This guide walks through building a 7-node Kubernetes cluster spread across NETWAYS Web Services (OpenStack) and AWS, managed end-to-end by Claudie.
This article has been created in collaboration with Samuel Stoličný from Berops. You can find him on LinkedIn if you want to learn more about Claudie.
Why This Setup
Running the same cluster across a German OpenStack provider and a hyperscaler gives you two things that are hard to get from either one alone. NWS keeps your core workloads on EU-owned infrastructure with predictable pricing and direct access to NETWAYS’ OpenStack expertise, which matters for data-sovereignty and operational support.
AWS puts extra capacity and managed features (Route 53, Managed Databases, Message Queues, Lambda) one hop away over the cluster network, so you can reach for them without moving the whole cluster. Claudie stitches the two providers together into one Kubernetes control plane, so applications schedule across both like it’s a single cloud.
What You Get
- 3 control-plane nodes (1 NWS, 2 AWS) which gives etcd a proper quorum (tolerates one node loss).
- 4 compute nodes (3 NWS, 1 AWS). The NWS-heavy split matches the intent of keeping
- most workloads on EU-sovereign infrastructure while keeping some capacity on AWS.
- A WireGuard VPN connecting all seven nodes into one cluster network.
- Cilium CNI and Longhorn storage installed by Claudie.
Prerequisites
- A Kubernetes cluster to run Claudie on (the “management cluster”). For testing, a local kind or minikube cluster is enough. For production use something more durable, since Claudie stores state in it.
kubectlconfigured against that cluster.- A MyNWS account with quota for at least 4 VMs, 4 floating IPs, and 200 GB of storage in the
HetznerNBG4region. - An AWS account with permissions to create EC2 instances, VPCs, and security groups in
us-east-2(or any region you prefer).
Step 1 – Install Claudie
Install cert-manager first. Claudie uses it for its admission webhook:
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.19.3/cert-manager.yamlThen deploy Claudie itself:
kubectl apply -f https://github.com/berops/claudie/releases/latest/download/claudie.yamlGive it a minute, then check that the pods are running in the claudie namespace:
kubectl get pods -n claudie
NAME READY STATUS RESTARTS AGE
ansibler-7b5f8c6d49-qvz2w 1/1 Running 0 60s
claudie-operator-59dd646bcf-bjsdr 1/1 Running 0 60s
kube-eleven-bdc799684-xlwws 1/1 Running 0 60s
kuber-7cc7cb797d-bvsrv 1/1 Running 0 60s
manager-69f96bb548-l5wqr 1/1 Running 0 60s
minio-0 1/1 Running 0 60s
minio-1 1/1 Running 0 60s
minio-2 1/1 Running 0 60s
minio-3 1/1 Running 0 60s
mongodb-858994c5cb-x8w77 1/1 Running 0 60s
nats-0 2/2 Running 0 60s
nats-1 2/2 Running 0 60s
nats-2 2/2 Running 0 60s
terraformer-667b9f4556-dkx4x 1/1 Running 0 60sAll pods should be Running. If some are still pending, wait and re-check.
For more detail (including optional network policies for hardening) see the Claudie Detailed guide.
Step 2 – Create Provider Credentials
You’ll need one set of MyNWS application credentials and one AWS IAM access key.
NETWAYS Web Services (OpenStack Application Credential)
NWS authentication is normally federated through NWS-ID (OIDC). Claudie needs static credentials, so the workflow is two-step: log in with your NWS-ID Project User, then create an OpenStack application credential that Claudie can use directly.
In the NWS Customer Interface, open your OpenStack project and switch on the OpenStack Project User option. Note the username (it looks like <your-id>-openstack-<hash>) and password.
Install the OpenStack CLI (openstack) and authenticate with the Project User:
export OS_AUTH_URL=https://cloud.netways.de:5000/v3/
export OS_IDENTITY_API_VERSION=3
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_USERNAME=<your-project-user> # e.g. 21133-openstack-427a2
export OS_PROJECT_NAME=<your-project-name> # often the same string
read -s OS_PASSWORD; export OS_PASSWORD # read password from clipboard, then EnterCreate the application credential:
openstack application credential create --role member claudie -f shell
id="..."
secret="..."
project_id="..."Save the id and secret. The secret is shown only once. The project_id from the output is also what you need for the secret. Then look up the domain ID:
openstack project show $(openstack token issue -c project_id -f value) \
-c domain_id -f value
defaultNWS uses the literal string default for the domain ID.
AWS (IAM Access Key)
Create an IAM user with AmazonEC2FullAccess (or a narrower policy that covers EC2, VPC, and security groups) and note the access key ID and secret access key.
Step 3 – Create Kubernetes Secrets
Create a namespace for your credentials and add both secrets:
kubectl create namespace claudie-secrets
kubectl create secret generic netways-secret \
--namespace=claudie-secrets \
--from-literal=authurl='https://cloud.netways.de:5000/v3/' \
--from-literal=domainid='default' \
--from-literal=projectid='<your-nws-project-id>' \
--from-literal=applicationcredentialid='<your-credential-id>' \
--from-literal=applicationcredentialsecret='<your-credential-secret>'
kubectl create secret generic aws-secret \
--namespace=claudie-secrets \
--from-literal=accesskey='<your-aws-access-key-id>' \
--from-literal=secretkey='<your-aws-secret-access-key>'Step 4 – Create the Claudie Manifest File
Save the following as netways-aws-cluster.yaml:
apiVersion: claudie.io/v1beta1
kind: InputManifest
metadata:
name: netways-aws-cluster
labels:
app.kubernetes.io/part-of: claudie
spec:
providers:
- name: netways-1
providerType: openstack
secretRef:
name: netways-secret
namespace: claudie-secrets
- name: aws-1
providerType: aws
secretRef:
name: aws-secret
namespace: claudie-secrets
nodePools:
dynamic:
- name: nws-ctrl
providerSpec:
name: netways-1
region: HetznerNBG4
zone: HetznerNBG4
externalNetworkName: public-network
count: 1
serverType: s1.medium
image: "Ubuntu Noble 24.04 LTS"
storageDiskSize: 50
- name: aws-ctrl
providerSpec:
name: aws-1
region: us-east-2
zone: us-east-2a
count: 2
serverType: t3.medium
image: ami-0ea3c35c5c3284d82
storageDiskSize: 50
- name: nws-cmp
providerSpec:
name: netways-1
region: HetznerNBG4
zone: HetznerNBG4
externalNetworkName: public-network
count: 3
serverType: s1.medium
image: "Ubuntu Noble 24.04 LTS"
storageDiskSize: 50
- name: aws-cmp
providerSpec:
name: aws-1
region: us-east-2
zone: us-east-2a
count: 1
serverType: t3.medium
image: ami-0ea3c35c5c3284d82
storageDiskSize: 50
kubernetes:
clusters:
- name: netways-aws-cluster
version: v1.32.0
network: 192.168.2.0/24
pools:
control:
- nws-ctrl
- aws-ctrl
compute:
- nws-cmp
- aws-cmpNotes on NWS-specific values:
serverType: s1.mediumis NWS’s general-purpose flavor (4 vCPU, 4 GB RAM, 50 GB local disk). Claudie boots VMs from the image directly, so anys*.*,s2.*,p1.*, ord1.*flavor with non-zero local disk works. List available flavors withopenstack flavor list.storageDiskSize: 50is unrelated to the local disk above. It creates an extra Cinder volume that Claudie attaches to each worker node for Longhorn storage. Controlplane nodes ignore this field.image: "Ubuntu Noble 24.04 LTS"is NWS’s official Ubuntu 24.04 image. Find the current name withopenstack image list --public. AWS AMIami-0ea3c35c5c3284d82is Ubuntu 24.04 LTS inus-east-2. Look up the right AMI for your region with the Ubuntu AMI locator.
Step 5 – Deploy the Cluster
Apply the manifest:
kubectl apply -f netways-aws-cluster.yaml
inputmanifest.claudie.io/netways-aws-cluster createdClaudie picks it up and starts provisioning. Check progress with:
kubectl get inputmanifest -n claudie
NAME STATUS
netways-aws-cluster IN_PROGRESSThe build moves through four phases: Terraformer (provisioning VMs), Ansibler (configuring nodes, installing WireGuard), KubeEleven (running kubeadm), and Kuber (installing CNI, storage, and post-setup). A 7-node build typically takes 12 to 18 minutes.
When it’s done you’ll see:
kubectl get inputmanifest -n claudie
NAME STATUS
netways-aws-cluster WATCHING_FOR_CHANGESWATCHING_FOR_CHANGES means Claudie has finished provisioning and is watching for any future edits to the manifest.
Step 6 – Access the Cluster
Claudie writes the generated kubeconfig as a Kubernetes secret in the claudie namespace. Pull it out and point kubectl at it:
kubectl get secrets -n claudie -l claudie.io/output=kubeconfig \
-o jsonpath='{.items[0].data.kubeconfig}' | base64 -d > netways-aws.kubeconfig
export KUBECONFIG=$PWD/netways-aws.kubeconfig
kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP OS-IMAGE
aws-ctrl-xxxxxxx-01 Ready control-plane 5m v1.32.0 192.168.2.2 Ubuntu 24.04.1 LTS
aws-ctrl-xxxxxxx-02 Ready control-plane 5m v1.32.0 192.168.2.3 Ubuntu 24.04.1 LTS
aws-cmp-yyyyyyy-01 Ready <none> 3m v1.32.0 192.168.2.7 Ubuntu 24.04.1 LTS
nws-ctrl-zzzzzzz-01 Ready control-plane 6m v1.32.0 192.168.2.1 Ubuntu 24.04.4 LTS
nws-cmp-wwwwwww-01 Ready <none> 3m v1.32.0 192.168.2.4 Ubuntu 24.04.4 LTS
nws-cmp-wwwwwww-02 Ready <none> 3m v1.32.0 192.168.2.5 Ubuntu 24.04.4 LTS
nws-cmp-wwwwwww-03 Ready <none> 3m v1.32.0 192.168.2.6 Ubuntu 24.04.4 LTSSeven nodes, three control-plane, four workers, all in Ready state. The INTERNAL-IP column shows the WireGuard overlay addresses Claudie assigned from the 192.168.2.0/24 WireGuard network range in your manifest. All inter-node traffic flows encrypted over that network regardless of which cloud the nodes are on.
In a representative test build, the warm cross-cloud round-trip latency between MyNWS in Nuremberg and AWS in Ohio (us-east-2a) measured 219-220 ms over the WireGuard mesh, which is essentially the speed-of-light minimum for transatlantic fiber. Same-region NWS-to-NWS hops are around 1 ms.
Step 7 – Clean Up
Delete the InputManifest to tear down all provisioned infrastructure:
kubectl delete inputmanifest netways-aws-cluster -n claudie
inputmanifest.claudie.io "netways-aws-cluster" deletedTerraformer destroys VMs, networks, routers, security groups, and floating IPs on both clouds. Teardown usually takes 3 to 5 minutes.
Troubleshooting
- NWS-ID OIDC keeps prompting for a token. Don’t put OIDC tokens in the secret. Claudie needs static application credentials. Switch on the OpenStack Project User in the NWS Customer Interface, then run
openstack application credential createfrom that session. - 401 Unauthorized on apply. The secret is missing one of the five required keys (
authurl,domainid,projectid,applicationcredentialid,applicationcredentialsecret) or has stale values from a rotated credential.
What’s Next
- NETWAYS Cloud documentation, full reference and guides.
- Claudie documentation, full reference and guides.
- Attach a load balancer following the Loadbalancing guide.
- Set up autoscaling with the autoscaling guide.
- For additional regions or providers, see the Openstack and AWS reference pages.
0 Comments