Kubernetes with Cilium: Traffic filtering based on L7

5 September, 2023

Justin Lamp
Justin Lamp
Senior Systems Engineer

Justin ist seit April 2022 als Systems Engineer Teil des NETWAYS-Teams. Ursprünglich aus Kiel, brachte ihn seine Leidenschaft für Open-Source-Technologien insbesondere OpenStack und Kubernetes, nach Nürnberg. Er begeistert sich für das Experimentieren mit neuen Technologien – vom Erlernen und Einrichten bis zur Fehlerbehebung. Der Wechsel zu Open Source gibt ihm die Freiheit, direkt mit den Technologien und deren Entwicklern zu arbeiten. In seiner Freizeit ist Justin sportlich aktiv: Leichtathletik, Schwimmen in der Ostsee, Radfahren sowie Mountainbiken, Rafting und Snowboarden gehören zu seinen Leidenschaften.

by | Sep 5, 2023

With the new version of the Cilium CNI on our Kubernetes service, you get the ability to filter traffic based on L7 properties. This is normally reserved for service meshes and can be very helpful in securing your services. In this tutorial, we will secure an API endpoint so that our client can only access certain routes. All other requests will not be forwarded to the service in the first place to minimize the load on the API.

Prerequisites

Before you start, you will need the following:

  1. A Kubernetes cluster in operation with Cilium.
  2. kubectl command line tool is installed and configured for communication with your cluster.
  3. Basic knowledge of Kubernetes pods, services and namespaces.

Create API + API client

$ kubectl apply -f \
https://gist.github.com/modzilla99/0cbdd39cc838e752ae8cdb5ec3cdd03a/\
raw/adb7218bb1e9195c42192d71809f6f7043e50cd7/deploy.yaml
configmap/apiconfig created
deployment.apps/api created
service/api created
pod/api-client created

We have now created a service called “api” that is bound to the deployment “api”. It has two endpoints. The /secret route should be private and /greetings should be publicly accessible. Since we have not defined any network policies, both routes are accessible to any pod running in the cluster.

Testing the API

So let’s try out the API first:

$ kubectl exec -it api-client -- curl -H 'X-Security-Header: mysecret' api/secret/mysecret
{
  "name": "mysecret",
  "secret": "syferghewg"
}

$ kubectl exec -it api-client -- curl api/greetings/World
Greetings, World!

$ kubectl exec -it api-client -- curl google.com
The document has moved

As you can see, the API is accessible without restrictions and the api-client pod also has full/unfiltered external access.

Restrict access

To minimize access, we want to restrict the pods’ access exclusively to our API and api.github.com. This goal can be effectively achieved with the following NetworkPolicy:

---
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-client-egress
spec:
  endpointSelector:
    # Apply policy to api client
    matchLabels:
      app: client
      type: production
  ingress:
  # Drop all incoming traffic
  - {}
  egress:
  # Allow API acccess to GitHub only via HTTPs
  - toFQDNs:
    - matchName: api.github.com
    toPorts:
    - ports:
      - port: "443"
  # Allow access to our API
  - toEndpoints:
    - matchLabels:
        app: api
        type: production
    toPorts:
    - ports:
      - port: "6090"
        protocol: TCP
  # Allow DNS queries
  - toEndpoints:
    - matchLabels:
        io.kubernetes.pod.namespace: kube-system
        k8s-app: kube-dns
    toPorts:
    - ports:
      - port: "53"
    rules:
      dns:
      - matchPattern: "*"

Now the pod can only communicate with our and the GitHub API and query the Kubernetes DNS server. Any request to other resources will now fail:

$ kubectl exec -it api-client -- curl -4 --connect-timeout 3 -v google.com
* processing: google.com
* Trying 142.250.184.238:80...
* ipv4 connect timeout after 2974ms, move on!
* Failed to connect to google.com port 80 after 3001 ms: Timeout was reached
* Closing connection
curl: (28) Failed to connect to google.com port 80 after 3001 ms: Timeout was reached

$ kubectl exec -it api-client -- curl -4 https://api.github.com/user
{
  "message": "Requires authentication",
  "documentation_url": "https://docs.github.com/rest/users/users#get-the-authenticated-user"
}

Next, we want to secure our API. We don’t want the pod to be able to communicate to the outside world and we only want it to allow access to our two routes. Since we want the /greetings endpoint to be public, we open access to everyone, while we restrict access to the /secret route to our API client.

---
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-api-ingress
spec:
  endpointSelector:
    matchLabels:
      app: api
      type: production
  egress:
  # Deny any external traffic
  - {}
  ingress:
  # Allow access to /secret route from our api client 
  - fromEndpoints:
    - matchLabels:
        app: client
        type: production
    toPorts:
    - ports:
      - port: "6090"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/secret/.*"
          # Require HTTP header for authorization
          headers:
          - 'X-Security-Header: mysecret'
  # Allow access to /greetings route from anyone
  - toPorts:
    - ports:
      - port: "6090"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/greetings/.*"

Now we can only access the /secret route with the required header and from the Api client pod:

$ kubectl exec -it api-client -- curl api/secret/mysecret
Access denied

$ kubectl exec -it api-client -- curl -H 'X-Security-Header: mysecret' api/secret/mysecret
{
  "name": "mysecret",
  "secret": "adasge53whbetn"
}

Conclusion

Cilium NetworkPolicies are a powerful way to control network traffic within your cluster. By defining rules for inbound and outbound traffic, you can secure your applications and isolate them when necessary. This tutorial has covered the basics of creating and using NetworkPolicies, but you can also create more complex policies to suit your specific needs. Need help with this topic? Send us a message or subscribe to our newsletter to receive more information on this topic.

Our portfolio

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

How did you like our article?