Our Kubernetes release review of Kubernetes v1.33 Octarine

Kubernetes v1.33 was released last Tuesday, and as always, we at NWS took a close look at what’s new, what features the project has gotten rid of, and which features stand out in particular. As always, you can find the highlights including practical examples in our Kubernetes release review!

Kubernetes v1.33 bears the release name Octarinenamed after the magical eighth color in Terry Pratchett’s fantasy universe. This name is fitting – new features such as sidecar containers or in-place scaling of pod resources really do feel magical. However, our Kubernetes release review starts with the announced deprecations.

Deprecations in Kubernetes v1.33 Octarine

As with every Kubernetes release, there are some features that have been deprecated. This usually happens due to project development in a new direction or after the emergence of a new, better standard. A good example is the Endpoints API.

Deprecation of the Endpoint API in favor of the EndpointSlices API

The EndpointSlices API has been categorized as stable since Kubernetes v1.21 and has therefore been the de facto successor to the old, simpler Endpoints API within Kubernetes for some time now. Although the Endpoints API was easy to understand and simple to implement, it did have problems here and there in larger setups.

These problems have been resolved by the introduction of the EndpointSlices API and new functions have been integrated, e.g. dual-stack networking. This deprecation will probably only affect you if you use the old Endpoints API directly as part of scripts or automations. A detailed blog post on the migration has already been announced.

The original Kubernetes Enhancement Proposal (KEP ) for deprecation can be found here.

Deprecation of the in-tree gitRepo volume driver

The gitRepo volume type has been deprecated since Kubernetes v1.11, which is almost 7 years ago. Since then, there have always been security concerns regarding the gitRepo volume type and the possibility of using it to achieve remote code execution on cluster nodes.

In Kubernetes v1.33, the source code has now been removed from the Kubernetes project. The release notes list git-sync or Init Container as useful alternatives, gitVolumes are still available in the Kubernetes API and pods with gitRepo volumes can be created. To execute such pods, the feature gate GitRepoVolumeDriver=true must be set on the Kubelets of a cluster, otherwise a transparent error message will be returned to the user.

This renewed opt-in means that gitRepo volumes can still be used for the next three versions and a migration strategy can be developed.

You can find the original CEP on deprecation here.

A complete overview of all five deprecations in Kubernetes v1.33 Octarine can be found in the release notes.

Highlights in Kubernetes v1.33 Octarine

Now that we’ve ticked off the missing features, we can move on to the more exciting part of our Kubernetes release review: The new features in Kubernetes v1.33 Octarine!

We find two features particularly interesting here: sidecar containers are now stable, and vertical scaling of pods in-place is beta. We will explain both changes in the next few paragraphs.

Sidecar container in Kubernetes v1.33

So-called sidecar containers have been an integral part of Kubernetes since its early days. Here, an additional container is defined as another regular container or an init container running before the container start within a pod. Common use cases are, for example, service meshes, log and metric scrapers, secret management and other auxiliary services.

Problems with previous sidecars as init containers

However, the existing implementation options repeatedly gave rise to problems, depending on the application:

• If the sidecar is defined as an init container, the kubelet waits until the init container is terminated before creating the main container. This model is therefore unusable for permanently running agents for network, logging and scraping.
• If the sidecar is defined as a regular container, the start sequence of the regular containers is not deterministic. It could therefore be that the actual application starts before the sidecar and therefore information cannot be logged, metrics cannot be scraped or a network connection cannot be established. The same applies to the scheduling of a pod.
• Due to the often lower resource requirements of sidecars compared to the application containers of a pod, sidecars are the preferred target of the kubelet if containers have to be terminated due to a lack of resources. In the case of a set restartPolicy: Never, e.g. for jobs, the sidecar is not restarted in such a case.

With the introduction of sidecar containers as an explicit deployment model, Kubernetes enables defined, reliable handling of such processes:

• Sidecar containers are init containers that continue to run even after pod startup and do not block the start of the application container.
• Sidecar containers are terminated at the time a pod is scheduled after the application containers.
• Regular init containers and sidecar containers can be declared together; the start sequence is still observed as usual.

This makes it much easier in Kubernetes since v1.29 to define and manage additional processes required for monitoring, logging, housekeeping and other application purposes.

Definition of sidecar containers

This is made possible by a new field restartPolicy in the specification of initContainers of a pod:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  labels:
    app: myapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: myapp
          image: alpine:latest
          command: ['sh', '-c', 'while true; do echo "logging" >> /opt/logs.txt; sleep 1; done']
          volumeMounts:
            - name: data
              mountPath: /opt
      initContainers:
        - name: logshipper
          image: alpine:latest
          restartPolicy: Always
          command: ['sh', '-c', 'tail -F /opt/logs.txt']
          volumeMounts:
            - name: data
              mountPath: /opt
      volumes:
        - name: data
          emptyDir: {}

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  labels:
    app: myapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: myapp
          image: alpine:latest
          command: ['sh', '-c', 'while true; do echo "logging" >> /opt/logs.txt; sleep 1; done']
          volumeMounts:
            - name: data
              mountPath: /opt
      initContainers:
        - name: logshipper
          image: alpine:latest
          restartPolicy: Always
          command: ['sh', '-c', 'tail -F /opt/logs.txt']
          volumeMounts:
            - name: data
              mountPath: /opt
      volumes:
        - name: data
          emptyDir: {}

The set restartPolicy Always in the Init Container logshipper ensures that the sidecar can be (re)started or stopped independently of the application container myapp and any other Init Containers. The sidecar is guaranteed to start before the application container and is terminated after it.

Unlike regular init containers, sidecards, like regular containers, support probes (livenessProbe, readinessProbe, startupProbe) to control their life cycle. With defined probes, sidecars are therefore also included in the evaluation of the ready/healthy status of a pod.

These new possibilities of sidecar management through the feature categorized as stable in Kubernetes v1.33 Octarine are definitely our personal highlight of this Kubernetes release review. You can find the underlying KEP here. Further information on sidecar containers can be found in the Kubernetes documentation.

In-place scaling of pod resources in Kubernetes v1.33

The second already mentioned feature that we would like to highlight in our Kubernetes release review relates to the vertical scaling of pod resources – without a previously necessary restart!

Kubernetes has long enabled the management of resources for pods and containers. In contrast to the horizontal scaling of workloads (by adding/removing workload replicas), vertical scaling has so far always been associated with a restart of all affected pods and containers.

Advantages of in-place pod resizing

The In-place resource resize for vertical scaling of Pods feature, which is currently categorized as beta, now enables precisely this process without the need for a restart. This opens up some interesting application possibilities:

• Stateful workloads can be scaled up without downtime.
• Allocated resources can be reduced when the load is low.
• Applications that require significantly more resources for initialization than in regular operation can be managed accordingly.

The Kubernetes documentation provides a good overview of the resizing of pod resources, including examples.

Definition of In-Place Pod Resizing

As the feature is active by default in Kubernetes v1.33 Octarine, nothing needs to be taken into account. The resizePolicy field is available for an explicit definition of the desired behavior of a pod in the event of a resizing. This can be used to define the restart behaviour of a pod after resizing for each affected resource:

apiVersion: v1
kind: Pod
metadata:
  name: resize-demo
spec:
  containers:
  - name: pause
    image: registry.k8s.io/pause:3.8
    resizePolicy:
    - resourceName: cpu
      restartPolicy: NotRequired # Default, but explicit here
    - resourceName: memory
      restartPolicy: RestartContainer
    resources:
      limits:
        memory: "200Mi"
        cpu: "700m"
      requests:
        memory: "200Mi"
        cpu: "700m"

apiVersion: v1
kind: Pod
metadata:
  name: resize-demo
spec:
  containers:
  - name: pause
    image: registry.k8s.io/pause:3.8
    resizePolicy:
    - resourceName: cpu
      restartPolicy: NotRequired # Default, but explicit here
    - resourceName: memory
      restartPolicy: RestartContainer
    resources:
      limits:
        memory: "200Mi"
        cpu: "700m"
      requests:
        memory: "200Mi"
        cpu: "700m"

In this example, it is specified that no restart of the pod is required for CPU resizing; however, when resizing the allocated memory, a restart of the container is required to guarantee propagation of the newly set resources.

The restart policies listed in the example (NotRequired or RestartContainer) are the only ones currently available. Also, the feature is currently only available for CPU and memory resources. A list of all current limitations can be found in the Kubernetes documentation.

The Vertical Pod Autoscaler(VPA) is not yet able to handle the new feature. An open PR on GitHub can be followed here.

Kubernetes v1.33 Octarine – Feels like magic!

Our conclusion of this Kubernetes release review: Kubernetes v1.33 Octarine more than lives up to its name. The newly introduced or continued changes to the project have the potential to relieve us of a lot of work in existing environments, e.g. for sidecar management or elaborate scaling frameworks, and feel “like magic” for newcomers and old hands alike.
You can find a complete overview of all changes included in Kubernetes v1.33 either in the official release notes or in the accompanying blog post of the Kubernetes project.

And as always, if you want to implement any of these features or need any other help running Kubernetes in the NETWAYS Cloud or anywhere else, feel free to contact one of our MyEngineers® at any time – we’d love to hear from you! Otherwise, we’ll see you in the next Kubernetes release review for v1.34 in August.