ReadWriteMany (RWX) with the NFS Ganesha Provisioner

ReadWriteMany (RWX) with the NFS Ganesha Provisioner


You have the desire that your application needs to scale across multiple nodes for load balancing, but needs access to a common PVC? For this purpose, you need a PVC that is RWX-enabled. As part of our Managed Kubernetes Cluster, it is possible to create a CSI Cinder block storage. However, due to technical limitations, this is only ReadWriteOnce (RWO) capable. We will show you here how we can still create a RWX-capable PVC using a workaround!

PVCs are explained in the tutorial Creating Persistent Volumes in Kubernetes, on which this tutorial builds.

Take, for example, a web server that has HTML pages, images, JS etc. stored on its PVC. These files are to be uploaded once and made available to all instances of the web server simultaneously. To better distribute the load, the deployment is scaled accordingly. Thus, several pods of the same type run on different servers.

To ensure that all pods – across host boundaries – have access to the files, we create a RWX Storageclass with the nfs-ganesha-server-and-external-provisioner. The basis for this is an NWS Managed Kubernetes Cluster. After completing the workaround, we have the option of creating PVCs that can be read and written to simultaneously by multiple pods.

A note beforehand: the setup described is not HA capable!



The tutorial assumes the following:

  • NWS Managed Kubernetes Cluster
  • kubectl
  • helm

You can find more information on these topics in our NWS Docs



With the following command we add the Helm Repository, which provides us with the nfs-ganesha-server-and-external-provisioner, called NFS Provisioner in the following.

$ helm repo add nfs-ganesha-server-and-external-provisioner \

After that we can immediately start with the installation. Note the settings that follow the –set parameter. These result in:

  • persistence.enabled=true
    This parameter ensures that our data is stored on a persistent volume and persists even after the pod restarts.
  •  persistence.storageClass=default
    Here it is specified that the storage class “standard” should be used for the persistent data.
  •  persistence.size=200Gi
    This parameter specifies the size of the PVC that the NFS Provisioner fetches for the files.

Note that the PVC size specified with persistence.size is shared for all NFS PVCs that are obtained from the NFS Provisioner. There are many other configuration options with which the NFS Provisioner can be adapted to your own needs. You can find these here

$ helm install nfs-server \
nfs-ganesha-server-and-external-provisioner/nfs-server-provisioner \
--set persistence.enabled=true \
--set persistence.storageClass=standard,persistence \
--set size=200Gi

If the NFS Provisioner could be installed successfully, an output like the following appears:

NAME: nfs-server
LAST DEPLOYED: Mon May 22 14:41:58 2023
NAMESPACE: default
STATUS: deployed
The NFS Provisioner service has now been installed.
A storage class named 'nfs' has now been created and is available to provision dynamic volumes.
You can use this storageclass by creating a `PersistentVolumeClaim` with the correct storageClassName attribute. For example:
    kind: PersistentVolumeClaim
    apiVersion: v1
      name: test-dynamic-volume-claim
      storageClassName: "nfs"
        - ReadWriteOnce
          storage: 100Mi

Now let’s take a look at the NFS Provisioner Pod we just created and see if it’s running:

kubectl get pods
NAME                                  READY   STATUS    RESTARTS   AGE
nfs-server-nfs-server-provisioner-0   1/1     Running   0          36m

And of course the associated storage class for the automatic provision of NFS RWX PVCs. We can now use this to dynamically create and use PVCs of the RWX type.

$ kubectl get storageclass
encrypted                             Delete          Immediate           true                   3h22m
encrypted-high-iops                          Delete          Immediate           true                   3h22m
high-iops                             Delete          Immediate           true                   3h22m
nfs                   cluster.local/nfs-server-nfs-server-provisioner   Delete          Immediate           true                   39m
nws-storage                           Delete          Immediate           true                   3h22m
standard (default)                          Delete          Immediate           true                   3h22m

Web server example

We now have a PVC with the property RWX which we can provision dynamically. Now let’s take a look at how this is integrated into a deployment.

We create two files; one for our web server deployment and one for the RWX PVC:

# nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
    app: webserver
  name: webserver
  replicas: 1
      app: webserver
      creationTimestamp: null
        app: webserver
      - image: nginx:latest
        name: nginx
        - mountPath: /files
          name: files
      - name: files
          claimName: nfs-pvc


# nfs-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
  name: nfs-pvc
  - ReadWriteMany
      storage: 5Gi
  storageClassName: nfs

Our deployment specifies that we want to run exactly one replica of the NGiNX Container. This integrates the dynamically created NFS PVC under /files. Now we have to feed the definitions to Kubernetes using kubectl:

$ kubectl apply -f nfs-pvc.yaml nginx-deployment.yaml
persistentvolumeclaim/nfs-pvc created
deployment.apps/webserver created

As we can see a Webserver pod is running:

$ kubectl get pods
NAME                                  READY   STATUS    RESTARTS   AGE
nfs-server-nfs-server-provisioner-0   1/1     Running   0          54m
webserver-5486dd9cf5-hfhnd            1/1     Running   0          114s

But since this is far from sufficient for our load, we will expand the deployment to 4 replications:

$ kubectl scale deployment webserver --replicas=4
deployment.apps/webserver scaled

A small check is recommended and lo and behold, all 4 pods are running happily:

$ kubectl get pods
NAME                                  READY   STATUS    RESTARTS   AGE
nfs-server-nfs-server-provisioner-0   1/1     Running   0          55m
webserver-5486dd9cf5-hfhnd            1/1     Running   0          3m9s
webserver-5486dd9cf5-nh9fl            1/1     Running   0          18s
webserver-5486dd9cf5-ss27f            1/1     Running   0          18s
webserver-5486dd9cf5-xl2lj            1/1     Running   0          18s

Now let’s check if the NFS is also working properly between the pods.
A first command shows us that /files is empty. Using a second command, we create the nfs-is-rwx file. From the third issue we can see that we have been successful. We created in one pod and the file immediately existed in another pod.

$ kubectl exec webserver-5486dd9cf5-hfhnd -- ls /files
$ kubectl exec webserver-5486dd9cf5-nh9fl -- touch /files/nfs-is-rwx
$ kubectl exec webserver-5486dd9cf5-xl2lj -- ls /files


You have now set up an NFS server, which uses a CSI Cinder block storage in the background. The NFS server uses the block storage in the background to provide an RWX PVC for your pods via NFS. As a result, we have circumvented the technical limitation of a CSI Cinder block device and you can now use your NWS Managed Kubernetes Cluster for more use cases.