To quickly recap:<\/p>\n
- \n
- \n
Originally PV was designed to be a piece of storage pre-allocated by administrator. Though after the introduction of Storage Class and Provisioner, users are able to dynamically provision PVs now.<\/p>\n<\/li>\n
- \n
PVC is a request for a PV. When used with Storage Class, it will trigger the dynamic provisioning of a matching PV.<\/p>\n<\/li>\n
- \n
PV and PVC are always one to one mapping.<\/p>\n<\/li>\n
- \n
Provisioner is a plugin used to provision PV for users. It helps to remove the administrator from the critical path of creating a workload that needs persistent storage.<\/p>\n<\/li>\n
- \n
Storage Class is a classification of PVs. The PV in the same Storage Class can share some properties. In most cases, while being used with a Provisioner, it can be seen as the Provisioner with predefined properties. So when users request it, it can dynamically provision PVs with those predefined properties.<\/p>\n<\/li>\n<\/ol>\n
But those are not the only ways to use persistent storage in Kubernetes.<\/p>\n
Take a deep dive into Best Practices in Kubernetes Networking<\/h5>\n
From overlay networking and SSL to ingress controllers and network security policies, we’ve seen many users get hung up on Kubernetes networking challenges. In this video recording, we dive into Kubernetes networking, and discuss best practices for a wide variety of deployment options.\n <\/p>\n
Watch the video<\/a><\/p>\n
Volume<\/h2>\n
In the previous article, I mentioned that there is also a concept of Volume in Kubernetes. In order to differentiate Volume from Persistent Volume, people sometimes call it In-line Volume, or Ephemeral Volume.<\/p>\n
Let me quote the definition of Volume<\/a> here:<\/p>\n
\n
A Kubernetes volume \u2026 has an explicit lifetime – the same as the Pod that encloses it. Consequently, a volume outlives any Containers that run within the Pod, and data is preserved across Container restarts. Of course, when a Pod ceases to exist, the volume will cease to exist, too. Perhaps more importantly than this, Kubernetes supports many types of volumes, and a Pod can use any number of them simultaneously.<\/p>\n
At its core, a volume is just a directory, possibly with some data in it, which is accessible to the Containers in a Pod. How that directory comes to be, the medium that backs it, and the contents of it are determined by the particular volume type used.<\/p>\n<\/blockquote>\n
One important property of Volume is that it has the same lifecycle as the Pod it belongs to. It will be gone if the Pod is gone. That\u2019s different from Persistent Volume, which will continue to exist in the system until users delete it. Volume can also be used to share data between containers inside the same Pod, but this isn\u2019t the primary use case, since users normally only have one container per Pod.<\/p>\n
So it\u2019s easier to treat Volume as a property of Pod, instead of as a standalone object. As the definition said, it represents a directory inside the pod, and Volume type defines what\u2019s in the directory. For example, Config Map Volume type will create configuration files from the API server in the Volume directory; PVC Volume type will mount the filesystem from the corresponding PV in the directory, etc. In fact, Volume is almost the only way to use storage natively inside Pod.<\/p>\n
It\u2019s easy to get confused between Volume, Persistent Volume and Persistent Volume Claim. So if you can imagine that there is a data flow, it will look like this: PV -> PVC -> Volume. PV contains the real data, bound to PVC, which used as Volume in Pod in the end.<\/p>\n
However, Volume is also confusing in the sense that besides PVC, it can be backed by pretty much any type of storage supported by Kubernetes directly.<\/p>\n
Remember we already have Persistent Volume, which supports different kinds of storage solutions. We also have Provisioner, which supports the similar (but not exactly the same) set of solutions. And we have different types of Volume as well.<\/p>\n
So, how are they different? And how to choose between them?<\/p>\n
Many ways of persisting data<\/h2>\n
Take AWS EBS for example. Let\u2019s start counting the ways of persisting data in Kubernetes.<\/p>\n
Volume Way<\/h3>\n
awsElasticBlockStore is a Volume type.<\/p>\n
You can create a Pod, specify a volume as awsElasticBlockStore, specify the volumeID, then use your existing EBS volume in the Pod.<\/p>\n
The EBS volume must exist before you use it with Volume directly.<\/p>\n
PV way<\/h3>\n
AWSElasticBlockStore is also a PV type.<\/p>\n
So you can create a PV that represents an EBS volume (assuming you have the privilege to do that), then create a PVC bound to it. Finally, use it in your Pod by specifying the PVC as a volume.<\/p>\n
Similar to Volume Way, EBS volume must exist before you create the PV.<\/p>\n
Provisioner way<\/h3>\n
kubernetes.io\/aws-ebs is also a Kubernetes built-in Provisioner for EBS.<\/p>\n
You can create a Storage Class with Provisioner kubernetes.io\/aws-ebs, then create a PVC using the Storage Class. Kubernetes will automatically create the matching PV for you. Then you can use it in your Pod by specifying the PVC as a volume.<\/p>\n
In this case, you don\u2019t need to create EBS volume before you use it. The EBS Provisioner will create it for you.<\/p>\n
Third-Party Way<\/h3>\n
All the options listed above are the built-in options of Kubernetes. There are also some third-party implementations of EBS in the format of Flexvolume driver, to help you hook it up to Kubernetes if you\u2019re not yet satisfied by any options above.<\/p>\n
And there are CSI drivers for the same purpose if Flexvolume doesn\u2019t work for you. (Why? More on this later.)<\/p>\n
VolumeClaimTemplate Way<\/h3>\n
If you\u2019re using StatefulSet, congratulations! You now have one more way to use EBS volume with your workload \u2013 VolumeClaimTemplate.<\/p>\n
VolumeClaimTemplate is a StatefulSet spec property. It provides a way to create matching PVs and PVCs for the Pod that Statefulset created. Those PVCs will be created using Storage Class so they can be created automatically when StatefulSet is scaling up. When a StatefulSet has been scaled down, the extra PVs\/PVCs will be kept in the system. So when the StatefulSet scales up again, they will be used again for the new Pods created by Kubernetes. We will talk more on StatefulSet later.<\/p>\n
As an example, let\u2019s say you created a StatefulSet named www with replica 3, and a VolumeClaimTemplate named data with it. Kubernetes will create 3 Pods, named www-0, www-1, www-2 accordingly. Kubernetes will also create PVC www-data-0 for Pod www-0, www-data-1 for www-1, and www-data-2 for www-2. If you scale the StatefulSet to 5, Kubernetes will create www-3, www-data-3, www-4 and www-data-4 accordingly. Then you scale the StatefulSet down to 1, all www-1 to www-4 will be deleted, but www-data-1 to www-data-4 will remain in the system. So when you decide to scale up to 5 again, Pod www-1 to www-4 will be created, and PVC www-data-1 will still serve Pod www-1, www-data-2 for www-2, etc. That\u2019s because the identity of Pod are stable in StatefulSet. The name and relationship are predictable when using StatefulSet.<\/p>\n
VolumeClaimTemplate is important for the block storage solutions like EBS and Longhorn. Because those solutions are inherently ReadWriteOnce, you cannot share it between the Pods. Deployment won\u2019t work well with them if you have more than one Pod running with persistent data. So VolumeClaimTemplate provides a way for the block storage solution to scale horizontally for a Kubernetes workload.<\/p>\n
How to choose between Volume, Persistent Volume and Provisioner<\/h2>\n
As you see, there are built-in Volume types, PV types, Provisioner types, plus external plugins using Flexvolume and\/or CSI. The most confusing part is that they just provide largely the same but also slightly different functionality.<\/p>\n
I thought, at least, there should be a guideline somewhere on how to choose between them.<\/p>\n
But I cannot find it anywhere.<\/p>\n
So I\u2019ve plowed through codes and documents, to bring you the comparison matrix, and the guideline that makes the most sense to me.
\nComparison of Volume, Persistent Volume and Provisioner<\/p>\n
![\"Sheng](\"https:\/\/rancher.com\/img\/bio\/sheng-yang.jpg\")
<\/p>\n