Kubernetes, a leading open source project for automating deployment, scaling, and management of containerized applications, announced version 1.10 today. Among the key features of this release are support for the Container Storage Interface (CSI), API aggregation, a new mechanism for supporting hardware devices, and more.
It’s also the first release since CoreOS joined Red Hat. CoreOS already had the opportunity to work closely with our new Red Hat colleagues through the Kubernetes community and we now have the opportunity to redouble our efforts to help forward Kubernetes as an open source and community-first project.
The Kubernetes project gave a sneak peek at the feature list of Kubernetes 1.10 when the beta was released, but here we’ll take a closer look at some of the more significant developments. First, however, it may be helpful to give a quick refresher on how Kubernetes is developed and new features are added to the system.
From alpha to stable
As you may know, Kubernetes is a system composed of a number of components and APIs. Not all of them can be developed simultaneously or reach maturity at the same time. Because of this, Kubernetes releases include features that are considered alpha, beta, and stable quality. The Kubernetes community defines these features as:
- Alpha features should be considered tentative. They may change dramatically by the time they’re considered production-ready, or they may be dropped entirely. These features are not enabled by default.
- Beta features are considered well-tested and will not be dropped from Kubernetes, but they may yet change. These features are available by default.
- Stable features are considered suitable for production use. Their APIs won’t change the way beta and alpha APIs are likely to, and it is often safe to assume they will be supported for “many subsequent versions” of Kubernetes.
As is usual, Kubernetes 1.10 includes a mix of features at each of these levels of maturity, and several of them merit special attention in this release.
API aggregation is stable
One feature that has graduated to stable in Kubernetes 1.10 is API aggregation, which allows Kubernetes community members to develop their own, custom API servers without making any changes to the core Kubernetes code repository. With this feature now stable, Kubernetes cluster admins can more confidently add these third-party APIs to their clusters in a production configuration. Our collective experience running API aggregation has helped identify and support the upstreams ability to graduate it to stable.
This is a powerful capability that allows developers to provide highly customized behaviors to Kubernetes clusters that return very different kinds of resources than the core Kubernetes APIs provide. This can be especially valuable for use cases where Custom Resource Definitions (CRDs), the primary Kubernetes extension mechanism, may not be fully featured enough.
Customization is something that has been requested by the community and the CoreOS team, now as the Red Hat team, has been focused on architecting Kubernetes in a way to make this possible. In November 2016, we introduced the Operator pattern, software that encodes domain knowledge and extends the Kubernetes API, enabling users to more easily deploy, configure, and manage applications. With API aggregation now considered stable, developers have more ways to use Kubernetes in unique, custom ways.
Standardized storage support
Support for the Container Storage Interface (CSI) specification has graduated to beta in Kubernetes 1.10, and it’s one of the more significant enhancements of this release. The goal of CSI is to create a standardized way for storage vendors to write plugins that will work across multiple container orchestration tools – including but not limited to Kubernetes. Among the capabilities it aims to provide are standardized ways to dynamically provision and deprovision storage volumes, attach or detach volumes from nodes, mount or unmount volumes from nodes, and so on.
Kubernetes was one of the first container orchestration tools to support CSI and the code can now be viewed as fairly mature. As a result, Kubernetes users can expect more storage options for their clusters, as the amount of development and integration work required of vendors is reduced.
A replacement for kube-dns
Most Kubernetes clusters use internal DNS for service discovery. And the default provider for this has been kube-dns, a Go wrapper around dnsmasq, which late last year suffered a slew of vulnerabilities. Work is being done to switch the default provider from Kube-DNS to CoreDNS, an independent project overseen by the CNCF that’s written in Go. As of Kubernetes 1.10, this work has moved into beta.
CoreDNS is built around plugins, and its goals include simplicity, speed, flexibility, and ease of service discovery, all of which are in keeping with the goals of the broader Kubernetes community, including the drive to move more functions out of the core Kubernetes code base and into their own projects.
Expanding support for performance sensitive workloads
Much work has been done across the community to support more performance-sensitive workloads in Kubernetes. For the 1.10 release, the DevicePlugins API has gone to beta. This is designed to provide the community a stable integration point for GPUs, high-performance networking interfaces, FPGAs, Infiniband, and many other types of devices, without requiring the vendor to add any custom code to the Kubernetes core repository.
Other advanced features have graduated to beta to better support CPU and memory sensitive applications. The static CPU pinning policy has graduated to beta to support CPU latency sensitive applications by pinning applications to particular cores. In addition, the cluster is able to schedule and isolate hugepages for those applications that demand them.
Pod security policy
Containers are just isolated processes on a host. Disable that isolation and you’re not running a container anymore.
Kubernetes offers several ways to enable privileged access to a host. These options are intended for workloads with special requirements, such as network plugins and host agents, but in the wrong hands they can also be extremely effective attack vectors against a node. Flip the privileged flag on a pod spec and a container can access all of the host’s devices. A workload with host networking enabled can get around network policy, while a workload that requests a host mount can gain access to the kubelet’s on-disk credentials.
Pod security policies are designed to reduce this attack surface by restricting the kinds of pods that can be run in a given namespace. Over the past couple of releases, the community has worked to get the feature to a usable state, and in 1.10 the API moves to its own API group from the deprecated extensions/v1beta1.
There is no true multi-tenancy without pod security policies. Over the next few releases, the community expects to see a measured rollout of PSP (similar to the RBAC rollout about a year ago starting in 1.6) as it attempts to improve the default security posture of Kubernetes. In the meantime, we encourage admins to enable this feature on their test clusters and begin experimenting with it. Your feedback can help improve Kubernetes security for everyone.
Adding identity to containers
Finally, one alpha-quality feature that’s worth calling attention to is the TokenRequest API, a replacement for service accounts, which gets us on the road to being able to assign identities to individual containers. Currently, multiple instances of the same container all share the same identity. Identifying them individually should facilitate the creation of policies that impact specific containers – for example, a policy could be created wherein containers not running on the user’s most locked-down, secured nodes could be denied TLS credentials.
TokenRequest also enables credentials targeted for services other than the API server. This can let applications safely attest to external services without hand over its Kubernetes credentials, and can harden use cases such as the Vault Kubernetes plugin.
As with alpha features, this is definitely a work in progress. But it is another important step toward hardening the security of Kubernetes clusters.
Onward and upward
As always, we congratulate the entire Kubernetes community for the hard work that went into making Kubernetes 1.10 one of the most feature-rich releases yet. A fast-moving open source projects, Kubernetes continues to mature and seek to adapt to meet the needs of its user base, thanks to the many contributors from across the ecosystem.
Work on Kubernetes 1.11 is already underway, with the release expected to ship in roughly three months. To have a look at what’s under development, or to get involved, join any of the many special interest groups (SIGs) where community collaboration take place. Red Hat and the CoreOS team are proud to work alongside the other members of the upstream community. Join the upstream community contributors, Cole Mickens and Stefan Schimanski, for a briefing about what’s new on March 28.