Kubernetes 1.11 was released last week — it was a huge accomplishment for everyone involved and the release includes a swath of new functionality. A key focus area for Heptio is making Kubernetes easier to deploy and upgrade, and part of that work includes making improvements to Heptio’s preferred tool for bootstrapping a Kubernetes cluster, kubeadm.
SIG-Cluster-Lifecycle leads the development on kubeadm, and Heptio was heavily involved in this group (which includes contributors such as Lucas Käldström, Leigh Schrandt, Lubomir Ivanov, Peter (XiangPeng) Zhao, Rostislav Georgiev, and Di Xu) to bring about some important improvements in 1.11.
In this post, we cover what kubeadm is, why Heptio recommends kubeadm, and how we are working with the Kubernetes community to move kubeadm forward.
What is kubeadm and when should I use it?
kubeadm is a tool included in every Kubernetes release that helps users set up a best-practice Kubernetes cluster. kubeadm provides support along the entire lifecycle of a Kubernetes cluster including creation, upgrade, and teardown.
Due to the variety of ways in which machines can be provisioned, the scope of kubeadm is intentionally limited to bootstrapping rather than provisioning — it is intended to be a building block, and higher level tools take advantage of kubeadm. kubeadm allows these higher level tools to ensure clusters are conformant and look as much alike as possible. Setting up add-ons such as CNI, the Kubernetes dashboard, and monitoring tools is outside the scope of kubeadm.
Check out the kubeadm repository and docs for more information.
Why does Heptio recommend kubeadm?
kubeadm is the primary deployment tool Heptio recommends for bootstrapping a Kubernetes cluster, based on some key advantages:
- Ease of use. kubeadm supports creation, upgrade, and teardown, and is a relatively easy tool for new users to adopt.
- Use on any infrastructure. Heptio customers need to deploy Kubernetes across a wide variety of infrastructure — including bare metal, VMware, AWS, Azure, GCE, and more. It’s important that a cluster created on one platform looks like another created somewhere else. We even created Heptio Sonobuoy to test conformance so you can be sure that clusters were set up correctly and will behave as expected on any infrastructure.
- Extendable. Heptio customers have unique enterprise requirements. kubeadm provides a phases command that allows you to execute steps individually, so you can customize actions as needed.
- Production ready. kubeadm rolls out secure Kubernetes clusters — adopting best practices such as enforcing RBAC, using secure communication between the control plane components and between the API server and kubelets, locking down the kubelet API, and more.
- Community contributions. kubeadm has become one of the most common ways to deploy Kubernetes, and as a result the community has rallied to harden kubeadm and make inroads on every release.
So where does kubeadm fit into a complete deployment solution for Heptio customers? kubeadm is not a one-click-install solution. As stated above, kubeadm is intended to be a building block and part of a larger solution. Heptio is investing significantly in this area to bring a declarative, API-driven model to cluster creation and operations, where clusters are treated as immutable (i.e. upgrades equate to a new deployment versus an in-place upgrade). Heptio plans to leverage and contribute to the upstream work on the Cluster API to make this real. More will be shared in the future.
What’s new in kubeadm in 1.11?
SIG-Cluster-Lifecycle leads the development on kubeadm and is focused on making Kubernetes easier to deploy and operate. Some key improvements the community brought to kubeadm in 1.11 include:
- A step-by-step guide to build an HA Kubernetes deployment with stacked masters (e.g. etcd members and control plane nodes are co-located). An external etcd cluster is also available as an option.
- An upgraded API version for the configuration file, v1alpha2. kubeadm will still be able to read v1alpha1 configuration, and will automatically convert the configuration to v1alpha2. This is a necessary step for kubeadm to align with the Cluster API work mentioned above.
- The first step towards a fully dynamic kubelet configuration — kubelet config allows overrides via config-map for a host of features for a kubelet, but includes a set of defaults that remove manual steps involved when initializing or joining across different host OSs and CRI implementations.
- Replacement of kube-dns by CoreDNS as the default DNS provider, which is valuable for users that have scaled out deployments. Existing deployments will automatically upgrade to CoreDNS via kubeadm.
- Better CRI integration (including auto detection for other CRI runtimes than Docker) and support for air-gapped environments.
- New CLI commands that are helpful for people setting up clusters, including: kubeadm config print-default, kubeadm config migrate, kubeadm config images list, kubeadm config images pull, and kubeadm upgrade node config.
- Several bug fixes, most notably: a fix for a security issue for etcd by closing the peer port that was previously left insecure, and additional security in the kubelet configuration.
A team of amazing Heptio engineers (including Tim St. Clair, Jason DeTiberus, Chuck Ha, Liz Frost, Ruben Orduz, and Jennifer Rondeau) were key contributors to SIG-Cluster-Lifecycle in bringing out these important additions to kubeadm in 1.11. In particular, Tim St. Clair does an incredible amount of work for this group by grooming and triaging issues and reviewing pull requests. It is important for Heptio to do this work upstream, so everyone in the community can rally around a common Kubernetes bootstrapping method which can provide for semantic consistency across environments.
See a full list of 1.11 kubeadm changes here.
What’s next for kubeadm?
The key focus areas for development moving forward are high availability and general availability (GA) of kubeadm. While users can achieve high availability with kubeadm manually today, the community is focused on bringing native support for things like replicated etcd and multiple, redundant API servers and other control plane components. GA will require some key additions such as getting the config API to v1beta1 (to ensure supported migrations and deprecation guarantees), support for reading multiple YAML documents, HA capabilities (both etcd and masters), and alignment with the Cluster API spec.
Heptio is fully committed to making Kubernetes easier to deploy and operate, and we will make continued investments in kubeadm and other tooling to bring those capabilities to everyone.
Interested in learning how Heptio can help you deploy and operate Kubernetes more efficiently, and take advantage of the innovation in kubeadm and other tools? Check out our Heptio Kubernetes Subscription (HKS) and consulting offerings, or reach out to us directly.