In this post, you will go from 3 Ubuntu 16.04 nodes to a basic Kubernetes cluster in a few simple steps. To accomplish this, you will be using Rancher Kubernetes Engine (RKE). To be able to use RKE, you will need 3 Linux nodes with Docker installed (see Requirements below).<\/p>\n
This won\u2019t be a production ready cluster, but enough to get you familiar with RKE, some Kubernetes and be able to play around with the cluster. Keep an eye out for the post for building a production ready cluster.<\/p>\n
Requirements<\/h3>\n\n- RKE<\/li>\n<\/ul>\n
You will be using RKE from your workstation. Download the latest version for your platform at:
\nhttps:\/\/github.com\/rancher\/rke\/releases\/latest<\/a><\/p>\n\n- kubectl<\/li>\n<\/ul>\n
After creating the cluster, we will use the default Kubernetes command-line tool called kubectl to interact with the cluster.
\nGet the latest version for your platform at:
\nhttps:\/\/kubernetes.io\/docs\/tasks\/tools\/install-kubectl\/<\/a><\/p>\n\n- 3 Ubuntu 16.04 nodes with 2(v)CPUs, 4GB of memory and with swap disabled<\/li>\n<\/ul>\n
Most commonly used Linux distribution is Ubuntu 16.04, this is what will be used in this post. Make sure swap is disabled by running swapoff -a and removing any swap entry in \/etc\/fstab. You must be able to access the node using SSH. As this is a multi-node cluster, the required ports<\/a> need to be opened before proceeding.<\/p>\n\n- Docker installed on each Linux node<\/li>\n<\/ul>\n
Kubernetes only validates Docker up to 17.03.2 (See https:\/\/github.com\/kubernetes\/kubernetes\/blob\/master\/CHANGELOG-1.11.md#external-dependencies<\/a>).
\nYou can use https:\/\/docs.docker.com\/install\/linux\/docker-ce\/ubuntu\/<\/a> to install Docker (make sure you install 17.03.2) or use this one-liner to install the correct version:
\ncurl https:\/\/releases.rancher.com\/install-docker\/17.03.sh<\/a> | sh<\/p>\nMake sure the requirements listed above are fulfilled before you proceed.<\/p>\n
How RKE works<\/h3>\n
RKE can be run from any platform (the binary is available for MacOS\/Linux\/Windows), in this example it will run on your workstation\/laptop\/computer. The examples in this post are based on MacOS\/Linux.<\/p>\n
RKE will connect to the nodes using a configured SSH private key (the nodes should have the matching SSH public key installed for the SSH user) and setup a tunnel to access the Docker socket (\/var\/run\/docker.sock by default, but configurable). This means that the configured SSH user must have access to the Docker socket on the machine, we will go over this in Creating the Linux user account.<\/p>\n
![\"\"](\"https:\/\/rancher.com\/img\/blog\/2018\/2018-09-26-setup-k8s-rkeworks.png\")
<\/p>\n
Creating the Linux user account<\/h3>\n
Note: Make sure Docker is installed following the instructions in the Requirements section above.<\/em><\/p>\nThe following steps need to be executed on every node. If you need to use sudo, prefix each command with sudo. If you already have users that can access the machine using a SSH key and can access the Docker socket, you can skip this step.<\/p>\n
# Login to the node
\n$ ssh [email protected]<\/a>
\n# Create a Linux user called rke, create home directory, and add to docker group
\n$ useradd -m -G docker rke
\n# Switch user to rke and create SSH directories
\n$ su – rke
\n$ mkdir $HOME\/.ssh
\n$ chmod 700 $HOME\/.ssh
\n$ touch $HOME\/.ssh\/authorized_keys
\n# Test Docker socket access
\n$ docker version
\nClient:
\nVersion: 17.03.2-ce
\nAPI version: 1.27
\nGo version: go1.7.5
\nGit commit: f5ec1e2
\nBuilt: Tue Jun 27 03:35:14 2017
\nOS\/Arch: linux\/amd64<\/p>\nServer:
\nVersion: 17.03.2-ce
\nAPI version: 1.27 (minimum version 1.12)
\nGo version: go1.7.5
\nGit commit: f5ec1e2
\nBuilt: Tue Jun 27 03:35:14 2017
\nOS\/Arch: linux\/amd64
\nExperimental: false<\/p>\n
Configuring SSH keys<\/h3>\n
In this post we will create new keys but feel free to use your existing keys. Just make sure you specify them correctly when we configure the keys in RKE.<\/p>\n
Note: If you want to use SSH keys with a passphrase, you will need to have ssh-agent running with the key added and specify \u2013ssh-agent-auth when running RKE.<\/em><\/p>\nCreating SSH key pair
\nCreating an SSH key pair can be done by using ssh-keygen , you can execute this on your workstation\/laptop\/computer. It is highly recommended to put a passphrase on your SSH private key. If you lose your SSH private key (and not have a passphrase on it), anyone can use it to access your nodes.<\/p>\n
$ ssh-keygen
\nGenerating public\/private rsa key pair.
\nEnter file in which to save the key ($HOME\/.ssh\/id_rsa):
\nEnter passphrase (empty for no passphrase):
\nEnter same passphrase again:
\nYour identification has been saved in $HOME\/.ssh\/id_rsa.
\nYour public key has been saved in $HOME\/.ssh\/id_rsa.pub.
\nThe key fingerprint is:
\nxxx<\/p>\n
After creating the SSH key pair, you should have the following files:<\/p>\n
\n- $HOME\/.ssh\/id_rsa (SSH private key, keep this secure)<\/li>\n
- $HOME\/.ssh\/id_rsa.pub (SSH public key)<\/li>\n<\/ul>\n
Copy the SSH public key to the nodes
\nTo be able to access the nodes using the created SSH key pair, you will need to install the SSH public key onto the nodes.<\/p>\n
Execute this for every node (where hostname is the IP\/hostname of the node):<\/p>\n
# Install the SSH public key on the node
\n$ cat $HOME\/.ssh\/id_rsa.pub | ssh hostname “sudo tee -a \/home\/rke\/.ssh\/authorized_keys”<\/p>\n
Note: This post is demonstrating how you create a separate user for RKE. Because of this, we can\u2019t use ssh-copy-id as it only works for installing keys to the same user as is used for the SSH connection.<\/em><\/p>\nSetup ssh-agent<\/p>\n
Note: If you chose not to put a passphrase on your SSH private key, you can skip this step.<\/em><\/p>\nThis needs to be executed on your workstation\/laptop\/computer:<\/p>\n
# Run ssh-agent and configure the correct environment variables
\n$ eval $(ssh-agent)
\nAgent pid 5151
\n# Add the private key to the ssh-agent
\n$ ssh-add $HOME\/.ssh\/id_rsa
\nIdentity added: $HOME\/.ssh\/id_rsa ($HOME\/.ssh\/id_rsa)<\/p>\n
Test SSH connectivity
\nLast step is to test if we can access the node using the SSH private key. This needs to be executed on your workstation\/laptop\/computer, replacing hostname with each of the nodes IP\/hostname):<\/p>\n
$ ssh -i $HOME\/.ssh\/id_rsa [email protected]<\/a> docker version
\nClient:
\nVersion: 17.03.2-ce
\nAPI version: 1.27
\nGo version: go1.7.5
\nGit commit: f5ec1e2
\nBuilt: Tue Jun 27 03:35:14 2017
\nOS\/Arch: linux\/amd64<\/p>\nServer:
\nVersion: 17.03.2-ce
\nAPI version: 1.27 (minimum version 1.12)
\nGo version: go1.7.5
\nGit commit: f5ec1e2
\nBuilt: Tue Jun 27 03:35:14 2017
\nOS\/Arch: linux\/amd64
\nExperimental: false<\/p>\n
Configuring and running RKE<\/h3>\n
Get RKE for your platform at:
\nhttps:\/\/github.com\/rancher\/rke\/releases\/latest<\/a>.<\/p>\nRKE will run on your workstation\/laptop\/computer.<\/p>\n
For this post I\u2019ve renamed the RKE binary to rke, to make the commands generic for each platform. You can do the same by running:<\/p>\n
Test if RKE can be successfully executed by using the following command:<\/p>\n
# Download RKE for MacOS (Darwin)
\n$ wget https:\/\/github.com\/rancher\/rke\/releases\/download\/v0.1.9\/rke_darwin-amd64
\n# Rename binary to rke
\nmv rke_darwin-amd64 rke
\n# Make RKE binary executable
\n$ chmod +x rke
\n# Show RKE version
\n$ .\/rke –version
\nrke version v0.1.9<\/p>\n
Next step is to create a cluster configuration file (by default it will be cluster.yml). This contains all information to build the Kubernetes cluster, like node connection info, what roles to apply to what node etcetera. All configuration options<\/a> can be found in the documentation. You can create the cluster configuration file by running .\/rke config and answering the questions. For this post, you will create a 3 node cluster with every role on each node (answer y for every role), and we will add the Kubernetes Dashboard as addon (Using https:\/\/raw.githubusercontent.com\/kubernetes\/dashboard\/master\/src\/deploy\/recommended\/kubernetes-dashboard.yaml<\/a>). To access the Kubernetes Dashboard, you need a Service Account token which will be created by adding https:\/\/gist.githubusercontent.com\/superseb\/499f2caa2637c404af41cfb7e5f4a938\/raw\/930841ac00653fdff8beca61dab9a20bb8983782\/k8s-dashboard-user.yml<\/a> to the addons.<\/p>\nRegarding answering the question to create the cluster configuration file:<\/p>\n
\n- The values in brackets, for instance [22] for SSH Port, are defaults and can just be used by pressing the Enter key.<\/li>\n
- The default SSH Private Key would do, if you have another key, please change it.\n
$ .\/rke config
\n[+] Cluster Level SSH Private Key Path [~\/.ssh\/id_rsa]: ~\/.ssh\/id_rsa
\n[+] Number of Hosts [1]: 3
\n[+] SSH Address of host (1) [none]: ip_or_dns_host1
\n[+] SSH Port of host (1) [22]:
\n[+] SSH Private Key Path of host (ip_or_dns_host1) [none]:
\n[-] You have entered empty SSH key path, trying fetch from SSH key parameter
\n[+] SSH Private Key of host (ip_or_dns_host1) [none]:
\n[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~\/.ssh\/id_rsa
\n[+] SSH User of host (ip_or_dns_host1) [ubuntu]: rke
\n[+] Is host (ip_or_dns_host1) a Control Plane host (y\/n)? [y]: y
\n[+] Is host (ip_or_dns_host1) a Worker host (y\/n)? [n]: y
\n[+] Is host (ip_or_dns_host1) an etcd host (y\/n)? [n]: y
\n[+] Override Hostname of host (ip_or_dns_host1) [none]:
\n[+] Internal IP of host (ip_or_dns_host1) [none]:
\n[+] Docker socket path on host (ip_or_dns_host1) [\/var\/run\/docker.sock]:
\n[+] SSH Address of host (2) [none]: ip_or_dns_host2
\n[+] SSH Port of host (2) [22]:
\n[+] SSH Private Key Path of host (ip_or_dns_host2) [none]:
\n[-] You have entered empty SSH key path, trying fetch from SSH key parameter
\n[+] SSH Private Key of host (ip_or_dns_host2) [none]:
\n[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~\/.ssh\/id_rsa
\n[+] SSH User of host (ip_or_dns_host2) [ubuntu]: rke
\n[+] Is host (ip_or_dns_host2) a Control Plane host (y\/n)? [y]: y
\n[+] Is host (ip_or_dns_host2) a Worker host (y\/n)? [n]: y
\n[+] Is host (ip_or_dns_host2) an etcd host (y\/n)? [n]: y
\n[+] Override Hostname of host (ip_or_dns_host2) [none]:
\n[+] Internal IP of host (ip_or_dns_host2) [none]:
\n[+] Docker socket path on host (ip_or_dns_host2) [\/var\/run\/docker.sock]:
\n[+] SSH Address of host (3) [none]: ip_or_dns_host3
\n[+] SSH Port of host (3) [22]:
\n[+] SSH Private Key Path of host (ip_or_dns_host3) [none]:
\n[-] You have entered empty SSH key path, trying fetch from SSH key parameter
\n[+] SSH Private Key of host (ip_or_dns_host3) [none]:
\n[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~\/.ssh\/id_rsa
\n[+] SSH User of host (ip_or_dns_host3) [ubuntu]: rke
\n[+] Is host (ip_or_dns_host3) a Control Plane host (y\/n)? [y]: y
\n[+] Is host (ip_or_dns_host3) a Worker host (y\/n)? [n]: y
\n[+] Is host (ip_or_dns_host3) an etcd host (y\/n)? [n]: y
\n[+] Override Hostname of host (ip_or_dns_host3) [none]:
\n[+] Internal IP of host (ip_or_dns_host3) [none]:
\n[+] Docker socket path on host (ip_or_dns_host3) [\/var\/run\/docker.sock]:
\n[+] Network Plugin Type (flannel, calico, weave, canal) [canal]:
\n[+] Authentication Strategy [x509]:
\n[+] Authorization Mode (rbac, none) [rbac]:
\n[+] Kubernetes Docker image [rancher\/hyperkube:v1.11.1-rancher1]:
\n[+] Cluster domain [cluster.local]:
\n[+] Service Cluster IP Range [10.43.0.0\/16]:
\n[+] Enable PodSecurityPolicy [n]:
\n[+] Cluster Network CIDR [10.42.0.0\/16]:
\n[+] Cluster DNS Service IP [10.43.0.10]:
\n[+] Add addon manifest URLs or YAML files [no]: yes
\n[+] Enter the Path or URL for the manifest [none]: https:\/\/raw.githubusercontent.com\/kubernetes\/dashboard\/master\/src\/deploy\/recommended\/kubernetes-dashboard.yaml
\n[+] Add another addon [no]: yes
\n[+] Enter the Path or URL for the manifest [none]: https:\/\/gist.githubusercontent.com\/superseb\/499f2caa2637c404af41cfb7e5f4a938\/raw\/930841ac00653fdff8beca61dab9a20bb8983782\/k8s-dashboard-user.yml
\n[+] Add another addon [no]: no<\/li>\n<\/ul>\n
When the last question is answered, the cluster.yml file will be created in the same directory as RKE was run:<\/p>\n
ls -la cluster.yml
\n-rw-r—– 1 user user 3688 Sep 17 12:50 cluster.yml<\/p>\n
You are now ready to build your Kubernetes cluster. This can be done by running rke up. Before you run the command, make sure the ports required<\/a> are opened between your workstation\/laptop\/computer and the nodes, and between each of the nodes. You can now build your cluster using the following command:<\/p>\n$ .\/rke up
\nINFO[0000] Building Kubernetes cluster
\n…
\nINFO[0151] Finished building Kubernetes cluster successfully<\/p>\n
If all went well, you should have a lot of output from the command but it should end with Finished building Kubernetes cluster successfully. It will also write a kubeconfig file as kube_config_cluster.yml . You can use that file to connect to your Kubernetes cluster.<\/p>\n
Exploring your Kubernetes cluster<\/h3>\n
Make sure you have kubectl installed, see https:\/\/kubernetes.io\/docs\/tasks\/tools\/install-kubectl\/<\/a> how to get it for your platform.<\/p>\nNote: When running kubectl, it automatically tries to use a kubeconfig from the default location; $HOME\/.kube\/config. In the examples, we explicitly specify the kubeconfig file using –kubeconfig kube_config_cluster.yml. If you don\u2019t want to specify the kubeconfig file every time, you can copy the file kube_config_cluster.yml to $HOME\/.kube\/config. (you probably need to create the directory $HOME\/.kube first)<\/em><\/p>\nStart with querying the server for its version:<\/p>\n
$ kubectl –kubeconfig kube_config_cluster.yml version<\/p>\n
Client Version: version.Info
\nServer Version: version.Info<\/p>\n
One of the first things to check, is if all nodes are in Ready state:<\/p>\n
$ kubectl –kubeconfig kube_config_cluster.yml get nodes
\nNAME STATUS ROLES AGE VERSION
\nhost1 Ready controlplane,etcd,worker 11m v1.11.1
\nhost2 Ready controlplane,etcd,worker 11m v1.11.1
\nhost3 Ready controlplane,etcd,worker 11m v1.11.1<\/p>\n
When you generated the cluster configuration file, you added the Kubernetes dashboard addon to be deployed on the cluster. You can check the status of the deployment using:<\/p>\n
$ kubectl –kubeconfig kube_config_cluster.yml get deploy -n kube-system -l k8s-app=kubernetes-dashboard<\/p>\n
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
\nkubernetes-dashboard 1 1 1 1 17m<\/p>\n
By default, the deployments are not exposed to the outside. If you want to visit the Kubernetes Dashboard in your browser, you will need to expose the deployment externally (which we will do in our demo application later) or use the built-in proxy functionality of kubectl. This will open the 127.0.0.1:8001 (your local machine on port 8001) and tunnel it to the Kubernetes cluster.<\/p>\n
Before you can visit the Kubernetes Dashboard, you need to retrieve the token to login to the dashboard. By default, it runs under a very limited account and will not be able to show you all the resources in your cluster. The second addon we added when creating the cluster configuration file created the account and token we need (this is based upon https:\/\/github.com\/kubernetes\/dashboard\/wiki\/Creating-sample-user<\/a>)<\/p>\nYou can retrieve the token by running:<\/p>\n
$ kubectl –kubeconfig kube_config_cluster.yml -n kube-system describe secret $(kubectl –kubeconfig kube_config_cluster.yml -n kube-system get secret | grep admin-user | awk ”) | grep ^token: | awk ‘{ print $2 }’<\/p>\n
eyJhbGciOiJSUzI1NiIs….<more_characters><\/p>\n
The string that is returned, is the token you need to login to the dashboard. Copy the whole string.<\/p>\n
Set up the kubectl proxy as follows:<\/p>\n
$ kubectl –kubeconfig kube_config_cluster.yml proxy<\/p>\n
Starting to serve on 127.0.0.1:8001<\/p>\n
And open the following URL:<\/p>\n
http:\/\/localhost:8001\/api\/v1\/namespaces\/kube-system\/services\/https:kubernetes-dashboard:\/proxy\/<\/a><\/p>\nWhen prompted for login, choose Token, paste the token and click Sign In.<\/p>\n
Note: When you don\u2019t get a login screen, open it manually by clicking Sign In on the top right.<\/em><\/p>\nRun a demo application<\/h3>\n
Last step of this post, running a demo application and exposing it. For this example you will run a demo application superseb\/rancher-demo, which is a web UI showing the scale of a deployment. It will be exposed using an Ingress, which is handled by the NGINX Ingress controller that is deployed by default. If you want to know more about Ingress, please see https:\/\/kubernetes.io\/docs\/concepts\/services-networking\/ingress\/<\/a><\/p>\nStart by deploying and exposing the demo application (which runs on port 8080):<\/p>\n
$ kubectl –kubeconfig kube_config_cluster.yml run –image=superseb\/rancher-demo rancher-demo –port 8080 –expose
\nservice\/rancher-demo created
\ndeployment.apps\/rancher-demo created<\/p>\n
Check the status of your deployment:<\/p>\n
$ kubectl –kubeconfig kube_config_cluster.yml rollout status deployment\/rancher-demo
\n…
\ndeployment “rancher-demo” successfully rolled out<\/p>\n
The command kubectl run is the easiest way to get a container running on your cluster. It takes an image parameter to specify the Docker image and a name at minimum. In this case, we also want to configure the port that this container exposes (internally), and expose it. What happened was that there was a Deployment created (and a ReplicaSet) with a scale of 1 (default), and a Service was created to abstract access to the pods (which can contain one or more containers, in this case 1). For more information on these subjects check the following links:<\/p>\n
\n- https:\/\/kubernetes.io\/docs\/tutorials\/kubernetes-basics\/deploy-app\/deploy-intro\/<\/a><\/li>\n
- https:\/\/kubernetes.io\/docs\/tutorials\/kubernetes-basics\/expose\/expose-intro\/<\/a><\/li>\n<\/ul>\n
RKE deploys the NGINX Ingress controller by default on every node. This opens op port 80 and port 443, and can serve as main entrypoint for any created Ingress. An Ingress can contain a single host or multiple, multiple paths, and you can configure SSL certificates. In this post you will configure a basic Ingress, making our demo application accessible on a certain hostname. In the example we will use rancher-demo.domain.test as hostname to access the demo application.<\/p>\n
Note: To access our test domain you have to add the domain name to \/etc\/hosts to visit the UI, as it\u2019s not a valid DNS name. If you have access to your own domain, you can add a DNS A record pointing to each of the nodes.<\/em><\/p>\n![\"\"](\"https:\/\/rancher.com\/img\/blog\/2018\/2018-09-26-setup-k8s-ingress.png\")
<\/p>\n
The only part that is not created, is the Ingress. Let\u2019s create an Ingress calledrancher-demo-ingress, having a host specification to match requests to our test domain (rancher-demo.domain.test), and pointing it to our Service called rancher-demo on port 8080. Save the following content to a file called ingress.yml:<\/p>\n
apiVersion: extensions\/v1beta1
\nkind: Ingress
\nmetadata:
\nname: rancher-demo-ingress
\nspec:
\nrules:
\n– host: rancher-demo.domain.test
\nhttp:
\npaths:
\n– path: \/
\nbackend:
\nserviceName: rancher-demo
\nservicePort: 8080<\/p>\n
Create this Ingress using kubectl:<\/p>\n
$ kubectl –kubeconfig kube_config_cluster.yml apply -f ingress.yml
\ningress.extensions\/rancher-demo-ingress created<\/p>\n
It is time to test accessing the demo application. You can try it on the command-line first, instruct curl to resolve the test domain to each of the nodes:<\/p>\n
# Get node IP addresses
\n$ kubectl –kubeconfig kube_config_cluster.yml get nodes
\nNAME STATUS ROLES AGE VERSION
\n10.0.0.1 Ready controlplane,etcd,worker 3h v1.11.1
\n10.0.0.2 Ready controlplane,etcd,worker 3h v1.11.1
\n10.0.0.3 Ready controlplane,etcd,worker 3h v1.11.1
\n# Test accessing the demo application
\n$ curl –resolve rancher-demo.domain.test:80:10.0.0.1 [http:\/\/rancher-demo.domain.test\/ping](http:\/\/rancher-demo.domain.test\/ping)<\/p>\n
{“instance”:”rancher-demo-5cbfb4b4-thmbh”,”version”:”0.1″}
\n$ curl –resolve rancher-demo.domain.test:80:10.0.0.2 [http:\/\/rancher-demo.domain.test\/ping](http:\/\/rancher-demo.domain.test\/ping)<\/p>\n
{“instance”:”rancher-demo-5cbfb4b4-thmbh”,”version”:”0.1″}
\n$ curl –resolve rancher-demo.domain.test:80:10.0.0.3
\n[http:\/\/rancher-demo.domain.test\/ping](http:\/\/rancher-demo.domain.test\/ping)<\/p>\n
{“instance”:”rancher-demo-5cbfb4b4-thmbh”,”version”:”0.1″}<\/p>\n
If you use the test domain, you will need to add it to your machine\u2019s \/etc\/hosts file to be able to reach it properly.<\/p>\n
echo “10.0.0.1 rancher-demo.domain.test” | sudo tee -a \/etc\/hosts<\/p>\n
Now visit http:\/\/rancher-demo.domain.test<\/a> in your browser.<\/p>\nIf this has all worked out, you can fill up the demo application a bit more by scaling up your Deployment:<\/p>\n
$ kubectl –kubeconfig kube_config_cluster.yml scale deploy\/rancher-demo –replicas=10
\ndeployment.extensions\/rancher-demo scaled<\/p>\n
Note: Make sure to clean up the \/etc\/hosts entry when you are done.<\/em><\/p>\nClosing words<\/h3>\n
This started as a post how to create a Kubernetes cluster in under 10 minutes, but along the way I tried to add some useful information how certain parts work. To avoid having a post that takes a day to read (explaining every part), there will be other posts describing certain parts. For now, I\u2019ve linked as much resources as possible to existing documentation where you can learn more.<\/p>\n
\n- RKE documentation<\/a><\/li>\n
- NGINX Ingress controller<\/a><\/li>\n
- Learn Kubernetes basics<\/a><\/li>\n<\/ul>\n
![\"Sebastiaan](\"https:\/\/rancher.com\/img\/bio\/sebastiaan-van-steenis.jpg\")
<\/p>\n
Sebastiaan van Steenis<\/p>\n
Support Engineer<\/p>\n
![\"github\"](\"https:\/\/rancher.com\/img\/icon-github.svg\")
<\/a><\/p>\n
You will be using RKE from your workstation. Download the latest version for your platform at: After creating the cluster, we will use the default Kubernetes command-line tool called kubectl to interact with the cluster. Most commonly used Linux distribution is Ubuntu 16.04, this is what will be used in this post. Make sure swap is disabled by running swapoff -a and removing any swap entry in \/etc\/fstab. You must be able to access the node using SSH. As this is a multi-node cluster, the required ports<\/a> need to be opened before proceeding.<\/p>\n Kubernetes only validates Docker up to 17.03.2 (See https:\/\/github.com\/kubernetes\/kubernetes\/blob\/master\/CHANGELOG-1.11.md#external-dependencies<\/a>). Make sure the requirements listed above are fulfilled before you proceed.<\/p>\n RKE can be run from any platform (the binary is available for MacOS\/Linux\/Windows), in this example it will run on your workstation\/laptop\/computer. The examples in this post are based on MacOS\/Linux.<\/p>\n RKE will connect to the nodes using a configured SSH private key (the nodes should have the matching SSH public key installed for the SSH user) and setup a tunnel to access the Docker socket (\/var\/run\/docker.sock by default, but configurable). This means that the configured SSH user must have access to the Docker socket on the machine, we will go over this in Creating the Linux user account.<\/p>\n Note: Make sure Docker is installed following the instructions in the Requirements section above.<\/em><\/p>\n The following steps need to be executed on every node. If you need to use sudo, prefix each command with sudo. If you already have users that can access the machine using a SSH key and can access the Docker socket, you can skip this step.<\/p>\n # Login to the node Server: In this post we will create new keys but feel free to use your existing keys. Just make sure you specify them correctly when we configure the keys in RKE.<\/p>\n Note: If you want to use SSH keys with a passphrase, you will need to have ssh-agent running with the key added and specify \u2013ssh-agent-auth when running RKE.<\/em><\/p>\n Creating SSH key pair $ ssh-keygen After creating the SSH key pair, you should have the following files:<\/p>\n Copy the SSH public key to the nodes Execute this for every node (where hostname is the IP\/hostname of the node):<\/p>\n # Install the SSH public key on the node Note: This post is demonstrating how you create a separate user for RKE. Because of this, we can\u2019t use ssh-copy-id as it only works for installing keys to the same user as is used for the SSH connection.<\/em><\/p>\n Setup ssh-agent<\/p>\n Note: If you chose not to put a passphrase on your SSH private key, you can skip this step.<\/em><\/p>\n This needs to be executed on your workstation\/laptop\/computer:<\/p>\n # Run ssh-agent and configure the correct environment variables Test SSH connectivity $ ssh -i $HOME\/.ssh\/id_rsa [email protected]<\/a> docker version Server: Get RKE for your platform at: RKE will run on your workstation\/laptop\/computer.<\/p>\n For this post I\u2019ve renamed the RKE binary to rke, to make the commands generic for each platform. You can do the same by running:<\/p>\n Test if RKE can be successfully executed by using the following command:<\/p>\n # Download RKE for MacOS (Darwin) Next step is to create a cluster configuration file (by default it will be cluster.yml). This contains all information to build the Kubernetes cluster, like node connection info, what roles to apply to what node etcetera. All configuration options<\/a> can be found in the documentation. You can create the cluster configuration file by running .\/rke config and answering the questions. For this post, you will create a 3 node cluster with every role on each node (answer y for every role), and we will add the Kubernetes Dashboard as addon (Using https:\/\/raw.githubusercontent.com\/kubernetes\/dashboard\/master\/src\/deploy\/recommended\/kubernetes-dashboard.yaml<\/a>). To access the Kubernetes Dashboard, you need a Service Account token which will be created by adding https:\/\/gist.githubusercontent.com\/superseb\/499f2caa2637c404af41cfb7e5f4a938\/raw\/930841ac00653fdff8beca61dab9a20bb8983782\/k8s-dashboard-user.yml<\/a> to the addons.<\/p>\n Regarding answering the question to create the cluster configuration file:<\/p>\n $ .\/rke config When the last question is answered, the cluster.yml file will be created in the same directory as RKE was run:<\/p>\n ls -la cluster.yml You are now ready to build your Kubernetes cluster. This can be done by running rke up. Before you run the command, make sure the ports required<\/a> are opened between your workstation\/laptop\/computer and the nodes, and between each of the nodes. You can now build your cluster using the following command:<\/p>\n $ .\/rke up If all went well, you should have a lot of output from the command but it should end with Finished building Kubernetes cluster successfully. It will also write a kubeconfig file as kube_config_cluster.yml . You can use that file to connect to your Kubernetes cluster.<\/p>\n Make sure you have kubectl installed, see https:\/\/kubernetes.io\/docs\/tasks\/tools\/install-kubectl\/<\/a> how to get it for your platform.<\/p>\n Note: When running kubectl, it automatically tries to use a kubeconfig from the default location; $HOME\/.kube\/config. In the examples, we explicitly specify the kubeconfig file using –kubeconfig kube_config_cluster.yml. If you don\u2019t want to specify the kubeconfig file every time, you can copy the file kube_config_cluster.yml to $HOME\/.kube\/config. (you probably need to create the directory $HOME\/.kube first)<\/em><\/p>\n Start with querying the server for its version:<\/p>\n $ kubectl –kubeconfig kube_config_cluster.yml version<\/p>\n Client Version: version.Info One of the first things to check, is if all nodes are in Ready state:<\/p>\n $ kubectl –kubeconfig kube_config_cluster.yml get nodes When you generated the cluster configuration file, you added the Kubernetes dashboard addon to be deployed on the cluster. You can check the status of the deployment using:<\/p>\n $ kubectl –kubeconfig kube_config_cluster.yml get deploy -n kube-system -l k8s-app=kubernetes-dashboard<\/p>\n NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE By default, the deployments are not exposed to the outside. If you want to visit the Kubernetes Dashboard in your browser, you will need to expose the deployment externally (which we will do in our demo application later) or use the built-in proxy functionality of kubectl. This will open the 127.0.0.1:8001 (your local machine on port 8001) and tunnel it to the Kubernetes cluster.<\/p>\n Before you can visit the Kubernetes Dashboard, you need to retrieve the token to login to the dashboard. By default, it runs under a very limited account and will not be able to show you all the resources in your cluster. The second addon we added when creating the cluster configuration file created the account and token we need (this is based upon https:\/\/github.com\/kubernetes\/dashboard\/wiki\/Creating-sample-user<\/a>)<\/p>\n You can retrieve the token by running:<\/p>\n $ kubectl –kubeconfig kube_config_cluster.yml -n kube-system describe secret $(kubectl –kubeconfig kube_config_cluster.yml -n kube-system get secret | grep admin-user | awk ”) | grep ^token: | awk ‘{ print $2 }’<\/p>\n eyJhbGciOiJSUzI1NiIs….<more_characters><\/p>\n The string that is returned, is the token you need to login to the dashboard. Copy the whole string.<\/p>\n Set up the kubectl proxy as follows:<\/p>\n $ kubectl –kubeconfig kube_config_cluster.yml proxy<\/p>\n Starting to serve on 127.0.0.1:8001<\/p>\n And open the following URL:<\/p>\n http:\/\/localhost:8001\/api\/v1\/namespaces\/kube-system\/services\/https:kubernetes-dashboard:\/proxy\/<\/a><\/p>\n When prompted for login, choose Token, paste the token and click Sign In.<\/p>\n Note: When you don\u2019t get a login screen, open it manually by clicking Sign In on the top right.<\/em><\/p>\n Last step of this post, running a demo application and exposing it. For this example you will run a demo application superseb\/rancher-demo, which is a web UI showing the scale of a deployment. It will be exposed using an Ingress, which is handled by the NGINX Ingress controller that is deployed by default. If you want to know more about Ingress, please see https:\/\/kubernetes.io\/docs\/concepts\/services-networking\/ingress\/<\/a><\/p>\n Start by deploying and exposing the demo application (which runs on port 8080):<\/p>\n $ kubectl –kubeconfig kube_config_cluster.yml run –image=superseb\/rancher-demo rancher-demo –port 8080 –expose Check the status of your deployment:<\/p>\n $ kubectl –kubeconfig kube_config_cluster.yml rollout status deployment\/rancher-demo The command kubectl run is the easiest way to get a container running on your cluster. It takes an image parameter to specify the Docker image and a name at minimum. In this case, we also want to configure the port that this container exposes (internally), and expose it. What happened was that there was a Deployment created (and a ReplicaSet) with a scale of 1 (default), and a Service was created to abstract access to the pods (which can contain one or more containers, in this case 1). For more information on these subjects check the following links:<\/p>\n RKE deploys the NGINX Ingress controller by default on every node. This opens op port 80 and port 443, and can serve as main entrypoint for any created Ingress. An Ingress can contain a single host or multiple, multiple paths, and you can configure SSL certificates. In this post you will configure a basic Ingress, making our demo application accessible on a certain hostname. In the example we will use rancher-demo.domain.test as hostname to access the demo application.<\/p>\n Note: To access our test domain you have to add the domain name to \/etc\/hosts to visit the UI, as it\u2019s not a valid DNS name. If you have access to your own domain, you can add a DNS A record pointing to each of the nodes.<\/em><\/p>\n The only part that is not created, is the Ingress. Let\u2019s create an Ingress calledrancher-demo-ingress, having a host specification to match requests to our test domain (rancher-demo.domain.test), and pointing it to our Service called rancher-demo on port 8080. Save the following content to a file called ingress.yml:<\/p>\n apiVersion: extensions\/v1beta1 Create this Ingress using kubectl:<\/p>\n $ kubectl –kubeconfig kube_config_cluster.yml apply -f ingress.yml It is time to test accessing the demo application. You can try it on the command-line first, instruct curl to resolve the test domain to each of the nodes:<\/p>\n # Get node IP addresses {“instance”:”rancher-demo-5cbfb4b4-thmbh”,”version”:”0.1″} {“instance”:”rancher-demo-5cbfb4b4-thmbh”,”version”:”0.1″} {“instance”:”rancher-demo-5cbfb4b4-thmbh”,”version”:”0.1″}<\/p>\n If you use the test domain, you will need to add it to your machine\u2019s \/etc\/hosts file to be able to reach it properly.<\/p>\n echo “10.0.0.1 rancher-demo.domain.test” | sudo tee -a \/etc\/hosts<\/p>\n Now visit http:\/\/rancher-demo.domain.test<\/a> in your browser.<\/p>\n If this has all worked out, you can fill up the demo application a bit more by scaling up your Deployment:<\/p>\n $ kubectl –kubeconfig kube_config_cluster.yml scale deploy\/rancher-demo –replicas=10 Note: Make sure to clean up the \/etc\/hosts entry when you are done.<\/em><\/p>\n This started as a post how to create a Kubernetes cluster in under 10 minutes, but along the way I tried to add some useful information how certain parts work. To avoid having a post that takes a day to read (explaining every part), there will be other posts describing certain parts. For now, I\u2019ve linked as much resources as possible to existing documentation where you can learn more.<\/p>\n Sebastiaan van Steenis<\/p>\n Support Engineer<\/p>\n
\nhttps:\/\/github.com\/rancher\/rke\/releases\/latest<\/a><\/p>\n\n
\nGet the latest version for your platform at:
\nhttps:\/\/kubernetes.io\/docs\/tasks\/tools\/install-kubectl\/<\/a><\/p>\n\n
\n
\nYou can use https:\/\/docs.docker.com\/install\/linux\/docker-ce\/ubuntu\/<\/a> to install Docker (make sure you install 17.03.2) or use this one-liner to install the correct version:
\ncurl https:\/\/releases.rancher.com\/install-docker\/17.03.sh<\/a> | sh<\/p>\nHow RKE works<\/h3>\n
<\/p>\nCreating the Linux user account<\/h3>\n
\n$ ssh [email protected]<\/a>
\n# Create a Linux user called rke, create home directory, and add to docker group
\n$ useradd -m -G docker rke
\n# Switch user to rke and create SSH directories
\n$ su – rke
\n$ mkdir $HOME\/.ssh
\n$ chmod 700 $HOME\/.ssh
\n$ touch $HOME\/.ssh\/authorized_keys
\n# Test Docker socket access
\n$ docker version
\nClient:
\nVersion: 17.03.2-ce
\nAPI version: 1.27
\nGo version: go1.7.5
\nGit commit: f5ec1e2
\nBuilt: Tue Jun 27 03:35:14 2017
\nOS\/Arch: linux\/amd64<\/p>\n
\nVersion: 17.03.2-ce
\nAPI version: 1.27 (minimum version 1.12)
\nGo version: go1.7.5
\nGit commit: f5ec1e2
\nBuilt: Tue Jun 27 03:35:14 2017
\nOS\/Arch: linux\/amd64
\nExperimental: false<\/p>\nConfiguring SSH keys<\/h3>\n
\nCreating an SSH key pair can be done by using ssh-keygen , you can execute this on your workstation\/laptop\/computer. It is highly recommended to put a passphrase on your SSH private key. If you lose your SSH private key (and not have a passphrase on it), anyone can use it to access your nodes.<\/p>\n
\nGenerating public\/private rsa key pair.
\nEnter file in which to save the key ($HOME\/.ssh\/id_rsa):
\nEnter passphrase (empty for no passphrase):
\nEnter same passphrase again:
\nYour identification has been saved in $HOME\/.ssh\/id_rsa.
\nYour public key has been saved in $HOME\/.ssh\/id_rsa.pub.
\nThe key fingerprint is:
\nxxx<\/p>\n\n
\nTo be able to access the nodes using the created SSH key pair, you will need to install the SSH public key onto the nodes.<\/p>\n
\n$ cat $HOME\/.ssh\/id_rsa.pub | ssh hostname “sudo tee -a \/home\/rke\/.ssh\/authorized_keys”<\/p>\n
\n$ eval $(ssh-agent)
\nAgent pid 5151
\n# Add the private key to the ssh-agent
\n$ ssh-add $HOME\/.ssh\/id_rsa
\nIdentity added: $HOME\/.ssh\/id_rsa ($HOME\/.ssh\/id_rsa)<\/p>\n
\nLast step is to test if we can access the node using the SSH private key. This needs to be executed on your workstation\/laptop\/computer, replacing hostname with each of the nodes IP\/hostname):<\/p>\n
\nClient:
\nVersion: 17.03.2-ce
\nAPI version: 1.27
\nGo version: go1.7.5
\nGit commit: f5ec1e2
\nBuilt: Tue Jun 27 03:35:14 2017
\nOS\/Arch: linux\/amd64<\/p>\n
\nVersion: 17.03.2-ce
\nAPI version: 1.27 (minimum version 1.12)
\nGo version: go1.7.5
\nGit commit: f5ec1e2
\nBuilt: Tue Jun 27 03:35:14 2017
\nOS\/Arch: linux\/amd64
\nExperimental: false<\/p>\nConfiguring and running RKE<\/h3>\n
\nhttps:\/\/github.com\/rancher\/rke\/releases\/latest<\/a>.<\/p>\n
\n$ wget https:\/\/github.com\/rancher\/rke\/releases\/download\/v0.1.9\/rke_darwin-amd64
\n# Rename binary to rke
\nmv rke_darwin-amd64 rke
\n# Make RKE binary executable
\n$ chmod +x rke
\n# Show RKE version
\n$ .\/rke –version
\nrke version v0.1.9<\/p>\n\n
\n[+] Cluster Level SSH Private Key Path [~\/.ssh\/id_rsa]: ~\/.ssh\/id_rsa
\n[+] Number of Hosts [1]: 3
\n[+] SSH Address of host (1) [none]: ip_or_dns_host1
\n[+] SSH Port of host (1) [22]:
\n[+] SSH Private Key Path of host (ip_or_dns_host1) [none]:
\n[-] You have entered empty SSH key path, trying fetch from SSH key parameter
\n[+] SSH Private Key of host (ip_or_dns_host1) [none]:
\n[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~\/.ssh\/id_rsa
\n[+] SSH User of host (ip_or_dns_host1) [ubuntu]: rke
\n[+] Is host (ip_or_dns_host1) a Control Plane host (y\/n)? [y]: y
\n[+] Is host (ip_or_dns_host1) a Worker host (y\/n)? [n]: y
\n[+] Is host (ip_or_dns_host1) an etcd host (y\/n)? [n]: y
\n[+] Override Hostname of host (ip_or_dns_host1) [none]:
\n[+] Internal IP of host (ip_or_dns_host1) [none]:
\n[+] Docker socket path on host (ip_or_dns_host1) [\/var\/run\/docker.sock]:
\n[+] SSH Address of host (2) [none]: ip_or_dns_host2
\n[+] SSH Port of host (2) [22]:
\n[+] SSH Private Key Path of host (ip_or_dns_host2) [none]:
\n[-] You have entered empty SSH key path, trying fetch from SSH key parameter
\n[+] SSH Private Key of host (ip_or_dns_host2) [none]:
\n[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~\/.ssh\/id_rsa
\n[+] SSH User of host (ip_or_dns_host2) [ubuntu]: rke
\n[+] Is host (ip_or_dns_host2) a Control Plane host (y\/n)? [y]: y
\n[+] Is host (ip_or_dns_host2) a Worker host (y\/n)? [n]: y
\n[+] Is host (ip_or_dns_host2) an etcd host (y\/n)? [n]: y
\n[+] Override Hostname of host (ip_or_dns_host2) [none]:
\n[+] Internal IP of host (ip_or_dns_host2) [none]:
\n[+] Docker socket path on host (ip_or_dns_host2) [\/var\/run\/docker.sock]:
\n[+] SSH Address of host (3) [none]: ip_or_dns_host3
\n[+] SSH Port of host (3) [22]:
\n[+] SSH Private Key Path of host (ip_or_dns_host3) [none]:
\n[-] You have entered empty SSH key path, trying fetch from SSH key parameter
\n[+] SSH Private Key of host (ip_or_dns_host3) [none]:
\n[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~\/.ssh\/id_rsa
\n[+] SSH User of host (ip_or_dns_host3) [ubuntu]: rke
\n[+] Is host (ip_or_dns_host3) a Control Plane host (y\/n)? [y]: y
\n[+] Is host (ip_or_dns_host3) a Worker host (y\/n)? [n]: y
\n[+] Is host (ip_or_dns_host3) an etcd host (y\/n)? [n]: y
\n[+] Override Hostname of host (ip_or_dns_host3) [none]:
\n[+] Internal IP of host (ip_or_dns_host3) [none]:
\n[+] Docker socket path on host (ip_or_dns_host3) [\/var\/run\/docker.sock]:
\n[+] Network Plugin Type (flannel, calico, weave, canal) [canal]:
\n[+] Authentication Strategy [x509]:
\n[+] Authorization Mode (rbac, none) [rbac]:
\n[+] Kubernetes Docker image [rancher\/hyperkube:v1.11.1-rancher1]:
\n[+] Cluster domain [cluster.local]:
\n[+] Service Cluster IP Range [10.43.0.0\/16]:
\n[+] Enable PodSecurityPolicy [n]:
\n[+] Cluster Network CIDR [10.42.0.0\/16]:
\n[+] Cluster DNS Service IP [10.43.0.10]:
\n[+] Add addon manifest URLs or YAML files [no]: yes
\n[+] Enter the Path or URL for the manifest [none]: https:\/\/raw.githubusercontent.com\/kubernetes\/dashboard\/master\/src\/deploy\/recommended\/kubernetes-dashboard.yaml
\n[+] Add another addon [no]: yes
\n[+] Enter the Path or URL for the manifest [none]: https:\/\/gist.githubusercontent.com\/superseb\/499f2caa2637c404af41cfb7e5f4a938\/raw\/930841ac00653fdff8beca61dab9a20bb8983782\/k8s-dashboard-user.yml
\n[+] Add another addon [no]: no<\/li>\n<\/ul>\n
\n-rw-r—– 1 user user 3688 Sep 17 12:50 cluster.yml<\/p>\n
\nINFO[0000] Building Kubernetes cluster
\n…
\nINFO[0151] Finished building Kubernetes cluster successfully<\/p>\nExploring your Kubernetes cluster<\/h3>\n
\nServer Version: version.Info<\/p>\n
\nNAME STATUS ROLES AGE VERSION
\nhost1 Ready controlplane,etcd,worker 11m v1.11.1
\nhost2 Ready controlplane,etcd,worker 11m v1.11.1
\nhost3 Ready controlplane,etcd,worker 11m v1.11.1<\/p>\n
\nkubernetes-dashboard 1 1 1 1 17m<\/p>\nRun a demo application<\/h3>\n
\nservice\/rancher-demo created
\ndeployment.apps\/rancher-demo created<\/p>\n
\n…
\ndeployment “rancher-demo” successfully rolled out<\/p>\n\n
<\/p>\n
\nkind: Ingress
\nmetadata:
\nname: rancher-demo-ingress
\nspec:
\nrules:
\n– host: rancher-demo.domain.test
\nhttp:
\npaths:
\n– path: \/
\nbackend:
\nserviceName: rancher-demo
\nservicePort: 8080<\/p>\n
\ningress.extensions\/rancher-demo-ingress created<\/p>\n
\n$ kubectl –kubeconfig kube_config_cluster.yml get nodes
\nNAME STATUS ROLES AGE VERSION
\n10.0.0.1 Ready controlplane,etcd,worker 3h v1.11.1
\n10.0.0.2 Ready controlplane,etcd,worker 3h v1.11.1
\n10.0.0.3 Ready controlplane,etcd,worker 3h v1.11.1
\n# Test accessing the demo application
\n$ curl –resolve rancher-demo.domain.test:80:10.0.0.1 [http:\/\/rancher-demo.domain.test\/ping](http:\/\/rancher-demo.domain.test\/ping)<\/p>\n
\n$ curl –resolve rancher-demo.domain.test:80:10.0.0.2 [http:\/\/rancher-demo.domain.test\/ping](http:\/\/rancher-demo.domain.test\/ping)<\/p>\n
\n$ curl –resolve rancher-demo.domain.test:80:10.0.0.3
\n[http:\/\/rancher-demo.domain.test\/ping](http:\/\/rancher-demo.domain.test\/ping)<\/p>\n
\ndeployment.extensions\/rancher-demo scaled<\/p>\nClosing words<\/h3>\n
\n
<\/p>\n<\/a><\/p>\n