![\"Imgur\"](\"https:\/\/i.imgur.com\/yURoPT9.jpg\")
<\/p>\nAn important and complex aspect of container orchestration is scheduling the application containers. Appropriate placement of containers onto the shared infrastructure resources that are available is the key to achieve maximum performance at optimum compute resource usage.<\/p>\n
<\/p>\n
Cattle, which is the default orchestration engine for Rancher 1.6, provided various scheduling abilities to effectively place services, as documented here<\/a>.<\/p>\n With the release of the 2.0 version based on Kubernetes, Rancher now utilizes native Kubernetes scheduling. In this article we will look at the scheduling methods available in Rancher 2.0 in comparison to Cattle\u2019s scheduling support.<\/p>\n Key terminology differences, implementing key elements, and transforming Compose to YAML<\/p>\n Watch the video<\/a><\/p>\n Based on the native Kubernetes behavior, by default, pods in a Rancher 2.0 workload will be spread across the nodes (hosts) that are schedulable and have enough free capacity. But just like the 1.6 version, Rancher 2.0 also facilitates:<\/p>\n Here is how scheduling in the 1.6 UI looks. Rancher lets you either run all containers on a specific host, specify hard\/soft host labels, or use affinity\/anti-affinity rules while deploying services.<\/p>\n And here is the equivalent node scheduling UI for Rancher 2.0 that provides the same features while deploying workloads.<\/p>\n Rancher uses the underlying native Kubernetes constructs to specify node affinity\/anti-affinity. Detailed documentation from Kubernetes is available here<\/a>.<\/p>\n Let\u2019s run through some examples that schedule workload pods using these node scheduling options, and then check how the Kubernetes YAML specs look like in comparison to the 1.6 Docker Compose config.<\/p>\n While deploying a workload (navigate to your Cluster > Project > Workloads), it is possible to schedule all pods in your workload to a specific node.<\/p>\n Here I am deploying a workload of scale = 2 using the nginx image on a specific node.<\/p>\n Rancher will choose that node if there is enough compute resource availability and no port conflicts if hostPort is used. If the workload exposes itself using a nodePort that conflicts with another workload, the deployment gets created successfully, but no nodePort service is created. Therefore, the workload doesn\u2019t get exposed at all.<\/p>\n On the Workloads tab, you can list the workload group by node. I can see both of the pods for my Nginx workload are scheduled on the specified node:<\/p>\n Now here is what this scheduling rule looks like in Kubernetes pod specs:<\/p>\n I added a label foo=bar to node1 in my Rancher 2.0 cluster to test the label-based node scheduling rules.<\/p>\n Here is how to specify a host label affinity rule in the Rancher 2.0 UI. A hard affinity rule means that the host chosen must satisfy all the scheduling rules. If no such host can be found, the workload will fail to deploy.<\/p>\n In the PodSpec YAML, this rule translates to field nodeAffinity. Also note that I have included the Rancher 1.6 docker-compose.yml used to achieve the same scheduling behavior using labels.<\/p>\n If you are a Rancher 1.6 user, you know that a soft rule means that the scheduler should<\/em> try to deploy the application per the rule, but can deploy even if the rule is not satisfied by any host. Here is how to specify this rule in Rancher 2.0 UI.<\/p>\n The corresponding YAML specs for the pod are shown below.<\/p>\n Apart from the key = value host label matching rule, Kubernetes scheduling constructs also support the following operators:<\/p>\n So to achieve anti-affinity, you can use the operators NotIn and DoesNotExist for the node label.<\/p>\n If you are a Cattle user, you will be familiar with a few other scheduling options available in Rancher 1.6:<\/p>\n If you are using these options in your Rancher 1.6 setups, it is possible to replicate them in Rancher 2.0 using native Kubernetes scheduling options. As of v2.0.8, there is no UI support for these options while deploying workloads, but you can always use them by importing the Kubernetes YAML specs on a Rancher cluster.<\/p>\n This 1.6 option lets you schedule containers to a host where a container with a specific label is already present. To do this on Rancher 2.0, use Kubernetes inter-pod affinity and anti-affinity feature<\/a>.<\/p>\n As noted in these docs, Kubernetes allows you to constrain which nodes your pod can be scheduled to based on pod labels rather than node labels.<\/p>\n One of the most-used scheduling features in 1.6 was anti-affinity to the service itself using labels on containers. To replicate this behavior in Rancher 2.0, we can use pod anti-affinity constructs in Kubernetes YAML specs. For example, consider a Nginx web workload. To ensure that pods in this workload do not land on the same host, you can use the podAntiAffinity construct as shown below. By specifying podAntiAffinity using labels, we ensure that each Nginx replica does not co-locate on a single node.<\/p>\n Using Rancher CLI<\/a>, this workload can be deployed onto the Kubernetes cluster. Note that the above deployment specifies three replicas, and I have three schedulable nodes in the Kubernetes cluster.<\/p>\n Since podAntiAffinity is specified, the three pods end up on different nodes. To further check how podAntiAffinity applies, I can scale up the deployment to four pods. Notice that the fourth pod cannot get scheduled since the scheduler cannot find another node to satisfy the podAntiAffinity rule.<\/p>\n While you are creating a service in Rancher 1.6, you can specify the memory reservation and mCPU reservation in the Security\/Host tab in the UI. Cattle will schedule containers for the service onto hosts that have enough available compute resources.<\/p>\n In Rancher 2.0, you can specify the memory and CPU resources required by your workload pods using resources.requests.memory and resources.requests.cpu under the pod container specs. You can find more detail about these specs here<\/a>.<\/p>\n When you specify these resource requests, the Kubernetes scheduler will assign the pod to a node with capacity.<\/p>\n Rancher 1.6 has the ability to specify container labels on the host to only allow specific containers to be scheduled to it.<\/p>\n To achieve this in Rancher 2.0, use the equivalent Kubernetes feature of adding node taints (like host tags) and using tolerations<\/a> in your pod specs.<\/p>\n In Rancher 1.6, a global service<\/a> is a service with a container deployed on every host in the environment.<\/p>\n If a service has the label io.rancher.scheduler.global: ‘true’, then the Rancher 1.6 scheduler will schedule a service container on each host in the environment. As mentioned in the documentation, if a new host is added to the environment, and the host fulfills the global service\u2019s host requirements, the service will automatically be started on it by Rancher.<\/p>\n The sample below is an example of a global service in Rancher 1.6. Note that just placing the required label is sufficient to make a service global.<\/p>\n version: ‘2’ How can we deploy a global service in Rancher 2.0 using Kubernetes?<\/p>\n For this purpose, Rancher deploys a Kubernetes DaemonSet<\/a> object for the user\u2019s workload. A DaemonSet<\/em> functions exactly like the Rancher 1.6 global service. The Kubernetes scheduler will deploy a pod on each node of the cluster, and as new nodes are added, the scheduler will start new pods on them provided they match the scheduling requirements of the workload.<\/p>\n Additionally, in 2.0, you can also limit a DaemonSet to be deployed to nodes that have a specific label, as mentioned here<\/a>.<\/p>\n If you are a Rancher 1.6 user, to migrate your global service to Rancher 2.0 using the UI, navigate to your Cluster > Project > Workloads view. While deploying a workload, you can choose the following workload types:<\/p>\n This is what the corresponding Kubernetes YAML specs look like for the above DaemonSet<\/em> workload:<\/p>\n apiVersion: apps\/v1beta2 To migrate a Rancher 1.6 global service to Rancher 2.0 using its Compose config, follow these steps.<\/p>\n You can convert the docker-compose.yml file from Rancher 1.6 to Kubernetes YAML using the Kompose<\/a> tool, and then deploy the application using either the Kubectl client tool<\/a> or Rancher CLI<\/a> in the Kubernetes cluster.<\/p>\n Consider the docker-compose.yml specs mentioned above where the Nginx service is a global service. This is how it can be converted to Kubernetes YAML using Kompose:<\/p>\n Now configure the Rancher CLI against your Kubernetes Cluster and deploy the generated *-daemonset.yaml file.<\/p>\n As shown above, my Kubernetes cluster has two worker nodes where workloads can be scheduled, and deploying the global-daemonset.yaml started two pods for the Daemonset, one on each node.<\/p>\n In this article, we reviewed how the various scheduling functionalities of Rancher 1.6 can be migrated to Rancher 2.0. Most of the scheduling techniques have equivalent options available in Rancher 2.0, or they can be achieved via native Kubernetes constructs.<\/p>\n In the upcoming article, I will explore a bit about how service discovery support in Cattle can be replicated in a Rancher 2.0 setup – stay tuned!<\/p>\n Prachi Damle<\/p>\n Principal Software Engineer<\/p>\nHow to Migrate from Rancher 1.6 to Rancher 2.1 Online Meetup<\/h5>\n
Node Scheduling<\/h2>\n
\n
![\"Imgur\"](\"https:\/\/i.imgur.com\/MiqZSGj.png\")
<\/p>\n![\"Imgur\"](\"https:\/\/i.imgur.com\/Keswh5R.png\")
<\/p>\nExample: Run All Pods on a Specific Node<\/h3>\n
![\"Imgur\"](\"https:\/\/i.imgur.com\/UylWMFP.png\")
<\/p>\n![\"Imgur\"](\"https:\/\/i.imgur.com\/uOLA1p4.png\")
<\/p>\n![\"Imgur\"](\"https:\/\/i.imgur.com\/KXOCG56.png\")
<\/p>\nExample: Host Label Affinity\/Anti-Affinity<\/h3>\n
![\"Imgur\"](\"https:\/\/i.imgur.com\/O0Qy0tk.png\")
<\/p>\nHost Label Affinity: Hard<\/h4>\n
![\"Imgur\"](\"https:\/\/i.imgur.com\/QU2uKw3.png\")
<\/p>\n![\"Imgur\"](\"https:\/\/i.imgur.com\/NkOA1td.png\")
<\/p>\nHost Label Affinity: Soft<\/h4>\n
![\"Imgur\"](\"https:\/\/i.imgur.com\/RVAlMIw.png\")
<\/p>\n![\"Imgur\"](\"https:\/\/i.imgur.com\/8KyEMrS.png\")
<\/p>\nHost Label Anti-Affinity<\/h4>\n
![\"Imgur\"](\"https:\/\/i.imgur.com\/t214I2R.png\")
<\/p>\nSupport for Other 1.6 Scheduling Options<\/h2>\n
\n
Schedule Using Container labels<\/h3>\n
![\"Imgur\"](\"https:\/\/i.imgur.com\/VvN6NOU.png\")
<\/p>\n![\"Imgur\"](\"https:\/\/i.imgur.com\/OZP7KqG.png\")
<\/p>\n![\"Imgur\"](\"https:\/\/i.imgur.com\/5DdYUHf.png\")
<\/p>\nResource-Based Scheduling<\/h3>\n
Schedule Only Specific Services to Host<\/h3>\n
Global Service<\/h2>\n
\nservices:
\nglobal:
\nimage: nginx
\nstdin_open: true
\ntty: true
\nlabels:
\nio.rancher.container.pull_image: always
\nio.rancher.scheduler.global: ‘true’<\/p>\nDeploying a DaemonSet<\/em> Using Rancher 2.0 UI<\/h3>\n
![\"Imgur\"](\"https:\/\/i.imgur.com\/rYzp93g.png\")
<\/p>\n
\nkind: DaemonSet
\nmetadata:
\nlabels:
\nworkload.user.cattle.io\/workloadselector: daemonSet-default-globalapp
\nname: globalapp
\nnamespace: default
\nspec:
\nselector:
\nmatchLabels:
\nworkload.user.cattle.io\/workloadselector: daemonSet-default-globalapp
\ntemplate:
\nmetadata:
\nlabels:
\nworkload.user.cattle.io\/workloadselector: daemonSet-default-globalapp
\nspec:
\naffinity: {}
\ncontainers:
\n– image: nginx
\nimagePullPolicy: Always
\nname: globalapp
\nresources: {}
\nstdin: true
\ntty: true
\nrestartPolicy: Always<\/p>\nDocker Compose to Kubernetes YAML<\/h3>\n
![\"Imgur\"](\"https:\/\/i.imgur.com\/zo3gj38.png\")
<\/p>\n![\"Imgur\"](\"https:\/\/i.imgur.com\/PR3m49c.png\")
<\/p>\nConclusion<\/h2>\n
![\"Prachi](\"https:\/\/rancher.com\/img\/bio\/prachi-damle.jpg\")
<\/p>\n