{"id":1356,"date":"2019-02-21T07:21:42","date_gmt":"2019-02-21T07:21:42","guid":{"rendered":"https:\/\/www.appservgrid.com\/paw93\/?p=1356"},"modified":"2019-03-07T20:19:16","modified_gmt":"2019-03-07T20:19:16","slug":"architecture-of-ranchers-docker-machine-integration","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw93\/index.php\/2019\/02\/21\/architecture-of-ranchers-docker-machine-integration\/","title":{"rendered":"Architecture of Rancher’s Docker-machine Integration"},"content":{"rendered":"

As you may have seen, Rancher recently announced\u00a0<\/a>our integration
\nwith docker-machine<\/a>. This
\nintegration will allow users to spin up Rancher compute nodes across
\nmultiple cloud providers right from the Rancher UI. In our initial
\nrelease, we supported Digital Ocean. Amazon EC2 is soon to follow and
\nwe\u2019ll continue to add more cloud providers as interest dictates. We
\nbelieve this feature will really help the Zero-to-Docker _(and
\nZero-to-Rancher)_ experience. But the feature itself is not the focus
\nof this post. In this post, I want to detail the software architerture
\nemployed to achieve this integration. First, it\u2019s important to
\nunderstand that everyhting in Rancher is an API resource with a process
\nlifecycle. Containers, images, networks, and accounts are all API
\nresources with their own process lifecycles. When you deploy a machine
\nin the Rancher UI, you\u2019re creating a machine resource. It has three
\nlife cycle processes: 1. Create 2. Bootstrap 3. Delete The create
\nprocess is kicked off when the user creates a machine in the UI. When
\nthe create process completes, it auotmatically kicks off the bootstrap
\nprocess. Delete (perhaps obviously) occurs when the user chooses to
\ndelete or destroy the host. Our integration with machine is achieved
\nthrough a microservice that hooks into Rancher machine lifecycle events
\nand execs out to the docker-machine binary accordingly. You can check
\nout the source code for this service here:
\nhttps:\/\/github.com\/rancherio\/go-machine-service<\/a>.
\nLogically, the interaction looks like this:
\n\"machine\"<\/a>
\n…Sorry for the bad graphic. Anyway…<\/em> When you spin up Rancher
\nwith docker run rancher\/server … with the default configuration, the
\nRancher API, Rancher Process Server, DB, and Machine Microservice are
\nall processes living inside that container (and in fact, the API and
\nprocess server are the same process). The docker-machine binary is in
\nthe container as well but only runs when it is called. You may at this
\npoint be wondering about that event bus. In Rancher, we keep eventing
\ndead-simple<\/em> and above all follow this principle:<\/p>\n

There is no such thing as reliable messaging.<\/p><\/blockquote>\n

So, that \u201cevent bus\u201d consists of the microservice making a POST
\nrequest to the \/subsribe API endpoint. The response is a stream of
\nnewline-terminated json events, similar in concept to the docker event
\nstream<\/a>. The
\nprocess server is responsible for firing (and refiring) events until it
\nreceives a reply event (another API POST) indicating the event was
\nhandled. Further event handlers are blocked until the current event
\nhandler replies successfully. The microservice is responsible for
\nhandling the events, replying, and acting idempotently so that refires
\ncan occur without ill-effect. So when the machine microservie receives a
\ncreate event, it translate the machine API resource\u2019s prooperties into
\na docker-machine cli command and execs out to it. Since the machine
\ncreation process is long lived, the service monitors the standard out
\nand error of the call and sends corresponding status updates to the
\nRancher server. These are then presented to the user in the UI. When
\ndocker-machine reports that the machine was successfully created, the
\nmicroservice will reply to the original event it received from the
\nRancher server. The successful end of the create event will cause the
\nprocess server to automatically kick off the bootstrap event, which
\nmakes it way right back down to the machine microservice. When that
\nevent is received, we\u2019ll again exec out to docker-machine to get the
\ndetails needed to connect to the machine\u2019s docker daemon. We do this by
\nexecuting the docker-machine config command and parsing the response.
\nWith the connection parameters in hand, the service fires up a rancher
\nagent on the machine via docker run … rancher\/agent …. This is the
\nexact same command that a user would run if they wanted to manaully join
\na server to Rancher. When that container is up and running, it will
\nreport into the Rancher server and start hooking into container
\nlifecycle events in much the same way that this service hooks into
\nmachine lifecycle events. From there, it\u2019s business as normal for the
\nRancher server and the machine\u2019s rancher-agent. That about does it for
\nthe technical architecture of our docker-machine integration. There are
\na lot more interesting but minor technical detail to share, but I
\ndidn\u2019t want to go too far off into the weeds in this post. I\u2019ll write
\nup some follow up post sharing those details in the not-too-distant
\nfuture. Finally, shout out (and thanks) to Evan
\nHaslett<\/a>, Ben
\nFirshman<\/a>, and the rest of the docker-machine
\nteam and community for the help along the way. We look forward to more
\nexciting work with the docker-machine, including getting RancherOS in
\nthere. If you\u2019d like to learn more about Rancher, please schedule a
\ndemo and we\u2019ll walk you through the latest features, and our
\nfuture roadmap. Note: This post also appears
\non Craig\u2019s personal blog
\nhere<\/a>.
\nFeel free to check out that blog for more software engineering
\ninsights.<\/em><\/p>\n

Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

As you may have seen, Rancher recently announced\u00a0our integration with docker-machine. This integration will allow users to spin up Rancher compute nodes across multiple cloud providers right from the Rancher UI. In our initial release, we supported Digital Ocean. Amazon EC2 is soon to follow and we\u2019ll continue to add more cloud providers as interest … <\/p>\n

Continue reading “Architecture of Rancher’s Docker-machine Integration”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-1356","post","type-post","status-publish","format-standard","hentry","category-kubernetes"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/1356","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/comments?post=1356"}],"version-history":[{"count":1,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/1356\/revisions"}],"predecessor-version":[{"id":1431,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/1356\/revisions\/1431"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/media?parent=1356"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/categories?post=1356"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/tags?post=1356"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}