{"id":814,"date":"2018-12-03T09:16:10","date_gmt":"2018-12-03T09:16:10","guid":{"rendered":"https:\/\/www.appservgrid.com\/paw93\/?p=814"},"modified":"2018-12-06T23:30:25","modified_gmt":"2018-12-06T23:30:25","slug":"continuous-delivery-of-everything-with-rancher-drone-and-terraform","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw93\/index.php\/2018\/12\/03\/continuous-delivery-of-everything-with-rancher-drone-and-terraform\/","title":{"rendered":"Continuous Delivery of Everything with Rancher, Drone, and Terraform"},"content":{"rendered":"<p>It\u2019s 8:00 PM. I just deployed to production, but nothing\u2019s working.<br \/>\n<em>Oh, wait. the production Kinesis stream doesn\u2019t exist, because the<br \/>\nCloudFormation template for production wasn\u2019t updated.<\/em> Okay, fix that.<br \/>\n9:00 PM. Redeploy. Still broken. <em>Oh, wait. The production config file<br \/>\nwasn\u2019t updated to use the new database.<\/em> Okay, fix that. Finally, it<br \/>\nworks, and it\u2019s time to go home. Ever been there? How about the late<br \/>\nnight when your provisioning scripts work for updating existing servers,<br \/>\nbut not for creating a brand new environment? Or, a manual deployment<br \/>\nstep missing from a task list? Or, a config file pointing to a resource<br \/>\nfrom another environment? Each of these problems stems from separating<br \/>\nthe activity of provisioning infrastructure from that of deploying<br \/>\nsoftware, whether by choice, or limitation of tools. The impact of<br \/>\ndeploying should be to allow customers to benefit from added value or<br \/>\nvalidate a business hypothesis. In order to accomplish this,<br \/>\ninfrastructure and software are both needed, and they normally change<br \/>\ntogether. Thus, a deployment can be defined as:<\/p>\n<ul>\n<li>reconciling the infrastructure needed with the infrastructure that<br \/>\nalready exists; and<\/li>\n<li>reconciling the software that we want to run with the software that<br \/>\nis already running.<\/li>\n<\/ul>\n<p>With Rancher, Terraform, and Drone, you can build a continuous delivery<br \/>\npipeline that lets you deploy this way. Let\u2019s look at a sample system:<br \/>\n<img decoding=\"async\" src=\"http:\/\/cdn.rancher.com\/wp-content\/uploads\/2017\/07\/28105255\/Sample-System.png\" alt=\"\" \/> This simple<br \/>\narchitecture has a server running two microservices,<br \/>\n[<a href=\"https:\/\/github.com\/PelotonTechIO\/happy-service\">happy-service<\/a>]<br \/>\nand<br \/>\n[<a href=\"http:\/\/lotonTechIO\/glad-service\">glad-service<\/a>].<br \/>\nWhen a deployment is triggered, you want the ecosystem to match this<br \/>\npicture, regardless of what its current state is. Terraform is a tool<br \/>\nthat allows you to predictably create and change infrastructure and<br \/>\nsoftware. You describe individual resources, like servers and Rancher<br \/>\nstacks, and it will create a plan to make the world match the resources<br \/>\nyou describe. Let\u2019s create a Terraform configuration that creates a<br \/>\nRancher environment for our production deployment:<\/p>\n<p>provider &#8220;rancher&#8221; {<br \/>\napi_url = &#8220;$&#8221;<br \/>\n}<\/p>\n<p>resource &#8220;rancher_environment&#8221; &#8220;production&#8221; {<br \/>\nname = &#8220;production&#8221;<br \/>\ndescription = &#8220;Production environment&#8221;<br \/>\norchestration = &#8220;cattle&#8221;<br \/>\n}<\/p>\n<p>resource &#8220;rancher_registration_token&#8221; &#8220;production_token&#8221; {<br \/>\nenvironment_id = &#8220;$&#8221;<br \/>\nname = &#8220;production-token&#8221;<br \/>\ndescription = &#8220;Host registration token for Production environment&#8221;<br \/>\n}<\/p>\n<p>Terraform has the ability to preview what it\u2019ll do before applying<br \/>\nchanges. Let\u2019s run terraform plan.<\/p>\n<p>+ rancher_environment.production<br \/>\ndescription: &#8220;Production environment&#8221;<br \/>\n&#8230;<\/p>\n<p>+ rancher_registration_token.production_token<br \/>\ncommand: &#8220;&lt;computed&gt;&#8221;<br \/>\n&#8230;<\/p>\n<p>The pluses and green text indicate that the resource needs to be<br \/>\ncreated. Terraform knows that these resources haven\u2019t been created yet,<br \/>\nso it will try to create them. Running terraform apply creates the<br \/>\nenvironment in Rancher. You can log into Rancher to see it. Now let\u2019s<br \/>\nadd an AWS EC2 server to the environment:<\/p>\n<p># A look up for rancheros_ami by region<br \/>\nvariable &#8220;rancheros_amis&#8221; {<br \/>\ndefault = {<br \/>\n&#8220;ap-south-1&#8221; = &#8220;ami-3576085a&#8221;<br \/>\n&#8220;eu-west-2&#8221; = &#8220;ami-4806102c&#8221;<br \/>\n&#8220;eu-west-1&#8221; = &#8220;ami-64b2a802&#8221;<br \/>\n&#8220;ap-northeast-2&#8221; = &#8220;ami-9d03dcf3&#8221;<br \/>\n&#8220;ap-northeast-1&#8221; = &#8220;ami-8bb1a7ec&#8221;<br \/>\n&#8220;sa-east-1&#8221; = &#8220;ami-ae1b71c2&#8221;<br \/>\n&#8220;ca-central-1&#8221; = &#8220;ami-4fa7182b&#8221;<br \/>\n&#8220;ap-southeast-1&#8221; = &#8220;ami-4f921c2c&#8221;<br \/>\n&#8220;ap-southeast-2&#8221; = &#8220;ami-d64c5fb5&#8221;<br \/>\n&#8220;eu-central-1&#8221; = &#8220;ami-8c52f4e3&#8221;<br \/>\n&#8220;us-east-1&#8221; = &#8220;ami-067c4a10&#8221;<br \/>\n&#8220;us-east-2&#8221; = &#8220;ami-b74b6ad2&#8221;<br \/>\n&#8220;us-west-1&#8221; = &#8220;ami-04351964&#8221;<br \/>\n&#8220;us-west-2&#8221; = &#8220;ami-bed0c7c7&#8221;<br \/>\n}<br \/>\ntype = &#8220;map&#8221;<br \/>\n}<\/p>\n<p># this creates a cloud-init script that registers the server<br \/>\n# as a rancher agent when it starts up<br \/>\nresource &#8220;template_file&#8221; &#8220;user_data&#8221; {<br \/>\ntemplate = &lt;&lt;EOF<br \/>\n#cloud-config<br \/>\nwrite_files:<br \/>\n&#8211; path: \/etc\/rc.local<br \/>\npermissions: &#8220;0755&#8221;<br \/>\nowner: root<br \/>\ncontent: |<br \/>\n#!\/bin\/bash<br \/>\nfor i in<br \/>\ndo<br \/>\ndocker info &amp;&amp; break<br \/>\nsleep 1<br \/>\ndone<br \/>\nsudo docker run -d &#8211;privileged -v \/var\/run\/docker.sock:\/var\/run\/docker.sock -v \/var\/lib\/rancher:\/var\/lib\/rancher rancher\/agent:v1.2.1 $$<br \/>\nEOF<\/p>\n<p>vars {<br \/>\nregistration_url = &#8220;$&#8221;<br \/>\n}<br \/>\n}<\/p>\n<p># AWS ec2 launch configuration for a production rancher agent<br \/>\nresource &#8220;aws_launch_configuration&#8221; &#8220;launch_configuration&#8221; {<br \/>\nprovider = &#8220;aws&#8221;<br \/>\nname = &#8220;rancher agent&#8221;<br \/>\nimage_id = &#8220;$&#8221;<br \/>\ninstance_type = &#8220;t2.micro&#8221;<br \/>\nkey_name = &#8220;$&#8221;<br \/>\nuser_data = &#8220;$&#8221;<\/p>\n<p>security_groups = [ &#8220;$&#8221;]<br \/>\nassociate_public_ip_address = true<br \/>\n}<\/p>\n<p># Creates an autoscaling group of 1 server that will be a rancher agent<br \/>\nresource &#8220;aws_autoscaling_group&#8221; &#8220;autoscaling&#8221; {<br \/>\navailability_zones = [&#8220;$&#8221;]<br \/>\nname = &#8220;Production servers&#8221;<br \/>\nmax_size = &#8220;1&#8221;<br \/>\nmin_size = &#8220;1&#8221;<br \/>\nhealth_check_grace_period = 3600<br \/>\nhealth_check_type = &#8220;ELB&#8221;<br \/>\ndesired_capacity = &#8220;1&#8221;<br \/>\nforce_delete = true<br \/>\nlaunch_configuration = &#8220;$&#8221;<br \/>\nvpc_zone_identifier = [&#8220;$&#8221;]<br \/>\n}<\/p>\n<p>We\u2019ll put these in the same directory as environment.tf, and run<br \/>\nterraform plan again:<\/p>\n<p>+ aws_autoscaling_group.autoscaling<br \/>\narn: &#8220;&#8221;<br \/>\n&#8230;<\/p>\n<p>+ aws_launch_configuration.launch_configuration<br \/>\nassociate_public_ip_address: &#8220;true&#8221;<br \/>\n&#8230;<\/p>\n<p>+ template_file.user_data<br \/>\n&#8230;<\/p>\n<p>This time, you\u2019ll see that rancher_environment resources is missing.<br \/>\nThat\u2019s because it\u2019s already created, and Rancher knows that it<br \/>\ndoesn\u2019t have to create it again. Run terraform apply, and after a few<br \/>\nminutes, you should see a server show up in Rancher. Finally, we want to<br \/>\ndeploy the happy-service and glad-service onto this server:<\/p>\n<p>resource &#8220;rancher_stack&#8221; &#8220;happy&#8221; {<br \/>\nname = &#8220;happy&#8221;<br \/>\ndescription = &#8220;A service that&#8217;s always happy&#8221;<br \/>\nstart_on_create = true<br \/>\nenvironment_id = &#8220;$&#8221;<\/p>\n<p>docker_compose = &lt;&lt;EOF<br \/>\nversion: &#8216;2&#8217;<br \/>\nservices:<br \/>\nhappy:<br \/>\nimage: peloton\/happy-service<br \/>\nstdin_open: true<br \/>\ntty: true<br \/>\nports:<br \/>\n&#8211; 8000:80\/tcp<br \/>\nlabels:<br \/>\nio.rancher.container.pull_image: always<br \/>\nio.rancher.scheduler.global: &#8216;true&#8217;<br \/>\nstarted: $STARTED<br \/>\nEOF<\/p>\n<p>rancher_compose = &lt;&lt;EOF<br \/>\nversion: &#8216;2&#8217;<br \/>\nservices:<br \/>\nhappy:<br \/>\nstart_on_create: true<br \/>\nEOF<\/p>\n<p>finish_upgrade = true<br \/>\nenvironment {<br \/>\nSTARTED = &#8220;$&#8221;<br \/>\n}<br \/>\n}<\/p>\n<p>resource &#8220;rancher_stack&#8221; &#8220;glad&#8221; {<br \/>\nname = &#8220;glad&#8221;<br \/>\ndescription = &#8220;A service that&#8217;s always glad&#8221;<br \/>\nstart_on_create = true<br \/>\nenvironment_id = &#8220;$&#8221;<\/p>\n<p>docker_compose = &lt;&lt;EOF<br \/>\nversion: &#8216;2&#8217;<br \/>\nservices:<br \/>\nglad:<br \/>\nimage: peloton\/glad-service<br \/>\nstdin_open: true<br \/>\ntty: true<br \/>\nports:<br \/>\n&#8211; 8000:80\/tcp<br \/>\nlabels:<br \/>\nio.rancher.container.pull_image: always<br \/>\nio.rancher.scheduler.global: &#8216;true&#8217;<br \/>\nstarted: $STARTED<br \/>\nEOF<\/p>\n<p>rancher_compose = &lt;&lt;EOF<br \/>\nversion: &#8216;2&#8217;<br \/>\nservices:<br \/>\nglad:<br \/>\nstart_on_create: true<br \/>\nEOF<\/p>\n<p>finish_upgrade = true<br \/>\nenvironment {<br \/>\nSTARTED = &#8220;$&#8221;<br \/>\n}<br \/>\n}<\/p>\n<p>This will create two new Rancher stacks; one for the happy service and<br \/>\none for the glad service. Running terraform plan once more will show<br \/>\nthe two Rancher stacks:<\/p>\n<p>+ rancher_stack.glad<br \/>\ndescription: &#8220;A service that&#8217;s always glad&#8221;<br \/>\n&#8230;<\/p>\n<p>+ rancher_stack.happy<br \/>\ndescription: &#8220;A service that&#8217;s always happy&#8221;<br \/>\n&#8230;<\/p>\n<p>And running terraform apply will create them. Once this is done,<br \/>\nyou\u2019ll have your two microservices deployed onto a host automatically<br \/>\non Rancher. You can hit your host on port 8000 or on port 8001 to see<br \/>\nthe response from the services:<br \/>\n<img decoding=\"async\" src=\"http:\/\/cdn.rancher.com\/wp-content\/uploads\/2017\/07\/28113145\/glad-happy-1024x872.png\" alt=\"\" \/> We\u2019ve created each<br \/>\npiece of the infrastructure along the way in a piecemeal fashion. But<br \/>\nTerraform can easily do everything from scratch, too. Try issuing a<br \/>\nterraform destroy, followed by terraform apply, and the entire<br \/>\nsystem will be recreated. This is what makes deploying with Terraform<br \/>\nand Rancher so powerful &#8211; Terraform will reconcile the desired<br \/>\ninfrastructure with the existing infrastructure, whether those resources<br \/>\nexist, don\u2019t exist, or require modification. Using Terraform and<br \/>\nRancher, you can now create the infrastructure and the software that<br \/>\nruns on the infrastructure together. They can be changed and versioned<br \/>\ntogether, too. In the future blog entries, we\u2019ll look at how to<br \/>\nautomate this process on git push with Drone. Be sure to check out the<br \/>\ncode for the Terraform configuration are hosted on<br \/>\n[<a href=\"https:\/\/github.com\/PelotonTechIO\/example-shared-infrastructure\">github<\/a>].<br \/>\nThe<br \/>\n<a href=\"https:\/\/github.com\/PelotonTechIO\/happy-service\">[happy-service]<\/a><br \/>\nand<br \/>\n[<a href=\"https:\/\/github.com\/PelotonTechIO\/glad-service\">glad-service<\/a>]<br \/>\nare simple nginx docker containers. <em>Bryce Covert is an engineer at<br \/>\n<a href=\"https:\/\/www.pelo.tech\">pelotech<\/a>. By day, he helps teams accelerate<br \/>\nengineering by teaching them functional programming, stateless<br \/>\nmicroservices, and immutable infrastructure. By night, he hacks away,<br \/>\ncreating point and click adventure games. You can find pelotech on<br \/>\nTwitter at <a href=\"https:\/\/twitter.com\/pelotechnology\">@pelotechnology<\/a>.<\/em><\/p>\n<p><a href=\"https:\/\/rancher.com\/continuous-delivery-of-everything\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>It\u2019s 8:00 PM. I just deployed to production, but nothing\u2019s working. Oh, wait. the production Kinesis stream doesn\u2019t exist, because the CloudFormation template for production wasn\u2019t updated. Okay, fix that. 9:00 PM. Redeploy. Still broken. Oh, wait. The production config file wasn\u2019t updated to use the new database. Okay, fix that. Finally, it works, and &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.appservgrid.com\/paw93\/index.php\/2018\/12\/03\/continuous-delivery-of-everything-with-rancher-drone-and-terraform\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Continuous Delivery of Everything with Rancher, Drone, and Terraform&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-814","post","type-post","status-publish","format-standard","hentry","category-kubernetes"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/814","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/comments?post=814"}],"version-history":[{"count":1,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/814\/revisions"}],"predecessor-version":[{"id":855,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/814\/revisions\/855"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/media?parent=814"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/categories?post=814"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/tags?post=814"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}