{"id":1043,"date":"2019-01-10T01:44:54","date_gmt":"2019-01-10T01:44:54","guid":{"rendered":"https:\/\/www.appservgrid.com\/paw93\/?p=1043"},"modified":"2019-01-12T10:25:37","modified_gmt":"2019-01-12T10:25:37","slug":"hidden-dependencies-with-microservices-rancher-labs","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw93\/index.php\/2019\/01\/10\/hidden-dependencies-with-microservices-rancher-labs\/","title":{"rendered":"Hidden Dependencies with Microservices | Rancher Labs"},"content":{"rendered":"<div class=\"wrapper p-t-md\">\n<div class=\"row center-xs\">\n<div class=\"col-md-8 col-xs-12\">\n<h1 class=\"m-t-0\">Hidden Dependencies with Microservices<\/h1>\n<div class=\"row m-0\"><a class=\"tag\" href=\"https:\/\/rancher.com\/tags\/cattle\">CATTLE<\/a><a class=\"tag\" href=\"https:\/\/rancher.com\/tags\/operations\">OPERATIONS<\/a><a class=\"tag\" href=\"https:\/\/rancher.com\/tags\/orchestration\">ORCHESTRATION<\/a><\/div>\n<\/div>\n<\/div>\n<hr \/>\n<\/div>\n<div class=\"wrapper\">\n<article class=\"main-content row center-xs\">\n<div class=\"col-md-8 col-xs-12\">\n<p>One of the great things about microservices is that they allow engineering to decouple software development from application lifecycle. Every microservice:<\/p>\n<ul>\n<li>can be written in its own language, be it Go, Java, or Python<\/li>\n<li>can be contained and isolated form others<\/li>\n<li>can be scaled horizontally across additional nodes and instances<\/li>\n<li>is owned by a single team, rather than being a shared responsibility among many teams<\/li>\n<li>communicates with other microservices through an API a message bus<\/li>\n<li>must support a common service level agreement to be consumed by other microservices, and conversely, to consume other microservices<\/li>\n<\/ul>\n<p>These are all very cool features, and most of them help to decouple various software dependencies from each other. But what is the operations point of view? While the cool aspects of microservices bulleted above are great for development teams, they pose some new challenges for DevOps teams. Namely:<\/p>\n<p><strong>Scalability:<\/strong>\u00a0Traditionally, monolithic applications and systems scaled vertically, with low dynamism in mind. Now, we need horizontal architectures to support massive dynamism \u2013 we need infrastructure as code (IaC). If our application is not a monolith, then our infrastructure cannot be, either.<\/p>\n<p><strong>Orchestration:<\/strong>\u00a0Containers are incredibly dynamic, but they need resources \u2013 CPU, memory, storage (SDS) and networking (SDN) when executed. Operations and DevOps teams need a new piece of software that knows which resources are available to run tasks fairly (if sharing resources with other teams), and efficiently.<\/p>\n<p><strong>System Discovery:<\/strong>\u00a0To merge dynamism and orchestration, you need a system discovery service. With microservices and containers, one can implement still use a configuration management database (CMDB), but with massive dynamism in mind. A good system has to be aware of every container deployment change, able to get or receive information from every container (metadata, labels), and provide a method for making this info available.<\/p>\n<p>There are many tools in the ecosystem one can choose. However, the scope of this article is not to do a deep dive into these tools, but to provide an overview of how to reduce dependencies between microservices and your tooling.<\/p>\n<h3 id=\"scenario-1-rancher-cattle\">Scenario 1: Rancher + Cattle<\/h3>\n<p>Consider the following scenario, where a team is using\u00a0<a href=\"http:\/\/rancher.com\/rancher\">Rancher<\/a>. Rancher is facilitating infrastructure as code, and using Cattle for orchestration, and Rancher discovery (metadata, DNS, and API) for managing system discovery. Assume that the DevOps team is familiar with this stack, but must begin building functionality for the application to run. Let\u2019s look at the dependency points they\u2019ll need to consider:<\/p>\n<ol>\n<li><strong>The IaC tool shouldn\u2019t affect the development or deployment of microservices.<\/strong>\u00a0This layer is responsible for providing, booting, and enabling communication between servers (VMs or bare metal). Microservices need servers to run, but it doesn\u2019t matter how those servers were provided. We should be able to change our IaC method without affecting microservices development or deployment paths.<\/li>\n<li><strong>Microservice deployments are dependent on orchestration.<\/strong>\u00a0The development path for microservices could be the same, but the deployment path is tightly coupled to the orchestration service, due to deployment syntax and format. There\u2019s no easy way to avoid this dependency, but it can be minimized by using different orchestration templates for specific microservice deployments.<\/li>\n<li><strong>Microservice developments could be dependent on system discovery.<\/strong>\u00a0It depends on the development path.<\/li>\n<\/ol>\n<p>Points (1) and (2) are relatively clear, but let\u2019s take a closer look at (3). Due to the massive dynamism in microservices architectures, when one microservice is deployed, it must be able to retrieve its configuration easily. It also needs to know where its peers and collaborating microservices are, and how they communicate (which IP, and via which port, etc). The natural conclusion is that for each microservice, you also define logic coupling it with service discovery. But what happens if you decide to use another orchestrator or tool for system discovery? Consider a second system scenario:<\/p>\n<h3 id=\"scenario-2-rancher-kubernetes-etcd\">Scenario 2: Rancher + Kubernetes + etcd<\/h3>\n<p>In the second scenario, the team is still using Rancher to facilitate Infrastructure as Code. However, this team has instead decided to use\u00a0<a href=\"http:\/\/kubernetes.io\/\">Kubernetes<\/a>\u00a0for orchestration and system discovery (using\u00a0<a href=\"https:\/\/coreos.com\/etcd\/\">etcd<\/a>). The team would have to create Kubernetes deployment files for microservices (Point 2, above), and refactor all the microservices to talk with Kubernetes instead of Rancher metadata (Point 3). The solution is to decouple services from configuration. This is easier said than done, but here is one possible way to do it:<\/p>\n<ul>\n<li>Define a container hierarchy<\/li>\n<li>Separate containers into two categories: executors and configurators<\/li>\n<li>Create generic images based on functionality<\/li>\n<li>Create application images from the generic executor images; similarly, create config-apps images from the generic configurator images<\/li>\n<li>Define logic for running multiple images in a collaborative\/accumulative mode<\/li>\n<\/ul>\n<p>Here\u2019s an example with specific products to clarify these steps:\u00a0<a href=\"http:\/\/cdn.rancher.com\/wp-content\/uploads\/2016\/12\/14162533\/Container-Hierarchies.jpg\"><img decoding=\"async\" src=\"http:\/\/cdn.rancher.com\/wp-content\/uploads\/2016\/12\/14162533\/Container-Hierarchies-1024x576.jpg\" alt=\"container-hierarchies\" \/><\/a>In the image above, we\u2019ve used:<\/p>\n<ul>\n<li><strong><a href=\"https:\/\/github.com\/rawmind0\/alpine-base\">base: alpine<\/a><\/strong>\u00a0Built from\u00a0<a href=\"https:\/\/alpinelinux.org\/\">Alpine Linux<\/a>, with some extra packages: OpenSSL, curl, and bash. This has the smallest OS footprint with package manager, based in\u00a0<a href=\"https:\/\/uclibc.org\/\">uClib<\/a>, and it\u2019s ideal for containerized microservices that don\u2019t need\u00a0<a href=\"https:\/\/www.gnu.org\/software\/libc\/\">glibc<\/a>.<\/li>\n<li><strong><a href=\"https:\/\/github.com\/rawmind0\/alpine-monit\">executor: monit<\/a>.<\/strong>\u00a0Built from the base above, with monitoring installed under \/opt\/monit. It\u2019s written in C with static dependencies, a small 2 MB footprint, and cool features.<\/li>\n<li><strong><a href=\"https:\/\/github.com\/rawmind0\/alpine-tools\">configurator: confd<\/a>.<\/strong>\u00a0Built from the base above, with confd and useful system tools under \/opt\/tools. It\u2019s written in Golang with static dependencies, and provides an indirect path for system discovery, due to supporting different backends like Rancher, etcd, and\u00a0<a href=\"https:\/\/www.consul.io\/\">Consul<\/a>.<\/li>\n<\/ul>\n<p>The main idea is to keep microservices development decoupled from system discovery; this way, they can run on their own, or complemented by another tool that provides dynamic configuration. If you\u2019d like to test out another tool for system discovery (such as etcd, ZooKeeper, or Consul), then all you\u2019ll have to do is develop a new branch for the configurator tool. You won\u2019t need to develop another version of the microservice. By avoiding reconfiguring the microservice itself, you\u2019ll be able to reuse more code, collaborate more easily, and have a more dynamic system. You\u2019ll also get more control, quality, and speed during both development and deployment. To learn more about hierarchy, images, and build I\u2019ve used here, you can access\u00a0<strong><a href=\"https:\/\/github.com\/rawmind0\/docker-alpine\">this repo<\/a><\/strong>on GitHub. Within are additional examples using Kafka and Zookeeper packages; these service containers can run alone (for dev\/test use cases), or with Rancher and Kubernetes for dynamic configuration with Cattle or Kubernetes\/etcd.\u00a0<strong>Zookeeper (repo\u00a0<a href=\"https:\/\/github.com\/rawmind0\/alpine-zk\">here<\/a>,<\/strong>\u00a067 MB)<\/p>\n<ul>\n<li>With Cattle (<a href=\"https:\/\/github.com\/rawmind0\/alpine-zk\/tree\/master\/rancher\">here<\/a>)<\/li>\n<li>With Kubernetes (<a href=\"https:\/\/github.com\/rawmind0\/alpine-zk\/tree\/master\/k8s\">here<\/a>)<\/li>\n<\/ul>\n<p><strong>Kafka (repo\u00a0<a href=\"https:\/\/github.com\/rawmind0\/alpine-kafka\">here<\/a>,<\/strong>\u00a0115 MB)<\/p>\n<ul>\n<li>Cattle (<a href=\"https:\/\/github.com\/rawmind0\/alpine-kafka\/tree\/master\/rancher\">here<\/a>)<\/li>\n<li>Kubernetes (<a href=\"https:\/\/github.com\/rawmind0\/alpine-kafka\/tree\/master\/k8s\">here<\/a>)<\/li>\n<\/ul>\n<h3 id=\"conclusion\">Conclusion<\/h3>\n<p>Orchestration engines and system discovery services can result in \u201chidden\u201d microservice dependencies if not carefully decoupled from microservices themselves. This decoupling makes it easier to develop, test and deploy microservices across infrastructure stacks without refactoring, and in turn allows users to build wider, more open systems with better service agreements.\u00a0<em>Raul Sanchez Liebana (<a href=\"https:\/\/twitter.com\/rawmindNet\">@rawmindNet<\/a>) is a DevOps engineer at Rancher<\/em>.<\/p>\n<\/div>\n<\/article>\n<\/div>\n<p><a href=\"https:\/\/rancher.com\/hidden-dependencies-with-microservices\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hidden Dependencies with Microservices CATTLEOPERATIONSORCHESTRATION One of the great things about microservices is that they allow engineering to decouple software development from application lifecycle. Every microservice: can be written in its own language, be it Go, Java, or Python can be contained and isolated form others can be scaled horizontally across additional nodes and instances &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.appservgrid.com\/paw93\/index.php\/2019\/01\/10\/hidden-dependencies-with-microservices-rancher-labs\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Hidden Dependencies with Microservices | Rancher Labs&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-1043","post","type-post","status-publish","format-standard","hentry","category-kubernetes"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/1043","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/comments?post=1043"}],"version-history":[{"count":1,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/1043\/revisions"}],"predecessor-version":[{"id":1067,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/1043\/revisions\/1067"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/media?parent=1043"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/categories?post=1043"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/tags?post=1043"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}