{"id":311,"date":"2018-10-16T08:52:19","date_gmt":"2018-10-16T08:52:19","guid":{"rendered":"https:\/\/www.appservgrid.com\/paw93\/?p=311"},"modified":"2018-10-16T21:05:20","modified_gmt":"2018-10-16T21:05:20","slug":"propagating-configuration-from-terraform-to-kubernetes-apps","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw93\/index.php\/2018\/10\/16\/propagating-configuration-from-terraform-to-kubernetes-apps\/","title":{"rendered":"Propagating configuration from Terraform to Kubernetes Apps"},"content":{"rendered":"<p>Feb 13, 2018\u00a0 By <a href=\"https:\/\/container-solutions.com\/author\/adam\/\">Adam Sandor<\/a><\/p>\n<p>I recently encountered an interesting problem while terraforming Kubernetes clusters on Google Cloud. Our Terraform configurations have some important information which is needed by applications running on the cluster. These are either values for which our Terraform resources are the primary source, or values which are outputs from them.<\/p>\n<p>Here are some examples of the types of values we want to propagate from Terraform resources to application Pods running on Kubernetes:<\/p>\n<ul>\n<li>Name of the GKE (Google Kubernetes Engine) cluster \u2013 specified by Terraform.<\/li>\n<li>Endpoint of the Cloud SQL database \u2013 generated by Terraform<\/li>\n<li>Static Client IP address \u2013 generated by GCP (Google Cloud Platform)<\/li>\n<\/ul>\n<p>These values are all available while Terraform is running but hard or impossible to access from a Pod running on the Kubernetes cluster. Most surprisingly the cluster name is not available in any kind of metadata on Google cloud but it has to be used when submitting custom container metrics to Google Stackdriver.<\/p>\n<p>To propagate these values conveniently to our application Pods we can use the <a href=\"https:\/\/www.terraform.io\/docs\/providers\/kubernetes\/index.html\">Terraform Kubernetes provider<\/a> to create <b>ConfigMaps<\/b> or <b>Secrets<\/b> (depending on the sensitivity of the information) in Kubernetes. Applications running on the cluster can then have the values injected as environment variables or files.<\/p>\n<p>The Kubernetes provider is quite new and in a general sense it\u2019s questionable whether you should be provisioning Kubernetes objects using Terraform. You can read more on this from Harshal Shah in this post: <a href=\"http:\/\/blog.infracloud.io\/terraform-helm-kubernetes\/\">http:\/\/blog.infracloud.io\/terraform-helm-kubernetes\/<\/a>. For our use case however using the Kubernetes provider is perfect.<\/p>\n<p>I was worried about the fact that we have to provision the Kubernetes cluster itself (using the Google Cloud provider <a href=\"https:\/\/www.terraform.io\/docs\/providers\/google\/r\/container_cluster.html\">container cluster resource<\/a>), and use the credentials this resource outputs to set up the Kubernetes provider. This kind of inter-provider dependency is somewhat undefined in Terraform but it turns out to work perfectly.<\/p>\n<p><a href=\"http:\/\/container-solutions.com\/content\/uploads\/2018\/02\/Propagating-data-from-Terraform-to-a-Kubernetes-application.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/container-solutions.com\/content\/uploads\/2018\/02\/Propagating-data-from-Terraform-to-a-Kubernetes-application-e1518174969535.png\" alt=\"Propagating data from Terraform to a Kubernetes application\" width=\"702\" height=\"440\" \/><\/a><\/p>\n<p>Here are snippets from the code used to do all this.<\/p>\n<p>Google Cloud provider setup and the Kubernetes cluster resource definition:<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<table>\n<tbody>\n<tr>\n<td>&nbsp;<\/p>\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>&nbsp;<\/td>\n<td>provider &#8220;google&#8221; {<\/p>\n<p>region = &#8220;europe-west3&#8221;<\/p>\n<p>credentials = &#8220;itsasecret&#8221;<\/p>\n<p>project = &#8220;adamsproject&#8221;<\/p>\n<p>}<\/p>\n<p>resource &#8220;google_container_cluster&#8221; &#8220;mycluster&#8221; {<\/p>\n<p>project = &#8220;adamsproject&#8221;<\/p>\n<p>name = &#8220;mycluster&#8221;<\/p>\n<p>zone = &#8220;europe-west3-a&#8221;<\/p>\n<p>initial_node_count = 1<\/p>\n<p>node_version = &#8220;$&#8221;<\/p>\n<p>min_master_version = &#8220;$&#8221;<\/p>\n<p>node_config {<\/p>\n<p>machine_type = &#8220;g1-small&#8221;<\/p>\n<p>disk_size_gb = 50<\/p>\n<p>}<\/p>\n<p>}<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p>The Kubernetes provider is configured using outputs from the google_container_cluster resource \u2013 namely the CA certificate, the client certificate and the client secret key. We have to base64 decode these.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<table>\n<tbody>\n<tr>\n<td><\/td>\n<td>provider &#8220;kubernetes&#8221; {<\/p>\n<p>host = &#8220;$&#8221;<\/p>\n<p>client_certificate = &#8220;$&#8221;<\/p>\n<p>client_key = &#8220;$&#8221;<\/p>\n<p>cluster_ca_certificate = &#8220;$&#8221;<\/p>\n<p>}<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p>Once the Kubernetes provider can access our new cluster, this resource definition will create a Secret with our database secrets:<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<table>\n<tbody>\n<tr>\n<td><\/td>\n<td>resource &#8220;kubernetes_secret&#8221; &#8220;cloudsql-db-credentials&#8221; {<\/p>\n<p>&#8220;metadata&#8221; {<\/p>\n<p>name = &#8220;cloudsql-db-credentials&#8221;<\/p>\n<p>}<\/p>\n<p>data {<\/p>\n<p>username = &#8220;$&#8221;<\/p>\n<p>password = &#8220;$&#8221;<\/p>\n<p>}<\/p>\n<p>}<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p>We can also create some ConfigMaps to hold other, less sensitive information. Here I\u2019m creating a database connection string to be used by Google CloudSQL Proxy:<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<table>\n<tbody>\n<tr>\n<td><\/td>\n<td>resource &#8220;kubernetes_config_map&#8221; &#8220;dbconfig&#8221; {<\/p>\n<p>&#8220;metadata&#8221; {<\/p>\n<p>name = &#8220;dbconfig&#8221;<\/p>\n<p>}<\/p>\n<p>data = {<\/p>\n<p>dbconnection = &#8220;$:$:$&#8221;<\/p>\n<p>}<\/p>\n<p>}<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p>I\u2019m not going into the details of how to get this configuration all the way to your application components as that is straightforward once you have ConfigMaps or Secrets created. Take a look at the Kubernetes documentation if you are not familiar with working with <a href=\"https:\/\/kubernetes.io\/docs\/tasks\/configure-pod-container\/configure-pod-configmap\/\">ConfigMaps<\/a> or <a href=\"https:\/\/kubernetes.io\/docs\/tasks\/inject-data-application\/distribute-credentials-secure\/\">Secrets<\/a>.<\/p>\n<h3>Summary<\/h3>\n<p>Terraform will inevitably become your primary source of truth for many variables in your environment. It will also know about variables that are set by the cloud provider during resource creation, like a random IP address. It\u2019s a very elegant way to use Kubernetes Secrets and ConfigMaps to pass these values to your application components which need them, while the Kubernetes provider in Terraform offers a perfect way to create these objects.<\/p>\n<p><a href=\"https:\/\/container-solutions.com\/propagating-configuration-from-terraform-to-kubernetes-apps\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Feb 13, 2018\u00a0 By Adam Sandor I recently encountered an interesting problem while terraforming Kubernetes clusters on Google Cloud. Our Terraform configurations have some important information which is needed by applications running on the cluster. These are either values for which our Terraform resources are the primary source, or values which are outputs from them. &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.appservgrid.com\/paw93\/index.php\/2018\/10\/16\/propagating-configuration-from-terraform-to-kubernetes-apps\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Propagating configuration from Terraform to Kubernetes Apps&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-311","post","type-post","status-publish","format-standard","hentry","category-kubernetes"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/311","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/comments?post=311"}],"version-history":[{"count":2,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/311\/revisions"}],"predecessor-version":[{"id":419,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/311\/revisions\/419"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/media?parent=311"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/categories?post=311"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/tags?post=311"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}