{"id":348,"date":"2018-10-16T10:36:38","date_gmt":"2018-10-16T10:36:38","guid":{"rendered":"https:\/\/www.appservgrid.com\/paw93\/?p=348"},"modified":"2018-10-17T08:21:06","modified_gmt":"2018-10-17T08:21:06","slug":"health-checking-grpc-servers-on-kubernetes","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw93\/index.php\/2018\/10\/16\/health-checking-grpc-servers-on-kubernetes\/","title":{"rendered":"Health checking gRPC servers on Kubernetes"},"content":{"rendered":"<p>Author: <a href=\"https:\/\/twitter.com\/ahmetb\" target=\"_blank\" rel=\"noopener\">Ahmet Alp Balkan<\/a> (Google)<\/p>\n<p><a href=\"https:\/\/grpc.io\" target=\"_blank\" rel=\"noopener\">gRPC<\/a> is on its way to becoming the lingua franca for<br \/>\ncommunication between cloud-native microservices. If you are deploying gRPC<br \/>\napplications to Kubernetes today, you may be wondering about the best way to<br \/>\nconfigure health checks. In this article, we will talk about<br \/>\n<a href=\"https:\/\/github.com\/grpc-ecosystem\/grpc-health-probe\/\" target=\"_blank\" rel=\"noopener\">grpc-health-probe<\/a>, a<br \/>\nKubernetes-native way to health check gRPC apps.<\/p>\n<p>If you\u2019re unfamiliar, Kubernetes <a href=\"\/docs\/tasks\/configure-pod-container\/configure-liveness-readiness-probes\/\">health<br \/>\nchecks<\/a><br \/>\n(liveness and readiness probes) is what\u2019s keeping your applications available<br \/>\nwhile you\u2019re sleeping. They detect unresponsive pods, mark them unhealthy, and<br \/>\ncause these pods to be restarted or rescheduled.<\/p>\n<p>Kubernetes <a href=\"https:\/\/github.com\/kubernetes\/kubernetes\/issues\/21493\" target=\"_blank\" rel=\"noopener\">does not<br \/>\nsupport<\/a> gRPC health<br \/>\nchecks natively. This leaves the gRPC developers with the following three<br \/>\napproaches when they deploy to Kubernetes:<\/p>\n<p><a href=\"\/images\/blog\/2019-09-30-health-checking-grpc\/options.png\"><img decoding=\"async\" src=\"https:\/\/d33wubrfki0l68.cloudfront.net\/0ba6676a7835bffc535bfeca611b6301f5f8b07a\/8a475\/images\/blog\/2019-09-30-health-checking-grpc\/options.png\" alt=\"options for health checking grpc on kubernetes today\" \/><\/a><\/p>\n<ol>\n<li>httpGet probe: Cannot be natively used with gRPC. You need to refactor<br \/>\nyour app to serve both gRPC and HTTP\/1.1 protocols (on different port<br \/>\nnumbers).<\/li>\n<li>tcpSocket probe: Opening a socket to gRPC server is not meaningful,<br \/>\nsince it cannot read the response body.<\/li>\n<li>exec probe: This invokes a program in a container\u2019s ecosystem<br \/>\nperiodically. In the case of gRPC, this means you implement a health RPC<br \/>\nyourself, then write and ship a client tool with your container.<\/li>\n<\/ol>\n<p>Can we do better? Absolutely.<\/p>\n<h2>Introducing \u201cgrpc-health-probe\u201d<\/h2>\n<p>To standardize the \u201cexec probe\u201d approach mentioned above, we need:<\/p>\n<ul>\n<li>a standard health check \u201cprotocol\u201d that can be implemented in any gRPC<br \/>\nserver easily.<\/li>\n<li>a standard health check \u201ctool\u201d that can query the health protocol easily.<\/li>\n<\/ul>\n<p>Thankfully, gRPC has a <a href=\"https:\/\/github.com\/grpc\/grpc\/blob\/v1.15.0\/doc\/health-checking.md\" target=\"_blank\" rel=\"noopener\">standard health checking<br \/>\nprotocol<\/a>. It<br \/>\ncan be used easily from any language. Generated code and the utilities for<br \/>\nsetting the health status are shipped in nearly all language implementations of<br \/>\ngRPC.<\/p>\n<p>If you<br \/>\n<a href=\"https:\/\/github.com\/grpc\/grpc\/blob\/v1.15.0\/src\/proto\/grpc\/health\/v1\/health.proto\" target=\"_blank\" rel=\"noopener\">implement<\/a><br \/>\nthis health check protocol in your gRPC apps, you can then use a standard\/common<br \/>\ntool to invoke this Check() method to determine server status.<\/p>\n<p>The next thing you need is the \u201cstandard tool\u201d, and it\u2019s the<br \/>\n<a href=\"https:\/\/github.com\/grpc-ecosystem\/grpc-health-probe\/\" target=\"_blank\" rel=\"noopener\">grpc-health-probe<\/a>.<\/p>\n<p><a href=\"\/images\/blog\/2019-09-30-health-checking-grpc\/grpc_health_probe.png\"><br \/>\n<img decoding=\"async\" src=\"https:\/\/d33wubrfki0l68.cloudfront.net\/69ff06cd73000bd1ed01b146a7ec52db326ce9b9\/1a564\/images\/blog\/2019-09-30-health-checking-grpc\/grpc_health_probe.png\" width=\"768\" \/><\/a><\/p>\n<p>With this tool, you can use the same health check configuration in all your gRPC<br \/>\napplications. This approach requires you to:<\/p>\n<ol>\n<li>Find the gRPC \u201chealth\u201d module in your favorite language and start using it<br \/>\n(example <a href=\"https:\/\/godoc.org\/github.com\/grpc\/grpc-go\/health\" target=\"_blank\" rel=\"noopener\">Go library<\/a>).<\/li>\n<li>Ship the<br \/>\n<a href=\"https:\/\/github.com\/grpc-ecosystem\/grpc-health-probe\/\" target=\"_blank\" rel=\"noopener\">grpc_health_probe<\/a><br \/>\nbinary in your container.<\/li>\n<li><a href=\"https:\/\/github.com\/grpc-ecosystem\/grpc-health-probe\/tree\/1329d682b4232c102600b5e7886df8ffdcaf9e26#example-grpc-health-checking-on-kubernetes\" target=\"_blank\" rel=\"noopener\">Configure<\/a><br \/>\nKubernetes \u201cexec\u201d probe to invoke the \u201cgrpc_health_probe\u201d tool in the<br \/>\ncontainer.<\/li>\n<\/ol>\n<p>In this case, executing \u201cgrpc_health_probe\u201d will call your gRPC server over<br \/>\nlocalhost, since they are in the same pod.<\/p>\n<h2>What\u2019s next<\/h2>\n<p>grpc-health-probe project is still in its early days and it needs your<br \/>\nfeedback. It supports a variety of features like communicating with TLS servers<br \/>\nand configurable connection\/RPC timeouts.<\/p>\n<p>If you are running a gRPC server on Kubernetes today, try using the gRPC Health<br \/>\nProtocol and try the grpc-health-probe in your deployments, and <a href=\"https:\/\/github.com\/grpc-ecosystem\/grpc-health-probe\/\" target=\"_blank\" rel=\"noopener\">give<br \/>\nfeedback<\/a>.<\/p>\n<h2>Further reading<\/h2>\n<ul>\n<li>Protocol: <a href=\"https:\/\/github.com\/grpc\/grpc\/blob\/v1.15.0\/doc\/health-checking.md\" target=\"_blank\" rel=\"noopener\">GRPC Health Checking Protocol<\/a> (<a href=\"https:\/\/github.com\/grpc\/grpc\/blob\/v1.15.0\/src\/proto\/grpc\/health\/v1\/health.proto\" target=\"_blank\" rel=\"noopener\">health.proto<\/a>)<\/li>\n<li>Documentation: <a href=\"https:\/\/kubernetes.io\/docs\/tasks\/configure-pod-container\/configure-liveness-readiness-probes\/\" target=\"_blank\" rel=\"noopener\">Kubernetes liveness and readiness probes<\/a><\/li>\n<li>Article: <a href=\"https:\/\/ahmet.im\/blog\/advanced-kubernetes-health-checks\/\" target=\"_blank\" rel=\"noopener\">Advanced Kubernetes Health Check Patterns<\/a><\/li>\n<\/ul>\n<p><a href=\"https:\/\/kubernetes.io\/blog\/2018\/10\/01\/health-checking-grpc-servers-on-kubernetes\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Author: Ahmet Alp Balkan (Google) gRPC is on its way to becoming the lingua franca for communication between cloud-native microservices. If you are deploying gRPC applications to Kubernetes today, you may be wondering about the best way to configure health checks. In this article, we will talk about grpc-health-probe, a Kubernetes-native way to health check &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.appservgrid.com\/paw93\/index.php\/2018\/10\/16\/health-checking-grpc-servers-on-kubernetes\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Health checking gRPC servers on Kubernetes&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-348","post","type-post","status-publish","format-standard","hentry","category-kubernetes"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/348","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/comments?post=348"}],"version-history":[{"count":1,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/348\/revisions"}],"predecessor-version":[{"id":489,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/348\/revisions\/489"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/media?parent=348"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/categories?post=348"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/tags?post=348"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}