{"id":3536,"date":"2018-11-17T11:36:05","date_gmt":"2018-11-17T11:36:05","guid":{"rendered":"https:\/\/www.appservgrid.com\/paw92\/?p=3536"},"modified":"2018-11-17T15:54:19","modified_gmt":"2018-11-17T15:54:19","slug":"bisected-the-unfortunate-reason-linux-4-20-is-running-slower","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw92\/index.php\/2018\/11\/17\/bisected-the-unfortunate-reason-linux-4-20-is-running-slower\/","title":{"rendered":"Bisected: The Unfortunate Reason Linux 4.20 Is Running Slower"},"content":{"rendered":"<p>After running a lot of tests and then bisecting the <a href=\"https:\/\/www.phoronix.com\/scan.php?page=search&amp;q=Linux+4.20\">Linux 4.20<\/a> kernel merge window, the reason for the significant slowdowns in the Linux 4.20 kernel for many real-world workloads is now known&#8230;<\/p>\n<p>This latest Linux 4.20 testing endeavor started out with seeing <a href=\"https:\/\/www.phoronix.com\/scan.php?page=news_item&amp;px=Linux-4.20-Initial-Slowdowns\">the Intel Core i9 performance pulling back<\/a> in many synthetic and real-world tests. This ranged from Rodinia scientific OpenMP tests taking 30% longer to Java-based DaCapo tests taking up to ~50% more time to complete to code compilation tests taking measurably longer to lower PostgreSQL database server performance to longer Blender3D rendering times. That happened with a Core i9 7960X and Core i9 7980XE test systems while the AMD Threadripper 2990WX performance was unaffected by the Linux 4.20 upgrade.<\/p>\n<p><em>In some cases this Linux 4.20 slowdown is enough where the Threadripper 2990WX is able to pick up extra wins over the Core i9 7980XE.<\/em><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.phoronix.net\/image.php?id=2018&amp;image=linux_420_regress2_med\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>When digging through more of my test system data, a set of systems I have running the latest Linux kernel Git benchmarks every other day also saw <a href=\"https:\/\/www.phoronix.com\/scan.php?page=news_item&amp;px=Linux-4.20-Performance-Pullback\">a significant pullback in performance<\/a> from the early days of the Linux 4.20 merge window up through the very latest kernel code as of today. Those affected systems weren&#8217;t high-end HEDT boxes but included a low-end Core i3 7100 as well as a Xeon E5 v3 and Core i7 systems. AMD systems though still didn&#8217;t appear impacted. Those tests also found workloads like the Smallpt renderer to slowdown significant, PHP performance to take a major dive, and other scientific workloads like HMMer also faced a major setback compared to the current Linux 4.19 stable series.<\/p>\n<p><em>Bisecting the Linux 4.20 kernel slowdown&#8230; The sizable difference during that process.<\/em><\/p>\n<p>With seeing clear performance regressions on a number of systems when running the latest Linux 4.20 code, and especially with being able to reproduce it on high-core-count hardware (thus significantly cutting down the kernel build times), this morning I kicked off the kernel bisecting process to see why this new kernel is causing many workloads to run so much slower than Linux 4.19. With the <a href=\"https:\/\/www.phoronix-test-suite.com\/\">Phoronix Test Suite<\/a> doing the heavy-lifting, the problematic commit was quickly uncovered.<\/p>\n<p>Going into this testing my thinking was perhaps an Intel P-State CPU frequency scaling driver regression as something that has caused some performance regressions in the past or perhaps a scheduler change. There&#8217;s also been a lot of Linux 4.20 changes in general that some unintentional regression must have slipped in there somewhere primarily hurting the Intel Linux performance&#8230; As a reminder, <a href=\"https:\/\/www.phoronix.com\/scan.php?page=news_item&amp;px=2018-Biggie-Kernel\">Linux 4.20 is the biggest kernel release of the year<\/a> in terms of lines of code changed with more than 354 thousand lines of new code added at the end of October when this merge window opened.<\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"\/\/www.phoronix.com\/image-viewer.php?id=linux-420-bisect&amp;image=stibp_420_1_lrg\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" src=\"\/\/www.phoronix.net\/image.php?id=linux-420-bisect&amp;image=stibp_420_1_med\" \/><\/a><\/p>\n<p>As outlined in the <a href=\"https:\/\/www.phoronix.com\/vr.php?view=27048\">Linux 4.20 feature overview<\/a>, there are a lot of exciting changes with this kernel. But why is it slower? More work on f!*#(# Spectre!<\/p>\n<p><a href=\"http:\/\/www.phoronix.com\/vr.php?view=27092\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>After running a lot of tests and then bisecting the Linux 4.20 kernel merge window, the reason for the significant slowdowns in the Linux 4.20 kernel for many real-world workloads is now known&#8230; This latest Linux 4.20 testing endeavor started out with seeing the Intel Core i9 performance pulling back in many synthetic and real-world &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.appservgrid.com\/paw92\/index.php\/2018\/11\/17\/bisected-the-unfortunate-reason-linux-4-20-is-running-slower\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Bisected: The Unfortunate Reason Linux 4.20 Is Running Slower&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3536","post","type-post","status-publish","format-standard","hentry","category-linux"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/3536","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/comments?post=3536"}],"version-history":[{"count":1,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/3536\/revisions"}],"predecessor-version":[{"id":3799,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/3536\/revisions\/3799"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/media?parent=3536"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/categories?post=3536"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/tags?post=3536"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}