{"id":11848,"date":"2019-03-17T14:17:13","date_gmt":"2019-03-17T14:17:13","guid":{"rendered":"http:\/\/www.appservgrid.com\/paw92\/?p=11848"},"modified":"2019-03-17T14:17:13","modified_gmt":"2019-03-17T14:17:13","slug":"perf-a-performance-monitoring-and-analysis-tool-for-linux","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw92\/index.php\/2019\/03\/17\/perf-a-performance-monitoring-and-analysis-tool-for-linux\/","title":{"rendered":"Perf- A Performance Monitoring and Analysis Tool for Linux"},"content":{"rendered":"<p>When we talk of performance in computing, we refer to the relationship between our resources and the tasks that they allows us to complete in a given period of time.<\/p>\n<div id=\"attachment_20647\" class=\"wp-caption aligncenter\">\n<p><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/Perf-Performance-Monitoring-Analysis-Linux-Tool.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-20647\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/Perf-Performance-Monitoring-Analysis-Linux-Tool.png\" alt=\"Perf- A Performance Monitoring and Analysis Tool for Linux\" width=\"720\" height=\"345\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p class=\"wp-caption-text\">Perf- A Performance Monitoring and Analysis Tool for Linux<\/p>\n<\/div>\n<p>In a day of fierceless competition between companies, it is important that we learn how to use what we have at the best of its capacity. The waste of hardware or software resources, or the lack of ability to know how to use them more efficiently, ends up being a loss that we just can\u2019t afford if we want to be at the top of our game.<\/p>\n<p>At the same time, we must be careful to not take our resources to a limit where sustained use will yield irreparable damage.<\/p>\n<p>In this article we will introduce you to a relatively new performance analysis tool and provide tips that you can use to monitor your Linux systems, including hardware and applications. This will help you to ensure that they operate so that you are capable to produce the desired results without wasting resources or your own energy.<\/p>\n<h3>Introducing and installing Perf in Linux<\/h3>\n<p>Among others, Linux provides a performance monitoring and analysis tool called conveniently\u00a0<strong>perf<\/strong>. So what distinguishes\u00a0<strong>perf<\/strong>\u00a0from other well-known tools with which you are already familiar?<\/p>\n<p>The answer is that\u00a0<strong>perf<\/strong>\u00a0provides access to the\u00a0<a href=\"https:\/\/www.tecmint.com\/command-line-tools-to-monitor-linux-performance\/\" target=\"_blank\" rel=\"noopener\">Performance Monitoring Unit<\/a>\u00a0in the CPU, and thus allows us to have a close look at the behavior of the hardware and its associated events.<\/p>\n<p>In addition, it can also\u00a0<a href=\"https:\/\/www.tecmint.com\/linux-performance-monitoring-and-file-system-statistics-reports\/\" target=\"_blank\" rel=\"noopener\">monitor software events, and create reports<\/a>\u00a0out of the data that is collected.<\/p>\n<p>You can install\u00a0<strong>perf<\/strong>\u00a0in RPM-based distributions with:<\/p>\n<pre># yum update &amp;&amp; yum install perf     [<strong>CentOS<\/strong> \/ <strong>RHEL<\/strong> \/ <strong>Fedora<\/strong>]\r\n# dnf update &amp;&amp; dnf install perf     [<strong>Fedora 23+<\/strong> releases]\r\n<\/pre>\n<p>In\u00a0<strong>Debian<\/strong>\u00a0and derivatives:<\/p>\n<pre># sudo aptitude update &amp;&amp; sudo aptitude install linux-tools-$(uname -r) linux-tools-generic\r\n<\/pre>\n<p>If\u00a0<code>uname -r<\/code>\u00a0in the command above returns extra strings besides the actual version (<strong>3.2.0-23-generic<\/strong>\u00a0in my case), you may have to type\u00a0<strong>linux-tools-3.2.0-23<\/strong>\u00a0instead of using the output of\u00a0<strong>uname<\/strong>.<\/p>\n<p>It is also important to note that\u00a0<strong>perf<\/strong>\u00a0yields incomplete results when run in a guest on top of\u00a0<a href=\"https:\/\/www.tecmint.com\/install-virtualbox-on-redhat-centos-fedora\/\" target=\"_blank\" rel=\"noopener\">VirtualBox<\/a>\u00a0or\u00a0<a href=\"https:\/\/www.tecmint.com\/install-vmware-workstation-11-in-linux\/\" target=\"_blank\" rel=\"noopener\">VMWare<\/a>\u00a0as they do not allow access to hardware counters as other virtualization technologies (such as\u00a0<a href=\"https:\/\/www.tecmint.com\/install-and-configure-kvm-in-linux\/\" target=\"_blank\" rel=\"noopener\">KVM<\/a>\u00a0or\u00a0<a href=\"https:\/\/www.tecmint.com\/citrix-xenserver-installation-and-network-configuration-in-linux\/\" target=\"_blank\" rel=\"noopener\">XEN<\/a>) do.<\/p>\n<p>Additionally, keep in mind that some\u00a0<strong>perf<\/strong>\u00a0commands may be restricted to root by default, which can be disabled (until the system is rebooted) by doing:<\/p>\n<pre># echo 0 &gt; \/proc\/sys\/kernel\/perf_event_paranoid\r\n<\/pre>\n<p>If you need to disable\u00a0<strong>paranoid<\/strong>\u00a0mode permanently, update the following setting in\u00a0<strong>\/etc\/sysctl.conf<\/strong>\u00a0file.<\/p>\n<pre>kernel.perf_event_paranoid = 0\r\n<\/pre>\n<h4>Subcommands<\/h4>\n<p>Once you have installed\u00a0<strong>perf<\/strong>, you can refer to its man page for a list of available subcommands (you can think of subcommands as special options that open a specific window into the system). For best and more complete results, use\u00a0<strong>perf<\/strong>\u00a0either as\u00a0<strong>root<\/strong>\u00a0or through\u00a0<strong>sudo<\/strong>.<\/p>\n<h4>Perf list<\/h4>\n<p>perf list (without options) returns all the symbolic event types (long list). If you want to view the list of events available in a specific category, use\u00a0<strong>perf<\/strong>\u00a0list followed by the category name ([<strong>hw|sw|cache|tracepoint|pmu|event_glob<\/strong>]), such as:<\/p>\n<p>Display list of software pre-defined events in Linux:<\/p>\n<pre># perf list sw \r\n<\/pre>\n<div id=\"attachment_20640\" class=\"wp-caption aligncenter\">\n<p><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/List-Software-Pre-defined-Events-in-Linux.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-20640\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/List-Software-Pre-defined-Events-in-Linux.png\" alt=\"List Software Pre-defined Events in Linux\" width=\"558\" height=\"281\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p class=\"wp-caption-text\">List Software Pre-defined Events in Linux<\/p>\n<\/div>\n<h4>Perf stat<\/h4>\n<p><strong>perf<\/strong>\u00a0stat runs a command and\u00a0<a href=\"https:\/\/www.tecmint.com\/install-web-vmstat-in-linux\/\" target=\"_blank\" rel=\"noopener\">collects Linux performance statistics<\/a>\u00a0during the execution of such command. What happens in our system when we run\u00a0<strong>dd<\/strong>?<\/p>\n<pre># perf stat dd if=\/dev\/zero of=test.iso bs=10M count=1\r\n<\/pre>\n<div id=\"attachment_20641\" class=\"wp-caption aligncenter\">\n<p><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/Collects-Performance-Statistics-of-Linux-Command.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-20641\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/Collects-Performance-Statistics-of-Linux-Command.png\" alt=\"Collects Performance Statistics of Linux Command\" width=\"711\" height=\"382\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p class=\"wp-caption-text\">Collects Performance Statistics of Linux Command<\/p>\n<\/div>\n<p>The stats shown above indicate, among other things:<\/p>\n<ol>\n<li>The execution of the\u00a0<strong>dd<\/strong>\u00a0command took\u00a0<strong>21.812281<\/strong>\u00a0milliseconds of CPU. If we divide this number by the \u201cseconds time elapsed\u201d value below (<strong>23.914596<\/strong>\u00a0milliseconds), it yields\u00a0<strong>0.912<\/strong>\u00a0(CPU utilized).<\/li>\n<li>While the command was executed,\u00a0<strong>15<\/strong>\u00a0context-switches (also known as process switches) indicate that the CPUs were switched 15 times from one process (or thread) to another.<\/li>\n<li><strong>2<\/strong>\u00a0CPU migrations is the expected result when in a 2-core CPU the workload is distributed evenly between the number of cores.<br \/>\nDuring that time (<strong>21.812281<\/strong>\u00a0milliseconds), the total number of CPU cycles that were consumed was\u00a0<strong>62,025,623<\/strong>, which divided by\u00a0<strong>0.021812281<\/strong>\u00a0seconds gives\u00a0<strong>2.843<\/strong>\u00a0GHz.<\/li>\n<li>If we divide the number of cycles by the total instructions count we get\u00a0<strong>4.9<\/strong>\u00a0Cycles Per Instruction, which means each instruction took almost 5 CPU cycles to complete (on average). We can blame this (at least in part) on the number of branches and branch-misses (see below), which end up wasting or misusing CPU cycles.<\/li>\n<li>As the command was executed, a total of\u00a0<strong>3,552,630<\/strong>\u00a0branches were encountered. This is the CPU-level representation of decision points and loops in the code. The more branches, the lower the performance. To compensate for this, all modern CPUs attempt to predict the flow the code will take.\u00a0<strong>51,348<\/strong>\u00a0branch-misses indicate the prediction feature was wrong\u00a0<strong>1.45%<\/strong>\u00a0of the time.<\/li>\n<\/ol>\n<p>The same principle applies to gathering stats (or in other words, profiling) while an application is running. Simply launch the desired application and after a reasonable period of time (which is up to you) close it, and\u00a0<strong>perf<\/strong>\u00a0will display the stats in the screen. By analyzing those stats you can identify potential problems.<\/p>\n<h4>Perf top<\/h4>\n<p><strong>perf top<\/strong>\u00a0is similar to\u00a0<a href=\"https:\/\/www.tecmint.com\/12-top-command-examples-in-linux\/\" target=\"_blank\" rel=\"noopener\">top command<\/a>, in that it displays an almost real-time system profile (also known as live analysis).<\/p>\n<p>With the\u00a0<code>-a<\/code>\u00a0option you will display all of the known event types, whereas the\u00a0<code>-e<\/code>\u00a0option will allow you to choose a specific event category (as returned by\u00a0<strong>perf list<\/strong>):<\/p>\n<p>Will display all cycles event.<\/p>\n<pre>perf top -a \r\n<\/pre>\n<p>Will display all cpu-clock related events.<\/p>\n<pre>perf top -e cpu-clock \r\n<\/pre>\n<div id=\"attachment_20642\" class=\"wp-caption aligncenter\">\n<p><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/Live-Analysis-of-Linux-Performance.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-20642\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/Live-Analysis-of-Linux-Performance.png\" alt=\"Live Analysis of Linux Performance\" width=\"637\" height=\"383\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p class=\"wp-caption-text\">Live Analysis of Linux Performance<\/p>\n<\/div>\n<p>The first column in the output above represents the percentage of samples taken since the beginning of the run, grouped by function Symbol and Shared Object. More options are available in\u00a0<strong>man perf-top<\/strong>.<\/p>\n<h4>Perf record<\/h4>\n<p><strong>perf<\/strong>\u00a0record runs a command and saves the statistical data into a file named\u00a0<strong>perf.data<\/strong>\u00a0inside the current working directory. It runs similarly to\u00a0<strong>perf stat<\/strong>.<\/p>\n<p>Type\u00a0<strong>perf record<\/strong>\u00a0followed by a command:<\/p>\n<pre># perf record dd if=\/dev\/null of=test.iso bs=10M count=1\r\n<\/pre>\n<div id=\"attachment_20643\" class=\"wp-caption aligncenter\">\n<p><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/Record-Command-Statistical-Data.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-20643\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/Record-Command-Statistical-Data.png\" alt=\"Record Command Statistical Data\" width=\"647\" height=\"161\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p class=\"wp-caption-text\">Record Command Statistical Data<\/p>\n<\/div>\n<h4>Perf report<\/h4>\n<p><strong>perf report<\/strong>\u00a0formats the data collected in\u00a0<strong>perf.data<\/strong>\u00a0above into a performance report:<\/p>\n<pre># sudo perf report\r\n<\/pre>\n<div id=\"attachment_20644\" class=\"wp-caption aligncenter\">\n<p><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/Perf-Linux-Performance-Report.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-20644\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/05\/Perf-Linux-Performance-Report.png\" alt=\"Perf Linux Performance Report\" width=\"702\" height=\"554\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p class=\"wp-caption-text\">Perf Linux Performance Report<\/p>\n<\/div>\n<p>All of the above\u00a0<strong>subcommands<\/strong>\u00a0have a dedicated man page that can be invoked as:<\/p>\n<pre># man perf-subcommand\r\n<\/pre>\n<p>where\u00a0<strong>subcommand<\/strong>\u00a0is either\u00a0<strong>list<\/strong>,\u00a0<strong>stat<\/strong>,\u00a0<strong>top<\/strong>,\u00a0<strong>record<\/strong>, or\u00a0<strong>report<\/strong>. These are the most frequently used subcommands; others are listed in the documentation (refer to the Summary section for the link).<\/p>\n<h3>Summary<\/h3>\n<p>In this guide we have introduced you to\u00a0<strong>perf<\/strong>, a performance monitoring and analysis tool for Linux. We highly encourage you to become familiar with its documentation which is maintained in\u00a0<a href=\"https:\/\/perf.wiki.kernel.org\/\" target=\"_blank\" rel=\"noopener\">https:\/\/perf.wiki.kernel.org<\/a>.<\/p>\n<p>If you find applications that are consuming a high percentage of resources, you may consider modifying the source code, or use other alternatives.<\/p>\n<p><a href=\"https:\/\/www.tecmint.com\/perf-performance-monitoring-and-analysis-tool-for-linux\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>When we talk of performance in computing, we refer to the relationship between our resources and the tasks that they allows us to complete in a given period of time. Perf- A Performance Monitoring and Analysis Tool for Linux In a day of fierceless competition between companies, it is important that we learn how to &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.appservgrid.com\/paw92\/index.php\/2019\/03\/17\/perf-a-performance-monitoring-and-analysis-tool-for-linux\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Perf- A Performance Monitoring and Analysis Tool for Linux&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-11848","post","type-post","status-publish","format-standard","hentry","category-linux"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/11848","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/comments?post=11848"}],"version-history":[{"count":1,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/11848\/revisions"}],"predecessor-version":[{"id":11849,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/11848\/revisions\/11849"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/media?parent=11848"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/categories?post=11848"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/tags?post=11848"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}