{"id":11839,"date":"2019-03-17T13:42:45","date_gmt":"2019-03-17T13:42:45","guid":{"rendered":"http:\/\/www.appservgrid.com\/paw92\/?p=11839"},"modified":"2019-03-17T13:42:45","modified_gmt":"2019-03-17T13:42:45","slug":"how-to-setup-and-manage-log-rotation-using-logrotate-in-linux","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw92\/index.php\/2019\/03\/17\/how-to-setup-and-manage-log-rotation-using-logrotate-in-linux\/","title":{"rendered":"How to Setup and Manage Log Rotation Using Logrotate in Linux"},"content":{"rendered":"

One of the most interesting (and perhaps one of the most important as well) directories in a Linux system is\u00a0\/var\/log<\/code>. According to the\u00a0Filesystem Hierarchy Standard<\/a>, the activity of most services running in the system are written to a file inside this directory or one of its subdirectories.<\/p>\n

Such files are known as\u00a0logs<\/strong>\u00a0and are the key to examining how the system is operating (and how it has behaved in the past).\u00a0Logs<\/strong>\u00a0are also the first source of information where administrators and engineers look while troubleshooting.<\/p>\n

If we look at the contents of\u00a0\/var\/log<\/code>\u00a0on a\u00a0CentOS\/RHEL\/Fedora<\/strong>\u00a0and\u00a0Debian\/Ubuntu<\/strong>\u00a0(for variety) we will see the following log files and subdirectories.<\/p>\n

Please note that the result may be somewhat different in your case depending on the services running on your system(s) and the time they have been running.<\/p>\n

In RHEL\/CentOS and Fedora<\/h4>\n
# ls \/var\/log\r\n<\/pre>\n
\n
\"Log<\/a><\/p>\n
Log Files and Directories under CentOS 7<\/p>\n<\/div>\n
In Debian and Ubuntu<\/h4>\n
# ls \/var\/log\r\n<\/pre>\n
\n
\"Log<\/a><\/p>\n
Log Files and Directories in Debian 8<\/p>\n
On both cases, we can observe that some of the log names end as expected in\u00a0\u201clog\u201d<\/strong>, while other are either renamed using a date (for example,\u00a0maillog-20160822<\/strong>\u00a0on\u00a0CentOS<\/strong>) or compressed (consider\u00a0auth.log.2.gz<\/strong>\u00a0and\u00a0mysql.log.1.gz<\/strong>\u00a0on\u00a0Debian<\/strong>).<\/p>\n
This is not a default behavior based on the chosen distribution, but can be changed at will using directives in the configuration files, as we will see in this article.<\/p>\n
If logs were kept forever, they would eventually end up filling the filesystem where\u00a0\/var\/log<\/strong>\u00a0resides. In order to prevent that, the system administrator can use a nice utility called\u00a0logrotate<\/strong>\u00a0to clean up the logs on a periodic basis.<\/p>\n
In few words,\u00a0logrotate<\/strong>\u00a0will rename or compress the main log when a condition is met (more about that in a minute) so that the next event is recorded on an empty file.<\/p>\n
In addition, it will remove\u00a0\u201cold\u201d<\/strong>\u00a0log files and will keep the most recent ones. Of course, we get to decide what\u00a0\u201cold\u201d<\/strong>\u00a0means and how often we want logrotate to clean up the logs for us.<\/p>\n
Installing Logrotate in Linux<\/h3>\n
To install\u00a0logrotate<\/strong>, just use your package manager:<\/p>\n
---------- On Debian and Ubuntu<\/strong> ---------- \r\n# aptitude update && aptitude install logrotate \r\n\r\n---------- On CentOS, RHEL and Fedora<\/strong> ---------- \r\n# yum update && yum install logrotate\r\n<\/pre>\n
It is worth and well to note that the configuration file (\/etc\/logrotate.conf<\/code>) may indicate that other, more specific settings may be placed on individual\u00a0.conf<\/code>\u00a0files inside\u00a0\/etc\/logrotate.d<\/strong>.<\/p>\n
Suggested Read:<\/b>\u00a0Manage System Logs (Configure, Rotate and Import Into Database) Using Logrotate<\/a><\/p>\n
This will be the case if and only if the following line exists and is not commented out:<\/p>\n
include \/etc\/logrotate.d\r\n<\/pre>\n
We will stick with this approach, as it will help us to keep things in order, and use the\u00a0Debia<\/strong>n box for the following examples.<\/p>\n
Options<\/h5>\n
Being a very versatile tool, logrotate provides plenty of directives to help us configure when and how the logs will be rotated, and what should happen right afterwards.<\/p>\n
Let\u2019s insert the following contents in\u00a0\/etc\/logrotate.d\/apache2.conf<\/strong>\u00a0(note that most likely you will have to create that file) and examine each line to indicate its purpose:<\/p>\n
apache2.conf<\/div>\n
\/var\/log\/apache2\/* {\r\n    weekly\r\n    rotate 3\r\n    size 10M\r\n    compress\r\n    delaycompress\r\n}\r\n<\/pre>\n
The first line indicates that the directives inside the block apply to all logs inside\u00a0\/var\/log\/apache2<\/strong>:<\/p>\n
    \n
  1. weekly<\/strong>\u00a0means that the tool will attempt to rotate the logs on a weekly basis. Other possible values are daily and monthly.<\/li>\n
  2. rotate 3<\/strong>\u00a0indicates that only 3 rotated logs should be kept. Thus, the oldest file will be removed on the fourth subsequent run.<\/li>\n
  3. size=10M<\/strong>\u00a0sets the minimum size for the rotation to take place to 10M. In other words, each log will not be rotated until it reaches 10MB.<\/li>\n
  4. compress<\/strong>\u00a0and\u00a0delaycompress<\/strong>\u00a0are used to tell that all rotated logs, with the exception of the most recent one, should be compressed.<\/li>\n<\/ol>\n
    Let\u2019s execute a dry-run to see what logrotate would do if it was actually executed now. Use the\u00a0-d<\/code>\u00a0option followed by the configuration file (you can actually run logrotate by omitting this option):<\/p>\n
    # logrotate -d \/etc\/logrotate.d\/apache2.conf\r\n<\/pre>\n
    The results are shown below:<\/p>\n
    \n
    \"Rotate<\/a><\/p>\n
    Rotate Apache Logs with Logrotate<\/p>\n<\/div>\n
    Instead of compressing the logs, we could rename them after the\u00a0date<\/strong>\u00a0when they were rotated. To do that, we will use the\u00a0dateext<\/code>\u00a0directive. If our date format is other than the default\u00a0yyyymmdd<\/strong>, we can specify it using\u00a0dateformat<\/strong>.<\/p>\n
    Suggested Read:<\/b>\u00a0Install \u2018atop\u2019 to Monitor Logging Activity of Linux System Processes<\/a><\/p>\n
    Note that we can even prevent the rotation from happening if the log is empty with notifempty. In addition, let\u2019s tell logrotate to mail the rotated log to the system administrator (gabriel@mydomain.com<\/strong>\u00a0in this case) for his \/ her reference (this will require a\u00a0mail server to be set up<\/a>, which is out of the scope of this article).<\/p>\n
    If you want to get mails about logrotate, you can setup Postfix mail server as show here:\u00a0Install Postfix Mail Server<\/a><\/p>\n
    This time we will use\u00a0\/etc\/logrotate.d\/squid.conf<\/strong>\u00a0to only rotate\u00a0\/var\/log\/squid\/access.log<\/strong>:<\/p>\n
    squid.conf<\/div>\n
    \/var\/log\/squid\/access.log {\r\n    monthly\r\n    create 0644 root root\r\n    rotate 5\r\n    size=1M\r\n    dateext\r\n    dateformat -%d%m%Y\r\n    notifempty\r\n    mail gabriel@mydomain.com\r\n}\r\n<\/pre>\n
    As we can see in the image below, this log did not need to be rotated. However, when the size condition is met\u00a0(size=1M<\/strong>), the rotated log will be renamed\u00a0access.log-25082016<\/strong>\u00a0(if the log was rotated on\u00a0August 25, 2016<\/strong>) and the main log (access.log<\/strong>) will be re-created with access permissions set to\u00a00644<\/strong>\u00a0and with\u00a0root<\/strong>\u00a0as owner and group owner.<\/p>\n
    Finally, when the number of logs finally reaches\u00a06<\/strong>, the oldest log will be mailed to\u00a0gabriel@mydomain.com<\/strong>.<\/p>\n
    \n
    \"Rotate<\/a><\/p>\n
    Rotate Squid Logs with Logrotate<\/p>\n<\/div>\n
    Now let\u2019s suppose you want to run a custom command when the rotation takes place. To do that, place the line with such command between the postrotate and endscript directives.<\/p>\n
    For example, let\u2019s suppose we want to send an email to root when any of the logs inside\u00a0\/var\/log\/myservice<\/strong>gets rotated. Let\u2019s add the lines in red to\u00a0\/etc\/logrotate.d\/squid.conf<\/strong>:<\/p>\n
    squid.conf<\/div>\n
    \/var\/log\/myservice\/* {\r\n\tmonthly\r\n\tcreate 0644 root root\r\n\trotate 5\r\n\tsize=1M\r\n    \tpostrotate\r\n   \t\techo \"A rotation just took place.\" | mail root\r\n    \tendscript<\/strong>\r\n}\r\n<\/pre>\n
    Last, but not least, it is important to note that options present in\u00a0\/etc\/logrotate.d\/*.conf<\/code>\u00a0override those in the main configuration file in case of conflicts.<\/p>\n
    Logrotate and Cron<\/h3>\n
    By default, the installation of logrotate creates a crontab file inside\u00a0\/etc\/cron.daily<\/strong>\u00a0named\u00a0logrotate<\/strong>. As it is the case with the other crontab files inside this directory, it will be executed daily starting at\u00a06:25 am<\/strong>\u00a0if anacron is not installed.<\/p>\n
    Suggested Read:<\/b>\u00a011 Cron Scheduling Task Examples in Linux<\/a><\/p>\n
     <\/p>\n<\/div>\n
    Otherwise, the execution will begin around\u00a07:35 am<\/strong>. To verify, watch for the line containing\u00a0cron.daily<\/strong>\u00a0in either\u00a0\/etc\/crontab<\/strong>\u00a0or\u00a0\/etc\/anacrontab<\/strong>.<\/p>\n
    Summary<\/h3>\n
    In a system that generates several logs, the administration of such files can be greatly simplified using logrotate. As we have explained in this article, it will automatically rotate, compress, remove, and mail logs on a periodic basis or when the file reaches a given size.<\/p>\n
    Just make sure it is set to run as a cron job and logrotate will make things much easier for you. For more details, refer to the man page.<\/p>\n
    Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
    One of the most interesting (and perhaps one of the most important as well) directories in a Linux system is\u00a0\/var\/log. According to the\u00a0Filesystem Hierarchy Standard, the activity of most services running in the system are written to a file inside this directory or one of its subdirectories. Such files are known as\u00a0logs\u00a0and are the key … <\/p>\n
    Continue reading “How to Setup and Manage Log Rotation Using Logrotate in Linux”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-11839","post","type-post","status-publish","format-standard","hentry","category-linux"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/11839","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/comments?post=11839"}],"version-history":[{"count":1,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/11839\/revisions"}],"predecessor-version":[{"id":11840,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/11839\/revisions\/11840"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/media?parent=11839"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/categories?post=11839"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/tags?post=11839"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}