Understanding logrotate utility
Logs are useful when you want to track usage or troubleshoot an application. As more information gets logged, however, log files use more disk space. Over time a log file can grow to unwieldy size. Running out of disk space because of a large log file is a problem, but a large log file can also slow down the process of resizing or backing up your virtual server. Additionally, it’s hard to look for a particular event if you have a million log entries to skim through. So it’s a good idea to keep log files down to a manageable size, and to prune them when they get too old to be of much use.
Fortunately, the logrotate utility makes log rotation easy. “Log rotation” refers to the practice of archiving an application’s current log, starting a fresh log, and deleting older logs. The system usually runs logrotate once a day, and when it runs it checks rules that can be customized on a per-directory or per-log basis.
How logrotate works
The system runs logrotate on a schedule, usually daily. On most distributions, the script that runs logrotate daily is located at
/etc/cron.daily/logrotate.
Some distributions use a variation. For example, on Gentoo the logrotate script is located at
/etc/cron.daily/logrotate.cron.
If you want logrotate to run more often (for hourly log rotation, for example) you need to use cron to run logrotate through a script in
/etc/cron.hourly.
When logrotate runs, it reads its configuration files to determine where to find the log files that it needs to rotate, how often the files should be rotated, and how many archived logs to keep.
logrotate.conf
The main logrotate configuration file is located at
/etc/logrotate.conf.
The file contains the default parameters that logrotate uses when it rotates logs. The file is commented, so you can skim it to see how the configuration is set up. Several of the specific commands in that file are described later in this article.
Note that one line in the file reads:
include /etc/logrotate.d
That directory contains most of the application-specific configuration files.
logrotate.d
Use the following command to list contents of the directory that stores application-specific log settings:
ls /etc/logrotate.d
Depending on how much is installed on your server, this directory might contain no files or several. In general, applications that are installed through your package manager will also create a config file in
/etc/logrotate.d.
Usually the directory contains a configuration file for your syslog service, which logrotate reads when it rotates the system logs. This file contains an entry for various system logs, along with some commands similar to those contained in
logrotate.conf.
NOTE: On versions of Ubuntu earlier than Karmic Koala (9.10) there is no entry for a syslog service. Before that release, the system logs were rotated by a
savelog command run from the/etc/cron.daily/sysklogd script.Inside an application file
As an example, consider the contents of a logrotate configuration file that might be put in place when you install Apache on a Fedora system:
/var/log/httpd/*log {
missingok
notifempty
sharedscripts
postrotate
/sbin/service httpd reload > /dev/null 2>/dev/null || true
endscript
}
When logrotate runs, it checks for any files in
/var/log/httpd that end in log and rotates them, if they aren’t empty. If it checks the httpd directory and doesn’t find any log files, it doesn’t generate an error. Then it runs the command in the postrotate/endscript block (in this case, a command that tells Apache to restart), but only after it has processed all the specified logs.
This example file does not contain some settings that are included in the
logrotate.conf file. The commands in logrotate.conf act as defaults for log rotation. You can specify different settings for any application when you want to override the defaults. For example, if you run a busy web server, you might want to include a daily command in Apache’s configuration block so that Apache’s logs will rotate daily instead of the default weekly rotation.
The next section describes some of the more commonly-used commands actually do in a logrotate configuration file.
Configuration commands
You can get a full list of commands used in logrotate configuration files by checking the man page:
man logrotate
This section describes the more commonly-used commands.
Remember, the configuration files for applications in
/etc/logrotate.d inherit their defaults from the main /etc/logrotate.conf file.Log files
A log file and its rotation behavior are defined by listing the log file (or files) followed by a set of commands enclosed in curly brackets. Most application configuration files will contain just one of these blocks, but it’s possible to put more than one in a file, or to add log file blocks to the main
logrotate.conf file.
You can list more than one log file for a block by using a wildcard in the name or by separating log files in the list with spaces. For example, to specify all files in the directory /var/foo that end in
.log, and the file/var/bar/log.txt, you would set up the block as follows: /var/foo/*.log /var/bar/log.txt {
rotate 14
daily
compress
delaycompress
sharedscripts
postrotate
/usr/sbin/apachectl graceful > /dev/null
Endscript
}
Rotate count
The
rotate command determines how many archived logs are returned before logrotate starts deleting the older ones. For example:rotate 4
This command tells logrotate to keep four archived logs at a time. If four archived logs exist when the log is rotated again, the oldest one is deleted to make room for the new archive.
Rotation interval
You can specify a command that tells logrotate how often to rotate a particular log. The possible commands include:
daily
weekly
monthly
yearly
If a rotation interval is not specified the log will be rotated whenever logrotate runs (unless another condition like
size has been set).
If you want to use a time interval other than the the defined ones, you need to use cron to create a separate configuration file. For example, if you want to rotate a particular log file hourly, you could create a file in
/etc/cron.hourly (you might need to create that directory too) that would contain a line like the following:/usr/sbin/logrotate /etc/logrotate.hourly.conf
Then you would put the configuration for that hourly run of logrotate (the log file location, whether or not to compress old files, and so on) into
/etc/logrotate.hourly.conf.Size
You can use the
size command to specify a file size for logrotate to check when determining whether to perform a rotation. The format of the command tells logrotate what units you’re using to specify the size:size 100k
size 100M
size 100G
The first example would rotate the log if it gets larger than 100 kilobytes, and the second if it’s larger than 100 megabytes, and the third if it’s over 100 gigabytes. I don’t recommend using a limit of 100G, mind you, the example just got a little out of hand there.
The size command takes priority over and replaces a rotation interval if both are set.
Compression
If you want archived log files to be compressed (in gzip format), you can include the following command, usually in
/etc/logrotate.conf:compress
Compression is normally a good idea, because log files are usually all text and text compresses well. If, however, you have some archived logs that you don’t want to compress, but you still want compression to be on by default, you can include the following command in an application-specific configuration:
nocompress
Another command of note in regard to compression is as follows:
delaycompress
This command is useful if you want to compress the archived logs, but want to delay the compression. When
delaycompress is active, an archived log is compressed the next time that the log is rotated. This can be important when you have a program that might still write to its old log file for a time after a fresh one is rotated in. Note that delaycompress works only if you have compress in your configuration.
An example of a good time to use
delaycompress would be when logrotate is told to restart Apache with the “graceful” or “reload” directive. Because old Apache processes do not end until their connections are finished, they could potentially try to log more items to the old file for some time after the restart. Delaying the compression ensures that you won’t lose those extra log entries when the logs are rotated.Postrotate
Logrotate runs the
postrotate script each time it rotates a log specified in a configuration block. You usually want to use this script to restart an application after the log rotation so that the app can switch to a new log.postrotate
/usr/sbin/apachectl restart > /dev/null
endscript
>/dev/null tells logrotate to pipe the command’s output to nowhere. In this case, you don’t need to view the the output if the application restarted correctly.
The
postrotate command tells logrotate that the script to run, starts on the next line, and theendscript command says that the script is done.Sharedscripts
Normally logrotate runs the
postrotate script every time it rotates a log. This is also true for multiple logs that use the same configuration block. For example, a web server configuration block that refers to both the access log and the error log will, if it rotates both, run the postrotate script twice (once for each file rotated). If both files are rotated, the web server is restarted twice.
To keep logrotate from running that script for every log, you can include the following command:
sharedscripts
This command tells logrotate to check all the logs for that configuration block before running the
postrotate script. If one or both of the logs is rotated, the postrotate script runs only once. If none of the logs is rotated, the postrotate script doesn’t run.Where to go next
This article provides an overview of what logrotate does and what kind of configuration options are available to you. You should now be able to explore the existing configurations and adapt them to your needs. To learn how to create an example configuration (to rotate the logs for custom virtual hosts), seeSample logrotate configurations and troubleshooting.
CHAPTER 2
Sample logrotate configuration and troubleshooting
Applying knowledge
In the previous article we talked about what logrotate does and how you can configure it. In this article we’ll apply this new knowledge to putting together a log rotation solution for a custom virtual host or two (or three, or four, etc.). We’ll also look at some options for testing and troubleshooting logrotate.
Tying it all together: virtual host logs
To show how you can use logrotate for your own applications, let’s look at an example that will come in handy for a lot of people: rotating logs for your custom virtual hosts. We’ll use apache for this example, but it can be tweaked pretty easily for other web servers like nginx or lighttpd, usually just by changing the postrotate script.
First we’ll want to create a file to hold the configuration that will tell logrotate what to do with the virtual host’s log files. We won’t edit the main config file or the web server’s config file, since there’s always a possibility that a future package upgrade might want to overwrite the config. Instead we’ll make our own. Let’s call it:
/etc/logrotate.d/virtualhosts
This example tosses all the virtual hosts into one file, but if you have one that’s busier than others you may want to create separate config files to handle the needs of your different domains. We’ll also specify several items that are probably already set in your main config, just so we cover all the bases.
The files
We’ll say that we have two virtual domains, domain1.com and domain2.com, and that the log files for each are in /home/demo/public_html/(domain name)/log. The first thing we’ll do in our config file is tell logrotate where to find the log files, then start the config block for them:
/home/demo/public_html/domain1.com/log/*log /home/demo/public_html/domain2.com/log/*log {
If you have more log directories or files to add, just insert them into that list.
Rotate
Next we’ll want to make sure logrotate only keeps as many old logs as we want:
rotate 14
We’ll use 14 in this example to keep two weeks’ worth of logs, but you can of course adjust that number to something suitable to your requirements.
Interval
Now we’ll tell the web server to rotate these logs daily (again, change it if you prefer a longer interval):
daily
Size (optional)
The size setting specifies a maximum size for your logs. When the log reaches that size a log rotation is triggered.
size 50M
The size setting is optional here because it actually overrides any time-based rotation conditions. You can specify both a time interval and a max size, but if you do the time interval setting will be ignored.
Compression
We’ll specify whether or not we want these logs to be compressed when they’re archived. For this example we’ll use delaycompress to account for the graceful restart of apache, which means we also need to turn compression on:
compress
delaycompress
Sharedscripts
You might have several virtual hosts, and that would mean several logs to rotate. To make sure the web server only gets restarted after all the rotations are done we add:
sharedscripts
Postrotate
We’ll specify a postrotate script that will restart the web server:
postrotate
/usr/sbin/apachectl graceful > /dev/null
endscript
And finally, we close the config block with a curly bracket:
}
The whole shebang
Once we bring it all together our config file will look like this:
/home/demo/public_html/domain1.com/log/*log /home/demo/public_html/domain2.com/log/*log {
rotate 14
daily
compress
delaycompress
sharedscripts
postrotate
/usr/sbin/apachectl graceful > /dev/null
endscript
}
You’ll want to test that, of course, either by making sure you’re watching things when the nightly cron jobs are run, or by running logrotate right now:
/usr/sbin/logrotate /etc/logrotate.conf
If you don’t get any errors back you should be okay. But if you want to be absolutely certain you can run through some of the tests we would use when we suspect something isn’t working right.
Testing logrotate
If you suspect logrotate is having some trouble, or you just want to make sure a new config you’ve put in place will work, there are some useful flags you can pass to logrotate when you run it from the command line:
Verbose
The verbose flag, “-v”, tells logrotate to say what it’s doing while it’s doing it. It’s very useful when trying to find out why logrotate doesn’t rotate a log when you want it to.
Debug
The debug flag, “-d”, tells logrotate to go through the motions of rotating logs but not actually rotate them. It can be handy if you want to test a new config file but don’t want any actual log rotation run when you do (if you’re working on a production server, for example).
The debug flag is good for checking that the config file is formatted properly and that logrotate can find the log files it would rotate. However, since it doesn’t actually run the rotations it doesn’t test some parts of the process like the postrotate scripts.
Force
The force flag, “-f”, forces logrotate to rotate all logs when it runs, whether or not they would normally need to be rotated at that time. If you want to thoroughly test logrotate’s configs this is the flag to use. Just remember that logrotate will be rotating logs and deleting old ones according to the configuration you’ve set up, so don’t accidentally rotate out a recent log you needed to keep.
The force flag can be useful if you’re convinced that logrotate should be rotating a log, but it isn’t. Forcing the issue will help you tell if the problem is that logrotate doesn’t think the log needed rotating (if you run with the force flag and the log is rotated), or if the problem is that logrotate isn’t able to affect the log file (if you run it and nothing happens to the log).
Note that if logrotate is set to add a date to the name of an archived log, not even using the force flag will get logrotate to make a new archive in the same day (since the name it would use for the archive is already taken). In that circumstance you may need to rename the most recent archive (for each log file in a given config block) before you can force a log rotation.
Combining flags
The testing flags can be used together quite effectively. To have logrotate tell you what it would do if you made it rotate everything, but not actually rotate anything, you can combine all three:
/usr/sbin/logrotate -vdf /etc/logrotate.conf
You’ll get treated to a long list of things logrotate would do, including which log files it would rotate and what it would do during that process.
If you then want to test all the rotate configs in their entirety - including the scripts run after rotations - you can run logrotate without the debug flag:
/usr/sbin/logrotate -vf /etc/logrotate.conf
All the logs will be rotated, and skimming the output should help you catch any obvious problems. You’ll also want to make sure that all your services are still running okay (that there was nothing wrong with the postrotate scripts), and that all the logs actually did get rotated.
How logrotate remembers
If you find that a log isn’t rotating even though it’s old enough that it should, a simple way to fix the problem is to manually run logrotate with the “-f” flag. But if you’re the sort who wants to know why something’s gone wrong, there’s one more file you can check before forcing a rotation:
/var/lib/logrotate.status
That file is where logrotate stores information about when it last rotated each log file. If you look inside you’ll see something like:
logrotate state -- version 2
"/var/log/acpid.log" 2010-6-18
"/var/log/iptables.log" 2010-6-18
"/var/log/uucp.log" 2010-6-29
...
It’s a straightforward format - the log file location is on the left, and the date when it was last rotated is on the right. Sometimes it can happen that the dates on your server get a little wonky (if you were tinkering with an NTP service or the like), and the date when a log was last rotated winds up being a future date. If that’s happened you’ll see it here.
If you want to check logrotate out with a particular log file but don’t want to force everything to rotate, you can delete the log’s entry from the logrotate status file. Then when you run logrotate normally it should create a new entry for the log with today’s date (even though it may not actually rotate the log - it uses that first run as a baseline if it’s just interval-based).
Summary
For something that runs quietly in the background and only really performs one type of task, logrotate does quite a bit. You hopefully understand logrotate better than you wanted to. At the least you should be able to set up new logrotate config files for your own purposes, either creating them from scratch or copying existing configs and modifying them appropriately. And most importantly, you can keep your logs from getting out of control.
No hay comentarios:
Publicar un comentario