rotatelogs Questions and Answers

Can rotatelogs write to log files over 2GB?

Platform 1.3 releases? Later releases?
AIX, HP-UX/PA-RISC, Linux, Solaris/SPARC No No
HP-UX/PA-RISC, Solaris/x86_64, z/OS N/A Yes
Windows No Yes

Why does rotatelogs not support large files on some platforms?

This applies to IBM HTTP Server 2.0 and later.

The web server and rotatelogs use file access interfaces provided by the Apache Portable Runtime (APR) library bundled with IBM HTTP Server. The APR provided with IBM HTTP Server on these platforms cannot support file offsets larger than 2GB without introducing an API incompatibility, which would break all current plug-in modules written for IBM HTTP Server. However, it was possible to enable large file support for applications which only append to files without introducing an API incompatibility.

The web server's internal support for error and access log files only appends to the end of log files and does not use file offsets, so the modifications to APR allow large log files when using the internal web server support.

The rotatelogs application interacts with log files in a more complex manner, including the use of file offsets. Thus, the APR changes don't enable large file support in rotatelogs.

Does rotatelogs rotate log files if no requests are received?

No, the rotate operation will not occur until IHS logs another request. If your configuration specifies that rotatelogs performs the rotation operation after 86400 seconds, and if IHS receives no requests after 86400 seconds have elapsed, the new log file will not yet be created. Then, when rotatelogs receives its next request to log, it will create the new log file and close the old one.

Does rotatelogs buffer data before writing to the log file?

No. However, data may be buffered in the operating system kernel after the web server writes the data but before rotatelogs can read it. This time is usually very brief.

Other programs can be used to filter data seen by rotatelogs, and those programs may introduce buffering. Example:

CustomLog "|grep -v \b200\b | /opt/IHS/bin/rotatelogs /opt/IHS/logs/grepped 86400 540" common

In this case, grep will buffer data internally until it has a full buffer, then rotatelogs will see many log records at the same time. (4096 bytes is a typical size for the buffer used by grep.)

Note: IBM HTTP Server 6.0 prior to 6.0.2.1, or IBM HTTP Server 2.0 prior to PK07831, does not support the type of CustomLog directive in the example above.

How can I rotate log files every night at midnight?

Specify a time interval of 86400 seconds. Use the -l option to rotate at midnight local time instead of midnight UTC. IBM HTTP Server 1.3 doesn't support the -l option; specify the UTC offset parameter instead.

Example:

CustomLog "|/usr/HTTPServer/bin/rotatelogs -l /www/logs/access_%Y-%m-%d" custom

This example includes the year, month, and day in the log file name. An example generated filename is /www/logs/access_2007-11-26.

Why are hour, minute, and second fields in generated filenames all zero when I rotate every 24 hours?

When rotating log files based on time intervals, the timestamp in generated filenames represents the beginning of the current interval, regardless of when the web server is started. The beginning of a 24-hour interval is always midnight, so hour, minute, and second fields are always zero. The beginning of a 1-hour interval is always the beginning of the hour, so minute and second fields are always zero.

Why is the timestamp always in UTC when I rotate based on file size?

The -l option and UTC offset parameters are not respected when rotating based on file size.
This is addressed by APAR PK56939.

Error Message: error writing to logfile xxx messages lost

This message can occur when the log has grown to 2 gigabytes on platforms that don't support large files with rotatelogs. In this case, shorten the rotation interval or use size-based rotation.

This message can also occur when there is insufficient free space on the filesystem used for logging.

If this error message occurs, the log file being written to by rotatelogs will be truncated and then new entries will continue to be written to the log.