Configuring logging
To customize logging, you can use the Configuration and Administration forms or edit directives in the proxy configuration file.
- Paths and file names to store log files
- Filters to include and exclude information from access log files
- Maintenance options to archive or delete logs
About logs
- Access logs:
- Access log—Tracks local administrative requests to the Caching Proxy itself.
- Cache access log—Tracks requests for objects that are in the cache.
- Proxy access log—Tracks requests that are proxied from origin servers.
- Event log—Tracks cache information messages.
- Error log—Tracks error messages that are related to the Caching Proxy.
Caching Proxy creates new log files each day at midnight. If the proxy is not running at midnight, new logs are created the first time that it is started that day. You can specify the directory and file name prefix for each log file; each log file that is created also contains a date suffix in the form .Mmmddyyyy (for example,.Apr142000).
Because logs can use a large amount of space, consider storing your log files on a storage device that is separate from the operating system and cache, to prevent errors.
Log file names and basic options
To specify the basic configuration of the proxy server logs, in the Configuration and Administration forms, select Server Configuration –> Logging –> Log Files. Specify the path and file name for each log file that you want to use. The current file name for each log is displayed in the corresponding text box; if you have not specified a path, the default path is displayed.
Information that is logged in to the proxy logs is not automatically written to the system log, but you can configure Caching Proxy to write to the system login addition to or instead of to its own logs. On the Log Files form, select the Log information to Syslog check box. The system log must be created before this option is chosen.
To specify that proxy server log information is written only to the system log, you must edit the proxy configuration file.
Related configuration file directives
- AccessLog — Name the path for the access log file
- CacheAccessLog — Specify the path for the cache access log files
- ErrorLog — Specify the file where server errors are logged
- EventLog — Specify the path for the event log file
- LogToSyslog — Specify whether to send access information to the system log (Linux and UNIX only)
- MaxLogFileSize — Specify the maximum size for each log file
- ProxyAccessLog — Name the path for the proxy access log file
Access log filters
- What was requested
- When it was requested
- Who requested it
- The method of the request
- The type of file that your server sent in response to the request
- The return code, which indicates whether the request was successful
- The size of the data that was sent
Reasons to control what is logged
- To reduce log size
The log files of a busy server can become large enough to fill the server's disk space. By default, all access requests are logged, which means that log entries are made not only for an HTML page but for each image that the page contains. Including only meaningful access requests can significantly reduce the number of entries in the log. For example, you might want to configure your access logs so that they include log entries for HTML page access requests, but not for requests for GIF images.
- To collect targeted information
For example, if you are interested in who is accessing your server from outside your company, you can filter out access requests that originate from IP addresses within your company. If you are interested in finding out the size of the audience for a particular website, you can create an access log that shows only the access requests for that URL.
Information that is excluded from access logs is not recorded in any access report and is not available for future use. Therefore, if you are unsure about how much access information you need to track, apply exclusion filters conservatively until you gain experience in monitoring your server.
Configuring access log filters
Access log entries can be filtered based on any of the following attributes:
- URL (files or directories)
- IP address or host name
- User agents
- Method
- MIME type
- Return code
To specify your filters, in the Configuration and Administration forms, select Server Configuration –> Logging –> Access Log Exclusions. Specify only those exclusions that you want. You do not need to use all the categories.
- In the section headed Do not log requests to the following Directories or Files in the Access Log, list the URL paths for which you want to exclude log entries.
- In the section headed Do not log requests from the following user-agents, list the proxy agents for which you want to exclude log entries.
- In the section headed Do not log requests from the following Hostnames or IP addresses, list the host names or IP addresses for which you want to exclude log entries.
- In the section headed Do not log requests with the following Methods:, check the boxes for any methods for which you want to exclude log entries.
- In the section headed Do not log requests for files of the following MIME types,
check the boxes for any MIME types for which you want to exclude log
entries. Note: This directive affects only the Proxy access log. It is not possible to filter a log listing these cached objects by their MIME type. Use AccessLogExcludeURL to do this.
- In the section headed Do not log requests with the following Return Codes, check the boxes for request return codes for which you want to exclude log entries.
Click Submit.
Related configuration file directives
- AccessLogExcludeMethod — Suppress log entries for files or directories requested by a specified method
- AccessLogExcludeMimeType — Suppress Proxy access log entries for specific MIME types
- AccessLogExcludeReturnCode — Suppress log entries for specific return codes
- AccessLogExcludeURL — Suppress log entries for specific files or directories
- AccessLogExcludeUserAgent — Suppress log entries from specific browsers
- NoLog — Suppress log entries for specific hosts or domains that match a template
Default log settings
- Default paths
All logs are enabled in the Caching Proxy default configuration. They are stored in the logs/ subdirectory of the installation directory. The default paths are the following:
- Local (administrative) access logs:
- Linux and UNIX: /opt/ibm/edge/cp/server_root/logs/local
- Windows: drive :\Program Files\IBM\edge\cachingproxy\cp\logs\local
- Cache access log:
- Linux and UNIX: /opt/ibm/edge/cp/server_root/logs/cache
- Windows: drive :\Program Files\IBM\edge\cachingproxy\cp\logs\cache
- Proxy access log:
- Linux and UNIX: /opt/ibm/edge/cp/server_root/logs/proxy
- Windows: drive :\Program Files\IBM\edge\cachingproxy\cp\logs\proxy
- Error log:
- Linux and UNIX: /opt/ibm/edge/cp/server_root/logs/error
- Windows: drive :\Program Files\IBM\edge\cachingproxy\cp\logs\error
- Event log:
- Linux and UNIX: /opt/ibm/edge/cp/server_root/logs/event
- Windows: drive :\Program Files\IBM\edge\cachingproxy\cp\logs\event
Each log file name is a combination of the base name and a date suffix in the form .Mmmddyyyy , for example, proxy.Feb292000.
- Local (administrative) access logs:
- Default formats
Logs are stored in the common file format by default. A combined log format is also available and can be set by adding the following line to the proxy configuration file (ibmproxy.conf).
The combined log format is similar to the common format but has added fields that show the referrer, the user agent, and cookie information. Local time format is the default time format.LogFileFormat combined
- Default content
By default, all access requests are recorded in the appropriate access log, and no access information is recorded in the system log. Error log information is written to the error log only, and event log information is written to the event log only.
- Default maintenance
In the default configuration, logs are not archived or deleted.
Maintaining and archiving logs
Caching Proxy uses a plug-in to manage logs.
- Compress and remove logs that are older than a specified age.
- Remove logs after they reach a specified age or after that log category reaches a specified collective size.
- Run your own program at midnight each night to maintain and archive the logs.
- If you choose Purge, set the age, file size, or both, to be used in determining which logs to delete. When files are purged by age and size, files older than the maximum age are purged before files larger than the maximum size. When files are purged by size, older logs are deleted first.
- If you choose Compress, set the age of logs to be compressed and the command to use for compressing log files (include all parameters). Also, set the maximum age of logs. After compressing the logs, the maintenance agent deletes compressed logs older than the maximum age.
Related configuration file directives
- CompressAge — Specify when to compress logs
- CompressDeleteAge — Specify when to delete logs
- CompressCommand — Specify the compression command and parameters
- LogArchive — Specify the behavior of log archiving
- Midnight — Specify the API plugin used to archive logs
- PurgeAge — Specify the age limit for a log
- PurgeSize — Specify the limit for the size of the log archive.
Log file scenario
- Logs must carry a local time stamp and must be in a common log file format.
- Access logs must be purged when they are more than 30 days old or when the logs reach a collective size of 25 MB.
- The following request types must not be logged to the access logs:
- Requests for GIF images
- Requests from hosts with IP addresses that match the pattern 130.128.*.*
- Redirection requests (these requests yield a return code 300 - 399)
- Optionally, select the Log Files form to specify paths for the access log files. (Default paths are provided.)
- Use the Log Archiving form to specify how
to archive the files:
- Specify Purge as the archiving method.
- Under When using Purge, complete the fields
as follows:
- Delete logs older than 30 Days
- Delete logs larger than 25 MB
- Use the Access Log Exclusions form to filter
log entries as follows:
- In the Do not log requests from the following hostnames or IP addresses list, add 130.128.*.* in the Excluded Host field.
- Under Do not log requests for files of the following MIME types, select the image/gif check box.
- Under Do not log requests with the following Return Codes, select the (3xx) Redirection check box.
LogArchive purge
PurgeAge 30
PurgeSize 25
AccessLogExcludeURL *.gif
NoLog 130.128.*.*
AccessLogExcludeReturnCode 300