Logstash and Bluemix log collectors
Liberty generates various events at runtime, such as log events, trace events, first failure data capture (ffdc) events, access log events, and garbage collection events. It is helpful to consolidate events from all servers so the events can be searched, filtered, and analyzed, particularly when you are managing many servers, or when you are running servers on different platforms (for example on dedicated hardware and in the cloud). It can also be helpful to store events on a separate server in cases where you use Liberty in environments that lack persistent file storage for problem determination data.
- Message log events
- Trace log events
- HTTP access log events (from the Liberty server)
- FFDC log events
- Garbage collection events (available for IBM JDKs only)
Logstash collector
The Logstash collector feature (logstashCollector-1.0 or logstashCollector-1.1) sends events to a Logstash server that you provide.Logstash can be used with the Elasticsearch search server and Kibana dashboard, all of which you provide, set up, and manage, to give a consolidated view of logs or other events from across your enterprise. Note that there are no separate processes or agents to be set up on the Liberty server machine when using the Logstash collector to forward events.

Event structure
The collectors send each event as a set of field name-value pairs. Each different type of event has its own set of fields. Knowing which fields each event has is useful when creating your own Kibana dashboards.| logstashCollector-1.0 Field Name | logstashCollector-1.1 / bluemixLogCollector-1.1 Field Name | Field Description |
|---|---|---|
| type | type | a string that identifies the type of event |
| datetime | was_datetime | time at which the event occurred |
| hostName | host | host name of the server that was the source of the event For more information about the rules to set the host name, see Using Logstash and Bluemix log collectors with Liberty running in Docker |
| wlpUserDir | was_userDir | user directory of the server that was the source of the event, for example, D:\wlp\usr |
| serverName | was_serverName | server name of the server that was the source of the event For more information about the rules to set the server name, see Using Logstash and Bluemix log collectors with Liberty running in Docker |
| sequence | was_sequence | sequence number of event (useful for sorting records with the same time stamp) |
- Message events
Table 2. Message events (type: "liberty_message":) logstashCollector-1.0 Field Name logstashCollector-1.1 / bluemixLogCollector-1.1 Field Name Field Description severity severity indicator (F = Fatal, E = Error, W = Warning, A = Audit, I = Info, O = SystemOut, R = SystemErr) loglevel severity indicator (FATAL, ERROR, WARNING, AUDIT, INFO, CONFIG, DETAIL)
messageId was_messageId message ID in the log line, which can be used to find out specific types of errors, for example, SRVE0250I methodName was_methodName method name from log record className was_className class name from log record loggerName module logger name from log record threadId was_threadId thread ID in the log line, for example, 00000015. Note that the thread ID is a string and not a number message message the message from log record, starting with the message ID - Trace events
Table 3. Trace events (type: "liberty_trace":) logstashCollector-1.0 Field Name logstashCollector-1.1 / bluemixLogCollector-1.1 Field Name Field Description severity severity indicator (1 = Fine, 2 = Finer, 3 = Finest, > = Entry, < = Exit) loglevel severity indicator (FINE, FINER, FINEST, ENTRY, EXIT) methodName was_methodName method name from log record className was_className class name from log record loggerName module logger name from log record threadId was_threadId thread ID in the log line, for example, 00000015. Note that the thread ID is a string and not a number message message the message from log record - HTTP access log events
Table 4. HTTP access log events (type: "liberty_accesslog":) logstashCollector-1.0 Field Name logstashCollector-1.1 / bluemixLogCollector-1.1 Field Name Field Description uriPath was_uriPath Path information for the requested URL. This does NOT contain the query parameters, for example, /pushworksserver/ push/apps/tags requestMethod was_requestMethod HTTP verb used, for example, GET remoteHost was_remoteHost remote host IP address, for example, 127.0.0.1 userAgent was_userAgent userAgent value in the request requestProtocol was_requestProtocol Protocol type, for example, HTTP/1.1 queryString was_queryString string representing query string from the HTTP request, for example, color=blue&size=large
bytesReceived was_bytesReceived bytes received in the URL, for example, 94 responseCode was_responseCode HTTP response code, for example, 200 elapsedTime was_elapsedTime time taken to serve the request, in microseconds requestHost was_requestHost request host IP address, for example, 127.0.0.1 requestPort was_requestPort port number of the request - FFDC events
Table 5. FFDC events (type: "liberty_ffdc":) logstashCollector-1.0 Field Name logstashCollector-1.1 / bluemixLogCollector-1.1 Field Name Field Description className was_className the class that emitted the FFDC entry exceptionName was_exceptionName the exception that was reported in the FFDC entry probeID was_probeID the unique identifier of the FFDC point within the class stackTrace was_stackTrace the stack trace of the FFDC incident objectDetails was_objectDetails the incident details for the FFDC incident threadId was_threadId the thread ID of the FFDC incident - Garbage collection events
Table 6. Garbage collection events (type: "liberty_gc":) logstashCollector-1.0 Field Name logstashCollector-1.1 / bluemixLogCollector-1.1 Field Name Field Description heap was_heap the total heap available usedHeap was_usedHeap the amount of heap used duration was_duration the duration for which garbage collection was run, in microseconds gcType was_gcType the type of garbage collection event (for example: Nursery, Global, etc.) reason was_reason the reason for the garbage collection
Using Logstash and Bluemix log collectors with Liberty running in Docker
For running Liberty servers in Docker containers or other virtualized environments, the host/hostName and serverName/was_serverName are set as follows:
- the value of the CONTAINER_HOST environment variable
- the value of the ${defaultHostName} Liberty configuration variable
- the canonical hostname as reported by the JDK
- the value of the CONTAINER_NAME environment variable
- the value of the ${wlp.server.name} Liberty configuration variable
When running Liberty servers in the Bluemix Container Service the CONTAINER_HOST and CONTAINER_NAME are already set for you.
When running Liberty servers in Liberty Docker Collectives the CONTAINER_HOST and CONTAINER_NAME are already set for you.
docker run -d -e LICENSE=accept -e CONTAINER_NAME=yourContainerName -e CONTAINER_HOST=yourContainerHost --name=yourContainerName yourImageName
Concept topic