Using the Logstash collector version 1.0

Use the Logstash collector feature in Liberty to collect log and other events from your Liberty servers and send them to a remote Logstash server. The collected events can be used for log analysis and troubleshooting purposes.

Procedure

  1. Set up Logstash V2.x following the instructions from Elasticsearch.
  2. Create or acquire certificate and key pair files for SSL for Logstash. The following example is the command for openSSL that can be used for generating a certificate and key pair. Customize the number of days the keys are valid as required.
    openssl req -x509 -newkey rsa:2048 -keyout logstash.key -out logstash.crt -days 365 -nodes
  3. For Logstash V2.x and Elasticsearch users, copy the sample into a liberty_logstash_template.json file. See sample logstash index template. Customize the _ttl defaults as required to indicate the number of milliseconds to keep records of each event type.
  4. For Logstash V2.x and Elasticsearch users, copy the sample into a liberty_logstash.conf file. See sample logstash filters. Customize lumberjack ssl_certificate path, ssl_key path, and port number as required. Customize Elasticsearch hosts and template path as required.
  5. Complete the following steps for each of the Liberty servers that you want to collect events from:
    1. Acquire or create a keystore for the Liberty server. To create a self-signed certificate, use the following command. Customize the server name, password, and subject as required.
      d:\wlp\bin\securityUtility createSSLCertificate --server=myServerName --password="Liberty" --subject=CN=myHostname,OU=defaultServer,O=ibm,C=us
    2. Import the logstash.crt file from step 2 into the trust.jks file in your server. Customize the wlp_install_dir and server name as required. When prompted for a password, use the certificate password from step 5a.
      d:\java\bin\keytool -import -noprompt -alias logstash -file logstash.crt -keystore wlp_install_dir\usr\servers\myServerName\resources\security\trust.jks -storepass Liberty
    3. Configure Logstash collector in the server.xml file in Liberty by adding the following content. Customize the logstashCollector list of sources, host name, and port as required.
      <featureManager>
          <feature>logstashCollector-1.0</feature>
      </featureManager>
      
      <keyStore id="defaultKeyStore" password="Liberty" />
      <keyStore id="defaultTrustStore" location="trust.jks" password="Liberty" />
      <ssl id="mySSLConfig" trustStoreRef="defaultTrustStore" keyStoreRef="defaultKeyStore" />
      
      <logstashCollector 
      	source="message,trace,garbageCollection,ffdc,accessLog" 
      	hostName="localhost" 
      	port="5043" 
      	sslRef="mySSLConfig">
      
          
          
      </logstashCollector>
      Note: Trace and access logs are high volume logs and require more network, CPU, and storage resources to collect.
  6. For users of Elasticsearch and Kibana V4.x, import the Kibana dashboard as follows:
    1. Save the Kibana dashboard JSON to a file on your local file system. See Sample Kibana dashboard for Elasticsearch and Kibana V4.x users.
    2. In Kibana, configure the index pattern by clicking Settings > Indices. Choose the Index name as logstash-* and Time-field name as datetime.
    3. Click Settings > Objects > Import to import the dashboard into Kibana. When prompted, provide the path to the file you saved in the previous step.
  7. Save the dashboard by using the save (disk) icon. Enter Liberty in the text box that is provided and click the save icon next to the text box. The next time that you go to Kibana from any browser, you can reload this dashboard by using the load icon, and clicking Liberty.

Results

You configured your Liberty servers to send events to your Logstash server and can now view your events in the Liberty dashboard by using Kibana.


Icon that indicates the type of topic Task topic



Timestamp icon Last updated: Monday, 5 December 2016
http://www14.software.ibm.com/webapp/wsbroker/redirect?version=cord&product=was-libcore-mp&topic=twlp_analytics_logstash
File name: twlp_analytics_logstash.html