Troubleshooting Java applications
If you have a problem with a Java™ application, you can use the diagnostics that are provided by CICS® and the JVM to determine the cause of the problem.
About this task
You can use freely available tools that perform real-time and offline analysis of a JVM, for example IBM® Health Center. For full details, see IBM Monitoring and Diagnostic Tools - Health Center.
For troubleshooting web applications that are running in a Liberty JVM server, see Troubleshooting Liberty JVM servers and Java web applications. For information about where to find log files see Controlling the location for JVM stdout, stderr, JVMTRACE, and dump output.
Procedure
- If you are unable to start a JVM server, check that the
setup of your Java installation
is correct. Use the CICS messages
and any errors in the stderr file for the JVM
to determine what might be causing the problem.
- Check that the correct version of the Java SDK is installed and that CICS has access to it in z/OS UNIX. For a list of supported SDKs, see High-level language and compiler support.
- Check that the USSHOME system initialization parameter is set in the CICS region. This parameter specifies the home for files on z/OS UNIX.
- Check that the JVMPROFILEDIR system initialization parameter is set correctly in the CICS region. This parameter specifies the location of the JVM profiles on z/OS UNIX.
- Check that the CICS region has read and run access to the z/OS UNIX directories that contain the JVM profiles.
- Check that the CICS region has write access to the working directory of the JVM. This directory is specified in the WORK_DIR option in the JVM profile.
- Check that the JAVA_HOME option in the JVM profiles points to the directory that contains the Java SDK.
- Check that SDFJAUTH is in the STEPLIB concatenation of the CICS startup JCL.
- If you are using WebSphere® MQ or DB2® DLL files, check that the 64-bit versions of these files are available to CICS.
- If you modify DFHAXRO to configure the Language Environment® enclave, ensure that the runtime options do not exceed 200 bytes and that the options are valid. CICS does not validate the options that you specify before it passes them to Language Environment. Check SYSOUT for any error messages from Language Environment.
- If your setup is correct, gather diagnostic information
to determine what is happening to the application and the JVM.
- To obtain the diagnostics, you must use PRINT_JVM_OPTIONS=TRUE. The default for this option is PRINT_JVM_OPTIONS=FALSE, so if it is left to default no options for diagnostics are presented. When you specify PRINT_JVM_OPTIONS=TRUE, all the options that are passed to the JVM at startup, including the contents of the class paths, are printed to SYSPRINT. The information is produced every time a JVM is started with this option in its profile.
- Check the dfhjvmout and dfhjvmerr files for information and error messages from the JVM. These files are in the directory that is specified by the WORK_DIR/applid/jvmserver option in the JVM profile. The files might have different names if the STDOUT and STDERR options were changed in the JVM profile.
- If the application is failing or performing poorly, debug
the application.
- If you receive java.lang.ClassNotFoundException errors and the transaction abends with the AJ05 code, the application might not be able to access IBM or vendor classes in the OSGi framework. For more information about how to fix this problem, see Upgrading the Java environment.
- Use the CEDX transaction to debug the application transaction. For a Liberty JVM server, if
you are using a URI map to match the inbound application request to an application transaction,
debug that transaction.
If you use the default transaction CJSA, you must set the
MAXACTIVE attribute to 1 on the DFHEDFTC transaction class (or DFHEDFTO transaction class if you use
CEDY).
This setting is required because a number of CJSA tasks might be running and you might
debug the wrong transaction. Do not use CEDX on the CJSA transaction in a production environment. - To use a debugger with the JVM server, you must set some options in the JVM profile. For more information, see Debugging a Java application.
- If you want to determine the status of OSGi bundles and
services, use the OSGi console. Set the following properties in the JVM profile:
-Dosgi.file.encoding=ISO-8859-1, and -Dosgi.console=host:port
where host is the host name of the system the JVM server is running on, and
port is a free port on the same system. While the
osgi.console.encoding property was designed to allow the OSGi console to use a
preferred encoding without putting the whole JVM into that encoding, an outstanding bug in the
Equinox OSGi framework prevents its use, instead you must set the file.encoding
value to an ASCII based encoding. If you are using an OSGi JVM server, add
OSGI_CONSOLE=TRUE to the JVM profile. If you are using a Liberty JVM server,
add the osgiConsole-1.0 feature to the
server.xml. Connect to the OSGi console by using a Telnet session with the host
and port properties you specified in the JVM profile.Note: If you type the exit command into the OSGi console, it will issue a system.exit(0) call to the environment that the JVMSERVER runs in. The command to disconnect your terminal from the OSGi console is disconnect. system.exit(0) is an abrupt stop of all threads and workload, and if left to continue processing, can leave the JVM and CICS in an indeterminate state. CICS is designed to perform an immediate shutdown in this eventuality to avoid subsequent complications. For this reason, it is important to control write access to both the JVM profile, and server.xml. A Liberty JVM server offers further protection by requiring inclusion of the osgiConsole-1.0 feature before the OSGi console is able to run. The OSGi console is primarily a development and debug aid, and is not expected to run in a production environment.
- If you are getting out-of-memory errors, it might indicate
that the JVM or CICS address
space was not allocated enough storage, the application might have
a memory leak, or the heap size might be insufficient.
- Use CICS statistics or a tool such as IBM Health Center to monitor the JVM. If the application has a memory leak, the amount of live data that remains after garbage collection gradually increases over time until the heap is exhausted. The JVM server statistics report the size of the heap after the last garbage collection and the maximum and peak size of the heap. For more information, see Analyzing Java applications using IBM Health Center.
- Run the storage reports for Language Environment to find out whether the amount of storage is sufficient. For more information, see Language Environment enclave storage for JVMs.
- If you are getting encoding errors when you install or
run a Java application, maybe
you set up conflicting or an unsupported combination of code pages. JVMs on z/OS typically
use an EBCDIC code page for file encoding; the default for non-Liberty
JVM servers is IBM1047 (or cp1047), but the JVM can use other code
pages for file encoding if required. CICS requires
an EBCDIC code page to handle character data and all JCICS calls must
use an EBCDIC code page. The code page is set in the LOCALCCSID system
initialization parameter for the CICS region.
- Check the JVM server logs to see whether any warning messages were issued relating to the value of LOCALCCSID. If this parameter is set to a non-EBCDIC code page, a code page that is not supported by the JVM, or an EBCDIC code page that is not supported (such as 930), the JVM server uses cp1047.
- JCICS calls use the code page that is specified in the LOCALCCSID system initialization parameter. If your application expects a different code page, you get encoding errors. To use a different code page for JCICS, set the -Dcom.cics.jvmserver.override.ccsid= parameter in the JVM profile.
- If you are using the -Dcom.cics.jvmserver.override.ccsid= parameter in the JVM profile, ensure that the CCSID is an EBCDIC code page. The application must use EBCDIC when it uses JCICS calls.
- If you are running SOAP processing in an Axis2 JVM server, ensure that the -Dfile.encoding JVM property specifies an EBCDIC property. If you specify a non-EBCDIC code page, such a UTF-8, the web service request fails and the response contains corrupted data.
- If you experience startup timeouts or timeouts
under workload, there are various parameters that you can tune to
help resolve the issue. The following give an indication of values
you can tune:
- Modify your -Dcom.ibm.cics.jvmserver.threadjoin.timeout setting to control how long an HTTP request waits to obtain a JVM server thread.
- Increase the THREADLIMIT value on the JVMSERVER resource.
- If THREADLIMIT is already set to the
maximum permitted value, then you might be attempting
to run more work than a single JVM server can handle.
Consider balancing the workload between multiple JVM servers or multiple
regions.
Alternatively, your CICS system might be unresponsive because of other constraints. Follow the standard procedures to diagnose performance problems. See Improving the performance of a CICS system.
What to do next
- Diagnostics for Java
Many of the usual sources of CICS diagnostic information contain information that applies to Java applications. In addition to the information supplied by CICS, there are a number of interfaces specific to the JVM that you can use for problem determination. - Troubleshooting Liberty JVM servers and Java web applications
If you have a problem with a Java web application, you can use the diagnostics that are provided by CICS and Liberty to determine the cause of the problem. - Controlling the location for JVM stdout, stderr, JVMTRACE, and dump output
Output from Java applications that are running in a JVM server can be written to the z/OS UNIX files. The z/OS UNIX files are named by the STDOUT, STDERR, and JVMTRACE options in the JVM server or routed to the JES log. - Managing the OSGi log files of JVM servers
The OSGi framework writes errors to a set of log files in the working directory of the JVM server. You can manage the number and size of the log files for each JVM server if the defaults are not appropriate for your environment. - CICS component tracing for JVM servers
In addition to the logging produced by Java, CICS provides some standard trace points in the SJ (JVM) and AP domains for 0, 1, and 2 trace levels. These trace points trace the actions that CICS takes in setting up and managing JVM servers. - Activating and managing tracing for JVM servers
You can activate JVM server tracing by turning on SJ and AP component tracing. Small amounts of trace are written to the internal trace table, but Java also writes out logging information to a unique file in zFS for each JVM server. This file does not wrap so you must manage its size in zFS. - Debugging a Java application
The JVM in CICS supports the Java Platform Debugger Architecture (JPDA), which is the standard debugging mechanism provided in the Java Platform. - The CICS JVM plug-in mechanism
In addition to the standard JPDA debug interfaces in the JVM, CICS provides a set of interception points (plug-ins) in the CICS Java middleware, which can be useful for debugging applications. You can use these plug-ins to insert additional Java programs immediately before and after the application Java code is run.
java_troubleshooting.html |
Last updated: Monday, 5 June 2017