Changing to the alternate, one-to-one threading library on Solaris 8 improves performance
 Technote (troubleshooting)
 
Problem(Abstract)
Solaris 8 ships with two threading libraries: many-to-many and one-to-one. The names refer to how the model maps Solaris threads to lwps and kernel threads. By default, the many-to-many thread library is used in Solaris 8. In certain circumstances, this leads to thread starvation and even hangs; eventually, kill -3 (used to create a thread dump) does not work. From experience, it has been noted that Oracle and IBM® WebSphere® Application Server running on Solaris 8 seems particularly prone to these problems.
 
Cause
By default, the many-to-many thread library is used on Solaris 8. Switching to the one-to-one library reduces the possibility of thread starvation and improves scalability. This improves the overall performance of the system. One-to-one is the default implementation in Solaris 9 and subsequent releases.
 
Resolving the problem

Determining which library is currently in use

Run pldd PID and check the output for the path to libthread.so. The pldd command lists details of all dynamic libraries linked into the process.
  • usr/lib/libthread.so
    The many-to-many threading library is currently in use.

  • usr/lib/lwp/libthread.so
    The one-to-one alternate threading library is currently in use.

Changing to the alternate threading library

There are two ways to change the threading library. The first method is preferred.
  • Change the LD_LIBRARY_PATH variable (instructions below)

    OR

  • Set the JVM Arg -XX:UseBoundThreads.

Changing to the alternate threading library on Version 5.x and 6.x

The LD_LIBRARY_PATH can be set at one place in the setupCmdLine.sh script:
  1. Edit the setupCmdLine.sh file located in the install_root/bin directory.

  2. Search on WAS_LIBPATH in the file.

  3. Append :/usr/lib/lwp to WAS_LIBPATH.

    For example:

    WAS_LIBPATH="$WAS_HOME"/bin:/opt/mqm/java/lib:/opt/wemps/lib:/usr/lib/lwp

  4. Restart the server instance.

Changing to the alternate threading library on Version 4.0 AE

  1. Edit the startupServer.sh script located in the install_root/bin directory.

  2. Search on $LD_LIBRARY_PATH in the file.

  3. Prefix $LD_LIBRARY_PATH with /usr/lib/lwp.

    For example:

    LD_LIBRARY_PATH="$WAS_LIBPATH":/usr/lib/lwp$LD_LIBRARY_PATH
    export LD_LIBRARY_PATH ;;

  4. Restart the server instance.

Benefits of changing the threading library

If you run pstack against the hung state, there is at least one thread stack with mutex_lock at the top and some form of signal processing further down in the stack.
  • Sample pstack:

    -----------------  lwp# 218  --------------------------------
    ff29bcfc
    ___lwp_mutex_lock (ff37e000, 1, 0, 0, 0, 2) + c
    ff359afc _park    (c8e81e30, ff37e000, 0, c8e81d70, 24d54, c9e81d70) + 11c
    ff3597bc _swtch   (c8e81d70, c8e81d70, ff37e000, 5, 1000, 0) + 424
    ff35b178 _mutex_adaptive_lock (ff3898ec, 4c00, 1000, fffeffff, 1, 4d58) + 160
    ...................................  __1cbCCompiledCodeSafepointHandlerbKhandle_illegal_instruction_exception6M_

  • Sample pstack:

    -----------------  lwp# 335 / thread# 4144  --------------------
    ff29ef88
    lwp_mutex_lock (ff385908)
    ff3680e0
     sigacthandler (21, e1381d70, e137ec70, 0, cb5e58, ff37e000) + 310
    --- called from signal handler with signal 33 (SIGLWP) ---
    ..................................................
    e53d55e4 Java_oracle_jdbc_oci8_OCIDBAccess_do_1execute (cd1744, e138032c,

In the preceding sample, if you see the indicators in a pstack of a hung JVM, change to the alternate thread library.

Full details of this fix are provided in Frequently Asked Questions About the Java HotSpot VM on the Sun support site.

A full description of Solaris threading.
 
 
Cross Reference information
Segment Product Component Platform Version Edition
Application Servers Runtimes for Java Technology Java SDK
 
 


Document Information


Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server > General
Operating system(s): Solaris
Software version: 6.0
Software edition:
Reference #: 1107291
IBM Group: Software Group
Modified date: Feb 23, 2006