|
| Problem | Solaris® 8 ships with two threading libraries: many-to-many and one-to-one. The names refer to how the model maps Solaris threads to lwps and kernel threads. By default, the many-to-many thread library is used in Solaris 8. In certain circumstances, this leads to thread starvation and even hangs; eventually, kill -3 (used to create a thread dump) does not work. In our experience, Oracle® and WebSphere® Application Server running on Solaris 8 seems particularly prone to these problems. | | Cause | Reasons for changing the threading libraryBy default, the many-to-many thread library is used on Solaris 8. Switching to the one-to-one library reduces the possibility of thread starvation and improves scalability. This improves the overall performance of the system. One-to-one is the default implementation in Solaris 9 and subsequent releases. | | Solution | Determiningwhich library is currently in use
Run plddPID and check the output for the path to libthread.so. The pldd command lists details of all dynamic libraries linked into the process.
- usr/lib/libthread.so
the many-to-many threading library is currently in use
- usr/lib/lwp/libthread.so
the one-to-one alternate threading library is currently in use
Changing to the alternate threading libraryThere are two ways to change the threading library. The first method is preferred.- Change the LD_LIBRARY_PATH variable (instructions below)
or
- Set the JVM Arg -XX:UseBoundThreads.
Changing to the alternate threading library on V4.0 AE releases
- Edit startupServer.sh located in the WAS_HOME/bin directory.
- Search on $LD_LIBRARY_PATH in the file.
- Prefix /usr/lib/lwp to $LD_LIBRARY_PATH
Example:
LD_LIBRARY_PATH="$WAS_LIBPATH":/usr/lib/lwp$LD_LIBRARY_PATH
export LD_LIBRARY_PATH ;;
- Restart the server instance.
Changing to the alternate threading library on V5.0 and V5.1 releases
LD_LIBRARY_PATH can be set at one place in setupCmdLine.sh.
- Edit the setupCmdLine.sh file located in the install_root/bin directory.
- Search on WAS_LIBPATH in the file.
- Add :/usr/lib/lwp to the WAS_LIBPATH.
Example:
WAS_LIBPATH="$WAS_HOME"/bin:/opt/mqm/java/lib:/opt/wemps/lib:/usr/lib/lwp
- Restart the server instance.
Benefits of changing the threading library
If you run pstack against the hung state, there is at least one thread stack with mutex_lock at the top and some form of signal processing further down in the stack.
- Sample pstack:
----------------- lwp# 218 --------------------------------
ff29bcfc ___lwp_mutex_lock (ff37e000, 1, 0, 0, 0, 2) + c
ff359afc _park (c8e81e30, ff37e000, 0, c8e81d70, 24d54, c9e81d70) + 11c
ff3597bc _swtch (c8e81d70, c8e81d70, ff37e000, 5, 1000, 0) + 424
ff35b178 _mutex_adaptive_lock (ff3898ec, 4c00, 1000, fffeffff, 1, 4d58) + 160
................................... __1cbCCompiledCodeSafepointHandlerbKhandle_illegal_instruction_exception6M_
- Sample pstack:
----------------- lwp# 335 / thread# 4144 --------------------
ff29ef88 lwp_mutex_lock(ff385908)
ff3680e0 sigacthandler(21, e1381d70, e137ec70, 0, cb5e58, ff37e000) + 310
--- called from signal handler with signal 33 (SIGLWP) ---
..................................................
e53d55e4 Java_oracle_jdbc_oci8_OCIDBAccess_do_1execute (cd1744, e138032c,
If you see the indicators above in a pstack of a hung JVM™, change to the alternate thread library.
Full details of this fix are provided on the Sun support site: http://java.sun.com/docs/hotspot/PerformanceFAQ.html#8
A full description of Solaris threading is provided here: http://java.sun.com/docs/hotspot/threads/threads.html
| |
| | |
| |
|
Product categories: Software, Application Servers, Distributed Application & Web Servers, WebSphere Application Server Operating system(s): Solaris Software version: 3.5, 4.0, 5.0, 5.1, 6.0 Software edition: Edition Independent Reference #: 1107291 IBM Group: Software Group Modified date: 2004-07-13
(C) Copyright IBM Corporation 2000, 2004. All Rights Reserved.
|