TroubleShooting: JMX API for WebSphere Application Server V5.0, 5.1, and 6.0
 Technote (troubleshooting)
 
Problem(Abstract)
TroubleShooting for problems with the Websphere Application Server JMX API component. This should help address common issues with this component before calling IBM support and save you time.
 
Resolving the problem

1. Learning more 2. Troubleshooting 3. Collecting data 4. Analyzing data

Steps to help resolve JMX API problems
  1. Are you receiving an exception while compiling JMX java client code?

    For example:

    java.lang.NoClassDefFoundError:com/ibm/ws/security/util/InvalidPasswordDecodingException
    • Yes, see this technote...
    • No, continue to the next question.

      Document needed
      Provide the output from the command prompt where javac compiler command ran.

      Solution
      Search for the missing class (InvalidPasswordDecodingException in this example) in the classpath of the javac executable.

      If running within WebSphere Application Server, WebSphere Network Deployment or WebSphere Application Server Client environment, search inside WAS_HOME\lib, JAVA_HOME\jre\lib or JAVA_HOME\jre\lib\ext directory. After you find the jar file that contains the missing class, add it into the classpath statement of the javac executable.

      In this example, WAS_HOME\lib\iwsorb.jar contains the InvalidPasswordDecodingException class file.

      # Start of Sample batch file to compile AdminClientTester.java code
      @setlocal
      @echo off
      call "C:/WebSphere/DeploymentManager/bin/setupCmdLine.bat"
      SET CONSOLE_ENCODING=-Dws.output.encoding=console
      "%JAVA_HOME%\bin\javac" -classpath .;%WAS_HOME%/lib/ras.jar;%WAS_HOME%/lib/naming.jar;%WAS_HOME%/lib/admin.jar;%WAS_HOME%/lib/wsexception.jar;%WAS_HOME%/lib/jmxc.jar AdminClientTester.java
      @endlocal

      #End of Sample batch file to compile AdminClientTester.java code

  2. Are you receiving an exception while running the JMX java client?
    • Yes, see this technote....
    • No, continue to the next question.

      #Start of Sample batch file to execute AdminClientTester.java code
      @setlocal
      @echo off
      call "C:/WebSphere/DeploymentManager/bin/setupCmdLine.bat"
      set NAMING_FACTORY=com.ibm.websphere.naming.WsnInitialContextFactory
      set LOGIN_SOURCE=-Dcom.ibm.CORBA.loginSource=stdin

      "%JAVA_HOME%\bin\java" %CLIENTSAS% %LOGIN_SOURCE% \
      -Dwas.install.root=%WAS_HOME% \
      -Djava.naming.factory.initial=%NAMING_FACTORY% \
      -Dwas.repository.root=%CONFIG_ROOT% \
      -Dcom.ibm.CORBA.BootstrapHost=%COMPUTERNAME% \
      -classpath .;%WAS_CLASSPATH%;%WAS_HOME%\lib\admin.jar;%WAS_HOME%\lib\wasjmx.jar;%WAS_HOME%\lib\namingclient.jar AdminClientTester %*

      #End of Sample batch file to execute AdminClientTester.java code

      Example
      If you use the wrong SOAP port number for connection, the following exception occurs because the port number given for the target host is not listening on that port; the Operating System refuses the connection.

      Failed in getting admin connection:
      com.ibm.websphere.management.exception.ConnectorException: ADMC0016E: Could not create SOAP Connector to connect to host wastest at port 8877
      ADMC0016E: Could not create SOAP Connector to connect to host wastest at port 8877
      .....
      Caused by: com.ibm.websphere.management.exception.ConnectorNotAvailableException
      .....
      Caused by: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.net.ConnectException: Connection refused: connect; targetException=java.lang.IllegalArgumentException: Error opening socket: java.net.ConnectException: Connection refused: connect]


      Document needed
      Provide the output from the command prompt where the javac compiler command ran.

      Solution
      Verify that the correct hostname and port number is specified for the given SOAP or RMI protocols.

  3. Are you receiving a security exception while running JMX java client?

    For example:
    • Yes, see this technote....
    • No, continue to the next question.


      Example 1
      com.ibm.websphere.management.exception.ConnectorException: ADMC0017E: Could not create RMI Connector to connect to host wastest.raleigh.ibm.com at port 32539
      at com.ibm.websphere.management.AdminClientFactory.createAdminClient(AdminClientFactory.java:394)
      at JMX_WAS_Test.main(JMX_WAS_Test.java:54)
      ---- Begin backtrace for nested exception
      java.lang.reflect.InvocationTargetException: java.lang.IllegalStateException: Error getting SecurityCurrent from the ORB
      at com.ibm.ws.security.util.LoginHelper.getSecurityCurrent(LoginHelper.java:183)
      at com.ibm.ws.security.util.LoginHelper.<init>(LoginHelper.java:65)
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.setupSecurityCredentials(RMIConnectorClient.java:126)
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.<init>(RMIConnectorClient.java:105)
      at java.lang.reflect.Constructor.newInstance(Native Method)
      at com.ibm.websphere.management.AdminClientFactory.createAdminClient(AdminClientFactory.java:340)
      at JMX_WAS_Test.main(JMX_WAS_Test.java:54)

      Example 2
      com.ibm.websphere.management.exception.ConnectorException: ADMC0017E: Could not create RMI Connector to connect to host wastest.raleigh.ibm.com at port 32539
      at com.ibm.websphere.management.AdminClientFactory.createAdminClient(AdminClientFactory.java:394)
      at JMX_WAS_Test.main(JMX_WAS_Test.java:54)
      ---- Begin backtrace for nested exception
      java.lang.reflect.InvocationTargetException: com.ibm.websphere.management.exception.ConnectorAuthenticationException: RMI Connector login failed
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.setupSecurityCredentials(RMIConnectorClient.java:157)
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.<init>(RMIConnectorClient.java:105)
      at java.lang.reflect.Constructor.newInstance(Native Method)
      at com.ibm.websphere.management.AdminClientFactory.createAdminClient(AdminClientFactory.java:340)
      at JMX_WAS_Test.main(JMX_WAS_Test.java:54)
      ---- Begin backtrace for nested exception
      org.omg.SecurityLevel2.LoginFailed: IDL:omg.org/SecurityLevel2/LoginFailed:1.0
      at com.ibm.ISecurityLocalObjectBaseL13Impl.LoginHelperImpl.request_login_controlled(LoginHelperImpl.java:1212)
      at com.ibm.ISecurityLocalObjectBaseL13Impl.LoginHelperImpl.request_login_controlled(LoginHelperImpl.java:821)
      at com.ibm.ISecurityLocalObjectBaseL13Impl.LoginHelperImpl.request_login_controlled(LoginHelperImpl.java:771)
      at com.ibm.ISecurityLocalObjectBaseL13Impl.LoginHelperImpl.request_login_controlled(LoginHelperImpl.java:750)
      at com.ibm.ISecurityLocalObjectBaseL13Impl.LoginHelperImpl.request_login(LoginHelperImpl.java:730)
      at com.ibm.ws.security.util.LoginHelper.login(LoginHelper.java:95)
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.setupSecurityCredentials(RMIConnectorClient.java:146)
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.<init>(RMIConnectorClient.java:105)
      at java.lang.reflect.Constructor.newInstance(Native Method)
      at com.ibm.websphere.management.AdminClientFactory.createAdminClient(AdminClientFactory.java:340)
      at JMX_WAS_Test.main(JMX_WAS_Test.java:54)

      Example 3
      CONNECTOR_TYPE = RMI
      SASloginSource = properties
      SASloginUserid= null
      SAS client config file=file:/home/incdev/JMX/IBM/sas.client.props
      Connecting to wastest:32539 using RMI
      security Credential not Provided on command prompt, may use properties file

      Unable to read required resource: implfactory.properties
      java.lang.NullPointerException
      at java.io.Reader.<init>(Reader.java:59)
      at java.io.InputStreamReader.<init>(InputStreamReader.java:84)
      at java.io.InputStreamReader.<init>(InputStreamReader.java:73)
      at java.util.Properties.load(Properties.java:184)
      at com.ibm.ws.util.ImplFactory$1.run(ImplFactory.java:38)
      at java.security.AccessController.doPrivileged(Native Method)
      at com.ibm.ws.util.ImplFactory.<clinit>(ImplFactory.java:33)
      at com.ibm.ejs.jts.jts.CurrentFactory.<clinit>(CurrentFactory.java:56)
      at com.ibm.ws.naming.jndicos.CNContextImpl.suspendTransaction(CNContextImpl.java:3851)
      at com.ibm.ws.naming.jndicos.CNContextImpl.cosResolve(CNContextImpl.java:3308)
      at com.ibm.ws.naming.jndicos.CNContextImpl.doLookup(CNContextImpl.java:1497)
      at com.ibm.ws.naming.jndicos.CNContextImpl.doLookup(CNContextImpl.java:1458)
      at com.ibm.ws.naming.jndicos.CNContextImpl.lookup(CNContextImpl.java:1168)
      at com.ibm.ws.naming.util.WsnInitCtx.lookup(WsnInitCtx.java:132)
      at javax.naming.InitialContext.lookup(InitialContext.java:345)
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.reconnect(RMIConnectorClient.java:207)
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.<init>(RMIConnectorClient.java:108)
      at java.lang.reflect.Constructor.newInstance(Native Method)
      at com.ibm.websphere.management.AdminClientFactory.createAdminClient(AdminClientFactory.java:340)
      at AdminClientTester.main(AdminClientTester.java:111)
      Failed in getting admin connection:
      com.ibm.websphere.management.exception.ConnectorException: ADMC0017E: Could not create RMI Connector to connect to host wastest at port 32539
      ADMC0017E: Could not create RMI Connector to connect to host wastest at port 32539
      com.ibm.websphere.management.exception.ConnectorException: ADMC0017E: Could not create RMI Connector to connect to host wastest at port 32539
      at com.ibm.websphere.management.AdminClientFactory.createAdminClient(AdminClientFactory.java:394)
      at AdminClientTester.main(AdminClientTester.java:111)
      ---- Begin backtrace for nested exception
      java.lang.reflect.InvocationTargetException: com.ibm.websphere.management.exception.ConnectorNotAvailableException
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.reconnect(RMIConnectorClient.java:215)
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.<init>(RMIConnectorClient.java:108)
      at java.lang.reflect.Constructor.newInstance(Native Method)
      at com.ibm.websphere.management.AdminClientFactory.createAdminClient(AdminClientFactory.java:340)
      at AdminClientTester.main(AdminClientTester.java:111)
      ---- Begin backtrace for nested exception
      javax.naming.NamingException: The JNDI operation "lookup"on the context "" with the name "RMIConnector" failed.  Please get the root cause Throwable contained in this NamingException for more information.  Root exception is java.lang.NoClassDefFoundError: Invalid Implementation Key, com.ibm.ws.transaction.TXCurrent
      at com.ibm.ws.util.ImplFactory.loadClassFromKey(ImplFactory.java:104)
      at com.ibm.ejs.jts.jts.CurrentFactory.<clinit>(CurrentFactory.java:56)
      at com.ibm.ws.naming.jndicos.CNContextImpl.suspendTransaction(CNContextImpl.java:3851)
      at com.ibm.ws.naming.jndicos.CNContextImpl.cosResolve(CNContextImpl.java:3308)
      at com.ibm.ws.naming.jndicos.CNContextImpl.doLookup(CNContextImpl.java:1497)
      at com.ibm.ws.naming.jndicos.CNContextImpl.doLookup(CNContextImpl.java:1458)
      at com.ibm.ws.naming.jndicos.CNContextImpl.lookup(CNContextImpl.java:1168)
      at com.ibm.ws.naming.util.WsnInitCtx.lookup(WsnInitCtx.java:132)
      at javax.naming.InitialContext.lookup(InitialContext.java:345)
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.reconnect(RMIConnectorClient.java:207)
      at com.ibm.ws.management.connector.rmi.RMIConnectorClient.<init>(RMIConnectorClient.java:108)
      at java.lang.reflect.Constructor.newInstance(Native Method)
      at com.ibm.websphere.management.AdminClientFactory.createAdminClient(AdminClientFactory.java:340)
      at AdminClientTester.main(AdminClientTester.java:111)

      Reason for security failure
      Typically, this is caused by incorrect credentials passed to the admin client using the sas.client.props or soap.client.props file.

      Example of soap.client.props
      #--------------------------------------------------------------
      # SOAP Client Security Enablement
      #
      # - security enabled status  ( false[default], true  )
      #--------------------------------------------------------------
      com.ibm.SOAP.securityEnabled=true

      # JMX SOAP connector identity
      com.ibm.SOAP.loginUserid=wasadmin
      com.ibm.SOAP.loginPassword=waspassword

      #--------------------------------------------------------------
      # SOAP Request Timeout
      #
      # - timeout             (specified in seconds [default 180], 0 implies no timeout)
      #
      #--------------------------------------------------------------
      com.ibm.SOAP.requestTimeout=180

      #--------------------------------------------------------------
      # SSL Configuration
      #
      # - keyStore and trustStore                     (fully qualified path to file)
      # - keyStorePassword and trustStorePassword     (string specifying password - encoded or not)
      #--------------------------------------------------------------

      #com.ibm.ssl.keyStore=C\:/WebSphere/DeploymentManager/etc/DummyClientKeyFile.jks
      com.ibm.ssl.keyStore=C\:/WebSphere/DeploymentManager/etc/WASV5ClientKeyFile.jks
      #com.ibm.ssl.keyStorePassword={xor}CDo9Hgw\=
      com.ibm.ssl.keyStorePassword=waspassword
      #com.ibm.ssl.trustStore=C\:/WebSphere/DeploymentManager/etc/DummyClientTrustFile.jks
      com.ibm.ssl.trustStore=C\:/WebSphere/DeploymentManager/etc/WASV5ClientTrustFile.jks
      #com.ibm.ssl.trustStorePassword={xor}CDo9Hgw\=
      com.ibm.ssl.trustStorePassword=waspassword

      #--------------------------------------------------------------
      #  URL protocol handler
      #
      # - stream protocol handler (implementaion of URL handler for "https" URL protocol type)
      #
      #  Note: For IBM JDK, it must be com.ibm.net.ssl.internal.www.protocol.
      #--------------------------------------------------------------
      java.protocol.handler.pkgs=com.ibm.net.ssl.internal.www.protocol

      Example of sas.client.props
      ============================
      traceSettingsFile=file:/C:/WebSphere/DeploymentManager/properties/TraceSettings.properties
      #--------------------------------------------------------------
      # Client Security Enablement
      #
      # - security enabled status  ( false, true [default] )
      #--------------------------------------------------------------
      com.ibm.CORBA.securityEnabled=true

      #--------------------------------------------------------------
      # RMI/IIOP Authentication Protocol (sas, csiv2, both [default])
      #
      # Specify "both" when communicating with 5.0x and previous release servers.
      # Specify "csiv2" when communicating with only 5.0x servers.
      # Specify "sas" when communicating with only previous release servers.
      #--------------------------------------------------------------
      com.ibm.CSI.protocol=both

      #--------------------------------------------------------------
      # Authentication Configuration
      #
      # - authenticationTarget       (BasicAuth [default], this is the only supported selection
      #    on a pure client for this release.  This is for message
      #    layer authentication only, SSL client certificate authentication
      #    is configured below under CSIv2 configuration.)
      # - authenticationRetryEnabled (enables authentication retries if login fails when
      #    loginSource=prompt or stdin)
      # - authenticationRetryCount   (the number of times to retry)
      # - source   (prompt [default], properties, keyfile, stdin, none)
      # - timeout (prompt timeout, specified in seconds, 0 min to 600 max [default 300])
      # - validateBasicAuth          (determines if immediate authentication after uid/pw login,
      #   or wait for method request to send uid/pw to server,
      #   setting this to false gives the previous release behavior.)
      # - securityServerHost  (when validateBasicAuth=true, this property might need to be set
      #    in order for security code to lookup SecurityServer.  Needs to be set to
      #    any running WebSphere server host in the cell you are authenticating to.
      # - securityServerPort    (when validateBasicAuth=true, this property might need to be set
      #    in order for security code to lookup SecurityServer.  Needs to be set to
      #    the bootstrap port of the host chosen above.
      # - loginUserid                (must be set if login source is "properties" )
      # - loginPassword           (must be set if login source is "properties" )
      # - principalName           (format: "realm/userid", only needed in cases where realm
      #                                        is required. Typically the realm is already provided by the
      #                                        server via the IOR and this property is not necessary).
      #
      #--------------------------------------------------------------
      com.ibm.CORBA.authenticationTarget=BasicAuth
      com.ibm.CORBA.authenticationRetryEnabled=true
      com.ibm.CORBA.authenticationRetryCount=3
      com.ibm.CORBA.validateBasicAuth=true
      com.ibm.CORBA.securityServerHost=wastest
      com.ibm.CORBA.securityServerPort=9917
      com.ibm.CORBA.loginTimeout=300
      #com.ibm.CORBA.loginSource=prompt
      com.ibm.CORBA.loginSource=properties
      #com.ibm.CORBA.loginSource=stdin
      #com.ibm.CORBA.loginSource=key file
      #com.ibm.CORBA.loginSource=none

      # RMI/IIOP user identity
      com.ibm.CORBA.loginUserid=wasadmin
      com.ibm.CORBA.loginPassword=waspassword
      com.ibm.CORBA.principalName=

      #--------------------------------------------------------------
      # CSIv2 Configuration (see InfoCenter for more information on these properties).
      #
      # This is where you enable SSL client certificate authentication.  Must also
      # specify a valid SSL keyStore below with a personal certificate in it.
      #--------------------------------------------------------------

      # Does this client support stateful sessions?
      com.ibm.CSI.performStateful=true

      # Does this client support/require BasicAuth (userid/password) client authentication?
      com.ibm.CSI.performClientAuthenticationRequired=false
      com.ibm.CSI.performClientAuthenticationSupported=true

      # Does this client support/require SSL client authentication?
      com.ibm.CSI.performTLClientAuthenticationRequired=false
      com.ibm.CSI.performTLClientAuthenticationSupported=false

      # Note: You can perform BasicAuth (uid/pw) and SSL client authentication (certificate)
      # simultaneously, however, the BasicAuth identity will always take precedence at the server.

      # Does this client support/require SSL connections?
      com.ibm.CSI.performTransportAssocSSLTLSRequired=false
      com.ibm.CSI.performTransportAssocSSLTLSSupported=true

      # Does this client support/require 40-bit cipher suites when using SSL?
      com.ibm.CSI.performMessageIntegrityRequired=true
      com.ibm.CSI.performMessageIntegritySupported=true
      # Note: This property is only valid when SSL connections are supported or required.

      # Does this client support/require 128-bit cipher suites when using SSL?
      com.ibm.CSI.performMessageConfidentialityRequired=false
      com.ibm.CSI.performMessageConfidentialitySupported=true
      # Note: This property is only valid when SSL connections are supported or required.

      #--------------------------------------------------------------
      # SSL Configuration
      #
      # - protocol                              (SSL [default], SSLv2, SSLv3, TLS, TLSv1)
      # - keyStoreType                     (JKS [default], JCEK, PKCS12)
      # - trustStoreType                    (JKS [default], JCEK, PKCS12)
      # - keyStore and trustStore      (fully qualified path to file)
      # - keyStoreClientAlias           (string specifying ssl certificate alias to use from keyStore)
      # - keyStorePassword and trustStorePassword   (string specifying password - encoded or not)
      # - cipher suites                       (refer to InfoCenter for valid ciphers)
      #
      #    com.ibm.ssl.enabledCipherSuites=enabled_cipher_suites
      #
      #    Note: The com.ibm.ssl.enabledCipherSuites property defines the cipher
      #              suites used for the SSL session. If this property is defined, it
      #              overrides the default cipher suites defined for the specified QOP.
      #
      #--------------------------------------------------------------
      com.ibm.ssl.protocol=SSL
      com.ibm.ssl.keyStoreType=JKS
      #com.ibm.ssl.keyStore=C\:/WebSphere/DeploymentManager/etc/DummyClientKeyFile.jks
      com.ibm.ssl.keyStore=C\:/WebSphere/DeploymentManager/etc/WASV5ClientKeyFile.jks
      #com.ibm.ssl.keyStorePassword={xor}CDo9Hgw\=
      com.ibm.ssl.keyStorePassword=waspassword
      com.ibm.ssl.trustStoreType=JKS
      #com.ibm.ssl.trustStore=C\:/WebSphere/DeploymentManager/etc/DummyClientTrustFile.jks
      com.ibm.ssl.trustStore=C\:/WebSphere/DeploymentManager/etc/WASV5ClientKeyFile.jks
      #com.ibm.ssl.trustStorePassword={xor}CDo9Hgw\=
      com.ibm.ssl.trustStorePassword=waspassword

      #--------------------------------------------------------------
      # Quality of Protection for the IBM protocol
      #
      # - perform  ( high [default], medium, low )
      #--------------------------------------------------------------
      com.ibm.CORBA.standardPerformQOPModels=high

      #--------------------------------------------------------------
      # CORBA Request Timeout (used when getting NO_RESPONSE exceptions, typically
      #                        during high-stress loads.  Specify on all processes
      #                        involved in the communications.)
      #
      # - timeout       (specified in seconds [default 180], 0 implies no timeout)
      #
      #    com.ibm.CORBA.requestTimeout=180
      #--------------------------------------------------------------
      com.ibm.CORBA.requestTimeout=180


      SSL Certificates
      If the server is not using the DummyServerKeyFile.jks and DummyServerTrustFile.jks, the certificates must be extracted from those jks files and added to DummyClientKeyFile.jks and DummyClientTrustFile.jks for SSL negotiation to work.

      Tracing the server side.
      Use the following trace specification on the server side:
      management=all=enabled:Admin=all=enabled:SSL=all=enabled

      Tracing the Client side:
      For RMI protocols, use the following property traceSettingsFile=TraceSettings.properties

      Example of TraceSettings.properties file
      ========================================
      ###################################################################
      # A property to specify where the output messages and trace (if enabled)
      # should be routed. Valid values include stdout or a fully qualified
      # file name. If a file is specified but cannot be opened, stdout is used.
      #
      # Sample invocations
      # To specify stdout - traceFileName=stdout
      # To specify a file on NT - traceFileName=c:\\MyTraceFile.log or
      #                                          traceFileName=c:/MyTraceFile.log
      # To specify a file on Unix - traceFileName=/usr/MyTraceFile.log
      ###################################################################
      #traceFileName=C:\\WebSphere\\DeploymentManager\\logs\\SASTraceFile.log
      traceFileName=SASTraceFile.log
      #traceFileName=stdout
      ###################################################################
      # Specify trace strings here. Trace strings take the form of:
      # logger={level}={type} where:
      #     level = entryexit || debug || event || all
      #     type =  enabled || disabled
      # examples:
      # com.ibm.ejs.ras.SharedLogBase=all=enabled enables all tracing for the
      #         single logger created in class com.ibm.ejs.ras.SharedLogBase.
      # com.ibm.ejs.*=debug=enabled enables debug tracing for all loggers with
      #         names starting with com.ibm.ejs.
      #
      # Multiple trace strings can be specified, one per line.
      ###################################################################
      *=all=enabled


      For a listing of all technotes, downloads, and educational materials specific to the component, search the WebSphere Application Server support site.
      http://www.ibm.com/software/webservers/appserv/was/support/
 
 
Cross Reference information
Segment Product Component Platform Version Edition
Application Servers Runtimes for Java Technology Java SDK
 
 


Document Information


Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server > Java Management Extensions (JMX) or JMX client API
Operating system(s): Windows
Software version: 6.0
Software edition:
Reference #: 1233694
IBM Group: Software Group
Modified date: Mar 31, 2006