Outbound JSSE (SSL) calls fail in Network Deployment environment with Global Security enabled
 Technote (troubleshooting)
 
Problem(Abstract)
An application making an outbound JSSE (SSL) call fails with an "Unknown Certificate" or "Bad Certificate" error. The problem could also manifest itself as a "Certificate Expired" error if the server being contacted uses Verisign or Thawte signed certificates. In IBM® WebSphere® Studio Application Developer or a stand-alone server, this problem should not occur.
 
Cause
IBM WebSphere Application Server uses Apache SOAP internally to perform System Management operations (for example synchronizing, and starting and stopping servers) in a Network Deployment environment. When Global Security is enabled, Apache SOAP uses Secure Socket Layers (SSL) to protect the communication between nodes. To do this, the following global properties are set in the JVM:

javax.net.ssl.keystore
javax.net.ssl.truststore

Because these properties are being set for each SOAP communication (Synchronizing is done every minute by default), applications that set these properties will have their values changed. The key files defined in the SSL repertoire being used by the SOAP service would then be used for outbound JSSE communication instead of the key files specified in the application. If the application relies on the cacerts file, the same problem will occur.

 
Resolving the problem
The solution varies based on the type of SSL connection between the application and the destination server. The recommended solution is to use Socket Factories, which define their own keystore/truststore, without using global system properties. The following Developerworks article has a Socket Factory example:

http://www-106.ibm.com/developerworks/java/library/j-customssl/

If the application cannot be changed for some reason, the following two work arounds can be used based on whether or not client-authentication is required.

If the remote server requires server-authentication (Default SSL behavior):

From the Administrative Console:

  1. Select Servers -> Application Servers -> <server_name> -> Administration Services -> JMX Connectors -> SOAPConnector -> Custom Properties

    Note: <server_name> is the server where the application is deployed

  2. Note the value of the sslConfig setting (<node_name>/DefaultSSLSettings unless it has been changed)

  3. Select Security -> SSL -> <sslConfig_value>

  4. Note the Trust File Name field value

  5. Add your signer certificate that is used to authenticate to the remote server to the trust store listed in step 4. You can use the Ikeyman or keytool utility to add the certificate.

  6. Restart WebSphere and try the application

If the remote server requires client-authentication (A client certificate is used in addition to a signer certificate):

The instructions below assume that you have created a keystore/truststore that contain your client certificate and signer certificate (We will call the keystore/truststore keystore.jks and truststore.jks) and that the Dummy keys are being used by WebSphere Application Server. If the Dummy keys have been replaced, substitute the relevant key names instead:

  1. Copy the keystore.jks and truststore.jks to the Install_Root /etc directory on the Node where the failing application is deployed

  2. Extract the private key from the keystore.jks (using Ikeyman) and add it as a signer certificate in the DummyServerTrustFile.jks on the Dmgr Install_Root /etc directory

  3. Extract the private key (Jserver) from the DummyServerKeyFile.jks using Ikeyman and add it as a signer certificate in the truststore.jks in the Node's Install_Root /etc directory

  4. Start the Deployment Manager and make sure it comes up properly

  5. Start the nodeagent and make sure it comes up and synchronizes with the Deployment Manager. Check the SystemOut.log for the nodeagent for SSL handshake errors.

    From the Administrative Console:

  6. Select Security -> SSL

  7. Click the New button on the SSL Configuration Repertoires page

  8. Use the information below to set up the new SSL repertoire -> Click OK and save changes

    Alias: CustomSSLSettings
    Key File Name: ${USER_INSTALL_ROOT}/etc/keystore.jks
    Key File Password: <password>
    Trust File Name: ${USER_INSTALL_ROOT}/etc/truststore.jks
    Trust File Password: <password>

  9. Select Servers -> Application Servers -> <server_name> -> Administration Services

    <server_name> is the server where the application is deployed

  10. Select JMX Connectors -> SOAPConnector -> Custom Properties -> sslConfig

  11. Change the Value field to match the name of the new repertoire you created -> Click OK and save changes

    You must use the full name of the repertoire
    (for example bullisManager/CustomSSLSettings)


  12. Start your Application Server and try your application
 
 
Cross Reference information
Segment Product Component Platform Version Edition
Application Servers Runtimes for Java Technology Java SDK
 
 


Document Information


Product categories: Software > Application Servers > Distributed Application & Web Servers > WebSphere Application Server > Java Security (JSSE/JCE)
Operating system(s): Windows
Software version: 6.1
Software edition:
Reference #: 1191941
IBM Group: Software Group
Modified date: Jan 31, 2005