Configuring Secure Socket Layer in WebSphere Application Server

• "What is Secure Socket Layer?" and related concepts
• Overview: WebSphere Application Server's use of Secure Socket Layer
• Configuring Secure Socket Layer between the Web browser and the Web server:
  • Configuring Secure Socket Layer for browsers
    
  • Configuring Secure Socket Layer for Web servers
    
  • Configuring Secure Socket Layer for IBM HTTP Server, specifically
• Configuring Secure Socket Layer for between the Web server plug-in and an application server
• Configuring default Secure Socket Layer settings

Overview: WebSphere Application Server's use of Secure Socket Layer

Secure Socket Layer (SSL) provides secure communication for several WebSphere Application Server components. In particular, SSL is used between:

• The Web server WebSphere Application Server plug-in acting as an SSL client, and an application server Web container acting as an SSL server using HTTPS.
• Am EJB client acting as an SSL client, and an application server acting as an SSL server using RMI/IIOP over SSL.
• A Web component, including servlets and Java Server Page (JSP) files, acting as an SSL client, and an external SSL secured Web site acting as an SSL server using HTTPS.
• An enterprise bean in an application server acting as an SSL client, and an external SSL secured EJB server acting as an SSL server using RMI/IIOP over SSL.
• Internal administrative and application server processes using RMI/IIOP over SSL on WebSphere Application Server Advanced Edition.
• Administrative clients, including the administrative console, XMLConfig, and WSCP, acting as an SSL client, and the administrative server acting as an SSL server using RMI/IIOP over SSL on WebSphere Application Server Advanced Edition.
• Administrative and application servers acting as an SSL client, and an SSL secured Lightweight Directory Access (LDAP) server acting as an SSL server over LDAPS on WebSphere Application Server Advanced Edition.

Configuring Secure Socket Layer between the Web browser and the Web server:

Configuring Secure Socket Layer for the browser

Configuring SSL for the browser is browser-specific. Refer to the respective browser documentation for instructions. When you type "https://..." instead of "http://...", the browser creates an SSL connection instead of a simple TCP connection to the Web server. Then, the browser either prompts you or fails to connect. When the browser prompts you, it asks if you want to trust the SSL secured Web server you trying to connect to, because there is no signer certificate in the memory corresponding to the certificate used to secure the Web server. This is normal if the Web server was secured using a self signed certificate or a Certificate Authority (CA) that does not have an agreement with the Web browser. The browser fails if it is unable to validate the Web server or unable to agree upon the level of security options.

Configuring Secure Socket Layer for the Web server

Configuring SSL for the Web server depends on the type of Web server. Refer to your Web server documentation for instructions. When SSL is enabled, an SSL key file is required. The key file should contain the CA signer certificates and any personal certificates. Client authentication is disabled by default. See the InfoCenter article "5.5.5: Mapping certificates to users for client authentication and authorization" for more information.

In order for the client certificate to be forwarded by the WebSphere Web server plug-in to the WebSphere Application Server, client authentication must be enabled for the Web server. Enabling client authentication in WebSphere Application Server is not required unless you want to authenticate the WebSphere Web server plug-in or any other clients connecting directly to the WebSphere Application Server through SSL.

Also, the virtual host for each application server using the SSL port (usually 443), needs to have a port alias added for the SSL port. See the InfoCenter article "6.6.16: Administering virtual hosts".

Configuring Secure Socket Layer for IBM HTTP Server

This section provides an example of configuring SSL for IBM HTTP Server. See the IBM HTTP Server documentation for the current and complete instructions. The httpd.conf.sample file in your Web server provides examples of all directives, including the SSL-related directives.

Create a key file using the IBM HTTP Server key management utility.

a. Create a directory at a location such as "product_installation_root/myKeys"
This directory is used to hold your SSL key files and certificates.
b. Start the Key Management Utility from the IBM HTTP Server Start menu.
To start this utility on a Windows platform, click Start -> Programs -> IBM HTTP Server -> Start Key Management Utility
c. Click the Key Database File menu and select New.
d. Specify settings and click OK:
• Key Database Type: CMS Key Database File
• File Name: WebServerKeys.kdb
• Location: The path to your "myKeys" directory
e. Enter a password for your SSL key file. You are asked to enter the password two times for confirmation.
f. Check the "Stash the password to a file?" option. Click OK. The "WebServerKeys.sth" file is created and contains an encoded form of the password. Encoding prevents a casual viewing of the password, but is not highly secure. Therefore, use the operating system permissions to prevent access to the "WebServerKeys.sth" file by unauthorized persons.
g. When you see the list of default Signer Certificates, click the Signer Certificates menu and select Personal Certificates.
If you have a server certificate from a CA, for example, VeriSign, you can click Import to import this certificate into your SSL key file. Then, you are prompted for the type and location of the file containing the server certificate.
If you do not have a valid server certificate from a CA, but want to test your system, click New Self-Signed.
You are prompted to enter a Key Label such as "Test" and Organization, such as "IBM". You can use the default values.


h.Click the Key Database File menu and select Close.

1. Add the following lines to the bottom of your httpd.conf file:

   LoadModule ibm_ssl_module modules/IBMModuleSSL128.dll(for Windows)
   

2. 		
   LoadModule ibm_ssl_module libexec/mod_ibm_ssl_128.so (for Unix)
   

3. 		
   AddModule mod_ibm_ssl.c (for Unix)
   

4. 		
   Listen 443
   

5. 		
   <VirtualHost:443>
   

6. 		
   SSLEnable
   

7. 		
   </VirtualHost>
   

8.  
   Keyfile "product_installation_root/myKeys/WebServerKeys.kdb"
   
   	
   

   
   (use a backward slash (\) instead of a forward slash (/)for Windows)

To enable CA and get IBM HTTP Server to send a request to the browser for a certificate, uncomment the last line containing "SSLClientAuth required". Your browser might prompt you to choose a certificate to send to the Web server in order to perform client authentication.

1. Start your IBM HTTP Server.
2. Test your configuration from a browser by entering a URL such as:

   https://localhost
   

3. If you are using a self-signed certificate, instead of a certificate issued by a CA, such as VeriSign, your browser should prompt you to trust the unknown signer of the server certificate. If you enabled client authentication, your browser might prompt you to select a certificate to send to the Web server in order to perform client authentication. The page should then be displayed.
4. Test your configuration from a browser on a sample application:

a. Add port 443 to the WebSphere Application Server Virtual Host alias settings: Start the WebSphere Application Server and open the administrative console. In the navigation tree, click on WebSphere Administrative Domain > Virtual Hosts. The Virtual Hosts properties displays in the property view. In the property view, click the default_app virtual host. On the General tab, click Add. A blank host alias is added to the end of the host alias table. Type *:443 in the new host alias field.

Click Apply.

b. Regenerate the plug-in configuration file to make sure the *:443 host alias is added.
In the navigation tree, click WebSphere Administrative Domain > Nodes > node_name. Right-click the node_name and select Regen Webserver Plugin from the pop-up menu.

c. Start the Default Server and test HTTPS access to any Web resource using the default_app virtual host like the snoop servlet from a browser by entering a URL such as:
https://localhost/servlet/snoop

Configuring Secure Socket Layer between the Web server plug-in and an application server

Configure SSL between the Web server plug-in and an application server, by first securing the connection between the Web server and Web browser with SSL. If security is enabled, HTTPS requests for application server Web resources sent to the Web server go through the SSL connection between the Web server plug-in and the application server, but regular HTTP requests go between the Web server plug-in and application server unencrypted. This section provides an example using a self-signed certificate, which is ok if you have full control over both the Web server machine and the WebSphere Application Server machine. They should be trusted in this case.

Step 1: Creating an Secure Socket Layer key file for the application server

Use the JSSE key management utility installed with the WebSphere Application Server product, or any other key management utility which writes to the JKS (.jks) format. The following is an example of how to create an SSL key file for your application server allowing only the plug-in to connect to the application server on its SSL port only if client authentication is enabled on the application server.

1. Start the JSSE key management utility. On Windows, start the Key Management Utility from the WebSphere Application Server entry on the Start menu.
2. Create a new key database file.
   a. Click Key Database File and select New.
   b. Specify settings:
   • Key database type: JKS
   • File Name: appServerKeys.jks
   • Location: your myKeys directory, such as "product_installation_root\myKeys\
   c. Click OK.
   d. Enter a password and click OK.
3. Delete all of the signer certificates.
4. Click Signer Certificates and select Personal Certificates.
5. Add a new self-signed certificate.
   a. Click New Self-Signed to add a self-signed certificate.
   b. Specify settings:
   • Key Label: appServerTest
   • Organization: IBM
   c. Click OK.
6. Extract the certificate from this self-signed certificate so that it can be imported into the signer certificate section of the application server SSL key file and the plug-in SSL key file.
   a. Click Extract Certificate.
   b. Specify settings:
   • Data Type: Base64-encoded ASCII data
   • Certificate file name: appServerKeys.arm
   • Location: the path to your myKeys directory
   c. Click OK.
7. Import the self-signed certificate as a signer certificate. If you are using a CA instead of a self-signed certificate, a special certificate called the CA certificate must be received from the CA separately and imported as the signer certificate instead of doing as follows for a self-signed certificate:
   a. Click Personal Certificates and select Signer Certificates.
   b. Click Add.
   c. Specify settings:
   • Data Type: Base64-encoded ASCII data
   • Certificate file name: appServerKeys.arm
   • Location: the path to your myKeys directory
   d. Click OK.
8. Enter "appServer signer" for the label and click OK.
9. Click Key Database File.
10. Select Exit. Make sure to exit or close the key database file each time it is modified to ensure the changes are picked up by the plug-in.

Step 2: Creating a Secure Socket Layer key file for the Web server plug-in

You must use the key management utility of GSKit installed when any WebSphere Application Server Web server plug-in is installed (<drive>:\Program Files\ibm\gsk5\gsk5ikm.exe in Windows and /opt/ibm/gsk5/gsk5ikm or /usr/ibm/gsk5/gsk5ibm in Unix). Lack of the GSKit indicates that a plug-in was not installed or there was a problem installing the plug-in. If you are using the IBM HTTP Server as your Web server, you can use the same SSL key file the Web server is using. However, it is recommended that separate SSL key files be used because the trust policy for the connection to the Web server is different than the trust policy for the connection to the application server. For example, you might want to allow many browsers to connect to the Web server's HTTPS port, but you only want to allow a small, well-known number of plug-ins to connect directly to a WebSphere application server HTTPS port. The following example shows how to create an SSL key file for your WebSphere plug-in containing a self-signed certificate:

1. Create the directory product_installation_root\myKeys.

1. Start the key management utility of GSKit.

The default path on Windows to this utility is <drive>:\Program Files\ibm\gsk5\gsk5ikm.exe and on Unix is /opt/ibm/gsk5/gsk5ikm or /usr/ibm/gsk5/gsk5ibm

1. Click Key Database File and select New.
2. Specify settings and click OK:
   • Key database type: CMS Key Database File
   • File name: plug-inKeys.kdb
   • Location: your myKeys directory
3. Enter a password for your SSL key file.
4. Select Stash the password to a file. Click OK.

1. When you see the list of default Signer Certificates, select the first certificate and click Delete.
2. Repeat the previous step until all of the signer certificates have been deleted.
3. Click Signer Certificates and select Personal Certificates.
4. Add a new self-signed certificate. If you are using a CA instead of a self-signed certificate, a special certificate called the CA certificate must be received from the CA separately and imported as the signer certificate instead of doing as follows for a self-signed certificate:
   a. Click New Self-Signed to add a self-signed certificate.
   b. Specify settings.
   • Key Label: plug-in
   • Organization: IBM
   c. Click OK.
5. Extract the certificate so that you can import it into the application server key file later.
   a. Click Extract Certificate.
   b. Specify settings:
   • Base64-encoded ASCII data: Data Type
   • Certificate file name: plug-inKeys.arm
   • Location: path to your myKeys directory
   c. Click OK.
6. Import the application server certificate. If you are using a CA instead of a self-signed certificate, a special certificate called the CA certificate must be received from the CA separately and imported as the signer certificate instead of doing as follows for a self-signed certificate:
   a. Go to the Signer Certificates.
   b. Click Add.
   c. Specify settings:
   • Data Type: Base64-encoded ASCII data
   • Certificate file name: appServerKeys.arm
   • Location: the path to your myKeys directory
   d. Click OK.
7. Type "appServer signer" for the label and click OK.
8. Click the Key Database File menu and select Close. Make sure to exit or close the key database file each time it is modified to ensure the changes are picked up by the plug-in.

Step 3: Import the plug-in certificate into the application server Secure Socket Layer key file

1. Start the JSSE key management utility.
2. Click Key Database File and select Open.
3. Find appServerKeys.jks in your myKeys directory and double-click on it.
4. Enter a password for your SSL key file.
5. Import the plug-in certificate.
   a. Go to the Signer Certificates.
   b. Click Add.
   c. Specify settings. If you are using a CA instead of a self-signed certificate, a special certificate called the CA certificate must be received from the CA separately and imported as the signer certificate instead of doing as follows for a self-signed certificate:
   • Data Type: Base64-encoded ASCII data
   • Certificate file name: plug-inKeys.arm
   • Location: the path to your myKeys directory
   d. Click OK.
6. Enter "plug-in signer" for the label and click OK.
7. Click Key Database File.
8. Select Exit. Make sure to exit or close the key database file each time it is modified to ensure the changes are picked up by the plug-in.

Step 4: Referencing the appServerKeys.jks key file in the Web container transport settings

Reference the appropriate SSL key file in the application server SSL settings configuration panel. The Security Center default SSL settings are not changed in the following example, because doing so affects SSL settings of other WebSphere Application Server components.

1. In the administrative console navigation tree, click WebSphere Administrative Domain > Nodes > node_name > Application Servers > application_server_name.
2. In the properties frame, click the Services tab.
3. In the Service table, click Web Container Service > Edit Properties.
4. In the Web Container Service window, click the Transport tab.
5. Click Add.
6. Specify settings in the General tab.
   • Transport host: *
   • Transport port: 9443 (or another port if this is being used)
7. Select Enable SSL.
8. Specify settings in the SSL configuration.
   • Key File Name: product_installation_root\myKeys\appServerKeys.jks
   • Key File Password: enter your password
   • Key File Format: JKS
   • Trust File Name: product_installation_root/myKeys/appServerKeys.jks
   • Trust File Password: enter your password
   • Trust File Format: JKS
9. Select Enable client authentication. Enabling client authentication ensures that only the plug-in containing the matching certificate is able to authenticate to the application server.
10. Save your changes.
11. In the navigation tree, right-click WebSphere Administrative Domain > Nodes > node_name to display a pop-up menu.
12. Select Regen Webserver Plugin to incorporate the transport settings into the plugin-cfg.xml file.

Step 5: Modifying the Web server plug-in configuration file

Now that you have created the SSL key file for the plug-in, edit the plug-in configuration file so that it references your key file. If you regenerate your plug-in file, you need to redo these changes, because they are set to the defaults. Find the "product_installation_root\config\plugin-cfg.xml file. The SSL key file is specified by the keyring property, and the stash file. The stash file contains an encoded password specified by the stash file property. Make sure that the path of plugin-cfg.xml is specified in your Web server configuration, for example, in "httpd.conf" for IBM HTTP Server. Look for the following section in the plugin-cfg.xml file and modify it to point to your key and stash files, created in Step 2:

 <ServerGroup Name="node name/app server name">

     <Server CloneID="unique identifier" Name="app server name">

         <Transport Hostname="node name" Port="9080" Protocol="http"/>

         <Transport Hostname="node name" Port="9443" Protocol="https">

             <Property name="keyring" value="product_installation_root\myKeys\plug-inKeys.kdb"/>

             <Property name="stashfile" value="product_installation_root\myKeys\plug-inKeys.sth"/>

         </Transport>

     </Server>

 </ServerGroup>

Step 6: If your Web server is on a separate machine than your WebSphere Application Server, you must copy the plugin-cfg.xml file to the Web server machine and replace the existing plugin-cfg.xml on the Web server machine.

Step 7: If your Web server is running as a user who does not have read permission for the keyring (.kdb) and stash (.sth) files, you must either change the user that the Web server runs as or the permissions of the keyring and stash files so that the Web server can read those files.

Step 8: Stop and restart the servers

Stop and restart the Web server and application server to activate the configuration.

Configuring default Secure Socket Layer settings

The default SSL settings affect every SSL connection. The effect of the default SSL settings can be bypassed for the SSL connection between the plug-in and the application server, and for the SSL connection between the WebSphere Application Server and the LDAP server using their respective customizable SSL settings. The key file contains the personal certificates required to act as an SSL server in the WebSphere Application Server. The trust file contains the signer certificates required to act as an SSL client in the WebSphere Application Server. The same file can be used as both the key and trust file, but it must contain both the personal and signer certificates required for the WebSphere Application Server to act as a SSL client or server. The WebSphere Application Server comes with sample default key and trust files including DummyServerKeyFile.jks and DummyServerTrustFile.jks. To ensure your applications are secure, these files should be replaced with customized key and trust files.

The following example creates a self-signed certificate to replace the sample certificate using one file containing both the personal and signer certificates.

Step 1: Creating an Secure Socket Layer key file for the default Secure Socket Layer settings

1. Start the JSSE key management utility.
   On Windows, start the key management utility from the WebSphere Application Server entry on the Start menu.
2. Create a new key database file.
   a. Click Key Database File and select New.
   b. Specify settings:
   • Key database type: JKS
   • File Name: SSLKeys.jks
   • Location: your myKeys directory, such as "product_installation_root\myKeys
   c. Click OK.
   d. Enter a password and click OK.
3. Delete all of the signer certificates.
4. Click Signer Certificates and select Personal Certificates.
5. Add a new self-signed certificate.
   a. Click New Self-Signed to add a self-signed certificate.
   b. Specify settings.
   • Key Label: SSL Server
   • Organization: IBM
   
   c. Click OK.
6. Extract the certificate from this self-signed certificate so that it can be imported into the signer certificate section of the application server's SSL key file and other trust files as needed.
   a. Click Extract Certificate.
   b. Specify settings:
   • Data Type: Base64-encoded ASCII data
   • Certificate file name: SSLKeys.arm
   • Location: the path to your myKeys directory
   c. Click OK.
7. Import the self-signed certificate as a signer certificate. If you are using a CA instead of a self-signed certificate, a special certificate called the CA certificate must be received from the CA separately and imported as the signer certificate instead of doing the following for a self-signed certificate:
   a.Click Personal Certificates and select Signer Certificates.
   b.Click Add.
   c.Specify settings:
   • Data Type: Base64-encoded ASCII data
   • Certificate file name: SSLKeys.arm
   • Location: the path to your myKeys directory
   d.Click OK.
8. Enter "SSL Signer" for the label and click OK.
9. Click Key Database File.
10. Click Exit. Make sure to exit or close the key database file each time it is modified to ensure the changes are picked up by the WebSphere Application Server.

Step 2: Reference the key file in WebSphere Application Server systems administration

Reference the appropriate SSL key file in the default SSL settings configuration panel or in the HTTPS SSL settings configuration panel.

1. In the navigation tree click Security Center.
2. Click Default SSL Configuration.
3. Specify settings in the default SSL configuration.
   • Key File Name: "product_installation_root\myKeys\SSLKeys.jks
   • Key File Password: enter your password
   • Key File Format: JKS
   • Trust File Name: "product_installation_root\myKeys\SSLKeys.jks
   • Trust File Password: enter your password
   • Trust File Format: JKS
   • Client Authentication: selected
4. Save your changes.

Step 3: Creating a Secure Socket Layer key file for the WebSphere Application Server administrative clients and external EJB clients

1. Start the JSSE key management utility.
   On Windows, start the key management utility from the WebSphere Application Server entry on the Start menu.
2. Create a new key database file.
   a.Click Key Database File and select New.
   b.Specify settings:
   • Key database type: JKS
   • File Name: ClientKeys.jks
   • Location: your myKeys directory, such as "product_installation_root\myKeys
   c. Click OK.
   d. Enter a password and click OK.
3. Delete all of the signer certificates.
4. Import the self-signed certificate as a signer certificate.
   a. Go to the Signer Certificates section.
   b. Click Add.
   c. Specify settings. If using a CA instead of a self-signed certificate, a special certificate called the CA certificate must be received from the CA separately and imported as the signer certificate instead of doing the following for a self-signed certificate:
   • Data Type: Base64-encoded ASCII data
   • Certificate file name: SSLKeys.arm
   • Location: the path to your myKeys directory
   d.Click OK.
5. Enter "Default SSL Signer" for the label and click OK.
6. Click Key Database File.
7. Select Exit. Make sure to exit or close the key database file each time it is modified to ensure the changes are picked up by the WebSphere Application Server.

Step 4: Reference the key file in WebSphere Application Server administrative clients and external EJB clients

1. Edit the "product_installation_root\properties\sas.client.props and any other copy or derivation of the sas.client.props file using a text editor and modify the following parameters as indicated:

   com.ibm.ssl.keyStoreType=JKS 
   
   com.ibm.ssl.keyStore=product_installation_root\myKeys\ClientKeys.jks 
   
   com.ibm.ssl.keyStorePassword=enter your password 
   
   com.ibm.ssl.trustStoreType=JKS 
   
   com.ibm.ssl.trustStore=product_installation_root\myKeys\ClientKeys.jks 
   
   com.ibm.ssl.trustStorePassword=enter your password 
   

2. Save the modified file

Step 5: (Optional) Add the SSLKeys.arm signer certificate to the plug-in key file

If any application server Web container transport settings SSL configuration uses the default SSL settings, SSLKeys.arm must be added as a signer certificate to the plug-in key file.

1. Start the GSKit key management utility.
2. Open the plug-in key file.
3. Import SSLKeys.arm as a signer certificate.
   a. Access the Signer Certificates section.
   b. Click Add.
   c. Specify settings:
   • Data Type: Base64-encoded ASCII data
   • Certificate file name: SSLKeys.arm
   • Location: the path to your myKeys directory
   d. Click OK.
4. Click the Key Database File menu and select Close.

Step 6: (Optional) Add the LDAP server signer certificate to the SSLKeys.jks file

If the LDAP server is configured to connect to the WebSphere Application Server through LDAPS, the LDAP server certificate needs to be added to the signer certificate section of SSLKeys.jks.

1. Export the certificate from the LDAP server.
2. Start the JSSE key management utility.
3. Open the SSLKeys.jks file.
4. Import the LDAP server certificate as a signer certificate.
   a. Access the Signer Certificates section.
   b. Click Add.
   c. Specify settings:
   • Data Type: Base64-encoded ASCII data
   • Certificate file name: LDAP server certificate file name
   • Location: the path to your LDAP server certificate file
   d. Click OK.
5. Click the Key Database File menu and select Close. Make sure to exit or close the key database file each time it is modified to ensure the changes are picked up by the WebSphere Application Server.

Step 7: Stop and restart the servers.

Stop and restart the Web server (if the plug-in key file was modified) and the WebSphere Application Server to activate the configuration.