This article describes how to configure Sun OpenSSO Enterprise 8.0 Update 1 in Federal Information Processing Standards (FIPS) mode. The following procedures use Sun Java System Web Server 7.0 as the OpenSSO Enterprise web container, with the NSS Certificate DB (certdb) as the key/certificate store.

Before You Begin

• After you enable FIPS mode, the bootstrap file cannot be decrypted (CR 6835816). Therefore, before you enable FIPS mode, backup the CONFIG_DIR/bootstrap file. Then, after you enable FIPS mode, replace the bootstrap file with the backup copy.

• If Web Server 7.0 has the Java security manager enabled, add the following additional permissions to the server.policy file:

  permission java.security.SecurityPermission "insertProvider.Mozilla-JSS";
  permission java.security.SecurityPermission "putProviderProperty.Mozilla-JSS";
  permission java.security.SecurityPermission "removeProvider.Mozilla-JSS";
  

Configuring the NSS Database in FIPS Mode

Configure the NSS database in FIPS mode. For example, using the modutil command:

modutil -fips true -dbdir location-of-your-nss-database

For information about modutil, see:
http://www.mozilla.org/projects/security/pki/nss/tools/modutil.html

Enabling the FIPS-140 Standard for Sun Java System Web Server 7.0

To enable the FIPS-140 Standard for Web Server 7.0, you must change the certdb password and enable FIPS mode as true. (By default, Web Server 7.0 sets the password to blank for its certdb.)

To Enable the FIPS-140 Standard for Web Server 7.0

Set the password for the internal PKCS11 token using either the Web Server 7.0 Admin Console or CLI command.

To Set the Password Using the Web Server 7.0 Admin Console

1. Log in to the Admin Console.
2. Go to the configuration page in the Admin console.
3. Click the Certificates > PKCS11 Tokens tab.
4. Click the PKCS11 token name (default is internal).
5. Select the Token State checkbox.
6. Enter the password information.
7. Save your changes.

To Set the Password Using Web Server 7.0 CLI

Execute the wadm command. For example:
wadm> set-token-pin --user=admin --password-file=admin.pwd --host=serverhost
--port=8989 --config=config1 --token=internal

To Enable FIPS mode for Web Server 7.0 With modutil

Use modutil in the WS70_ROOT/bin directory. For example:
modutil -fips true -dbdir location-of-your-nss-database

By default, the NSS database is in the config directory for the Web Server 7.0 instance.

To Pull the Changes into the Admin Server

If you use certutil or modutil to modify files in the config directory, you must pull the changes into the Web Server 7.0 Admin Server. For example, using wadm:

wadm pull-config --user=admin --password-file=admin-pwfile --host=server-host --port=8989 --config=config1 node1

To Test the FIPS Mode Change

After you enable FIPS mode for Web Server 7.0, confirm that FIPS is enabled by restarting server. You should see a new prompt for the certdb password. For example:

> Please enter the PIN for the "NSS FIPS 140-2 Certificate DB" token:

Configuring an OpenSSO Enterprise 8.0 Instance Using the Console

Before You Begin

• jss4.jar file – The WS_INSTALL_DIR/lib/jss4.jar file must be compatible with the NSS version you are using. If necessary, download a compatible jss4.jar file and copy it to the WS_INSTALL_DIR/lib directory.

• Multiple OpenSSO Enterprise 8.0 instances – If you are configuring multiple OpenSSO Enterprise 8.0 instances that are part of a site, first add and configure all instances in the site in non-FIPS mode. Then, after all instances are added and configured for the site, configure the instances in FIPS mode.

To Configure an OpenSSO Enterprise 8.0 Instance Using the Console

To enable a single OpenSSO Enterprise 8.0 instance in FIPS mode, you must first configure the instance to use the JSS-based implementation class for encryption, Secure Random, SSL sockets, and the HTTPS Protocol Handler.

1. Log in to the OpenSSO Enterprise Administration Console.
2. Click Configuration > Servers and Sites, and then the Server Name instance.
3. Click the Security tab.
4. Click the Inheritance Settings button.
5. Uncheck the Encryption class, FIPS Mode, and Secure Random Factory Class properties.
6. Click Save and then Back to Server Profile.
7. Change Encryption class to com.iplanet.services.util.JSSEncryption.
8. Change Secure Random Factory Class to com.iplanet.am.util.JSSSecureRandomFactoryImpl.
9. Check Yes for FIPS Mode.
10. Click Save and then the Advanced tab.
11. Change the com.iplanet.security.SSLSocketFactoryImpl property to com.iplanet.services.ldap.JSSSocketFactory.
12. Click Add and add following property with the value:
    java.protocol.handler.pkgs=com.iplanet.services.comm
13. Click Add and add following property with the value:
    com.iplanet.am.admin.cli.certdb.dir=path-to-FIPS-enabled-NSS-certdb
14. Click Save.
15. Restart the OpenSSO Enterprise server instance.