Assume FAM/OpenSSO server has been deployed using the same build mentioned below.
The opensso server url is http://myhost.red.iplanet.com:8080/opensso. 
The glassfish server instance is at http://myhost.red.iplanet.com:8090.

1. download appserver_v9_agent.zip from opensso stable build 2

2. unzip it to an install directory say /myagent

3. cd to /myagent/j2ee_agents/appserver_v9_agent/bin

4. chmod 755 agentadmin

5. stop the agent container.

6. start installation:  agentadmin --install

************************************************************************
Welcome to the Access Manager Policy Agent for Sun Java(TM) System
Application Server 8.1/8.2/9.0/9.1. If the Policy Agent is used with
Federation Manager services, User needs to enter information relevant to
Federation Manager.

************************************************************************


Enter the complete path to the directory which is used by Application Server
to store its configuration Files. This directory uniquely identifies the
Application Server instance that is secured by this Agent.
[ ? : Help, ! : Exit ]
Enter the Application Server Config Directory Path
[/var/opt/SUNWappserver/domains/domain1/config]:/space/products/glassfish/glassfish/domains/domain1/config

Enter the name of the Application Server instance that is secured by this
Agent.
[ ? : Help, < : Back, ! : Exit ]
Enter the Application Server Instance name [server]: 


Enter the URL where the Access Manager is running. Please include the
deployment URI also as shown below:
(http://myserver.company.com:8080/opensso)
[ ? : Help, < : Back, ! : Exit ]
Access Manager URL: http://myhost.red.iplanet.com:8080/opensso


Enable this field only when the agent is being installed on a remote server
instance host.
[ ? : Help, < : Back, ! : Exit ]
Is Domain administration server host remote ? [false]: 

Enter the Agent protected Application Server URL
[ ? : Help, < : Back, ! : Exit ]
Agent URL: http://myhost.red.iplanet.com:8090


Enter the deployment URI for the Agent Application. This Application is used
by the agent for internal housekeeping.
[ ? : Help, < : Back, ! : Exit ]
Enter the Deployment URI for the Agent Application [/agentapp]: 


Enter a valid Encryption Key.
[ ? : Help, < : Back, ! : Exit ]
Enter the Encryption Key [RjZAM5H5cDkhC0X5x4cHkMX7K0OtTm1L]: 


Enter a valid Agent profile name. Before proceeding with the agent
installation, please ensure that a valid Agent profile exists in Access
Manager.
[ ? : Help, < : Back, ! : Exit ]
Enter the Agent Profile name: myagent1


Enter the path to a file that contains the password to be used for identifying
the Agent.
[ ? : Help, < : Back, ! : Exit ]
Enter the path to the password file: /home/huacui/apassword


Enter true only if agent is being installed on a remote instance from the
Domain Administration server host. 
[ ? : Help, < : Back, ! : Exit ]
Is the agent being installed on the DAS host for a remote instance ? [false]: 


-----------------------------------------------
SUMMARY OF YOUR RESPONSES
-----------------------------------------------
Application Server Config Directory :
/space/products/glassfish/glassfish/domains/domain1/config 
Application Server Instance name : server 
Access Manager URL : http://myhost.red.iplanet.com:8080/opensso 
Domain Administration Server Host is remote : false 
Agent URL : http://myhost.red.iplanet.com:8090 
Deployment URI for the Agent Application : /agentapp 
Encryption Key : RjZAM5H5cDkhC0X5x4cHkMX7K0OtTm1L 
Agent Profile name : myagent1 
Agent Profile Password file name : /home/huacui/apassword 
Agent installed on the DAS host for a remote instance : false 

Verify your settings above and decide from the choices below.
1. Continue with Installation
2. Back to the last interaction
3. Start Over
4. Exit
Please make your selection [1]:

Creating a backup for file
/space/products/glassfish/glassfish/domains/domain1/config/login.conf
...DONE.

Creating a backup for file
/space/products/glassfish/glassfish/domains/domain1/config/server.policy
...DONE.

Adding Agent Realm to
/space/products/glassfish/glassfish/domains/domain1/config/login.conf
file ...DONE.

Adding java permissions to
/space/products/glassfish/glassfish/domains/domain1/config/server.policy
file ...DONE.

Creating directory layout and configuring Agent file for Agent_001
instance ...DONE.

Reading data from file /home/huacui/apassword and encrypting it ...DONE.

Generating audit log file name ...DONE.

Creating tag swapped AMAgent.properties file for instance Agent_001 ...DONE.

Creating a backup for file
/space/products/glassfish/glassfish/domains/domain1/config/domain.xml
...DONE.

Adding Agent parameters to
/space/products/glassfish/glassfish/domains/domain1/config/domain.xml
file ...DONE.


SUMMARY OF AGENT INSTALLATION
-----------------------------
Agent instance name: Agent_001
Agent Bootstrap file location:
/export/home/space/agents/glassfish/j2ee_agents/appserver_v9_agent/Agent_001/config/AMAgent.properties
Agent Configuration file location
/export/home/space/agents/glassfish/j2ee_agents/appserver_v9_agent/Agent_001/config/AMAgentConfiguration.properties
Agent Audit directory location:
/export/home/space/agents/glassfish/j2ee_agents/appserver_v9_agent/Agent_001/logs/audit
Agent Debug directory location:
/export/home/space/agents/glassfish/j2ee_agents/appserver_v9_agent/Agent_001/logs/debug


Install log file location:
/export/home/space/agents/glassfish/j2ee_agents/appserver_v9_agent/logs/audit/install.log

Thank you for using Access Manager Policy Agent


7. Agent install is done. need to create an agent profile myagent1 in FAM server.

8. build2 FAM doesn't have agent console UI support. have to use famadm create-agent CLI to create an agent profile in FAM.
cd to /myagent/j2ee_agents/appserver_v9_agent/Agent_001/config
edit the file AMAgentConfiguration.properties with the following changes:

8.1. find the property com.sun.identity.agents.config.access.denied.uri, and set its value to /agentsample/authentication/accessdenied.html. so it looks like
com.sun.identity.agents.config.access.denied.uri = /agentsample/authentication/a
ccessdenied.html

8.2. find the property com.sun.identity.agents.config.login.form[0], and set its value to /agentsample/authentication/login.html. so it looks like
com.sun.identity.agents.config.login.form[0]=/agentsample/authentication/login.html

8.3. find the property com.sun.identity.agents.config.logout.uri[], and set its value to /agentsample/logout. so it looks like
com.sun.identity.agents.config.logout.uri[agentsample] = /agentsample/logout

8.4. find the property com.sun.identity.agents.config.notenforced.uri[0], and set its value to a list of URIs. so it looks like
com.sun.identity.agents.config.notenforced.uri[0] = /agentsample/public/*
com.sun.identity.agents.config.notenforced.uri[1] = /agentsample/images/*
com.sun.identity.agents.config.notenforced.uri[2] = /agentsample/styles/*
com.sun.identity.agents.config.notenforced.uri[3] = /agentsample/index.html
com.sun.identity.agents.config.notenforced.uri[4] = /agentsample/
com.sun.identity.agents.config.notenforced.uri[5] = /agentsample

8.5. find the property com.sun.identity.agents.config.privileged.attribute.type[0] = Role, change it to com.sun.identity.agents.config.privileged.attribute.type[0] = Group.

8.6. find the property com.sun.identity.agents.config.privileged.attribute.tolowercase[Role] = false, change it to 
com.sun.identity.agents.config.privileged.attribute.tolowercase[Group] = false.

8.6. find the property com.iplanet.services.debug.level, and set
com.iplanet.services.debug.level=message

9. find the famAdminTools.zip from the same opensso bits and unzip it into a directory say /cli-b2
do: setup -f /opensso (assume /opensso is the config directory)
this will create a opensso/bin in the same dir.
cd to opensso/bin.
copy AMAgentConfiguration.properties from agent to opensso/bin.
append one line to end of AMAgentConfiguration.properties file.
userpassword=myagent1 (myagent1 is the password you gave while installing the agent)
create a text file say adminpasswd that contains the amadmin password in cleartext.
run famadm create-agent command to create an agent called myagent1 (the agent profile name you gave while installing the agent):
famadm create-agent -b myagent1 -t J2EEAgent -u amadmin -f adminpasswd -D AMAgentConfiguration.properties 

this command will create an agent profile myagent1 in the FAM server.
use "famadm show-agent -b myagent1 -u amadmin -f adminpasswd" command to verify the agent profile account. 

Now very important, due to some b2 issue, the agent is not immediately usable. RESTART THE FAM SERVER. 
Try use myagent1/myagent1 to login to console to verify the authentication of the agent profile user.

10. Now restart the agent container. 

11. deploy agentapp.war, it is located at /myagent/j2ee_agents/appserver_v9_agent/etc. this is a housekeeping app for agent. it receives notifications from FAM 8 server and passes them on to the agent.

12. the agent should now function.



How to set up the sample application

1. On the agent machine, cd /myagent/j2ee_agents/appserver_v9_agent/sampleapp
   This directory has the sample app source and depolyable files.
   if your FAM server's root suffix is "dc=opensso,dc=java,dc=net", then you don't have to change anything. Just deploy the agentsample.ear file located in dist directory. 
   If not, you need to modify sun-application.xml and sun-web.xml in etc directory by replacing the "dc=opensso,dc=java,dc=net" with your root suffix. then you need to rebuild the ear file following the instuction in the readme.txt section "Compiling and Assembling the Application". Now, deploy the agentsample.ear file located in dist directory.

2. login to FAM console with amadmin user.

3. Go to main console page, and click on Access Control tab

4. click on realm "opensso", click on Subjects tab, click on User tab. create a new user called "chris" with password "chris". click on Group tab, create groups "manager" and "employee". assign the user "chris" to both "manager" and "employee".

5. go to Policies tab. create a new policy p1. create a rule r1 with resource name http://myhost.red.iplanet.com:8090/agentsample/*, allow actions GET and POST. click on Save button to save the rule r1. then in the same policy, create a Subject s1, assign group "manager" and "employee" to the subject s1. save the subject, and most importantly save the policy p1.

6. the sample application setup is done. open up a browser and enter http://myhost.red.iplanet.com:8090/agentsample. On the left hand side frame, there are three links. 

J2EE Declarative Security  
J2EE Security API  
URL Policy Enforcement

Click on URL Policy Enforcement, on the right frame, a page shows up with a link saying "Invoke a Servlet Protected by URL Policy". click on the link, the agent will take you to the FAM login page. enter chirs/chris. the browser should show you a successful invocation page if things go well. exercise the other two links in a similar manner.