Sun Gathering Debug Data for Sun xVM Ops Center 2.1

original author: Peter Charpentier, October 11, 2007
updated by: Drew Gilbert, April 27, 2008
updated by: Lubos Kosco, May 16, 2008
updated by: Lubos Kosco, February 16, 2009

This Tech Note is intended for anyone who needs to open a Service Request for a Sun xVM Ops Center issue with the Sun Support Center.

Introduction

Sun™ xVM Ops Center is a Solaris™ and Linux operating system (OS) life cycle management tool that allows you to provision new systems, manage their updates and configuration changes, and eventually redeploy systems for new purposes. You can also inventorize your systems and on supported ones update their firmware. You can subscribe to update and provisioning channels to manage Solaris, Red Hat, and SUSE systems.

This technical note describes how to collect data that the Sun Support Center requires to debug problems with a Sun xVM Ops Center system. By collecting this data before you open a Service Request, you can substantially reduce the time needed to analyze and resolve a problem.

This technical note covers Sun xVM Ops Center on the Solaris OS and Linux platforms. The information applies to all types of environments, including test, pre-production, and production. To reduce performance impact, verbose debugging is used only when necessary. It is possible that the problem might disappear when you configure logging for debug mode. In most cases, the debug data described here is sufficient to analyze the problem.

If your problem does not conveniently fit into any of the specific categories that are presented here, supply the general information described in Collecting Basic Debug Data for Sun xVM Ops Center and clearly explain your problem.

If the information you provide is not sufficient to find the root cause of the problem, the Sun Support Center will ask for more details, as needed.

Overview of Collecting Debug Data

There are five basic steps for collecting debug data for a Sun xVM Ops Center problem. This document provides information about the first and last Step below.

1. Collect the basic problem and system information.
2. Collect the specific problem information (for example, installation logs, output, and other data).
3. Enable and disable debug modes for the various parts of the product.
4. Create a tar.gz (tar.z) file of all the information and upload it to the Sun Support Center.
5. Create a Service Request with the Sun Support Center.

Collecting Basic Debug Data for Sun xVM Ops Center

This section describes the kinds of debug data that you need to provide based on the kind of problem that you are experiencing. Note that the script will detect which OS you are running, but also which parts are installed on a specific machine, such as agent and server software.

To Enable and Restore Debugging

1. Go to the Sun Gathering Debug Data web page and find appropriate Sun GDD - Sun xVM Ops Center (alternatively visit http://kenai.com/projects/sun-gdd-xvmoc/downloads ).
2. Download the sun-gdd-xvmoc-VERSION-collect_noarch.zip file.
3. Expand the file and install the Sun-GDD-xVMOC.shscript. This gives you access to the actual script that is used to collect data and enable or restore debugging.
   Note - Run all scripts in this procedure as root.
4. Run the script with the following attribute to enable debugging for the agent.

   # ./Sun-GDD-xVMOC.sh debug 

5. Run the script with the following attribute to restore the debug level to that set prior to installing the Sun-GDD-xVMOC.shscript.

   # ./Sun-GDD-xVMOC.sh restore 

6. The agent software will automaticall restart to effect the changes.

To Collect Data Regarding the Update Channel Agent and Server

1. Ensure that the Sun-GDD-xVMOC.shscript is installed.
2. To collect the debug data, run the script as root:

   # ./Sun-GDD-xVMOC.sh collect

3. These scripts create a dump file in /var/tmp (or designated path) that should be uploaded to Sun after a Service Request is logged.
   Note - These scripts automatically detect if an agent and server are installed, and copies all the files that are found.

Additional Command-Line Options

Sample gathering for case 123456 without looking for new updates of the script

# ./Sun-GDD-xVMOC.sh -c 123456 -n collect

In this case, the log data will be collected without prompting for a location with case number 123456 and the resultant file will be located at:

• /var/tmp/123456-sun-gdd-xvmoc-`hostname`_`date "+%d-%m-%y--%H.%M.%S"`.tar.gz

Collection of xVM Ops Center domain data (jmxdump)

# ./Sun-GDD-xVMOC.sh -u ADMIN_USERNAME_OF_xVMOC -n collect

Collection will collect domain data of xVM Ops Center in textual form after the password is provided.

Disable restart of processes

When any debug or restore options are used, the script automatically restarts the relevant processes. If there is a requirement to delay restarting, this behavior can be bypassed with the -r option, for example:

# ./Sun-GDD-xVMOC.sh -r debug

The user will then be prompted to manually restart the agent processes. Note, that you need to know how to restart the appropriate component.

Automatic debug, collect and restore

When using the option auto-collect you will go through automated cycle of debug, wait for key press or certain time, collect, restore:

# ./Sun-GDD-xVMOC.sh -t 600 auto-collect

Script will enable debugging and increase logger sizes, it will wait for 600 seconds and then collect logs and restore back to normal mode

Note - The -t option is to be used only for auto-collect

Help Screen From the Sun GDD Sun xVM Ops Center Script

To get a help screen please run.

# ./Sun-GDD-xVMOC.sh -h 

Technical Script Details

The following debug information is required for the various software elements.
Note - Agent,Server are all parts of Update Channel!

Debug Information for the Agent

The following information is added to the .uce.rc file.

• A copy of the original .uce.re file to the .uce.rc.SunGDD file to enable debugging.
• The following lines are added:

  ( all ) ( log.__file.default_logger-level, "DETAILED" { "SEVERE","ERROR","WARNING","INFO","DEBUG","FINE","DETAILED" } );
  ( all ) ( log.__file.uce_agent_app-level, "DETAILED" { "SEVERE","ERROR","WARNING","INFO","DEBUG","FINE","DETAILED" } ); 

• The agent is restarted automatically.
  To restore to previous debug level, copy back the .uce.rc.SunGDD file if it exists.

Debug Information for the Server

The following information is added to the .uce.rc file.

• A copy of the original .uce.re file to the .uce.rc.SunGDD file to enable debugging.
• The following lines are added:

  ( all ) ( log.__file.uce_server_cgi-level, "DETAILED" { "SEVERE","ERROR","WARNING","INFO","DEBUG","FINE","DETAILED" } );
  ( all ) ( log.__file.uce_server_cgi_publisher-level, "DETAILED" { "SEVERE","ERROR","WARNING","INFO","DEBUG","FINE","DETAILED" } );
  ( all ) ( log.__file.uce_server_scheduler-level, "DETAILED" { "SEVERE","ERROR","WARNING","INFO","DEBUG","FINE","DETAILED" } );
  ( all ) ( log.__file.default_logger-level, "DETAILED" { "SEVERE","ERROR","WARNING","INFO","DEBUG","FINE","DETAILED" } );
  ( all ) ( invisible.debug.__group1.debug_mode, true ); 

• The server is restarted automatically.

Debug Information for Sun xVM Ops Center Components

The following debugging is enabled.

• All appropriate instances of cacao (satellite-default, proxy, agent) are fully enabled for debug logs. These instances have increased logging file size and increase count of rolling log files. Backups of original configs are kept in cacao.properties.SunGDD files.
• Graphical Sun xVM Ops Center components are also fully debug enabled and their log files increased in size. Backups of config files are kept in logging.properties.SunGDD and log4j.xml.SunGDD .
• The appropriate Sun xVM Ops Center components are restarted automatically.

Creating a Service Request With the Sun Support Center

When you create a Service Request through the Sun Support Center, either online or by phone, provide the following information:

• A clear problem description
• Details of the state of the system, both before and after the problem started
• Impact on end users
• All recent software and hardware changes
• Any actions already attempted
• Whether the problem is reproducible; when reproducible, provide the detailed test case
• Whether a preproduction or test environment is available
• Name and location of the archive file containing the debug data

Upload your debug data archive file to the following locations:

• http://supportfiles.sun.com

Note - When opening a Service Request by phone with the Dispatch Team, provide a summary of the problem, then send the details in a text file named Description.txt. Be sure to include Description.txt in the archive along with the rest of your debug data.

Reporting Problems

Use the following email aliases to report problems with this document and its associated scripts:

• To provide feedback: gdd-feedback@sun.com (alternatively users@sun-gdd-xvmoc.kenai.com)
• To report problems: gdd-issue-tracker@sun.com (alternatively issues@sun-gdd-xvmoc.kenai.com)

More Information

For more information about Sun xVM Ops Center, go to the Sun xVM Software Information Exchange.

To get an inside perspective, visit the Sun xVM blog. The goal of the blog is to share information with customers who either have already implemented or will implement these products in the future. Those blogs also provide important information around features, training, and other key activities.