Debugging and Analyzing HPC Applications

This section explains how to use various tools to debug and analyze your HPC applications.

Using DTrace Capability for HPC Applications

About Using DTrace With Sun Studio

The NetBeans DTrace GUI plug-in is required for effectively using DTrace capability for HPC applications. This plug-in provides a Graphical User Interface (GUI) for executing DTrace scripts that can be installed in the Sun Studio IDE. You can execute DTrace scripts from the GUI, even those which are embedded in shell scripts.

To visualize the output of DTrace, use Chime. Chime is a graphical tool for visualizing DTrace aggregations. Chime is fully integrated with the NetBeans DTrace GUI plug-in.

To execute DTrace, you need to have the required user privileges. If you do not have the required user privileges, execute the following command as the root user to grant the required privileges:

# usermod -K defaultpriv=basic,dtrace_user,dtrace_proc,dtrace_kernel <userId>

Note - The user hpcuser is already provided with the required privileges.

How to Download and Install the DTrace Plug-In

To download the NetBeans DTrace GUI plug-in, follow these steps:

1. Invoke Sun Studio using the desktop icon.
2. Select Tools->Plugins.
3. From the Plugins window, select the Available Plugins tab.
4. Select the DTrace plug-in, and click the Install button at bottom left.
   The Plug-in Installer window appears.
5. Follow the instructions on the Plug-in Installer window to install the plug-in.
6. Once plug-in installation is complete, select Window->DTrace.
7. Exit from Sun Studio.
8. Execute the script /export/home/hpcuser/STUDIO_CONFIG/studioConfig.sh.
9. Invoke Sun Studio again using the desktop icon.

How to Use DTrace With an MPI Application

To use DTrace with an MPI application, follow these steps:

1. From the Sun Studio main menu, select Window > DTrace to open the DTrace plug-in.
   
   
2. Select the DTrace tab and select the Chime tab.
   
   
   
3. Select the required script.
   A sample visualization created for mpicount script is selected in the following example.
   
   
4. Execute the MPI_DTRACE project which invokes the binary mpicommleak before executing the mpicount script.
   The binary is built as shown in the previous figure.
5. Use the following command to get the process ID for mpicommleak.

   $ ps \-ef \| grep mpicomm
   root  3723  3721   0 17:06:20 pts/7      0:00  /MPI_EXAMPLES/dtrace/mpicommleak
   

   In this example, the process ID of mpicommleak is 3723.

6. Click Run to start the Chime visualization.
   You are prompted to enter the process ID.
   
   
7. Type the process ID and click OK.
   The chime visualization appears.
   The Chime visualization for mpicount.d is shown in the following figure.
   
   For more information on how to create the Chime visualization, see http://opensolaris.org/os/project/dtrace-chime/new_displays/wizard. You can also access the DTrace user guide at http://docs.sun.com/app/docs/doc/817-6223/6mlkidlh7?q=trunc&a=view.

Using D-Light Capability for HPC Applications

D-Light is another plug-in available for the Sun Studio IDE. The plug-in offers a variety of instrumentation that take advantage of the DTrace debugging and performance analysis functionality in the Solaris operating system. It helps in analyzing your application's behavior.

How to Use D-Light With an MPI Application

To use D-Light with an MPI application, follow these steps:

1. From the Sun Studio main menu, select Tools > D-Light Tool to open the D-Light Tool plug-in.
   
   
2. Drag-and-drop the required D-Light instrument to the right pane.
   The following figure shows a Clock Profiler and Heap Monitor instruments.
   
   
3. Click the D-Light toolbar button and select Run Executable.
   This allows you to select the required binary to execute.
   
   The output of the selected instruments appear as the executable starts running.
   
   
4. Click Stop to stop the instruments.
   
   For more information on how to use the D-Light tool, see http://developers.sun.com/sunstudio/documentation/tutorials/d_light_tutorial/index.html.

Using Analyzer for Performance Analysis

The Performance Analyzer is a graphical data analysis tool that analyzes the performance data and displays the data in tabular and graphical forms. It can be used to collect clock profiling metrics, hardware counter metrics, synchronization delay metrics, memory allocation metrics, MPI tracing metrics, and so on. The Performance Analyzer GUI consists of eight elements and some of the most important ones are described here.

• MPI Timeline – Graphically displays the MPI activity that occurred during an application's run. The horizontal axis shows the elapsed time and the vertical axis shows the MPI process rank. MPI_send and MPI_Recv messages can be clicked and their source can be visualized using the Show Call Stack if available button.
  
• MPI Chart – Generates scatter plots and histograms to visualize the performance data of MPI functions and MPI messages. Use the MPI chart controls to visualize the data in X, Y, 2-D charts. Use the MPI data-zooming and data-filtering controls to broaden or narrow your view of the data in the MPI Timeline and MPI Charts.
  
• Functions – Shows a list consisting of functions and their metrics like CPU usage, MPI send, MPI receives, and MPI wait times.
  
• Callers-Callees – Shows the selected function in a pane in the center, with callers of that function in a pane above, and callees of that function in a pane below. It also displays detailed metrics for the selected object in the right pane.
  
• Source – Contains the source file of the experiment.
  
• Disassembly – Shows a disassembly listing of the object file containing the selected function, annotated with performance metrics for each instruction.
  
  For more information about Performance Analyzer, see http://wikis.sun.com/display/MPIAnalyzer/MPI+Analyzer+Tutorial.