Sun Grid Engine Information Center
Accounting and Reporting Console (ARCo)
Index

ARCo Configuration Files and Scripts

About dbwriter

The dbwriter component writes and deletes the reporting data in the reporting database. It performs the following tasks:

• Reads raw data from the reporting file and writes this raw data to the reporting database.
• Calculates derived values. You can configure which values to calculate, as well as the rules that govern the calculations.
• Deletes outdated data. You can configure how long to keep data.

The sge_qmaster component generates the reporting file. You can configure the generation of the reporting file. See the attribute reporting_params in the man page sge_conf(5), and the attribute report_variables in the man page host_conf(5).

When dbwriter starts up, it calculates derived values. dbwriter also deletes outdated records at startup. If dbwriter runs in continuous mode, dbwriter continues to calculate derived values and to delete outdated records at hourly intervals, or at whatever interval you specify. See Derived Values and Deletion Rules.

You can specify in a XML file the values that you want to calculate and the records that you want to delete. The path to this file is specified during installation. To change the path to the file, edit the DBWRITER_CALCULATION_FILE parameter in the dbwriter.conf file

inst_dbwriter Command Options

The inst_dbwriter script, used for installing dbwriter, is located at $SGE_ROOT/dbwriter and supports the following options:

• -nosmf – Disables SMF for Solaris 10+ machines. Instead the regular RC scripts are used.
• -upd – Removes old RC scripts not containing the SGE_CLUSTER_NAME and starts the installation. This option must be used if you are upgrading from version prior to 6.2.
• -rmrc – Removes 6.2 RC scripts or SMF service.
• -h – Prints usage text to stdout.

If no option is specified, installation is started.

dbwriter Configuration Parameters

During dbwriter module installation, the following configuration parameters are collected. These parameters are stored in the $SGE_ROOT/$SGE_CELL/common/dbwriter.conf file. Changes to the dbwriter.conf file require restarting the dbwriter.

Table – dbwriter Configuration Parameters

sgedbwriter Command Options

The sgedbwriter script, used for starting and stopping dbwriter, is located at $SGE_ROOT/$SGE_CELL/common and supports the following sub-commands:

• start – Starts the dbwriter as a background process. If no options are supplied, this is the default behavior. You can supply three options with the start command:
  • -debug – Start the dbwriter in in debug mode. This allows you to attach a debugger.
  • -debug_port port-number – Specify a port to use for debugging. The default is 8000.
  • -nosmf – Force no SMF
• stop – Stops the dbwriter process.
• print setting – Prints the specified dbwriter setting to stdout. The following settings are available:
  • pid_file – Prints the default pid file
  • log_file – Print the default log file
  • spool_dir – Print the default spool directory
• -h – Prints usage text to stdout.

The content environment variable JVMARGS is treated as options for the Java virtual machine. If $JAVA_HOME is, set the Java virtual machine at $JAVA_HOME/bin/java is started.

About Reporting

The reporting file contains the following types of data:

• Host load values and consumable resources
• Queue consumable resources
• Job logging
• Job accounting
• Share-tree usage
• Advance reservation logging
• Advance reservation accounting

Enabling Generation of the Reporting File

When the Grid Engine system is first installed, the reporting file is disabled. To use ARCo, you must enable the reporting file for the cluster. Once enabled, the reporting file will be generated by sge_qmaster. By default, the reporting file is located in $SGE_ROOT/$SGE_CELL/common. The path to the file is stored in the DBWRITER_REPORTING_FILE parameter of the dbwriter.conf file.

Once the reporting file is enabled, the dbwriter can read raw data from the reporting file and write it to the reporting database.

For information about configuring the generation of the reporting file, see the reporting_params parameter of the sge_conf(5) man page, and the report_variables parameter of the host_conf(5) man page. For more information about configuring the reporting file, see the reporting(5) man page. For complete details about installing and configuring ARCo, see Installing the Accounting and Reporting Console.

How to Enable Generation of the Reporting File From the Command Line

To enable reporting from the command line, use the qconf -mconf command to set the reporting_params attributes, as described in the last step of How to Enable Generation of the Reporting File With QMON.

How to Enable Generation of the Reporting File With QMON

1. To enable reporting with QMON, on the QMON Main Control window click the Cluster Configuration button.
   
   
2. On the Cluster Configuration dialog box, select the global host, and click Modify.
   
   
3. On the Cluster Settings dialog box, click the Advanced Settings tab.
   
   
4. In the Reporting Parameters field, set the following parameters:
   • Set accounting to true. true is the default value.
   • Set reporting to true.
   • Set flush_time to 00:00:15. 00:00:15 is the default value.
   • Set joblog to true.
   • Set sharelog to 00:00:00. 00:00:00 is the default value.

Reporting Module Configuration Parameters

During reporting module installation, the following configuration parameters are collected. These parameters are stored in the $SGE_ROOT/$SGE_CELL/arco/reporting/config.xml file. Changes to the config.xml file require restarting the smcwebserver.

• The <database> element includes several attributes that configure the database connection for the application to use. This element includes two sub-elements and several attributes. Attributes include the following:
  • name
  • host
  • port
  • schema
  • clusterName
    Sub-elements include the following:
  • <driver>
  • <user>, which has three attributes:
    • name
    • passwd
    • maxConnections
• The <appUser> element identifies each user that is permitted to use the reporting feature. One appUser element is provided for each user that is permitted to use the reporting feature.

  Note You can edit the config.xml file to add additional users. Provide another appUser element for each user to add.

• The <storage> element defines the storage of ARCo queries and results. This element includes three sub-elements:
  • <root> defines the path of the spool directory
  • <queries> defines the directory where to store queries
  • <results> defines the directory where to store results

Example - Reporting Module Configuration File

The following config.xml example illustrates a single cluster configuration. For a multiple cluster configuration, there would be multiple <database> tags.

<configuration>
 <!--
   Configure the database connection to be used by the application
 -->
 <database name="arco" host="host.domain" port="5432" schema="public" clusterName="testsuite">
    <driver type="postgres">
       <javaClass>org.postgresql.Driver</javaClass>
    </driver>
    <user name="arco_read" passwd="ed5sq937d20ecf5c" maxConnections="10"/>
 </database>

 <applUser>
    admin
 </applUser>
 <applUser>
    sgetest1
 </applUser>
 <applUser>
    sgetest2
 </applUser>

 <storage>
   <root>/var/spool/arco</root>
   <queries>queries</queries>
   <results>results</results>
 </storage>
</configuration>

Other ARCo Utilities

arcorun

The arcorun utility enables you to view and run ARCo queries from the command line. You can view query output in XML (default), CSV, PDF or HTML format. You can also set values for late-binding parameters.For more information, see the arcorun(1) man page.

Note You must run the arcorun utility on a host from which the ARCo spooling directory (default: /var/spool/arco) is accessible.

Example – Running a Query

A query is run by simply invoking the arcorun command with
the name of the query as the argument:

     % $SGE_ROOT/$SGE_CELL/arco/reporting/arcorun Statistics

If a query name contains whitespaces you have to put
double-quotes around the query name:

     % $SGE_ROOT/$SGE_CELL/arco/reporting/arcorun "Host Load"

updatedb.sh

The updatedb.sh utility enables you to preview changes that will be performed on your database. You supply your existing database parameters and choose y in the following prompt:

Shall we only print all sql statements which will be executed during the upgrade? (y/n) [y] >> 

After that, the SQL commands that will be executed during update/upgrade are printed to the stdout.

It is not recommended to use this as a substitute for a regular dbwriter update/upgrade. If you would choose option n, the SQL commands would be executed and only your database definition would be updated, but you would still need to perform regular dbwriter re-installation to also update other parts of dbwriter that might have changed.