Sun Grid Engine Information Center
Index

Installing Sun Grid Engine Software

To effectively install Sun Grid Engine, perform the following tasks in the order shown:

• Printable Installing Sun Grid Engine Guide
• Planning the Installation
• Supported Operating Platforms
• Loading the Distribution Files on a Workstation
• Installing Sun Grid Engine Interactively
• Installing Security Features
• Upgrading Sun Grid Engine Software
• Verifying Sun Grid Engine Installation
• Automating the Installation Process
• Installing SMF Services
• Installing a JMX-Enabled System
• Removing Grid Engine Software
• Microsoft Services for UNIX
• User Management for Sun Grid Engine on Windows Hosts
• Other Installation Issues

In addition, you might need to perform one or more related tasks:

• To be able to gather statistics, view reports, and other accounting tasks, see Installing the Accounting and Reporting Console.

• To be able to distribute resources between different services according to configurable Service Level Agreements (SLAs), see Installing the Service Domain Manager (SDM) Module.

To print this section, see the Printable Installing Sun Grid Engine Guide.

<< Previous: Installing Sun Grid Engine

Next: Loading the Distribution Files on a Workstation >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Planning the Installation

Whether you have installed previous versions of the Sun Grid Engine software or this is your first time, you must do some planning before you extract and install the software. This section describes the decisions that you must make, and, wherever possible, gives you criteria on which you can base your decisions. This section consists of the following topics:

Decisions That You Must Make

You must make several decisions before you can plan the installation:

• Decide whether your system of networked computer hosts that run Sun Grid Engine 6.2 software is to be a single cluster or a collection of sub-clusters, called cells. Cells enable you to install separate instances of the Grid Engine software but share the binary files across those instances.
• Select the machines that are to be Grid Engine system hosts. Determine the host type of each machine: master host, shadow master host, administration host, submit host, execution host, or a combination.
• Ensure that all users of the Grid Engine system have the same user names on all submit and execution hosts.

  Note Hosts running Windows as an operating system cannot have the master host or shadow master host host types.

• Decide how to order Grid Engine software directories. For example, you could organize directories as a complete tree on each workstation, or you could cross-mount directories, or you could set up a partial directory tree on some workstations. You must also decide where to locate each Grid Engine software installation directory, sge-root.
• Decide on the site's queue structure.
• Determine whether to define network services as an NIS file or as local to each workstation in /etc/services.
• Use the information in the following section to gather the information that you need to complete the installation worksheet.

Gather the Necessary Information

Before you install the Grid Engine software, you must plan how to achieve the results that fit your environment. This section helps you make the decisions that affect the rest of the procedure. Write down your installation plan in a table similar to the following example. You can view the worksheet alone (for printing).

If you are going to install Grid Engine 6.2 on a Windows system, acquire and install Microsoft Services For UNIX. See Microsoft Services For UNIX for more information.

If you are going to install Grid Engine 6.2 on a Windows system, create the required Certificate Security Protocol (CSP) certificates before installing Grid Engine. See How to Install a CSP-Secured System for information about CSP certificates.

Check Other Grid Engine Installation Issues for applicability.

Disk Space Requirements

The Grid Engine software directory tree has the following fixed disk space requirements:

• 50 Mbytes for the installation files without any binaries
• Between 60 and 100 Mbytes for each set of binaries

The ideal disk space for Grid Engine system spool directories is as follows:

• 50-200 Mbytes for the master host spool directories
• 50-200 Mbytes for the Berkeley DB spool directories

The spool directories of the master host and of the execution hosts are configurable and need not reside under the default location, sge-root.

Note You must satisfy several Windows platform-specific prerequisites before you can install Grid Engine on hosts that are running the Windows operating system. You might need to install additional software on your computer which might require additional disk space. See Microsoft Services For UNIX.

$SGE_ROOT Directory

You must create a directory into which to load the contents of the distribution media. This directory is called the root directory, or $SGE_ROOT. When the Grid Engine system is running, this directory stores the current cluster configuration and all other data that must be spooled to disk.

Note For efficient spooling, place the spooling directories somewhere other than within $SGE_ROOT.

Use a valid path name for the directory that is network-accessible on all hosts. For example, if the file system is mounted using automounter, set $SGE_ROOT to /usr/SGE6, not to /tmp_mnt/usr/SGE6.

Note Throughout this information space, the $SGE_ROOT environment variable is used to refer to the directory into which the Sun Grid Engine software is installed.

The $SGE_ROOT directory is the top level of the Grid Engine software directory tree. On startup, each Grid Engine software component in a cell needs read access to the $SGE_ROOT/$SGE_CELL/common directory. When Grid Engine software is installed as a single cluster, the value of $SGE_CELL is default.

For ease of installation and administration, this directory should be readable on all hosts on which you intend to run the Grid Engine software installation procedure. For example, you can select a directory that is available across a network file system, such as NFS. If you choose to select file systems that are local to the hosts, you must copy the installation directory to each host before you start the installation procedure for the particular machine. See File Access Permissions for a description of required permissions.

Directory Organization

When determining the directory organization, you must decide the following:

• The directory organization, for example, whether you will install a complete software tree on each workstation, cross-mounted directories, or a partial directory tree on some workstations.
• Where to locate each $SGE_ROOT root directory.

Note Because changing the installation directory or the spool directories requires a new installation of the system, use extra care to select a suitable installation directory. Note that all important information from a previous installation can be preserved.

By default, the installation procedure installs the Grid Engine software, man pages, spool areas, and the configuration files in a directory hierarchy under the installation directory as shown in the following figure. If you accept this default behavior, you should install or select a directory with the access permissions that are described in File Access Permissions.

Figure – Sample Directory Hierarchy

You can choose to put the spool areas in other locations during the primary installation. See Configuring Queues for more detailed instructions.

Cells

You can set up the Grid Engine system as a single cluster or as a collection of loosely coupled clusters called cells. The $SGE_CELL environment variable indicates the cluster being referenced. When the Grid Engine system is installed as a single cluster, $SGE_CELL is not set, and the value default is assumed for the cell value.

Cluster Name

The $SGE_CLUSTER_NAME environment variable supports unique naming of the cluster. Unlike the $SGE_CELL variable, there are restrictions on $SGE_CLUSTER_NAME. If you decide to use Grid Engine SMF services on Solaris 10 or later hosts, you must select a new $SGE_CLUSTER_NAME. This name becomes part of the name of the Sun Grid Engine SMF services. The $SGE_CLUSTER_NAME is also used to distinguish multiple rc files for different clusters.

note If your $SGE_CELL name already reflects the desired cluster name and also satisfies $SGE_CLUSTER_NAME restrictions, set the cluster name to the $SGE_CELL value. Otherwise, the proposed default value is pSGE_QMASTER_PORT, which uniquely identifies the running cluster by the port on which its qmaster daemon is running. See SMF administration for more information.

User Names

For the Grid Engine system to verify that users submitting jobs have permission to submit them on the desired execution hosts, users' names must be identical on the submit and execution hosts. You might therefore have to change user names on some machines, because Grid Engine user names map directly to system user accounts.

Note User names on the master host are not relevant for permission checking. These user names do not have to match or even exist.

Installation Accounts

You can install the Grid Engine software either as the root user or as an unprivileged user, for example, your own user account. However, if you install the software when you are logged in as an unprivileged user, the installation allows only that user to run Grid Engine jobs. Access is denied to all other accounts. Installing the software when you are logged in as root resolves this restriction. However, root permission is required for the complete installation procedure. Also, if you install as an unprivileged user, you are not allowed to use the qrsh, qtcsh, or qmake commands, nor can you run tightly integrated parallel jobs.

Note To use SMF on Solaris 10 or later hosts and control the Grid Engine software services as an unprivileged user, perform the following additional steps as root user (or user with appropriate permissions): For a local user: Create the solaris.smf.manage.sge authorization echo "solaris.smf.manage.sge:::Manage Grid Engine Services::" >> /etc/security/auth_attr Create the new role sge_smf: roleadd -c "Grid Engine SMF Administrator" -g <group> -d <home_dir> -u <UID> \ -s <profile_shell> -A "solaris.smf.manage.sge" "sge_smf" Assign the just-created role sge_smf to the user: usermod -R "sge_smf" <normal_user> For a distributed name service, such as NIS, NIS+, or LDAP: Create the solaris.smf.manage.sge authorization on your name server echo "solaris.smf.manage.sge:::Manage Grid Engine Services::" >> /etc/security/auth_attr Create the Grid Engine SMF Management profile on your name server echo "Grid Engine SMF Management:::Can manage Grid Engine SMF services:auths=solaris.smf.manage.sge;" >> /etc/security/prof_attr Create the new role sge_smf and assign it to the user: /usr/sadm/bin/smrole add -D <domain_name> - -n "sge_smf" -a <normal_user> \ -d <home_dir> -c "Grid Engine SMF Administrator" -p "Grid Engine SMF Management" After this is done a non-root user <normal_user> will be able to manage Grid Engine SMF services via svcadm command or the startup scripts.

File Access Permissions

If you install the software logged in as root, you might have a problem configuring root read/write access for all hosts on a shared file system. Therefore, you might have problems putting the $SGE_ROOT files onto a network-wide file system.

You can force Grid Engine software to run all Grid Engine system components through a non-root administrative user account, for example sgeadmin. With this setup, this particular user needs only read/write access to the shared $SGE_ROOT file system.

The installation procedure asks whether files should be created and owned by an administrative user account. If you answer "Yes" and provide a valid user name, files are created by this user. Otherwise, the user name under which you run the installation procedure is used. Create an administrative user, and answer "Yes" to this question.

Make sure in all cases that the account used for file handling on all hosts has read/write access to the $SGE_ROOT directory. Also, the installation procedure assumes that the host from which you access the Grid Engine software distribution media can write to the $SGE_ROOT directory.

Note The name of the root user on Windows hosts depends on the system language of the Windows operating system. You can even change the name of the root user. The default name for many languages is the name Administrator. If your Windows host is a member of a Windows domain, only the local Administrator is the root user. Neither the members of the Administrators group, nor the domain Administrator, nor a member of the Domain Admins group are the root user. See User Management for Sun Grid Engine on Windows Hosts for more information about users on Windows hosts.

Network Services

Determine whether your site's network services are defined in an NIS database or in an /etc/services file that is local to each workstation. If your site uses NIS, determine the host name of your NIS server so that you can add entries to the NIS services map.

The Grid Engine system services are sge_execd and sge_qmaster. To add the services to your NIS map, choose reserved, unused port numbers. The following examples show sge_qmaster and sge_execd entries.

sge_qmaster 6444/tcp
sge_execd 6445/tcp

Master Host

The master host controls the Grid Engine system. This host runs the master daemon sge_qmaster.

The master host must comply with the following requirements:

• The host must be a stable platform.
• The host must not be excessively busy with other processing.
• At least 60 to 120 Mbytes of unused main memory must be available to run the Grid Engine system daemons. For very large clusters that include many hundreds or thousands of hosts and tens of thousands of jobs in the system at any time, 1 GByte or more of unused main memory might be required and 2 CPUs might be beneficial.
• The master host must be installed before shadow master execution, administration, or submit hosts.
• (Optional) The Grid Engine software directory, $SGE_ROOT, should be installed locally to cut down on network traffic.

  Note Windows hosts cannot act as master hosts.

Shadow Master Hosts

These hosts back up the functionality of sge_qmaster in case the master host or the master daemon fails. To be a shadow master host, a machine must have the following characteristics:

• It must run sge_shadowd.
• It must share sge_qmaster status, job information, and queue configuration information that is logged to disk. In particular, the shadow master hosts need read/write root or administration user access to the sge_qmaster spool directory and to the $SGE_ROOT/$SGE_CELL/common directory.
• The $SGE_ROOT/$SGE_CELL/common/shadow_masters file must contain a line defining the host as a shadow master host.

  Note If no cell name is specified during installation, the value of $SGE_CELL is default.

The shadow master host facility is activated for a host as soon as these conditions are met. You do not need to restart the Grid Engine system daemons to make a host into a shadow master host.

Note Windows hosts cannot act as shadow master hosts.

Spool Directories under the Root Directory

During the installation of the master host, you must specify the location of a spooling directory. This directory is used to spool jobs from execution hosts that do not have a local spooling directory.

Note If you are using a Windows execution host, you must use the local spooling directory.

• On the master host, spool directories are maintained under qmaster-spool-dir. The location of qmaster-spool-dir is defined during the master host installation process. The default value of qmaster-spool-dir is $SGE_ROOT/$SGE_CELL/spool/qmaster.
• On each execution host, a spool directory called execd-spool-dir is defined during the execution host installation processes. The default value of execd-spool-dir is $SGE_ROOT/$SGE_CELL/spool/execution-host-name. You will get better performance from execution hosts with local spooling directories than from execution hosts that have NFS mounted the master host's spooling directory.

  Note If no cell name is specified during installation, the value of $SGE_CELL is default.

You do not need to export these directories to other machines. However, exporting the entire $SGE_ROOT tree and making it write-accessible for the master host and all executable hosts makes administration easier.

Note If you use a Lustre fileshare as the spool directory, you should disable file striping for these directories. Refer to the Lustre Operations Manual for information on how to disable file striping.

Choosing Between Classic Spooling and Database Spooling

During the installation, you are given the option to choose between classic spooling and Berkeley DB spooling. If you choose Berkeley DB spooling, you are then given the option to spool to a local directory or to a separate host, known as a Berkeley DB spooling server.

Using a Berkeley DB spooling server might provide better performance than classic spooling. Part of this performance increase is because the master host can make non-blocking writes to the database, but has to make blocking writes to the text file used by classic spooling. Also consider file format and data integrity. Writing to the Berkeley DB provides a greater level of data integrity than writing to a text file. However, a text file stores data in a format that you can read and edit. Normally, you do not need to read these files, but the spooling directory contains the messages from the system daemons, which can be useful for debugging.

Database Server and Spooling Host

The master host can store its configuration and state to a Berkeley DB spooling database. The spooling database can be installed on the master server or on a separate host. When the Berkeley DB spools into a local directory on the master host, the performance is better. If you want to set up a shadow master host, you need to use a separate Berkeley DB spooling server (host). In this case, you have to choose a host with a configured RPC service. The master host connects through RPC to the Berkeley DB.

Note This configuration does not provide a High-Availability (HA) solution. For example, scripts of pending jobs are not spooled through BDB spool server and thus are not available for a shadow master.

With the introduction of NFS4 software available with the Solaris ^TM 10 operating system, you can use Berkeley DB spooling on a network file system. You could not use Berkeley DB spooling on previous NFS versions. This circumstance allows a shadow host installation spooled on Berkeley DB without setting up an additional Berkeley DB Spooling Server.

Caution Although using a shadow master host is more reliable, using a separate Berkeley DB spooling host results in a potential security hole. RPC communication as used by the Berkeley DB can be easily compromised. Only use this alternative if your site is secure and if users can be trusted to access the Berkeley DB spooling host by means of TCP/IP communication.

If you choose to use Berkeley DB spooling without a shadow master, you do not need to set up a separate spooling server. Likewise, if you choose not to use Berkeley DB spooling, you can set up a shadow master host without setting up a separate spooling server.

Once you determine whether you need a separate spooling server, you will also need to determine the location for the spooling directory. The spooling directory must be local to the spooling server. A default value for the location of the spooling directory is recommended during installation, but this default value is not suitable when the file server is different from the master host.

The requirements for the Berkeley DB spooling host are similar to the requirements for the master host:

• The host must be a stable platform.
• The host must not be excessively busy with other processing.
• At least 60 to 120 Mbytes of unused main memory must be available to run the Grid Engine system daemons. For very large clusters that include many hundreds or thousands of hosts and tens of thousands of jobs in the system at any time, one GByte or more of unused main memory might be required and two CPUs might be beneficial.
• (Optional) A separate spooling host must be installed before the master host.
• (Optional) The $SGE_ROOT directory should be installed locally, to cut down on network traffic.

Execution Hosts

Execution hosts run the jobs that users submit to the Grid Engine system. An execution host must first be set up as an administration host. You run an installation script on each execution host. For more information, see How to Install Execution Hosts.

Group IDs

You need to provide a range of IDs that will be assigned dynamically for jobs. The range must be big enough to provide enough numbers for the maximum number of Grid Engine jobs running at a single moment on a single host.

A group ID is assigned to each Grid Engine job to monitor the resource utilization of the job. Each job will be assigned a unique ID while it is running. For example, a range of 20000-20100 allows 100 jobs to run concurrently on a single host. You can change the group ID range for your cluster configuration at any time, but the values in the UNIX group ID range must be unused on your system.

Administration Hosts

Operators and managers of the Grid Engine system use administration hosts to perform administrative tasks such as reconfiguring queues or adding Grid Engine users.

The master host installation script automatically makes the master host an administration host. During the master host installation process, you can add other administration hosts. You can also manually add administration hosts on the master host at any time after installation.

Submit Hosts

Jobs can be submitted and controlled from submit hosts. The master host installation script automatically makes the master host a submit host.

Cluster Queues

The installation procedure creates a default cluster queue structure, which is suitable for getting acquainted with the system. The default queue can be removed after installation.

Note No matter what directory is used for the installation of the software, the administrator can change most settings that were created by the installation procedure. This change can be made while the system is running.

Consider the following when determining a queue structure:

• Whether you need cluster queues for sequential, interactive, parallel, and other job types
• Which queue instances to put on which execution hosts
• How many job slots are needed in each queue

For more detailed information on administering cluster queues, see Configuring Queues.

Scheduler Profiles

You can choose from three scheduler profiles during the installation process: normal, high, and max. You can use these predefined profiles as a starting point for Grid Engine tuning.

Using these profiles, you can optimize the scheduler for one or more of the following:

• The amount of information that is tracked about a scheduling run
• The load adjustment during a scheduling run
• Interval scheduling (the default) or immediate scheduling

You can choose from three scheduler profiles:

• normal – This profile uses load adaptation and interval scheduling, and reports all the information that the scheduler gathers during the dispatch cycle. This profile is the starting point for most grids. Use this profile if your highest priority is gathering and reporting information about a scheduling run.
• high – This profile is more appropriate for a large cluster, where throughput is more important than gathering and reporting all the information from the scheduler. This profile also uses interval scheduling. Use this profile if you want to get better performance at the cost of getting less information about your scheduling runs.
• max – This profile disables all information gathering and reporting, enables immediate scheduling, and disables load adaptation. Immediate scheduling is very useful for sites with high throughput and very short running jobs. The advantage of immediate scheduling decreases as runtime of the jobs increases. This profile can be used in clusters of any size where only throughput is important and everything else is a lower priority.

For more information on how to configure scheduling, see Administering the Scheduler.

Installation Method

Several methods are available for installing the Grid Engine software:

• Interactive
• Interactive, with increased security
• Automated, using the inst_sge script and a configuration file
• Upgrade

To decide which installation method you should use, consider the following factors.

• Do you already have the Grid Engine software installed and running?
  • If so, you will probably want to upgrade. The upgrade process is described in Upgrading From a Previous Version of Sun Grid Engine Software.
  • If not, the master host installation is only done once. The master host is typically installed interactively, as described in Installing the Sun Grid Engine Software Interactively.
• Do you need to install just a few execution hosts?
  If so, then you will probably want to install them interactively, as described in Installing the Sun Grid Engine Software Interactively.
• Do you need to install a large number of execution hosts?
  If so, then you might want to perform automated installation, using the inst_sge script and a configuration file. See Using the inst_sge Utility and a Configuration Template.
• Do you require your grid to use encryption?
  If so, you have to perform an interactive installation with increased security. See Installing the Increased Security Features.

Check the Other Installation Issues Appendix

If you are installing Grid Engine on a Linux system or on a system with IPMP, see Other Grid Engine Installation Issues for important information.

<< Previous: Installing Sun Grid Engine

Next: Loading the Distribution Files on a Workstation >>

Index

Installation Planning Checklist

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Supported Operating Platforms for Sun Grid Engine

The Sun Grid Engine 6.2 software supports the following operating systems and platforms:

• Solaris 10, 9, and 8 Operating Systems (SPARC Platform Edition)
• Solaris 10 and 9 Operating Systems (x86 Platform Edition)
• Solaris 10 Operating System (x64 Platform Edition)
• Apple Mac OS X 10.4 (Tiger), PPC platform
• Apple Mac OS X 10.4 (Tiger), x86 platform
• Apple Mac OS X 10.5 (Leopard), x86 platform
• Hewlett Packard HP-UX 11.00 or higher, 32 bit
• Hewlett Packard HP-UX 11.00 or higher, 64 bit (including HP-UX on IA64)
• IBM AIX 5.1, 5.3
• Linux x86, kernel 2.4, 2.6, glibc >= 2.3.2
• Linux x64, kernel 2.4, 2.6, glibc >= 2.3.2
• Linux IA64, kernel 2.4, 2.6, glibc >= 2.3.2
• Microsoft Windows Server 2003, Windows XP Professional with at least Service Pack 1, Windows 2000 Server with at least Service Pack 3, or Windows 2000 Professional with at least Service Pack 3

<< Previous: Planning the Installation

Next: Installing Sun Grid Engine Software Interactively >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Loading the Distribution Files on a Workstation

Software Distribution

The Sun Grid Engine 6.2 software is distributed on CD-ROM and through electronic download. For information on how to access CD-ROMs, ask your system administrator or refer to your local system documentation. The CD-ROM distribution contains a directory named Sun_Grid_Engine_6_2. The product distribution is in this directory, in both tar.gz format and the pkgadd format. The pkgadd format is provided for the Solaris Operating System (Solaris OS). For all supported operating systems, the software is distributed in tar.gz format.

How to Load the Distribution Files on a Workstation

Before You Begin

Ensure that the file systems and directories that are to contain the Grid Engine software distribution and the spool and configuration files are set up properly by setting the access permissions as defined in File Access Permissions.

Steps

1. Provide access to the distribution media.
   If you downloaded the software, rather than getting it on CD-ROM, just unzip the files into a directory. This directory must be located on a file system that has at least 350 MBytes free space.
   
   
2. Log in to a system.
   Log in preferably on a system that has a direct connection to a file server.
   
   
3. Create the installation directory.
   Create an installation directory as described in $SGE_ROOT Installation Directory.

   # mkdir /opt/sge6-2
   

   In these instructions, the installation directory is abbreviated as sge-root.
   
   

4. Install the binaries for all binary architectures that are to be used by any of your master, execution, and submit hosts in your Grid Engine system cluster.
   You can use either the pkgadd method or the tar method.

pkgadd Method

The pkgadd format is provided for the Solaris Operating System. To facilitate remote installation, the pkgadd directories are also provided in zip files.

You can install the following packages:

As you type the following commands, you must be prepared to respond to script questions about your base directory, sge-root, and the administrative user. The script requests the choices that you made during the planning steps of this installation. See Decisions That You Must Make for further details.

At the command prompt, type the following commands, responding to the script questions.

# cd cdrom_mount_point/Sun_Grid_Engine_6_2
# pkgadd -d ./Common/Packages SUNWsgeec

Depending on the Solaris binary that you need, type one of the following commands:

# pkgadd -d ./Solaris_sparc/Packages SUNWsgee 
# pkgadd -d ./Solaris_sparc/Packages SUNWsgeex 
# pkgadd -d ./Solaris_x86/Packages SUNWsgeei 
# pkgadd -d ./Solaris_x64/Packages SUNWsgeeax 

tar Method

For all supported operating systems, the software is distributed in tar.gz format.

The following table contains files that you need to install, regardless of platform.

The tar files that contain platform-specific binaries use the naming convention of sge-6_2-bin-architecture.tar.gz.

The following table lists the platform-specific binaries. Install the file for each platform that you need to support. Note that each platform has its own directory under Sun_Grid_Engine_6_2.

Type the following commands at the command prompt. In the example, <basedir> is the abbreviation for the full directory, cdrom-mount-point/Sun_Grid_Engine_6_2.

% su 
# cd <sge-root>
# gzip -dc <basedir>/Common/tar/sge-6_2-common.tar.gz | tar xvpf -
# gzip -dc <basedir>/Solaris_sparc/tar/sge-6_2-bin-solsparc32.tar.gz | tar xvpf -
# gzip -dc <basedir>/Solaris_sparc/tar/sge-6_2-bin-solsparc64.tar.gz | tar xvpf -
# SGE_ROOT=<sge-root>; export SGE_ROOT
# util/setfileperm.sh $SGE_ROOT

<< Previous: Planning the Installation

Next: Installing Sun Grid Engine Software Interactively >>

<< Previous: Loading the Distribution Files on a Workstation

Next: Upgrading Sun Grid Engine Software >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Installing the Sun Grid Engine Software Interactively

This section covers the interactive installation of the Sun Grid Engine software and consists of the following topics:

Note The instructions in this section assume that you are installing the software on a computer running the Solaris TM Operating System. Any difference in functionality created by other operating system architecture that the Grid Engine software runs on is documented in files starting with the string arc_depend in the $SGE_ROOT/doc directory. The remainder of the file name indicates the operating system architectures to which the comments in the files apply, as in the arc_depend_irix.asc file. Also note that there are several prerequisites that you must satisfy for Windows systems before you can install Grid Engine. See Microsoft Services For UNIX for further details. This section does not cover the upgrade process or the installation of the Accounting and Reporting Module, ARCo. For information about upgrading, see Upgrading From a Previous Release of Grid Engine Software. For information about installing ARCo, see Installing the Accounting and Reporting Console.

Interactive Installation Overview

Note The instructions in this section are for a new Grid Engine system only. For instructions on how to install a new system with additional security protection, see Installing the Increased Security Features. For instructions on how to upgrade an existing installation of an earlier version of the Grid Engine software, see Upgrading From a Previous Release of Grid Engine Software.

Full installation includes the following tasks:

• Running an installation script on the master host and on every execution host in the Grid Engine system
• Registering information about administration hosts and submit hosts

Performing an Installation

The following sections describe how to install all the components of the Grid Engine system, including the master, execution, administration, and submit hosts. If you need to install the system with enhanced security, see Installing the Increased Security Features before you continue installation. For more information about installing Grid Engine SMF services see Installing the SMF Services before you start the installation.

• How to Install the Master Host
• How to Install Execution Hosts
• Registering Administration Hosts
• Registering Submit Hosts
• How to Install the Berkeley DB Spooling Server

<< Previous: Loading the Distribution Files on a Workstation

Next: Upgrading Sun Grid Engine Software >>

<< Previous: Installing Sun Grid Engine Interactively

Next: How to Install Execution Hosts >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

• Example Master Host Installation

How to Install the Master Host

The master host installation procedure creates the appropriate directory hierarchy that the master daemon requires and starts the Grid Engine master daemon sge_qmaster on the master host. The master host is also registered as a host with administrative and submit permission. The installation procedure creates a default configuration for the system on which it is run. The installation script queries the system for the type of operating system. The script then makes meaningful settings based on this information.

If, at any time during the installation, you think something went wrong, you can quit the installation procedure and restart it.

Before You Begin

• Extract the Grid Engine software, as described in Loading the Distribution Files on a Workstation.
• If you have decided to use an administrative user, as described in User Names, you should create that user before installing the master host.

  Note Windows hosts cannot act as master hosts.

Steps

1. Log in to the master host as root.
   
   
2. If the $SGE_ROOT environment variable is not set, set it by typing:

   # SGE_ROOT=<path_to_installation_directory (the directory MUST contain all SGE files such as SGE binaries)>; export SGE_ROOT
   

   To confirm that you have set the $SGE_ROOT environment variable, type:

   # echo $SGE_ROOT
   

3. Go to the installation directory.
   • If the directory where the installation files reside is visible from the master host, change directory (cd) to the installation directory sge-root, and then proceed to the next step.
   • If the directory is not visible and cannot be made visible, do the following:
     • Create a local installation directory, sge-root, on the master host.
     • Copy the installation files to the local installation directory sge-root across the network, for example, by using ftp or rcp.
     • Change directory (cd) to the local sge-root directory.
       
       
4. Type the install_qmaster command, adding the -csp flag if you are installing using the Certificate Security Protocol method described in Installing the Increased Security Features.
   This command starts the master host installation procedure. You are asked several questions, and you might be required to run some administrative actions.
   For a complete installation example, see Example Master Host Installation.

   # ./install_qmaster
   Welcome to the Grid Engine installation
   ---------------------------------------
   
   Grid Engine qmaster host installation
   -------------------------------------
   
   .
   .
   .
   
   The qmaster installation procedure will take approximately 5-10 minutes.
   
   Hit <RETURN> to continue >> 
   

5. Choose an administrative account owner.
   See Step 5 in the Example Master Host Installation.
   
   
6. Verify the $SGE_ROOT directory setting.
   In the example shown Step 6 of the Example Master Host Installation, the value of $SGE_ROOT is /opt/sge62.
   
   
7. Set up the TCP/IP services for the Grid Engine software.
   See Step 7 in the Example Master Host Installation.
   If TCP/IP services have not been configured, you will be notified. To configure TCP/IP services:
   
   1. Start a new terminal session or window to add the information /etc/services file or your NIS maps.
      
   2. Add the correct ports to the /etc/services file or your NIS services map, as described in Network Services.
      The following example adds entries for both sge_qmaster and sge_execd to your /etc/services file.

      ...
      sge_qmaster     6444/tcp
      sge_execd       6445/tcp
      

   3. Save your changes and return to the window where the installation script is running.
      
      
8. Type the name of your cell or accept the default cell name.
   See Step 8 in the Example Master Host Installation.
   The use of Grid Engine system cells is described in Cells.
   • If you have decided to use cells, type the cell name now.
   • If you have decided not to use cells, press the Return key.
     
     
9. Set up a unique cluster name.
   See Step 9 in the Example Master Host Installation.
   For more information, see Cluster Name.
   • To accept the default cluster name, press the Return key.
   • To enter a new cluster name, type the cluster name and press the Return key.
     
     
10. Specify a spool directory.
    See Step 10 in the Example Master Host Installation.
    For guidelines on disk space requirements for the spool directory, see Disk Space Requirements. For information on where spool directory is installed, see Spool Directories Under the Root Directory.
    • To accept the default spool directory, press the Return key.
    • If you want to use a different spool directory, then answer y to the prompt and provide a complete path name to the directory.
      
      
11. Specify whether you plan to use Windows-based execution hosts.
    See Step 11 in the Example Master Host Installation.
    • If you do not plan to use Windows support, answer No.
    • If you want Windows support, answer Yes. You will be asked some Windows-specific questions later in the installation process. These questions will be marked as WINDOWS-ONLY.
      
      
12. Verify or set the correct file permissions.
    See Step 12 in the Example Master Host Installation.
    • If you used pkgadd or you know that the file permissions are correct, answer y to accept the current permissions.
    • Answer n if you need to verify or change the file permissions.
    • WINDOWS ONLY – If you specified that you wanted Windows Execution Host support in the previous question, you should let the script set the file permissions for you.
      
      
13. Specify whether all Grid Engine hosts for this cluster are located in a single DNS domain.
    See Step 13 in the Example Master Host Installation.
    • If all of your Grid Engine system hosts are located in a single DNS domain, then answer y. Grid Engine will not care if domain information is supplied. hostA and hostA.foo.com are equivalent.
    • If all of your Grid Engine system hosts are not located in a single DNS domain, then answer n. You will be asked to configure a default domain to use in case a host is specified without domain information.
      
      
14. Watch while Grid Engine creates directories according to the information that you provided so far.
    See Step 14 in the Example Master Host Installation.
    
    
15. Specify whether you want to use classic spooling or Berkeley DB.
    See Step 15 in the Example Master Host Installation. By default, Grid Engine uses Berkeley Database spooling.
    For more information on how to determine the type of spooling mechanism you want, please see Choosing Between Classic Spooling and Database Spooling.
    
    • If you choose Berkeley DB spooling, you are asked to choose whether to use a local directory or a Berkeley DB Spooling Server.

      Tip To use a shadow master host for increased availability of the database, use the Berkeley DB Spooling Server.

      • To use a Berkeley DB spooling server, enter y. To install the Berkeley DB Spooling Server:
        
        1. Start a new terminal session or window and install the software, as described in How to Install the Berkeley DB Spooling Server.
           
        2. After you have installed the software on the spooling server, return to the master installation window, and press the Return key.
           
        3. Type the name of the spooling server.
           In Step 15 of the Example Master Host Installation, vector is the host name of the spooling server.
           
        4. Type the name of the spooling directory.
           In Step 15 of the Example Master Host Installation, /opt/sge62/default/spool/spooldb is the spooling directory.
           
      • If you do not want to use a Berkeley DB spooling server, type n. You are asked to provide the complete path to the database directory. If the directory does not exist, it is created.
        
    • To specify classic spooling, type classic.
      
      
16. Type a range of IDs that will be assigned dynamically for jobs.
    See Step 16 in the Example Master Host Installation.
    For more information, see Group IDs.
    
    
17. Verify the spooling directory for the execution daemon.
    See Step 17 in the Example Master Host Installation.
    The Grid Engine administrator must have access to create and write into this directory. For information on spooling, see Spool Directories Under the Root Directory.
    
    
18. Type the email address of the user who should receive problem reports.
    See Step 18 in the Example Master Host Installation. In the example, the user who will receive problem reports is me@my.domain.
    
    
19. Verify the configuration parameters.
    See Step 19 in the Example Master Host Installation.
    • If configuration parameters are correct, Grid Engine proceeds to create the local configuration.
    • If configuration parameters are not correct, type y to change them.
      
      
20. Specify whether you want the daemons to start when the system is booted.
    See Step 20 in the Example Master Host Installation.
    
    
21. WINDOWS-ONLY – If you specified that you want Windows support, you are asked to create Certificate Security Protocol (CSP) certificates.
    Even if the system is not running in CSP mode, it is necessary to create certain CSP certificates for Windows support. These certificates are automatically generated during the master host installation. For instructions on how to transfer these certificates to the Windows execution hosts, see Step 6 of How to Install a CSP-Secured System.
    
    
22. WINDOWS-ONLY – Add the Windows Administrator name to the Grid Engine manager list.
    
    
23. Identify the hosts that you will later install as execution hosts.
    See Step 23 in the Example Master Host Installation.

    Tip You can list hosts individually, separated by a blank space, or you can supply a file that contains host names.

    Note You can use the master host for executing jobs. To do so, you must carry out the execution host installation for the master machine. However, if you use a very slow machine as master host, or if your cluster is significantly large, do not use the master host as an execution host.

24. Select a scheduler profile.
    See Step 24 in the Example Master Host Installation.
    For information on how to determine which profile you should use, see Scheduler Profiles.
    Once you answer this question, the installation process is complete. Several screens of information will be displayed before the script exits.
    
    
25. WINDOWS-ONLY – Copy the certificate files to the Windows execution hosts.
    You can use a script to perform this function.

    Tip To use this functionality without being asked for a password, the root user should use rsh or ssh to access the execution hosts.

26. Create the environment variables ($SGE_ROOT and $SGE_CELL) for use with the Grid Engine software.
    See Step 26 in the Example Master Host Installation.

    Note If no cell name was specified during installation, the value of cell is default.

    • If you are using a C shell, type the following command:

      % source $SGE_ROOT/$SGE_CELL/common/settings.csh
      

    • If you are using a Bourne shell or Korn shell, type the following command:

      $ . $SGE_ROOT/$SGE_CELL/common/settings.sh
      

See Also For details about how you can verify that the execution host has been set up correctly, see How to Verify That the Daemons Are Running on the Master Host.

<< Previous: Installing Sun Grid Engine Interactively

Next: How to Install Execution Hosts >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Example Master Host Installation

The following example shows a complete Sun Grid Engine master host installation. Remember that this is only one step in the entire Sun Grid Engine installation process. The steps in this example coordinate with the master host installation description at How to Install the Master Host.

Steps 1-4

001   % su -
002   # cd sge-install-dir
003   # ./install_qmaster
004   Grid Engine License is displayed. 
005
006   Do you agree with that license? (y/n) [n] >>
007
008   Welcome to the Grid Engine installation
009   ---------------------------------------
010   
011   Grid Engine qmaster host installation
012   -------------------------------------
013   
014   Before you continue with the installation please read these hints:
015   
016      - Your terminal window should have a size of at least
017        80x24 characters
018   
019      - The INTR character is often bound to the key Ctrl-C.
020        The term >Ctrl-C< is used during the installation if you
021        have the possibility to abort the installation
022   
023   The qmaster installation procedure will take approximately 5-10 minutes.
024   
025   Hit <RETURN> to continue >> 
026   

Step 5

027   Grid Engine admin user account
028   ------------------------------
029
030   The current directory
031
032   /opt/sge62
033
034   is owned by user
035
036   myusername
037
038   If user >root< does not have write permissions in this directory on *all*
039   of the machines where Grid Engine will be installed (NFS partitions not
040   exported for user >root< with read/write permissions) it is recommended to
041   install Grid Engine that all spool files will be created under the user id
042   of user >myusername<.
043
044   IMPORTANT NOTE: The daemons still have to be started by user >root<.
045
046   Do you want to install Grid Engine as admin user >myusername< (y/n) [y] >> 
047
048   Installing Grid Engine as admin user >myusername<
049
050   Hit <RETURN> to continue >> 
051   Choosing Grid Engine admin user account
052   ---------------------------------------
053   
054   You may install Grid Engine that all files are created with the user id of an
055   unprivileged user.
056   
057   This will make it possible to install and run Grid Engine in directories
058   where user >root< has no permissions to create and write files and directories.
059   
060      - Grid Engine still has to be started by user >root<
061   
062      - This directory should be owned by the Grid Engine administrator
063   
064   Do you want to install Grid Engine
065   under an user id other than >root< (y/n) [y] >> y
066   
067   Choosing a Grid Engine admin user name
068   --------------------------------------
069   
070   Please enter a valid user name >> sgeadmin
071   
072   Installing Grid Engine as admin user >sgeadmin<
073   
074   Hit <RETURN> to continue >>
075

Step 6

  
076   Checking $SGE_ROOT directory
077   ----------------------------
078   
079   The Grid Engine root directory is:
080   
081      $SGE_ROOT = /opt/sge62
082   
083   If this directory is not correct (e.g. it may contain an automounter
084   prefix) enter the correct path to this directory or hit <RETURN>
085   to use default [/opt/sge62] >> 
086   
087   Your $SGE_ROOT directory: /opt/sge62
088   
089   Hit <RETURN> to continue >> 
090  

Step 7
Two actions – one for qmaster, one for execd

091   Grid Engine TCP/IP communication service
092   ----------------------------------------
093
094   The port for sge_qmaster is currently set by the shell environment.
095
096      SGE_QMASTER_PORT = 10500
097
098   Now you have the possibility to set/change the communication ports by using the
099   >shell environment< or you may configure it via a network service, configured
100   in local >/etc/services<, >NIS< or >NIS+<, adding an entry in the form
101
102      sge_qmaster <port_number>/tcp
103
104   to your services database and make sure to use an unused port number.
105
106   How do you want to configure the Grid Engine communication ports?
107
108    Using the >shell environment<:                           [1]
109
110    Using a network service like >/etc/services<, >NIS/NIS+<: [2]
111
112   (default: 1) >> 1 
113
114    Using the environment variable
115
116      $SGE_QMASTER_PORT=10500
117
118   as port for communication.
119
120   Hit <RETURN> to continue >> 
121
122   Grid Engine TCP/IP communication service
123   ----------------------------------------
124
125   The port for sge_execd is currently set by the shell environment.
126
127      SGE_EXECD_PORT = 10501
128
129   Now you have the possibility to set/change the communication ports by using the
130   >shell environment< or you may configure it via a network service, configured
131   in local >/etc/services<, >NIS< or >NIS+<, adding an entry in the form
132
133      sge_execd <port_number>/tcp
134
135   to your services database and make sure to use an unused port number.
136
137   How do you want to configure the Grid Engine communication ports?
138
139   Using the >shell environment<:                           [1]
140
141   Using a network service like >/etc/services<, >NIS/NIS+<: [2]
142
143   (default: 1) >> 1
144
145   Using the environment variable
146
147      $SGE_EXECD_PORT=10501
148
149   as port for communication.
150
151   Hit <RETURN> to continue >> 

Step 8

 
152   Grid Engine cells
153   -----------------
154   
155   Grid Engine supports multiple cells.
156  
157   If you are not planning to run multiple Grid Engine clusters or if you don't
158   know yet what is a Grid Engine cell it is safe to keep the default cell name
159   
160      default
161   
162   If you want to install multiple cells you can enter a cell name now.
163
164   The environment variable
165   
166      $SGE_CELL=<your_cell_name>
167   
168   will be set for all further Grid Engine commands.
169   
170   Enter cell name [default] >> 
171   
172   Using cell >default<. 
173   Hit <RETURN> to continue >> 
174  

Step 9

175   Unique cluster name
176   -------------------
177
178   The cluster name uniquely identifies a specific Sun Grid Engine cluster.
179   The cluster name must be unique throughout your organization. The name 
180   is not related to the SGE cell.
181 
182   The cluster name must start with a letter ([A-Za-z]), followed by letters, 
183   digits ([0-9]), dashes (-) or underscores (_).
184
185   Enter new cluster name or hit <RETURN>
186   to use default [p10500] >> 
187
188   Your $SGE_CLUSTER_NAME: p10500
189
190   Hit <RETURN> to continue >> 

Step 10

 
191   Grid Engine qmaster spool directory
192   -----------------------------------
193   
194   The qmaster spool directory is the place where the qmaster daemon stores
195   the configuration and the state of the queuing system.
196   
197   The admin user >myusername< must have read/write access
198   to the qmaster spool directory.
199   
200   If you will install shadow master hosts or if you want to be able to start
201   the qmaster daemon on other hosts (see the corresponding section in the
202   Grid Engine Installation and Administration Manual for details) the account
203   on the shadow master hosts also needs read/write access to this directory.
204   
205   The following directory
206   
207   [/opt/sge62/default/spool/qmaster]
208   
209   will be used as qmaster spool directory by default!
210   
211   Do you want to select another qmaster spool directory (y/n) [n] >> 
212  

Step 11

 
213   Windows Execution Host Support
214   ------------------------------
215                                                                                   
216   Are you going to install Windows Execution Hosts? (y/n) [n]
217  

Step 12

 
218   Verifying and setting file permissions
219   --------------------------------------
220   
221   Did you install this version with >pkgadd< or did you already
222   verify and set the file permissions of your distribution (y/n) [y] >> 
223
224   Verifying and setting file permissions
225   --------------------------------------
226
227   We may now verify and set the file permissions of your Grid Engine
228   distribution.
229
230   This may be useful since due to unpacking and copying of your distribution
231   your files may be unaccessible to other users.
232
233   We will set the permissions of directories and binaries to
234
235      755 - that means executable are accessible for the world
236
237   and for ordinary files to
238
239      644 - that means readable for the world
240
241   Do you want to verify and set your file permissions (y/n) [y] >>
242
243   Verifying and setting file permissions and owner in >3rd_party<
244   Verifying and setting file permissions and owner in >bin<
245   Verifying and setting file permissions and owner in >ckpt<
246   Verifying and setting file permissions and owner in >examples<
247   Verifying and setting file permissions and owner in >inst_sge<
248   Verifying and setting file permissions and owner in >install_execd<
249   Verifying and setting file permissions and owner in >install_qmaster<
250   Verifying and setting file permissions and owner in >lib<
251   Verifying and setting file permissions and owner in >mpi<
252   Verifying and setting file permissions and owner in >pvm<
253   Verifying and setting file permissions and owner in >qmon<
254   Verifying and setting file permissions and owner in >util<
255   Verifying and setting file permissions and owner in >utilbin<
256   Verifying and setting file permissions and owner in >catman<
257   Verifying and setting file permissions and owner in >doc<
258   Verifying and setting file permissions and owner in >include<
259   Verifying and setting file permissions and owner in >man<
260
261   Your file permissions were set
262
263   Hit <RETURN> to continue >>
264   

Step 13

265   Select default Grid Engine hostname resolving method
266   ----------------------------------------------------
267   
268   Are all hosts of your cluster in one DNS domain? If this is
269   the case the hostnames
270   
271      >hostA< and >hostA.foo.com<
272   
273   would be treated as equal, because the DNS domain name >foo.com<
274   is ignored when comparing hostnames.
275   
276   Are all hosts of your cluster in a single DNS domain (y/n) [y] >>   
277   
278   Ignoring domainname when comparing hostnames.
279   
280   Hit <RETURN> to continue >> 
281   

Step 14

 
282   Making directories
283   ------------------
284   
285   creating directory: /opt/sge62/default/spool/qmaster
286   creating directory: /opt/sge62/default/spool/qmaster/job_scripts
287   Hit <RETURN> to continue >> 
288  

Step 15

 
289   Setup spooling
290   --------------
291   Your SGE binaries are compiled to link the spooling libraries
292   during runtime (dynamically). So you can choose between Berkeley DB 
293   spooling and Classic spooling method. 
294   Please choose a spooling method (berkeleydb|classic) [berkeleydb] >> 
295   
296   The Berkeley DB spooling method provides two configurations!
297   
298   1) Local spooling:
299   The Berkeley DB spools into a local directory on this host (qmaster host)
300   This setup is faster, but you can't setup a shadow master host
301   
302   2) Berkeley DB Spooling Server:
303   If you want to setup a shadow master host, you need to use
304   Berkeley DB Spooling Server!
305   In this case you have to choose a host with a configured RPC service.
306   The qmaster host connects via RPC to the Berkeley DB. This setup is more
307   failsafe, but results in a clear potential security hole. RPC communication
308   (as used by Berkeley DB) can be easily compromised. Please only use this
309   alternative if your site is secure or if you are not concerned about
310   security. Check the installation guide for further advice on how to achieve
311   failsafety without compromising security.
312   
313   Do you want to use a Berkeley DB Spooling Server? (y/n) [n] >> y
314   
315   Berkeley DB Setup
316   
317   -----------------
318   Please, log in to your Berkeley DB spooling host and execute "inst_sge -db"
319   Please do not continue, before the Berkeley DB installation with
320   "inst_sge -db" is completed, continue with <RETURN>
321   
322   Berkeley Database spooling parameters
323   -------------------------------------
324   
325   Please enter the name of your Berkeley DB Spooling Server! >> vector
326   
327
328   Do you want to use a Berkeley DB Spooling Server? (y/n) [n] >> 
329
330   Hit <RETURN> to continue >> 
331
332   Berkeley Database spooling parameters
333   -------------------------------------
334
335   Please enter the Database Directory now, even if you want to spool locally,
336   it is necessary to enter this Database Directory. 
337
338   Default: [/opt/sge62/default/spool/spooldb] >> /tmp/dom/spooldb
339
340   Dumping bootstrapping information
341   Initializing spooling database
342
343   Hit <RETURN> to continue >> 

Step 16

 
344   Grid Engine group id range
345   --------------------------
346   
347   When jobs are started under the control of Grid Engine an additional group id
348   is set on platforms which do not support jobs. This is done to provide maximum
349   control for Grid Engine jobs.
350   
351   This additional UNIX group id range must be unused group id's in your system.
352   Each job will be assigned a unique id during the time it is running.
353   Therefore you need to provide a range of id's which will be assigned
354   dynamically for jobs.
355   
356   The range must be big enough to provide enough numbers for the maximum number
357   of Grid Engine jobs running at a single moment on a single host. E.g. a range
358   like >20000-20100< means, that Grid Engine will use the group ids from
359   20000-20100 and provides a range for 100 Grid Engine jobs at the same time
360   on a single host.
361   
362   You can change at any time the group id range in your cluster configuration.
363   
364   Please enter a range >> 20000-20100
365   
366   Using >20000-20100< as gid range. Hit <RETURN> to continue >> 
367  

Step 17

 
368   Grid Engine cluster configuration
369   ---------------------------------
370   
371   Please give the basic configuration parameters of your Grid Engine
372   installation:
373   
374      <execd_spool_dir>
375   
376   The pathname of the spool directory of the execution hosts. User >myusername<
377   must have the right to create this directory and to write into it.
378   
379   Default: [/opt/sge62/default/spool] >>  
380  

Step 18

 
381   Grid Engine cluster configuration (continued)
382   ---------------------------------------------
383   <administator_mail>
384   
385   The email address of the administrator to whom problem reports are sent.
386   
387   It is recommended to configure this parameter. You may use >none<
388   if you do not wish to receive administrator mail.
389   
390   Please enter an email address in the form >user@foo.com<.
391   
392   Default: [none] >> me@my.domain
393  

Step 19

 
394   The following parameters for the cluster configuration were configured:
395   
396      execd_spool_dir        /opt/sge62/default/spool
397      administrator_mail     me@my.domain
398   
399   Do you want to change the configuration parameters (y/n) [n] >> n
400   
401   Creating local configuration
402   ----------------------------
403   Creating >act_qmaster< file
404   Adding default complex attributes
405   Adding SGE default usersets
406   Adding >sge_aliases< path aliases file
407   Adding >qtask< qtcsh sample default request file
408   Adding >sge_request< default submit options file
409   Creating >sgemaster< script
410   Creating >sgeexecd< script
411   Creating settings files for >.profile/.cshrc<
412   
413   Hit <RETURN> to continue >> 
414  

Step 20

415   qmaster startup script
416   ----------------------
417
418   Do you want to start qmaster automatically at machine boot?
419   NOTE: If you select "n" SMF will be not used at all! (y/n) [y] >> 
420
421
422   Hit <RETURN> to continue >> 
423 
424   Grid Engine qmaster startup
425   ---------------------------
426
427   Starting qmaster daemon. Please wait ...
428   Hit <RETURN> to continue >> 

Step 23

 
429   Adding Grid Engine hosts
430   ------------------------
431   
432   Please now add the list of hosts, where you will later install your execution
433   daemons. These hosts will be also added as valid submit hosts.
434   
435   Please enter a blank separated list of your execution hosts. You may
436   press <RETURN> if the line is getting too long. Once you are finished
437   simply press <RETURN> without entering a name.
438
439   You also may prepare a file with the hostnames of the machines where you plan
440   to install Grid Engine. This may be convenient if you are installing Grid
441   Engine on many hosts.
442   
443   Do you want to use a file which contains the list of hosts (y/n) [n] >> n
444   
445   Adding admin and submit hosts
446   -----------------------------
447   
448   Please enter a blank seperated list of hosts.
449   
450   Stop by entering <RETURN>. You may repeat this step until you are
451   entering an empty list. You will see messages from Grid Engine
452   when the hosts are added.
453   
454   Host(s): host1 host2 host3 host4
455   
456   host1 added to administrative host list
457   host1 added to submit host list
458   host2 added to administrative host list
459   host2 added to submit host list
460   host3 added to administrative host list
461   host3 added to submit host list
462   host4 added to administrative host list
463   host4 added to submit host list
464   Hit <RETURN> to continue >> 
465
466   Adding admin and submit hosts
467   -----------------------------
468
469   Please enter a blank seperated list of hosts.
470
471   Stop by entering <RETURN>. You may repeat this step until you are
472   entering an empty list. You will see messages from Grid Engine
473   when the hosts are added.
474
475   Host(s): 
476   Finished adding hosts. Hit <RETURN> to continue >> 
477
478   If you want to use a shadow host, it is recommended to add this host
479   to the list of administrative hosts.
480
481   If you are not sure, it is also possible to add or remove hosts after the
482   installation with <qconf -ah hostname> for adding and <qconf -dh hostname>
483   for removing this host
484
485   Attention: This is not the shadow host installation
486   procedure.
487   You still have to install the shadow host separately
488
489   Do you want to add your shadow host(s) now? (y/n) [y] >> 
490
491   Adding Grid Engine shadow hosts
492   -------------------------------
493
494   Please now add the list of hosts, where you will later install your shadow
495   daemon.
496
497   Please enter a blank separated list of your execution hosts. You may
498   press <RETURN> if the line is getting too long. Once you are finished
499   simply press <RETURN> without entering a name.
500
501   You also may prepare a file with the hostnames of the machines where you plan
502   to install Grid Engine. This may be convenient if you are installing Grid
503   Engine on many hosts.
504 
505   Do you want to use a file which contains the list of hosts (y/n) [n] >> 
506
507   Adding admin hosts
508   ------------------
509
510   Please enter a blank seperated list of hosts.
511
512   Stop by entering <RETURN>. You may repeat this step until you are
513   entering an empty list. You will see messages from Grid Engine
514   when the hosts are added.
515
516   Host(s): es-ergb01-01
517   adminhost "es-ergb01-01" already exists
518   Hit <RETURN> to continue >> 
519
520   Please enter a blank seperated list of hosts.
521
522   Stop by entering <RETURN>. You may repeat this step until you are
523   entering an empty list. You will see messages from Grid Engine
524   when the hosts are added.
525
526   Host(s): 
527   Finished adding hosts. Hit <RETURN> to continue >> 
528  
529   Creating the default <all.q> queue and <allhosts> hostgroup
530   -----------------------------------------------------------
531   
532   root@myhost added "@allhosts" to host group list
533   root@myhost added "all.q" to cluster queue list
534   
535   Hit <RETURN> to continue >> 
536  
537   No CSP system installed!
538   No CSP system installed!

Step 24

 
539   Scheduler Tuning
540   ----------------
541   The details on the different options are described in the manual. 
542
543   Configurations
544    --------------
545   1) Normal
546       Fixed interval scheduling, report scheduling information,
547       actual + assumed load
548
549   2) High
550       Fixed interval scheduling, report limited scheduling information,
551       actual load
552
553   3) Max
554       Immediate Scheduling, report no scheduling information,
555       actual load
556
557   Enter the number of your preferred configuration and hit <RETURN>! 
558   Default configuration is [1] >> 
559
560
561   We're configuring the scheduler with >Normal< settings!
562   Do you agree? (y/n) [y] >> 
563
564   changed scheduler configuration

Step 26

565   Using Grid Engine
566   -----------------
567
568   You should now enter the command:
569
570      source /scratch2/myusername/sge62/default/common/settings.csh
571
572   if you are a csh/tcsh user or
573
574      # . /scratch2/myusername/sge62/default/common/settings.sh
575
576   if you are a sh/ksh user.
577
578   This will set or expand the following environment variables:
579
580      - $SGE_ROOT         (always necessary)
581      - $SGE_CELL         (if you are using a cell other than >default<)
582      - $SGE_CLUSTER_NAME (always necessary)
583      - $SGE_QMASTER_PORT (if you haven't added the service >sge_qmaster<)
584      - $SGE_EXECD_PORT   (if you haven't added the service >sge_execd<)
585      - $PATH/$path       (to find the Grid Engine binaries)
586      - $MANPATH          (to access the manual pages)
587
588   Hit <RETURN> to see where Grid Engine logs messages >> 
589
590   Grid Engine messages
591   --------------------
592
593   Grid Engine messages can be found at:
594
595      Startup messages can be found in SMF service log files.
596      You can get the name of the log file by calling svcs -l <SERVICE_NAME> 
597      E.g.: svcs -l svc:/application/sge/qmaster:p10500
598
599   After startup the daemons log their messages in their spool directories.
600
601      Qmaster:     /scratch2/myusername/sge62/default/spool/qmaster/messages
602      Exec daemon: <execd_spool_dir>/<hostname>/messages
603
604
605   Grid Engine startup scripts
606   ---------------------------
607
608   Grid Engine startup scripts can be found at:
609
610      /scratch2/myusername/sge62/default/common/sgemaster (qmaster)
611      /scratch2/myusername/sge62/default/common/sgeexecd (execd)
612
613   Do you want to see previous screen about using Grid Engine again (y/n) [n] >> 
614
615   Your Grid Engine qmaster installation is now completed
616   ------------------------------------------------------
617
618   Please now login to all hosts where you want to run an execution daemon
619   and start the execution host installation procedure.
620
621   If you want to run an execution daemon on this host, please do not forget
622   to make the execution host installation in this host as well.
623
624   All execution hosts must be administrative hosts during the installation.
625   All hosts which you added to the list of administrative hosts during this
626   installation procedure can now be installed.
627
628   You may verify your administrative hosts with the command
629
630      # qconf -sh
631
632   and you may add new administrative hosts with the command
633
634      # qconf -ah <hostname>
635
636   Please hit <RETURN> >> 
637
638   sge_qmaster successfully installed!

<< Previous: How to Install the Master Host

Next: Registering Administration Hosts >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

• Example Execution Host Installation

How to Install Execution Hosts

The execution host installation procedure creates the appropriate directory hierarchy required by sge_execd, and starts the sge_execd daemon on the execution host. This section describes how to install execution hosts interactively from the command line. You can automate the installation of execution of multiple hosts by using the procedure described in Automating the Installation Process.

Before You Begin

Before installing an execution host, you first need to install the master server as described in How to Install the Master Host and share the common directory.

Windows-Only You must satisfy several prerequisites before you can install Grid Engine execution hosts with Windows operating systems. You might have to install additional software on your computer. See Microsoft Services for UNIX. See the steps described in How to Install a CSP-Secured System – Steps 6a, 6b and 6c. After the installation, each user has to register their Windows password with Grid Engine using the sgepasswd client application. See User Management for Sun Grid Engine on Windows Hosts for more information.

Steps

1. Log in to the execution host as root.
   
   
2. As you did for the master installation, either copy the installation files to a local installation directory sge-root or use a network installation directory.
   
   
3. If the $SGE_ROOT environment variable is not set, set it by typing:

   # SGE_ROOT=<path_to_install/unpacked_directory>; export SGE_ROOT
   

   To confirm that you have set the $SGE_ROOT environment variable, type:

   # echo $SGE_ROOT
   

   
   

4. Change directory (cd) to the installation directory, sge-root.
   
   
5. Verify that the execution host has been declared on the administration host.
   • If you do not see the name of this execution host in the output of the qconf -sh command, you will need to declare it as an administration host.
     • Start a new terminal session or window.
     • In that window, log into the master host.
     • Declare the execution host as an administration host, using the qconf command.

       # qconf -ah quark
       quark added to administrative host list
       

     • Log back out of the master host, and continue with the installation of the execution host.
       
       
6. Type the install_execd command, adding the -csp flag if you are installing using the Certificate Security Protocol method described in Installing the Increased Security Features.
   This command starts the execution host installation procedure.
   For a complete installation example, see Example Execution Host Installation.

   # ./install_execd
   Welcome to the Grid Engine execution host installation
   ------------------------------------------------------
   
   .
   .
   .
   
   The execution host installation will take approximately 5 minutes.
   
   Hit <RETURN> to continue >> 
   

   
   

7. Verify the $SGE_ROOT directory setting.
   In the example shown in lines 27 through 41 of the Example Execution Host Installation, the value of $SGE_ROOT is /scratch2/myusername/sge62.
   
   
8. Type the name of your cell or accept the default cell name.
   See lines 042 through 076 of the Example Execution Host Installation.
   The use of Grid Engine system cells is described in Cells.
   • If you have decided to use cells, then type the cell names now.
   • If you have decided not to use cells, then press the Return key.
     
     
9. The install script checks to see what ports have been defined for the execution daemon.
   See lines 077 through 085 of the Example Execution Host Installation.
   If no ports have been defined, you will be asked to define them.
   
   
10. The install script checks to see whether the admin user already exists.
    If the admin user already exists, the script continues uninterrupted. If the admin user does not exist, the script shows the following screen where you must supply a password for the admin user. After the admin user is created, press the Return key.

    Local Admin User
    ----------------
    
    The local admin user sgeadmin, does not exist!
    The script tries to create the admin user.
    Please enter a password for your admin user >>
    

    Creating admin user sgeadmin, now ...
    
    Admin user created, hit <ENTER> to continue!
    

    
    

11. Verify the execution host has been declared as an administration host.
    See lines 086 through 092 of the Example Execution Host Installation.
    
    
12. Specify whether you want to use a local spool directory.
    See lines 093 through 122 of the Example Execution Host Installation.
    For information on spooling, see Spool Directories Under the Root Directory.
    • If you do not want a local spool directory, answer n.
    • If you do want a local spool directory, answer y.
      In the example, /tmp/dom/execs is used as the local spool directory on domain.com. Choose any directory that meets the disk space requirements described in Disk Space Requirements.
      
      
13. Specify whether you want execd to start automatically at boot time.
    See lines 123 through 131 of the Example Execution Host Installation.
    You might not want to install the startup script if you are installing a test cluster or you would rather start the daemon manually on reboot.
    
    
14. WINDOWS ONLY – Choose whether to display the GUI for Windows jobs.
    See lines 132 through 163 of the Example Execution Host Installation.
    A Grid Engine Helper Service is included with the Sun Grid Engine distribution. This service enables Windows jobs to display a GUI on the visible desktop of the execution host. The visible desktop is either the desktop of the user currently logged in on the execution host or the desktop of the next user who will log in. It is not the log in screen.
    The Helper Service is a independent component loosely coupled with the execution daemon. The startup of the Helper Service is plugged in the Services dialog box in the Windows control panel. You can install only one Helper Service per host. There can be only one execution daemon installed per Helper Server.
    The installation script asks during the installation of a execution host whether you want to see the GUI of Windows jobs.
    
    
15. Specify a queue for this host.
    See lines 164 through 183 of the Example Execution Host Installation.
    Once you answer this question, the installation process is complete. Several screens of information will be displayed before the script exits.
    
    
16. Create the environment variables ($SGE_ROOT and $SGE_CELL) for use with the Grid Engine Software.
    See lines 184 through 234 of the Example Execution Host Installation.

    Note If no cell name was specified during installation, the value of cell is default.

    • If you are using a C shell, type the following command:

      % source $SGE_ROOT/$SGE_CELL/common/settings.csh
      

    • If you are using a Bourne shell or Korn shell, type the following
      command:

      $ . $SGE_ROOT/$SGE_CELL/common/settings.sh
      

See Also For details about how you can verify that the execution host has been set up correctly, see How to Verify that the Daemons are Running on the Execution Hosts.

<< Previous: How to Install the Master Host

Next: Registering Administration Hosts >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Example Execution Host Installation

The following example shows a complete Sun Grid Engine execution host installation. Before you install the execution host, you need to first install the master server as described in How to Install the Master Host. The line numbers in this example are referred to from the execution host installation description at How to Install Execution Hosts.

Steps 1-6

001   % su -
002   # qstat -f
003   # ./install_execd
004
005   Welcome to the Grid Engine execution host installation
006   ------------------------------------------------------
007
008   If you haven't installed the Grid Engine qmaster host yet, you must execute
009   this step (with >install_qmaster<) prior the execution host installation.
010
011   For a sucessful installation you need a running Grid Engine qmaster. It is
012   also necessary that this host is an administrative host.
013
014   You can verify your current list of administrative hosts with
015   the command:
016
017      # qconf -sh
018
019   You can add an administrative host with the command:
020
021      # qconf -ah <hostname>
022
023   The execution host installation will take approximately 5 minutes.
024
025   Hit <RETURN> to continue >> 
026

Step 7

027   Checking $SGE_ROOT directory
028   ----------------------------
029
030   The Grid Engine root directory is:
031
032      $SGE_ROOT = /scratch2/myusername/sge62
033
034   If this directory is not correct (e.g. it may contain an automounter
035   prefix) enter the correct path to this directory or hit <RETURN>
036   to use default [/scratch2/myusername/sge62] >> 
037
038   Your $SGE_ROOT directory: /scratch2/myusername/sge62
039
040   Hit <RETURN> to continue >> 
041

Step 8

042   Grid Engine cells
043   -----------------
044
045   Please enter cell name which you used for the qmaster
046   installation or press <RETURN> to use [default] >> 
047
048   Using cell: >default<
049
050   Hit <RETURN> to continue >> 
051
052   ... set owner of /var/sgeCA/port10500 to bofur+myusername
053
054   ... copy /var/sgeCA/port10500/default/userkeys/root to 
055   /var/sgeCA/port10500/default/userkeys/bofur+Administrator
056   cp: /var/sgeCA/port10500/default/userkeys/root: No such file or directory
057
058   ... copy /var/sgeCA/port10500/default/userkeys/root to 
059   /var/sgeCA/port10500/default/userkeys/Administrator
060   cp: /var/sgeCA/port10500/default/userkeys/root: No such file or directory
061
062   ... copy /var/sgeCA/port10500/default/userkeys/myusername to 
063   /var/sgeCA/port10500/default/userkeys/bofur+myusername
064
065   ... set owner of /var/sgeCA/port10500/default/userkeys/Administrator to Administrator
066
067   ... set owner of /var/sgeCA/port10500/default/userkeys/bofur+Administrator to bofur+Administrator
068
069   ... set owner of /var/sgeCA/port10500/default/userkeys/myusername to myusername
070
071   ... set owner of /var/sgeCA/port10500/default/userkeys/bofur+myusername to bofur+myusername
072
073   ... remove old /var/sgeCA/port10500/default/userkeys/root certificates
074
075   WINDOWS certificates are copied and permissions are set!
076

Step 9

077   Grid Engine TCP/IP communication service
078   ----------------------------------------
079
080   The port for sge_execd is currently set BOTH as service and by the
081   shell environment
082
083      SGE_EXECD_PORT = 10501
084      sge_execd service set to port 725
085

Step 10
If the admin user already exists, the script automatically skips this step. See How to Install Execution Hosts for more information.

Step 11

086   Checking hostname resolving
087   ---------------------------
088
089   This hostname is known at qmaster as an administrative host.
090
091   Hit <RETURN> to continue >>
092

Step 12

093   Local execd spool directory configuration
094   -----------------------------------------
095
096   During the qmaster installation you've already entered a global
097   execd spool directory. This is used, if no local spool directory is configured.
098
099   Now you can configure a local spool directory for this host.
100   ATTENTION: The local spool directory doesn't have to be located on a local
101   drive. It is specific to the <local> host and can be located on network drives,
102   too. But for performance reasons, spooling to a local drive is recommended.
103
104   FOR WINDOWS USER: On Windows systems the local spool directory MUST be set
105   to a local harddisk directory.
106   Installing an execd without local spool directory makes the host unuseable.
107   Local spooling on local harddisk is mandatory for Windows systems.
108
109   Do you want to configure a local spool directory
110   for this host (y/n) [n] >> y
111
112   Please enter the local spool directory now! >> /tmp/dom/execs
113   Using local execd spool directory [/tmp/dom/execs]
114   Hit <RETURN> to continue >> 
115
116   Creating local configuration
117   ----------------------------
118   myusername@domain.com modified "domain.com" in configuration list
119   Local configuration for host >domain.com< created.
120
121   Hit <RETURN> to continue >> 
122

Step 13

123   execd startup script
124   --------------------
125
126   We can install the startup script that will
127   start execd at machine boot (y/n) [y] >> n
128
129
130   Hit <RETURN> to continue >> 
131

Step 14

132   Windows Helper Service Installation
133   ---------------------------------------
134
135   If you're going to run Windows job's using GUI support, you have
136   to install the Windows Helper Service
137   Do you want to install the Windows Helper Service? (y/n) [n] >> y
138
139   Testing, if a service is already installed!
140
141      ... a service is already installed!
142      ... stopping service!
143      ... uninstalling old service!
144   Service successfully uninstalled.
145
146
147      ... moving new service binary!
148      ... installing new service!
149   Service successfully installed.
150
151
152      ... starting new service!
153
154   Hit <RETURN> to continue >> 
155
156   Grid Engine execution daemon startup
157   ------------------------------------
158
159   Starting execution daemon. Please wait ...
160      starting sge_execd
161
162   Hit <RETURN> to continue >> 
163

Step 15

164   Adding a queue for this host
165   ----------------------------
166
167   We can now add a queue instance for this host:
168
169      - it is added to the >allhosts< hostgroup
170      - the queue provides 1 slot(s) for jobs in all queues
171        referencing the >allhosts< hostgroup
172
173   You do not need to add this host now, but before running jobs on this host
174   it must be added to at least one queue.
175
176   Do you want to add a default queue instance for this host (y/n) [y] >> 
177
178   No modification because "bofur" already exists in "hostlist" of "hostgroup"
179   root@domain.com modified "@allhosts" in host group list
180   root@domain.com modified "all.q" in cluster queue list
181
182   Hit <RETURN> to continue >> 
183

Step 16

184   Using Grid Engine
185   -----------------
186
187   You should now enter the command:
188
189      source /scratch2/myusername/sge62/default/common/settings.csh
190
191   if you are a csh/tcsh user or
192
193      # . /scratch2/myusername/sge62/default/common/settings.sh
194
195   if you are a sh/ksh user.
196
197   This will set or expand the following environment variables:
198
199      - $SGE_ROOT         (always necessary)
200      - $SGE_CELL         (if you are using a cell other than >default<)
201      - $SGE_CLUSTER_NAME (always necessary)
202      - $SGE_QMASTER_PORT (if you haven't added the service >sge_qmaster<)
203      - $SGE_EXECD_PORT   (if you haven't added the service >sge_execd<)
204      - $PATH/$path       (to find the Grid Engine binaries)
205      - $MANPATH          (to access the manual pages)
206
207   Hit <RETURN> to see where Grid Engine logs messages >> 
208
209   Grid Engine messages
210   --------------------
211
212   Grid Engine messages can be found at:
213
214      /tmp/qmaster_messages (during qmaster startup)
215      /tmp/execd_messages   (during execution daemon startup)
216
217   After startup the daemons log their messages in their spool directories.
218 
219      Qmaster:     /scratch2/myusername/sge62/default/spool/qmaster/messages
220      Exec daemon: <execd_spool_dir>/<hostname>/messages
221
222
223   Grid Engine startup scripts
224   ---------------------------
225
226   Grid Engine startup scripts can be found at:
227 
228      /scratch2/myusername/sge62/default/common/sgemaster (qmaster)
229      /scratch2/my/sge62/default/common/sgeexecd (execd)
230
231   Do you want to see previous screen about using Grid Engine again (y/n) [n] >> 
232
233   Your execution daemon installation is now completed.
234

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

• Example Berkeley DB Spooling Server Installation

How to Install the Berkeley DB Spooling Server

The installation procedure installs the Grid Engine software necessary for Berkeley DB spooling.

1. Load the Grid Engine software onto a local file system.
   For details on how to extract the files, see How to Load the Distribution Files On a Workstation.
   
   
2. Log in to the spooling server host as root.
   
   
3. If the $SGE_ROOT environment variable is not set, set it by typing:

   # SGE_ROOT=sge-root; export SGE_ROOT
   

   To confirm that you have set the $SGE_ROOT environment variable, type:

   # echo $SGE_ROOT
   

4. Change to the installation directory.

   # cd $SGE_ROOT
   

5. Type the inst_sge command with the -db option.

   # sge-root/inst_sge -db
   

   This command starts the spooling server installation procedure. You are asked several questions. If you think something went wrong, you can quit the installation procedure and restart it at any time.

6. Choose an administrative account owner.

   Choosing Grid Engine admin user account
   ---------------------------------------
   
   You may install Grid Engine that all files are created with the user id of an 
   unprivileged user.
   
   This will make it possible to install and run Grid Engine in directories
   where user >root< has no permissions to create and write files and directories.
   
      - Grid Engine still has to be started by user >root<
   
      - this directory should be owned by the Grid Engine administrator
   
   Do you want to install Grid Engine
   under an user id other than >root< (y/n) [y] >> y
   
   Choosing a Grid Engine admin user name
   --------------------------------------
   
   Please enter a valid user name >> sgeadmin
   Installing Grid Engine as admin user >sgeadmin<
   
   Hit <RETURN> to continue >>
   

7. Verify the $SGE_ROOT directory setting.
   In the following example, the value of $SGE_ROOT is /opt/sge62.

   Checking $SGE_ROOT directory
   ----------------------------
   
   The Grid Engine root directory is:
   
      $SGE_ROOT = /opt/sge62
   
   If this directory is not correct (e.g. it may contain an automounter
   prefix) enter the correct path to this directory or hit <RETURN>
   to use default [/opt/n1ge6] >> 
   
   Your $SGE_ROOT directory: /opt/sge62
   
   Hit <RETURN> to continue >> 
   

8. Type the name of your cell.
   The use of Grid Engine system cells is described in Cells.

   Grid Engine cells
   -----------------
   
   Grid Engine supports multiple cells.
   
   If you are not planning to run multiple Grid Engine clusters or if you don't
   know yet what is a Grid Engine cell it is safe to keep the default cell name
   
      default
   
   If you want to install multiple cells you can enter a cell name now.
   
   The environment variable
   
      $SGE_CELL=<your_cell_name>
   
   will be set for all further Grid Engine commands.
   
   Enter cell name [default] >> 
   

9. Select Berkeley DB spooling.

   Setup spooling
   --------------
   Your SGE binaries are compiled to link the spooling libraries
   during runtime (dynamically). So you can choose between Berkeley DB 
   spooling and Classic spooling method.
   Please choose a spooling method (berkeleydb|classic) [berkeleydb] >> 
   

10. Verify your host name.
    In this example, the installation script is being run on host2.

    Berkeley Database spooling parameters
    -------------------------------------
    
    You are going to install an RPC Client/Server mechanism!
    In this case, qmaster will
    contact an RPC server running on a separate server machine.
    If you want to use the SGE shadowd, you have to use the 
    RPC Client/Server mechanism.
    
    Enter database server name or 
    hit <RETURN> to use default [host2] >> 
    

11. Type the directory path of your spooling directory.
    You might need to change this path if this directory is NFS mounted, or if you do not have write permissions to this directory.

    Enter the database directory
    or hit <RETURN> to use default [/opt/sge62/default//spooldb] >> 
    
    creating directory: /opt/sge62/default//spooldb
    

12. Start the RPC server.

    Now we have to startup the rc script
    >/opt/sge62/default/common/sgebdb< 
    on the RPC server machine
    
    If you already have a configured Berkeley DB Spooling Server,
    you have to restart the Database with the rc script now and continue with >NO<
    
    Shall the installation script try to start the RPC server? (y/n) [y] >> y
    Starting rpc server on host host2!
    The Berkeley DB has been started with these parameters:
    
    Spooling Server Name: host2
    DB Spooling Directory: /opt/sge62/default//spooldb
    
    Please remember these values, during Qmaster installation
    you will be asked for them! Hit <RETURN> to continue!
    

13. Specify whether you want Berkeley DB service to start automatically at boot time.

    Berkeley DB startup script
    --------------------------
    
    We can install the startup script that
    Grid Engine is started at machine boot (y/n) [y] >> y
    

    Once you answer this question, the installation process is complete.
    
    

14. Create the environment variables for use with the Grid Engine software.

    Note If no cell name was specified during installation, the value of $SGE_CELL is default.

    • If you are using a C shell, type the following command:

      % source $SGE_ROOT/$SGE_CELL/common/settings.csh
      

    • If you are using a Bourne shell or Korn shell, type the following command:

      $ . $SGE_ROOT/$SGE_CELL/common/settings.sh
      

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Example Berkeley DB Spooling Server Installation

The following example shows a complete Berkeley DB Spooling Server installation. Remember that this is only one step in the entire Sun Grid Engine installation process. The line numbers in this example are referred to from the spooling server installation description at How to Install the Berkeley DB Spooling Server.

Steps 1-4

# cd $SGE_ROOT

# sge-root/inst_sge -db

Choosing Grid Engine admin user account
---------------------------------------

You may install Grid Engine that all files are created with the user id of an 
unprivileged user.

This will make it possible to install and run Grid Engine in directories
where user >root< has no permissions to create and write files and directories.

   - Grid Engine still has to be started by user >root<

   - this directory should be owned by the Grid Engine administrator

Do you want to install Grid Engine
under an user id other than >root< (y/n) [y] >> y

Choosing a Grid Engine admin user name
--------------------------------------

Please enter a valid user name >> sgeadmin
Installing Grid Engine as admin user >sgeadmin<

Hit <RETURN> to continue >>

Checking $SGE_ROOT directory
----------------------------

The Grid Engine root directory is:

   $SGE_ROOT = /opt/n1ge6

If this directory is not correct (e.g. it may contain an automounter
prefix) enter the correct path to this directory or hit <RETURN>
to use default [/opt/n1ge6] >> 

Your $SGE_ROOT directory: /opt/n1ge6

Hit <RETURN> to continue >> 

Grid Engine cells
-----------------

Grid Engine supports multiple cells.

If you are not planning to run multiple Grid Engine clusters or if you don't
know yet what is a Grid Engine cell it is safe to keep the default cell name

   default

If you want to install multiple cells you can enter a cell name now.

The environment variable

   $SGE_CELL=<your_cell_name>

will be set for all further Grid Engine commands.

Enter cell name [default] >> 

Setup spooling
--------------
Your SGE binaries are compiled to link the spooling libraries
during runtime (dynamically). So you can choose between Berkeley DB 
spooling and Classic spooling method.
Please choose a spooling method (berkeleydb|classic) [berkeleydb] >> 

Berkeley Database spooling parameters
-------------------------------------

You are going to install an RPC Client/Server mechanism!
In this case, qmaster will
contact an RPC server running on a separate server machine.
If you want to use the SGE shadowd, you have to use the 
RPC Client/Server mechanism.


Enter database server name or 
hit <RETURN> to use default [host2] >> 

Enter the database directory
or hit <RETURN> to use default [/opt/n1ge6/default//spooldb] >> 

creating directory: /opt/n1ge6/default//spooldb

Now we have to startup the rc script
>/opt/n1ge6/default/common/sgebdb< 
on the RPC server machine

If you already have a configured Berkeley DB Spooling Server,
you have to restart the Database with the rc script now and continue with >NO<

Shall the installation script try to start the RPC server? (y/n) [y] >> y
Starting rpc server on host host2!
The Berkeley DB has been started with these parameters:

Spooling Server Name: host2
DB Spooling Directory: /opt/n1ge6/default//spooldb

Please remember these values, during Qmaster installation
you will be asked for them! Hit <RETURN> to continue!

Berkeley DB startup script
--------------------------

We can install the startup script that
Grid Engine is started at machine boot (y/n) [y] >> y

<< Previous: How to Install Execution Hosts

Next: Registering Submit Hosts >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Registering Administration Hosts

The master host is implicitly allowed to run administrative tasks and to submit, monitor, and delete jobs. The master host does not require any additional installation or configuration to perform administration functions. By contrast, pure administration hosts do require registration.

Note You can also install administration hosts by using the QMON graphical user interface. See Configuring Administration Hosts With QMON.

To register an administration host from the command line:

1. On the master host, log in to the Grid Engine system administrative account, for example, the sgeadmin account.
   
   
2. Type the following command:

   % qconf -ah <admin-host-name>[,...]
   

<< Previous: How to Install Execution Hosts

Next: Registering Submit Hosts >>

<< Previous: Registering Administration Hosts

Next: Installing Sun Grid Engine Interactively >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Registering Submit Hosts

Note You can also install submit hosts by using the QMON graphical user interface. See Configuring Submit Hosts With QMON.

To register a submit host from the command line:

1. On the master host, log in to the Grid Engine system administrative account, for example, the sgeadmin account.
   
   
2. Type the following command:

   % qconf -as <submit-host-name>[,...]
   

Refer to About Hosts and Daemons for more details and other means to configure the different host types.

<< Previous: Registering Administration Hosts

Next: Installing Sun Grid Engine Interactively >>

<< Previous: Installing Sun Grid Engine Software Interactively

Next: Verifying Sun Grid Engine Installation >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Installing the Increased Security Features

Use the instructions in this section to set up your system more securely. These instructions will help you set up your system with Certificate Security Protocol (CSP)-based encryption.

Installing the increased security features consists of the following topics:

Why Install the Increased Security Features?

Instead of transferring messages in clear text, the messages in this secure system are encrypted with a secret key. The secret key is exchanged using a public/private key protocol. Users present their certificates through the Grid Engine system to prove identity. Users receive the certificate to ensure that they are communicating with the correct systems. After this initial announcement phase, communication continues transparently in encrypted form. The session is valid only for a certain period, after which the session must be re-announced.

Additional Setup Required

The steps required to set up the Certificate Security Protocol enhanced version of the Grid Engine system are similar to the standard setup. You generally follow the instructions in Planning the Installation, Loading the Distribution Files on a Workstation, How to Install the Master Host, How to Install Execution Hosts and Registering Administration Hosts.

However, the following additional tasks are required:

• Generating the Certificate Authority (CA) system keys and certificates on the master host by calling the installation script with the -csp flag
• Distributing the system keys and certificates to the execution and submit hosts using a secure method such as ssh
• Generating user keys and certificates automatically, after master installation
• Adding new users

How to Install a CSP-Secured System

Install the Grid Engine software as outlined in Performing an Installation, with the following exception: use the additional flag -csp when invoking the various installation scripts. To install a CSP-secured system do the following:

1. Change the master host installation procedure.
   Type the following command and respond to the prompts from the installation script.

   # ./install_qmaster -csp
   

   
   

2. Supply the following information to generate the CSP certificates and keys:
   • Two-letter country code, for example, US for the United States
   • State
   • Location, such as a city
   • Organization
   • Organizational unit
   • CA email address
     
     As the installation proceeds, the Certificate Authority is created. A CA specific to the Grid Engine system is created on the master host. The directories that contain information relevant to security are as follows:
   • The publicly accessible CA and daemon certificate are stored in

     $SGE_ROOT/$SGE_CELL/common/sgeCA
     

   • The corresponding private keys are stored in

     /var/sgeCA/{sge_service| portSGE_QMASTER_PORT}/cell/private
     

   • User keys and certificates are stored in

     /var/sgeCA/{sge_service| portSGE_QMASTER_PORT}/cell/userkeys/$USER
     

     
     

3. The script prompts you for site information.
   
   
4. Confirm whether the information you supplied is correct.
   
   
5. Continue the installation.
   After the security-related setup of the master host sge_qmaster is finished, the script prompts you to continue with the rest of the installation procedure, as in the following example:

   SGE startup script
   --------------------
   
   Your system wide SGE startup script is installed as:
   
        "/scratch2/eddy/sge_sec/default/common/sgemaster"
   
   Hit Return to continue >>
   

   
   

6. Transfer the directory that contains the private key and the random file to each execution host.
   1. As root on the master host, type the following commands to prepare to copy the private keys to the machines you set up as execution hosts:

      # umask 077
      # cd /
      # tar cvpf /var/sgeCA/port536.tar /var/sgeCA/port536/default
      

   2. As root on each execution host, use the following commands to securely copy the files:

      # umask 077
      # cd /
      # scp masterhost:/var/sgeCA/port536.tar . 
      # umask 022
      # tar xvpf /port536.tar
      # rm /port536.tar
      

      Note On a Windows execution host, the tar utility cannot restore the ownerships and permissions. In this case, the Administrator must set the ownerships and permissions manually.

   3. Type the following command to verify the file permissions:

      # ls -lR /var/sgeCA/port536/
      

      
      The output should look like the following example:

      /var/sgeCA/port536/:
      total 2
      drwxr-xr-x   4 eddy     other        512 Mar  6 10:52 default
      /var/sgeCA/port536/default:
      total 4
      drwx------   2 eddy     staff        512 Mar  6 10:53 private
      drwxr-xr-x   4 eddy     staff        512 Mar  6 10:54 userkeys
      /var/sgeCA/port536/default/private:
      total 8
      -rw-------   1 eddy     staff        887 Mar  6 10:53 cakey.pem
      -rw-------   1 eddy     staff        887 Mar  6 10:53 key.pem
      -rw-------   1 eddy     staff       1024 Mar  6 10:54 rand.seed
      -rw-------   1 eddy     staff        761 Mar  6 10:53 req.pem
      /var/sgeCA/port536/default/userkeys:
      total 4
      dr-x------   2 eddy     staff        512 Mar  6 10:54 eddy
      dr-x------   2 root     staff        512 Mar  6 10:54 root
      /var/sgeCA/port536/default/userkeys/eddy:
      total 16
      -r--------   1 eddy     staff       3811 Mar  6 10:54 cert.pem
      -r--------   1 eddy     staff        887 Mar  6 10:54 key.pem
      -r--------   1 eddy     staff       2048 Mar  6 10:54 rand.seed
      -r--------   1 eddy     staff        769 Mar  6 10:54 req.pem
      /var/sgeCA/port536/default/userkeys/root:
      total 16
      -r--------   1 root     staff       3805 Mar  6 10:54 cert.pem
      -r--------   1 root     staff        887 Mar  6 10:54 key.pem
      -r--------   1 root     staff       2048 Mar  6 10:53 rand.seed
      -r--------   1 root     staff        769 Mar  6 10:54 req.pem
      

      
      

7. Install the Grid Engine software on each execution host.

   # cd $SGE_ROOT
   # ./install_execd -csp
   

   
   

8. Respond to the prompts from the installation script.
   The execution host installation procedure creates the appropriate directory hierarchy required by sge_execd, and starts the sge_execd daemon on the execution host.
   If the root user does not have write permissions in the $SGE_ROOT directory on all of the machines where Grid Engine software will be installed, you are asked whether to install the software as the user to whom the directory belongs. If you answer yes, you must install the security-related files into that user's $HOME/.sge directory, as shown in the following example.

   % su - sgeadmin
   % source $SGE_ROOT/default/common/settings.csh
   % $SGE_ROOT/util/sgeCA/sge_ca -copy
   % logout
   

   In the above example, sgeadmin is the name of the user who owns the installation directory.
   
   

9. After completing all remaining installation steps, refer to the instructions below in How to Generate Certificates and Private Keys for Users.

How to Generate Certificates and Private Keys for Users

To use the CSP-secured system, the user must have access to a user-specific certificate and private key. The most convenient method of gaining access is to create a text file identifying the users.

1. On the master host, create and save a text file that identifies users.
   Use the format of the file myusers.txt shown in the following example. The fields of the file are UNIX_username:Gecos_field:email_address.

   eddy:Eddy Smith:eddy@my.org
   sarah:Sarah Miller:sarah@my.org
   leo:Leo Lion:leo@my.org
   

   
   

2. As root on the master host, type the following command:

   # $SGE_ROOT/util/sgeCA/sge_ca -usercert myusers.txt
   

   
   

3. Confirm by typing the following command:

   # ls -l /var/sgeCA/port536/default/userkeys
   

   This directory listing produces output similar to the following example.

   dr-x------  2 eddy  staff        512 Mar  5 16:13 eddy
   dr-x------  2 sarah staff        512 Mar  5 16:13 sarah
   dr-x------  2 leo   staff        512 Mar 5 16:13 leo
   

   
   

4. Tell each user to install security related files in their directories.
   Tell each user listed in the file (myusers.txt in the example) to install the security-related files in their $HOME/.sge directories by typing the following commands.

   % source $SGE_ROOT/default/common/settings.csh
   % $SGE_ROOT/util/sgeCA/sge_ca -copy
   

   Users should see the following confirmation (user eddy in the example).

   Certificate and private key for user
   eddy have been installed
   

   For every Grid Engine software installation, a subdirectory for the corresponding SGE_QMASTER_PORT number is installed. The following example, based on the myusers.txt file, is a result of issuing the command preceding the output.

   % ls -lR $HOME/.sge
   
   /home/eddy/.sge:
   total 2
   drwxr-xr-x  3 eddy staff        512 Mar  5 16:20 port536
   
   /home/eddy/.sge/port536:
   total 2
   drwxr-xr-x  4 eddy staff        512 Mar  5 16:20 default
   
   /home/eddy/.sge/port536/default:
   total 4
   drwxr-xr-x  2 eddy staff        512 Mar  5 16:20 certs
   drwx------  2 eddy staff        512 Mar 5 16:20 private
   
   /home/eddy/.sge/port536/default/certs:
   total 8
   -r--r--r--  1 eddy staff       3859 Mar  5 16:20 cert.pem
   
   /home/eddy/.sge/port536/default/private:
   total 6
   -r--------  1 eddy staff        887 Mar  5 16:20 key.pem
   -r--------  1 eddy staff       2048 Mar 5 16:20 rand.seed
   

How to Renew Certificates

1. Change to $SGE_ROOT and become root on the master host.

   # tcsh
   # source $SGE_ROOT/default/settings.csh
   

   Note This assumes that $SGE_CELL is the default.

   
   

2. Edit $SGE_ROOT/util/sgeCA/renew_all_certs.csh, and change the number of days that the certificates are valid:

    # extend the validity of the CA certificate by
     set CADAYS = 365
     # extend the validity of the daemon certificate by
     set DAEMONDAYS = 365
     # extend the validity of the user certificate by
     set USERDAYS = 365
   

   
   

3. Run the changed script.

   # util/sgeCA/renew_all_certs.csh
   

   Note The default for all extension times is 365 days from the day the script is run.

   
   

4. Replace the old certificates against the new ones on all hosts that installed them locally.
   That is, under /var/sgeCA/..., see the execution daemon installation.
   
   
5. If users have copied certificates and keys to $HOME/.sge, they have to repeat $SGE_ROOT/util/sgeCA/sge_ca -copy to have access to the renewed certificates.

Checking Certificates

The following sections provide examples of commands related to certificates, where arch is your system architecture, as in sol-sparc64. Depending on what you want to do, type one or more of the following commands.

Displaying a Certificate

Type the following as one string with a space between the -in and the ~/.sge components.

% $SGE_ROOT/utilbin/arch/opensslx509 -in 
~/.sge/port536/default/certs/cert.pem -text

Check Issuer

Type the following as one string with a space between the -in and the ~/.sge components.

% $SGE_ROOT/utilbin/arch/opensslx509 -issuer -in 
~/.sge/port536/default/certs/cert.pem -noout

Check Subject

Type the following as one string with a space between the -in and the ~/.sge components.

% $SGE_ROOT/utilbin/arch/opensslx509 -subject -in 
~/.sge/port536/default/certs/cert.pem -noout

Show Email of Certificate

Type the following as one string with a space between the -in and the ~/.sge components.

% $SGE_ROOT/utilbin/arch/opensslx509 -email -in 
~/.sge/default/port536/certs/cert.pem -noout

Show Validity

Type the following as one string with a space between the -in and the ~/.sge components.

% $SGE_ROOT/utilbin/arch/opensslx509 -dates -in 
~/.sge/default/port536/certs/cert.pem -noout

Show Fingerprint

Type the following as one string with a space between the -in and the ~/.sge components.

% $SGE_ROOT/utilbin/arch/opensslx509 -fingerprint -in 
~/.sge/port536/default/certs/cert.pem -noout

<< Previous: Installing Sun Grid Engine Software Interactively

Next: Verifying Sun Grid Engine Installation >>

<< Previous: Installing Sun Grid Engine Software Interactively

Next: Installing the Increased Security Features >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Upgrading From a Previous Version of Sun Grid Engine Software

Note The following instructions will work only on the Sun Grid Engine 6.2 RR release. The upgrade procedure is only able to upgrade your software from version 6.0 update 2 or higher. If you are running an older version of the Sun Grid Engine software, such as 5.3 or 6.0, you must upgrade to version 6.0 update 2 or higher and then upgrade again to version 6.2 as explained below. See Upgrading from 5.3 to 6.0.

• Example Upgrade for Cloned Cluster Configuration
• Upgrading from 5.3 to 6.0

About Upgrading the Software

Note The upgrade procedure is now partly destructive. See the constraints. The LD_LIBRARY_PATH variable is not set in Grid Engine 6.2 software. Remove the existing LD_LIBRARY_PATH settings from 6.0 before you start a 6.2 installation. Before you begin the upgrade process, make sure that you source the existing $SGE_ROOT/$SGE_CELL/common/settings.sh or $SGE_ROOT/$SGE_CELL/common/settings.csh file.

The upgrade procedure uses the cluster configuration information from the older version of the software to install the Grid Engine 6.2 software on the master host. Beginning with the Sun Grid Engine 6.2 release, you can install 6.2 to a different $SGE_ROOT or $SGE_CELL and transfer the old configuration to this cluster. This method is called cloned cluster configuration. You might want to use this method to accomplish the following:

• To test the upgrade before making the real upgrade.
• To keep the old cluster running.

Before You Upgrade

Choose one of the following methods to upgrade to 6.2:

• New 6.2 installation (different $SGE_ROOT or $SGE_CELL) using the same configuration as was used for the old cluster (cloned cluster configuration).
  If you use the cloned cluster configuration, you do not have to stop or in any way affect the original cluster. You simply install a new qmaster and transfer the configuration from the old cluster to the new one. Then, you manually restart the new execution daemons on all the original execution hosts.
  The disadvantage of the cloned configuration method is that you have to install the new qmaster and might loose some of the configuration information during the upgrade (see the constraints). Another disadvantage is that the original execution host will now have twice as many slots - one set for the old cluster and one for the new one.
• Real upgrade of the existing cluster (same $SGE_ROOT and $SGE_CELL.)

Constraints

The following constraints apply to both upgrade methods:

• Dynamic and static load values will be lost (only static values will be recreated).
• The sharetree usage will be lost.
• Neither jobs nor advanced reservations (ARs) will be replicated.
• There might be running or pending jobs in the cluster when the configuration is saved. If you decide to install the new Sun Grid Engine version in the same $SGE_ROOT and $SGE_CELL, then you must remove all jobs from the old cluster before the old cluster is shutdown and the new software is installed.
• The previous state of a disabled queue will be lost if the queue config initial_state is set to default.

Additional Constraints for the New 6.2 Installation with Cloned Configuration

For the cloned cluster configuration, you must also define several new variables and directories that must be different from the original settings:

• $SGE_ROOT
• $SGE_CELL
• $SGE_CLUSTER_NAME
• $SGE_QMASTER_PORT
• $SGE_EXECD_PORT
• Master daemon spooling directory (qmaster_spool_dir)
• Execution daemon spooling directory (execd_spool_dir)
• Group ID range for the jobs (gid_range)

Caution Only one SGE_Helper_Service.exe can run on an execution host. You cannot use the same Windows execution host for a 6.0 or 6.1 cluster and a 6.2 cluster.

Note Because there have been significant changes in the Grid Engine 6.2 software, loading the configuration adds and removes some configuration attributes. Adding and removing configuration attributes might affect the operation of the cluster. To ensure stability, you should always follow this process: Upgrade to the new $SGE_ROOT or $SGE_CELL (cloned cluster configuration). Test that the original cluster configuration did not change and that the functionality of the cluster remains intact. Perform the real upgrade of the original cluster, if desired.

Back Up the Configuration of the Old Cluster

You can create this backup at any time before you start the upgrade procedure. The upgrade is the same for both types of the upgrade procedures. To create the backup, at least the qmaster daemon must be running.

What the Backup Contains

The backup saves the following files:

• arseqnum
• jobseqnum
• act_qmaster
• bootstrap
• cluster_name
• host_aliases
• qtask
• sge_aliases
• sge_ar_request
• sge_request
• sge_qstat
• sge_qquota
• sge_qstat
• shadow_masters
• accounting
• dbwriter.conf
• jmx directory

Caution During the upgrade procedure, you can select the next job ID. Do not select a job ID that is less than the last job ID in the accounting file in the backup. If you do, the accounting file will contain some job IDs twice. This leads to unexpected behaviors. To avoid the problem, accept the suggested default for the next job ID. The upgrade procedure calculates a safe value for the default.

The backup process creates the following files:

• sge_root - old $SGE_ROOT
• sge_cell - old $SGE_CELL
• ports - old $SGE_QMASTER_PORT and $SGE_EXECD_PORT
• win_hosts - A list of registered windows execution hosts at the time of the backup

The standard qconf client is used to save the complete cluster configuration.

How to Back Up the Cluster

1. Either download the backup script or get the backup script from the Sun Grid Engine 6.2 common package (util/upgrade_modules/save_sge_config.sh).
2. (Optional) Verify that the script is executable.
3. Source the $SGE_ROOT/$SGE_CELL/common/settings.sh (or .csh) file of the original cluster.
4. Run the backup script.
   The backup script has one argument, which is the path to the directory in which to store the backup. The directory must not already exist, but the user must have permission to create it.

   Note You must run the backup script on an admin host (qconf -sh) as a manager or operator user (typically sgeadmin).

   # ./save_sge_config.sh /backups/sge_6.1_June10_2008
   

   The backup process displays a message confirming that the backup succeeded.

How to Install the 6.2 Software Using the Cloned Cluster Configuration Method

Caution Do not make both the new cluster and the old cluster available to your users. If you do, execution hosts would offer the original amount of slots for both clusters and might become overloaded.

1. Back up the original cluster settings as described in How to Back Up the Cluster.
2. (Optional) ARCo Upgrade Prerequisites
   If you use ARCo and you want to have the data from the old and new cluster in the same ARCo database, you cannot install the dbwriter on the new cluster, specifying the old dbwriter's database parameters, unless the dbwriter from the old cluster is stopped and all the data from the old cluster are inserted in the database. After installing dbwriter (with the same database parameters) on the new cluster, you must not again start the dbwriter on the old cluster, otherwise your database will be compromised.
   1. Wait to install ARCo on the new cluster until all the jobs are drained from the old cluster, the cluster is stopped and the old reporting file is processed completely.
      There should be no reporting or reporting.processing file in the $SGE_ROOT/$SGE_CELL/common directory of the old cluster.

      Note Jobs can be submitted and the reporting file generated on the new cluster, as long as there is no dbwriter installed on the new cluster.

      Caution There cannot be more than one dbwriter process writing into the same ARCo database and schema. If you create a new ARCo database for the new cluster, you cannot later merge it with the old ARCo database, due to the primary key constraints.

      Once the reporting file on the old cluster is processed, on dbwriter host:

   2. Source the cluster settings.sh (or .csh) file.
   3. Stop the dbwriter:

      # $SGE_ROOT/$SGE_CELL/common/sgedbwriter stop
      

      
      
      

3. Extract the new 6.2 binaries and common files to the new $SGE_ROOT directory.
   
   
4. Start the new upgrade installation of the qmaster from the new $SGE_ROOT directory.

   # ./inst_sge -upd
   

   This starts the upgrade procedure. See the Example Upgrade for Cloned Cluster Configuration.

   Tip To enable or disable some additional features like JMX, CSP, or use old IJS, you must provide additional flags to the upgrade script the same way you would for qmaster installation. For example, to upgrade a cluster and enable JMX thread in qmaster and CSP mode run: ./inst_sge -upd -jmx -csp

   
   

5. Accept the displayed license.
   
   
6. Enter the complete path to the backup directory.
   For example, /backups/sge_6.1_June10_2008. See Step 6 in the example.
   
   
7. Enter the new $SGE_ROOT directory.
   The default is the current directory. For more information, see SGE_ROOT. See Step 7 in the example.
   
   
8. Select a new $SGE_CELL directory.
   The default is the $SGE_CELL directory from the backup. For more information, see SGE_CELL. See Step 8 in the example.
   
   
9. Select a new SGE_QMASTER_PORT number.
   The default is the $SGE_QMASTER_PORT number from the backup + 2. See Step 9 in the example.
   
   
10. Select a new SGE_EXECD_PORT number.
    The default is the $SGE_EXECD_PORT number from the backup + 2. See Step 10 in the example.
    
    
11. Select a new qmaster spooling directory
    The default is $SGE_ROOT/$SGE_CELL/spool/qmaster. See Step 11 in the example.
    
    
12. Select a new $SGE_CLUSTER_NAME.
    The default is p$SGE_QMASTER_PORT. For more information, see SGE_CLUSTER_NAME. See Step 12 in the example.
    
    
13. (Optional) Choose the JMX configuration.
    For more information about JMX, see JMX guide.
    If you started the upgrade using the -jmx option, one of the following choices appears:
    • Choose if you want to use JMX settings from the backup or use new settings.
      This question appears when JMX exists in the backup.
    • Choose a JMX port.
      This question appears when JMX does not exist in the backup.
      
      
14. Select a spooling method.
    For more information on choosing a spooling mechanism, see Choosing Between Classic Spoooling and Database Spooling. See Step 14 in the example.
    
    
15. Choose if you want to use interactive jobs support (IJS) settings from the backup or use the new defaults for 6.2.
    In most cases, you should use the new defaults which enable the new interactive jobs support. Step 15 in the example shows the new defaults.

    Caution If you changed QLOGIN_DAEMON, QLOGIN_COMMAND, RLOGIN_DEAMON, RLOGIN_COMMAND, RSH_DEAMON, or RSH_COMMAND configuration attributes, you should verify that the new IJS will not break your site-specific settings.

    
    
    

16. Choose the group id range
    The default is the last group id from the backup + 100 and same range. See Step 16 in the example.
    
    
17. Select the next job ID.
    The default is old jobseqnum + 1000, rounded up to the nearest 1000. See Step 17 in the example.
    
    
18. (Optional) Select the next AR ID.
    This question appears only if arseqnum is in the backup. The default is old arseqnum + 1000, rounded up to the nearest 1000. See Step 18 in the example.
    
    
19. Select automatic startup options.
    See Step 19 in the example.
    One of the following choices appears:
    • Choose whether to run qmaster as an SMF service.
      This question appears only on systems that run at least version 10 of the Solaris OS.
    • Choose whether to use RC scripts for qmaster.
      This question appears on platforms that are not running at least version 10 of the Solaris OS or if you started the upgrade using the -nosmf option.
      
      
20. Load the old configuration.
    See Step 20 in the example.
    If this step fails with a critical error:
    1. Check the log file /tmp/sge_backup_date.log.
    2. Try to reload the configuration through the $SGE_ROOT/util/upgrade_modules/load_sge_config.sh script and the arguments displayed in the previous step.
    3. If the preceding steps do not resolve the problem, stop the upgrade process.
       
       
21. (Optional) Upgrade ARCo.
    If you use ARCo, you need to upgrade it. If you want to use the same ARCo database, copy the $SGE_ROOT/$SGE_CELL/common/dbwriter.conf from the old cluster into the same directory on the new cluster, it will be sourced and you will be only prompted to enter any missing information during the installtion of dbwriter. See Upgrading ARCo step 6.
    
    
22. Run the post upgrade procedures

    Info The post-upgrade procedures are easier when you have root access to all machines through ssh or rsh without having to enter a password. To use rsh instead of the default ssh, run the ./inst_sge command with -rsh argument. Example: # ./inst_sge -upd-execd -rsh

    1. Initialize the local execd spool directories
       This step creates the local execd spool directories on the execd hosts with the correct permissions. Run the following command as root from the master host in $SGE_ROOT directory:

       # ./inst_sge -upd-execd
       

    2. (Optional) Create new RC scripts for the whole cluster.

       Caution This command removes old RC scripts. To keep the old RC scripts, do not run this command.

       To start the services automatically after a reboot, run the following command as root from the master host in $SGE_ROOT directory:

       ## ./inst_sge -upd-rc
       

    3. (Optional) Install or update the Windows helper service.
       Perform this step to use the Windows execution hosts with the 6.2 cluster. When connecting to each Windows execution host, you are prompted for an administrator user to connect to the Windows host. If all your Windows hosts share the same administrative user, set the environment variable SGE_WIN_ADMIN to that user to access all Windows hosts without additional user intervention. Example:

       (sh, bash)# export SGE_WIN_ADMIN=Administrator
       (csh,tcsh)# setenv SGE_WIN_ADMIN Administrator
       

       To install or update the Windows helper service, run the following command as root from the master host in $SGE_ROOT directory:

       # ./inst_sge -upd-win
       

       Caution Only one SGE_Helper_Service.exe can run on an execution host. You cannot use the same Windows execution host for a 6.0 or 6.1 cluster and a 6.2 cluster.

       
       
       

23. Start the new execution daemons.
    Optionally, if you can login without typing a password, you can start the whole cluster as root user from the $SGE_ROOT directory with a single command:

    # ./inst_sge -start-all
    

    This command starts the master daemon, shadow daemons, and all execution daemons.

Upgrade is complete.

How to Upgrade the Original Cluster to 6.2 Software (Real Upgrade)

1. (Optional) Test the cloned cluster, if you used the cloned cluster configuration method to transfer the configuration to a new 6.2 cluster.
   
   
2. Back up the original cluster settings as described in How to Back Up the Cluster.
   
3. Stop the scheduler:

   # qconf -ks
   

   
   

4. Verify that no jobs are running on the cluster.
   
   
5. Stop the old cluster:

   # qconf -ke all
   # $SGE_ROOT/$SGE_CELL/common/sgemaster stop
   

   
   

6. (Optional) Stop the Berkeley DB server, if your cluster uses Berkeley DB server spooling.
   On the BDB server host:
   1. Source the cluster settings.sh (or .csh) file.
   2. Type the following command:

      # $SGE_ROOT/$SGE_CELL/common/sgebdb stop
      

      
      

7. (Optional) If you use ARCo, ensure that the reporting file has been completely processed by the dbwriter.
   There should be no reporting or reporting.processing file in the $SGE_ROOT/$SGE_CELL/common directory.
   Once the reporting file is processed, on dbwriter host:
   1. Source the cluster settings.sh (or .csh) file.
   2. Stop the dbwriter:

      # $SGE_ROOT/$SGE_CELL/common/sgedbwriter stop
      

      Warning If you use ARCo, you must completely process the reporting file and stop the dbwriter before you continue.

      
      

8. Extract the new 6.2 binaries and common files to the $SGE_ROOT directory.

   Caution Do not remove any of the $SGE_ROOT directory contents, except for the case where the new Sun Grid Engine 6.2 binaries differ from the existing installation. For example, you might have used your custom lx26-amd64 binaries, but Sun Grid Engine 6.2 uses lx24-amd64 even for 2.6 kernels. In that case you must remove the old binaries manually! You must ensure that all binaries for all used architectures were updated and no architecture with the old version remains in the $SGE_ROOT directory.

   
   
   

9. Start the new upgrade on the original qmaster host from the $SGE_ROOT directory.

   # ./inst_sge -upd
   

   Tip To enable or disable some additional features like JMX, CSP, or to use the old IJS, you must provide additional flags to the upgrade script in the same way that you would for qmaster installation. For example, to upgrade a cluster and enable the JMX thread in qmaster and use CSP mode, run the following command: ./inst_sge -upd -jmx -csp

   
   

10. Accept the displayed license.
    
    
11. Enter the complete path to the backup directory.
    For example, /backups/sge_6.1_June10_2008.
    
    

    Caution In case you you don't specify the original $SGE_ROOT and $SGE_CELL in the next two steps, the upgrade type attempted will not be the real upgrade! Instead the clone cluster configuration method will be used.

12. Enter the $SGE_ROOT directory.
    The default is the current directory. For more information, see SGE_ROOT.
    
    
13. Enter the $SGE_CELL directory.
    The default is default. For more information, see SGE_CELL.
    
    
14. Select a new $SGE_CLUSTER_NAME.
    The default value is one of the following, depending on which is found first:
    • The existing SGE_CLUSTER_NAME ($SGE_ROOT/$SGE_CELL/common/cluster-name)
    • The SGE_CLUSTER_NAME from the backup
    • p$SGE_QMASTER_PORT
      For more information, see SGE_CLUSTER_NAME.
      
      
15. (Optional) Select the JMX configuration.
    For more information about JMX, see JMX guide.
    If you started the upgrade using the -jmx option, one of the following choices appears:
    • Choose if you want to use JMX settings from the backup or use new settings.
      This question appears when JMX exists in the backup.
    • Choose a JMX port.
      This question appears when JMX does not exist in the backup.
      
      
16. Choose if you want to keep the spooling method from the backup.
    
    
17. (Optional) Select a spooling method.
    This is displayed if you chose not to use backup in the previous screen. See example. For more information on choosing a spooling mechanism, see Choosing Between Classic Spooling and Database Spooling.
    
    
18. Choose if you want to use interactive jobs support (IJS) settings from the backup or use the new defaults for 6.2.
    In most cases, you should use the new defaults which enable the new interactive jobs support.

    Caution If you changed QLOGIN_DAEMON, QLOGIN_COMMAND, RLOGIN_DEAMON, RLOGIN_COMMAND, RSH_DEAMON, or RSH_COMMAND configuration attributes, you should verify that the new IJS will not break your site-specific settings.

    
    

19. Select the next job ID.
    The default is old jobseqnum + 1000, rounded up to the nearest 1000.
    
    
20. (Optional) Select the next AR ID.
    This question appears only if arseqnum is in the backup. The default is old arseqnum + 1000, rounded up to the nearest 1000.
    
    
21. Choose automatic startup options.
    One of the following choices appears:
    • Choose whether to run qmaster as an SMF service.
      This question appears only on systems that run at least version 10 of the Solaris OS.
    • Choose whether to use RC scripts for qmaster.
      This question appears on platforms that are not running at least version 10 of the Solaris OS or if you started the upgrade using the -nosmf option.
      
      
22. Load the old configuration.
    If this step fails with a critical error:
    1. Check the log file /tmp/sge_backup_date.log.
    2. Try to reload the configuration through the $SGE_ROOT/util/upgrade_modules/load_sge_config.sh script and the arguments displayed in the previous step.
    3. If the preceding steps do not resolve the problem, stop the upgrade process.
       
       
23. (Optional) Copy the binaries and the common directory to all the hosts in the cluster, if not on a shared file system
    If you use local binaries or a local common directory for each host, you must copy all the new binaries and the common directory locally to each host. Ensure that all binaries are updated and no architecture with the old version remains in the $SGE_ROOT directory.

    Note If you do not perform this operation the qmaster host will have Sun Grid Engine 6.2 binaries, while the rest of the cluster will still have the old version and will not work as desired!

    
    
    

24. (Optional) Upgrade ARCo.
    If you use ARCo, you need to upgrade it.See Upgrading ARCo step 6.
    
    
25. Run the post upgrade procedures

    Info The post-upgrade procedures are easier when you have root access to all machines through ssh or rsh without having to enter a password. To use rsh instead of the default ssh, run the ./inst_sge command with -rsh argument. Example: # ./inst_sge -upd-execd -rsh

    1. Initialize the local execd spool directories
       This step creates the local execd spool directories on the execd hosts with the correct permissions. Run the following command as root from the master host in $SGE_ROOT directory:

       # ./inst_sge -upd-execd
       

    2. (Optional) Create new RC scripts for the whole cluster.

       Caution This command removes old RC scripts. To keep the old RC scripts, do not run this command.

       To start the services automatically after a reboot, run the following command as root from the master host in $SGE_ROOT directory:

       ## ./inst_sge -upd-rc
       

    3. (Optional) Install or update the Windows helper service.
       Perform this step to use the Windows execution hosts with the 6.2 cluster. When connecting to each Windows execution host, you are prompted for an administrator user to connect to the Windows host. If all your Windows hosts share the same administrative user, set the environment variable SGE_WIN_ADMIN to that user to access all Windows hosts without additional user intervention. Example:

       (sh, bash)# export SGE_WIN_ADMIN=Administrator
       (csh,tcsh)# setenv SGE_WIN_ADMIN Administrator
       

       To install or update the Windows helper service, run the following command as root from the master host in $SGE_ROOT directory:

       # ./inst_sge -upd-win
       

       Caution Only one SGE_Helper_Service.exe can run on an execution host. You cannot use the same Windows execution host for a 6.0 or 6.1 cluster and a 6.2 cluster.

       
       
       

26. Start the new execution daemons.
    Optionally, if you can login without typing a password, you can start the whole cluster as root user from the $SGE_ROOT directory with a single command:

    # ./inst_sge -start-all
    

    This command starts the master daemon, shadow daemons, and all execution daemons.

Upgrade is complete.

<< Previous: Installing Sun Grid Engine Software Interactively

Next: Installing the Increased Security Features >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Upgrading Sun Grid Engine Software
Index

Example Upgrade for Cloned Cluster Configuration

The following upgrade example uses a copy of the existing cluster configuration with a different $SGE_CELL. This example does not use JMX and there are no Service Tags. The steps in this example are referred to from the software upgrade description at How to Install the 6.2 Software Using the Cloned Cluster Configuration Method.

Steps 4 and 5

# ./inst_sge -upd

Welcome to the Grid Engine Upgrade Procedure
--------------------------------------------

Before you continue with the upgrade, read these hints:

   - Your terminal window should have a size of at least
     80x24 characters

   - At any time during the upgrade process, use your standard
     interrupt key to abort the upgrade. Typically, the interrupt
     key combination is Ctrl-C.

The upgrade procedure will take approximately 1-2 minutes.

Hit <RETURN> to continue >>

Step 6

Type the complete path to the Grid Engine configuration backup directory.
-------------------------------------------------------------------------
Backup directory  >> /tmp/bck

Found backup from GE 6.1u4 version created on 2008-06-10_10:56:29
Continue with this backup directory (y/n) [y] >>

Step 7

The Grid Engine root directory is:

   $SGE_ROOT = /sge

If this directory is not correct (e.g. it may contain an automounter
prefix) enter the correct path to this directory or hit <RETURN>
to use default [/sge] >>

Your $SGE_ROOT directory: /sge

Hit <RETURN> to continue >>

Step 8

Grid Engine cells
-----------------

Grid Engine supports multiple cells.

If you are not planning to run multiple Grid Engine clusters or if you don't
know yet what is a Grid Engine cell it is safe to keep the default cell name

   default

If you want to install multiple cells you can enter a cell name now.

The environment variable

   $SGE_CELL=<your_cell_name>

will be set for all further Grid Engine commands.

Enter cell name [default] >> new_cell

Using cell >new_cell<.
Hit <RETURN> to continue >>

Step 9

Grid Engine TCP/IP communication service
----------------------------------------

The port for sge_qmaster is currently set by the shell environment.

   SGE_QMASTER_PORT = 21640

Now you have the possibility to set/change the communication ports by 
using the
 >shell environment< or you may configure it via a network service, 
configured
in local >/etc/service<, >NIS< or >NIS+<, adding an entry in the form

    sge_qmaster <port_number>/tcp

to your services database and make sure to use an unused port number.

How do you want to configure the Grid Engine communication ports?

Using the >shell environment<:                           [1]

Using a network service like >/etc/service<, >NIS/NIS+<: [2]

(default: 1) >>

Grid Engine TCP/IP communication service
----------------------------------------

Using the environment variable

   $SGE_QMASTER_PORT=21640

as port for communication.

Do you want to change the port number? (y/n) [n] >>

Step 10

Grid Engine TCP/IP communication service
----------------------------------------

The port for sge_execd is currently set by the shell environment.

   SGE_EXECD_PORT = 21641

Now you have the possibility to set/change the communication ports by 
using the
 >shell environment< or you may configure it via a network service, 
configured
in local >/etc/service<, >NIS< or >NIS+<, adding an entry in the form

    sge_execd <port_number>/tcp

to your services database and make sure to use an unused port number.

How do you want to configure the Grid Engine communication ports?

Using the >shell environment<:                           [1]

Using a network service like >/etc/service<, >NIS/NIS+<: [2]

(default: 1) >>

Grid Engine TCP/IP communication service
----------------------------------------

Using the environment variable

   $SGE_EXECD_PORT=21641

as port for communication.

Do you want to change the port number? (y/n) [n] >>

Step 11

Grid Engine qmaster spool directory
-----------------------------------

The qmaster spool directory is the place where the qmaster daemon stores
the configuration and the state of the queuing system.

The admin user >sgeadmin< must have read/write access
to the qmaster spool directory.

If you will install shadow master hosts or if you want to be able to start
the qmaster daemon on other hosts (see the corresponding section in the
Grid Engine Installation and Administration Manual for details) the account
on the shadow master hosts also needs read/write access to this directory.

The following directory

[/sge/new_cell/spool/qmaster]

will be used as qmaster spool directory by default!

Do you want to select another qmaster spool directory (y/n) [n] >>

Step 12

Unique cluster name
-------------------

The cluster name uniquely identifies a specific Sun Grid Engine cluster.
The cluster name must be unique throughout your organization. The name
is not related to the SGE cell.

The cluster name must start with a letter ([A-Za-z]), followed by letters,
digits ([0-9]), dashes (-) or underscores (_).

Enter new cluster name or hit <RETURN>
to use default [p21640] >>

Your $SGE_CLUSTER_NAME: p21640

Hit <RETURN> to continue >>

Step 14

creating directory: /sge/new_cell/spool/qmaster/job_scripts

Setup spooling
--------------
Your SGE binaries are compiled to link the spooling libraries
during runtime (dynamically). So you can choose between Berkeley DB
spooling and Classic spooling method.
Please choose a spooling method (berkeleydb|classic) [berkeleydb] >> classic

Initializing spooling database

Hit <RETURN> to continue >>

Step 15

Interactive Job Support (IJS) Selection
---------------------------------------

The backup configuration includes information for running
interactive jobs. Do you want to use the IJS information from
the backup ('y') or use new default values ('n') (y/n) [y] >> n


Using new interactive job support default setting for a new installation.
Hit <RETURN> to continue >>

Creating >act_qmaster< file

Step 16

Grid Engine group id range
--------------------------

When jobs are started under the control of Grid Engine an additional 
group id is set on platforms which do not support jobs. This is done 
to provide maximum control for Grid Engine jobs.

This additional UNIX group id range must be unused group id's in your 
system. Each job will be assigned a unique id during the time it is 
running. Therefore you need to provide a range of id's which will 
be assigned dynamically for jobs.

The range must be big enough to provide enough numbers for the 
maximum number of Grid Engine jobs running at a single moment on 
a single host. E.g. a range like >20000-20100< means, that Grid Engine 
will use the group ids from 20000-20100 and provides a range for 
100 Grid Engine jobs at the same time on a single host.

You can change at any time the group id range in your cluster configuration.

Please enter a range [34299-34498] >>

Using >34299-34498< as gid range. Hit <RETURN> to continue >>

Grid Engine cluster configuration
---------------------------------

Please give the basic configuration parameters of your Grid Engine
installation:

   <execd_spool_dir>

The pathname of the spool directory of the execution hosts. User >sgeadmin<
must have the right to create this directory and to write into it.

Default: [/sge/new_cell/spool] >>

Grid Engine cluster configuration (continued)
---------------------------------------------

<administrator_mail>

The email address of the administrator to whom problem reports are sent.

It is recommended to configure this parameter. You may use >none<
if you do not wish to receive administrator mail.

Please enter an email address in the form >user@foo.com<.

Default: [sgeadmin@qmaster.com] >>

The following parameters for the cluster configuration were configured:

   execd_spool_dir        /sge/new_cell/spool
   administrator_mail     sgeadmin@qmaster.com

Do you want to change the configuration parameters (y/n) [n] >>

Step 17

Provide a value to use for the next job ID.
-------------------------------------------

Backup contains last job ID 1. As a suggested value, we added 1000
to that number and rounded it up to the nearest 1000.
Increase the value, if appropriate.
Choose the new next job ID [2000] >> 

Hit <RETURN> to continue >>

Step 18

Provide a value to use for the next AR ID.
------------------------------------------

Backup contains last AR ID 1. As a suggested value, we added 1000
to that number and rounded it to the nearest 1000.
Increase the value, if appropriate.
Choose the new next AR ID [2000] >>

Hit <RETURN> to continue >>

Step 19

Creating >sgemaster< script
Creating >sgeexecd< script
Creating settings files for >.profile/.cshrc<

Hit <RETURN> to continue >>

qmaster startup script
----------------------

Do you want to start qmaster automatically at machine boot?
NOTE: If you select "n" SMF will be not used at all! (y/n) [y] >> n


Grid Engine qmaster startup
---------------------------

Starting qmaster daemon. Please wait ...
   starting sge_qmaster
Hit <RETURN> to continue >>

Step 20

Last step - load configuration from the backup
----------------------------------------------

load command: /sge/util/upgrade_modules/load_sge_config.sh /tmp/bck -mode "copy" -log C -newijs "false" -gid_range "34299-34498" -admin_mail "sgeadmin@qmaster.com" -execd_spool_dir "/sge/new_cell/spool"


Hit <RETURN> to continue >>


Loading saved cluster configuration from /tmp/bck (log in 
/tmp/sge_backup_load_2008-06-13_17:42:28.log)...

Loading saved cluster configuration from /tmp/bck (log in /tmp/sge_backup_load_2008-06-13_17:42:28.log)...
Done

If loading the configuration succeeded run these additional commands:
REQUIRED:
inst_sge -upd-execd
   This command initializes all execd spool directories.

inst_sge -upd-win
   This command connects to all Windows execution hosts and installs
   the new Windows helper service on each host.
   WARNING: If a helper service from a previous release is running
            on this host, the new helper service overwrites it. The
            host will run only in a 6.2 cluster.
   TIP: This action requires to enter a windows administrator user for each
        host interactively. If all your systems share the same administrator you
        can set the environment variable SGE_WIN_ADMIN to that user name.
        E.g.: (sh, bash) export SGE_WIN_ADMIN=Administrator
              (csh,tcsh) setenv SGE_WIN_ADMIN Administrator

OPTIONAL:
inst_sge -upd-rc
   This command creates new autostart scripts for the new cluster
   and removes any conflicting files.
   TIP: To disable SMF on Solaris systems, use the command
        inst_sge -upd-rc -nosmf

TIP: Use inst_sge -post-upd to do all above actions

Index

How to Upgrade the Software From 5.3 to 6.0 Update 2

Before You Begin

Be sure to review Planning the Installation for the information that you will need during the upgrade process. If you have decided to use an administrative user, as described in User Names, you should create that user now. This procedure assumes that you have already extracted the Grid Engine software, as described in Loading the Distribution Files on a Workstation.

Note While you can run Grid Engine 6.0 software concurrently with your older version of Grid Engine software, you should run the upgrade procedure when there are no running jobs.

Steps

1. Log in to the master host as root.
   
   
2. Load the distribution files.
   For details, see Loading the Distribution Files on a Workstation.
   
   
3. Ensure that you have set the $SGE_ROOT environment variable by typing:

   # echo $SGE_ROOT
   

   If the $SGE_ROOT environment variable is not set, set it now by typing:

   # SGE_ROOT=sge-root; export SGE_ROOT
   

   
   

4. Change to the sge-root installation directory.
   Select one of the two following options:
   • If the directory where the installation files reside is visible from the master host, change directories (cd) to the installation directory sge-root, and then proceed to Step 4 of How to Install the Master Host.
   • If the directory is not visible and cannot be made visible, do the following:
     • Create a local installation directory, sge-root, on the master host.
     • Copy the installation files to the local installation directory sge-root across the network (for example, by using ftp or rcp).
     • Change directories (cd) to the local sge-root directory.
       
5. Run the upgrade command on the master host, and respond to the prompts.
   This command starts the master host installation procedure. You are asked several questions, and you might be required to run some administrative actions.
   
   The syntax of the upgrade command is:

   inst_sge -upd 5.3-sge-root-directory 5.3-cell-name
   

   In the following example, the 5.3 sge-root directory is /sge/gridware and the cell name is default.

   # ./inst_sge -upd /sge/gridware default
   Welcome to the Grid Engine Upgrade
   ----------------------------------
   
   Before you continue with the installation please read these hints:
   
      - Your terminal window should have a size of at least
        80x24 characters
   
      - The INTR character is often bound to the key Ctrl-C.
        The term >Ctrl-C< is used during the upgrade if you
        have the possibility to abort the upgrade
   
   The upgrade procedure will take approximately 5-10 minutes.
   After this upgrade you will get a running qmaster and schedd with
   the configuration of your old installation. If the upgrade was
   successfully completed it is necessary to install your execution hosts
   with the install_execd script.
   
   Hit <RETURN> to continue >>
   

   
   

6. Choose an administrative account owner.
   In the following example, the value of sge-root is /opt/n1ge6, and the administrative user is sgeadmin.

   Grid Engine admin user account
   ------------------------------
   
   The current directory
   
      /opt/n1ge6
   
   is owned by user
   
      sgeadmin
   
   If user >root< does not have write permissions in this directory on *all*
   of the machines where Grid Engine will be installed (NFS partitions not
   exported for user >root< with read/write permissions) it is recommended to
   install Grid Engine that all spool files will be created under the user id
   of user >sgeadmin<.
   
   IMPORTANT NOTE: The daemons still have to be started by user >root<.
   
   Do you want to install Grid Engine as admin user >sgeadmin< (y/n) [y] >>
   

   
   

7. Verify the $SGE_ROOT directory setting.
   In the following example, the value of $SGE_ROOT is /opt/n1ge6.

   Checking $SGE_ROOT directory
   ----------------------------
   
   The Grid Engine root directory is:
   
      $SGE_ROOT = /opt/n1ge6
   
   If this directory is not correct (e.g. it may contain an automounter
   prefix) enter the correct path to this directory or hit <RETURN>
   to use default [/opt/n1ge6] >>
   

   
   

8. Set up the TCP/IP services for the Grid Engine software.
   1. If the TCP/IP services have not been configured, respond to the installation messages.

      Grid Engine TCP/IP service >sge_qmaster<
      ----------------------------------------
      
      There is no service >sge_qmaster< available in your >/etc/services< file
      or in your NIS/NIS+ database.
      
      You may add this service now to your services database or choose a port number.
      It is recommended to add the service now. If you are using NIS/NIS+ you should
      add the service at your NIS/NIS+ server and not to the local >/etc/services<
      file.
      
      Please add an entry in the form
      
        sge_qmaster <port_number>/tcp
      
      to your services database and make sure to use an unused port number.
      
      Please add the service now or press <RETURN> to go to entering a port number >>
      

   2. Start a new terminal session or window to add information to the /etc/services file or your NIS maps.
   3. Add the correct ports to the /etc/services file or your NIS services map, as described in Network Services.
      The following example shows how you might edit your /etc/services file.

      ...
      sge_qmaster     536/tcp
      sge_execd       537/tcp
      

      Note In this example, the entries for both sge_qmaster and sge_execd are added to /etc/services. Subsequent steps in this example assume that both entries have been made.

   4. Save your changes and return to the window where the installation script is running.

      Please add the service now or press <RETURN> to go to entering a port number >>
      

      Press <RETURN>. The installation procedure displays the following output:

      sge_qmaster 536
      
      Service >sge_qmaster< is now available.
      
      Hit <RETURN> to continue >>
      
      Grid Engine TCP/IP service >sge_execd<
      --------------------------------------
      
      Using the service
      
         sge_execd
      
      for communication with Grid Engine.
      
      Hit <RETURN> to continue >>
      

      
      

9. Enter the name of your cell or press Return to use the default.
   The use of Grid Engine system cells is described in Cells.

   Grid Engine cells
   -----------------
   
   Grid Engine supports multiple cells.
   
   If you are not planning to run multiple Grid Engine clusters or if you don't
   know yet what is a Grid Engine cell it is safe to keep the default cell name
   
      default
   
   If you want to install multiple cells you can enter a cell name now.
   
   The environment variable
   
      $SGE_CELL=<your_cell_name>
   
   will be set for all further Grid Engine commands.
   
   Enter cell name [default] >>
   

   If you have decided not to use cells, the installation process displays the following information:

   Using cell >default<.
   Hit <RETURN> to continue >>
   

   
   

10. Specify a spool directory.
    For guidelines on disk space requirements for the spool directory, see Disk Space Requirements. For information on where the spool directory is installed, see Spool Directories Under the Root Directory.

    Grid Engine qmaster spool directory
    -----------------------------------
    
    The qmaster spool directory is the place where the qmaster daemon stores
    the configuration and the state of the queuing system.
    
    The admin user >sgeadmin< must have read/write access
    to the qmaster spool directory.
    
    If you will install shadow master hosts or if you want to be able to start
    the qmaster daemon on other hosts (see the corresponding section in the
    Grid Engine Installation and Administration Manual for details) the account
    on the shadow master hosts also needs read/write access to this directory.
    
    The following directory
    
    [/opt/n1ge6/default/spool/qmaster]
    
    will be used as qmaster spool directory by default!
    
    Do you want to select another qmaster spool directory (y/n) [n] >>
    

    • If you want to accept the default spool directory, press Return to continue.
    • If you do not want to accept the default spool directory, then answer y. In the following example the /my/spool directory is specified as the master host spool directory.

      Do you want to select another qmaster spool directory (y/n) [n] >> y
      
      Please enter a qmaster spool directory now! >>/my/spool
      

      
      

11. Set the correct file permissions.

    Verifying and setting file permissions
    --------------------------------------
    
    Did you install this version with >pkgadd< or did you already
    verify and set the file permissions of your distribution (y/n) [y] >> n
    
    Verifying and setting file permissions
    --------------------------------------
    
    We may now verify and set the file permissions of your Grid Engine
    distribution.
    
    This may be useful since due to unpacking and copying of your distribution
    your files may be unaccessible to other users.
    
    We will set the permissions of directories and binaries to
    
       755 - that means executable are accessible for the world
    
    and for ordinary files to
    
       644 - that means readable for the world
    
    Do you want to verify and set your file permissions (y/n) [y] >> y
    
    Verifying and setting file permissions and owner in >3rd_party<
    Verifying and setting file permissions and owner in >bin<
    Verifying and setting file permissions and owner in >ckpt<
    Verifying and setting file permissions and owner in >examples<
    Verifying and setting file permissions and owner in >install_execd<
    Verifying and setting file permissions and owner in >install_qmaster<
    Verifying and setting file permissions and owner in >mpi<
    Verifying and setting file permissions and owner in >pvm<
    Verifying and setting file permissions and owner in >qmon<
    Verifying and setting file permissions and owner in >util<
    Verifying and setting file permissions and owner in >utilbin<
    Verifying and setting file permissions and owner in >catman<
    Verifying and setting file permissions and owner in >doc<
    Verifying and setting file permissions and owner in >man<
    Verifying and setting file permissions and owner in >inst_sge<
    Verifying and setting file permissions and owner in >bin<
    Verifying and setting file permissions and owner in >lib<
    Verifying and setting file permissions and owner in >utilbin<
    
    Your file permissions were set
    
    Hit <RETURN> to continue >>
    

    
    

12. Specify whether all of your Grid Engine system hosts are located in a single DNS domain.

    Select default Grid Engine hostname resolving method
    ----------------------------------------------------
    
    Are all hosts of your cluster in one DNS domain? If this is
    the case the hostnames
    
       >hostA< and >hostA.foo.com<
    
    would be treated as eqal, because the DNS domain name >foo.com<
    is ignored when comparing hostnames.
    
    Are all hosts of your cluster in a single DNS domain (y/n) [y] >>
    

    1. If all of your Grid Engine system hosts are located in a single DNS domain, then answer y.

       Are all hosts of your cluster in a single DNS domain (y/n) [y] >> y
       
       Ignoring domainname when comparing hostnames.
       
       Hit <RETURN> to continue >>
       

    2. If all of your Grid Engine system hosts are not located in a single DNS domain, then answer n.

       Are all hosts of your cluster in a single DNS domain (y/n) [y] >> n
       
       The domainname is not ignored when comparing hostnames.
       
       
       Hit <RETURN> to continue >>
       
       Default domain for hostnames
       ----------------------------
       
       Sometimes the primary hostname of machines returns the short hostname
       without a domain suffix like >foo.com<.
       
       This can cause problems with getting load values of your execution hosts.
       If you are using DNS or you are using domains in your >/etc/hosts< file or
       your NIS configuration it is usually safe to define a default domain
       because it is only used if your execution hosts return the short hostname
       as their primary name.
       
       If your execution hosts reside in more than one domain, the default domain
       parameter must be set on all execution hosts individually.
       
       Do you want to configure a default domain (y/n) [y] >>
       

    3. Press Return to continue.
       • If you want to specify a default domain, then answer y. In the following example, sun.com is specified as the default domain.

         Do you want to configure a default domain (y/n) [y] >> y
         
         
         Please enter your default domain >> sun.com
         
         Using >sun.com< as default domain. Hit <RETURN> to continue >>
         

       • If you do not want to specify a default domain, then answer n. In the following example, sun.com is specified as the default domain.

         Do you want to configure a default domain (y/n) [y] >> n
         

         
         

13. Press Return to continue.

    Making directories
    ------------------
    
    creating directory: default/common
    creating directory: /opt/n1ge6/default/spool/qmaster
    creating directory: /opt/n1ge6/default/spool/qmaster/job_scripts
    Hit <RETURN> to continue >>
    

    
    

14. Specify whether you want to use classic spooling or Berkeley DB.
    For more information on choosing the spooling mechanism, see Database Server and Spooling Host.

    Setup spooling
    --------------
    Your SGE binaries are compiled to link the spooling libraries
    during runtime (dynamically). So you can choose between Berkeley DB
    spooling and Classic spooling method.
    Please choose a spooling method (berkeleydb|classic) [berkeleydb] >>
    

    1. If you want to specify Berkeley DB spooling, press Return to continue.

       Please choose a spooling method (berkeleydb|classic) [berkeleydb] >>
       
       The Berkeley DB spooling method provides two configurations!
       
       1) Local spooling:
       The Berkeley DB spools into a local directory on this host (qmaster host)
       This setup is faster, but you can't setup a shadow master host
       
       2) Berkeley DB Spooling Server:
       If you want to setup a shadow master host, you need to use
       Berkeley DB Spooling Server!
       In this case you have to choose a host with a configured RPC service.
       The qmaster host connects via RPC to the Berkeley DB. This setup is more
       failsafe, but results in a clear potential security hole. RPC communication
       (as used by Berkeley DB) can be easily compromised. Please only use this
       alternative if your site is secure or if you are not concerned about
       security. Check the installation guide for further advice on how to achieve
       failsafety without compromising security.
       
       Do you want to use a Berkeley DB Spooling Server? (y/n) [n] >>
       

       • If you want to use a Berkeley DB spooling server, enter y.

         Do you want to use a Berkeley DB Spooling Server? (y/n) [n] >> y
         
         Berkeley DB Setup
         
         -----------------
         Please, log in to your Berkeley DB spooling host and execute "inst_sge -db"
         Please do not continue, before the Berkeley DB installation with
         "inst_sge -db" is completed, continue with <RETURN>
         

         Note Do not press Return until you have completed the Berkeley DB installation on the spooling server.

         Follow these steps to set up a Berkeley DB spooling server:

       1. Start a new terminal session or window.
       2. Log in to the spooling server.
       3. Install the software as described in How to Install the Berkeley DB Spooling Server.
       4. After you have installed the software on the spooling server, return to the master installation window, and press Return to continue.
       5. Enter the name of the spooling server. In the following example, vector is the host name of the spooling server.

          Berkeley Database spooling parameters
          -------------------------------------
          
          Please enter the name of your Berkeley DB Spooling Server! >> vector
          

       6. Enter the name of the spooling directory. In the following example, /opt/n1ge6/default/spooldb is the spooling directory.

          Please enter the Database Directory now!
          
          Default: [/opt/n1ge6/default/spooldb] >>
          Dumping bootstrapping information
          Initializing spooling database
          
          Hit <RETURN> to continue >>
          

       • If you do not want to use a Berkeley DB spooling server, enter n.

         Do you want to use a Berkeley DB Spooling Server? (y/n) [n] >> n
         
         
         Hit <RETURN> to continue >>
         
         Berkeley Database spooling parameters
         -------------------------------------
         
         Please enter the Database Directory now, even if you want to spool locally
         it is necessary to enter this Database Directory.
         
         Default: [/opt/n1ge6/default/spool/spooldb] >>
         

         Then specify an alternate directory, or press Return to continue.

         creating directory: /opt/n1ge6/default/spool/spooldb
         Dumping bootstrapping information
         Initializing spooling database
         
         Hit <RETURN> to continue >>
         

    2. If you want to specify classic spooling, then enter classic.

       Please choose a spooling method (berkeleydb|classic) [berkeleydb] >> classic
       
       Dumping bootstrapping information
       Initializing spooling database
       
       
       Hit <RETURN> to continue >>
       

       
       

15. Enter a group ID range.
    For more information, see Group IDs.

    Grid Engine group id range
    --------------------------
    
    When jobs are started under the control of Grid Engine an additional group id
    is set on platforms which do not support jobs. This is done to provide maximum
    control for Grid Engine jobs.
    
    This additional UNIX group id range must be unused group id's in your system.
    Each job will be assigned a unique id during the time it is running.
    Therefore you need to provide a range of id's which will be assigned
    dynamically for jobs.
    
    The range must be big enough to provide enough numbers for the maximum number
    of Grid Engine jobs running at a single moment on a single host. E.g. a range
    like >20000-20100< means, that Grid Engine will use the group ids from
    20000-20100 and provides a range for 100 Grid Engine jobs at the same time
    on a single host.
    
    You can change at any time the group id range in your cluster configuration.
    
    Please enter a range >> 20000-20100
    
    Using >20000-20100< as gid range. Hit <RETURN> to continue >>
    

    
    

16. Verify the spooling directory for the execution daemon.
    For information on spooling, see Spool Directories Under the Root Directory.

    Grid Engine cluster configuration
    ---------------------------------
    
    Please give the basic configuration parameters of your Grid Engine
    installation:
    
       <execd_spool_dir>
    
    The pathname of the spool directory of the execution hosts. User >sgeadmin<
    must have the right to create this directory and to write into it.
    
    Default: [/opt/n1ge6/default/spool] >>
    

    
    

17. Enter the email address of the user who should receive problem reports.
    In this example, the user who will receive problem report is me@my.domain.

    Grid Engine cluster configuration (continued)
    ---------------------------------------------
    
    <administator_mail>
    
    The email address of the administrator to whom problem reports are sent.
    
    It's is recommended to configure this parameter. You may use >none<
    if you do not wish to receive administrator mail.
    
    Please enter an email address in the form >user@foo.com<.
    
    Default: [none] >> me@my.domain
    

    Once you answer this question, the installation process is complete. The system displays several screens of information before the script exits.
    
    The upgrade process uses your existing configuration to customize the installation. Output similar to the following is displayed:

    Creating >act_qmaster< file
    Creating >sgemaster< script
    Creating >sgeexecd< script
    creating directory: /tmp/centry
    Reading in complex attributes.
    Reading in administrative hosts.
    Reading in execution hosts.
    Reading in submit hosts.
    Reading in users:
        User "as114086".
        User "md121042".
    Reading in usersets:
        Userset "defaultdepartment".
        Userset "deadlineusers".
        Userset "admin".
        Userset "bchem1".
        Userset "bchem2".
        Userset "bchem3".
        Userset "bchem4".
        Userset "damtp7".
        Userset "damtp8".
        Userset "damtp9".
        Userset "econ1".
        Userset "staff".
    Reading in calendars:
        Calendar "always_disabled".
        Calendar "always_suspend".
        Calendar "test".
    Reading in projects:
        Project "ap1".
        Project "ap2".
        Project "high".
        Project "low".
        Project "p1".
        Project "p2".
        Project "staff".
    Reading in parallel environments:
        PE "bench_tight".
        PE "make".
    Creating settings files for >.profile/.cshrc<
    

    Caution Do not rename any of the binaries of the distribution. If you use any scripts or tools in your cluster that monitor the daemons, make sure to check for the new names.

    
    

18. Create the environment variables for use with the Grid Engine software.

    Note If no cell name was specified during installation, the value of $SGE_CELL is default.

    • If you are using a C shell, type the following command:

      % source $SGE_ROOT/$SGE_CELL/common/settings.csh
      

    • If you are using a Bourne shell or Korn shell, type the following command:

      $ . $SGE_ROOT/$SGE_CELL/common/settings.sh
      

      
      

19. Install or upgrade the execution hosts.
    There are two ways that you can install the Sun Grid Engine software on your execution hosts: installation or upgrade. If you install the execution hosts, the local spool directory configuration, and some execd parameters will be overwritten. If you upgrade the execution hosts, those files will remain untouched.
    1. To upgrade the software on the execution host, you need to log into each execution host and run the following command:

       # $SGE_ROOT/inst_sge -x -upd
       

    2. To install the software on the execution host:
       • If you only have a few execution hosts, you can install them interactively. You need to log into each execution host, and run the following command:

         # $SGE_ROOT/inst_sge -x
         

         Complete instructions for installing execution hosts interactively are in How to Install Execution Hosts.

       • If you have a large number of execution hosts, you should consider installing them non-interactively.
         Instructions for installing execution hosts in an automated way are in Using the inst_sge Utility and a Configuration Template.
         
         
20. If you have configured load sensors on your execution hosts, you will need to copy these load sensors to the new directory location.
    
    
21. Check your complexes.
    Both the structure of complexes and the rules for configuring complexes have changed. You can use qconf -sc to list your complexes. Review the log file that was generated during the master host upgrade, update.pid. The update.pid file will be placed in the master host spool directory, which is $SGE_ROOT/$SGE_CELL/spool/ by default.
    
    If necessary, you can use qconf -mc to reconfigure your complexes. For details, see Configuring Complex Resource Attributes.
    
    
22. Reconfigure your queues.
    During the upgrade process, a single default cluster queue is created. Within this queue you will find all of your installed execution hosts. It is recommended that you reconfigure your queues. For details, see Configuring Queues.

<< Previous: Installing Sun Grid Engine Software Interactively

Next: Installing Sun Grid Engine >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Verifying Sun Grid Engine Installation

Verifying the Installation

The verification phase includes the following tasks:

• Ensuring that the master daemon is running on the master host
• Ensuring that the daemons are running on all execution hosts
• Ensuring that you can run simple commands
• Submitting test jobs

To ensure that the Grid Engine system daemons are running, look for the sge_qmaster daemon on the master host and the sge_execd daemon on the execution hosts. Once you have verified that the daemons are running, you can try to use commands and prepare to submit jobs.

Note If no cell name was specified during installation, the value of $SGE_CELL is default.

How to Verify That the Daemon Is Running on the Master Host

1. Log in to the master host.
   Look in the file $SGE_ROOT/$SGE_CELL/common/act_qmaster to see if you really are on the master host.
   
   
2. Verify that the master daemon is running.
   • On BSD-based UNIX systems, type the following command:

     % ps -ax | grep sge
     

     You should see output similar to the following example.

     14676 p1 S <  4:47 /gridware/sge/bin/solaris/sge_qmaster
     

   • On systems running a UNIX System 5-based operating system (such as the Solaris Operating System), type the following command:

     % ps -ef | grep sge
     

     You should see output similar to the following example.

     root 439 1 0 Jun 2 ? 3:37 /gridware/sge/bin/solaris/sge_qmaster
     

     
     

3. If you do not see the appropriate string, restart the daemon.
   To start the master host daemon, sge_qmaster:

   # $SGE_ROOT/$SGE_CELL/common/sgemaster  start
   

   
   

4. Continue the verification process.
   After you have verified that the master host and the execution host daemons are running, continue the verification process. See How to Run Simple Commands.

How to Verify That the Daemons Are Running on the Execution Hosts

1. Log in to the execution hosts on which you ran the execution host installation procedure.
   
   
2. Verify that the daemons are running.
   • On BSD-based UNIX systems, type the following command:

     % ps -ax | grep sge
     

     You should see output similar to the following example.

     14688 p1 S <    4:27  /gridware/sge/bin/solaris/sge_execd
     

   • On systems running a UNIX System 5-based operating system (such as the Solaris Operating System), type the following command:

     % ps -ef | grep sge
     

     You should see output similar to the following example.

     root 171 1 0 Jun 22 ? 7:11 /gridware/sge/bin/solaris/sge_execd
     

     
     

3. If you do not see similar output, restart the daemon.

   # $SGE_ROOT/$SGE_CELL/common/sgeexecd  start
   

   
   

4. Continue the verification process.
   After you have verified that the master host and the execution host daemons are running, continue the verification process. See How to Run Simple Commands below for details.

How to Run Simple Commands

If both the necessary daemons are running on the master and execution hosts, the Grid Engine software should be operational. Check by issuing a trial command.

1. Log in to either the master host or another administrative host.
   In your standard search path, make sure to include $SGE_ROOT/bin.
   
   
2. From the command line, type the following command:

   % qconf -sconf
   

   This qconf command displays the current global cluster configuration Basic Cluster Configuration.
   If this command fails, your $SGE_ROOT environment variable is not set correctly.
   

   1. Check whether the environment variables SGE_EXECD_PORT and SGE_QMASTER_PORT are set in the script files, $SGE_ROOT/$SGE_CELL/common/settings.csh or $SGE_ROOT/$SGE_CELL/common/settings.sh.

      Note If no cell name was specified during installation, the value of $SGE_CELL is default.

      • If so, make sure that the environment variables SGE_EXECD_PORT and {{SGE_QMASTER_PORT} are set to the correct value before you try the command again.
      • If not, verify whether your NIS services map contains entries for sge_qmaster and sge_execd.
        If the SGE_EXECD_PORT and SGE_QMASTER_PORT variables are not used in these files, then the services database (/etc/services or the NIS services map for example) on the machine from which you run the command must provide entries for both sge_qmaster and sge_execd. If these entries do not exist, add an entry to the machine's services database, giving it the same value as is configured on the master host.
        
   2. Retry the qconf command.
      
      
3. Try to submit test jobs.

How to Submit Test Jobs

Before you start submitting batch scripts to the Grid Engine system, check to see whether your site's standard shell resource files (.cshrc, .profile, or .kshrc) as well as your personal resource files contain commands such as stty. Batch jobs do not have a terminal connection by default, and therefore calls to stty result in an error.

1. Log in to the master host.
   
   
2. Type the following command.

   % rsh <exec-host-name> date
   

   The exec-host-name refers to one of the already installed execution hosts. You should try this test on all execution hosts if your login or home directories differ from host to host. The rsh command should give you output similar to the date command run locally on the master host. If any additional lines contain error messages, you must fix the cause of the errors before you can run a batch job successfully.
   
   For all command interpreters, you can check on an actual terminal connection before you run a command such as stty.
   The following is an example of a Bourne shell script to test the terminal connection.

   tty -s 
   if [ $? = 0 ]; then
      stty erase ^H
   fi
   

   
   The following example shows C shell syntax.

   tty -s
   if ( $status = 0 ) then
      stty erase ^H
   endif
   

   
   

3. Submit one of the sample scripts contained in the $SGE_ROOT/examples/jobs directory.

   % qsub $SGE_ROOT/examples/jobs/simple.sh
   

   
   

4. Use the qstat command to monitor the job's behavior.
   For more information about submitting and monitoring batch jobs, see Submitting Batch Jobs.
   
   
5. After the job finishes executing, check your home directory for the redirected stdout/stderr files script-name.ejob-id and script-name.ojob-id.
   The job-id is a consecutive unique integer number assigned to each job.

In case of problems, see Improving Grid Engine Performance.

<< Previous: Installing Sun Grid Engine Software Interactively

Next: Installing Sun Grid Engine >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Automating the Installation Process

This section describes how you can automate the software installation process for the following reasons:

• To install the Grid Engine software on many hosts

• To install the Grid Engine software without user interaction

This section consists of the following topics:

About Automatic Installation

You can use the $SGE_ROOT/inst_sge utility to install and uninstall Sun Grid Engine master hosts, execution hosts, shadow host and Berkeley DB spooling server hosts. You can also use this utility to backup automatically the Sun Grid Engine configuration and accounting data.

Note Using the Berkeley DB Spooling Server host does not provide high availability, and it has no authentication mechanism. It should only be used on a closed network with fully trusted users.

You can use the inst_sge utility in interactive mode to supplant any of the commands that were described in Installing the Grid Engine Software Interactively.

To simplify automatic installation and backup processes, use the configuration templates that are located in the $SGE_ROOT/util/install_modules directory.

The automatic installation requires no user interaction. No messages are displayed on the terminal during the installation.

When the installation finishes, a message indicates where the installation log file resides. The name of the installation log file format is install_hostname_timestamp.log. Normally, you can find information about errors during installation in this file. In case of serious errors though, the installation script might not be able to move the log file into the spool directory. In this situation, the log file is placed in the /tmp directory.

Special Considerations

The first step in performing an automatic installation is to set up a configuration file. You can find configuration file templates in the $SGE_ROOT/util/install_modules directory. Consider the following as you plan your automatic installation:

• To use automatic installation on remote hosts, the root user must be able to access those hosts through rsh or ssh without supplying a password.
• For local spooling, that is, spooling on the master host, no special configuration is needed. However, the directory where the spooling occurs must not be on an NFS version 3 volume. You may use an NFS version 4 volume for local spooling.
• To run the Berkeley DB spooling server on a host other than the master host, you must install and configure RPC services on this separate host.

To perform this step manually before you start the automatic installation, use the following command:

./inst_sge -db

You can also use the following command to install automatically the Berkeley DB Spooling Server:

% ./inst_sge -db -m -x -auto <full-path-to-configuration-file>

This command checks the SPOOLING_SERVER entry within the configuration file and starts the Berkeley DB installation on the server host.

Note If you start the automatic installation on the master host, the entire cluster can be installed with one command. The automatic installation script accesses the remote hosts through rsh or ssh and starts the installation remotely. This process requires a well-configured configuration file, which each host must be able to read. That file should be installed on each host or shared through NFS.

Using the inst_sge Utility and a Configuration Template

To automate system installation, use the inst_sge utility in combination with a configuration file. See Configuration File Templates.

Note You cannot use the auto installation procedure to install remotely a Windows execution host. You must run the auto installation procedure directly on the Windows execution host.

How to Automate the Master Host Installation

Before You Begin

You need to complete the planning process as outlined in Planning the Installation.

In addition, you need to be able to connect to each of the remote hosts using the rsh or ssh commands, without supplying a password. If this type of access is not allowed on your network, you cannot use this method of installation.

Steps

1. Create a copy of the configuration template, $SGE_ROOT/util/install_modules/inst_template.conf.

   # cd $SGE_ROOT/util/install_modules
   # cp inst_template.conf my_configuration.conf
   

   
   

2. Edit your configuration template, using the values from the worksheet you completed in Planning the Installation.
   The configuration file template includes liberal comments to help you decide where appropriate information belongs. See Configuration File Templates.
   
   
3. Log in as root on the system that you want to be the Sun Grid Engine master host.
   
   
4. Create the $SGE_ROOT directory.
   The $SGE_ROOT directory is the root directory of the Sun Grid Engine software hierarchy, for example /opt/sge62.
   
   
5. Go to the $SGE_ROOT directory and start the installation.

   # cd $SGE_ROOT
   # ./inst_sge -m -auto <full-path-to-configuration-file>
   

The -m option starts the master host installation and installs the master daemon on the local machine. In addition, the -auto option sets up any remote hosts, as specified in the configuration file.

Note You cannot install remotely a master host. You must always install a master host locally.

To prevent data loss or destroying an already installed cluster, the automatic installation terminates if the configured $SGE_CELL directory or the configured Berkeley DB spooling directory already exists. If the installation terminates, the script displays the reason for the termination on the screen.

A log file of the master installation is created in the $SGE_ROOT/default/spool/qmaster directory. The file name is created using the format install_hostname_date_time.log.

Tip You can also combine options if you want to perform multiple installations with one command. For example, the following command installs the master daemon on the local machine and installs all execution hosts that are configured in the configuration file:

./inst_sge -m -x -auto <full-path-to-configuration-file>

a. Wait for notification that the installation has completed.

b. When the automatic installation exits successfully, it displays a message similar to the following:

The Install log can be found in the 
{{/opt/sge62/spool/install_myhost_30mar2007_090152.log}} file.

The installation log file includes any script or error messages that were generated during installation. If the qmaster_spooling_dir directory exists, the log files will be in that directory. If the directory does not exist, the log files will be in the /tmp directory.

Troubleshooting If you do not want your execution hosts to spool locally, be sure to set EXECD_SPOOL_DIR_LOCAL="", with no space between the double quotes ("").

Automating Other Installations Through a Configuration File

In addition to installing the master host, you can perform a variety of other automatic installations using a similar process. The actual form of the inst_sge command differs slightly, and different sections of the configuration file apply. This section provides some examples.

• To install a shadow host, use the following form of the command:

  inst_sge -sm -auto <full-path-to-configuration-file>
  

  Tip To install more than one shadow host, enter the host names in the <SHADOW_HOST> parameter section within the configuration file.

• You can install a separate execution host installation if the master host was installed without identified compute hosts or if you need to add additional compute hosts. For the execution host installation, you also need to have a configuration file.
  
  To install all configured execution hosts, use the following form of the command:

  inst_sge -x -auto <full-path-to-configuration-file>
  

• To install the Berkeley database server, use the following form of the command:

  inst_sge -db -auto <full-path-to-configuration-file>
  

See Configuration File Templates.

Automatic Installation With Increased Security (CSP)

The automatic installation also supports the Certificate Security Protocol (CSP) mode described in Installing the Increased Security Features. To use the CSP security mode, you must fill out the CSP parameters of the template files. The parameters are as follows:

# This section is used for csp installation mode.
# CSP_RECREATE recreates the certs on each installation, if true.
# In case of false, the certs will be created, if not existing.
# Existing certs won't be overwritten. (mandatory for csp install)
CSP_RECREATE="true"

# The created certs won't be copied, if this option is set to false
# If true, the script tries to copy the generated certs. This
# requires passwordless ssh/rsh access for user root to the
# execution hosts
CSP_COPY_CERTS="false"

# csp information, your country code (only 2 characters)
# (mandatory for csp install)
CSP_COUNTRY_CODE="DE"

# your state (mandatory for csp install)
CSP_STATE="Germany"

# your location, eg. the building (mandatory for csp install)
CSP_LOCATION="Building"

# your organisation (mandatory for csp install)
CSP_ORGA="Organisation"

# your organisation unit (mandatory for csp install)
CSP_ORGA_UNIT="Organisation_unit"

# your email (mandatory for csp install)
CSP_MAIL_ADDRESS="name@yourdomain.com"

To start the installation, type the following command:

inst_sge -m -csp -auto template-file-name

Note Certificates are created during the installation process. These certificates have to be copied to each host of the installed cluster. The installation process can do this for you; however, you need to perform the following steps to allow the installation process appropriate permissions to copy the certificates: Use rsh/rcp or ssh/scp on each host. Provide the root user with access to each host over ssh or rsh, without entering a password.

Automatic Uninstallation

You can also uninstall hosts automatically.

Note Uninstall all compute hosts before you uninstall the master host. If you uninstall the master host first, you have to uninstall all execution hosts manually.

To ensure that you have a clean environment, always source the $SGE_ROOT/$SGE_CELL/common/settings.csh file before proceeding.

Uninstalling Execution Hosts

During the execution host uninstallation, all configuration information for the targeted hosts is deleted. The uninstallation attempts to stop the exec hosts in a graceful manner.

First, the queue instances associated with the target host of the uninstallation will be disabled, so that new jobs will not be started. Then, in sequence, the following actions are done on each of the running jobs: checkpoint the job; reschedule the job; do forced rescheduling of the job.

At this point, the queue instance will be empty, and the execution daemon will be shut down, then the configuration, global spool directory or local spool directory will be removed.

The configuration file template has a section for identifying hosts that can be uninstalled automatically. Look for this section:

# Remove this execution hosts in automatic mode 
EXEC_HOST_LIST_RM="host1 host2 host3 host4"

Every host in the EXEC_HOST_LIST_RM list will be automatically removed from the cluster.

To start the automatic uninstallation of execution hosts, type the following command:

% ./inst_sge -ux -auto <full-path-to-configuration-file>

Uninstalling the Master Host

The master host uninstallation removes all of the Sun Grid Engine configuration files. After the uninstallation procedure completes, only the binary files remain. If you think that you will need the configuration information after the uninstallation, perform a backup of the master host. The master host uninstallation supports both interactive and automatic mode.

To start the automatic uninstallation of the master host, type the following command:

% ./inst_sge -um -auto <full-path-to-configuration-file>

This command performs the same procedure as in interactive mode, except the user is not prompted for confirmation of any steps and all terminal output is suppressed. Once the uninstall process is started, it cannot be stopped.

Uninstalling the Shadow Host

To start the automatic uninstallation of the shadow host, type the following command:

% ./inst_sge -usm -auto <full-path-to-configuration-file>

Automatic Backup

The automatic backup procedure backs up configuration and accounting data in much the same way as the interactive backup procedure. You can run the automatic backup procedure as a cron job if you want to schedule unattended or periodic backups. The automatic backup requires a configuration file, for which a template is located in the $SGE_ROOT/util/install_modules/backup_template.conf file.

Comments within the configuration file template indicate what values to use for your environment.

Starting an Automatic Backup

After you set up the configuration file, type the following command to start the automatic backup:

% ./inst_sge -bup -auto <full-path-to-configuration-file>

To prevent overwriting existing backup files, a date/time combination is added to the end of the backup directory name that is specified in the configuration file.

Example - Backup Configuration File

#--------------------------------------------------- 
# Autobackup Configuration File Template 
#--------------------------------------------------- 
# Please, enter your SGE_ROOT here (mandatory) 
SGE_ROOT="/opt/gridengine" 
# Please, enter your SGE_CELL here (mandatory) 
SGE_CELL="default" 
# Please, enter your Backup Directory here 
# After backup you will find your backup files here (mandatory) 
# The autobackup will add a time /date combination to this dirname 
# to prevent an overwriting! 
BACKUP_DIR="/opt/backups/ge_backup" 
# Please, enter true to get a tar/gz package 
# and false to copy the files only (mandatory) 
TAR="true" 
# Please, enter the backup file name here. (mandatory) 
BACKUP_FILE="backup.tar"

Troubleshooting Automatic Installation and Uninstallation

The following errors might be encountered during auto-installation:

If your network does not allow user root to have permissions to connect to other hosts through rsh or ssh without asking for a password, the automatic installation will not work remotely. In this case, log in to the host and use the following command to start the automatic installation locally on each host:

% ./inst_sge -x -noremote -auto /tmp/install_config_file.conf

Supplemental Information

• Configuration File Templates

Index

Configuration File Templates

Configuration file templates are located in the $SGE_ROOT/util/install_modules directory.

Example - Configuration File

#-------------------------------------------------
# SGE default configuration file
#-------------------------------------------------

# Use always fully qualified pathnames, please

# SGE_ROOT Path, this is basic information
#(mandatory for qmaster and execd installation)
SGE_ROOT="/opt/n1ge61"

# SGE_QMASTER_PORT is used by qmaster for communication
# Please enter the port in this way: 1300
# Please do not this: 1300/tcp
#(mandatory for qmaster installation)
SGE_QMASTER_PORT="6444"

# SGE_EXECD_PORT is used by execd for communication
# Please enter the port in this way: 1300
# Please do not this: 1300/tcp
#(mandatory for qmaster installation)
SGE_EXECD_PORT="6445"

# CELL_NAME, will be a dir in SGE_ROOT, contains the common dir
# Please enter only the name of the cell. No path, please
#(mandatory for qmaster and execd installation)
CELL_NAME="default"

# ADMIN_USER, if you want to use a different admin user than the owner,
# of SGE_ROOT, you have to enter the user name, here
# Leaving this blank, the owner of the SGE_ROOT dir will be used as 
admin user
ADMIN_USER=""

# The dir, where qmaster spools this parts, which are not spooled by DB
#(mandatory for qmaster installation)
QMASTER_SPOOL_DIR="/opt/n1ge61/default/spool/qmaster"

# The dir, where the execd spools (active jobs)
# This entry is needed, even if your are going to use
# berkeley db spooling. Only cluster configuration and jobs will
# be spooled in the database. The execution daemon still needs a spool
# directory  
#(mandatory for qmaster installation)
EXECD_SPOOL_DIR="/opt/n1ge61/default/spool"

# For monitoring and accounting of jobs, every job will get
# unique GID. So you have to enter a free GID Range, which
# is assigned to each job running on a machine.
# If you want to run 100 Jobs at the same time on one host you
# have to enter a GID-Range like that: 16000-16100
#(mandatory for qmaster installation) 
GID_RANGE="20000-20100"

# If SGE is compiled with -spool-dynamic, you have to enter here, which
# spooling method should be used. (classic or berkeleydb)
#(mandatory for qmaster installation)
SPOOLING_METHOD="berkeleydb"

# Name of the Server, where the Spooling DB is running on
# if spooling methode is berkeleydb, it must be "none", when
# using no spooling server and it must containe the servername
# if a server should be used. In case of "classic" spooling,
# can be left out
DB_SPOOLING_SERVER="none"

# The dir, where the DB spools
# If berkeley db spooling is used, it must contain the path to
# the spooling db. Please enter the full path. (eg. /tmp/data/spooldb)
# Remember, this directory must be local on the qmaster host or on the
# Berkeley DB Server host. No NSF mount, please
DB_SPOOLING_DIR="/opt/n1ge61/default/spooldb"

# A List of Host which should become admin hosts
# If you do not enter any host here, you have to add all of your hosts
# by hand, after the installation. The autoinstallation works without
# any entry
ADMIN_HOST_LIST="host1"

# A List of Host which should become submit hosts
# If you do not enter any host here, you have to add all of your hosts
# by hand, after the installation. The autoinstallation works without
# any entry
SUBMIT_HOST_LIST="host1"

# A List of Host which should become exec hosts
# If you do not enter any host here, you have to add all of your hosts
# by hand, after the installation. The autoinstallation works without
# any entry
# (mandatory for execution host installation)
EXEC_HOST_LIST="host1"

# The dir, where the execd spools (local configuration)
# If you want configure your execution daemons to spool in
# a local directory, you have to enter this directory here.
# If you do not want to configure a local execution host spool directory
# please leave this empty
EXECD_SPOOL_DIR_LOCAL=""

# If true, the domainnames will be ignored, during the hostname resolving
# if false, the fully qualified domain name will be used for name resolving
HOSTNAME_RESOLVING="true"

# Shell, which should be used for remote installation (rsh/ssh)
# This is only supported, if your hosts and rshd/sshd is configured,
# not to ask for a password, or promting any message.
SHELL_NAME="rsh"

# Enter your default domain, if you are using /etc/hosts or NIS configuration
DEFAULT_DOMAIN="none"

# If a job stops, fails, finnish, you can send a mail to this adress
ADMIN_MAIL="my.name@sun.com"

# If true, the rc scripts (sgemaster, sgeexecd, sgebdb) will be added,
# to start automatically during boottime
ADD_TO_RC="true"

#If this is "true" the file permissions of executables will be set to 755
#and of ordinary file to 644.  
SET_FILE_PERMS="true"

# This option is not implemented, yet.
# When a exechost should be uninstalled, the running jobs will be rescheduled
RESCHEDULE_JOBS="wait"

# Enter a one of the three distributed scheduler tuning configuration sets
# (1=normal, 2=high, 3=max)
SCHEDD_CONF="1"

# The name of the shadow host. This host must have read/write permission
# to the qmaster spool directory
# If you want to setup a shadow host, you must enter the servername
# (mandatory for shadowhost installation)
SHADOW_HOST="hostname"

# Remove this execution hosts in automatic mode
# (mandatory for unistallation of executions hosts)
EXEC_HOST_LIST_RM="host2 host3 host4"

# This is a Windows specific part of the auto isntallation template
# If you going to install windows executions hosts, you have to enable the
# windows support. To do this, please set the WINDOWS_SUPPORT variable
# to "true". ("false" is disabled)
# (mandatory for qmaster installation, by default WINDOWS_SUPPORT is
# disabled)
WINDOWS_SUPPORT="false"

# Enabling the WINDOWS_SUPPORT, recommends the following parameter.
# The WIN_ADMIN_NAME will be added to the list of SGE managers.
# Without adding the WIN_ADMIN_NAME the execution host installation
# won't install correctly.
# WIN_ADMIN_NAME is set to "Administrator" which is default on most
# Windows systems. In some cases the WIN_ADMIN_NAME can be prefixed with
# the windows domain name (eg. DOMAIN+Administrator)
# (mandatory for qmaster installation)
WIN_ADMIN_NAME="Administrator"

# This parameter set the number of parallel installation processes.
# The prevent a system overload, or exeeding the number of open file
# descriptors the user can limit the number of parallel install processes.
# eg. set PAR_EXECD_INST_COUNT="20", maximum 20 parallel execd are installed.
PAR_EXECD_INST_COUNT="20"

<< Previous: Installing Sun Grid Engine Software Interactively

Next: Installing Sun Grid Engine Software Interactively

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Installing SMF Services

The Service Management Facility (SMF) is a new feature in Solaris 10. It provides a unified model for controlling services, replaces RC scripts, handles service dependencies, provides better service availability, and speeds up boot process. If you do not use at least Version 10 of the Solaris OS in your cluster, or you do not plan to use SMF, continue with Installing Sun Grid Engine Software Interactively.

Note SMF is now the default for the hosts that run at least Version 10 of the Solaris OS. If you want to use the old behavior (RC files) for the Solaris hosts, you need to start the installation with the -nosmf option. Use the following command: ./inst_sge -x -nosmf

Installing SMF services includes the following topics:

Why Install SMF Services?

SMF provides a unified administrative model of the persistent services. It solves many challenges of the previous approaches. All services have a common place for log files. Persistent services are automatically restarted on failure. For more information, see SMF documentation.

Additional Setup Required

If you want unprivileged users to use SMF services, you should create a role sge_admin. Assign this role to the users who should be able to manipulate the Grid Engine SMF services as described here.

Then, you can simply answer y when prompted to use SMF during the installation.

How Do SMF Services Compare to the Normal Services?

The biggest difference between SMF and normal services is that SMF does not consider kill -9 to be a correct service shutdown. SMF interprets kill -9 to restart the service.

Within the SMF framework, a service is uniquely identified by its fault resource management identifier (FMRI).

qmaster Daemon

Service name (FMRI) is svc:/application/sge/qmaster:$SGE_CLUSTER_NAME.

¹ - Restart the daemon if RC scripts were installed

shadowd Daemon

Service name (FMRI) is svc:/application/sge/shadowd:$SGE_CLUSTER_NAME.

¹ - Restart the daemon if RC scripts were installed

execd Daemon

Service name (FMRI) is svc:/application/sge/execd:$SGE_CLUSTER_NAME.

¹ - Restart the daemon if RC scripts were installed

Berkeley RPC Server

Service name (FMRI) is svc:/application/sge/bdb:$SGE_CLUSTER_NAME.

¹ - Restart the server if RC scripts were installed

dbwriter Software

Service name (FMRI) is svc:/application/sge/dbwriter:$SGE_CLUSTER_NAME.

¹ - Restart the dbwriter if RC scripts were installed

Additionally you may use new SMF interfaces to interact with services. For more information, see the svcadm(1M) man page. New actions: Action Command Start service temporary svcadm enable -t FMRI Start service permanently (across reboots) svcadm enable FMRI Stop service temporary svcadm disable -t FMRI Stop service permanently (across reboots) svcadm disable FMRI Restart service svcadm reboot FMRI

<< Previous: Installing Sun Grid Engine Software Interactively

Next: Verifying Sun Grid Engine Installation >>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Installing a JMX-Enabled System

The JMX agent functionality enables access to a subset of sge_qmaster functionality via the JMX protocol. For Sun Grid Engine 6.2, the main purpose of the JMX agent is to provide an interface between the SDM Grid Engine adapter and the Sun Grid Engine system.

Additional Setup Required

The steps required to set up the JMX agent feature of Grid Engine are similar to the standard setup. You generally follow the instructions in Planning the Installation, Loading the Distribution Files on a Workstation, How to Install the Master Host, How to Install Execution Hosts and Registering Administration Hosts.
However, you have to perform a few additional tasks:

• Generating necessary configuration files on the master host by calling the installation script with the -jmx flag and depending on the JMX specific installation settings the optional generation of certificates, keys and keystore files.
• Optional distribution of security relevant files to the execution and submit hosts using a secure method such as ssh
• Generating user keys, certificates and keys automatically, after master installation
• Adding new users
• Tweaking of JMX-specific files

How to Install a JMX Agent-Enabled System

Install the Grid Engine software as outlined in Performing an Interactive Installation, with the following exception: use the additional flag -jmx when invoking the qmaster installation scripts.

To install a JMX agent enabled system do the following:

1. Change the master host installation procedure.
   Type the following command and respond to the prompts from the installation script.

   # ./install_qmaster -jmx [-csp]
   

2. Supply the following information to generate necessary configuration files and optionally the certificates, keys and keystores:
   • JMX agent options
     • JAVA_HOME (mandatory)
     • Additional JVM arguments (optional)
     • JMX MBean server port (mandatory)
     • JMX ssl authentication (default: true)
     • JMX ssl client authentication (default: true)
     • JMX ssl server keystore path
       (/var/sgeCA/{sge_qmaster| port$SGE_QMASTER_PORT}/$SGE_CELL/private/keystore)
     • JMX ssl server keystore password
   • Optional certificate specific options, if there is no CA available
     • Two-letter country code, for example, US for the United States
     • State
     • Location, such as a city
     • Organization
     • Organizational unit
     • CA email address
       As the installation proceeds, several JMX specific configuration files are installed:
       **jvm_threads is set to 1 instead of 0 if JMX is enabled in $SGE_ROOT/$SGE_CELL/common/bootstrap:

       ...
       jvm_threads             1
       ...
       

   • Several JMX agent specific configuration files are generated as:

     $SGE_ROOT/$SGE_CELL/common/jmx/jaas.config
     $SGE_ROOT/$SGE_CELL/common/jmx/java.policy
     $SGE_ROOT/$SGE_CELL/common/jmx/jmxremote.access
     $SGE_ROOT/$SGE_CELL/common/jmx/jmxremote.password
     $SGE_ROOT/$SGE_CELL/common/jmx/logging.properties
     $SGE_ROOT/$SGE_CELL/common/jmx/management.properties
     

     For a detailed description, see the comments in the files and the description below.
     
     Optionally the Certificate Authority is created. The directories that contain information relevant to security are as follows:

     • The publicly accessible CA and daemon certificate are stored in $SGE_ROOT/$SGE_CELL/common/sgeCA
     • The publicly accessible user certificates are stored in $SGE_ROOT/$SGE_CELL/common/sgeCA/usercerts
     • The corresponding private keys and keystore are stored in /var/sgeCA/{sge_qmaster| port$SGE_QMASTER_PORT}/$SGE_CELL/private
     • User keys, certificates and keystore are stored in /var/sgeCA/{sge_qmaster| port$SGE_QMASTER_PORT}/$SGE_CELL/userkeys/$USER
       
3. The script prompts you for site information.
   
   
4. Confirm whether the information you supplied is correct.
   
   
5. Continue the installation.
   After the security-related setup of the master host is finished, the script prompts you to continue with the rest of the installation procedure, as in the following example:

   SGE startup script
   --------------------
   
   Your system wide SGE startup script is installed as:
   
        "/scratch2/eddy/sge_sec/$SGE_CELL/common/sgemaster"
   
   Hit Return to continue >>
   

   
   

6. Proceed to the next task.
   For more information, see How to Generate Certificates and Private Keys for Users.

How to Generate Certificates, Private Keys and Keystores for Users

To use the CSP-secured system, the user must have access to a user-specific certificate, private key and keystore. Usually the steps outlined in How to Generate Certificates and Private Keys for Users are performed. After that the following procedure can be done to generate the corresponding keystore files for the users.

1. As root on the master host run the following command:

   # $SGE_ROOT/util/sgeCA/sge_ca -userks [-kspwf <kspwf-file>]
   

   
   

2. Confirm that the creation has been successful.

   # ls -lR /var/sgeCA/port$SGE_QMASTER_PORT/$SGE_CELL/userkeys
   /var/sgeCA/port$SGE_QMASTER_PORT/$SGE_CELL/userkeys/:
   total 8
   drwx------   2 eddy 	staff        512 Mar 13 11:33 eddy
   drwx------   2 sarah 	staff        512 Mar 13 11:33 sarah
   drwx------   2 leo 	staff        512 Mar 13 11:33 leo
   
   /var/sgeCA/port$SGE_QMASTER_PORT/$SGE_CELL/userkeys/eddy:
   total 16
   -rw-------   1 eddy staff       1586 Mar 13 11:32 cert.pem
   -rw-------   1 eddy staff        891 Mar 13 11:32 key.pem
   -rw-------   1 eddy staff       3031 Mar 13 11:33 keystore
   -rw-------   1 eddy staff       1024 Mar 13 11:32 rand.seed
   -rw-------   1 eddy staff        818 Mar 13 11:32 req.pem
   ...
   

Checking Certificates, Keys and Keystores

To confirm that these files contain the intended information, use the following commands:

• To display a certificate:

  $SGE_ROOT/util/sgeCA/sge_ca -print <whereever>/cert.pem
  

• To display a key:

  $SGE_ROOT/util/sgeCA/sge_ca -printkey <whereever>/key.pem
  

To display a keystore or truststore:

$JAVA_HOME/bin/keytool -list -v -keystore <whereever>/keystore

The keystore password must be entered to see all entries otherwise only the certificates are visible. For more information, see Java keytool documentation.

JMX Configuration Files

The following configuration files are installed into $SGE_ROOT/$SGE_CELL/common/jmx and are explained in detail here. Manual modification is usually not necessary and the preinstalled configurations should be sufficient.

jaas.config

Before using the JMX interface, you must run a special authentication against sge_qmaster. This process adds the correct principle that gives you the necessary role to access the JMX interfaces in read-only or read-write mode. The responsible section in the jaas.config file is named GridwareConfig or TestConfig (for testing only).
In general, the jaas.config file defines which login modules are used for which application case. The choice of the login module is defined either in a configuration file like management.properties or programmatically.
The jaas.config file contains different sections and allows the replacement of the authentication mechanism, e.g. authentication via unix pam or via LDAP (see the GridwareConfig section and TestConfig section below). The different modules can be stacked. For a general overview of Jaas, see http://java.sun.com/developer/technicalArticles/Security/jaasv2/. Here the procedure consists of two steps in the GridwareConfig section:

• Authenticate the user (for example with keystore or Unix login or with LDAP).
• In the JGDILogin module, add the JMXPrincipal that gives the defined role to the user. This role is used later in the jmx.access file to check if the user has read-only or read-write access.

/*
 * Default login configuration for qmaster's jmx server
 */
GridwareConfig {

    /**
     *  Accepts all clients which have a certificate which is signed with
     *  the CA certificate.
     */
    com.sun.grid.security.login.GECATrustManagerLoginModule requisite
         caTop="${com.sun.grid.jgdi.caTop}";

    /*
     *  Accepts all clients which has a valid username/password.
     *
     *  The username/password validation is done with the authuser binary (included
     *  in the Grid Engine distribution, $SGE_ROOT/utilbin/$ARCH/authuser).
     *
     *  ATTENTION: The authuser binary needs the suid bit. It does not work if grid
     *  engine is installed on a nosuid file system.
     *
     */
    com.sun.grid.security.login.UnixLoginModule requisite
        sge_root="${com.sun.grid.jgdi.sgeRoot}"
        auth_method="system";

    /*
     * Username password authentication against LDAP.
     *
     * Alternative username/password authentication if
     * com.sun.grid.security.login.UnixLoginModule is not working.
     *
     * The LDAP specific parameters have to be adjusted to the local requirements
     * For details please have a look at the LdapLoginModule javadocs.
     *
     * ATTENTION: The LdapLoginModule is only available in java 6. The
     * parameter libjvm_path must point to a java 6 jvm
     * (qconf -sconf | grep libjvm_path)
     */
    /*
    com.sun.security.auth.module.LdapLoginModule requisite
        userProvider="ldap://sun-ds/ou=people,dc=sun,dc=com"
        userFilter="(&(uid={USERNAME})(objectClass=inetOrgPerson))"
        useSSL=false;
    */

    /*
     *  The JGDILoginModule adds a JGDIPrincipal to the subject. The username of
     *  the JGDIPrincipal is the name of the first trusted principal. This name
     *  treated as username for gdi communication.
     *  For each login a new jgdi session id is created.
     *
     *  In the jmxremote.access file users who can access the system are defined
     *  Any principal matching these entries is given the corresponding role.
     *  Usually a jmxPrincipal is defined to give a user access to the system.
     *  (e.g. com.sun.grid.security.login.UserPrincipal = xyz &
     *        jmxPrincipal="controlRole" gives user xyz access under controlRole
     *  )
     */
    com.sun.grid.jgdi.security.JGDILoginModule optional
        trustedPrincipal="com.sun.grid.security.login.UserPrincipal"
        trustedPrincipal1="com.sun.security.auth.UserPrincipal"
        jmxPrincipal="controlRole";
};

/*
 *  TestConfig accepts any user. Only for testing
 */
TestConfig {

    com.sun.grid.security.login.TestLoginModule requisite;

    com.sun.grid.jgdi.security.JGDILoginModule optional
        trustedPrincipal="com.sun.grid.security.login.UserPrincipal"
        jmxPrincipal="controlRole";
};

/*
 *  Mandatory if native jgdi is used with a csp system
 *  (e.g. jgdish in csp mode)
 */
jgdi {
   com.sun.security.auth.module.KeyStoreLoginModule required
                                                    keyStoreURL="file:./keystore"
                                                    debug=false;
};

java.policy

The java.policy file that is used by the JGDIAgent restricts the possibilities of code that can be run in sge_qmaster's JVM.

Usually changes here are only necessary to change the access to a subset of the overall functionality. To tweak the policy settings to your needs it is useful to run the JMX server with security debugging enabled and to consult the generated logging files. (qconf -mconf, additional_jvm_args = -Djavax.net.debug=ssl -Djava.security.debug=access,failure)

/*
**
** with LdapLoginModule
** grant principal com.sun.security.auth.UserPrincipal "controlRole"
**
** with jmxremote.password
** grant principal javax.management.remote.JMXPrincipal "controlRole"
**
*/
grant codeBase "file:${com.sun.grid.jgdi.sgeRoot}/lib/jgdi.jar"  {
   permission java.net.SocketPermission   "*:1024-", "accept,connect";
   permission java.net.SocketPermission   "localhost:1024-", "listen,resolve";
   permission java.lang.RuntimePermission "loadLibrary.jgdi";
   permission java.lang.RuntimePermission "shutdownHooks";
   permission java.lang.RuntimePermission "setContextClassLoader";
   permission javax.security.auth.AuthPermission "createLoginContext.jgdi";
   permission javax.security.auth.AuthPermission "doAs";
   permission javax.security.auth.AuthPermission "getSubject";
   permission java.util.PropertyPermission "*", "read";
   permission java.util.logging.LoggingPermission "control";

   permission java.lang.FilePermission "${com.sun.grid.jgdi.sgeRoot}/${com.sun.grid.jgdi.sgeCell}/common/jmx/-", "read";
   permission java.io.FilePermission "${com.sun.grid.jgdi.sgeRoot}/util/-", "execute";
   permission java.io.FilePermission "${com.sun.grid.jgdi.sgeRoot}/utilbin/-", "execute";
   permission javax.management.MBeanServerPermission "createMBeanServer";
   permission javax.management.MBeanPermission "*", "*";
   permission javax.management.MBeanTrustPermission "register";
   permission java.lang.management.ManagementPermission "monitor";
   permission java.lang.management.ManagementPermission "control";

   permission java.lang.RuntimePermission "setIO";
   permission java.io.FilePermission      "jgdi.stdout", "write";
   permission java.io.FilePermission      "jgdi.stderr", "write";
   permission java.io.FilePermission      "jgdi0.log.lck", "delete";
   permission java.io.FilePermission      "${com.sun.grid.jgdi.sgeRoot}/${com.sun.grid.jgdi.sgeCell}/common/jmx/*", "read";
   permission java.io.FilePermission      "${com.sun.grid.jgdi.sgeRoot}/lib/-", "read";
   permission java.lang.RuntimePermission "accessClassInPackage.sun.management.jmxremote";
   permission java.lang.RuntimePermission "accessClassInPackage.sun.management.resources";
   permission java.lang.RuntimePermission "accessClassInPackage.sun.management";
   permission java.lang.RuntimePermission "accessClassInPackage.sun.rmi.server";
   permission java.lang.RuntimePermission "accessClassInPackage.sun.management.snmp.util";
   permission java.lang.RuntimePermission "accessClassInPackage.sun.rmi.registry";

   permission java.util.PropertyPermission "java.rmi.server.randomIDs", "write";

   permission javax.security.auth.AuthPermission "modifyPrincipals";
   permission javax.security.auth.AuthPermission "createLoginContext.*";
   permission javax.security.auth.AuthPermission "createLoginContext.JMXPluggableAuthenticator";
   permission java.security.SecurityPermission "createAccessControlContext";

   permission javax.management.remote.SubjectDelegationPermission "javax.management.remote.JMXPrincipal.controlRole";
};

grant principal javax.management.remote.JMXPrincipal "controlRole" {
   permission javax.management.MBeanPermission "com.sun.grid.jgdi.management.mbeans.JGDIJMX#*", "*";
   permission javax.management.MBeanPermission "sun.management.*#*", "*";
   permission javax.security.auth.AuthPermission "createLoginContext.jgdi";
   permission javax.security.auth.AuthPermission "doAs";
   permission javax.security.auth.AuthPermission "getSubject";
   permission java.util.PropertyPermission "*", "read";
   permission java.util.PropertyPermission "user.timezone", "read,write";
   permission java.util.logging.LoggingPermission "control";
   permission java.io.FilePermission      "${com.sun.grid.jgdi.sgeRoot}/lib/-", "read";
   permission java.lang.management.ManagementPermission "monitor";
   permission java.net.SocketPermission "*", "resolve";

   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#-[java.lang:type=OperatingSystem]", "isInstanceOf";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#-[java.lang:type=OperatingSystem]", "getAttribute";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#ProcessCpuTime[java.lang:type=OperatingSystem]", "getAttribute";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#Name[java.lang:type=OperatingSystem]", "getAttribute";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#Version[java.lang:type=OperatingSystem]", "getAttribute";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#Arch[java.lang:type=OperatingSystem]", "getAttribute";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#AvailableProcessors[java.lang:type=OperatingSystem]", "getAttribute";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#CommittedVirtualMemorySize[java.lang:type=OperatingSystem]", "getAttribute";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#TotalPhysicalMemorySize[java.lang:type=OperatingSystem]", "getAttribute";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#FreePhysicalMemorySize[java.lang:type=OperatingSystem]", "getAttribute";   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#TotalSwapSpaceSize[java.lang:type=OperatingSystem]", "getAttribute";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#FreeSwapSpaceSize[java.lang:type=OperatingSystem]", "getAttribute";
   permission javax.management.MBeanPermission "javax.management.MBeanServerDelegate#-[JMImplementation:type=MBeanServerDelegate]", "addNotificationListener";
   permission javax.management.MBeanPermission "javax.management.MBeanServerDelegate#-[JMImplementation:type=MBeanServerDelegate]", "isInstanceOf";
   permission javax.management.MBeanPermission "javax.management.MBeanServerDelegate#-[JMImplementation:type=MBeanServerDelegate]", "getMBeanInfo";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#-[java.lang:type=OperatingSystem]", "queryNames";
   permission javax.management.MBeanPermission "java.util.logging.Logging#-[java.util.logging:type=Logging]", "queryNames";
   permission javax.management.MBeanPermission "javax.management.MBeanServerDelegate#-[JMImplementation:type=MBeanServerDelegate]", "queryNames";
   permission javax.management.MBeanPermission "java.util.logging.Logging#-[java.util.logging:type=Logging]", "isInstanceOf";
   permission javax.management.MBeanPermission "java.util.logging.Logging#-[java.util.logging:type=Logging]", "getMBeanInfo";
   permission javax.management.MBeanPermission "com.sun.management.UnixOperatingSystem#-[java.lang:type=OperatingSystem]", "getMBeanInfo";

};

grant {
   permission java.util.logging.LoggingPermission "control";
   permission java.util.PropertyPermission "*", "read";
   permission java.util.PropertyPermission "user.timezone", "write";
   permission java.lang.RuntimePermission "setIO";
   permission java.lang.RuntimePermission "loadLibrary.jgdi";
   permission java.io.FilePermission      "jgdi.stdout", "write";
   permission java.io.FilePermission      "jgdi.stderr", "write";
   permission java.io.FilePermission      "${com.sun.grid.jgdi.sgeRoot}/lib/-", "read";
   permission java.io.FilePermission      "${com.sun.grid.jgdi.sgeRoot}/util/arch", "execute";
   permission java.io.FilePermission      "${com.sun.grid.jgdi.sgeRoot}/utilbin/-", "execute";
   permission javax.security.auth.AuthPermission "modifyPrincipals";
   permission java.io.FilePermission "${com.sun.grid.jgdi.caTop}", "read";
   permission java.io.FilePermission "${com.sun.grid.jgdi.caTop}/cacert.pem", "read";
   permission java.io.FilePermission "${com.sun.grid.jgdi.caTop}/ca-crl.pem", "read";
   permission java.io.FilePermission "${com.sun.grid.jgdi.caTop}/usercerts/-", "read";
   permission java.io.FilePermission "${com.sun.grid.jgdi.serverKeystore}", "read";
};

/*
grant {
   permission java.security.AllPermission;
};
*/

management.properties

This file describes the general JMX server configuration and the default template looks similar to this example and is usually adapted automatically during the installation process replacing the @@SGE_*@@ variables by concrete values.
The meaning of the @@SGE_*@@ variables is:

• @@SGE_JMX_PORT@@ is the configured JMX port
• @@SGE_JMX_SSL@@ is true or false if SSL shall be enabled for JMX or not
• @@SGE_JMX_SSL_CLIENT@@ is true or false if client authentication is required
• @@SGE_JMX_SSL_KEYSTORE@@ the keystore used for enabled SSL
• @@SGE_JMX_SSL_KEYSTORE_PW@@ the corresponding keystore password
• @@SGE_ROOT@@ the $SGE_ROOT root directory
• @@SGE_CELL@@ the $SGE_CELL name usually 'default'

#####################################################################
#  Default Configuration File for JGDI JMX
#####################################################################
#
# The Management Configuration file (in java.util.Properties format)
# will be read if one of the following system properties is set:
#    -Dcom.sun.grid.jgdi.management.jmxremote.port=<port-number>
# or -Dcom.sun.grid.jgdi.management.config.file=<this-file>
#
# The default Management Configuration file is:
#
#       $SGE_ROOT/{$SGE_CELL|default}/common/jmx/management.properties
#
# ################ Management Agent Port #########################
#
# For setting the JMX RMI agent port use the following line
# com.sun.grid.jgdi.management.jmxremote.port=<port-number>
com.sun.grid.jgdi.management.jmxremote.port=@@SGE_JMX_PORT@@

#####################################################################
#        RMI Management Properties
#####################################################################
#
# If system property -Dcom.sun.grid.jgdi.management.jmxremote.port=<port-number>
# is set then
#     - A MBean server is started
#     - JRE Platform MBeans are registered in the MBean server
#     - RMI connector is published  in a private readonly registry at
#       specified port using a well known name, "jmxrmi"
#     - the following properties are read for JMX remote management.
#
# The configuration can be specified only at startup time.
# Later changes to above system property (e.g. via setProperty method),
# this config file, the password file, or the access file have no effect to the
# running MBean server, the connector, or the registry.
#

#
# ###################### RMI SSL #############################
#
# com.sun.grid.jgdi.management.jmxremote.ssl=true|false
#      Default for this property is true. (Case for true/false ignored)
#      If this property is specified as false then SSL is not used.
#

#For RMI monitoring without SSL use the following line
# com.sun.grid.jgdi.management.jmxremote.ssl=false
com.sun.grid.jgdi.management.jmxremote.ssl=@@SGE_JMX_SSL@@

# com.sun.grid.jgdi.management.jmxremote.ssl.enabled.cipher.suites=<cipher-suites>
#      The value of this property is a string that is a comma-separated list
#      of SSL/TLS cipher suites to enable. This property can be specified in
#      conjunction with the previous property "com.sun.management.jmxremote.ssl"
#      in order to control which particular SSL/TLS cipher suites are enabled
#      for use by accepted connections. If this property is not specified then
#      the SSL RMI Server Socket Factory uses the SSL/TLS cipher suites that
#      are enabled by default.
#

# com.sun.grid.jgdi.management.jmxremote.ssl.enabled.protocols=<protocol-versions>
#      The value of this property is a string that is a comma-separated list
#      of SSL/TLS protocol versions to enable. This property can be specified in
#      conjunction with the previous property "com.sun.management.jmxremote.ssl"
#      in order to control which particular SSL/TLS protocol versions are
#      enabled for use by accepted connections. If this property is not
#      specified then the SSL RMI Server Socket Factory uses the SSL/TLS
#      protocol versions that are enabled by default.
#

# com.sun.grid.jgdi.management.jmxremote.ssl.need.client.auth=true|false
#      Default for this property is false. (Case for true/false ignored)
#      If this property is specified as true in conjunction with the previous
#      property "com.sun.management.jmxremote.ssl" then the SSL RMI Server
#      Socket Factory will require client authentication.
#

#For RMI monitoring with SSL client authentication use the following line
#com.sun.grid.jgdi.management.jmxremote.ssl.need.client.auth=true
com.sun.grid.jgdi.management.jmxremote.ssl.need.client.auth=@@SGE_JMX_SSL_CLIENT@@


#
# ################ RMI User authentication ################
#
# com.sun.grid.jgdi.management.jmxremote.authenticate=true|false
#      Default for this property is true. (Case for true/false ignored)
#      If this property is specified as false then no authentication is
#      performed and all users are allowed all access.
#

# For RMI monitoring without any checking use the following line
# com.sun.grid.jgdi.management.jmxremote.authenticate=false
com.sun.grid.jgdi.management.jmxremote.authenticate=true

#
# ################ RMI Login configuration ###################
#
# com.sun.grid.jgdi.management.jmxremote.login.config=<config-name>
#      Specifies the name of a JAAS login configuration entry to use when
#      authenticating users of RMI monitoring.
#
#      Setting this property is optional - the default login configuration
#      specifies a file-based authentication that uses the password file.
#
#      When using this property to override the default login configuration
#      then the named configuration entry must be in a file that gets loaded
#      by JAAS. In addition, the login module(s) specified in the configuration
#      should use the name and/or password callbacks to acquire the user's
#      credentials. See the NameCallback and PasswordCallback classes in the
#      javax.security.auth.callback package for more details.
#
#      If the property "com.sun.management.jmxremote.authenticate" is set to
#      false, then this property and the password & access files are ignored.
#

# For a non-default login configuration use the following line
# com.sun.grid.jgdi.management.jmxremote.login.config=<config-name>
com.sun.grid.jgdi.management.jmxremote.login.config=GridwareConfig

#
# ################ RMI Password file location ##################
#
# com.sun.grid.jgdi.management.jmxremote.password.file=filepath
#      Specifies location for password file
#      This is optional - default location is
#      $JRE/lib/management/jmxremote.password
#
#      If the property "com.sun.grid.jgdi.management.jmxremote.authenticate" is set to
#      false, then this property and the password & access files are ignored.

# For a non-default password file location use the following line
# com.sun.grid.jgdi.management.jmxremote.password.file=filepath
com.sun.grid.jgdi.management.jmxremote.password.file=@@SGE_ROOT@@/@@SGE_CELL@@/common/jmx/jmxremote.password

#
# ################ RMI Access file location #####################
#
# com.sun.grid.jgdi.management.jmxremote.access.file=filepath
#      Specifies location for access  file
#      This is optional - default location is
#      $JRE/lib/management/jmxremote.access
#
#      If the property "com.sun.management.jmxremote.authenticate" is set to
#      false, then this property and the password & access files are ignored.
#      Otherwise, the access file must exist and be in the valid format.
#      If the access file is empty or non-existent then no access is allowed.
#

# For a non-default access file location use the following line
# com.sun.grid.jgdi.management.jmxremote.access.file=filepath
com.sun.grid.jgdi.management.jmxremote.access.file=@@SGE_ROOT@@/@@SGE_CELL@@/common/jmx/jmxremote.access


# For the JGDI keystore module use this settings for the server keystore and keystore password
com.sun.grid.jgdi.management.jmxremote.ssl.serverKeystore=@@SGE_JMX_SSL_KEYSTORE@@
com.sun.grid.jgdi.management.jmxremote.ssl.serverKeystorePassword=@@SGE_JMX_SSL_KEYSTORE_PW@@

jmx.access

The jmx access file defines which principals are mapped to a special role.

######################################################################
#     Default Access Control File for Remote JMX(TM) Monitoring
######################################################################
#
# Access control file for Remote JMX API access to monitoring.
# This file defines the allowed access for different roles.  The
# password file (jmxremote.password by default) defines the roles and their
# passwords.  To be functional, a role must have an entry in
# both the password and the access files.
#
# Default location of this file is $JRE/lib/management/jmxremote.access
# You can specify an alternate location by specifying a property in
# the management config file $JRE/lib/management/management.properties
# (See that file for details)
#
# The file format for password and access files is syntactically the same
# as the Properties file format.  The syntax is described in the Javadoc
# for java.util.Properties.load.
# Typical access file has multiple  lines, where each line is blank,
# a comment (like this one), or an access control entry.
#
# An access control entry consists of a role name, and an
# associated access level.  The role name is any string that does not
# itself contain spaces or tabs.  It corresponds to an entry in the
# password file (jmxremote.password).  The access level is one of the
# following:
#       "readonly" grants access to read attributes of MBeans.
#                   For monitoring, this means that a remote client in this
#                   role can read measurements but cannot perform any action
#                   that changes the environment of the running program.
#       "readwrite" grants access to read and write attributes of MBeans,
#                   to invoke operations on them, and to create or remove them.
#         This access should be granted to only trusted clients,
#                   since they can potentially interfere with the smooth
#         operation of a running program
#
# A given role should have at most one entry in this file.  If a role
# has no entry, it has no access.
# If multiple entries are found for the same role name, then the last
# access entry is used.
#
#
# Default access control entries:
# o The "monitorRole" role has readonly access.
# o The "controlRole" role has readwrite access.

monitorRole   readonly
controlRole   readwrite

jmx.password

This is also a possible simple authentication mechanism though not recommended. Usually the jaas login module is preferred since it is much more flexible. You can specify a password for the different roles there. If a simple login mechanism is required it is recommended to change management.properties to use TestConfig instead of GridwareConfig, which allows any valid Unix user to connect to JGDI JMX server without a password.

logging.properties

To enable JGDI and JMX logging the delivered logging file has to be adjusted and sge_qmaster or at least the JMX server has to be restarted. The generated logging files default to jgdi0.log, jgdi.stderr and jgdi.stdout in the master spooling directory. The logging can also be influenced by changing the additional_jvm_args configuration to enable additional debugging messages for example.

#
#  Java Logging Configuration for JMX MBean server
#

# Specify the handlers to create in the root logger
# (all loggers are children of the root logger)
# The following creates two handlers

# Per default we log to the console
#handlers = java.util.logging.ConsoleHandler

# Use FileHandler
handlers = java.util.logging.FileHandler

# ------------------------------------------------------------------------------
#  Definition of log levels
# ------------------------------------------------------------------------------
# Set the default logging level for the root logger
.level = INFO
#com.sun.grid.jgdi.JGDI.level = FINE
#com.sun.grid.jgdi.rmi.level = FINE
#com.sun.grid.jgdi.configuration.xml.XMLUtil.level = FINE
#com.sun.grid.jgdi.configuration.ClusterQueueTestCase.level = FINE
#com.sun.grid.jgdi.management.level = FINER
#com.sun.grid.jgdi.event.level = FINER
# For authuser login module debugging
#com.sun.grid.security.login.level = FINER
#com.sun.grid.util.expect.level = FINER

# ------------------------------------------------------------------------------
#  Settings for ConsoleHandler
# ------------------------------------------------------------------------------
# Set the default logging level for new ConsoleHandler instances
java.util.logging.ConsoleHandler.level = INFO

# Set the default formatter for new ConsoleHandler instances
java.util.logging.ConsoleHandler.formatter = com.sun.grid.jgdi.util.SGEFormatter

# ------------------------------------------------------------------------------
#  Settings for FileHandler
# ------------------------------------------------------------------------------
# Set the default logging level for new FileHandler instances
java.util.logging.FileHandler.level = ALL
# qmaster runs in qmaster spool dir, so the file is created there
java.util.logging.FileHandler.pattern=jgdi%u.log
java.util.logging.FileHandler.formatter=com.sun.grid.jgdi.util.SGEFormatter

#
# Possible columns:
#
#   time    timestamp of the log message
#   host    hostname of the log message
#   name    name of the logger
#   thread  id of the thread
#   level   log level (short form)
#   source  class and method name
#   level_long log_level long form
#
com.sun.grid.jgdi.util.SGEFormatter.columns = time thread source level message

#
#  Print the stacktrace of the log record
#
com.sun.grid.jgdi.util.SGEFormatter.withStacktrace=true

#
#  Delimiter between columns
#
com.sun.grid.jgdi.util.SGEFormatter.delimiter = |

Testing and Troubleshooting

To connect to the JMX server jconsole can be used for testing. It is the responsibility of the administrator to allow/disallow access to the system via JMX. To force also client authentication of jconsole the management.properties file must be configured with:

• com.sun.grid.jgdi.management.jmxremote.ssl=true
• com.sun.grid.jgdi.management.jmxremote.ssl.need.client.auth=true

% jconsole -J-Djava.security.manager=java.rmi.RMISecurityManager \
 -J-Djava.security.policy=$SGE_ROOT/util/rmiconsole.policy \
 -J-Djavax.net.ssl.trustStore=<server truststore> \
 [-J-Djavax.net.ssl.keyStore=/<safe>/mykeystore \
  -J-Djavax.net.ssl.keyStorePassword=<mykeystore_pw> \
  -J-Djavax.net.ssl.keyPassword=<mykeystore_pw> ] \
 [-J-Djavax.net.debug=ssl]

where <server truststore> usually is either:
/var/sgeCA/port5322/$SGE_CELL/private/keystore
(only the server certificate is accessible without password)
or a special truststore is made available by the administrator:

keytool -export -alias "root" \ 
        -keystore /var/sgeCA/port$SGE_QMASTER_PORT/$SGE_CELL/private/keystore -rfc -file /tmp/jmxserver.cer

keytool -import -file /tmp/jmxserver.cer -keystore /tmp/truststore
Enter keystore password:  <pwd>
...
Trust this certificate? [no]:  yes
Certificate was added to keystore

The optional arguments are required if client authentication is set to true or for debugging.

The following simple example can be used to connect via JMX and monitor events

% java [-Dcom.sun.grid.jgdi.keyStore=\          /var/sgeCA/port$SGE_QMASTER_PORT/$SGE_CELL/private/keystore \
-Dcom.sun.grid.jgdi.caTop="$SGE_ROOT/$SGE_CELL/common/sgeCA" \
-Djava.util.logging.config.file=util/shell_logging.properties ] \
-cp $SGE_ROOT/lib/juti.jar:$SGE_ROOT/lib/jgdi.jar \
com.sun.grid.jgdi.examples.jmxeventmonitor.Main

The optional arguments can be skipped and serve only to preset the login dialog with useful values. If a connection has been established once a preferences file is written, that is reused afterwards.
To have the correct environment variables set the SGE settings.(c)sh file has to be sourced. To get access to the keystore the command must be run by the admin user in the example above.

For troubleshooting the following settings and files might give some additional insights:

• Messages file in the master spool directory if the JMX server can't be started
• $SGE_ROOT/$SGE_CELL/common/bootstrap to check if jvm_threads is enabled at all
• jgdi* log files in the master spool directory are the main source for finding out the reason for failure analysis
• $SGE_ROOT/$SGE_CELL/common/jmx/logging.properties to enable more detailed logging
• qconf -mconf with an additional_jvm_args parameter
  For example, add these two arguments -Djava.security.debug=all -Djavax.net.debug=ssl to trace any permission and authentication problems.

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Removing the Grid Engine Software

How to Remove the Software Interactively

To remove the software interactively, follow the steps below.

Note Remove the software from the execution hosts before removing it from the master host. If you remove the software from the master host first, you cannot automate the removal of the software from the execution hosts.

1. Ensure that your environment variables are set up properly.

   Note If no cell name was specified during installation, the value of $SGE_CELL is default.

   • If you are using a C shell, type the following command:

     # source $SGE_ROOT/$SGE_CELL/common/settings.csh
     

   • If you are using a Bourne or Korn shell, type the following command:

     # . $SGE_ROOT/$SGE_CELL/common/settings.sh
     

     
     

2. On the master host, issue the $SGE_ROOT/inst_sge -ux command.
   This example uninstalls the execution hosts: host1, host2 and host3.

   # $SGE_ROOT/inst_sge -ux -host "host1 host2 host3"
   

   Note You are not prompted for any information during this process. However, the output from this process will be displayed to the terminal window where you run the command.

   
   

3. (Optional) If you have any shadow master hosts, uninstall them:

   # $SGE_ROOT/inst_sge -usm -host "host4"
   

   
   

4. Uninstall the master host.

   # $SGE_ROOT/inst_sge -um
   

How to Remove the Software Using the inst_sge Utility and a Configuration Template

Unlike the interactive uninstallation method, the automated uninstallation method suppresses output during the process. Also, the automated method requires a properly formatted configuration file.

To remove the software using the inst_sge utility and a configuration template, follow these steps:

Note Remove the software from the execution hosts before removing it from the master host. If you remove the software from the master host first, you cannot automate the removal of the software from the execution hosts.

1. Ensure that your environment variables are set up properly.

   Note If no cell name was specified during installation, the value of $SGE_CELL is default.

   • If you are using a C shell, type the following command:

     # source $SGE_ROOT/$SGE_CELL/common/settings.csh
     

   • If you are using a Bourne or Korn shell, type the following command:

     # . $SGE_ROOT/$SGE_CELL/common/settings.sh
     

     
     

2. Create a copy of the configuration template, $SGE_ROOT/util/inst_sge_modules/inst_sge_template.conf.

   # cd $SGE_ROOT/util/inst_sge_modules/
   # cp inst_sge_template.conf  my_configuration.conf
   

   
   

3. Edit your configuration template.
   Every host that is in the EXEC_HOST_LIST_RM list will be removed.

   # Remove these execution hosts in automatic mode
   EXEC_HOST_LIST_RM="host1 host2 host3 host4"
   

   
   

4. On the master host type the $SGE_ROOT/inst_sge -ux -auto command.
   This example uninstalls the execution hosts: host1, host2 and host3.
   Type the following command as one string, with a space between the -auto and the $SGE_ROOT/util/inst_sge_modules/my_configuration.conf components.

   # $SGE_ROOT/inst_sge -ux -auto $SGE_ROOT/util/inst_sge_modules/my_configuration.conf
   

   Note You are not prompted for any information during this process. However, the output from this process will be displayed to the terminal window where you run the command.

   
   

5. (Optional) If you have any shadow master hosts, uninstall them.
   Type the following command as one string, with a space between the -auto and the $SGE_ROOT/util/inst_sge_modules/my_configuration.conf components.

   # $SGE_ROOT/inst_sge -usm -auto $SGE_ROOT/util/inst_sge_modules/my_configuration.conf
   

   
   

6. Uninstall the master host.
   Type the following command as one string, with a space between the -auto and the $SGE_ROOT/util/inst_sge_modules/my_configuration.conf components.

   # $SGE_ROOT/inst_sge -um -auto $SGE_ROOT/util/inst_sge_modules/my_configuration.conf
   

<< Previous: Installing Sun Grid Engine

Next: User Management on Windows Hosts>>

Sun Grid Engine Information Center
Installing Sun Grid Engine
Index

Microsoft Services for UNIX

• Changing Default Behavior to Case Sensitivity
• Configuring User Name Mapping
• Disabling DEP
• Enable suid Behavior for Interix Programs

Overview

Microsoft Windows Services for UNIX (SFU) makes it possible to integrate some Windows operating systems into existing UNIX environments. SFU provides components that simplify network administration and user management across the UNIX and Windows platforms. You can use SFU to do the following:

• Integrate Windows hosts into Grid Engine clusters. This means that the execution and client environment of Grid Engine can be used on Microsoft Windows hosts. You must use Grid Engine in combination with SFU for this to occur.
• Access the network file system (NFS). This makes it possible for you to share files between the UNIX and Windows environments.
• Possibly access account and password services on UNIX and Windows systems (PCNFS, NIS) using the user mapping service.
• Synchronize passwords and map authentication credentials between the UNIX and Windows operating systems. You can use the "single sign-on" capability for Windows and UNIX environments.
• Execute UNIX shell scripts and applications to run on Windows platform-based computers in full-featured UNIX environments.

Interix, SFU's UNIX environment subsystem, offers the following features:

• A complete, high-performance UNIX environment. You can use the csh shell or the ksh shell.
• Several hundred tools and utilities.
• A complete set of development tools and libraries that make it possible to port your UNIX-based applications to the Interix sub-system.

Unsupported Grid Engine Functionality

The following Grid Engine components are not supported in a Microsoft Windows environment and cannot be used on Windows Hosts even though they are standard to a Grid Engine installation:

• Master and Scheduler (sge_qmaster and sge_shadowd)
• Graphical User Interface (qmon)
• DRMAA
• qsh client command

System Requirements

The following system requirements apply to the SFU installation:

• You must install at least Version 5.0 of Internet Explorer, before running the SFU setup.
• You cannot install SFU on a system running Microsoft Services for Network File System. For example, Microsoft Services for NFS is a component of Windows Storage Server 2003.
• You must install the latest Windows service pack before installing SFU and Grid Engine. Then, you can install additional Windows service packs as they become available.
• The hard disk requirements for an SFU installation depend on which components you need to install. The following installation parameters apply:
  • The minimum disk space required is 20 MB.
  • The maximum disk space requirement is 360 MB.
  • SFU must be installed on a partition that is formatted with the NTFS file system.
• You must disable Data Execution Prevention (DEP). DEP is not compatible with some parts of SFU and might cause segmentation faults. See http://support.microsoft.com/kb/875352 for more information about DEP. To disable DEP, see Disabling DEP.

You can find more details concerning SFU requirements at http://www.microsoft.com/windows/sfu/.

Services for UNIX Installation

Microsoft's SFU is required to install Grid Engine successfully. You can download SFU from Microsoft. Search the site for "Windows Services for Unix" to find the current download information.

1. Get the SFU distribution media.
   
   
2. Execute the application to unzip the files into a directory.
   This directory must be located on a file system that has at least 480 MBytes free space.
   
   
3. Log in to the Windows system with the Administrator account.
   
   
4. Start the setup.exe application that you unpacked previously.
   
   
   
5. Enter your User name and Organization.
   
   
   
6. Accept the license agreement for SFU.
   
   
   
7. Choose the standard installation (recommended) or the custom installation.
   
   If disk space is limited, you might want to choose the custom installation. Make sure that you install at least the following components:
   
   • Utilities -> Base Utilities
   • Interix GNU components -> Interix GNU utilities
   • Remote connectivity components -> Telnet Server and Windows Remote Shell
   • If you intend to use NFS shared file systems, you also need Authentication tools for NFS -> User Mapping and Server for NFS Authentication.
     
8. Depending on the Windows operating system, you might be presented with the following two options concerning SFU security settings, shown in the dialog box below:
   
   If you need further information, consult Microsoft's SFU documentation.
   
   
9. Configure User Name Mapping.

   Note User Name Mapping is part of SFU and not part of Sun Grid Engine. Consult Microsoft documentation and support to set up user mapping correctly.

   Your selection in the dialog box, shown below, depends on the hosts and services that are currently provided in your Windows and UNIX environments. If there is no Remote User Mapping server in your environment, then you should select Local User Name Mapping Server.
   

   Note You should install SFU and enable the User Name Mapping service on your host that acts as a Domain Controller for your windows environment. All other hosts should contact that Remote User Name Mapping Server. If you choose Local User Name Mapping Server, then you might either select Network Information Services (NIS) to access your passwd and group NIS-maps. Otherwise, select l if you can provide the files yourself.

   See Configuring User Name Mapping for further details.
   
   

10. Depending on your previous selections, you can either enter the NIS Domain name and NIS Server name or the path of the passwd and group files.
    
    Below is an example of the files that have the standard UNIX format. This means that you can also use your /etc/passwd and /etc/group files from your UNIX environment.

    C:\Unix\etc\passwd 
    root:x:0:0:UNIX root user:/home/root:/bin/tcsh
    user1:x:1002:100:Full name of user1:/home/user1:/bin/tcsh
    C:\Unix\etc\group
    root::0:
    

    Note Some NIS maps do not contain an entry for the root user. If this is the case, follow these steps to map Administrator to root: First create a password file containing the root entry. If the SFU installation is finished, start the Services for UNIX Administration application and create the mapping: Administrator <-> root. Switch to NIS mapping. Use simple mapping or add manual mappings. At this point the installation starts installing components. Wait until all components are installed.

11. When the installation process finishes, you might need to reboot the machine, depending on the version of Windows that you are using.
    
    
12. Make sure that the Interix Subsystem Startup starts during boot time.
    If you intend to use NFS shares and user mapping, then also start Client for NFS and User Name Mapping.
    Depending on the installation options and your version of the Windows operating system, one or more of these services are disabled by default.