Grid Engine Home > Administering > Configuring Resource Attributes >

Defining Consumable Resources

Introducing Consumable Resources

Consumable resources, or consumables, provide an efficient way to manage limited resources. The complex also builds the framework for the system's consumable resources facility. The resource attributes that are defined in the complex can be attached to the global cluster, to a host, or to a queue instance. The attached attribute identifies a resource with the associated capability. Attribute definitions in the Grid Engine complex define how resource attributes should be interpreted. During the scheduling process, the availability of resources and the job requirements are taken into account. The Grid Engine system also performs the bookkeeping and the capacity planning that is required to prevent over-subscription of consumable resources.

Typical consumable resources include:

• Available free memory
• Unoccupied software licenses
• Free disk space
• Available bandwidth on a network connection

The definition of a resource attribute includes the following:

• Name of the attribute
• Shortcut to reference the attribute name
• Value type of the attribute, for example, STRING, RESTRING, TIME, or any other complex(5) type
• Relational operator used by the scheduler
• Requestable flag, which determines whether users can request the attribute for a job
• Consumable flag, which identifies the attribute as a consumable resource
• Default request value that is taken into account for consumable attributes if jobs do not explicitly specify a request for the attribute
• Urgency value, which determines job priorities on a per resource basis

Defining Consumable Resources

To enable consumable resource management, you must define the total capacity of a resource. You can define resource capacity globally for the cluster, for specified hosts, and for specified queues. These categories can supersede each other in the given order. Thus a host can restrict availability of a global resource, and a queue can restrict host resources and global resources.

The consumption of the resource is then monitored by Grid Engine software internal bookkeeping. Jobs are dispatched only if the internal bookkeeping indicates that sufficient consumable resources are available.

You define resource capacities by using the complex_values attribute in the queue and host configurations. The complex_values definition of the global host specifies global cluster consumable settings. For more information, see the host_conf(5) and queue_conf(5) man pages, as well as Configuring Queues and Configuring Hosts.

To each consumable attribute in a complex_values list, a value is assigned that denotes the maximum available amount for that resource. The internal bookkeeping subtracts from this total the assumed resource consumption by all running jobs as expressed through the jobs' resource requests.

Multiplied Resource Requests Versus Non-Multiplied Resource Requests

By default Sun Grid Engine performs multiplied resource requests, which means that a consumable resource request is multiplied by the number of slots allocated to a parallel job. The configuration for multiplied resource requests is designated by a YES flag in the consumable column of the job row in the complex definition.

The following multiplied resource request is explained below:

qsub -l mem=100M -pe make=8

Sun Grid Engine multiples the consumable resource request (100 M) by the number of slots allocated for the parallel job (8). The consumable usage is split across the queues and hosts on which the job runs. If four tasks run on host A and four tasks run on host B, the job consumes 400 Mbytes on each host.

While multiplied resource requests typically work well, in the case of software licenses, it is more practical to make a per job request, or a non-multiplied resource request, which debits the exact amount requested. Starting in Sun Grid Engine 6.2u2, you can configure the complex to accept non-multiplied resource requests by changing the jobs consumable flag from YES to JOB, as shown below:

#name   shortcut   type   relop   requestable   consumable   default   urgency 
#-----------------------------------------------------------------------------
jobs       j        INT    <=          YES           JOB        0        0 

For more on the complex configuration, see the queue_conf(5) man page.

Examples – Defining Consumable Resources

Only numeric attributes can be configured as consumables. Numeric attributes are attributes whose type is INT, DOUBLE, MEMORY, or TIME.

In the QMON Main Control window, click the Complex Configuration button. The Complex Configuration dialog box appears.

To enable the consumable management for an attribute, set the Consumable flag for the attribute in the complex configuration.

To set up other consumable resources, follow these examples: