Grid Engine Home > Administering > Using Job Submission Verifiers >
JSV Communication Protocol
After you have configured the necessary JSV(s) and you submit a job, the verification process begins. All configured JSVs are executed in the order described in JSV Verification Process. Client JSV instances communicate directly with the client process and server JSV instances communicate directly with the master daemon process. The following are described below:
Possible JSV Results
Each configured JSV yields a result. A submitted job must be accepted by all configured JSVs in order to be given access and run on the Grid Engine system. For more information, see JSV Verification Process.
Each JSV instance yields one of the following results:
ACCEPT – Job is accepted. If there are more configured JSVs, the job moves forward to the next configured JSV.
ACCEPT WITH CORRECTION – The job is corrected and then accepted. If there are more configured JSVs, the job moves forward to the next configured JSV.
REJECT WAIT – The job is temporarily rejected. The submitting user receives code outlining the error and can choose to submit the job again.
REJECT – The job is rejected. The submitting user receives an error message. The verification process stops.
Command References for the JSV Communication Protocol
A client or the master daemon sends a set of commands to the JSV Instance to do the following:
- Indicate that a new job should be added
- Provide the job specification for a job that should be added
- (optional) Provide the job environment that will be available in the job
- Trigger the verification of a job and wait for the verification result
In response, the JSV Instance sends commands that do the following:
- Indicate its state in the communication process
- Change the job specification or environment of a job that should be verified
- Report messages or errors
- Report the verification result
| Command | Description | Sent By | ||
|---|---|---|---|---|
| START blahblahblahmoremmoremoreblah |
The START command initiates each new job verification. | Client/Master Daemon | ||
| SEND ENV | If the client or the master daemon receives a SEND ENV command from a JSV instance before the STARTED command, then it will not only pass job parameters with PARAM commands but also with ENV commands. ENV commands provide the JSV instance with information about environment variables that will be exported to the job environment if the job is accepted. By default, the job environment is not passed to JSV instances. | JSV Instance | ||
| STARTED | The STARTED command is sent by a JSV instance to indicate that it is ready to receive PARAM commands and ENV commands for a new job verification. Note: It will only receive environment commands if it sends a SEND ENV before the STARTED command. | JSV Instance | ||
| PARAM parameter value | Multiple PARAM commands are sent to a JSV instance after the JSV sends a STARTED command. The sum of all PARAM commands represents a job specification for the job that is scheduled to be verified. As response a JSV script might also send back PARAM commands to the client or master daemon process to change the specification of a received job. The sum of all PARAM commands in that case describe only the modifications that have to be applied. The PARAM command has the following arguments:
|
Client/Master Daemon or JSV Instance | ||
| BEGIN | After a JSV instance has received all of the ENV commands and PARAM commands for a job that is scheduled to be verified, the client or the master daemon will trigger the verification process by sending one BEGIN command. Then, it waits for the PARAM commands and the ENV commands to be sent back from the JSV instance to modify the job specification. The JSV script or binary then uses the RESULT command to indicate that the verification process is finished for a job. | Client/Master Daemon | ||
| RESULT result_type message | After the verification of a job is done, a JSV script or binary sends a RESULT command with one of the following messages:
|
JSV Instance | ||
| ERROR message | An ERROR message is reported by the JSV instance to the client or master daemon, if an error is encountered. If the error occurs during a job verification, the job that is currently being verified is rejected. The JSV binary or script then restarts and becomes available to receive a new verification task. | JSV instance | ||
| LOG log_level message | The LOG command can be used whenever the client or the server expects input from a JSV instance. As log_level the word INFO, WARNING or ERROR can be used to create a corresponding log message.
|
JSV instance | ||
| ENV modifier name value (optional) |
The ENV is an optional command that only has to be implemented by a JSV instance if the SEND ENV command is sent by this JSV before the STARTED command is sent. This is the only case that the client or the master daemon uses one or more ENV command to pass the environment variables to the JSV instance. The variables are exported to the job environment when the job starts. The client and the master daemon will only send ENV commands with the modifier ADD. JSV instances modify environment variables by sending back ENV commands and by using the modifiers ADD, MOD and DEL. | JSV instance or Client/Master Daemon | ||
| QUIT | The QUIT has no additional arguments. If this command is sent to a JSV instance, then it should terminate itself immediately. | JSV instance |
Examples of Communication Between a Client or the Master Daemon and a JSV Instance
Example 1
The following job submission example produces the output found below:
> qsub -pe pe1 3 -hard -l a=1,b=5
-soft -l q=all.q $SGE_ROOT/examples/jobs/sleeper.sh 12
 | Note New line characters which terminate each line in the communication protocol are omitted. |
| Sent by Client or Master Daemon | Sent by JSV Instance |
|---|---|
| START | |
| SEND ENV | |
| STARTED | |
| PARAM VERSION 1.0 | |
| PARAM CONTEXT client | |
| PARAM CLIENT qsub | |
| PARAM USER yourusername | |
| PARAM GROUP staff | |
| PARAM CMDNAME /sge_root/examples/jobs/sleeper.sh | |
| PARAM CMDARGS 1 | |
| PARAM CMDARG0 12 | |
| PARAM l_hard a=1,b=5 | |
| PARAM l_soft q=all.q | |
| PARAM M yourusername@hostname | |
| PARAM N Sleeper | |
| PARAM o /dev/null | |
| PARAM pe_name pe1 | |
| PARAM pe_min 3 | |
| PARAM pe_max 3 | |
| PARAM S /bin/sh | |
| BEGIN | |
| RESULT STATE ACCEPT |
Example 2
The following script produces the output shown below:
qsub -V $SGE_ROOT/examples/jobs/sleeper.sh 120
| Sent by Client or Master Daemon | Sent by JSV Instance |
|---|---|
| START | |
| SEND ENV | |
| STARTED | |
| PARAM VERSION 1.0 | |
| PARAM CONTEXT server | |
| PARAM CLIENT qmaster | |
| PARAM USER yourusername | |
| PARAM GROUP yourusername | |
| ARAM JOB_ID 3 | |
| PARAM CMDNAME sleep | |
| PARAM CMDARGS 1 | |
| PARAM CMDARG0 120 | |
| PARAM A sge | |
| PARAM b y | |
| PARAM M yourusername@host | |
| PARAM N sleep | |
| ENV ADD USER yourusername | |
| ENV ADD LOGNAME yourusername | |
| ENV ADD HOME /Users/yourusername | |
| ENV ADD PATH /Users/yourusername/Test/MAINTRUNC/bin/lx26-x86:/Users/ yourusername/Test/MAINTRUNC/bin/lx26-x86:/Users/yourusername/Software /apache-ant-1.6.5/bin:/Users/yourusername/Software/unison-2.9.1/bin: /Users/yourusername/Software/indent-2.2.9/bin:/Users/yourusername/ Software/ctgas-5.6/bin:/Users/yourusername/Software/ncftp-3.2.0/bin:/Users /yourusername/Test/grm_yourusername_1/dist/bin:/Users/yourusername/Software/ gridengine-devel/bin:/Users/yourusername/Software/gridengine-devel/lx26-x86 /bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games |
|
| ENV ADD MAIL /var/mail/yourusername | |
| ENV ADD SHELL /bin/tcsh | |
| ENV ADD TERM xterm-color | |
| ENV ADD LANG en_US.UTF-8 | |
| ENV ADD HOSTTYPE i486-linux | |
| ENV ADD VENDOR intel | |
| ENV ADD OSTYPE linux | |
| ENV ADD MACHTYPE i486 | |
| ENV ADD SHLVL 1 | |
| ENV ADD PWD /Users/yourusername | |
| ENV ADD GROUP yourusername | |
| ENV ADD HOST hostname | |
| ENV ADD REMOTEHOST cibule | |
| ENV ADD DISPLAY cibule:0.0 | |
| ENV ADD CVSROOT :pserver:yourusername@cvs.sunsource.net:/cvs | |
| ENV ADD GRIDENGINE_ARCH lx26-x86 | |
| ENV ADD EDITOR /usr/bin/vim | |
| ENV ADD BROWSER /Applications/Firefox.app/Contents/MacOS/firefox | |
| ENV ADD SGE_ROOT /Users/yourusername/Test/MAINTRUNC | |
| ENV ADD SGE_QMASTER_PORT 5000 | |
| ENV ADD SGE_EXECD_PORT 5001 | |
| ENV ADD SGE_SOURCE /Users/yourusername/SourceSun/MAINTRUNC.gridengine/source | |
| ENV ADD SGE_TEST /Users/yourusername/SourceSun/MAINTRUNC.testsuite/src | |
| ENV ADD SGE_CELL default | |
| ENV ADD SGE_CLUSTER_NAME p5000 | |
| ENV ADD MANPATH /Users/yourusername/Test/MAINTRUNC/man:/Users/yourusername/Test/MAINTRUNC/man:/usr/ share/man:/usr/local/man |
|
| ENV ADD SGE_ARCH lx26-x86 | |
| ENV ADD SGE_BUILDARCH LINUX86 | |
| BEGIN | |
| RESULT STATE REJECT Binary job is rejected |
Example 3
The following script produces the output shown below:
qsub -V -b y sleep 120
| Sent by Client or Master Daemon | Sent by JSV Instance |
|---|---|
| START | |
| SEND ENV | |
| STARTED | |
| PARAM VERSION 1.0 | |
| PARAM CONTEXT server | |
| PARAM CLIENT qmaster | |
| PARAM USER yourusername | |
| PARAM GROUP yourusername | |
| PARAM JOB_ID 2 | |
| PARAM CMDNAME /Users/yourusername/Test/MAINTRUNC/examples/jobs/sleeper.sh | |
| PARAM CMDARGS 1 | |
| PARAM CMDARG0 120 | |
| PARAM A sge | |
| PARAM M yourusername@hostname | |
| PARAM N Sleeper | |
| PARAM S /bin/sh | |
| ENV ADD USER yourusername | |
| ENV ADD LOGNAME yourusername | |
| ENV ADD HOME /Users/yourusername | |
| ENV ADD PATH /Users/yourusername/Test/MAINTRUNC/bin/lx26-x86:/Users/yourusername/ Test/MAINTRUNC/bin/lx26-x86:/Users/yourusername/Software/apache-ant-1.6.5/bin:/Users /yourusername/Software/unison-2.9.1/bin:/Users/yourusername/Software/indent-2.2.9 /bin:/Users/yourusername/Software/ctgas-5.6/bin:/Users/yourusername/Software /ncftp-3.2.0/bin:/Users/yourusername/Test/grm_yourusername_1/dist/bin:/Users/ yourusername/Software/gridengine-devel/bin:/Users/yourusername/Software/ gridengine-devel/lx26-x86/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games |
|
| ENV ADD MAIL /var/mail/yourusername | |
| ENV ADD SHELL /bin/tcsh | |
| ENV ADD TERM xterm-color | |
| ENV ADD LANG en_US.UTF-8 | |
| ENV ADD HOSTTYPE i486-linux | |
| ENV ADD VENDOR intel | |
| ENV ADD OSTYPE linux | |
| ENV ADD MACHTYPE i486 | |
| ENV ADD SHLVL 1 | |
| ENV ADD PWD /Users/yourusername | |
| ENV ADD GROUP yourusername | |
| ENV ADD HOST hostname | |
| ENV ADD REMOTEHOST cibule | |
| ENV ADD DISPLAY cibule:0.0 | |
| ENV ADD CVSROOT :pserver:yourusername@cvs.sunsource.net:/cvs | |
| ENV ADD GRIDENGINE_ARCH lx26-x86 | |
| ENV ADD EDITOR /usr/bin/vim | |
| ENV ADD BROWSER /Applications/Firefox.app/Contents/MacOS/firefox | |
| ENV ADD SGE_ROOT /Users/yourusername/Test/MAINTRUNC | |
| ENV ADD SGE_QMASTER_PORT 5000 | |
| ENV ADD SGE_EXECD_PORT 5001 | |
| ENV ADD SGE_SOURCE /Users/yourusername/SourceSun/MAINTRUNC.gridengine/source | |
| ENV ADD SGE_TEST /Users/yourusername/SourceSun/MAINTRUNC.testsuite/src | |
| ENV ADD SGE_CELL default | |
| ENV ADD SGE_CLUSTER_NAME p5000 | |
| ENV ADD MANPATH /Users/yourusername/Test/MAINTRUNC/man:/Users/yourusername/ Test/MAINTRUNC/man:/usr/share/man:/usr/local/man |
|
| ENV ADD SGE_ARCH lx26-x86 | |
| ENV ADD SGE_BUILDARCH LINUX86 | |
| BEGIN | |
| RESULT STATE ACCEPT Job is accepted |
|
Participate
|
Learn More
|
Sign up or Log in to add a comment or watch this page.