next up previous contents
Next: Job Examples Up: Grid Jobs in Detail Previous: edg-brokerinfo Submission Information   Contents

Submitting and Monitoring Jobs

As at present Grid usage is foreseen for production, jobs must be debugged and working correctly in a non-Grid environment before trying Grid submission. When your job is running correctly, you must create the related JDL to be able to submit the job to the Grid. In the JDL file you must specify the name of all required files, at least stdin, stdout, stderr and any other required files, the name of the executable and any required arguments. If the executable is an image stored at your UI, it must be transmitted to the Grid using the InputSandbox.

Before submitting the job, verify that you are using the correct configuration file. A configuration file is needed to specify RB and LB when different from the standard setup. The need arises when the user is trying to access a different environment or the default RB is down for maintainance, failure, etc.

The required information in a configuration file is the LB URL and the RB DNS and looks like the following one:

%%beginLB%%
https://grid009g.cnaf.infn.it:7846
%%endLB%%
%%beginRB%%
grid009g.cnaf.infn.it[:7771]
%%endRB%%
N.B. note the optional port number spcified with the RB IP name. In general this information is not necessary, but in one site where I had problems submitting production jobs, I was requested to add the port number information.
The configuration file may also specify default values for other paramenters, as follows:
## Stage IN/OUT Storage Paths ##
DEFAULT_STORAGE_AREA_IN = /tmp
## Default values for Mandatory Attributes ##
requirements = other.Active
rank = - other.EstimatedTraversalTime
## Job Submission User Interface Version
Version = 1.2.15
## Default location for storage of log files
ErrorStorage= /tmp
## Default values for number of Retries on Fatal Error:
RetryCountLB = 1
RetryCountJobId = 1
## Number of timeout seconds for dg-log-transfer API call
LoggingTimeout = 10

When you invoke dg-job-submit, the procedure at first tests the JDL file and detects any formal error. If the check step is succesful, the job is submitted to the Grid and the Job Id is returned to the user. As Job Id is a URL of the following type:
https://LB-DNS:port/UI-IP address/nnnn?RB-DNS:port/
where nnnn is job submission time to uniquely identify your job, the user should store the name to be able to retrieve the job.
One choice is the builtin option -o [-output] Ids.list where all job Ids are appended to file Ids.list. In this way all your Ids are stored, but if you do not cleanup the file you will end with a very long list of jobs that might be confusing.
Another choice is the redirection of the job submission log to a file with a specific name, so that you can collect job Ids in some preferred way that helps getting job execution information.
Finally you can use the notify option to receive job information at your preferred e-mail address and keep job Ids in a mail folder.
When working frequently with the Grid, either use some web portal like Genius or create your own scripts not to forget supplying all required and/or useful options.


After you have submitted the job as follows:
$ dg-job-submit -c ~/CNAF.cfg argexe.jdl > Id3
periodically check job status for job termination:
$ dg-job-status `grep https Id3 | awk '{print $2}'`
When job status is OutputReady, retrieve your output files:
$ dg-job-get-output `grep https Id3 | awk '{print $2}'`
the command prints log information and the output file directory:
have been successfully retrieved and stored in the directory:
/tmp/105924161111958

Output files (the ones in the Output Sandbox) are stored in the /tmp area in a subdirectory with the same name of the time field in the Job Id. As the /tmp area is cleared at re-boot, move the files you want to keep into a permanenet area.


do-job-submit Options
Option Meaning
--help print usage info & options
--notify e-mail-adr send messages to e-mail address
--config SITE.cfg specify configuration file
--resource CEID specify CE required for the execution
--input CE.list choose a CE from the list
--output ID.list append Job ID to list
--hours nn specify proxy duration
--logfile sub.log write log info to file



next up previous contents
Next: Job Examples Up: Grid Jobs in Detail Previous: edg-brokerinfo Submission Information   Contents
luvisetto 2003-07-25