This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
qcg-pilotjobs [2019/01/23 14:30] bbosak@man.poznan.pl [How to use QCG PilotJob ?] |
qcg-pilotjobs [2019/01/31 11:47] bbosak@man.poznan.pl |
||
---|---|---|---|
Line 6: | Line 6: | ||
===== What is QCG PilotJob? ===== | ===== What is QCG PilotJob? ===== | ||
- | The QCG PilotJob is a computing job playing a role of a container over a number of subordinate computing jobs. It allows to execute many subordinate jobs in a single scheduling system allocation. Direct submission of a large group of jobs to a scheduling system can result in long aggregated time to finish as each single job is scheduled independently and waits in a queue. On the other hand the submission of a group of jobs can be restricted or even forbidden by administrative policies defined on clusters. One can argue that there are available job array mechanisms in many systems, however the traditional job array mechanism allows to run only bunch of jobs having the same resource requirements while jobs being parts of a multiscale simulation by nature vary in requirements and therefore need more flexible solutions. On a technical level QCG PilotJob is exposed as a lightweight service - QCG PilotJob Manager. | + | QCG PilotJob is a computing job playing a role of a container over a number of subordinate computing jobs. It allows to execute many subordinate jobs in a single scheduling system allocation. Direct submission of a number separate jobs to a scheduling system can result in long aggregated time to finish as each single job is scheduled independently and waits in a queue. Moreover the submission of a numerous of jobs can be restricted or even forbidden by administrative policies defined on clusters. One can argue that there are available job array mechanisms in many systems, however the traditional job array mechanism allows to run only bunch of jobs having the same resource requirements while jobs being parts of a multiscale simulation by nature vary in requirements and therefore need more flexible solutions. On a technical level QCG PilotJob is exposed as a lightweight service - QCG PilotJob Manager. |
===== How to use QCG PilotJob ? ===== | ===== How to use QCG PilotJob ? ===== | ||
Line 106: | Line 106: | ||
] | ] | ||
</code> | </code> | ||
+ | |||
+ | In this example, the first executed job will be ''date1'', then ''sleep'' and then ''date2''. The thing to note is the fact that the order of specification of jobs is important when there are dependencies between tasks applied. Thus, the ''sleep'' job has to be defined after ''date1'' and ''date2'' after ''sleep''. | ||
This is very basic scenario, but in a similar way QCG PilotJob system supports definition of more advanced use cases, e.g scenarios including loops and/or parallel processing. | This is very basic scenario, but in a similar way QCG PilotJob system supports definition of more advanced use cases, e.g scenarios including loops and/or parallel processing. |