nextflow.executor

The nextflow.executor package defines the executor interface and implements several built-in executors.

Class Diagram

run
init
submit
submit
submit
init
submit
submit
ProcessDef
Executor
TaskProcessor
TaskPollingMonitor
capacity : int
submitRateLimit : RateLimiter
pollIntervalMillis : long
dumpInterval : Duration
LocalPollingMonitor
maxCpus : int
maxMemory : long
AbstractGridExecutor
LocalExecutor
LocalTaskHandler
NativeTaskHandler
BashWrapperBuilder
bean : TaskBean
copyStrategy : ScriptFileCopyStrategy
GridTaskHandler
TaskBean
ScriptFileCopyStrategy
SimpleFileCopyStrategy
stageinMode : String
stageoutMode : String
targetDir : Path
workDir : Path

Note

Some classes may be excluded from the above diagram for brevity.

Notes

The Executor class is the base class for all Nextflow executors. The main purpose of an Executor is to submit tasks to an underlying compute environment, such as an HPC scheduler or cloud batch executor. It uses a TaskMonitor to manage the lifecycle of all tasks and TaskHandlers to manage each individual task. Most executors use the same polling monitor, but each executor implements its own task handler to customize it for a particular compute environment. See nextflow.processor for more details about these classes.

The built-in executors include the local executor (LocalExecutor) and the various grid executors (SLURM, PBS, LSF, etc), all of which extend AbstractGridExecutor. The LocalExecutor implements both “local” tasks (processes with a script or shell block) and “native” tasks (processes with an exec block).

The BashWrapperBuilder is used by executors to generate the wrapper script (.command.run) for a task, from a template script called command-run.txt, as well as the task configuration and the execution environment.