Nextflow Basics: Processes

Notice
This text was originally translated using AI tools.

Introduction#

A Process is the smallest unit where actual work occurs within a Nextflow workflow. Each process runs in an independent working environment (an isolated directory), receiving data via the channels we previously learned about and returning results back into channels.

If a channel is a 'conveyor belt' carrying raw materials (data), a process is like a 'factory station' that processes those materials.

Key Features#

The most distinct feature of a Nextflow process is the isolation of the execution environment. Every time a process is executed, it creates a temporary working directory (work/...) and operates exclusively within it.

• Input Files: Only the necessary files are brought into the working directory via symbolic links.
• Conflict Prevention: Even if 100 instances of the same process run in parallel, they each work in different directories, so errors due to overlapping filenames do not occur.

Basic Structure of a Process#

A Nextflow process is largely composed of the following sections:

process <NAME> {
    [ directives ]       // Execution environment settings (cpus, memory, container...)
 
    input:               // Definition of data to receive from a channel
    <input_definition>
 
    output:              // Definition of results to send to the next channel
    <output_definition>
 
    when:                // (Optional) Execution conditions
    <condition>
 
    script:              // The actual shell script or command to execute
    """
    <commands>
    """
}

Core Components

• Input
  Defines the data coming through the channel as variables or filenames to be used inside the process.
  Qualifiers like val, path, and tuple can be used.

• Output
  Defines which produced results should be passed to the next stage after the process is complete.
  Files or values defined here are returned in the form of a channel.

• Script
  The most critical part, where you write the actual commands you want to execute (bash, python, R, etc.).
  You can create dynamic scripts by referencing Nextflow variables using the $variable syntax.

• Directives
  Specify the resources to be used by the process. Various settings can be used, such as:

  • cpus 4: Assigns 4 CPUs.
  • memory '8 GB': Assigns 8 GB of memory.
  • container 'ubuntu:latest': Uses a specific Docker image.

Conclusion#

We have explored Channels and Processes. In the next post, we will configure a functional workflow using both of them.