SLURM Workload Manager
Plugin to provide access to Nodeum Data Mover directly from Slurm.
- 01 About SLURM
- 02 Burst Buffer
- 03 Data Mover Automation
Slurm
Slurm Workload Manager is a job scheduler which is used by many of the world's supercomputers and computer clusters.
It provides three key functions:
- Allocate exclusive and/or non-exclusive access to resources (computer nodes) to users for some duration of time so they can perform work,
- Provide a framework for starting, executing, and monitoring work, typically a parallel job such as Message Passing Interface (MPI) on a set of allocated nodes,
- Arbitrate contention for resources by managing a queue of pending jobs.
Slurm is the workload manager on about 60% of the TOP500 supercomputers.
Nodeum Integration
SLURM includes natively the burst buffer feature. Nodeum SLURM integration is a tight integration to execute unlimited number of data movement operations across any different types of storage nodes.
AUTOMATE DATA MOVER EXECUTION FROM SBACTH
From a SLURM side, slurmctld run the hook in the Burst Buffer which creates the task for Stage in and Stage out. This is done through the interface between Burst Buffer and Nodeum ND Client which allows movement tasks creation.
Parameters compatible with Nodeum
Parameters must be defined in the sbatch to operate with the Data Mover:
- the type of operation
- where the source data are stored
- where the data has to be copied
- if overwrite has to be set
- …
GET IN TOUCH
Let's start your next project. Need help to implement the integration.