POLICY-BASED DATA ORCHESTRATION

Automated solution for scheduling task executions 

Workflow Manager

Policy-Based Workflow Manager

We design an automated solution for scheduling task executions 

Data Movement Workflows

System allows the creation of different data movement tasks. This can be defined per data set and per storage but also with a certain level of policy : Migration, Archiving (Active, Offline), Protection.

Extended Data Set Selector

An unlimited data set selector brings granularity in the data you have to select. Movement can be executed between any type of storage from storage where hot data are stored to cold storage.

Scheduling

The internal scheduler can execute automatically, manual or scheduled task.

Filtering

Two different types of filtering are available : basic and advanced. Advanced provides embedded scripting editor. 

Nodeum - Product Shoot - Data Manager-1
Workflow Manager

Scalable - Safe - Optimized Data Movement

icon-set-nodeum-cleanup-white_scale
OPTIMIZE TO FILE MANAGEMENT PROCESSING

The solution is designed to optimize the file data management processing for different types of files. Here are three types of concepts that help to understand:

  • The system is designed to automate the processing order,
  • the solution is designed around multi-threaded file batching technology. Each task movement is defined by 3 stages: preparation, execution, and finalization. Tasks run in // with a batch system creating jobs, each job contain a list of files, the number of files in each job is determine by a maximum number of files or a when the capacity of all files is reached.
  • includes different queuing systems which allow to execute in // the different type of actions which must be performed during a data movement processing.

The benefit is to not lose time in discovery phase, files start to be copied once the first batch of data has been received. The discovery process continues to run during the copy

icon-set-nodeum-cleanup-white_long-term-29
PRIORITY MANAGEMENT

A priority level feature is available to each task execution; this feature is linked together with the tape drive allocation.

The solution allows a task which a higher priority to pause another task currently copying file to a tape. This will eject the tape, free up the drive remount the tape requested with the new task; and start the treatment.

icon-set-nodeum-cleanup-white_metadata
RESOURCE MANAGEMENT

The workflow manager system provides natively mechanism to prevent task movement overload.

  • By default, there is a memory threshold set at 80%. That’s mean, if the memory is over 80%, it is not possible to execute additional task movement.
  • There is the possibility to execute task to clean the disk cache layer,
  • There is a workflow for rehydration (which means for us the defragmentation of tapes).

The solution verifies before transferring any file the Status of any storage (available or not).

Nodeum always provides a clean stop of the processing if there is any problem in the source or destination storage.

 Even the source can be stopped – not reachable - …. Or the same for the destination. The behaviour is that the task is tagged as “stop by system”, logs task is updated with the root cause, error code, … And even the file logging is updated with status of each file not well processed and again why a display of the reason.

Workflow Manager

Data Integrity Verification

Nodeum provides data integrity validation in each workflow, the three main different mechanisms are available : checksums (e.g. CRC32), non-cryptographic hashes (e.g. xxHash) and cryptographic hashes (e.g. MD5).

md5checksum
MD5

MD5 hash algorithm is a commonly used function for validating data integrity. An MD5 checksum is a 32 digit hexadecimal number that represents the hash of the contents of a file.

The calculation of an MD5 is an industry standard so the integrity can be checked on any system.

XXHASH

It is an Extremely fast Hash algorithm, the hash is faster to be generated. It is highly portable, and hashes are identical across all platforms (little / big endian). This is more and more popular in the video device, Nodeum uses the xxHash64be algorithm which is compatible with other product and software in the industries

xxhash64

Get Started With Nodeum

Start the download of the free edition,
or compare the features of the different editions.

Get STARTED

Searchable Catalog

Catalog in real-time data stored in any of your storage systems (primary and secondary storages). Accessing and finding your data have never been easier.

Leverage Cloud ML/AI

Enrich data stored on premise or in cloud with the best Cloud based ML/AI engines. Get the most of these platforms to facilitate the organization and classification of your contents. 

Reporting & Analysis

Storage data usage statistics reporting and data lifecycle overview. Generate your reports on the current data usage, control and master your data lifecycle.

TCO Simulator

Toolbox to control the cost of your storage usage. Provide simulation of the cost impact of a new Storage Data Management Strategy.

Software-Defined

Turn any Linux platform to a Nodeum appliance in deploying easily an Ansible package. Get the most of your hardware, avoid any locking and get the most of next releases.

Highly Scalable

Scalability in data volumes, number of files and overall performance allow to grow with your needs in keeping the same user experience.

Metadata

Enjoy the capability to customize, tag and retrieve your files' metadata. Facilitate your organisation 

Intuitive Interface

Provide a natural and unique experience for any user using the solution. This helps to keep focus on your business and productivity.

Key Features

Discover the full set of features available into Nodeum. Develop a business focus data management strategy to unlock the value and potential of your data.