Nodeum Client
Connect Production Pipelines & Business Workflows with your Storage.
Quickly.
Connect Production Pipelines & Business Workflows with your Storage.
Quickly.
The Nodeum Client nd
command line tool provides a modern set of commands to execute data movement operations with Nodeum. The nd
commandline tool is built for compatibility with the Nodeum v2
for expected functionality and behavior.
nd
has the following syntax:
nd [GLOBALFLAGS] COMMAND --help
See Command Quick Reference for a list of supported commands.
Copyright
nd
is a property of Nodeum and its subsidaries, if any.
The intellectual and technical concepts contained herein are proprietary to Nodeum
and its subsidaries and may be covered by Belgium and Foreign Patents, patents in process,
and are protected by trade secret or copyright law.
Dissemination of this information or reproduction of this material is strictly forbidden unless
prior written permission is obtained from Nodeum.
Related Version: 2.0.6
Install the nd command line tool onto the host machine. Go in the section that corresponds to the host machine operating system or environment:
The following commands add a temporary extension to your system PATH for running the nd bash client.
Defer to your operating system instructions for making permanent modifications to your system PATH.
Alternatively, execute nd by navigating to the parent folder and running ./nd --help
curl https://get.nodeum.io/public/nd-2.0.6-linux-amd64 \
--create-dirs \
-o $HOME/nodeum-binaries/nd
chmod +x $HOME/nodeum-binaries/nd
export PATH=$PATH:$HOME/nodeum-binaries/
nd --help
curl https://get.nodeum.io/public/nd-2.0.6-darwin-amd64 \
--create-dirs \
-o $HOME/nodeum-binaries/nd
chmod +x $HOME/nodeum-binaries/nd
export PATH=$PATH:$HOME/nodeum-binaries/
nd --help
Open the following file in a browser:
https://get.nodeum.io/public/nd-2.0.6-windows-amd64
Once download, rename the file to nd.exe
and execute the file by double clicking on it, or by running the following in the command prompt or powershell:
\path\to\nd.exe --help
# wget https://get.nodeum.io/public/nd-2.0.6-x86_64.rpm
# dnf install ./nd-2.0.6-x86_64.rpm
[[email protected] ~]$ nd
NAME:
nd - Nodeum CLI
USAGE:
nd [global options] command [command options] [arguments...]
VERSION:
2.0.6
COMMANDS:
admin
config configure the Nodeum Client
copy, cp create copy task
move, mv create move task
task
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--json output as JSON (default: false)
--config value path to configuration file (default: <config-dir>/config.json) [$ND_CONFIG]
--config-dir value, -C value path to configuration folder (default: "/home/nodeum01/.config/.nd") [$ND_CONFIG_DIR]
--alias value alias in configuration file for authentication (default: "default") [$ND_ALIAS]
--url value URL of Nodeum [$ND_URL]
--access-token value for API authentication (1st authentication method) [$ND_ACCESS_TOKEN]
--refresh-token value for API authentication (1st authentication method, not saved in config) [$ND_REFRESH_TOKEN]
--authorization-endpoint value for Device Authorization Flow (2nd authentication method)
--token-endpoint value for Device Authorization Flow (2nd authentication method)
--client-id value for Device Authorization Flow (2nd authentication method)
--scopes value for Device Authorization Flow (2nd authentication method)
--persist-session persist Device Authorization session on disk for 1 hour (default: true)
--persist-session-renew if persist session is enabled, renew the token (default: false)
--username value for API authentication (3rd authentication method) [$ND_USERNAME]
--password value for API authentication (3rd authentication method) [$ND_PASSWORD]
--anonymous no login (default: false)
--help, -h show help (default: false)
--version, -v print the version (default: false)
The following table lists nd commands:
Name | Shortcut | Description |
---|---|---|
admin | Access the administration commands | |
config | Configure the Nodeum client | |
copy | cp | Create a task to copy data between two storages |
move | mv | Create a task to move data between two storages |
task | List detailed information about created task | |
help | h | Displays a summary of command usage and parameters on the terminal |
Note: the nd client provides a Bash completion mechanism to facilitate the search of commands.
Name | Shortcut | Description | Default |
---|---|---|---|
–json | Output as JSON | false | |
–config value | -C value | Path to configuration file | /home/nodeum01/.config/.nd/config.json |
–config-dir value | -C value | Path to configuration folder (default: “/home/nodeum01/.config/.nd”) | |
–alias value | Alias in configuration file for authentication | default | |
–url value | URL of Nodeum | ||
–access-token value | for API authentication (1st authentication method) | ||
–refresh-token value | for API authentication (1st authentication method, not saved in config) | ||
–authorization-endpoint value | for Device Authorization Flow (2nd authentication method) | ||
–token-endpoint value | for Device Authorization Flow (2nd authentication method) | ||
–client-id value | for Device Authorization Flow (2nd authentication method) | ||
–scopes value | for Device Authorization Flow (2nd authentication method) | ||
–persist-session | persist Device Authorization session on disk for 1 hour | true | |
–persist-session-renew | if persist session is enabled, renew the token | false | |
–username value | for API authentication (3rd authentication method) | ||
–password value | for API authentication (3rd authentication method) | ||
–anonymous | no login | false | |
–help | -h | show help | false |
–version | -v | print the version | false |
Parameters are available for each data movement task.
Name | Description | Value |
---|---|---|
–md project_name=<value> | Name of project defined in OpenStack | string |
–md project_domain_name=<value> | Name of the project’s domain defined in OpenStack | string |
–md user_domain_name=<value> | Name of the project’s user defined in OpenStack | string |
–md region_name=<value> | Name of the region’s project defined in OpenStack | string |
Parameters are available for each data movement task.
Name | Shortcut | Description | Value | Default value |
---|---|---|---|---|
–help | -h | Show help | false | |
–no-run | Create a task and don’t launch the task directly | false | ||
–name value | -n | Name of task | string | automatically generated |
–comment value | Comment of task | string | empty | |
–overwrite value | Overwrite all identical files already stored at destination | true - false | false | |
–priority value | Priority of the task, between 0 and 9 (0 is the highest priority) | 0 - 9 | 0 | |
–recursive | -R | Execute a recursive copy of the folder. If subfolders are present, the service will also copy the contents of each subfolder | false | |
–working-dir value | –wd | Defines the workpath to be kept at destination | . - … - path | 0 |
–ignore-hidden value | Task will not handle hidden file(s) | true - false | false | |
–progress value | Display live progress when running the task | true - false | true | |
–processed-nodes value | Display the processed nodes when running a task when the flag–progress is set | none, error, all | error |
Parameters are available for each data movement task.
Name | Shortcut | Description | Value | Default value |
---|---|---|---|---|
–context-uid value | –uid | Define the User ID which will handle the movement | integer | unset |
–context-gid value | –gid | Define the Group ID which will handle the movement | integer | unset |
–defer | when requesting the run of the task, will defer it for later with an unique ID | false | ||
–parallel value | Define the number of mover which will handle the movement. Maximum value is determined by the deployed implementation. | 1-20 | 1 | |
–callback type | Add callback. Format is type:./path/to/file | ./path/to/file | ||
–trigger-md key=value | –md key=value | Set metadata on the trigger. Format is key=value. Accepts multiple inputs | key=value | |
–task-md key=value | Set metadata on the task. Format is key=value. Accepts multiple inputs | key=value | ||
–files-md key=value | Set metadata on the files. Format is key=value. Accepts multiple inputs | key=value |
nd uses a JSON formatted configuration file used for storing certain kinds of information, such as the authentication and authorization options.
For Linux and macOS, the default configuration file location is .config/.nd/config.json
which is store in the $HOME
. For Windows, the configuration file is stored in $AppData$
.
You can display the configuration file location in using the command `nd --help´:
[[email protected] ~]$ nd --help
NAME:
nd - Nodeum CLI
....
--config value path to configuration file (default: <config-dir>/config.json) [$ND_CONFIG]
--config-dir value, -C value path to configuration folder (default: "/home/nodeum01/.config/.nd") [$ND_CONFIG_DIR]
You can use the --config value
where value is the path to a JSON formatted configuration file that nd uses for storing data.
The ND_CONFIG
environment variable can be used to set the value.
nd client allows SSL configuration to communicate with the Data Mover service which listen in HTTPS.
It is required to add the public certificate generated with the server.
The public certificate file has to be stored in this config-dir
folder: .config/.nd/certs/CAs/
.
The certificates on the server side have been generated following this command:
openssl11 req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes \
-keyout ./nodeum_certs/private.key \
-out ./nodeum_certs/public.crt \
-subj "/CN=nodeum-qualif.mt-c.local" \
-addext "subjectAltName=DNS:nodeum.domain.local,DNS:localhost,IP:127.0.0.1,IP:1.1.1.1"
Where 1.1.1.1
is the ip of the nginx interface and nodeum.domain.local
is its hostname including domain name.
Two authentifcation options are available:
Command | Description |
---|---|
nd config save |
this command save the nd configuration for authentication |
nd config review |
this command display the information related to the credentials |
The nd client provides a basic method of authentication in using Username / Password credentials.
[[email protected]]$ nd config save \
--url=http://<url> \
--username=<username> \
--password=<password>
Option | Description |
---|---|
--url |
this refer to the node which hosts the following service DATA MANAGEMENT WEB SERVICES |
--username |
this is the username to grant authorization to the service |
--password |
this is the password associated to the username that will grant the access to the service |
The nd client provides also an OpenID authentication mechanism. In this case, nd Client has to be configured with the approriated IDP to handle proper token management.
The --persist-session-renew
option can be defined to true to force a request token process any time the user has to interfact with the nd client.
[[email protected]]$ nd config save \
--url=http://<url> \
--authorization-endpoint=<authorization endpoint> \
--token-endpoint=<token endpoint> \
--client-id=<client id> \
--scopes="openid,profile,offline_access,email"
Option | Description |
---|---|
--url |
this refer to the node which hosts the following service DATA MANAGEMENT WEB SERVICES |
--authorization-endpoint |
this is the endpoint url to grant authorization to the service |
--token-endpoint |
this url is used to programmatically request tokens |
--client-id |
this refer to the client identifier which is provided by the OpenID provider |
Alias & default flags allow structuration of different group of settings. Different alias can be defined in the configuration file.
Alias and flags are declared in the configuration file ~/.config/.nd/config.json"
.
Default flags are defined for each available parameter command.
Flags can be overwritted in the nd command.
Example in config file:
{
// ...
// default: name of alias used by default
"default": {
// ...
"default_flags": {
// this is the name of the command, example: nd copy
"copy": {
// this is the flag, example: nd copy --md project_domain_name=MYPROJECTDOMAIN --md user_domain_name=MYUSERDOMAIN
"md": ["project_domain_name=MYPROJECTDOMAIN", "user_domain_name=MYUSERDOMAIN"]
}
}
}
// myorganisation: name of another alias
"myorganisation": {
// ...
}
}
The nd command allows the usage of alias, example : nd --alias myorganisation copy
Command | Description |
---|---|
nd admin status |
retrieve the status and the health of each services part of the cluster |
nd admin logs |
retrieve all Logs and return their contents |
nd admin status
command requests the Data Management Monitoring service to retrieve the status and health of each services part of the cluster.
This command returns list of services. The following information are displayed:
╭───┬───────────────────┬────────────────────────────┬─────────────────┬────────────┬───────────╮
│ │ NAME │ VERSION │ ADDRESS │ UPTIME │ MEMORY │
├───┼───────────────────┼────────────────────────────┼─────────────────┼────────────┼───────────┤
│ ● │ micro.http.broker │ ff.http.broadcast │ 10.2.2.83:50050 │ N/A │ N/A │
│ ● │ nodeum.dispatcher │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.84:50010 │ 172h11m8s │ 34.92 MB │
│ ● │ nodeum.dispatcher │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.83:50010 │ 172h13m32s │ 22.61 MB │
│ ● │ nodeum.finalizer │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.83:50030 │ 171h6m21s │ 25.39 MB │
│ ● │ nodeum.finalizer │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.85:50030 │ 171h8m40s │ 24.34 MB │
│ ● │ nodeum.front │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.83:50000 │ N/A │ N/A │
│ ● │ nodeum.monitoring │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.85:50060 │ 171h2m47s │ 17.20 MB │
│ ● │ nodeum.mover │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.84:50020 │ 172h4m48s │ 25.23 MB │
│ ● │ nodeum.mover │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.85:50020 │ 172h4m48s │ 25.63 MB │
│ ● │ nodeum.scheduler │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.83:50040 │ 172h18m6s │ 6.91 MB │
│ ● │ mongo │ 4.4.17 │ 10.2.2.83:27017 │ 268h0m44s │ 212.86 MB │
│ ● │ mongo │ 4.4.17 │ 10.2.2.84:27017 │ 268h0m44s │ 212.86 MB │
│ ● │ mongo │ 4.4.17 │ 10.2.2.85:27017 │ 268h0m44s │ 212.86 MB │
│ ● │ mysql │ 10.6.11-MariaDB │ 10.2.2.84:3306 │ 150h21m0s │ N/A │
│ ● │ redis-sentinel │ 6.2.6 │ 10.2.2.83:26379 │ 150h16m52s │ N/A │
│ ● │ redis-sentinel │ 6.2.6 │ 10.2.2.84:26379 │ 150h16m52s │ N/A │
│ ● │ redis-sentinel │ 6.2.6 │ 10.2.2.85:26379 │ 150h16m52s │ N/A │
│ ● │ redis │ 6.2.6 │ 10.2.2.83:6379 │ 285h37m50s │ N/A │
│ ● │ redis │ 6.2.6 │ 10.2.2.84:6379 │ 169h59m59s │ N/A │
│ ● │ redis │ 6.2.6 │ 10.2.2.85:6379 │ 169h59m59s │ N/A │
│ ● │ kafka │ 3.2.0 │ 10.2.2.84:9092 │ N/A │ N/A │
│ ● │ kafka │ 3.2.0 │ 10.2.2.83:9092 │ N/A │ N/A │
│ ● │ kafka │ 3.2.0 │ 10.2.2.85:9092 │ N/A │ N/A │
│ ● │ etcd │ 3.4.13 │ 10.2.2.83:2379 │ N/A │ N/A │
│ ● │ etcd │ 3.4.13 │ 10.2.2.84:2379 │ N/A │ N/A │
│ ● │ etcd │ 3.4.13 │ 10.2.2.85:2379 │ N/A │ N/A │
╰───┴───────────────────┴────────────────────────────┴─────────────────┴────────────┴───────────╯
╭───┬───────────────────┬────────────────────────────┬─────────────────┬────────────┬───────────╮
│ │ NAME │ VERSION │ ADDRESS │ UPTIME │ MEMORY │
├───┼───────────────────┼────────────────────────────┼─────────────────┼────────────┼───────────┤
│ ● │ micro.http.broker │ ff.http.broadcast │ 10.2.2.83:50050 │ N/A │ N/A │
│ ● │ nodeum.dispatcher │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.84:50010 │ 172h11m8s │ 34.92 MB │
│ ● │ nodeum.dispatcher │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.83:50010 │ 172h13m32s │ 22.61 MB │
│ ● │ nodeum.finalizer │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.83:50030 │ 171h6m21s │ 25.39 MB │
│ ● │ nodeum.finalizer │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.85:50030 │ 171h8m40s │ 24.34 MB │
│ ● │ nodeum.front │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.83:50000 │ N/A │ N/A │
│ ● │ nodeum.monitoring │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.85:50060 │ 171h2m47s │ 17.20 MB │
│ ● │ nodeum.mover │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.84:50020 │ 172h4m48s │ 25.23 MB │
│ ● │ nodeum.mover │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.85:50020 │ 172h4m48s │ 25.63 MB │
│ ● │ nodeum.scheduler │ v2.0.0-beta7-22-g9ffff06ba │ 10.2.2.83:50040 │ 172h18m6s │ 6.91 MB │
│ ● │ mongo │ 4.4.17 │ 10.2.2.83:27017 │ 268h0m44s │ 212.86 MB │
│ ● │ mongo │ 4.4.17 │ 10.2.2.84:27017 │ 268h0m44s │ 212.86 MB │
│ ● │ mongo │ 4.4.17 │ 10.2.2.85:27017 │ 268h0m44s │ 212.86 MB │
│ ● │ mysql │ 10.6.11-MariaDB │ 10.2.2.84:3306 │ 150h21m0s │ N/A │
│ ● │ redis-sentinel │ 6.2.6 │ 10.2.2.83:26379 │ 150h16m52s │ N/A │
│ ● │ redis-sentinel │ 6.2.6 │ 10.2.2.84:26379 │ 150h16m52s │ N/A │
│ ● │ redis-sentinel │ 6.2.6 │ 10.2.2.85:26379 │ 150h16m52s │ N/A │
│ ● │ redis │ 6.2.6 │ 10.2.2.83:6379 │ 285h37m50s │ N/A │
│ ● │ redis │ 6.2.6 │ 10.2.2.84:6379 │ 169h59m59s │ N/A │
│ ● │ redis │ 6.2.6 │ 10.2.2.85:6379 │ 169h59m59s │ N/A │
│ ● │ kafka │ 3.2.0 │ 10.2.2.84:9092 │ N/A │ N/A │
│ ● │ kafka │ 3.2.0 │ 10.2.2.83:9092 │ N/A │ N/A │
│ ● │ kafka │ 3.2.0 │ 10.2.2.85:9092 │ N/A │ N/A │
│ ● │ etcd │ 3.4.13 │ 10.2.2.83:2379 │ N/A │ N/A │
│ ● │ etcd │ 3.4.13 │ 10.2.2.84:2379 │ N/A │ N/A │
│ ● │ etcd │ 3.4.13 │ 10.2.2.85:2379 │ N/A │ N/A │
│ ○ │ nodeum.mover │ │ │ │ │
│ ○ │ nodeum.finalizer │ │ │ │ │
╰───┴───────────────────┴────────────────────────────┴─────────────────┴────────────┴───────────╯
nd admin logs
command requests the Data Management Log Management Services to retrieve all Logs and return their contents.
[[email protected] ~]$ nd admin logs --help
NAME:
nd admin logs - display application logs
USAGE:
nd admin logs [command options] [arguments...]
OPTIONS:
--since value, -S value show logs not older than the specified date
--until value, -U value show logs not newer than the specified date
--tag value, -u value [ --tag value, -u value ] filter by tags
--help, -h show help (default: false)
This command allows different parameters:
Option | Description |
---|---|
since | to only show logs not older than a specified date |
until | to only show logs not newer than a specified dat |
tag | to filter the logs per type of services. Example nodeum.monitoring |
The Logs can be exported in using the standard OS mechanism of exportation
./nd admin logs > 'nodeum_site-name_log.txt'
Where nodeum_site-name_log.txt
is the file name and site represents the name of the site.
Follow these steps to create a task:
The definition of a working directory allows to define where the files will be stored at the destination. Different options are available, they are described in the following definition.
With --wd=.
Source | Destination | Result |
---|---|---|
nod://source/folder/FILE.txt |
nod://dest/directory/ |
nod://dest/directory/FILE.txt |
nod://source/folder/FILE.txt |
nod://dest/RENAMED.txt |
nod://dest/RENAMED.txt |
nod://source/folder/ |
nod://dest/directory/ |
nod://dest/directory/FILE.txt |
nod://source/folder/ |
nod://dest/directory |
nod://dest/directory/FILE.txt |
nod://source/folder |
nod://dest/directory/ |
nod://dest/directory/folder/FILE.txt |
nod://source/folder |
nod://dest/directory |
nod://dest/directory/FILE.txt |
With --wd=..
Source | Destination | Result |
---|---|---|
nod://source/folder/FILE.txt |
nod://dest/directory/ |
nod://dest/directory/folder/FILE.txt |
nod://source/folder/FILE.txt |
nod://dest/RENAMED.txt |
nod://dest/RENAMED.txt |
nod://source/folder/ |
nod://dest/directory/ |
nod://dest/directory/folder/FILE.txt |
nod://source/folder/ |
nod://dest/directory |
nod://dest/directory/FILE.txt |
nod://source/folder |
nod://dest/directory/ |
nod://dest/directory/source/folder/FILE.txt |
nod://source/folder |
nod://dest/directory |
nod://dest/directory/FILE.txt |
This command send a copy request to the data mover service from a storage A (nod://posix_storage/) to a storage B (nod-cloud://cloud_storage/)
nd copy \ --md project_name=<my project name> \ --md project_domain_name=<my projectdomain name> \ --md user_domain_name=<my user domain name> \ nod://posix_storage/path/subpath/ \ nod-cloud://cloud_storage/container
nd copy \
--md project_name=<my project name> \
--md project_domain_name=<my projectdomain name> \
--md user_domain_name=<my user domain name> \
nod://posix_storage/path/subpath/ \
nod-cloud://cloud_storage/container
nd copy \ --md project_name=<my project name> \ --md project_domain_name=<my project domain name> \ --md user_domain_name=<my user domain name> \ --working-dir nod://largedata2_pool/storagetestdata/ \ --recursive \ nod://posix_storage/path/subpath/ \ nod-cloud://cloud_storage/container
nd copy \
--md project_name=<my project name> \
--md project_domain_name=<my project domain name> \
--md user_domain_name=<my user domain name> \
--working-dir nod://posix_storage/path/ \
--recursive \
nod://posix_storage/path/subpath/ \
nod-cloud://cloud_storage/container
nd copy \ --md project_name=<my project name> \ --md project_domain_name=<my project domain name> \ --md user_domain_name=<my user domain name> \ --working-dir nod://posix_storage/path/ \ --recursive \ --ignore-hidden \ nod://posix_storage/path/subpath/ \ nod-cloud://cloud_storage/container
nd copy \
--md project_name=<my project name> \
--md project_domain_name=<my project domain name> \
--md user_domain_name=<my user domain name> \
--working-dir nod://posix_storage/path/ \
--recursive \
--ignore-hidden \
nod://posix_storage/path/subpath/ \
nod-cloud://cloud_storage/container
nd copy \ --md project_name=<my project name> \ --md project_domain_name=<my project domain name> \ --md user_domain_name=<my user domain name> \ --working-dir nod://posix_storage/path/ \ --recursive \ --ignore-hidden \ nod-cloud://cloud_storage/container/path/ \ nod://posix_storage/path/
nd copy \
--md project_name=<my project name> \
--md project_domain_name=<my project domain name> \
--md user_domain_name=<my user domain name> \
--working-dir nod-cloud://cloud_storage/container/ \
--recursive \
--ignore-hidden \
nod-cloud://cloud_storage/container/path/ \
nod://posix_storage/path/
nd move \ --md project_name=<my project name> \ --md project_domain_name=<my project domain name> \ --md user_domain_name=<my user domain name> \ --working-dir nod://posix_storage/path/ \ --recursive \ --ignore-hidden \ nod://posix_storage/path/subpath/ \ nod-cloud://cloud_storage/container
nd move \
--md project_name=<my project name> \
--md project_domain_name=<my project domain name> \
--md user_domain_name=<my user domain name> \
--working-dir nod://posix_storage/path/ \
--recursive \
--ignore-hidden \
nod://posix_storage/path/subpath/ \
nod-cloud://cloud_storage/container
nd move \ --md project_name=<my project name> \ --md project_domain_name=<my project domain name> \ --md user_domain_name=<my user domain name> \ --working-dir nod-cloud://cloud_storage/container/ \ --recursive \ --ignore-hidden \ nod-cloud://cloud_storage/container/path/ \ nod://posix_storage/path/
nd move \
--md project_name=<my project name> \
--md project_domain_name=<my project domain name> \
--md user_domain_name=<my user domain name> \
--working-dir nod-cloud://swift_connector/my_new_container/ \
--recursive \
--ignore-hidden \
nod-cloud://cloud_storage/container/path/ \
nod://posix_storage/path/
nd task list
This command list all tasks created by the user in the data mover service.
The columns describe:
Output | Description |
---|---|
TASK ID | ID of the task |
TASK NAME | Name of the task defined during the creation |
COMMENT | Associated comment |
CREATED BY | User who has created the task |
+--------------------------+------------------------------------------------------------+---------+-------------------+-----------------------+
| TASK ID | TASK NAME | COMMENT | CREATED BY | LAST EXECUTION STATUS |
+--------------------------+------------------------------------------------------------+---------+-------------------+-----------------------+
| 6287a774a91db0b194e97d8d | From /largedata2_pool/storagetestdata/data5 to my_pool1 | | Valery Guilleaume | done |
| 628b45f5a91db04211739dc6 | From /largedata2_pool/storagetestdata/data5 to my_pool1 | | Valery Guilleaume | done |
| 628b468ea91db04211739dc9 | From /largedata2_pool/storagetestdata/data5 to my_pool1 | | Valery Guilleaume | done |
| 6331d52ba91db02d6797e6ae | From nod://largedata2_pool/storagetestdata/ to vg--1598 | | Valery Guilleaume | stopped_by_user |
| 6331d5aca91db02d6797e6b1 | From nod://largedata2_pool/storagetestdata/ to vg--1598 | | Valery Guilleaume | stopped_by_user |
| 6331d5c5a91db02d6797e6b4 | From nod://largedata2_pool/storagetestdata/ to vg--1590 | | Valery Guilleaume | stopped_by_user |
| 6331d677a91db02d6797e6b7 | From nod://largedata2_pool/storagetestdata/ to vg--1598 | | Valery Guilleaume | done |
| 6331d692a91db02d6797e6ba | From nod://largedata2_pool/storagetestdata/ to vg--1598 | | Valery Guilleaume | stopped_by_user |
| 6331d948a91db02d6797e6c6 | From nod://largedata2_pool/storagetestdata/ to vg--1598 | | Valery Guilleaume | stopped_by_user |
| 6333ffdda91db091264b68a5 | From nod://largedata2_pool/storagetestdata/ to vg--1500 | | Valery Guilleaume | finished with warning |
| 6333fff6a91db092d94b68a2 | From nod://largedata2_pool/storagetestdata/ to vg--1500 | | Valery Guilleaume | finished with warning |
| 63355214a91db0397f128dbf | From nod://largedata2_pool/storagetestdata/ to vg--1500 | | Valery Guilleaume | done |
| 63355218a91db0397f128dc2 | From nod://largedata2_pool/storagetestdata/ to vg--1500 | | Valery Guilleaume | done |
| 63355220a91db0a6ec22073e | From nod://largedata2_pool/storagetestdata/ to vg--1501 | | Valery Guilleaume | done |
| 6335549aa91db0397f128dc5 | From nod://largedata2_pool/storagetestdata/ to vg--1500 | | Valery Guilleaume | done |
| 6335549da91db0397f128dc8 | From nod://largedata2_pool/storagetestdata/ to vg--1502 | | Valery Guilleaume | done |
| 633554a1a91db0a6ec220741 | From nod://largedata2_pool/storagetestdata/ to vg--1506 | | Valery Guilleaume | done |
| 63358216a91db0397f128dcb | From nod://largedata2_pool/storagetestdata/ to vg--1500 | | Valery Guilleaume | finished with warning |
| 6335822da91db0397f128dce | From nod://largedata2_pool/storagetestdata/ to vg--1502 | | Valery Guilleaume | done |
| 633584d5a91db0397f128dd1 | From nod://largedata2_pool/storagetestdata/ to vg--1502 | | Valery Guilleaume | finished with warning |
| 6336b341a91db0397f128dd4 | From nod://largedata2_pool/storagetestdata/ to vg--1500 | | Valery Guilleaume | done |
| 6336b35da91db0c2985ff420 | From nod://largedata2_pool/storagetestdata/ to vg--1500 | | Valery Guilleaume | done |
+--------------------------+------------------------------------------------------------+---------+-------------------+-----------------------+
| NUMBER OF TASK(S) 22 | | | |
+--------------------------+------------------------------------------------------------+---------+-------------------+-----------------------+
At the end of each task execution, the task result is displayed.
INFO Connecting with device flow...
INFO Connected with user Valery Guilleaume
Processed size ... done! [8.17MB in 39s]
Processed items ... done! [471 in 39s]
ID: 633ecc74975854bbc799a424
Task ID: 633ecc74a91db0f38f7abc2e
Name: From nod://largedata2_pool/storagetestdata/ to vguilleaume
Comment:
Created by: Valery Guilleaume
Nodes: 471 / 471
Size: 8.17 MB / 8.17 MB
Status: done
Anytime the status of task can be displayed.
# nd task status 633ecc74a91db0f38f7abc2e
where 633ecc74a91db0f38f7abc2e
is the id of the task
INFO Connecting with device flow...
INFO Connected with user Valery Guilleaume
ID: 633ecc74975854bbc799a424
Task ID: 633ecc74a91db0f38f7abc2e
Name: From nod://largedata2_pool/storagetestdata/ to vguilleaume
Comment:
Created by: Valery Guilleaume
Nodes: 471 / 471
Size: 8.17 MB / 8.17 MB
Status: done
nd task list-exec 6389c04605e7b8ff6df35cc4
This command list all tasks executed by the user in the data mover service.
The columns describe:
Output | Description |
---|---|
ID | ID of the executed task |
STARTED AT | Date when the task has been started |
FINISHED AT | Date when the task has been finished |
NODES | Number of files copied / Total number of files to be copied |
SIZE | Size of files copied / Total size of files to be copied |
STATUS | Status of the executed task |
+--------------------------+---------------------+---------------------+---------+---------------------+-----------------+
| ID | STARTED AT | FINISHED AT | NODES | SIZE | STATUS |
+--------------------------+---------------------+---------------------+---------+---------------------+-----------------+
| 6389c105be32a3507d90a6ea | 02 Dec 22 10:10 CET | 02 Dec 22 10:27 CET | 86 / 86 | 5.89 GB / 5.89 GB | done |
| 6389e5e191c3044a236f7baf | 02 Dec 22 12:47 CET | 02 Dec 22 12:49 CET | 3 / 86 | 108.38 MB / 5.89 GB | stopped_by_user |
| 6389e64391c3044a236f7c09 | 02 Dec 22 12:49 CET | 02 Dec 22 12:50 CET | 2 / 86 | 19.52 kB / 5.89 GB | stopped_by_user |
| 6389f33d91c3044a236f7c63 | 02 Dec 22 13:44 CET | 02 Dec 22 14:00 CET | 22 / 86 | 1.76 GB / 5.89 GB | stopped_by_user |
| 6389f704c4df1940f3253444 | 02 Dec 22 14:00 CET | 02 Dec 22 14:03 CET | 5 / 86 | 108.41 MB / 5.89 GB | stopped_by_user |
| 6389f7bac4df1940f325349e | 02 Dec 22 14:03 CET | 02 Dec 22 14:04 CET | 2 / 86 | 19.52 kB / 5.89 GB | stopped_by_user |
| 6389f812c4df1940f32534f8 | 02 Dec 22 14:05 CET | 02 Dec 22 14:06 CET | 2 / 86 | 19.52 kB / 5.89 GB | stopped_by_user |
| 6389fc3cc4df1940f3253552 | 02 Dec 22 14:23 CET | 02 Dec 22 16:11 CET | 0 / 0 | 0.00 B / 0.00 B | stopped_by_user |
| 638a188cfbd1fa439d567ae2 | 02 Dec 22 16:23 CET | 02 Dec 22 16:25 CET | 2 / 86 | 19.52 kB / 5.89 GB | stopped_by_user |
| 638a1916fbd1fa439d567b3c | 02 Dec 22 16:26 CET | 02 Dec 22 16:27 CET | 2 / 86 | 19.52 kB / 5.89 GB | stopped_by_user |
| 638a1986fbd1fa439d567b96 | 02 Dec 22 16:28 CET | 02 Dec 22 16:30 CET | 2 / 86 | 22.02 kB / 5.89 GB | stopped_by_user |
| 638a21c5fbd1fa439d567bf0 | 02 Dec 22 17:03 CET | 05 Dec 22 11:18 CET | 86 / 86 | 5.89 GB / 5.89 GB | done |
+--------------------------+---------------------+---------------------+---------+---------------------+-----------------+
Any questions ? Contact us