Protocol S3 is Used to communicate with Cloud Storage and objects storage providers:
Nodeum supports different protocols to communicate with public cloud provider, private cloud provider and with object storage vendor.
Protocol Supported
Nodeum supports these following protocols :
- S3
- Swift
- Azure Blob
Provider supported
We validated and certified the connection of our product with different object storage provider.
The following compatibility matrix highlight the different technologies and vendors Nodeum is compatible with:
| Provider | Product | Type | Protocol | Nodeum Connector | Primary Storage | Secondary Storage |
| Amazon | AWS S3 Storage | Public | S3 | Amazon AWS S3 Storage | YES | YES |
| CEPH | On-Premise | S3 | Generic S3 | YES | YES | |
| Cloudian | HyperStore | On-Premise | S3 | Cloudian HyperStore | YES | YES |
| DellEMC ECS | ECS | On-Premise | S3 | DellEMC ECS | YES | YES |
| FileBase | Public | S3 | Generic S3 | YES | YES | |
| Google Cloud Storage | Public | Google Storage | Google Cloud Storage | YES | YES | |
| Huawei | OceanStore | On-Premise | S3 | Generic S3 | YES | YES |
| IBM | IBM Cloud Object Storage (COS) | On-Premise | S3 | Generic S3 | YES | YES |
| MinIO | On-Premise | S3 | Generic S3 | YES | YES | |
| Microsoft | Azure Blob | Public | Blob | Microsoft Azure Blob Storage | YES | YES |
| NetApp | StorageGRID | On-Premise | S3 | Generic S3 | YES | YES |
| OpenStack SwIFT (°) |
non-Keystone Keystone v2.0 Keystone v3.0 |
On-Premise | Swift | Openstack Swift | YES | YES |
| Quantum |
Activescale ActiveScale Cold Storage |
On-Premise | S3 | Quantum ActiveScale | YES | YES |
| Scality | Scality Ring | On-Premise | S3 | Scality Ring | YES | YES |
| Wasabi Hot Cloud Storage | Public | S3 | Wasabi Hot Cloud Storage | YES | YES |
You can find more details about each vendor integration into the following section. Lean more
(°) Google Cloud and OpenStack Swift support only files size less or equal to 1 GB.
S3 Backend supported
- S3FS
s3fs is a FUSE filesystem backed by Amazon S3 that allows to mount an S3 bucket as a local filesystem. It stores files natively and transparently in S3 (i.e., you can use other
programs to access the same files).
- Rclone
Rclone is a command line program to manage files on cloud storage. It is a feature rich alternative to cloud vendors' web storage interfaces.
Security
SSE Encryption is supported and https can be activated.
Available Options for S3FS
| Option(s) | Default Value | Description |
|
connect_timeout |
"300" (seconds) | It is the time to wait for connection before giving up |
|
default_acl |
“private” | By default, it is the "private" ACL which set the default canned ACL to apply to all written s3 objects, Others ACL be set according to Amazon S3 implementation. The following URL lists these options : https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl |
|
ecs |
disable | This option instructs a query to the ECS container credential metadata address instead of the instance metadata address. |
|
enable_content_md5 |
disable | Allow check data integrity of uploads via the MD5 checksum. This can add CPU overhead to transfers. |
|
iam_role |
no IAM role | This option requires the IAM role name or "auto". If you specify "auto", it will automatically use the IAM role names that are set to an instance. If you specify this option without any argument, it is the same as that you have specified the "auto". |
|
ibm_iam_auth |
is not using IBM IAM authentication | This option instructs the use of IBM IAM authentication. In this mode, the AWSAccessKey and AWSSecretKey will be used as IBM's Service-Instance-ID and APIKey, respectively. |
|
ibm_iam_endpoint |
||
|
list_object_max_keys |
“1000” | This specify the maximum number of keys returned by S3 list object API. |
|
max_stat_cache_size |
"100,000" entries (about 40MB) | maximum number of entries in the stat cache, and this maximum is also treated as the number of symbolic link cache. |
|
multipart_size |
10 | Part size, in MB, for each multipart request. The minimum value is 5 MB and the maximum value is 5 GB. Default value (10MB), means that file bigger than 10MB will be // and file smaller than 10MB will be serial. |
|
multireq_max |
20 | This is about the maximum number of parallel request for listing objects |
|
no_check_certificate |
The server certificate won't be checked against the available certificate authorities. This is very useful if there is self signed certificate. | |
|
nomultipart |
Disable multipart uploads | |
|
parallel_count |
5 | Number of parallel request for uploading big objects by multipart post request and sends parallel requests. It is necessary to set this value depending on a CPU and a network band. |
|
readwrite_timeout |
"120" (seconds) | It is the time to wait between read/write activity before giving up |
|
retries |
5 |
It is the number of times the system will do a a retry when a S3 transaction failed. |
|
singlepart_copy_limit |
512 |
Maximum size, in MB, of a single-part copy before trying multipart copy. |
|
ssl_verify_hostname |
2 |
When 0, do not verify the SSL certificate against the hostname. 1 or 2 ? |
|
storage_class |
"standard" | This option has to be supported by the storage vendor, it allow to store the objects with specified storage class. Possible values are standard, standard_ia, onezone_ia, reduced_redundancy and intelligent_tiering. |
%20W%26G%20Dark%20Back_150px.png?width=70&name=Nodeum(Single)%20W%26G%20Dark%20Back_150px.png)
Let Us Know What You Thought about this Post.
Put your Comment Below.