Subscribe to Email Updates

    S3 Object Storage

    Protocol S3 is Used to communicate with Cloud Storage and objects storage providers:

     

    Nodeum supports differnent protocols to communicate with public cloud provider, private cloud provider and with object storage vendor.

    One of them is S3 but Nodeum allows also connections with Swift, Azure Blob, ....

    We have validated and certified the connection of our product with different object storage provider.

    Provider supported

    The following compatibility matrix highlight the different technologies and vendors Nodeum is compatible with:

    Storage Provider Type Protocol
    Amazon S3 Public AWS S3
    Microsoft Azure Public Azure Blob
    Google Cloud (°) Public Google Storage
    Wasabi Hot Cloud Storage Public S3 compatible
    Cloudian HyperStore Private / On Premise S3 compatible
    Dell EMC ECS Private / On Premise S3 compatible
    Scality Ring Private / On Premise S3 compatible
    OpenStack Swift (°) Private / On Premise Swift
    IBM Cloud Object Storage (COS) Private / On Premise S3 compatible
    NetApp StorageGRID Private / On Premise S3 compatible

    You can find more details about each vendor integration into the following section. Lean more 

    (°) Google Cloud and OpenStack Swift support only files size less or equal to 1 GB.

    Security

    The system prevents any lock-in and always give the freedom to the customer to schedule data movement from any cloud provider to any alternative cloud provider. This can be schedule in the frequency the customer decides.

     

    Available Options

    Option(s) Default Value Description

    default_acl

    “private” By default, it is the "private" ACL which set the default canned ACL to apply to all written s3 objects,
    Others ACL be set according to Amazon S3 implementation. The following URL lists these options :
    https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl

    retries

    5

    It is the number of times the system will do a a retry when a S3 transaction failed.

    storage_class

    "standard" This option has to be supported by the storage vendor, it allow to store the objects with specified storage class. Possible values are standard, standard_ia, onezone_ia, reduced_redundancy and intelligent_tiering.

    connect_timeout

    "300" (seconds) It is the time to wait for connection before giving up

    readwrite_timeout

    "120" (seconds) It is the time to wait between read/write activity before giving up

    list_object_max_keys

    “1000” This specify the maximum number of keys returned by S3 list object API.

    max_stat_cache_size

    "100,000" entries (about 40MB) maximum number of entries in the stat cache, and this maximum is also treated as the number of symbolic link cache.

    no_check_certificate

      The server certificate won't be checked against the available certificate authorities. This is very useful if there is self signed certificate.

    ssl_verify_hostname

    2 When 0, do not verify the SSL certificate against the hostname. 1 or 2 ?

    multireq_max

    20 This is about the maximum number of parallel request for listing objects

    parallel_count

    5 Number of parallel request for uploading big objects. by multipart post request,and sends parallel requests. It is necessary to set this value depending on a CPU and a network band.

    multipart_size

    10 Part size, in MB, for each multipart request. The minimum value is 5 MB and the maximum value is 5 GB.

    singlepart_copy_limit

    512 Maximum size, in MB, of a single-part copy before trying multipart copy.

    nomultipart

      Disable multipart uploads

    enable_content_md5

    disable Allow check data integrity of uploads via the MD5 checksum. This can add CPU overhead to transfers.

    ecs

    disable This option instructs a query to the ECS container credential metadata address instead of the instance metadata address.

    iam_role

    no IAM role This option requires the IAM role name or "auto". If you specify "auto", it will automatically use the IAM role names that are set to an instance. If you specify this option without any argument, it is the same as that you have specified the "auto".

    ibm_iam_auth

    is not using IBM IAM authentication This option instructs the use of IBM IAM authentication. In this mode, the AWSAccessKey and AWSSecretKey will be used as IBM's Service-Instance-ID and APIKey, respectively.

    ibm_iam_endpoint

    https://iam.bluemix.net Sets the URL to use for IBM IAM authentication.

     

    Related Categories

    Hardware Matrix S3 object storage

    You may also like:

    Hardware Matrix Product Specification

    Tape Library Storage Compatibility

    Nodeum is natively compatible with the following Tape Library and Tape Drive. The solution manages the tape library auto...

    Hardware Matrix

    NAS Storage - Compatibility

    List of NAS Storage Compatible to connect to Nodeum Storage Provider Type NAS All series providing NFS or SMB file shari...

    Hardware Matrix

    Hardware Compatibility Matrix

    List of Server compatible to deploy Nodeum X86 SERVER

    Let Us Know What You Thought about this Post.

    Put your Comment Below.

    Learn and grow with award-winning support and a thriving community behind you.

    Get the free version