Object Storage CLI
    • PDF

    Object Storage CLI

    • PDF

    Article Summary

    Available in Classic and VPC

    This guide describes initial settings and brief instructions for Object Storage CLI, which is compatible with Amazon S3 CLI.

    Initial settings

    It describes how you can install and authenticate Object Storage CLI to use it.

    Installation

    The installation command is as follows:

    pip install awscli==1.15.85
    
    Note

    Authentication

    The following describes how to set the authentication information.

    ~$ aws configure
    AWS Access Key ID [****************leLy]: ACCESS_KEY_ID
    AWS Secret Access Key [None]: SECRET_KEY
    Default region name [None]: [Enter]
    Default output format [None]: [Enter]
    

    Cautions

    It describes precautions when using it in servers with assigned roles. Servers with assigned roles can use Object Storage commands without additional configurations.
    When using in servers with assigned roles, use the authentication key search API among the Metadata APIs. However, since a timeout can occur during this process, you can use the following command to extend the Metadata API timeout if necessary.

    export AWS_METADATA_SERVICE_TIMEOUT=10
    

    When using CLIs, the --endpoint-url option is a required value. For calling domains by region, see the Object Storage API guide.

    Command help

    Run the following command to check the help for smooth use of Object Storage CLI.

    aws help
    aws <command> help
    aws <command> <subcommand> help
    

    Commands

    It describes the Object Storage CLI.

    Create bucket

    The following is the command to create a bucket.

    • high-level

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 mb s3://<bucket_name>
      
    • API-level

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api create-bucket --bucket <bucket_name>
      
      Note

      For awscli version 2, --region us-east-1 option is required when running the create bucket command (mb).

    View bucket list

    The following is the command to search the bucket list.

    • high-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 ls
      
    • API-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api list-buckets
      

    Search object list

    Search whole object list

    The following is the command to search the whole object list.

    • high-level

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 ls s3://<bucket_name>
      
    • API-level

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api list-objects --bucket <bucket_name>
      

    Search object containing specific strings

    The following is the command to search the object list containing a specific string (e.g., foo) in the object name.

    • API-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api list-objects --bucket <bucket_name> --prefix <prefix> --query "Contents[?contains(Key, 'foo')]"
      

    Search size

    Search bucket size

    The option to search the bucket size can only be done by high-level command. The following is the command to search the bucket size.

    • high-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 ls s3://<bucket_name> --recursive --human-readable --summarize
      

    Search folder size

    The option to search the folder size can only be done by high-level command. The following is the command to search the folder size.

    • high-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 ls s3://<bucket_name>/<folder_name> --recursive --human-readable --summarize
      

    Upload object

    The following is the command to upload an object.

    • high-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 cp <local_file_path> s3://<bucket_name>[/<object_name>]
      
    • API-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api put-object --bucket <bucket_name> --key <object_name> --body <local_file_path>
      

    Upload multipart

    Follow the sequence in order to successfully execute multipart upload.

    1. Initiate multipart upload
      Enter the following command to generate and return the UploadId value.

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api create-multipart-upload --bucket <bucket_name> --key <object_name>
      
    2. Upload multipart

    Enter the following command to upload a part and return the ETag value of the uploaded part.

    aws --endpoint-url=https://kr.object.ncloudstorage.com s3api upload-part --bucket <bucket_name> --key <object_name> --part-number <sequential_integer> --body <local_file_path> --upload-id <upload_id>
    
    1. Complete multipart upload
      Enter the following command to complete the multipart upload of the parts uploaded above.

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api complete-multipart-upload --bucket <bucket_name> --key <object_name> --upload-id <upload_id> --multipart-upload file://<json_file>
      
      • json_file examples:
      {
        "Parts": [
          {"ETag": "90e6f8b09862bb7fdc080c89d53dbc65", "PartNumber": 1},
          {"ETag": "f158dfe47ee46ce22a80986cf8f4bdbb", "PartNumber": 2},
           ......
          {"ETag": "7d29c5f2bc97aebc6a395b52a0fb9d88", "PartNumber": 7}
        ]
      }
      

    Download object

    The following is the command to download an object.

    • high-level
      aws --endpoint-url=http://kr.object.ncloudstorage.com s3 cp s3://<bucket_name>/<object_name> <local_file_path>
      
    • API-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api get-object --bucket <bucket_name> --key <object_name> <local_file_path>
      

    Delete object

    The following is the command to delete an object.

    • high-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 rm s3://<bucket_name>/<object_name>
      
    • API-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api delete-object --bucket <bucket_name> --key <object_name>
      

    Delete all objects within a bucket/folder

    The following is the command to delete all objects within a bucket or folder.

    • high-level
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 rm s3://<bucket_name>/[<folder_name>/] --recursive
      

    Delete incomplete multipart upload

    The following describes how to check for incomplete multipart uploads and delete them:

    1. Search incomplete multipart upload
      You can search the information about the incomplete multipart upload by using the following command.

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api list-multipart-uploads --bucket <bucket_name>
      
    2. Delete incomplete multipart upload
      Use the command below to delete the incomplete multipart upload by entering upload_id searched above.

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api abort-multipart-uploads --bucket <bucket_name> --key <object_name> --upload-id <upload_id>
      

    Delete bucket

    The following is the command to delete a bucket.

    • high-level

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 rb s3://<bucket_name>
      

      You can delete a bucket that is not empty by adding the force option. Delete all objects belonging to a bucket before deleting the bucket.

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 rb s3://<bucket_name> --force
      
    • API-level

      aws --endpoint-url=https://kr.object.ncloudstorage.com s3api delete-bucket --bucket <bucket_name>
      
      Note

      Buckets that are not empty can't be deleted at the API-level.

    Sync

    The sync feature can only be performed with the high-level command. If you use the --delete option, then the files that don't exist in the source are also deleted in the destination. The sync command is as follows:

    • high-level
      • Sync all files in a bucket (folder) of the Object Storage to a local directory

        aws --endpoint-url=https://kr.object.ncloudstorage.com s3 sync s3://<bucket_name>[/<object_name>] <local_directory_name>
        
      • Sync all files in a local directory to a bucket (folder) of the Object Storage

        aws --endpoint-url=https://kr.object.ncloudstorage.com s3 sync <local_directory_name> s3://<bucket_name>[/<object_name>]
        
      • Sync all files in a bucket (folder) of the Object Storage to another bucket (folder)

        aws --endpoint-url=https://kr.object.ncloudstorage.com s3 sync s3://<source_bucket_name>[/<destination_object_name>] s3://<destination_bucket_name>[/<destination_object_name>]
        
        Note

        For awscli version 2, we recommend using version 1.15.x. because running the sync command can result in errors due to calling unsupported APIs.

    Access control list (ACL)

    You can set access control list (ACL) permissions for files in an Object Storage bucket (folder). The ACL feature can only be performed with the API-level command. The command is as follows:

    • API-level
      • Set the files in the Object Storage bucket as public
        aws --endpoint-url=https://kr.object.ncloudstorage.com s3api put-object-acl --bucket <bucket_name> --key <object_name> --acl public-read
        
      • Set the files in an Object Storage bucket as private
        aws --endpoint-url=https://kr.object.ncloudstorage.com s3api put-object-acl --bucket <bucket_name> --key <object_name> --acl private
        

    By using it with search list command (ls), you can set all files uploaded to an Object Storage bucket (folder) public. The command is as follows:

    • Set all files in the Object Storage bucket as public

      Caution

      This is a public setting for all files in the bucket, so be careful when executing the command.

      aws --endpoint-url=https://kr.object.ncloudstorage.com/ s3 ls s3://<bucket_name> --recursive | awk '{cmd="aws --endpoint-url=https://kr.object.ncloudstorage.com/ s3api put-object-acl --bucket <bucket_name> --acl public-read --key "$4; system(cmd)}'
      

    Debugging

    The commands related to debugging are as follows:

    • View detailed logs using the --debug* option
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 cp <local_file_path> s3://<bucket_name>[/<object_name>] --debug
      
    • Save as a log file
      aws --endpoint-url=https://kr.object.ncloudstorage.com s3 cp <local_file_path> s3://<bucket_name>[/<object_name>] --debug 2> debug.log
      

    Set CORS

    It sets CORS (Cross-Origin Resource Sharing) for an Object Storage bucket. Once an Object Storage bucket receives a cross-origin request, it checks the CORS configuration and allows the cross-origin request using the first CORS rule that matches the browser request. The following conditions have to be met for a CORS rule to be matched.

    • The request's Origin header must match the AllowedOrigins element.
    • The request method or Access-Control-Request-Method header (preflight request) must match the AllowedMethods element.
    • All headers specified in the Access-Control-Request-Method header (preflight request) must match the AllowedHeaders element. CORS configuration can only be done by the API-level command.

    The command is as follows:

    • API-level
      • Set CORS rules defined with a JSON file to the Object Storage bucket
        aws --endpoint-url=https://kr.object.ncloudstorage.com s3api put-bucket-cors --bucket <bucket_name> --cors-configuration file://cors.json
        
        cors.json:
        {
          "CORSRules": [
          {
              "AllowedHeaders": ["*"],
              "AllowedMethods": ["GET","PUT"],
              "AllowedOrigins": ["*"],
              "MaxAgeSeconds": 3000
            }
          ]
        }
        
      • Search the CORS rule set in the Object Storage bucket
        aws --endpoint-url=https://kr.object.ncloudstorage.com s3api get-bucket-cors --bucket <bucket_name>
        

    Was this article helpful?

    Changing your password will log you out immediately. Use the new password to log back in.
    First name must have atleast 2 characters. Numbers and special characters are not allowed.
    Last name must have atleast 1 characters. Numbers and special characters are not allowed.
    Enter a valid email
    Enter a valid password
    Your profile has been successfully updated.