Object Storage CLI

English

Object Storage CLI

Article summary

Did you find this summary helpful?

Thank you for your feedback

Available in Classic and VPC

This guide describes initial settings and brief instructions for Object Storage CLI, which is compatible with Amazon S3 CLI.

Initial settings

It describes how you can install and authenticate Object Storage CLI to use it.

Installation

The installation command is as follows:

pip install awscli==1.15.85

Note

Since awscli version 1.16.x does not support some features, we recommend using version 1.15.x.
awscli version 1.15.85 supports Python 3.8 or earlier.
See the following for related documents.
- Document: https://pypi.org/project/awscli/1.15.85/
- git: https://github.com/aws/aws-cli/tree/1.15.85

Authentication

The following describes how to set the authentication information.

~$ aws configure
AWS Access Key ID [****************leLy]: ACCESS_KEY_ID
AWS Secret Access Key [None]: SECRET_KEY
Default region name [None]: [Enter]
Default output format [None]: [Enter]

Cautions

It describes precautions when using it in servers with assigned roles. Servers with assigned roles can use Object Storage commands without additional configurations.
When using in servers with assigned roles, use the authentication key search API among the Metadata APIs. However, since a timeout can occur during this process, you can use the following command to extend the Metadata API timeout if necessary.

export AWS_METADATA_SERVICE_TIMEOUT=10

When using CLIs, the --endpoint-url option is a required value. For calling domains by region, see the Object Storage API guide.

Command help

Run the following command to check the help for smooth use of Object Storage CLI.

aws help
aws <command> help
aws <command> <subcommand> help

Commands

It describes the Object Storage CLI.

Create bucket

The following is the command to create a bucket.

high-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 mb s3://<bucket_name>

API-level
```
aws --endpoint-url=https://kr.object.ncloudstorage.com s3api create-bucket --bucket <bucket_name>
```
Note
For awscli version 2, --region us-east-1 option is required when running the create bucket command (mb).

View bucket list

The following is the command to search the bucket list.

high-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 ls

API-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api list-buckets

Search object list

Search whole object list

The following is the command to search the whole object list.

high-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 ls s3://<bucket_name>

API-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api list-objects --bucket <bucket_name>

Search object containing specific strings

The following is the command to search the object list containing a specific string (e.g., foo) in the object name.

API-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api list-objects --bucket <bucket_name> --prefix <prefix> --query "Contents[?contains(Key, 'foo')]"

Search size

Search bucket size

The option to search the bucket size can only be done by high-level command. The following is the command to search the bucket size.

high-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 ls s3://<bucket_name> --recursive --human-readable --summarize

Search folder size

The option to search the folder size can only be done by high-level command. The following is the command to search the folder size.

high-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 ls s3://<bucket_name>/<folder_name> --recursive --human-readable --summarize

Upload object

The following is the command to upload an object.

high-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 cp <local_file_path> s3://<bucket_name>[/<object_name>]

API-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api put-object --bucket <bucket_name> --key <object_name> --body <local_file_path>

Upload multipart

Follow the sequence in order to successfully execute multipart upload.

Initiate multipart upload
Enter the following command to generate and return the UploadId value.

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api create-multipart-upload --bucket <bucket_name> --key <object_name>

Upload multipart

Enter the following command to upload a part and return the ETag value of the uploaded part.

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api upload-part --bucket <bucket_name> --key <object_name> --part-number <sequential_integer> --body <local_file_path> --upload-id <upload_id>

Complete multipart upload
Enter the following command to complete the multipart upload of the parts uploaded above.

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api complete-multipart-upload --bucket <bucket_name> --key <object_name> --upload-id <upload_id> --multipart-upload file://<json_file>

json_file examples:

{
  "Parts": [
    {"ETag": "90e6f8b09862bb7fdc080c89d53dbc65", "PartNumber": 1},
    {"ETag": "f158dfe47ee46ce22a80986cf8f4bdbb", "PartNumber": 2},
     ......
    {"ETag": "7d29c5f2bc97aebc6a395b52a0fb9d88", "PartNumber": 7}
  ]
}

Download object

The following is the command to download an object.

high-level

aws --endpoint-url=http://kr.object.ncloudstorage.com s3 cp s3://<bucket_name>/<object_name> <local_file_path>

API-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api get-object --bucket <bucket_name> --key <object_name> <local_file_path>

Delete object

The following is the command to delete an object.

high-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 rm s3://<bucket_name>/<object_name>

API-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api delete-object --bucket <bucket_name> --key <object_name>

Delete all objects within a bucket/folder

The following is the command to delete all objects within a bucket or folder.

high-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 rm s3://<bucket_name>/[<folder_name>/] --recursive

Delete incomplete multipart upload

The following describes how to check for incomplete multipart uploads and delete them:

Search incomplete multipart upload
You can search the information about the incomplete multipart upload by using the following command.
```
aws --endpoint-url=https://kr.object.ncloudstorage.com s3api list-multipart-uploads --bucket <bucket_name>
```

Delete incomplete multipart upload
Use the command below to delete the incomplete multipart upload by entering upload_id searched above.

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api abort-multipart-uploads --bucket <bucket_name> --key <object_name> --upload-id <upload_id>

Delete bucket

The following is the command to delete a bucket.

high-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 rb s3://<bucket_name>

You can delete a bucket that is not empty by adding the force option. Delete all objects belonging to a bucket before deleting the bucket.

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 rb s3://<bucket_name> --force

API-level

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api delete-bucket --bucket <bucket_name>

Note

Buckets that are not empty can't be deleted at the API-level.

Sync

The sync feature can only be performed with the high-level command. If you use the --delete option, then the files that don't exist in the source are also deleted in the destination. The sync command is as follows:

high-level

Sync all files in a bucket (folder) of the Object Storage to a local directory

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 sync s3://<bucket_name>[/<object_name>] <local_directory_name>

Sync all files in a local directory to a bucket (folder) of the Object Storage

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 sync <local_directory_name> s3://<bucket_name>[/<object_name>]

Sync all files in a bucket (folder) of the Object Storage to another bucket (folder)
```
aws --endpoint-url=https://kr.object.ncloudstorage.com s3 sync s3://<source_bucket_name>[/<destination_object_name>] s3://<destination_bucket_name>[/<destination_object_name>]
```
Note
For awscli version 2, we recommend using version 1.15.x. because running the sync command can result in errors due to calling unsupported APIs.

Access control list (ACL)

You can set access control list (ACL) permissions for files in an Object Storage bucket (folder). The ACL feature can only be performed with the API-level command. The command is as follows:

API-level

Set the files in the Object Storage bucket as public

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api put-object-acl --bucket <bucket_name> --key <object_name> --acl public-read

Set the files in an Object Storage bucket as private

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api put-object-acl --bucket <bucket_name> --key <object_name> --acl private

By using it with search list command (ls), you can set all files uploaded to an Object Storage bucket (folder) public. The command is as follows:

Set all files in the Object Storage bucket as public

Caution

This is a public setting for all files in the bucket, so be careful when executing the command.

aws --endpoint-url=https://kr.object.ncloudstorage.com/ s3 ls s3://<bucket_name> --recursive | awk '{cmd="aws --endpoint-url=https://kr.object.ncloudstorage.com/ s3api put-object-acl --bucket <bucket_name> --acl public-read --key "$4; system(cmd)}'

Debugging

The commands related to debugging are as follows:

View detailed logs using the --debug* option

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 cp <local_file_path> s3://<bucket_name>[/<object_name>] --debug

Save as a log file

aws --endpoint-url=https://kr.object.ncloudstorage.com s3 cp <local_file_path> s3://<bucket_name>[/<object_name>] --debug 2> debug.log

Set CORS

It sets CORS (Cross-Origin Resource Sharing) for an Object Storage bucket. Once an Object Storage bucket receives a cross-origin request, it checks the CORS configuration and allows the cross-origin request using the first CORS rule that matches the browser request. The following conditions have to be met for a CORS rule to be matched.

The request's Origin header must match the AllowedOrigins element.
The request method or Access-Control-Request-Method header (preflight request) must match the AllowedMethods element.
All headers specified in the Access-Control-Request-Method header (preflight request) must match the AllowedHeaders element. CORS configuration can only be done by the API-level command.

The command is as follows:

API-level

Set CORS rules defined with a JSON file to the Object Storage bucket

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api put-bucket-cors --bucket <bucket_name> --cors-configuration file://cors.json

cors.json:
{
  "CORSRules": [
  {
      "AllowedHeaders": ["*"],
      "AllowedMethods": ["GET","PUT"],
      "AllowedOrigins": ["*"],
      "MaxAgeSeconds": 3000
    }
  ]
}

Search the CORS rule set in the Object Storage bucket

aws --endpoint-url=https://kr.object.ncloudstorage.com s3api get-bucket-cors --bucket <bucket_name>

Was this article helpful?

What's Next

Archive Storage CLI

Table of contents

Initial settings
Command help
Commands