MinIO

Here you will find an example of how to configure the MinIO Client (mc), a free S3 client for Linux, macOS and Windows.

MinIO client - S3 via commandline

The open source software MinIO differs from other clients like Cyberduck and WinSCP, as it does not offer a user interface, but only the command line interface (cli) Therefore minIO is more commonly used by users with higher technical knowledge who can teach themselves via documentation and help from online communities.

Download of the MinIO client

In the GitHub-Repository of the MinIO client, you find multiple download options. For example binary files for Linux, maxOS or Windows.

S3-Connections via command line (Linux & Windows)

Using the client in the command line is done by starting with the command mc which represents the client. Afterwards you might add typical commands like the linux commands cp, ls, cat, head, mv, and rm for S3 compatible storage services.

Configuration file setup

While it's possible to manually enter all parameters via the command line, using a configuration file is recommended. Using such a file (config.json) offers advantages in terms of consistency, reproducibility, and security. The configuration file is created by setting an alias and can then be manually modified if necessary. The command to create the configuration file is

mc alias set <my_resource_alias> https://global.datastorage.nrw:443/

The name for the placeholder term my_resource_alias can be freely chosen. The resulting configuration file can be found under ~/.mc/config.json under Linux or ?%USERPROFILE%\.mc\config.json under Windows. It should look like the following code:

{
    "version": "10",
    "aliases": {
        "my_resource_alias": {
        "url": "https://global.datastorage.nrw:443/",
        "accessKey": "<READ_OR_WRITE_ACCESS_KEY_FROM_COSCINE>",
        "secretKey": "<READ_OR_WRITE_SECRET_KEY_FROM_COSCINE>",
        "api": "s3v4",
        "path": "off"
        }
    }
} 

The access key and secret key for input into the configuration file, can be found in the resource configuration in Coscine. Depending on the usecase, you can add the key pair for either read or write permissions.

File operations:

After the creation of the configuration file, you can use different commands with the client mc. Typical operations are listing, uploading files and downloading files. For more information please visit the corresponding Git Repository.

Warning

For access to the Datastorage.nrw the virtual host style needs to be used, to satisfy the demands of the manufacturer. Additionally the option "path" needs to be set to "off".

Regarding folder actions, we recommend taking a look at the documentation on Managing Files and Folders. There you will find a list of actions supported by Coscine and information on future developments.

List files and versions

Usecase	Command
List files in a resource	mc ls my_resource_alias/<bucket name>
List file versions	mc ls --versions my_resource_alias/<bucket name>

Download

Usecase	Command
Download a file from a resource	mc cp my_resource_alias/<bucket name>/data.csv ./

Upload and Deletion of files

Keep in mind that overwriting an existing file will create a new file or object version. File versions will block storage space until the end of the retention period.

Upload

Usecase	Command
Upload a file into a resource	mc cp ./localFile.csv my_resource_alias/<bucket name>/

Delete

Deleted files can not restored from user side. Please evaluate any deletion action carefully. Therefore it is recommended to use the optional --dry-run flag to try out the result of the deletion, especially for recursive deletions. Trying out the deletion in a dry run will show the results of the deletion action, without executing the deletion command. In case the result equals the expectation, the command can be repeated without the dry run flag to execute the deletion.

Usecase	Command
Deletion of a single file	mc rm my_resource_alias/<bucket name>/myFile.csv
Preview of a recursive deletion	mc rm --recursive --dry-run my_resource_alias/<bucket name>/myFile.csv
Recursive deletion	mc rm --recursive my_resource_alias/<bucket name>/myFile.csv