WORM
WORM stands for Write once, read many. As this name suggests, it is a resource for storing research data with high protection requirements. Once stored, data can never be deleted, changed or overwritten. This ensures a high level of security when storing the data.
Warning: WORM resources can never be deleted, modified or overwritten
Once files have been stored in a WORM resource, they can no longer be deleted or edited! This is why this type of resource is only suitable for data that absolutely requires such high levels of security. For most research data, Web and S3 resources already provide the necessary functionality. Careless use of WORM resources wastes storage space unnecessarily, which is then no longer available to you or other researchers. Before applying for WORM storage space, you must consult your local Coscine support team. These teams will be happy to discuss your options with you!.
Properties of a WORM resource
- Storage space: Storage space for WORM resources must always be requested via Coscine-JARDS.
- Use cases: Data with high security requirements, large volumes of data, automated data submission, metadata management using Readme files, use of S3 storage space in conjunction with Jupyter Notebooks
- Access methods: User interface, API, S3
- Metadata: The entry of metadata via the metadata profile is not mandatory. The storage space application in Coscine-JARDS must therefore clearly explain what research data is involved and how it will be ensured that this data is described and made discoverable by means of metadata. If metadata, like the associated data, is to be made immutable, it can be stored as files (e.g. README files) in the S3 bucket. Metadata maintained via the user interface remains editable.
- Other special considerations: To submit a storage space request for a WORM resource, you must first have a consultation with your local Coscine support team. Users wishing to carry out complex S3 operations will find an overview of the S3 policies used in Coscine here.
Creating and using a WORM resource
Once the storage space request has successfully passed the review process and the quota has been allocated to the relevant project, you can create WORM resources within that project. Click on the menu icon to open the Resource Configuration tab. There you will find the credentials you need to connect directly to the resource via the S3 protocol using an [S3 client].
Warning: Recommended S3 clients
The Coscine team is unable to provide full support for the use of S3 clients. The S3 clients WinSCP, Cyberduck and MinIO Client have already been successfully tested with Coscine. You can find more detailed information on how to use them on their respective documentation pages. The Coscine team has no experience with the functionality of other applications (e.g. rclone, S3CMD).
Overview of credentials and connection parameters
To connect to WORM resources, certain data is required; the names of these may vary depending on the tool. Here, therefore, is a brief explanation of the necessary parameters and their common names.
Connection parameters for DataStorage.nrw resources
| Name | Value | Note |
|---|---|---|
| Server or Endpoint | global.datastorage.nrw | - |
| Port | 443 | - |
| Bucket Name | XXXXXXXX-XXXX-XXXX-... | - |
| Path | Same as Bucket Name | Warning! Depending on the tool, the path can be found in the advanced or optional settings. However, it is essential for using S3 storage! |
Hint
When connecting to DataStorage.nrw resources, it is recommended that you specify the hostname using the virtual host style. Using the path style may result in connection problems.
Two sets of login credentials are generated automatically. One set for read and write access, and one set that allows read-only access.
| Name(s) | Value | Note |
|---|---|---|
| Access Key (ID) | read_XXXXXX... or write_XXXXXX | Read and Write Access Keys only work with their respective Secret Keys. |
| Secret (Access) Key | XXXXXXXX-XXXX-XXXX-XXXX-... | - |