Versioning of files
With the start of provisioning for DataStorage.nrw resources, a new feature is also activated: File Versioning.
This works in a similar way to the Git commit history by retaining all file versions. If a file with the same name as an existing file is uploaded, the new file will be saved, but the old file will be kept as the previous version instead of being overwritten. There is no maximum number of file versions that are retained. File versioning in Coscine is designed to allow the recovery of deleted files after a short period of time and not to keep versioned files forever.
Note
Please note that file versions block storage space within your resource, even if the respective file versions are not displayed in the UI or API. Storage space blocked by deleted versions is freed up 7 days after deletion.
Manual Deletion of Versions
Currently Versions can not be deleted manually and the policy which defines the retention period can not be changed. For the future an integration into the UI is planned, to allow individual settings regarding deletion of versions.
Display of Versions
For Web and S3 resources information about file versions can be displayed in the user interface.
Display in Web UI:
By default the total quota used is displayed. By clicking the button in the image below, it enables highlighting the degree, how much of the currently used filesize stems from backup versions. Before the activation you will see the total used quota in a dark blue or red color, depending on the total usage of available quota. After the activation, a darker color is used for the current file versions and a lighter color to display the outdated versions, which will be automatically deleted at the end of the retention period. The activation of the button does trigger the changed disply in the resource overview and the popup modal of the resource detail information. While the versioning display offers information how much total space will be freed after the retention time, it is unclear if this might happen in 7 days or earlier, because of individual files different deletion dates.
Display in S3 Clients:
Typically in S3 Clients you do not see versions. It may happen that individual S3 clients might display versions, but even then those versions can not be deleted and will occupy the space until their scheduled deletion.
Quota management - best practices to save space
Versions can become a huge factor when planning how much quota you need. Try to plan ahead how often you will update files.
Update frequency
If you update your whole data once per week, you constantly have two versions. If you update daily, you have seven versions up to eight, depending on the difference between reupload time versus the time of the scheduled deletion. Keep your update frequency reasonable
Folder reuploads
If you are regularly reuploading newer versions of files, do that individually and upload them into the folder to save your quota. A typical case how unnecessary versions are created is by reuploading folders, where only few files have been changed. This typical behavior is learnt cause it is convenient and works well with modern operating systems like Windows/Linux/Mac OS. Unless those systems, coscine does not yet offer functions for skipping overwriting operations for files with the same filename and size. In case whole folders are reuploaded, a new version is created for all files, which in the end occupies quota. Upload files individually, not whole folders