Skip to content

API Guide

Index

  1. Preface
  2. Getting Started
  3. The Coscine REST API
  4. Further Information
  5. Coscine Python Binding

Preface

The traditional and most common way of using Coscine is through the web interface. In addition to that Coscine provides a REST API as an alternative means of interfacing with the system.

At first using the REST API may seem a little more complicated than using the web interface but it rewards users with the possibility for automation (of workflows)), integration (into existing software) and many more convenient ways that take over some of your work in dealing with research data.

Smart users may be able to design their workflow in such a way, that after they set up their toolchain they do not need to interface with Coscine in any shape or form. It should be the ultimate goal to offload as much work as possible to computer programs, scripts and the like.

This project serves as a simple introduction to the Coscine API. Over the course of this tutorial we will build our own python binding to get to know the API. You can choose to skip the tutorial if you would just like to incorporate the python connector in your python project without having to think too much about what is going on behind the scenes.

Getting Started

Before we dive into the tutorial we need to ensure that we both know what we are talking about.

This guide is using python 3.x - a very simple programming language that anyone is be able to pick up in an hour or less. If you do not know how to use python but do have some knowledge of other programming languages, then you should be able to follow the tutorial and adapt the code to your needs. However, if you do not have a technical background or are kind of computer illiterate then you should take a look at a simple python 3.x tutorial. You do not need any of the advanced stuff, just a solid foundation. We recommend the Python Programming tutorial by Derek Banas as an entry point. It will get you started with installing and grasping the basic concepts of the python programming language. Once you are set up you can proceed with the following steps.

The Coscine REST API

Now lets see what this API fuzz is all about...

API Token

To be able to use the API we require an API token. An API Token is a means of authenticating us to Coscine without a username or password. We usually create a token for each script or process that is interacting with the API in some way. A detailed explanation and instructions on how to create a token can be found here. Once we have a token we are ready to go.

Creating our base class

Before we start talking to the API we'll set up a basic little framework to make things easier along the line. We define a class Coscine that shall be responsible for abstracting our interaction with the Coscine API. In the class we define two properties: * The Coscine API base url * our token for authentification.

Immediately when creating an instance of our class we would like to set our token used for all future requests we make. Since our token alone is useless when used in request headers we must preceed it by the identifier bearer. This identifies our token as a token to the server, otherwise it would just appear to be some kind of value. Preceeding that value with bearer gives that value a name and officially makes it a token in the eyes of the server.

class Coscine:
        # Coscine API base url
        API = "https://coscine.rwth-aachen.de/coscine/api/"

        token = None

        def __init__(self, token):
            self.token = "Bearer " + token

It should be noted that a token should be kept secret. If someone gets a hold of your token then they can do whatever they want inside one of your project. Therefore you should never save your token inside of your sourcecode if you plan to make that code public i.e. upload it to github or store it on a public server. Store your token in an external file and load that file before creating an instance of the API class. Add your token file to your gitignore if you are using github. You would not want your research project get deleted by anonymous users online or your research data stolen by foreign companies. Keep your data safe by keeping your token secret!

Making requests

To be able to talk to the REST API we need some kind of interface, that lets us send and receive messages over the internet. Luckily, the python module requests does provide that kind of functionality in a simple to use manner. Using the requests module we can perform get, post, put and delete requests and thus are able to fully interact with a RESTful API. With each successful request we get an answer back from the server. Such answers are encoded in JSON format. Since we are using python there is already a module available that lets us decode JSON into a python object using one simple function call.

Now let us use the stuff we just learnt and incorporate it to build the basic framework of our API binding.

import requests

class Coscine:
        # Coscine API base url
        API = "https://coscine.rwth-aachen.de/coscine/api/"

        token = None

        def __init__(self, token):
            self.token = "Bearer " + token

        def get(self, endpoint):
            url = self.API + endpoint
            response = requests.get(url, headers = {
                "Authorization": self.token}, verify=True)
                return(json.loads(response.text))

        def put(self, endpoint, data):
            url = self.API + endpoint
            response = requests.put(url, headers = {
                "Authorization": self.token}, data = data)
            return(json.loads(response.text))

        def post(self, endpoint, data):
            url = self.API + endpoint
            response = requests.post(url, headers = {
                "Authorization": self.token}, data = data)
            return(json.loads(response.text))

        def delete(self, endpoint):
            url = self.API + endpoint
            response = requests.delete(url, headers = {
                "Authorization": self.token}, data = data)
            return(json.loads(response.text))

Now with that out of the way we can finally start interacting with the API!

API Structure

Coscines API is structured into different endpoints, each taking care with one part of the Coscine environment. You will have a seperate endpoint for user interaction, project configuration, metadata management and so on.

Endpoint Description
Notifications The NotificationApi handles the sync and async sending of notification.
Resources This api provides methods for interacting with resources.
Project This api provides methods to access, create and update project information.
User This api provides methods to access and update user information.
TreeApi The TreeApi handles the retrieving or storing metadata to a certain path.
Organization This api provides methods for interacting with organizations.
Notices This project provides the Rest Api for dealing with markdown files.
BlobApi The BlobApi handles the interaction with binary large objects in the CoScInE environment.

We'll demonstrate how to use some of the functionality exposed by these endpoints by creating a sample use case.

Let's assume we already created some Project X via the web interface. Within this project lies a Resource containing some data. We would like to fetch this data now. To do that, we need to interact with the Project and with the Resources Endpoint.

cos = Coscine(our_token)
# Get Project id
projects = cos.get("Coscine.Api.Project/Project")
id = None
for p in projects:
    if p.displayName == "X":
        id = p.id
if id is not None:
    # Get a list of the projects resources
    resources = cos.get("Coscine.Api.Project/" + id + "/Resources")

Now let's upload a file to an existing resource. This is a very common use case and probably one of the most used features of the API.

from coscine import *

tokenfile = open("./token.txt", "r")
token = tokenfile.read()
tokenfile.close()

cos = Coscine(token)

# Get a list of all resources associated with us 
resources = cos.get("Coscine.Api.Resources/Resource")

# Search for a specfic resource (Here with placeholder name 'NAME')
id = None
for resource in resources:
    if resource["displayName"] == 'NAME':
        id = resource["id"]

# Read the file we'd like to upload
filename = "research_data.txt"
fd = open(filename, "r")
data = fd.read()
fd.close()

# Use the blob api to upload our file to the resource
cos.put("Coscine.Api.Blob/Blob/" + id + "/" + filename, data)

At this point our file has been uploaded to the resource, but it is lacking all of our important metadata! Therefore we need to upload some metadata and associate it with the file. In order to get a template for our metadata we can utilize the coscine form generator.

Specify an Application Profile, a Shape URL and select a MimeType for the form generator to create a JSON template which you can embed inside of your application. A detailed tutorial on how to use the form generator can be found here. Once you have got your template, it is time to fill in the values and associate it with your upload.

metadata = {
    "@type": "https://purl.org/coscine/ap/engmeta/",
    "http://purl.org/dc/terms/publisher": {
        "@value": "John Doe",
        "@type": "http://www.w3.org/2001/XMLSchema#string"
    },
    "http://purl.org/dc/terms/creator": {
        "@value": "John Doe",
        "@type": "http://www.w3.org/2001/XMLSchema#string"
    },
    "http://purl.org/dc/terms/created": {
        "@value": "2021-03-01",
        "@type": "http://www.w3.org/2001/XMLSchema#date"
    },
    "http://purl.org/dc/terms/issued": {
        "@value": "2021-03-02",
        "@type": "http://www.w3.org/2001/XMLSchema#date"
    },
    "http://purl.org/dc/terms/available": {
        "@value": "2021-04-30",
        "@type": "http://www.w3.org/2001/XMLSchema#date"
    },
    "http://www.ub.uni-stuttgart.de/dipling#version": {
        "@value": "1.0",
        "@type": "http://www.w3.org/2001/XMLSchema#string"
    },
    "http://www.ub.uni-stuttgart.de/dipling#step": {
        "@value": "2",
        "@type": "http://www.w3.org/2001/XMLSchema#string"
    }
}
# Change values of the metadata dictionary here

# Upload metadata and associate with file
cos.put("Coscine.Api.Tree/Tree/" + id + "/" + filename, metadata)

Further information

You can find further information of the API by visiting the respective API endpoint documentation. A list of all endpoints and their documentation can be found here.

Coscine Python Binding

The final Coscine python binding can be downloaded by cloning this repository. Simply copy the directory to the destination you would like to use it from and include it in your program.