In the following sections, we'll demonstrate the API usage with examples using the API server that is running on Pittsburgh Super Computing. The API base URI is https://ccd4.vm.bridges.psc.edu/ccd-api.
This API requires user to be authenticated. Before using this API, the user will need to go to Causal-Web App and create an account.
Table of Contents
- Getting JSON Web Token(JWT)
- 1. Data Management
- Upload small data file
- Resumable data file upload
- List all dataset files of a user
- Get the detail information of a dataset file based on ID
- Delete physical dataset file and all records from database for a given file ID
- Summarize dataset file
- List all prior knowledge files of a given user
- Get the detail information of a prior knowledge file based on ID
- Delete physical prior knowledge file and all records from database for a given file ID
- 2. Causal Discovery
- List all the available causal discovery algorithms
- List details of given algorithm by ID
- List all data types
- List all independent tests
- List tests based on a given data type
- Liat all scores
- List scores based on a given data type
- List all algorithm parameters based on the provided test and score
- Add a new job to run the desired algorithm on a given data file
- List all running jobs
- Check the job status for a given job ID
- Cancel a running job
- 3. Result Management
After registration in Causal Web App, the email and password can be used to authenticate against the Causal REST API to get the access token (we use JWT) via HTTP Basic Auth.
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/jwt
In basic auth, the user provides the username and password, which the HTTP client concatenates (username + ":" + password), and base64 encodes it. This encoded string is then sent using a Authorization
header with the "Basic" schema. For instance user email [email protected]
whose password is 123
.
POST /ccd-api/jwt HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Basic ZGVtb0BwaXR0LmVkdToxMjM=
Once the request is processed successfully, the user ID together with a JWT will be returned in the response for further API queries.
{
"userId": 22,
"jwt": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA0Mjg1OTcsImlhdCI6MTQ3NTg0NjgyODU5N30.FcE7aEpg0u2c-gUVugIjJkzjhlDu5qav_XHtgLu3c6E",
"issuedTime": 1475846828597,
"lifetime": 3600,
"expireTime": 1475850428597,
"wallTime": [
1,
3,
6
]
}
We'll need to use this userId
in the URI path of all subsequent requests. And this jwt
expires in 3600 seconds(1 hour), so the API consumer will need to request for another JWT otherwise the API query to other API endpoints will be denied. And this JWT will need to be sent via the HTTP Authorization
header as well, but using the Bearer
schema.
The wallTime
field is designed for users who want to specify the the maximum CPU time when Slurm handles the jobs on PSC. Normally, a job is expected to finish before the specified maximum walltime. After the walltime reaches the maximum, the job terminates regardless whether the job processes are still running or not. In this example, you can pick 1 hour, 3 or 6 hours as the wallTime.
Note: querying the JWT endpoint again before the current JWT expires will generate a new JWT, which makes the old JWT expired automatically. And this newly generated JWT will be valid in another hour unless there's another new JWT being queried.
Since this API is developed with Jersey, which supports WADL. So you can view the generated WADL by going to https://ccd4.vm.bridges.psc.edu/ccd-api/application.wadl?detail=true
and see all resource available in the application. Accessing to this endpoint doesn't require authentication.
Basically, all the API usage examples are grouped into three categories:
- Data Management
- Causal Discovery
- Result Management
And all the following examples will be issued by user 22
whose password is 123
.
At this point, you can upload two types of data files: tabular dataset file(either tab delimited or comma delimited) and prior knowledge file.
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/dataset/upload
This is a multipart file upload via an HTML form, and the client is required to use name="file"
to name their file upload field in their form.
Generated HTTP request code example:
POST /ccd-api/22/dataset/upload HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename=""
Content-Type:
----WebKitFormBoundary7MA4YWxkTrZu0gW
If the Authorization header is not provided, the response will look like this:
{
"timestamp": 1465414501443,
"status": 401,
"error": "Unauthorized",
"message": "User credentials are required.",
"path": "/22/dataset/upload"
}
This POST request will upload the dataset file to the target server location and add corresponding records into database. And the response will contain the following pieces:
{
"id": 6,
"name": "Lung-tetrad_hv.txt",
"creationTime": 1466622267000,
"lastModifiedTime": 1466622267000,
"fileSize": 3309465,
"md5checkSum": "b1db7511ee293d297e3055d9a7b46c5e",
"fileSummary": {
"variableType": null,
"fileDelimiter": null,
"numOfRows": null,
"numOfColumns": null
}
}
The prior knowledge file upload uses a similar API endpoint:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/priorknowledge/upload
Due to there's no need to summarize a prior knowledge file, the response of a successful prior knowledge file upload will look like:
{
"id": 6,
"name": "Lung-tetrad_hv.txt",
"creationTime": 1466622267000,
"lastModifiedTime": 1466622267000,
"fileSize": 3309465,
"md5checkSum": "ugdb7511rt293d29ke3055d9a7b46c9k"
}
In addition to the regular file upload described in Example 6, we also provide the option of stable and resumable large file upload. It requires the client side to have a resumable upload implementation. We currently support client integrated with Resumable.js, whihc provides multiple simultaneous, stable and resumable uploads via the HTML5 File API. You can also create your own client as long as al the following parameters are set correctly.
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/chunkupload
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/chunkupload
In this example, the data file is splited into 3 chunks. The upload of each chunk consists of a GET request and a POST request. To handle the state of upload chunks, a number of extra parameters are sent along with all requests:
resumableChunkNumber
: The index of the chunk in the current upload. First chunk is1
(no base-0 counting here).resumableChunkSize
: The general chunk size. Using this value andresumableTotalSize
you can calculate the total number of chunks. Please note that the size of the data received in the HTTP might be lower thanresumableChunkSize
of this for the last chunk for a file.resumableCurrentChunkSize
: The size of the current resumable chuck.resumableTotalSize
: The total file size.resumableType
: The file type of the resumable chuck, e.e., "text/plain".resumableIdentifier
: A unique identifier for the file contained in the request.resumableFilename
: The original file name (since a bug in Firefox results in the file name not being transmitted in chunk multipart posts).resumableRelativePath
: The file's relative path when selecting a directory (defaults to file name in all browsers except Chrome).resumableTotalChunks
: The total number of chunks.
Generated HTTP request code example:
GET /ccd-api/22/chunkupload?resumableChunkNumber=2&resumableChunkSize=1048576&resumableCurrentChunkSize=1048576&resumableTotalSize=3309465&resumableType=text%2Fplain&resumableIdentifier=3309465-large-datatxt&resumableFilename=large-data.txt&resumableRelativePath=large-data.txt&resumableTotalChunks=3 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
This GET request checks if the data chunk is already on the server side. If the target file chunk is not found on the server, the client will issue a POST request to upload the actual data.
Generated HTTP request code example:
POST /ccd-api/22/chunkupload HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryMFjgApg56XGyeTnZ
------WebKitFormBoundaryMFjgApg56XGyeTnZ
Content-Disposition: form-data; name="resumableChunkNumber"
2
------WebKitFormBoundaryMFjgApg56XGyeTnZ
Content-Disposition: form-data; name="resumableChunkSize"
1048576
------WebKitFormBoundaryMFjgApg56XGyeTnZ
Content-Disposition: form-data; name="resumableCurrentChunkSize"
1048576
------WebKitFormBoundaryMFjgApg56XGyeTnZ
Content-Disposition: form-data; name="resumableTotalSize"
3309465
------WebKitFormBoundaryMFjgApg56XGyeTnZ
Content-Disposition: form-data; name="resumableType"
text/plain
------WebKitFormBoundaryMFjgApg56XGyeTnZ
Content-Disposition: form-data; name="resumableIdentifier"
3309465-large-datatxt
------WebKitFormBoundaryMFjgApg56XGyeTnZ
Content-Disposition: form-data; name="resumableFilename"
large-data.txt
------WebKitFormBoundaryMFjgApg56XGyeTnZ
Content-Disposition: form-data; name="resumableRelativePath"
large-data.txt
------WebKitFormBoundaryMFjgApg56XGyeTnZ
Content-Disposition: form-data; name="resumableTotalChunks"
3
------WebKitFormBoundaryMFjgApg56XGyeTnZ
Content-Disposition: form-data; name="file"; filename="blob"
Content-Type: application/octet-stream
------WebKitFormBoundaryMFjgApg56XGyeTnZ--
Each chunk upload POST will get a 200 status code from response if everything works fine.
And finally the md5checkSum string of the reassemabled file will be returned once the whole file has been uploaded successfully. In this example, the POST request that uploads the third chunk will response this:
b1db7511ee293d297e3055d9a7b46c5e
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/dataset
Generated HTTP request code example:
GET /ccd-api/22/dataset HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Accept: application/json
A JSON
formatted list of all the input dataset files that are associated with user 22
will be returned.
[
{
"id": 8,
"name": "data_small.txt",
"creationTime": 1467132449000,
"lastModifiedTime": 1467132449000,
"fileSize": 278428,
"md5checkSum": "ed5f27a2cf94fe3735a5d9ed9191c382",
"fileSummary": {
"variableType": "continuous",
"fileDelimiter": "tab",
"numOfRows": 302,
"numOfColumns": 123
}
},
{
"id": 10,
"name": "large-data.txt",
"creationTime": 1467134048000,
"lastModifiedTime": 1467134048000,
"fileSize": 3309465,
"md5checkSum": "b1db7511ee293d297e3055d9a7b46c5e",
"fileSummary": {
"variableType": null,
"fileDelimiter": null,
"numOfRows": null,
"numOfColumns": null
}
},
{
"id": 11,
"name": "Lung-tetrad_hv (copy).txt",
"creationTime": 1467140415000,
"lastModifiedTime": 1467140415000,
"fileSize": 3309465,
"md5checkSum": "b1db7511ee293d297e3055d9a7b46c5e",
"fileSummary": {
"variableType": "continuous",
"fileDelimiter": "tab",
"numOfRows": 302,
"numOfColumns": 608
}
}
]
You can also specify the response format as XML in your request
Generated HTTP request code example:
GET /ccd-api/22/dataset HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Accept: application/xml
And the response will look like this:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<datasetFileDTOes>
<datasetFile>
<id>8</id>
<name>data_small.txt</name>
<creationTime>2016-06-28T12:47:29-04:00</creationTime>
<lastModifiedTime>2016-06-28T12:47:29-04:00</lastModifiedTime>
<fileSize>278428</fileSize>
<md5checkSum>ed5f27a2cf94fe3735a5d9ed9191c382</md5checkSum>
<fileSummary>
<fileDelimiter>tab</fileDelimiter>
<numOfColumns>123</numOfColumns>
<numOfRows>302</numOfRows>
<variableType>continuous</variableType>
</fileSummary>
</datasetFile>
<datasetFile>
<id>10</id>
<name>large-data.txt</name>
<creationTime>2016-06-28T13:14:08-04:00</creationTime>
<lastModifiedTime>2016-06-28T13:14:08-04:00</lastModifiedTime>
<fileSize>3309465</fileSize>
<md5checkSum>b1db7511ee293d297e3055d9a7b46c5e</md5checkSum>
<fileSummary>
<variableType xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
<fileDelimiter xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
<numOfRows xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
<numOfColumns xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
</fileSummary>
</datasetFile>
<datasetFile>
<id>11</id>
<name>Lung-tetrad_hv (copy).txt</name>
<creationTime>2016-06-28T15:00:15-04:00</creationTime>
<lastModifiedTime>2016-06-28T15:00:15-04:00</lastModifiedTime>
<fileSize>3309465</fileSize>
<md5checkSum>b1db7511ee293d297e3055d9a7b46c5e</md5checkSum>
<fileSummary>
<fileDelimiter>tab</fileDelimiter>
<numOfColumns>608</numOfColumns>
<numOfRows>302</numOfRows>
<variableType>continuous</variableType>
</fileSummary>
</datasetFile>
</datasetFileDTOes>
Form the above output, we can also tell that data file with ID 10 doesn't have all the fileSummary
field values set, we'll cover this in the dataset summarization section.
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/dataset/{id}
Generated HTTP request code example:
GET /ccd-api/22/dataset/8 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
And the resulting response looks like this:
{
"id": 8,
"name": "data_small.txt",
"creationTime": 1467132449000,
"lastModifiedTime": 1467132449000,
"fileSize": 278428,
"fileSummary": {
"md5checkSum": "ed5f27a2cf94fe3735a5d9ed9191c382",
"variableType": "continuous",
"fileDelimiter": "tab",
"numOfRows": 302,
"numOfColumns": 123
}
}
API Endpoint URI pattern:
DELETE https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/dataset/{id}
Generated HTTP request code example:
DELETE /ccd-api/22/dataset/8 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
And this will result a HTTP 204 No Content status in response on success, which means the server successfully processed the deletion request but there's no content to response.
So from the first example we can tell that file with ID 10 doesn't have variableType
, fileDelimiter
, numOfRows
, and numOfColumns
specified under fileSummary
. Among these attributes, variableTypeand
fileDelimiter` are the ones that users will need to provide during this summarization process.
Before we can go ahead to run the desired algorithm with the newly uploaded data file, we'll need to summarize the data by specifing the variable type and file delimiter.
Required Fields | Description |
---|---|
id |
The data file ID |
variableType |
continuous, discrete, mixed |
fileDelimiter |
tab or comma |
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/dataset/summarize
Generated HTTP request code example:
POST /ccd-api/22/dataset/summarize HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Content-Type: application/json
{
"id": 1,
"variableType": "continuous",
"fileDelimiter": "comma"
}
This POST request will summarize the dataset file and generate a response (JSON or XML) like below:
{
"id": 10,
"name": "large-data.txt",
"creationTime": 1467134048000,
"lastModifiedTime": 1467134048000,
"fileSize": 3309465,
"md5checkSum": "b1db7511ee293d297e3055d9a7b46c5e",
"fileSummary": {
"variableType": "continuous",
"fileDelimiter": "tab",
"numOfRows": 302,
"numOfColumns": 608
}
}
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/priorknowledge
Generated HTTP request code example:
GET /ccd-api/22/priorknowledge HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Accept: application/json
A JSON
formatted list of all the input dataset files that are associated with user 22
will be returned.
[
{
"id": 9,
"name": "data_small.prior",
"creationTime": 1467132449000,
"lastModifiedTime": 1467132449000,
"fileSize": 278428,
"md5checkSum": "ed5f27a2cf94fe3735a5d9ed9191c382"
},
{
"id": 12,
"name": "large-data.prior",
"creationTime": 1467134048000,
"lastModifiedTime": 1467134048000,
"fileSize": 3309465,
"md5checkSum": "b1db7511ee293d297e3055d9a7b46c5e"
}
]
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/priorknowledge/{id}
Generated HTTP request code example:
GET /ccd-api/22/priorknowledge/9 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
And the resulting response looks like this:
{
"id": 9,
"name": "data_small.prior",
"creationTime": 1467132449000,
"lastModifiedTime": 1467132449000,
"fileSize": 278428,
"md5checkSum": "ed5f27a2cf94fe3735a5d9ed9191c382"
}
API Endpoint URI pattern:
DELETE https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/priorknowledge/{id}
Generated HTTP request code example:
DELETE /ccd-api/22/priorknowledge/9 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
And this will result a HTTP 204 No Content status in response on success, which means the server successfully processed the deletion request but there's no content to response.
Once the data file is uploaded and summaried, you can start running a Causal Discovery Algorithm on the uploaded data file.
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/algorithms
Generated HTTP request code example:
GET /ccd-api/22/algorithms HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
[
{
"id": "bpc",
"name": "BPC",
"description": "",
"requireTest": false,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "fas",
"name": "FAS",
"description": "",
"requireTest": true,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "fask",
"name": "FASK",
"description": "",
"requireTest": false,
"requireScore": true,
"acceptKnowledge": true
},
{
"id": "fask-concatenated",
"name": "FASK Concatenated",
"description": "",
"requireTest": false,
"requireScore": true,
"acceptKnowledge": true
},
{
"id": "fci",
"name": "FCI",
"description": "",
"requireTest": true,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "fges",
"name": "FGES",
"description": "",
"requireTest": false,
"requireScore": true,
"acceptKnowledge": true
},
{
"id": "fges-mb",
"name": "FGES-MB",
"description": "",
"requireTest": false,
"requireScore": true,
"acceptKnowledge": true
},
{
"id": "fofc",
"name": "FOFC",
"description": "",
"requireTest": false,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "ftfc",
"name": "FTFC",
"description": "",
"requireTest": false,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "gfci",
"name": "GFCI",
"description": "",
"requireTest": true,
"requireScore": true,
"acceptKnowledge": true
},
{
"id": "glasso",
"name": "GLASSO",
"description": "",
"requireTest": false,
"requireScore": false,
"acceptKnowledge": false
},
{
"id": "imgs_cont",
"name": "IMaGES Continuous",
"description": "",
"requireTest": false,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "imgs_disc",
"name": "IMaGES Discrete",
"description": "",
"requireTest": false,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "mbfs",
"name": "MBFS",
"description": "",
"requireTest": true,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "mgm",
"name": "MGM",
"description": "",
"requireTest": false,
"requireScore": false,
"acceptKnowledge": false
},
{
"id": "mimbuild",
"name": "MIMBuild",
"description": "",
"requireTest": false,
"requireScore": false,
"acceptKnowledge": false
},
{
"id": "pc-all",
"name": "PC All",
"description": "",
"requireTest": true,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "rfci",
"name": "RFCI",
"description": "",
"requireTest": true,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "ts-fci",
"name": "TsFCI",
"description": "",
"requireTest": true,
"requireScore": false,
"acceptKnowledge": true
},
{
"id": "ts-gfci",
"name": "TsGFCI",
"description": "",
"requireTest": true,
"requireScore": true,
"acceptKnowledge": true
},
{
"id": "ts-imgs",
"name": "TsIMaGES",
"description": "",
"requireTest": false,
"requireScore": true,
"acceptKnowledge": true
}
]
You can also just query the details of a specific algorithm:
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/algorithms/{algoId}
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/datatypes
Generated HTTP request code example:
[
"continuous",
"discrete",
"mixed"
]
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/tests
Generated HTTP request code example:
[
{
"id": "bdeu",
"name": "BDeu Test",
"supportedDataTypes": [
"discrete"
]
},
{
"id": "chi-square",
"name": "Chi Square Test",
"supportedDataTypes": [
"discrete"
]
},
{
"id": "cond-correlation",
"name": "Conditional Correlation Test",
"supportedDataTypes": [
"continuous"
]
},
{
"id": "cond-gauss-lrt",
"name": "Conditional Gaussian Likelihood Ratio Test",
"supportedDataTypes": [
"mixed"
]
},
{
"id": "correlation-t",
"name": "Correlation T Test",
"supportedDataTypes": [
"continuous"
]
},
{
"id": "disc-bic",
"name": "Discrete BIC Test",
"supportedDataTypes": [
"discrete"
]
},
{
"id": "fisher-z",
"name": "Fisher Z Test",
"supportedDataTypes": [
"continuous"
]
},
{
"id": "g-square",
"name": "G Square Test",
"supportedDataTypes": [
"discrete"
]
},
{
"id": "mixed-var-polynominal-likelihood-ratio",
"name": "Mixed Variable Polynomial Likelihood Ratio Test",
"supportedDataTypes": [
"mixed"
]
},
{
"id": "multinomial-logistic-regression-likelihood-ratio",
"name": "Multinomial Logistic Regression Likelihood Ratio Test",
"supportedDataTypes": [
"mixed"
]
},
{
"id": "multinomial-logistic-regression-wald",
"name": "Multinomial Logistic Retression Wald Test",
"supportedDataTypes": [
"mixed"
]
},
{
"id": "sem-bic",
"name": "SEM BIC Test",
"supportedDataTypes": [
"continuous"
]
}
]
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/tests/continuous
Generated HTTP request code example:
[
{
"id": "cond-correlation",
"name": "Conditional Correlation Test",
"supportedDataTypes": [
"continuous"
]
},
{
"id": "correlation-t",
"name": "Correlation T Test",
"supportedDataTypes": [
"continuous"
]
},
{
"id": "fisher-z",
"name": "Fisher Z Test",
"supportedDataTypes": [
"continuous"
]
},
{
"id": "sem-bic",
"name": "SEM BIC Test",
"supportedDataTypes": [
"continuous"
]
}
]
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/scores
Generated HTTP request code example:
[
{
"id": "bdeu",
"name": "BDeu Score",
"supportedDataTypes": [
"discrete"
]
},
{
"id": "cond-gauss-bic",
"name": "Conditional Gaussian BIC Score",
"supportedDataTypes": [
"mixed"
]
},
{
"id": "conditional-gaussian-other-bic",
"name": "Conditional Gaussian Other BIC Score",
"supportedDataTypes": [
"mixed"
]
},
{
"id": "disc-bic",
"name": "Discrete BIC Score",
"supportedDataTypes": [
"discrete"
]
},
{
"id": "disc-mixed-bic",
"name": "Discrete Mixed BIC Score",
"supportedDataTypes": [
"mixed"
]
},
{
"id": "fisher-z",
"name": "Fisher Z Score",
"supportedDataTypes": [
"continuous"
]
},
{
"id": "mixed-var-polynominal-bic",
"name": "Mixed Variable Polynomial BIC Score",
"supportedDataTypes": [
"mixed"
]
},
{
"id": "multinomial-logistic-regression-bic",
"name": "Multinomial Logistic Regression BIC Score",
"supportedDataTypes": [
"mixed"
]
},
{
"id": "sem-bic",
"name": "Sem BIC Score",
"supportedDataTypes": [
"continuous"
]
},
{
"id": "sem-bic-deterministic",
"name": "Sem BIC Score Deterministic",
"supportedDataTypes": [
"continuous"
]
}
]
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/scores/discrete
Generated HTTP request code example:
[
{
"id": "bdeu",
"name": "BDeu Score",
"supportedDataTypes": [
"discrete"
]
},
{
"id": "disc-bic",
"name": "Discrete BIC Score",
"supportedDataTypes": [
"discrete"
]
}
]
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/algorithmParameters
Field | Description |
---|---|
algoId |
The target algorithm ID string, required |
testId |
The ID of indenpendence test, not all algorithms require a test |
scoreId |
The ID of score method, not all algorithms require a score |
Generated HTTP request code example:
POST /ccd-api/1/algorithmParameters/ HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwOi8vbG9jYWxob3N0LyIsInVpZCI6MSwiZXhwIjoxNTE1NjIwNjQ4NTA2LCJpYXQiOjE1MTU2MTcwNDg1MDZ9.OvS1DNCRtgqNuOw0EX3TAxZxb998gvL84ZIRb3CykY0
Content-Type: application/json
{
"algoId": "fges",
"scoreId": "fisher-z"
}
Because fges
requires a score but donesn't require a test, the above request body doesn't have testId
specified. And if you accidently add the that, the API will response with an error message.
[
{
"name": "alpha",
"description": "Cutoff for p values (alpha) (min = 0.0)",
"valueType": "Double",
"defaultValue": 0.01
},
{
"name": "faithfulnessAssumed",
"description": "Yes if (one edge) faithfulness should be assumed",
"valueType": "Boolean",
"defaultValue": true
},
{
"name": "symmetricFirstStep",
"description": "Yes if the first step step for FGES should do scoring for both X->Y and Y->X",
"valueType": "Boolean",
"defaultValue": false
},
{
"name": "maxDegree",
"description": "The maximum degree of the graph (min = -1)",
"valueType": "Integer",
"defaultValue": 100
},
{
"name": "verbose",
"description": "Yes if verbose output should be printed or logged",
"valueType": "Boolean",
"defaultValue": false
},
{
"name": "bootstrapSampleSize",
"description": "The number of bootstraps (min = 0)",
"valueType": "Integer",
"defaultValue": 0
},
{
"name": "bootstrapEnsemble",
"description": "Ensemble method: Preserved (0), Highest (1), Majority (2)",
"valueType": "Integer",
"defaultValue": 1
}
]
This is a POST request and the algorithm details and data file id will need to be specified in the POST body as a JSON when you make the request.
They also share a common JSON structure as of their input, for example:
Input JSON Fields | Description |
---|---|
algoId |
The target algorithm ID string, required |
testId |
The ID of indenpendence test, not all algorithms require a test |
scoreId |
The ID of score method, not all algorithms require a score |
datasetFileId |
The dataset file ID, integer |
priorKnowledgeFileId |
The optional prior knowledge file ID, integer |
algoParameters |
Algorithm specific parameters, JSON object |
skipDataValidation |
If to skip the data validation, boolean, default false. Optional |
jvmOptions |
Optional advanced Options For Java Virtual Machine (JVM), JSON object. Currently only support maxHeapSize (Gigabyte, max value is 100) |
hpcParameters |
Optional arameters for High-Performance Computing, JSON array of key-value objects. Currently only support wallTime |
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/newjob
Generated HTTP request code example:
POST /ccd-api/1/newjob HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwOi8vbG9jYWxob3N0LyIsInVpZCI6MSwiZXhwIjoxNTE1NjIwNjQ4NTA2LCJpYXQiOjE1MTU2MTcwNDg1MDZ9.OvS1DNCRtgqNuOw0EX3TAxZxb998gvL84ZIRb3CykY0
Content-Type: application/json
{
"algoId": "fges",
"scoreId": "fisher-z",
"datasetFileId": 1,
"algoParameters": [
{
"key": "alpha",
"value": 0.4
},
{
"key": "faithfulnessAssumed",
"value": false
},
{
"key": "maxDegree",
"value": 10
}
],
"jvmOptions": {
"maxHeapSize": 100
}
}
In this example, we are running the "FGES" algorithm with "BDeu Test" and "BDeu Score" on the file of ID 8. We also set the wallTime as 1 hour. And this call will return the job info with a 201 Created response status code.
{
"id": 25,
"algoId": "fges",
"status": 0,
"addedTime": 1515617400509,
"resultFileName": "fges_Retention.txt_1515617400499.txt",
"resultJsonFileName": "fges_Retention.txt_1515617400499.json",
"errorResultFileName": "error_fges_Retention.txt_1515617400499.txt"
}
From this response we can tell that the job ID is 25, and the result file name will be fges_Retention.txt_1515617400499.txt
if everything goes well. If something is wrong an error result file with name error_fges_Retention.txt_1515617400499.txt
will be created.
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/jobs
Generated HTTP request code example:
GET /ccd-api/22/jobs/ HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Content-Type: application/json
Then you'll see the information of all jobs that are currently running:
[
{
"id": 32,
"algoId": "fges",
"addedTime": 1468436085000
},
{
"id": 33,
"algoId": "gfci",
"addedTime": 1468436087000
}
]
Once the new job is submitted, it takes time and resources to run the algorithm on the server. During the waiting, you can check the status of a given job ID:
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/jobs/{id}
Generated HTTP request code example:
GET /ccd-api/22/jobs/32 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
This will either return "Pending" or "Completed".
Sometimes you may want to cancel a submitted job.
API Endpoint URI pattern:
DELETE https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/jobs/{id}
Generated HTTP request code example:
DELETE /ccd-api/22/jobs/8 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
This call will response either "Job 8 has been canceled" or "Unable to cancel job 8". It's not guranteed that the system can always cencal a job successfully.
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/results
Generated HTTP request code example:
GET /ccd-api/22/results HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
The response to this request will look like this:
[
{
"name": "fges_sim_data_20vars_100cases.csv_1466171729046.txt",
"creationTime": 1466171732000,
"lastModifiedTime": 1466171732000,
"fileSize": 1660
},
{
"name": "fges_data_small.txt_1466172140585.txt",
"creationTime": 1466172145000,
"lastModifiedTime": 1466172145000,
"fileSize": 39559
}
]
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/results/{result_file_name}
Generated HTTP request code example:
GET /ccd-api/22/results/fges_data_small.txt_1466172140585.txt HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
On success, you will get the result file back as text file content. If there's a typo in file name of the that file doesn't exist, you'll get either a JSON or XML message based on the accept
header in your request:
The response to this request will look like this:
{
"timestamp": 1467210996233,
"status": 404,
"error": "Not Found",
"message": "Resource not found.",
"path": "/22/results/fges_data_small.txt_146172140585.txt"
}
Since we can list all the algorithm result files, based on the results, we can also choose multiple files and run a comparison.
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/results/compare
The request body is a JSON that contains an array of result files to be compared.
Generated HTTP request code example:
POST /ccd-api/22/results/compare HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
{
"resultFiles": [
"FGESc_sim_data_20vars_100cases.csv_1466171729046.txt",
"FGESc_data_small.txt_1467305104859.txt"
]
}
When you specify multiple file names, use the !!
as a delimiter. This request will generate a result comparison file with the following content (shortened version):
fges_sim_data_20vars_100cases.csv_1466171729046.txt fges_data_small.txt_1467305104859.txt
Edges In All Same End Point
NR4A2,FOS 0 0
X5,X17 0 0
MMP11,ASB5 0 0
X12,X8 0 0
hsa_miR_654_3p,hsa_miR_337_3p 0 0
RND1,FGA 0 0
HHLA2,UBXN10 0 0
HS6ST2,RND1 0 0
SCRG1,hsa_miR_377 0 0
CDH3,diag 0 0
SERPINI2,FGG 0 0
hsa_miR_451,hsa_miR_136_ 0 0
From this comparison, you can see if the two algorithm graphs have common edges and endpoints.
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/results/comparisons
Generated HTTP request code example:
GET /ccd-api/22/results/comparisons HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
The response will show a list of comparison files:
[
{
"name": "result_comparison_1467385923407.txt",
"creationTime": 1467385923000,
"lastModifiedTime": 1467385923000,
"fileSize": 7505
},
{
"name": "result_comparison_1467387034358.txt",
"creationTime": 1467387034000,
"lastModifiedTime": 1467387034000,
"fileSize": 7505
},
{
"name": "result_comparison_1467388042261.txt",
"creationTime": 1467388042000,
"lastModifiedTime": 1467388042000,
"fileSize": 7533
}
]
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/results/comparisons/{comparison_file_name}
Generated HTTP request code example:
GET /ccd-api/22/results/comparisons/result_comparison_1467388042261.txt HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Then it returns the content of that comparison file (shortened version):
fges_sim_data_20vars_100cases.csv_1466171729046.txt fges_data_small.txt_1467305104859.txt
Edges In All Same End Point
NR4A2,FOS 0 0
X5,X17 0 0
MMP11,ASB5 0 0
X12,X8 0 0
hsa_miR_654_3p,hsa_miR_337_3p 0 0
RND1,FGA 0 0
HHLA2,UBXN10 0 0
HS6ST2,RND1 0 0
SCRG1,hsa_miR_377 0 0
CDH3,diag 0 0
SERPINI2,FGG 0 0