DRAFT Data Process Service

Overview

The Data Process Service is a web service hosted by Fusion Registry which accepts a dataset (multiple formats supported) and a process instruction, which tells the Fusion Registry how to process the data, for example validate, map, transform, export.

Workflow

The Data Process Service has the following workflow

1. Load Data - Data is submitted using HTTP Post. The server provides a token to be able to perform further actions on the data
2. Run Process - The process to execute is sent to the server, along with the token of which dataset to apply the process to. The server provides a token to track the process
3. Track Progress - A GET request to track the progress of a running process, using the processes' unique token as supplied by the server
4. Export Data - If the final stage of a Process is to store the data in a temporary store, then it can be exported in any data format supported by Fusion Registry

Any data loaded is stored for 2 minutes, if there is no activity on the data 2 minutes after it is loaded, it will be evicted from the system.

Load Data

Entry Point	/ws/secure/dataprocess/load
Access	Restricted
Http Method	POST
Accepts	Any Supported Data Format
Compression	Zip files supported
Content-Type	1. multipart/form-data (if attaching file) – the attached file must be in field name of uploadFile 2. application/text or application/xml (if submitting data in the body of the POST)
Response Format	JSON
Response Statuses	200 - Data file received 400 - If not dataset provided 401 - Unauthorized (if access has been restricted) 500 - Server Error

Response

 {
   "token" : "uid123"
 }

Run Process

Entry Point	/ws/secure/dataprocess/run/{token}
Access	Restricted
Http Method	POST
Accepts	Process Instruction (JSON format)
Content-Type	1. multipart/form-data (if attaching file) – the attached file must be in field name of uploadFile 2. application/json (if submitting data in the body of the POST)
Response Format	JSON
Response Statuses	200 - Data file received 400 - If the token does not match a known dataset 401 - Unauthorized (if access has been restricted) 500 - Server Error

Response

 {
   "token" : "uid123"
 }

Track Process

Entry Point	/ws/secure/dataprocess/status/{token}
Access	Restricted
Http Method	GET
Response Format	JSON
Response Statuses	200 - Data file received 404 - If the token does not match a known process 401 - Unauthorized (if access has been restricted) 500 - Server Error

Response Example

 {
   "ProcessToken" : "abcd",
   "StartTime"    : 123456,   //unix time milliseconds since 1970
   "EndTime"      : null,     //unix time milliseconds since 1970
   "Status"       : 1         //0=running,1=success,2=error
   "Steps"        :
     {
        "Step1" : 
          {
            "StartTime"   : 123456,  //unix time milliseconds since 1970
            "EndTime"     : null,    //unix time milliseconds since 1970
            "Series" 	   : 12,
            "Rows"	   : 132
          }
     }
 }