Write APIs

Government Admin users can use Write APIs to automatically and programmatically publish data to data.gov.sg without needing to log into the UI

---

Prerequisites

• You must be a government officer

• Your email has admin permissions to manage datasets

Step 1 - Generate API Keys:

• Create an API key from the admin dashboard.

• Store the API key in a secure location. It will be used later.

• Step by step details can be found here: How to generate API keys

Step 2 - Verify API Connectivity:

Who Am I

GEThttps://api-production.data.gov.sg/v2/admin/api/auth/whoami

Authorization

apikeyAuth

Header parameters

Response

200

Successful response

Body

application/json

object

Request

JavaScriptCurlPython

const response = await fetch('https://api-production.data.gov.sg/v2/admin/api/auth/whoami', {
    method: 'GET',
    headers: {
      "Authorization": "apikey <token>"
    },
});
const data = await response.json();

Run a get request against the above endpoint, using your admin api key in the header:

https://api-production.data.gov.sg/v2/admin/api/auth/whoami

A 200 response means that you're connected.

Common issues:

1. Check your API URL - did you use the correct URL above?

2. Check your API key - did you use the correct API key? Try regenerating the API key and testing again

3. Are the network configurations set up to connect to the internet? data.gov.sg is an internet platform and require internet connectivity

Step 3 - Publish Dataset:

• To push data, an existing dataset must exist. The dataset can be created from the admin dashboard.

• The dataset must already contain existing data.

• The dataset must be published.

• For more details on publishing data: How to publish data

Using Write APIs involves three steps:

• Generating upload link

• Uploading file to link

• Polling for upload status

Step 4 - Create Upload / Append URL:

If you are creating a new dataset or replacing an existing dataset completely, then generate an upload link:

Get Create Upload URL

GEThttps://api-production.data.gov.sg/v2/admin/api/datasets/{datasetId}/upload-link

Path parameters

datasetId*string

Example: "d_263ee33e4d309a0bba75a1fd30ab7035"

Header parameters

Response

201

Created

Headers

Body

application/json

object

Request

JavaScriptCurlPython

const response = await fetch('https://api-production.data.gov.sg/v2/admin/api/datasets/{datasetId}/upload-link', {
    method: 'GET',
    headers: {},
});
const data = await response.json();

If you are appending rows to an existing dataset, then generate an append link. Please ensure the column data types and names are all the same and no new columns have been added. This append API only supports appending rows to an existing dataset and not appending columns:

Get Create Append URL

GEThttps://api-production.data.gov.sg/v2/admin/api/datasets/{datasetId}/append-link

Path parameters

datasetId*string

Example: "d_263ee33e4d309a0bba75a1fd30ab7035"

Query parameters

Header parameters

Response

201

Created

Headers

Body

application/json

object

Request

JavaScriptCurlPython

const response = await fetch('https://api-production.data.gov.sg/v2/admin/api/datasets/{datasetId}/append-link', {
    method: 'GET',
    headers: {},
});
const data = await response.json();

Find the datasetId within the dataset url For example, for this url: https://beta.data.gov.sg/datasets/d_07c63be0f37e6e59c07a4ddc2fd87fcb/view The datasetId is the following: d_07c63be0f37e6e59c07a4ddc2fd87fcb Hence, the endpoint used would be: https://api-production.data.gov.sg/v2/admin/api/datasets/d_07c63be0f37e6e59c07a4ddc2fd87fcb/upload-link

Step 5 - Upload Tabular File:

Upload Tabular File

Note that this uses the URL provided the response of "Upload Dataset Data Step 1"

PUThttps://api-production.data.gov.sg/attachments.data.gov.sg/c/6047/private/d_2a6070fa301695904e1a626434189e59/VW50aXRsZWQgRGF0YXNldA/d_2a6070fa301695904e1a626434189e59.csv

Query parameters

Body

*/*

string

Example: "\"name,title,email\\njanedoe,SWE,[e-mail]\""

Response

200

OK

Headers

Body

text/plain

string

Request

JavaScriptCurlPython

const response = await fetch('https://api-production.data.gov.sg/attachments.data.gov.sg/c/6047/private/d_2a6070fa301695904e1a626434189e59/VW50aXRsZWQgRGF0YXNldA/d_2a6070fa301695904e1a626434189e59.csv', {
    method: 'PUT',
    headers: {
      "Content-Type": "*/*"
    },
    body: JSON.stringify("\"name,title,email\\njanedoe,SWE,[e-mail]\""),
});
const data = await response.json();

Response

200

text

It’s important to ensure that you are sending a PUT request with only the file data. While POST-ing form-data is allowed, the file received will not be the same and may cause errors.

Common issues:

1. Check that you're using a PUT request

2. Check that you're using the generated upload url instead of the one provided above

Step 6 - Get Latest Ingestion Status:

Get Latest Ingestion Status

GEThttps://api-production.data.gov.sg/v2/admin/api/datasets/{datasetId}/ingestion-status

Path parameters

datasetId*string

Example: "d_263ee33e4d309a0bba75a1fd30ab7035"

Header parameters

Response

200

Legend

Order	Ingestion status	Description
1	Pending	The dataset validation and ingestion process has been started.
2	Pending Validation	The dataset has been queued for validation.
2.1	Validation Failed	The dataset contains invalid values causing validation failure. Please review the reported errors, make the necessary changes and try again.
2.2	Validation Passed	The dataset has been validated successfully, but not yet ingested.
3	Pending Ingestion	The dataset has been queued for ingestion.
3.1	Ingestion Failed	The dataset failed to be ingested. Please try again later.
3.2	Ingestion Success	The dataset has been ingested successfully.

Body

application/json

object

Request

JavaScriptCurlPython

const response = await fetch('https://api-production.data.gov.sg/v2/admin/api/datasets/{datasetId}/ingestion-status', {
    method: 'GET',
    headers: {},
});
const data = await response.json();

The validation and ingestion process of DataGovSG is asynchronous and long running. The upload request from the previous section will return before the process is complete. In order to get the status for validation and ingestion, manual polling is required.

Here are the possible validation and ingestion statuses:

Order	Status	Description
1	Pending	The validation and ingestion process has been started
2	Pending Validation	The dataset has been queued for validation
2.1	Validation Failed	The dataset contains invalid values causing validation failures. Please review the reported errors, make the necessary changes and try again.
2.2	Validation Passed	The dataset has been validated successfully, but not yet ingested.
3	Pending Ingestion	The dataset has been queued for ingestion.
3.1	Ingestion Failed	The dataset failed to be ingested. Please try again later.
3.2	Ingestion Success	The dataset has been ingested successfully.

Raw API Schema:

Updated as of 20th June 2024

16KB

v2-write-api-specification.yml

For further queries feel free to reach out to us: Contact the Data.gov.sg team