Lumminary API 1.0
Added: 31/12/2018
Updated at: 21/06/2021
# Introduction
The Lumminary API was built to allow third parties to interact with Lumminary customers and gain access to their genetic data. The Lumminary API is fast, scalable and highly secure. All requests to the Lumminary API take place over SSL, which means all communication of Customer data is encrypted.
Before we dive in, some definitions. This is what we mean by:
|Term|Definition|
|-----------|-----------|
|**Third party**|A third party (also referred to as "partner" or as "you") is a company which offers services and products using genetic data.|
|**Lumminary clients**|The Lumminary client (also referred to as "customer") is an individual who has created an account on the Lumminary platform.|
|**Lumminary**|This is us - our services including the Lumminary platform, the API, the DNA App Store, the DNA Vault, the "Connect with Lumminary" button, and the website in its totality. |
|**CWL**|This is the acronym for the "Connect with Lumminary" button.|
|**dataset**|This is the term we use when we refer to a customer's genetic data.|
|**Lumminary API**|This is a library/module that you can use to integrate your apps with the Lumminary platform.|
|**Lumminary toolkit**|This is a stand alone application which helps you integrate with Lumminary without writing any code or interacting with the Lumminary API.|
Let's dive in, now.
* [**Overview**](#section/Introduction/Overview)
* [**Install Lumminary API Client and Toolkit**](#Install-Lumminary-API-Client-andor-Toolkit)
* [**Obtaining credentials**](#Obtaining-credentials)
* [**Query customers authorizations**](#Query-customers-authorizations)
* [**Query customer genetic data**](#Query-customer-genetic-data)
* [**Submit reports**](#Submit-reports)
* [**"Connect with Lumminary" button**](#the-connect-with-lumminary-button)
* [**API specs**](#tag/Lumminary)
## Overview
In order to use Lumminary services, you'll need to install the Lumminary API Client or Toolkit. The Lumminary API Client and Toolkit are available in multiple programming languages, and we also provide a sandbox environment which you can use for integration and tests.
There are a couple of differences between the API Client and the Toolkit. Mainly, it's about the ease of use for integration. The Toolkit is basically a stand-alone application that facilitates the integration with the Lumminary API without the need to modify your already existing code.
You use the Lumminary API Client when you want to integrate it inside your own application. This means it gives you full flexibility regarding the integration into your own workflow.
You use the Lumminary Toolkit for an integration where the Toolkit is placed alongside your own application. You can use the Toolkit from the CLI - for example, to run a cronjob that processes incoming orders. The Toolkit uses the Lumminary API Client.
# Install Lumminary API Client and/or Toolkit
We provide the Lumminary API Client and Toolkit in multiple programming languages - default are PHP (minimum version 7.0), Python2.7 and Python3. However, if you need them in another language (Java, Obj-C, JavaScript, C#, Perl, CURL), please contact us.
## How to install the Lumminary API Client
#### PHP example:
The PHP Lumminary API Client is available at: https://github.com/Lumminary/lumminary-api-client-php
If you are already using [Composer](https://getcomposer.org), you can import the project by adding the following to your `composer.json`
```json
"repositories": [
{
"type": "git",
"url": "https://github.com/Lumminary/lumminary-api-client-php.git",
"reference": "master"
}
],
"require": {
"lumminary/api-client-php": "v1.0.6"
}
```
Then run `composer update lumminary/api-client-php`
#### Python example:
The Python Lumminary API Client is available at: https://github.com/Lumminary/lumminary-api-client-python
To install directly, run
```bash
pip install git+https://git@github.com/Lumminary/lumminary-api-client-python.git@v1.0.7#egg=lumminary-api-client
```
Or add the following line in your requirements.txt
```bash
git+https://git@github.com/Lumminary/lumminary-api-client-python.git@v1.0.7#egg=lumminary-api-client
```
## How to install the Lumminary Toolkit
#### PHP example:
The PHP Lumminary Toolkit is available at: https://github.com/Lumminary/lumminary-toolkit-php
To install the Lumminary Toolkit, run the following command where 'lumminary-toolkit-directory' is the directory where you want to install the Lumminary Toolkit:
`git clone git@github.com:Lumminary/lumminary-toolkit-php.git lumminary-toolkit-directory`
#### PYTHON example:
The Python Lumminary Toolkit is available at: https://github.com/Lumminary/lumminary-toolkit-python
To install the Lumminary Toolkit, run the following command where 'lumminary-toolkit-directory' is the directory where you want to install the Lumminary Toolkit:
```bash
git clone git@github.com:Lumminary/lumminary-toolkit-python.git lumminary-toolkit-directory
cd lumminary-toolkit-directory
virtualenv env
source env/bin/activate
pip install -r requirements.txt
```
Note that before running the toolkit, you should have the virtualenv enabled, like so : `source lumminary-toolkit-directory/env/bin/activate`
## Toolkit Setup
We recommend to run the Toolkit in a cronjob; at every run it will check for new Authorizations (Orders) and will download them; afterwards it will check for a new reports folder inside the old authorizations, process the reports, and delete the processed Authorizations and Reports from your server.
The first step after you clone the Lumminary Toolkit project for your language is to configure it with your credentials.
Go to the Lumminary Toolkit base diretory `cd lumminary-toolkit-directory`. Under the Toolkit directory, you will find a file `config/config_template.json` which has the following structure:
```json
{
"api_key":
You can obtain it from the Lumminary Admin |
| product_uuid | `"1234-1234-1234-1234"` |Your product UUID. This value can be obtained from the Lumminary Admin |
| api_host | `"https:\/\/api.lumminary.com\/v1"` | The API endpoint to use.
For the sandbox environment you can use "https:\/\/sandbox-api.lumminary.com\/v1" |
| output_root | `"/var/lumminary-orders/product1/"` | The base directory where the Toolkit places the Authorizations for your Product
The Lumminary Toolkit will *never* overwrite Authorization data or create this directory, to protect from overwrites or typos |
| export_handler |`"export_handler_tsv"` | If you have a custom export handler, then your Lumminary contact will provide you with the name of your export handler. |
| operations |`["pull_datasets", "push_reports"]` | These are two optional parameters that define the tasks that the Toolkit should perform. Possible values are:
1. `pull_datasets` - this tells the Toolkit to download the Customer Authorization (Customer details and genetic data)
2. `push_reports` - this tells the Toolkit to push the results to the API; see below for more details|
| optional | `{}` | Export handler specific value |
After updating the config file for your Toolkit, it should look similar to (Note that these are all dummy values) :
```json
{
"api_key": "TiiU...bqg==",
"product_uuid": "1234-1234-1234-1234",
"api_host": "https:\/\/api.lumminary.com\/v1",
"output_root": "\/var\/lumminary-orders\/product1\/",
"export_handler": "export_handler_tsv",
"product_name": "product 1",
"operations": [
"pull_datasets",
"push_reports"
],
"optional": {
"dna_data_filename": "dna-data.tsv",
"authorization_metadata_filename": "authorization-metadata.json"
}
}
```
You can now save and exit the text editor `:wq` and start polling the API for new Authorizations :
Python
```bash
# While still in the
`{ "credentials": { "username": "username@example.com", "password": "your generated password", "url": "https://your-website.com/report"}}`
The `url` should point to a login page that upon authentication redirects the user to the report page. You can find the customer's email address in the `authorization-metadata.json` and the `password` attribute must be a secure password. Please refer to the Note underneath this table. |
|The product is a physical product| Create a file with the name `result.json` into the `tmp_reports` directory, with the following content:
`{"physical_product": { "physical_product_completed": true }}`
This should be done upon dispatch. Please refer to the Note underneath this table. |
|An error occurred| Create a file with the name `result.json` into the `tmp_reports` directory, with the following content:
`{ "unfulfillable": {"error": "Reasons for why it is unfulfillable", }}`
The error message is for Lumminary internal usage, and it won't be visible to the customer. This will delete your Authorization, making it unuseable thereafter. So please use this only for unfixable errors, and never for temporary errors you attempt to resolve. Please refer to the Note underneath this table. |
###### Note
For each scenario above, we recommend you use a temporary directory to avoid uploading incomplete files or reports. So your workflow should be:
* create a temporary directory inside the `
This impacts the reference allele, location, and based on the dbSNP build, also the SNP's accession |
| `genotypedAlleles` | `genotyped_alleles` | The genotype value of the customer's queried SNP.
If the attribute of this SNP has the `phased` flag set to True,
the first items in the lists for all SNPs will be on the same inherited chromosome,
and analogous for the second item.
If the SNP is unphased, the order of the items is irrelevant |
|`phased` | `phased` | A boolean. True if the SNP is known to be phased, False otherwise |
|`chromosomeAccession` | `chromosome_accession` | This is the chromosome accession number where the SNP is located; in the format of NC_00x |
|`location` | `location` | This is the customer's SNP's location on the chromosome |
When trying to access any customer's SNP for which you don't have permission, an `Unauthorized` exception will be raised.
## Get all authorized SNPs
The function below returns all SNPs your product has access to. These are all the SNPs configured as mandatory for your product, as well as all SNPs that are configured as optional and available in the customer's data set. We encourage you to use this option if you need to get all available SNPs, because it is faster than fetching SNP details one by one.
For example, fetching all authorized SNPs:
#### PHP example:
```php
$datasetSnps = null;
// check to see if you have access to the customer genetic data
if (isset($productAuthorization["scopes"]["dataset"]))
{
// get all authorized SNPs
$datasetSnps = $apiClient->getClientSnpGroup(
$productAuthorization["clientUuid"],
$productAuthorization["scopes"]["dataset"]
);
}
```
#### Python example:
```python
datasetSnps = None
# check to see if you have access to the customer genetic data
if hasattr(productAuthorization.scopes, "dataset"):
# get all authorized SNPs
datasetSnps = apiClient.get_client_snp_group(
productAuthorization.client_uuid,
productAuthorization.scopes.dataset
)
```
##### The datasetSnps variable will be a list of objects each having the following attributes:
| Attribute name PHP | Attribute name Python | Description |
|:-------------------------:|:-------------------------:|:----------------------------------------------------------|
| `snpId` | `snp_id` | The rsid of the SNP |
| `referenceGenome` |`reference_genome` | The reference genome known to be used by the Dataset's source.
This impacts the reference allele, location, and based on the dbSNP build, also the SNP's accession |
| `genotypedAlleles` | `genotyped_alleles` | The genotype value of the customer's queried SNP.
If the attribute of this SNP has the `phased` flag set to True,
the first items in the lists for all SNPs will be on the same inherited chromosome,
and analogous for the second item.
If the SNP is unphased, the order of the items is irrelevant |
|`phased` | `phased` | A boolean. True if the SNP is known to be phased, False otherwise |
|`chromosomeAccession` | `chromosome_accession` | This is the chromosome accession number where the SNP is located; in the format of NC_00x |
|`location` | `location` | This is the customer's SNP's location on the chromosome |
When trying to access any customer's SNP for which you don't have permission, an `Unauthorized` exception will be raised.
## Get Genes
Along with individual SNPs, you can also get all the authorized SNPs from a particular gene, that are available in the customer's dataset.
Here, by genes, we refer strictly to the genomic region that produces some protein, without considering regulatory or noncoding regions that influence gene expression.
The gene name (known as symbol) can be from either of these two databases - NCBI and Ensembl.
For example, fetching details for a gene symbol:
#### PHP example
```php
$geneSymbol = "C1ORF159";
$geneDetails = null;
// check to see if you have access to the customer genetic data
if (isset($productAuthorization["scopes"]["dataset"]))
{
// get all authorized SNPs within a gene
$geneDetails = $apiClient->getClientGene(
$productAuthorization["clientUuid"],
$productAuthorization["scopes"]["dataset"],
$geneSymbol
);
}
```
#### Python example
```python
geneSymbol = "C1ORF159"
geneDetails = None
# check to see if you have access to the customer genetic data
if hasattr(productAuthorization.scopes, "dataset"):
# get all authorized SNPs within a gene
geneDetails = apiClient.get_client_gene(
productAuthorization.client_uuid,
productAuthorization.scopes.dataset,
geneSymbol
)
```
##### All the geneDetails object attributes are
| Attribute name PHP | Attribute name Python | Description |
|:---------------------:|:---------------------:|:-----------------------------------------------------------------------------------------|
| `molecularLocation` | `molecular_location` | An object containing the location of the gene within the chromosome - see below the molecular location object structure |
| `snps` | `snps` | A list of SNP objects present in the gene - see below the SNP object structure |
| `symbol` | `symbol` | The gene's accession string (name) |
##### Molecular location attributes
| Attribute name PHP | Attribute name Python | Description |
|:-------------------------:|:---------------------:|:-----------------------------------------------------------------------------------------|
| `chromosomeAccession` | `chromosome_accession` | The scaffold/chromosome on which the gene is placed |
| `start` | `start` | The gene's start position on the scaffold |
| `stop` | `stop` | The gene's stop position on the scaffold |
##### SNP object structure
| Attribute name PHP | Attribute name Python | Description |
|:-------------------------:|:---------------------:|:-----------------------------------------------------------------------------------------|
| `referenceGenome` | `reference_genome` | The reference genome known to be used by the Dataset's source.
This impacts the reference allele, location, and based on the dbSNP build, also the SNP's accession|
| `genotypedAlleles` | `genotyped_alleles` | The genotype value of the customer's queried SNP.
If the attribute of this SNP has the `phased` flag set to True,
the first items in the lists for all SNPs will be on the same inherited chromosome,
and analogous for the second item.
If the SNP is unphased, the order of the items is irrelevant |
| `phased` | `phased` | A boolean. True if the SNP is known to be phased, False otherwise |
| `chromosomeAccession` | `chromosome_accession` | This is the chromosome accession number where the SNP is located; in the format of NC_00x |
| `location` | `location` | This is the customer's SNP's location on the chromosome |
## Get customer genetic data in 23andMe tsv format
If your platform is already compatible with 23andMe genotype data files, you can use this specific function to generate data in the 23andMe format - list of rows in tab separated values.
#### PHP example:
```PHP
$authorizationDnaData = $apiClient->authorizationDnaData($productAuthorization["authorizationUuid"]);
```
#### Python example
```python
authorizationDnaData = apiClient.authorization_dna_data(productAuthorization.authorization_uuid)
```
`authorizationDnaData` contains a list of rows in a tsv (tab delimited values)/csv format (23andme-compatible)
# Submit reports
After you finish analysing the customer's genetic data, we need to inform the customer their analysis is complete. To do this, you will notify us using the function below. Finally, the customer will then:
* access their report file through a written electronic document (eg. .pdf or .doc)
* access their report on your website under an account with a username and a password or
* receive a physical product
## How to submit a report file
When you submit such a report file, Lumminary will save this document into the customer's account, from which the customer will then be able to access it directly.
#### PHP example
```php
$pathToReportFile = 
```html
```
##### b. Black button version
```html
```
## Button configuration
Each button has 2 attributes which need to be configured:
1. **data-partner-uuid** where you have to add your partner UUID (you have received the partner UUID after filling in the form for product registration).
2. **data-request** which is a string obtained by encrypting a serialized JSON (you have received the CWL encryption key after filling in the form for product registration). See details below.
#### Data-request object
The data-request object has a standard format which needs to be preserved. It is formed of two types of data, some mandatory and some optional. You can use the optional fields to add any metadata or other information for your own use. The data-request object is going to be returned with the response from the authentication without being altered.
##### Mandatory information
The mandatory information is a list of scopes which you ask the client to grant permission for. These scopes are comma delimited, and the possible options are: `address`, `email`, `dataset`.
The scopes _address_, _email_, and _dataset_ can be used in any combination; you must request at least one scope.
| Attribute name | Description |
|:-----------------:|:----------------------------------------------------|
| `address` | Requests access to a customer's name and address. |
| `email` | Requests access to a customer's email address. |
| `dataset` | Requests access to a customer's genetic data |
#### PHP example:
```php
$objAuthorizationRequest ["scopes"] = "address,dataset,email";
```
#### Python example:
```python
objAuthorizationRequest ["scopes"] = "address,dataset,email"
```
Product UUID is your `productUuid` for which you ask customer permissions.
#### PHP example:
```php
$objAuthorizationRequest["productUuid"] = $credentials->getProductUuid();
```
#### Python example:
```python
objAuthorizationRequest["productUuid"] = credentials.product_uuid
```
##### Optional information
In the optional part of the object, you can add any useful data, such as customer ID, session ID, or any parameter which can help you identify the response from Lumminary.
#### PHP example:
```php
$objAuthorizationRequest["customData"] = array();
$objAuthorizationRequest["customData"]["customerId"] =
```
## Connect with Lumminary summary of user interaction
When a customer clicks on the “Connect with Lumminary” button, a pop-up window opens. After they choose which genetic file to share, the pop-up will automatically close and the user will be redirected to your callback URL in the parent window. Your callback URL needs to be predefined in the Lumminary partner portal.
The GET request from the client to your callback URL will contain two querystring parameters - `request` and `response`:
1. `request` – this is exactly the same request that you previously sent in the `data-request` field. You can decrypt it with the CWL encryption key which you used to encrypt it.
2. `response` – the response is an urlencoded encrypted serialized JSON object which contains the Authorization UUID and the Authorization UTC unix timestamp. You will use the Authorization UUID to get the customer's data with the Lumminary API Client. The response string is encrypted with your CWL encryption key, the same as the `data-request` parameter.
In order to decrypt the `response` parameter, you can use the following function:
#### PHP example:
```php
// the entire callback URL, including the querystring parameters
$callbackUrlWithPayload = "https://partnerwebsite.con/callback?request=...&response=...";
$cwlReturnObject = Lumminary\Client\LumminaryApi::cwl_url_query_extract(
$callbackUrlWithPayload,
$partnerCwlKey
);
```
#### Python example:
```python
// the entire callback URL, including the querystring parameters
callback_url_with_payload = "https://partnerwebsite.con/callback?request=...&response=..."
cwlReturnObject = apiClient.cwl_url_query_extract(
callback_url_with_payload,
partner_cwl_key
)
```
The `cwlReturnObject` will now contain an object like the example below:
```json
{
"request":