Skip to content

Latest commit

 

History

History
111 lines (73 loc) · 4.48 KB

README.md

File metadata and controls

111 lines (73 loc) · 4.48 KB

AskBeacon Code Base

🛑 AskBeacon is meant to operate in conjunction with sBeacon. There are detailed instructions on how to set up sBeacon with terraform templates in this rep https://github.com/aehrc/terraform-aws-serverless-beacon. To ensure the connection between sBeacon and askBeacon is robust and secure please get in touch via http://bioinformatics.csiro.au to obtain tailored (i.e. security sensitive) instructions. In the meantime we encourage people to get familiar with the system through the set up demo here. 🛑

Demo Instance

We have hosted a demo instance to allow potential researchers to perform basic evaluation of the system. The demo instance uses 1000 Genomes Phase 3 data. You can access the demo instance using the following link.

🛑 Variant queries of 1000 genomes phase 3 data is associated with GRCH37. Hence queries must be of the following form;

Get me all the individuals having a variant in first chromosome between 11856377 to 11856379 under assembly id "GRCH37"

Launch the demo

Please use the following credentials

Please note that this AskBeacon instance demonstrates examples 2 and 3 from the paper (i.e. subsets of the 1000 Genome Project). Interactive access to any complete datasets, such as Parkinson's Progression Markers Initiative (example 1) can only be make available by the original data custodians, so this demo is meant solely to showcase AskBeacon's functionality rather than to support research.

AskBeacon Web Interface

sbacon

AskBeacon is accessible from Analytics tab followed by AskBeacon analysis selection as follows.

sbacon

AskBeacon can also be acessed through the Query tab followed by clicking the chat icon. This capability is for automatically populating the sBeacon UI to interact with the canned sBeacon visualization options.

sbacon

AskBeacon Core Logic

AskBeacon core logic is presented in this repository. This repository contains the following jupyter notebooks and their functionality is as follows.

1 - Extract Information

This notebook outlines how we extract information pertaining to a Beacon query from the user.

2 - Ontology Retrieval

This notebook demonmstrate the ontology indexing and retrieval process based on extracted information. This uses the following files.

terms.csv

This directly comes from sBeacon backend. sBeacon keeps a terms table and has the following format.

term label scope
OBI:0000070 genotyping assay cohorts
OBI:0000070 genotyping assay cohorts

embeddings.csv

This is the resulting file from the embedding of ontology terms. This is loaded into memory and indexed using docarray for ontology lookup.

term label scope embedding
OBI:0000070 genotyping assay cohorts [-0.06088845431804657, ...]
OBI:0000070 genotyping assay cohorts [-0.06088845431804657, ...]

3 - Extraction

This is the extraction portion of the AskBeacon analytics facility. The notebook outlines how we generate the extractor code using the Beacon V2 SDK.

4 - Execution

This notebooks shows how we generate the code that we'll be executing to perform the analysis. Note that this notebook relies on the following file.

metadata.json

This file has the summary information of the extracted data from the previous step.

{
  "table_names": ["data"],
  "table_metadata": [
    [
      [
        "id",
        "diseases",
        ...
      ],
      [
        "int",
        "list",
        "dict",
        ...
      ]
    ]
  ]
}

Utils

You can find the utils we have used under following directories.

utils

This has pydantic models, templates, vector db and code sanitisers used in our implementation

analytics_utils

This has the runners of extractors and execution codes. You might also find how we generate the metadata for metadata.json here as well.