BioMart 2007 Arek Kasprzyk European Bioinformatics Institute BOSC Vienna, July 2007

Data Flow Mart JAVA PERL Source data DAS Web GUI Command line Desktop GUI Web Service

Data Flow JAVA PERL Mart DAS Web GUI Command line Desktop GUI Web Service

Admin Tools

Recent developments (0.4- 0.6) MartBuilder MartView Web services API DAS Central Server More deployers

MartBuilder

MartView

Web services

API

DAS

Central Server

More deployers

Data Flow Mart JAVA PERL Source data DAS Web GUI Command line Desktop GUI Web Service

MartBuilder

MartBuilder

MartBuilder

MartView

API my $initializer = BioMart::Initializer->new('registryFile'=>$confFile); my $registry = $initializer->getRegistry; my $query = BioMart::Query->new('registry'=>$registry,'virtualSchemaName'=>’central_server_1'); $query->setDataset(" hsapiens_gene_ensembl "); $query->addFilter(" chromosome_name ", [” 1 "]); $query->addAttribute(" ensembl_gene_id "); $query->addAttribute(" ensembl_transcript_id "); $query->addAttribute(” ensembl_peptide_id "); $query->setDataset(“ msd ”); $query->addFilter(” experiment_type ", [” NMR "]); $query->addAttribute(" pdb_id "); $query->addAttribute(” resolution "); $query->addAttribute(” release_date "); $query->addAttribute(” header "); my $query_runner = BioMart::QueryRunner->new(); $query_runner->execute($query); $query_runner->printResults();

Web service <Query virtualSchemaName=&quot;central_server_1&quot;> <Dataset name=&quot; hsapiens_gene_ensembl &quot; > <Filter name=&quot; chromosome_name &quot; value=&quot; 1 &quot;/> <Attribute name=&quot; ensembl_gene_id &quot;/> <Attribute name=&quot; ensembl_transcript_id &quot;/> <Attribute name=&quot; ensembl_peptide_id &quot;/> </Dataset> <Dataset name=&quot; msd &quot;> <Filter name=&quot; experiment_type &quot; value=” NMR &quot;/> <Attribute name=&quot; pdb_id &quot;/> <Attribute name=” resolution &quot;/> <Attribute name=” release_date &quot;/> <Attribute name=” header &quot;/> </Dataset> </Query>

MartService Meta data(GET) Marts Datasets Configuration Queries (POST)

Meta data(GET)

Marts

Datasets

Configuration

Queries (POST)

Meta data http://www.mycompany.com/mypath/martservice ? Marts type=registry Datasets type=datasets&mart=mymart Configuration type=configuration&dataset=mydataset

Marts

type=registry

Datasets

type=datasets&mart=mymart

Configuration

type=configuration&dataset=mydataset

Query -O 5utr.dat <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?> <!DOCTYPE Query> <Query virtualSchemaName = &quot;default&quot; count = &quot;&quot; softwareVersion = &quot;0.5&quot; > <Dataset name=&quot; hsapiens_gene_ensembl &quot; > <Attribute name=&quot; ensembl_gene_id &quot;/> <Attribute name=&quot; ensembl_transcript_id &quot;/> <Filter name=&quot; chromosome_name &quot; value=&quot; 1 &quot;/> <Filter name=&quot; band_end &quot; value=” p36.33 &quot; / > <Filter name=&quot; band_start &quot; value=” q44 &quot;/> </Dataset> <Dataset name=&quot; msd &quot;> <Attribute name=&quot; pdb_id &quot;/> <Attribute name=” experiment_type &quot;/> <Filter name=&quot; experiment_type &quot; value=” NMR &quot;/> </Dataset> </Query> wget -q 'http://www.biomart.org/biomart/martservice?query=

Results Ordered according to Datasets Attributes Default Format TSV Can be altered by specifying a formatter

Ordered according to

Datasets

Attributes

Default Format TSV

Can be altered by specifying a formatter

Genomic data

Uniprot, MSD, ArrayExpress

Model organism databases

Developmental models

Proteomics

Name Fragment Position Alleles strand SNP1 AL139258 1659852 T/A 1 SNP2 NT_25698 2569873 C/T -1 SNP3 chr13 1125698 C/G 1 Genetics of Infectious and Autoimmune Diseases, Pasteur Institute, INSERM U730, Paris, France. Target SNP selection for the study of type 1 diabetes (T1D), malaria and dengue Data conversion and integration Ensembl HapMap NCBI UCSC Priopriatery data Diabetes-Gene Association DataBase Combined proprietary and public data

CAPRISA understanding HIV pathogenesis and epidemiology as well as HIV/AIDS treatment and prevention Clinical Data MID Cellular Immunity Humoral Immunity HLA Typing Sequence & Sequence Related Pipeline

Unilever Human study to evaluate Omics in assessing safety indicators Study of skin inflammation in response to detergent Skin samples taken and analyzed with multiple Omics techniques. Blood Skin biopsy Microdialysis

Human study to evaluate Omics in assessing safety indicators

Study of skin inflammation in response to detergent

Skin samples taken and analyzed with multiple Omics techniques.

Blood

Skin biopsy

Microdialysis

1. Filter 2. Attributes 3. Results Use Example 1 All genes in the human genome up-regulated in Pancreatic Adenocarcinomas (PDACs) vs Normal Pancreas (ND))

1. Filter 2. Attributes 3. Results Use Example 2 all upstream sequences for all genes on chromosome 1 up-regulated in Pancreatic Adenocarcinomas (PDACs) vs Normal Pancreas (ND))

1. Filter 2. Attributes Use Example 3 3. Results Just Finished my experiment and would like to get the overlaps between my results and those reported in previous studies !

Web service

Perl

DAS

Bioconductor package biomaRt

Galaxy

Taverna

Central Server (www.biomart.org)

www.biomart.org/biomart/ martservice

Future plans

New configuration system Normalized Based on a partition table concept Unified pointer system -> relational attribute Configuration merge - implicit federation Write to the db Run time slice and dice of a registry object rather than combinatorial pre-compilation

Normalized

Based on a partition table concept

Unified pointer system -> relational attribute

Configuration merge - implicit federation

Write to the db

Run time slice and dice of a registry object rather than combinatorial pre-compilation

New configuration system Scalability Updates and maintenance of large configurations Run time server scalability (cache and memory) Scalable for multiple mart users (single instance - security) Scalable for alternative configurations (new MartGUI framework)

Scalability

Updates and maintenance of large configurations

Run time server scalability (cache and memory)

Scalable for multiple mart users (single instance - security)

Scalable for alternative configurations (new MartGUI framework)

New MartGUI framework Components Alternative DS Configurations Alternative GUIs (MView, MQForm,MSForm etc) Alternative Analyzers/Vizualizers (optional install) Extensible Custom extensions to the components Common interface Formatters, DAS, Analyzers, Visualizers Importable/Exportable pair interface

Components

Alternative DS Configurations

Alternative GUIs (MView, MQForm,MSForm etc)

Alternative Analyzers/Vizualizers (optional install)

Extensible

Custom extensions to the components

Common interface

Formatters, DAS, Analyzers, Visualizers

Importable/Exportable pair interface

New GUI framework Old ‘GUI unit’: full registry+MartView+default formatters Customization limited to colors and headers New ‘GUI unit’: RegistrySlice+ MartGUI+Visualizer/Analyzer Combine units into your unique functional environment Functional level customization

Old ‘GUI unit’:

full registry+MartView+default formatters

Customization limited to colors and headers

New ‘GUI unit’:

RegistrySlice+ MartGUI+Visualizer/Analyzer

Combine units into your unique functional environment

Functional level customization

New GUI framework Gene Id conversion Functional annotation Compare two gene lists Analyze gene list Draw distribution Full search Draw bla bla chart Home Welcome to my data mining website SITE HEADER

New GUI framework Gene Id conversion Functional annotation Compare two gene lists Analyze gene list Genbank Trembl Uniprot Submit Draw distribution Full search paste your ids here Draw bla bla chart Hugo Home SITE HEADER

New GUI framework Home Gene Id converter Fu Full search Welcome to my data mining website

New GUI framework Hugo Genebank Uniprot Swissprot Submit paste your ids here Home Fu Full search Gene Id conversion

Cytogenetic distribution of pancreatic cancer genes satisfying my query (histogram)

Cytogenetic distribution of pancreatic cancer genes satisfying my query (ideogram)

Cytogenetic distribution of chromosomal aberrations in pancreatic cancer

New GUI framework

New GUI framework

New configuration tool MartConfigurator Handles a complete registry object Defines GUI units Automated service discovery Manual link override Automated updates for large configurations Improved user interaction

MartConfigurator

Handles a complete registry object

Defines GUI units

Automated service discovery

Manual link override

Automated updates for large configurations

Improved user interaction

Credits Martians Syed Haider Richard Holland Damian Smedley Contributors Steffen Durinck (NCI, NIH) Eric Just (Northwestern University) Don Gilbert (Indiana University) Darin London (Duke University) Will Spooner (CSHL) Gudmundur Thorisson (CSHL) Benoit Ballester (Universite de la Mediterranee) James Smith (Ensembl) Arne Stabenau (Ensembl) Andreas Kahari (Ensembl) Craig Melsopp (Ensembl) Katerina Tzouvara (EBI) Paul Donlon (Unilever )

Martians

Syed Haider

Richard Holland

Damian Smedley

Contributors

Steffen Durinck (NCI, NIH)

Eric Just (Northwestern University)

Don Gilbert (Indiana University)

Darin London (Duke University)

Will Spooner (CSHL)

Gudmundur Thorisson (CSHL)

Benoit Ballester (Universite de la Mediterranee)

James Smith (Ensembl)

Arne Stabenau (Ensembl)

Andreas Kahari (Ensembl)

Craig Melsopp (Ensembl)

Katerina Tzouvara (EBI)

Paul Donlon (Unilever )