Successfully reported this slideshow.

DREAM Challenge


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

DREAM Challenge

  1. 1. DREAM Challenge Gustavo Stolovitzky, IBM Computational Biology Center Andrea Califano, Columbia University
  2. 2. DREAM• DREAM is a Dialogue for Reverse Engineering Assessments and Methods.• The main objective is to catalyze the interaction between experiment and theory in the area of cellular network inference and quantitative model building in systems biology
  3. 3. Challenges• Network Topology and Parameter Inference Challenge• Sage Bionetworks - DREAM Breast Cancer Prognosis C• The DREAM-Phil Bowen ALS Prediction Prize4Life• NCI-DREAM Drug Sensitivity Prediction Challenge
  4. 4. Project/Challenges• Network Topology and Parameter Inference challenge • Develop/apply optimization methods including the selection of the most informative experiments, to accurately estimate parameters and predict outcomes of perturbations in Systems Biology models given • In Model 1 the complete structure of the model (including expressions for the kinetic rate laws) for a gene regulatory network composed of 9 genes. Protein and mRNA are explicitly modeled. • In Model 2 an incomplete structure of the model, with missing regulatory links, for a gene regulatory network composed of 11 genes. Here, participants will also have to find the missing links. Only proteins are explicitly modeled.
  5. 5. DREAM Phil Bowen ALS Prediction Prize4Life• Goal • Challenge is to predict the progression of disease in ALS patients based on the patient’s current disease status • Specifically develop an approach to predict a given patient’s disease status within a year’s time based on 3 months of data• Data • Includes demographics, medical and family history data, functional measures, vital signs, and lab data (blood chemistry/hematology/urinalysis) collected at multiple times• Disease progression will be calculated as the average change in (ALSFRS) Amyotrophic Lateral Sclerosis Functional Rating Scale over a year’s time from enrollment in a clinical trial• At the end of the challenge, the prediction submitted (based on 3 months of data) will be compared against the actual ALSFRS slope experienced by the patient over a year
  6. 6. Output• Improve disease prediction beyond the current capabilities by • Developing more accurate (sensitive and specific) methods of predicting progression, • Identifying markers (variables) that would enable a determination of expected future disease progression earlier on in the course of the disease• Validation • Validate the model against a subset of patients that are neither part of the training set nor the final validation (test) set. • Submit the actual code written in R language (“Validation Code”) and InnoCentive will run the code against the interim validation data set
  7. 7. NCI-DREAM Drug Sensitivity Prediction Challenge• Use genomic information to build models capable of ranking the sensitivity of cancer cell lines to a set of small molecule compounds or their combinations
  8. 8. Sub Challenges• Sub Challenge 1 • Predict the sensitivity of breast cancer cell lines to previously untested compounds • Model capable of ranking the sensitivity of 18 breast cancer cell lines to 31 compounds • Challenge in this case is to link the drug effects to the underlying genetics of the 53 cell lines.• Sub Challenge 2 • Predicting compound combinations that have a synergistic effect in reducing viability of a DLBCL cell line • Predict the activity of pairs of compounds in the DLBCL LY3 cell line from expression profiles acquired after treatment of the cell line with each of 14 individual compounds
  9. 9. Sage Bionetworks - DREAM Breast Cancer Prognosis Challenge• Background • Molecular diagnostics for cancer therapeutic decision making are among the most promising applications of genomic technology • Molecular profiles have proved particularly powerful in adding prognosis information to standard clinical practice in breast cancer• Trends emerging • Genes defining predictive signatures of the same phenotype often do not overlap across studies • Predictive signatures are not very robust • No consensus regarding the most accurate signatures or computational methods for inferring predictive signatures • No consensus regarding the added value of incorporating molecular data in addition to or instead of traditionally used clinical covariates
  10. 10. Goal/Challenge• Goal • To assess the accuracy of computational models designed to predict breast cancer survival, based on clinical information about the patients tumor as well as genome-wide molecular profiling data including gene expression and copy number profiles• Challenge • Create a community-based effort to provide an unbiased assessment of models and methodologies for the prediction of breast cancer survival • Common dataset will be provided to all participants, with a validation dataset held out for model evaluation • Novel dataset will be generated at the end of the Challenge and used to provide a final, unbiased score for each model
  11. 11. DATA• Training data set from METABRIC cohort of 2000 breast cancer samples • Include detailed clinical annotations • 10 median year survival time, gene expression and copy number data• Additional breast cancer datasets curated by Sage Bionetworks • Can be use in the model development• Web based platform called Synapse • Enable transparent reproducible model building and analysis workflows as well as sharing of data, tools and models with the challenge community• Validation dataset • Derived from 300- 500 fresh frozen primary tumors with the same clinical annotations and survival data as the METABRIC cohort
  12. 12. DATA• Survival data  • Survival data is loaded into R as a Surv object as defined in the R survival package. • This object is simply a 2 column matrix with sample names on the rows and columns: • time – time from diagnosis to last follow up. • status – weather the patient was alive at last follow up time• Feature data • Gene expression data. • Performed on the Illumina HT 12v3 platform • Loaded as Bioconductor ExpressionSet object • Data normalized• Copy number data. • Performed on the Affymetrix SNP 6.0 platform • Loaded as Bioconductor ExpressionSet object • Data normalized• Clinical covariates • Loaded as a data.frame  object with features
  13. 13. Covariates Features
  14. 14. Covariates Features
  15. 15. Submission• Models built for this Challenge will be constructed using the R programming language and uploaded to a common platform (Synapse) provided by Sage Bionetworks• Models will be uploaded as R objects implementing a function called customPredict() that returns a vector of survival predictions when given a set of feature data as input• customPredict() function will be run by a validation script for each submitted model and resulting predictions will be scored• Phase 3 submissions must be accompanied by a write-up that includes a short description of the approach used in the final model
  16. 16. Scoring• Challenge models will be scored by calculating the concordance index between the predicted survival and the true survival information in the validation dataset (accounting for the censor variable indicating whether the patient was alive at last follow-up)• Final assessment of models and the determination of the best performer will be based on the concordance index of predictions on the test dataset in Phase 3 of the Challenge.• In addition, other scoring metrics will be considered depending on the suggestions of the community throughout the Challenge
  17. 17. Time Line• Deadline for submitting models for the Breast Cancer Prognosis Challenge is 5PM EST October 15th• Best performers will be announced at the DREAM 7 Conference taking place in San Francisco on November 12 to 16• Final assessment of all models in newly generated data • Additional cohort of 350 breast cancer samples with archived fresh frozen tumor samples has been identified by Anne-Lise Borresen-Dale of Oslo University Hospital and a generous donation has been made by the Avon Foundation to obtain gene expression and copy number data on these samples • Currently curation of the clinical records of this patient cohort to harmonize with the current METABRIC dataset and generation of the genomic profiling data for these samples is being carried on • Aim is to generate these data by the November 12 DREAM conference