This document discusses data management requirements for predictive modeling using large datasets from multiple clinical, specimen, and lab repositories. It notes the need to assemble complete and up-to-date datasets while maintaining quality assurance and transparency. Over time, data storage systems experience problems with exponential data growth, manual data curation difficulties, and challenges integrating heterogeneous databases across different research groups. The document examines a spectrum of potential data management approaches and highlights collaborative networks and use of open source platforms as ways to address these issues.

1. Data Management for Predictive ToolsPaul Fearn, MBANLM Informatics Research FellowBiomedical and Health InformaticsUniversity of Washington | Fred Hutchinson Cancer Research CenterSeattle, WashingtonPROSTATE CANCER: PREDICTIVE MODELS FOR DECISION MAKINGApril 7th – 9th, 2011 - MSKCC - New York, NY

2. Data Management RequirementsNeed to assemble large datasets for predictive modelingPooling data across sites, systems and countriesLinking data across clinical, specimen and lab repositoriesQuality assurance (for reproducibility of results)Tradeoffs between accuracy and reproducibility of data pointsTransparency of data processingComplete and up-to-date datasetsEase to access, sort, filter and export dataStatistical analysis in Stata, R, SPSS, SAS, ExcelSQL queries and reportsSustainabilitySecondary (N-ary) use of clinical and research dataCumulative cost of data entryCumulative cost of staff training and turnoverCumulative risks and opportunity costs of staff entrenchment

3. The Growth ProblemLu Z. PubMed and Beyond. Database 2011;2011:baq036 21245076[pmid]

4. The Growth Problemhttp://www.ncbi.nlm.nih.gov/genbank/genbankstats.html

5. The Growth Problemhttp://www.ncbi.nlm.nih.gov/books/NBK44423/

6. The Breaking Point1000 cases

7. The Growth ProblemMicrosoft Access databases1999 ProstateDB 1.02000 PRDB / ProstabaseColdFusion & SQL Server web-based database2002 Valhalla 1.0 – 1.1Prostate2003 Valhalla 1.2 (7,994 patients)Billing/EMR compliant populated clinic formsASP.NET & SQL Server web-based database2004 CAISIS 2.0 – 2.1 (26,470 patients)Integrated bladder, kidney, testis2005 CAISIS 3.0 – 3.1 (44,000 patients)Prostatectomy eForm, protocol manager, tumor maps2006 CAISIS 3.5 – (55,000 patients)GU and Urology Prostate Follow-up eForms2007 CAISIS 4.0 – (80,000 patients)Metadata, dynamic forms, new diseases and eForms2008 CAISIS 4.1 – (98,000 patients)Email eForms, advanced find, specimen tracking2009 CAISIS 4.5 – (120,000+ patients)Project tracking, patient education, virtual fields, reporting module2010 CAISIS 5.0x

8. The Curation ProblemIncreasing volume of dataMore data points for annotationClinical / patientGenomic / biologicalPublic health / environmentParallel curation issues in modern clinical and biological research databases (Krallinger 2008*)Development of NLP system to support clinical research operations (Savova 2010**)*18834499[pmid], **20819853[pmid]

9. On the Other Hand…Long tail of research effortsSmall heterogeneous labs and projectsSubsets of dataSpecialized requirementsInnovative approaches

10. Spectrum of ApproachesOne dataset per project (i.e. study based systems)Registry databases (i.e. one treatment or disease)Data warehouse or data repositoryCommon schema (data model)“Amalgamation” of heterogeneous datasetsCommon security and accessCommon syntax (data format)Defined links between recordsIndexed for searching and retrievalFederation / grid of semantically integrated dataCommon vocabulary / terminologyFormal models (caBIG)

11. Loosely Linking Datahttp://www.ncbi.nlm.nih.gov/sites/gquery

12. Tightly Integrating DataVocabulary / TerminologyNCI Thesaurus (NCIt)NLM UMLSStandard data modelscaBIG / caDSRHL7/FDA/NCI CDISC / BRIDGWeb services*Common syntax / format*Stein. Creating a bioinformatics nation. Nature (2002) vol. 417 (6885) pp. 119-20 12000935[pmid]

13. The CAISIS System

14. Appendix: 394 people at 60 sites visited from Aug, 2008 to Jun, 2009DrivingFlying

15. Rise of collaborative networks (e.g. CTSAs)

16. Costly curation and support of research databases

17. Widespread and large scale implementation of EMRs

18. Development of data warehouses and repositories

19. Integrating biospecimen repository systems and data

20. Difficulties accessing and retrieving research data

21. Skewed distribution of data systems

22. Prevalence of Microsoft Access and Excel solutions

23. Shifts to less expensive and more open source platforms

24. REDCap, CAISIS, caTissue, Python and BioconductorAppendix: Site Visit Findings

25. Appendix: Clinical SystemsSurgical ReportsRadiation Therapy ReportsPathology ReportsLaboratory ReportsRadiology ReportsReview of Systems and Patient Reported OutcomesElectronic Medical / Health RecordsRegistration / demographicsClinical trials eligibility and recruitmentScheduling and operations

26. Appendix: Engaging Patients in Data ManagementPre-first visit questionnairesWeb-based survey systems (e.g. REDCap)Patient reported outcomesLongitudinal follow-up processTablets, iPads and mobile applications