Your SlideShare is downloading. ×
Sustainability and Cochrane Reviews: How technology can help
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Sustainability and Cochrane Reviews: How technology can help


Published on

My plenary talk at the UK Contributors' Meeting held in Loughborough, UK, 20-21 March 2012

My plenary talk at the UK Contributors' Meeting held in Loughborough, UK, 20-21 March 2012

  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Improvements in internal process, of course, affect consumers of our information downstream. I see sustainability as working both ways – internally via improved production process, priority-setting, etc. and externally in the health info marketplace reaching our audiences and changing health care. But, of course, “internal” involves working with partners, other orgs, etc.
  • From the intro to the Collaboration’s information technology strategy paper. Indeed, we were the first all-electronic medical journal. We were never in print and technology has advanced our cause. BUT, are we using technology to the betterment of our cause NOW? So, BUT....
  • Estimate is now nearly 10 years old!
  • Some technologies we’re already using, and they’re helping...
  • Refer to Phil Alderson talk, if possible...
  • The red text indicates where in the Review XML the data is extracted from, not that you can read this!
  • A mockup of the final product
  • So, in the data model and ontology we‘ve been working on in the Star Trek program of work (details to follow!), we‘re looking at drawing together the links between the PICOs. So, pulling together data via queries to easily answer questions like: “What‘s been compared to what and in what population for what outcome, etc.??“
  • So, could we use linked data technology to assist in the first step in doing overviews and network MA, drawing the map and visualizing the geometry of the interventions and comparisons...
  • Nice segue into...
  • Simple example with search...
  • First, we‘ll need to start thinking outside of the “container“ of the Review...
  • When data from Reviews is removed its context within the Review, we need to be careful that the context „travels“ with it...
  • Some of these developments require us, an organisation, to think differently about our content. The one-size-fits-all “container“ of the Cochrane Review will need to be flexible in order for use to meet new user demands and to allow for content that travels freely (any device, any platform, any context), retains its context and meaning so that people know they can trust it and allows us to create new “products“ to meet these new user demands.
  • The technologies that underpin this can be used at the level of the web itself AND within and across individual datasets. We are investigating both.Current web = web of documentsData in documents (mostly HTML/XML, etc.) not structuredLinked data allows for structuring data so that both humans and computers can understand itFor example, Cochrane Review XML is highly-structured but relationships not explicitIf they were, we could query across dataset and link other datasets for complex queriesThe web turns into a giant database (the vision, anyway)Search results display can be improvedContent enriched with external data and re-packaging of our dataMany other possiblitiesCurrent web = web of documentsMostly in HTML/XMLGood for telling computers how to display informationNot good for embedding meaning and relationships between bits of informationLinked Data technologies allow for structuring data so that both humans and computers can understand itIt’s all about structured data
  • Every presentation about the web and technology nowadays has to include a cloud image!!
  • The CRS is an exciting new project that could facilitate better linking of data. (Refer to something Gordon said in his talk, if possible/applicable).
  • Explain Star Trek joke and general framework here...
  • Explain Star Trek joke and general framework here...
  • We use diagrams like this to conceptualize the relationships. This is the findings ontology developed by Lorne Becker.
  • This is Rachel Marshall from the CEU’s work on modeling the PICOs and understanding the concepts and relationships between Reviews and studies...
  • Here is the Marshall-o-gram with GRADE and RoB per outcome modelled...
  • Reviews include comparisons. Comparisons include outcomes that are compared. Etc. This is by no means complete but was meant to assist in a “proof of concept“ exercise we called “interrogating the XML“ of Cochrane Reviews.
  • These questions build on each other in increasing complexity...
  • Just so you can visualize the potential application of this: In this Pubmed search, each Cochrane box has pointers to the Cochrane review(s) that included that trial.
  • Just so you can visualize the potential application of this: In this Pubmed search, each Cochrane box has pointers to the Cochrane review(s) that included that trial.
  • What sorts of bias are most prevalent in this particular body of research/clinical question of interest?
  • We could look at answering these kinds of questions which involve external datasets and mashups...
  • Cochrane could take the lead and model the entire knowledge space of Evidence-based Health Care in these semantic standards and create a giant triple store with our data. Then, others in the EbHC would use our ontologies and refer to our data and thus we would drive “the conversation” around the data.
  • Then, once we start throwing in other datasets, the triple store becomes even more powerful (note: not sure I drew all lines between all datasets, but you get the idea)…
  • For example, Volkswagen have done this for the car industry. They modeled the domain of car options with a car options ontology and are now positioned with “first mover” advantage in the car industry in leveraging semantic technologies.
  • This crazy image is the Linked Data cloud which shows all the various datasets currently in the web of data. The pink area is the life sciences area and includes PubMed, DrugBank and others.
  • Transcript

    • 1. Sustainability & Cochrane Reviews How technology can helpChris MavergamesDirector of Web DevelopmentThe Cochrane Collaboration
    • 2. Structure of this talk• Sustainability of Review production (internal sustainability) ▫ Technologies to assist Review production ▫ PICOtron and Overviews of Reviews• Sustainability in the health information marketplace (external sustainability) ▫ “Nimble” content and thinking outside Review “container” ▫ Linked data and Star Trek• Summary
    • 3. Information Technology Strategy“Information and Communication Technology (ICT) is at the heart of The Cochrane Collaboration ... To a very large extent, the success of The Cochrane Collaboration has been based on its investment in ICT.” - From the Collaboration‟s information technology strategy paper (ISSF)Our data and systems are great, but...
    • 4. Can we make themachines work a bit harder?
    • 5. Some critical points• For internal processes ▫ How to automate w/o creating more work ▫ Important that methodology remains sound• For external consumers ▫ Retain context of findings and results ▫ Create better understanding, not confusion
    • 6. Sustainability ofReview production (and updating)
    • 7. Producing Reviews - challenges• We have this incredibly complex, methodologically rigorous process and aim to be global and comprehensive with a mostly-volunteer workforce• We‟ve just cleared 5,000 reviews and its taken nearly 20 years, and these need to be maintained and updated• Estimates say we need min. 10,000 Reviews to be comprehensive (estimate 10 years old!)
    • 8. Cochrane Reviews are fantasticbut...• …creating them is a long and laborious process
    • 9. Cochrane Reviews are fantasticbut...• …creating them is a long and laborious process Help!
    • 10. Review production assistance• Reference managers ▫ EndNote, Zotero, Connotea, Mendeley and others• Screening/appraising references ▫ ScreenToGo App (email: for public beta)• Automated abstract screening/appraisal ▫ Semi-automated appraisal Support Vector Machines (SVMs)• Software covering multiple steps in systematic review production ▫ EPPI-Reviewer ( ▫ Distiller SR ( ▫ EROS (Early Review Organizing Software ▫ Automated data extraction from Cochrane Reviews (reading the XML)• GRADEpro• CRS Slide courtesy: Rachel Marshall
    • 11. What if authors could...?• Pull in Risk of Bias assessment information already done on trials or studies• See studies already included or excluded in other Reviews• Other links between studies and Reviews via the CRS – more later in Star Trek section• Use datasets like Drugbank and Diseasome for auto-completion or help filling out fields in RevMan for standardisation
    • 12. Could we use technology to assistwith…?• Using MeSH mappings to show coverage of Reviews• Using study links to show where gaps exist• Priority-setting• Creating derivative “views” or products from Cochrane data
    • 13. PICOtron
    • 14. PICOtron and Cochrane ClinicalAnswers• Project to semi-automate populating Cochrane Clinical Answers, a derivative product of The Cochrane Library• Python script written by Iain Marshall that extracts data from Cochrane Reviews• Script automatically and randomly reconfigures Review titles into one of 30 question formats
    • 15. PICOtron• Data is pulled from various areas of the Review XML• Depending on whether the result is significant and favored intervention, a sentence (narrative result per PICO) explaining these results is automatically generated• Then, an author checks these and combines this information across PICOs into a “clinical answer“
    • 16. PICOtron output
    • 17. Overviews of Reviews &Network Meta-analysis
    • 18. Help in preparing of Overviews of Reviewsand Network Meta-analysis driven by linked data Image from Lorne Becker
    • 19. Example of a trial level paroxetine synthesis sertraline citalopram 12 new generation escitalopram fluoxetine antidepressants: fluvoxamine Which ones are the milnacipran most efficacious? venlafaxine reboxetine bupropion mirtazapine duloxetineThanks to Georgia Salanti. Source:
    • 20. Network of Randomized Trials sertraline milnacipran reboxetine paroxetine mirtazapine duloxetine fluvoxamine escitalopram citalopram bupropion venlafaxine fluoxetineThanks to Georgia Salanti. Source:
    • 21. Network of Randomized Trials sertraline milnacipran reboxetine No trials paroxetine comparing mirtazapine reboxetine to bupropion duloxetine ? available in Cochrane Reviews fluvoxamine escitalopram citalopram bupropion venlafaxine fluoxetineThanks to Georgia Salanti. Source:
    • 22. Other ideas?Perhaps we can gain insight into how toimprove Review production processes fromhow our users actually use our output? segue ...
    • 23. Sustainability in the health information marketplace
    • 24. “The problem is not information overload but filter failure.” – Clay Shirky• Cochrane provides a vital curation, filtering and evaluation/quality-assessment role and we need to make this clear
    • 25. Again, Cochrane Reviews arefantastic BUT...• There are problems that limit their use by some people ▫ Difficult to wade through all of the text ▫ Difficult to understand the figures, terminology, and other bits of the Review ▫ Hard to compare interventions without reading multiple Reviews ▫ Moving from studies in CENTRAL to Reviews that included that study difficult ▫ Can be difficult to find the Review you seek
    • 26. • Search for “Prozac” – no reviews• Search for “fluoxetine” – 27 reviewsSearching The Cochrane Library
    • 27. Ideally, we‟d restructure ourcontent for different users• Beginning to do this now: ▫ for consumers ▫ Cochrane Clinical Answers for clinicians• BUT ▫ Takes a lot of work to reformulate reviews & authors, CRGs, etc are busyWouldn‟t it be nice if we could automate or semi-automate some of these processes?
    • 28. Thinking outside Review container
    • 29. Thinking outside Review container
    • 30. Making our content “nimble”Structured and linked data can help makeour content “nimble”Nimble content can: • Travel Freely • Retain Context Meaning • Create New Products - R. Lovinger, Razorfish
    • 31. Linked data
    • 32. What is linked data? Semantic Web is made up of: Linked Data & Web of Data Which all together comprise Web 3.0
    • 33. Current web = Web of documents
    • 34. Docs are linked not data in docs
    • 35. Machines aren„t good at readingweb pages• Data on the web is meant for human consumption• Machines need the data to be structured• Once structured, information can be more easily shared within datasets and across web pages
    • 36. It„s about...• Taking the complex relationships, interactions and dependencies in our data and modeling them in semantic web language and concepts for machine processing...• So that we can do things like: ▫ Gain insights into our data ▫ Help with priority setting ▫ Repackage it for different users ▫ Later, perhaps machine can infer new knowledge from our data and/or when our data is combined with other datasets
    • 37. Cochrane Register of Studies
    • 38. CRS and CENTRAL• Lack of unique study IDs a real problem• CRS solves this by providing a unique ID for all studies that can be referenced• Better linking of data about trials and to Reviews ▫ Example: Using forest plots to generate related studies lists for CENTRAL• Possibilities with linking to external sources such as PubMed
    • 39. Insert witty Star Trek reference here!
    • 40. Findings ontology Image: Lorne Becker
    • 41. Marshall-o-gram
    • 42. Marshall-o-gram
    • 43. Cochrane Review ontology
    • 44. We can…• Ask questions that use data from several different reviews• Improve search• Make it easier for people to find Cochrane Reviews• Link data from studies and Reviews better• Enhance the experience of our users by including data from other datasets
    • 45. A question using multiple reviews I’ve done a search for trials on a particular intervention for dementia. I want to know which of the trials have been included in a Cochrane Review and a summary of the risks of bias for the entire set of trials.
    • 46. Links to the relevant Review forthose trials that were included
    • 47. This study was one of 38 studies included in the Cochrane Review, <Title Here>. Click here to see the full reviewLinks to the relevant Review forthose trials that were included
    • 48. RoB Summary for Cochrane Reviews ondementia These figures summarize Risks of Bias from the trials included in the reviews in your search
    • 49. Enhancing the User Experience Make search work better
    • 50. You Say “Paracetamol”I Say “Acetaminophen”• Or, one could say any of these:Abenol (CA), Acephen, AnadinParacetamol (UK), Apo-Acetaminophen (CA), Aspirin Free Anacin, Atasol (CA), Calpol (UK), Cetaphen, Childrens Tylenol Soft Chews, Disprol (UK), Exdol (CA), Feverall, Galpamol (UK), Genapap, Genebs, Infants Pain Reliever, Mandanol (UK), Nortemp, Pain Eze, Panadol (UK), Robigesic (CA), Silapap, Tycolene, Tylenol 8 Hour, Tylenol, Tylenol Arthritis, Uni- Ace, Valorin
    • 51. Cochrane leading in Web 3.0?
    • 52. CRS/ CDSR CENTRAL HTAs DARE CMREbHC Semantic Platform
    • 53. CRS/ CDSR CENTRAL UMLS Drug Bank Diseasome HTAs DARE Symptom CMR * BBC Health Ontology OntologyEbHC Semantic Platform * Not yet created
    • 54. Cochrane and EbHC ontology?
    • 55. Will Cochrane have a bubblehere someday?
    • 56. Summary• Technology IS at the heart of what we do• For both internal and external applications, we can leverage these tools to further our mission• Requires that we think differently about the “container“ of the Review• Our data needs to become “nimble“ to meet future user needs• We should proceed slowly, incrementally - What are the “quick wins“?• Cochrane has the chance to lead in Web 3.0
    • 57. Thank you