Recommended
More Related Content
Viewers also liked
Viewers also liked (11)
More from jodischneider
More from jodischneider (20)
Recently uploaded
Recently uploaded (20)
Herrenhauser big data poster: Decision support on overwhelming amounts of data 2015-03-26
- 1. How to provide decision support on overwhelming amounts of data? Jodi Schneider - INRIA • jschneider@pobox.com Step 1: Understand the decision process & criteria • Ethnography o Participant Observation o Interviews • Annotation o Argumentation theory Step 2: Build a computer support system • Web standards o Develop an OWL ontology o Structure data in RDF format o Query with SPARQL • Human computer interaction: Design Step 3: Test & improve the system • Human computer interaction: User testing & Design 2. Method 3. Decision Support Case Study: Wikipedia Article Deletion Step 1: Understand the decision process & criteria Figure 3 - "CriteriaFilter" (red) improves on the native Wikipedia interface (blue), except in terms of perceived effort. 1. Challenges • Managing large amounts of data • Understanding the decision-making process • Designing interfaces that support decision making 4. Discussion • Process depends on determining key factors in the decision. • New application of method to medication safety: Which drugs shouldn't be taken together? o Provide support to evidence curators o Ontologies: micropublication, nanopublication Problems identified from interviews & participation: • Large volume: 500 deletion discussions per week • Consensus is difficult to determine. • Newcomers don't understand process & standards. Results from annotation: • Identified key criteria in discussions: Notability, Sources, Maintenance, Bias • Classified comments by key criteria. • Validated classification two ways: Interannotator agreement (.64-.82 κ) Coverage (key criteria used in 90% of comments). Step 3: Test and improve the "CriteriaFilter" system • Developed the WikipediaDeletion ontology. • Embedded the classification from the manual annotation into web pages. • Wrote custom SPARQL queries to retrieve all comments by factor. • Made “CriteriaFilter” interface by embedding queries into JavaScript. • 20 users perform tasks with both "CriteriaFilter" and the native Wikipedia interface. • Statistically significant improvements in 3 areas: o perceived usefulness o perceived ease of use o information completeness • Strong overall preference for "CriteriaFilter" (84%). • Qualitative feedback used to improve the next version. Step 2: Build a computer support system "CriteriaFilter" Figure 1 – Wikipedia deletion discussion Figure 2 – “CriteriaFilter” interface
Editor's Notes
- Information on our Poster Sessions and Lightning Talks Our poster boards are 100 (width) times 140 (height) cm wide (39, 37 times 55 inches). This means that in order to use the space provided in the most optimal way, your poster should be vertically orientated. Please note that you will have to bring your posters with you to the conference. There will be no possibility to print out the poster at the conference site. Pins, thumbtacks e.g. to attach the poster to the board are provided by us. ==== My poster will cover two case studies on providing decision support. For the 3- slide talk I will focus on the overall methodology. My newest work is about using human-machine partnerships to improve information retrieval and decision support in the biosciences. Thousands of people each year are harmed by taking medicines together, in part because current sources of information about drug-drug interactions do not agree. In ongoing work, we are using ontologies and human annotation to model the key assertions and supporting evidence from scientific papers. Our work is a prototype for a mass-collaborative system that combines text mining and human annotation to help synthesize key information about drug-drug information into a semantic knowledge base. My dissertation developed a methodology for providing decision support and information synthesis. I applied this to Wikipedia, the popular encyclopedia. Each week, about 500 borderline articles are considered for deletion from Wikipedia. These articles are discussed by groups of 2 to 200 people whose written comments are the basis of the decision. We showed that clustering topics in these comments in a new interface provides statistically significant improvements over the native Wikipedia discussion interface in terms of perceived usefulness, perceived ease of use, and information completeness. The commonality in both of these case studies is the use of both human and machine aspects. Structuring text into ontologies enables sophisticated queries using SPARQL--which can be used in smart search interfaces that summarize information. Both humans and machines can contribute to structuring text, and have complementary advantages: Text mining is fast (and can speed human analysis) while human work is accurate (and can improve subsequent text mining).q