SlideShare a Scribd company logo
1 of 37
Download to read offline
Ed H. Chi, Lichan Hong, Julie Heiser, Stu Card	

Palo Alto Research Center	


The user study portion of this research has been
funded in part by ARDA NIMD program.


                               Ed H. Chi VAST 2006   1
Ed H. Chi, Lichan Hong, Julie Heiser, Stu Card,
Michelle Gumbrecht	

Palo Alto Research Center	



The user study portion of this research has been
funded in part by ARDA NIMD program.
                               Ed H. Chi VAST 2006   2
Reading is an essential human activity.	

    •    Giant leaps forward is marked by new and better ways to
         find, correlate, and comprehend information.	





                   Ed H. Chi VAST 2006          Copyright 2004 PARC   3
Many books digitized in the
                                                               (goal)
  digital library effort.	

                90%
                                                                                (current)
    •  Amazon, Google, and




                                        RECALL
                                            75%
        CMU’s Million Book
        Project.	

Intelligent Analysts spend an
    enormous amount of time                      0%
    Reading! [Pirolli, Lee, Card]	

                  0
                                                                   TIME




                  Ed H. Chi VAST 2006                     Copyright 2004 PARC               4
Subject Indexes as a new reading “device”	

   •  Invented in the 15th Century [Dewar98]	

   •  Method of design refined thru centuries 	





                           Peter Heylyn's 1652 Cosmographie in Four Bookes


               Ed H. Chi VAST 2006                        Copyright 2004 PARC   5
Instead of generating new
indexes using IR
techniques, why not
enhance them?	

Take advantage of
centuries of experience
in building subject
indexes.	





                Ed H. Chi VAST 2006   Copyright 2004 PARC   6
Ed H. Chi VAST 2006   Copyright 2004 PARC   7
Readers need help in directing their attention to the most
relevant passages to their topic of interest.	


Idea: conceptually highlight passages and keywords that
are related to user search keywords.	





                Ed H. Chi VAST 2006      Copyright 2004 PARC   8
Conceptually highlight any relevant
User first type search keywords:                   passages and keywords	

     “anthrax symptoms”	





                                                                         Draw user attention	





                            Ed H. Chi VAST 2006             Copyright 2004 PARC                   9
Biohazard 	

   •  by Ken Alibek 	

   •  non-fiction retelling of
      his experiences
      working on biological
      weapons in the
      former Soviet Union. 	

   •  13 index pages in two
      columns, consisting
      829 entries	



                Ed H. Chi VAST 2006   Copyright 2004 PARC   10
Associated Entries
                      underlined in red




                       Exact Matches
                           in red




Ed H. Chi VAST 2006               Copyright 2004 PARC   11
Ed H. Chi VAST 2006   Copyright 2004 PARC   12
Ed H. Chi VAST 2006   Copyright 2004 PARC   13
Ed H. Chi VAST 2006   Copyright 2004 PARC   14
Ed H. Chi VAST 2006   Copyright 2004 PARC   15
Goal: Compare how users find, compare, and comprehend
information using the ScentIndex and 3Book versus the
physical book.	

It’s not clear to us that the ScentIndex would be better,
because:	

   •    Unfamiliarity with 3Book Interface (page turning, clicking on page
        numbers, use of search box)	

   •    Inability to grasp the ScentIndex concept (reorganization might
        be confusing, harder to read the index page on screen)	

   •    Readability of the Screen (hard to read a large number of pieces
        of text)	

   •    Users might be very good at using the physical book index.	



                   Ed H. Chi VAST 2006            Copyright 2004 PARC        16
Study Design:	

    •    Within-subject	

    •    Interface condition (ScentIndex vs. Physical Book Index), and 	

    •    Task Type (find, compare, comprehend), 	

    •    with the order of the interface used and expertise level as the
         between-subject variables.	

Subjects: 16 subjects (8 experts on the content, 8 novices)	

Materials: subjects used PC with two LCD monitors, and the
   physical Alibek book.	





                     Ed H. Chi VAST 2006            Copyright 2004 PARC      17
Overall, the ScentIndex eBook performed better
against the physical Book. 	

Faster Speed: 	

   •  Subjects using the ScentIndex were faster in
      completing their tasks no matter whether they were
      experts or novices, F(1,12)=12.96, p<.01.	

More Accurate:	

   •  Answers that they provided while using ScentIndex
      interface were more accurate, F(1,12)=3.991, p=.06. 	




                Ed H. Chi VAST 2006        Copyright 2004 PARC   18
Ed H. Chi VAST 2006   Copyright 2004 PARC   19
The difficulty seems to be, not so much that we publish unduly
in view of the extent and variety of present-day interests, but
rather that publication has been extended far beyond our present
ability to make real use of the record.” --- V. Bush 	


Indeed, our goal is to enhance current Browsing Interfaces
for more productive reading.	





                 Ed H. Chi VAST 2006        Copyright 2004 PARC    20
The user study portion of this research has been funded in part by
contract #MDA904-03-C-0404 to Stuart K. Card and Peter Pirolli from
the Advanced Research and Development Activity, Novel
Intelligence from Massive Data program. 	

We thank Jock Mackinlay for some fruitful conversation about the
interaction of the eBook; Michelle Gumbrecht and Tan Lee for
running some of our experiments; Pam Desmond for help in the data
analysis, and Brian Tramontana for the video production.	


Contact:	

Ed H. Chi (chi@acm.org)	





                   Ed H. Chi VAST 2006        Copyright 2004 PARC     21
Ed H. Chi VAST 2006   Copyright 2004 PARC   22
Ed H. Chi VAST 2006   Copyright 2004 PARC   23
Ed H. Chi VAST 2006   Copyright 2004 PARC   24
Ed H. Chi VAST 2006   Copyright 2004 PARC   25
Page
                          Textures

                 sample              Renderer
    scan

 Page
Images                   Words +
           extract       Locations
   OCR

                          Word Association
 Text
           compute            Matrix


                     Sentence                   Scent
         parse       Structure               Highlights


                     Ed H. Chi VAST 2006                  Copyright 2004 PARC   26
Ed H. Chi VAST 2006   Copyright 2004 PARC   27
Early proposal of an indexing system: Memex [Bush45]	

Electronic Books: Rocket eBook, SoftBook Reader, DigiPaper
[Huttenlocher00], DjVu [DjVu Zone03], PDF [Adobe03], MS Reader
[MS03].	

3D Electronic Books: SGI Demo Book [SGI93], WebBook [Card96],
British Library Turning the Pages [BL03], 3Book [Card03].	

Computer help search systems such as Apple or Microsoft.	

Google or AltaVista provide highlighting and searching	

Automatic Indexing in IR: use noun-phrases and parsers to create
indexes [Wacholder01, Nevill-Manning99]. Scatter/Gather [Cutting91].	

Concept similar to: Word Co-occurrence [Schuetze99], InfoScent and
Spreading Activation [Chi01, Chi00].	





                   Ed H. Chi VAST 2006           Copyright 2004 PARC      28
HyperText Book Systems: SuperBook [Remde87] provides a dynamic
TOC with fisheye DOI.	





                 Ed H. Chi VAST 2006        Copyright 2004 PARC   29
Two issues:	

    •    M is calculated using a 40 word window	

    •    Caveat: Exact word matches do not always show up.	

         •    Solution: Insert large values onto the diagonal	





                       Ed H. Chi VAST 2006                 Copyright 2004 PARC   30
An experimenter without prior knowledge of how the ScentIndex
system works devised a total of 12 tasks. 	

The tasks were divided into two groups of six tasks each. 	

    •    Between the two sets of questions, half of the subjects received one set
         first; the other received the other set first.	

    •    Tasks from one group were designed to be one-to-one equivalents of
         the other group. 	

    •    Of these six tasks, two were Simple Fact Retrieval questions, two were
         Dispersed Comparison questions, and two were Comprehension
         questions. 	





                     Ed H. Chi VAST 2006               Copyright 2004 PARC          31
Initial Survey (on computing and search experiences.)	

4 expert and 4 novice subjects used the Book interface first, and the
other eight used the ScentIndex first. 	

Subjects were trained to use the ScentIndex immediately before they
needed to use it. 	

Each task was given on a separate sheet of paper. Read, understand each
question completely before they start the task. 	

    •    time limit for each task (simple retrieval=2min, comparison=4min,
         comprehension=6min) with one minute warnings. 	

    •    For each interface, subjects performed the simple fact retrievals first,
         the dispersed comparisons second, and the comprehension questions
         last. 	

Post Survey (comments, preferences).	



                      Ed H. Chi VAST 2006               Copyright 2004 PARC        32
Simple Fact Retrieval:	

      •    The last natural occurring case of WHICH virus occurred in Somalia in 1977?	

      •    Who received a state award for developing a Q fever weapon?	

Dispersed Comparision:	

      •    What is the death rate of smallpox and tularemia? Which virus has a higher
           death rate?	

      •    What year did Russia open negotiations with Iraq for large fermentation vessels?
           What year did Vladimir Kryuchkov become chairman of the KGB? Which
           occurred first?	

Comprehension:	

      •    Pasechnik’s defection to the West had grave implications for the Soviet
           biowarfare program. Match the person with the fact that describes how they’re
           involved:	

      •    Persons: Frolov, Chernyayev, Karpov, Vinogradov	

      •    Facts: A. First told Alibek about Pasechnik’s defection. B. Deputy minister who
           refused to sign formal diplomatic reply. C. Given demarche that said US have
           “new information”, presumably given by Pasechnik. D. Told American visitors
           that Pasechnik’s jetstream milling machine was for “salt”.	



                            Ed H. Chi VAST 2006                Copyright 2004 PARC            33
Time to Completion	

                                                       6
Participants using the
ScentIndex interface performed




                                          ln(completion time) in seconds
                                                                           5.5

tasks faster than those using the
                                                                            5
Book, F(1,12)=12.96, p<.01.	

Many tasks not completed in                                                4.5
the time allotted using the Book
interface. 	

                                                              4                            Book

    •    6/7 for simple retrieval, 	

                                                                   ScentIndex
                                                                           3.5
    •    7/8 for comparison, 	

    •    3/5 for comprehension.	

                                                                            3
                                                                                 Simple      Dispersed          Comprehension




                              Ed H. Chi VAST 2006                                         Copyright 2004 PARC                   34
Natural log transformation on the
completion time 	

                                                                             6
As predicted, experts performed




                                            ln(completion time) in seconds
tasks faster than novices overall 	

                                        5
                                                                                                                     5.435
     •    (Expert Mean=4.85, S.D.=.
                                                                             4                   4.987
          212, Novice Mean=4.58,
          S.D.=.212, F(1,12)=17.7, p<.                                       3   3.722
          01.) 	

                                                                             2
     •    There were no interactions. 	

Simple Retrieval < Dispersed                                                 1
Comparison < Comprehension, F
                                                                             0
(2,24)=204, p<.01. 	

                                                           Sim ple      Dispersed          Com prehension




                                Ed H. Chi VAST 2006                                        Copyright 2004 PARC                    35
Converted the scores for each task to a percentage.	


     (measured in     Simple          Dispersed    Comprehen
     points)          Retrieval       Comparison   sion
     ScentIndex       M=1.88          M=1.88       M=1.77
     eBook            S.D.=.342       S.D.=.269    S.D.=.284

     Book             M=1.75          M=1.58       M=1.84
                      SD=.447         S.D.=.516    S.D.=.259


We found that users performed better using the ScentIndex, reaching
marginal significance F(1,12)=3.991, p=.06. 	

We found no difference between experts and novices.	




                    Ed H. Chi VAST 2006            Copyright 2004 PARC   36
Users overwhelmingly preferred the ScentIndex interface (15/16)	

    •    “can search using keyword combinations”	

    •    “clicking on page number to navigate”	

    •    “highlighting enables faster scanning and skimming”	

    •    “easier to compare index entries because it’s all on 1 page.”	

Some users mentioned that they prefer paper for extensive reading	

Potential Future work:	

    •    Compare with search engines (organize results by relevancy).	

    •    Understand difference between this technique and keyword finding.	

    •    Limit the page number list of each relevant index entry to only the pages
         that are relevant to the keywords specified. 	





                      Ed H. Chi VAST 2006                 Copyright 2004 PARC        37

More Related Content

Similar to Smart eBooks: ScentIndex and ScentHighlight research published at VAST2006

What is digital humanities ,By: Professor Lili Saghafi
What is digital humanities ,By: Professor Lili SaghafiWhat is digital humanities ,By: Professor Lili Saghafi
What is digital humanities ,By: Professor Lili SaghafiProfessor Lili Saghafi
 
Does DH Scholarship Take Place in the Lab?
Does DH Scholarship Take Place in the Lab?Does DH Scholarship Take Place in the Lab?
Does DH Scholarship Take Place in the Lab?Shawn Day
 
Learning Systems for Science
Learning Systems for ScienceLearning Systems for Science
Learning Systems for ScienceIan Foster
 
Research software susainability
Research software susainabilityResearch software susainability
Research software susainabilityDaniel S. Katz
 
Looking for Commonsense in the Semantic Web
Looking for Commonsense in the Semantic WebLooking for Commonsense in the Semantic Web
Looking for Commonsense in the Semantic WebValentina Presutti
 
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06jodischneider
 
MS-Presentation-new template arid university.pptx
MS-Presentation-new template arid university.pptxMS-Presentation-new template arid university.pptx
MS-Presentation-new template arid university.pptxNimraTariq69
 
Designing Useful and Usable Augmented Reality Experiences
Designing Useful and Usable Augmented Reality Experiences Designing Useful and Usable Augmented Reality Experiences
Designing Useful and Usable Augmented Reality Experiences Yan Xu
 
Leslie Johnston: Library Big Data Repository Services, Open Repositories 2012
Leslie Johnston: Library Big Data Repository Services, Open Repositories 2012Leslie Johnston: Library Big Data Repository Services, Open Repositories 2012
Leslie Johnston: Library Big Data Repository Services, Open Repositories 2012lljohnston
 
Detecting Incongruity Between News Headline and Body Text via a Deep Hierarch...
Detecting Incongruity Between News Headline and Body Text via a Deep Hierarch...Detecting Incongruity Between News Headline and Body Text via a Deep Hierarch...
Detecting Incongruity Between News Headline and Body Text via a Deep Hierarch...Seoul National University
 
Synthesizing knowledge from disagreement -cwi-2015-04-23
Synthesizing knowledge from disagreement -cwi-2015-04-23Synthesizing knowledge from disagreement -cwi-2015-04-23
Synthesizing knowledge from disagreement -cwi-2015-04-23jodischneider
 
Materials design using knowledge from millions of journal articles via natura...
Materials design using knowledge from millions of journal articles via natura...Materials design using knowledge from millions of journal articles via natura...
Materials design using knowledge from millions of journal articles via natura...Anubhav Jain
 
Applications of Natural Language Processing to Materials Design
Applications of Natural Language Processing to Materials DesignApplications of Natural Language Processing to Materials Design
Applications of Natural Language Processing to Materials DesignAnubhav Jain
 
Research Objects for FAIRer Science
Research Objects for FAIRer Science Research Objects for FAIRer Science
Research Objects for FAIRer Science Carole Goble
 
The Materials Project: Experiences from running a million computational scien...
The Materials Project: Experiences from running a million computational scien...The Materials Project: Experiences from running a million computational scien...
The Materials Project: Experiences from running a million computational scien...Anubhav Jain
 
Construction and Querying of Dynamic Knowledge Graphs
Construction and Querying of Dynamic Knowledge GraphsConstruction and Querying of Dynamic Knowledge Graphs
Construction and Querying of Dynamic Knowledge GraphsSutanay Choudhury
 
Building better knowledge graphs through social computing
Building better knowledge graphs through social computingBuilding better knowledge graphs through social computing
Building better knowledge graphs through social computingElena Simperl
 
Accelerating materials design through natural language processing
Accelerating materials design through natural language processingAccelerating materials design through natural language processing
Accelerating materials design through natural language processingAnubhav Jain
 
Accelerated Materials Discovery Using Theory, Optimization, and Natural Langu...
Accelerated Materials Discovery Using Theory, Optimization, and Natural Langu...Accelerated Materials Discovery Using Theory, Optimization, and Natural Langu...
Accelerated Materials Discovery Using Theory, Optimization, and Natural Langu...Anubhav Jain
 
Neuromarketing @ HEC Montreal-
Neuromarketing @ HEC Montreal-Neuromarketing @ HEC Montreal-
Neuromarketing @ HEC Montreal-Sylvain Senecal
 

Similar to Smart eBooks: ScentIndex and ScentHighlight research published at VAST2006 (20)

What is digital humanities ,By: Professor Lili Saghafi
What is digital humanities ,By: Professor Lili SaghafiWhat is digital humanities ,By: Professor Lili Saghafi
What is digital humanities ,By: Professor Lili Saghafi
 
Does DH Scholarship Take Place in the Lab?
Does DH Scholarship Take Place in the Lab?Does DH Scholarship Take Place in the Lab?
Does DH Scholarship Take Place in the Lab?
 
Learning Systems for Science
Learning Systems for ScienceLearning Systems for Science
Learning Systems for Science
 
Research software susainability
Research software susainabilityResearch software susainability
Research software susainability
 
Looking for Commonsense in the Semantic Web
Looking for Commonsense in the Semantic WebLooking for Commonsense in the Semantic Web
Looking for Commonsense in the Semantic Web
 
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
 
MS-Presentation-new template arid university.pptx
MS-Presentation-new template arid university.pptxMS-Presentation-new template arid university.pptx
MS-Presentation-new template arid university.pptx
 
Designing Useful and Usable Augmented Reality Experiences
Designing Useful and Usable Augmented Reality Experiences Designing Useful and Usable Augmented Reality Experiences
Designing Useful and Usable Augmented Reality Experiences
 
Leslie Johnston: Library Big Data Repository Services, Open Repositories 2012
Leslie Johnston: Library Big Data Repository Services, Open Repositories 2012Leslie Johnston: Library Big Data Repository Services, Open Repositories 2012
Leslie Johnston: Library Big Data Repository Services, Open Repositories 2012
 
Detecting Incongruity Between News Headline and Body Text via a Deep Hierarch...
Detecting Incongruity Between News Headline and Body Text via a Deep Hierarch...Detecting Incongruity Between News Headline and Body Text via a Deep Hierarch...
Detecting Incongruity Between News Headline and Body Text via a Deep Hierarch...
 
Synthesizing knowledge from disagreement -cwi-2015-04-23
Synthesizing knowledge from disagreement -cwi-2015-04-23Synthesizing knowledge from disagreement -cwi-2015-04-23
Synthesizing knowledge from disagreement -cwi-2015-04-23
 
Materials design using knowledge from millions of journal articles via natura...
Materials design using knowledge from millions of journal articles via natura...Materials design using knowledge from millions of journal articles via natura...
Materials design using knowledge from millions of journal articles via natura...
 
Applications of Natural Language Processing to Materials Design
Applications of Natural Language Processing to Materials DesignApplications of Natural Language Processing to Materials Design
Applications of Natural Language Processing to Materials Design
 
Research Objects for FAIRer Science
Research Objects for FAIRer Science Research Objects for FAIRer Science
Research Objects for FAIRer Science
 
The Materials Project: Experiences from running a million computational scien...
The Materials Project: Experiences from running a million computational scien...The Materials Project: Experiences from running a million computational scien...
The Materials Project: Experiences from running a million computational scien...
 
Construction and Querying of Dynamic Knowledge Graphs
Construction and Querying of Dynamic Knowledge GraphsConstruction and Querying of Dynamic Knowledge Graphs
Construction and Querying of Dynamic Knowledge Graphs
 
Building better knowledge graphs through social computing
Building better knowledge graphs through social computingBuilding better knowledge graphs through social computing
Building better knowledge graphs through social computing
 
Accelerating materials design through natural language processing
Accelerating materials design through natural language processingAccelerating materials design through natural language processing
Accelerating materials design through natural language processing
 
Accelerated Materials Discovery Using Theory, Optimization, and Natural Langu...
Accelerated Materials Discovery Using Theory, Optimization, and Natural Langu...Accelerated Materials Discovery Using Theory, Optimization, and Natural Langu...
Accelerated Materials Discovery Using Theory, Optimization, and Natural Langu...
 
Neuromarketing @ HEC Montreal-
Neuromarketing @ HEC Montreal-Neuromarketing @ HEC Montreal-
Neuromarketing @ HEC Montreal-
 

More from Ed Chi

2017 10-10 (netflix ml platform meetup) learning item and user representation...
2017 10-10 (netflix ml platform meetup) learning item and user representation...2017 10-10 (netflix ml platform meetup) learning item and user representation...
2017 10-10 (netflix ml platform meetup) learning item and user representation...Ed Chi
 
HCI Korea 2012 Keynote Talk on Model-Driven Research in Social Computing
HCI Korea 2012 Keynote Talk on Model-Driven Research in Social ComputingHCI Korea 2012 Keynote Talk on Model-Driven Research in Social Computing
HCI Korea 2012 Keynote Talk on Model-Driven Research in Social ComputingEd Chi
 
Location and Language in Social Media (Stanford Mobi Social Invited Talk)
Location and Language in Social Media (Stanford Mobi Social Invited Talk)Location and Language in Social Media (Stanford Mobi Social Invited Talk)
Location and Language in Social Media (Stanford Mobi Social Invited Talk)Ed Chi
 
Crowdsourcing using MTurk for HCI research
Crowdsourcing using MTurk for HCI researchCrowdsourcing using MTurk for HCI research
Crowdsourcing using MTurk for HCI researchEd Chi
 
CIKM 2011 Social Computing Industry Invited Talk
CIKM 2011 Social Computing Industry Invited TalkCIKM 2011 Social Computing Industry Invited Talk
CIKM 2011 Social Computing Industry Invited TalkEd Chi
 
WikiSym 2011 Closing Keynote
WikiSym 2011 Closing KeynoteWikiSym 2011 Closing Keynote
WikiSym 2011 Closing KeynoteEd Chi
 
CSCL 2011 Keynote on Social Computing and eLearning
CSCL 2011 Keynote on Social Computing and eLearningCSCL 2011 Keynote on Social Computing and eLearning
CSCL 2011 Keynote on Social Computing and eLearningEd Chi
 
Replication is more than Duplication: Position slides for CHI2011 panel on re...
Replication is more than Duplication: Position slides for CHI2011 panel on re...Replication is more than Duplication: Position slides for CHI2011 panel on re...
Replication is more than Duplication: Position slides for CHI2011 panel on re...Ed Chi
 
Tutorial on Using Amazon Mechanical Turk (MTurk) for HCI Research
Tutorial on Using Amazon Mechanical Turk (MTurk) for HCI ResearchTutorial on Using Amazon Mechanical Turk (MTurk) for HCI Research
Tutorial on Using Amazon Mechanical Turk (MTurk) for HCI ResearchEd Chi
 
Crowdsourcing for HCI Research with Amazon Mechanical Turk
Crowdsourcing for HCI Research with Amazon Mechanical TurkCrowdsourcing for HCI Research with Amazon Mechanical Turk
Crowdsourcing for HCI Research with Amazon Mechanical TurkEd Chi
 
Eddi: Topic Browsing of Twitter Streams
Eddi: Topic Browsing of Twitter StreamsEddi: Topic Browsing of Twitter Streams
Eddi: Topic Browsing of Twitter StreamsEd Chi
 
Large Scale Social Analytics on Wikipedia, Delicious, and Twitter (presented ...
Large Scale Social Analytics on Wikipedia, Delicious, and Twitter (presented ...Large Scale Social Analytics on Wikipedia, Delicious, and Twitter (presented ...
Large Scale Social Analytics on Wikipedia, Delicious, and Twitter (presented ...Ed Chi
 
Model-based Research in Human-Computer Interaction (HCI): Keynote at Mensch u...
Model-based Research in Human-Computer Interaction (HCI): Keynote at Mensch u...Model-based Research in Human-Computer Interaction (HCI): Keynote at Mensch u...
Model-based Research in Human-Computer Interaction (HCI): Keynote at Mensch u...Ed Chi
 
Zerozero88 Twitter URL Item Recommender
Zerozero88 Twitter URL Item RecommenderZerozero88 Twitter URL Item Recommender
Zerozero88 Twitter URL Item RecommenderEd Chi
 
Model-Driven Research in Social Computing
Model-Driven Research in Social ComputingModel-Driven Research in Social Computing
Model-Driven Research in Social ComputingEd Chi
 
ASC Disaster Response Proposal from Aug 2007
ASC Disaster Response Proposal from Aug 2007ASC Disaster Response Proposal from Aug 2007
ASC Disaster Response Proposal from Aug 2007Ed Chi
 
Using Information Scent to Model Users in Web1.0 and Web2.0
Using Information Scent to Model Users in Web1.0 and Web2.0Using Information Scent to Model Users in Web1.0 and Web2.0
Using Information Scent to Model Users in Web1.0 and Web2.0Ed Chi
 
China HCI Symposium 2010 March: Augmented Social Cognition Research from PARC...
China HCI Symposium 2010 March: Augmented Social Cognition Research from PARC...China HCI Symposium 2010 March: Augmented Social Cognition Research from PARC...
China HCI Symposium 2010 March: Augmented Social Cognition Research from PARC...Ed Chi
 
2010-03-10 PARC Augmented Social Cognition Research Overview
2010-03-10 PARC Augmented Social Cognition Research Overview2010-03-10 PARC Augmented Social Cognition Research Overview
2010-03-10 PARC Augmented Social Cognition Research OverviewEd Chi
 
2010-02-22 Wikipedia MTurk Research talk given in Taiwan's Academica Sinica
2010-02-22 Wikipedia MTurk Research talk given in Taiwan's Academica Sinica2010-02-22 Wikipedia MTurk Research talk given in Taiwan's Academica Sinica
2010-02-22 Wikipedia MTurk Research talk given in Taiwan's Academica SinicaEd Chi
 

More from Ed Chi (20)

2017 10-10 (netflix ml platform meetup) learning item and user representation...
2017 10-10 (netflix ml platform meetup) learning item and user representation...2017 10-10 (netflix ml platform meetup) learning item and user representation...
2017 10-10 (netflix ml platform meetup) learning item and user representation...
 
HCI Korea 2012 Keynote Talk on Model-Driven Research in Social Computing
HCI Korea 2012 Keynote Talk on Model-Driven Research in Social ComputingHCI Korea 2012 Keynote Talk on Model-Driven Research in Social Computing
HCI Korea 2012 Keynote Talk on Model-Driven Research in Social Computing
 
Location and Language in Social Media (Stanford Mobi Social Invited Talk)
Location and Language in Social Media (Stanford Mobi Social Invited Talk)Location and Language in Social Media (Stanford Mobi Social Invited Talk)
Location and Language in Social Media (Stanford Mobi Social Invited Talk)
 
Crowdsourcing using MTurk for HCI research
Crowdsourcing using MTurk for HCI researchCrowdsourcing using MTurk for HCI research
Crowdsourcing using MTurk for HCI research
 
CIKM 2011 Social Computing Industry Invited Talk
CIKM 2011 Social Computing Industry Invited TalkCIKM 2011 Social Computing Industry Invited Talk
CIKM 2011 Social Computing Industry Invited Talk
 
WikiSym 2011 Closing Keynote
WikiSym 2011 Closing KeynoteWikiSym 2011 Closing Keynote
WikiSym 2011 Closing Keynote
 
CSCL 2011 Keynote on Social Computing and eLearning
CSCL 2011 Keynote on Social Computing and eLearningCSCL 2011 Keynote on Social Computing and eLearning
CSCL 2011 Keynote on Social Computing and eLearning
 
Replication is more than Duplication: Position slides for CHI2011 panel on re...
Replication is more than Duplication: Position slides for CHI2011 panel on re...Replication is more than Duplication: Position slides for CHI2011 panel on re...
Replication is more than Duplication: Position slides for CHI2011 panel on re...
 
Tutorial on Using Amazon Mechanical Turk (MTurk) for HCI Research
Tutorial on Using Amazon Mechanical Turk (MTurk) for HCI ResearchTutorial on Using Amazon Mechanical Turk (MTurk) for HCI Research
Tutorial on Using Amazon Mechanical Turk (MTurk) for HCI Research
 
Crowdsourcing for HCI Research with Amazon Mechanical Turk
Crowdsourcing for HCI Research with Amazon Mechanical TurkCrowdsourcing for HCI Research with Amazon Mechanical Turk
Crowdsourcing for HCI Research with Amazon Mechanical Turk
 
Eddi: Topic Browsing of Twitter Streams
Eddi: Topic Browsing of Twitter StreamsEddi: Topic Browsing of Twitter Streams
Eddi: Topic Browsing of Twitter Streams
 
Large Scale Social Analytics on Wikipedia, Delicious, and Twitter (presented ...
Large Scale Social Analytics on Wikipedia, Delicious, and Twitter (presented ...Large Scale Social Analytics on Wikipedia, Delicious, and Twitter (presented ...
Large Scale Social Analytics on Wikipedia, Delicious, and Twitter (presented ...
 
Model-based Research in Human-Computer Interaction (HCI): Keynote at Mensch u...
Model-based Research in Human-Computer Interaction (HCI): Keynote at Mensch u...Model-based Research in Human-Computer Interaction (HCI): Keynote at Mensch u...
Model-based Research in Human-Computer Interaction (HCI): Keynote at Mensch u...
 
Zerozero88 Twitter URL Item Recommender
Zerozero88 Twitter URL Item RecommenderZerozero88 Twitter URL Item Recommender
Zerozero88 Twitter URL Item Recommender
 
Model-Driven Research in Social Computing
Model-Driven Research in Social ComputingModel-Driven Research in Social Computing
Model-Driven Research in Social Computing
 
ASC Disaster Response Proposal from Aug 2007
ASC Disaster Response Proposal from Aug 2007ASC Disaster Response Proposal from Aug 2007
ASC Disaster Response Proposal from Aug 2007
 
Using Information Scent to Model Users in Web1.0 and Web2.0
Using Information Scent to Model Users in Web1.0 and Web2.0Using Information Scent to Model Users in Web1.0 and Web2.0
Using Information Scent to Model Users in Web1.0 and Web2.0
 
China HCI Symposium 2010 March: Augmented Social Cognition Research from PARC...
China HCI Symposium 2010 March: Augmented Social Cognition Research from PARC...China HCI Symposium 2010 March: Augmented Social Cognition Research from PARC...
China HCI Symposium 2010 March: Augmented Social Cognition Research from PARC...
 
2010-03-10 PARC Augmented Social Cognition Research Overview
2010-03-10 PARC Augmented Social Cognition Research Overview2010-03-10 PARC Augmented Social Cognition Research Overview
2010-03-10 PARC Augmented Social Cognition Research Overview
 
2010-02-22 Wikipedia MTurk Research talk given in Taiwan's Academica Sinica
2010-02-22 Wikipedia MTurk Research talk given in Taiwan's Academica Sinica2010-02-22 Wikipedia MTurk Research talk given in Taiwan's Academica Sinica
2010-02-22 Wikipedia MTurk Research talk given in Taiwan's Academica Sinica
 

Recently uploaded

SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 

Recently uploaded (20)

SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 

Smart eBooks: ScentIndex and ScentHighlight research published at VAST2006

  • 1. Ed H. Chi, Lichan Hong, Julie Heiser, Stu Card Palo Alto Research Center The user study portion of this research has been funded in part by ARDA NIMD program. Ed H. Chi VAST 2006 1
  • 2. Ed H. Chi, Lichan Hong, Julie Heiser, Stu Card, Michelle Gumbrecht Palo Alto Research Center The user study portion of this research has been funded in part by ARDA NIMD program. Ed H. Chi VAST 2006 2
  • 3. Reading is an essential human activity. •  Giant leaps forward is marked by new and better ways to find, correlate, and comprehend information. Ed H. Chi VAST 2006 Copyright 2004 PARC 3
  • 4. Many books digitized in the (goal) digital library effort. 90% (current) •  Amazon, Google, and RECALL 75% CMU’s Million Book Project. Intelligent Analysts spend an enormous amount of time 0% Reading! [Pirolli, Lee, Card] 0 TIME Ed H. Chi VAST 2006 Copyright 2004 PARC 4
  • 5. Subject Indexes as a new reading “device” •  Invented in the 15th Century [Dewar98] •  Method of design refined thru centuries Peter Heylyn's 1652 Cosmographie in Four Bookes Ed H. Chi VAST 2006 Copyright 2004 PARC 5
  • 6. Instead of generating new indexes using IR techniques, why not enhance them? Take advantage of centuries of experience in building subject indexes. Ed H. Chi VAST 2006 Copyright 2004 PARC 6
  • 7. Ed H. Chi VAST 2006 Copyright 2004 PARC 7
  • 8. Readers need help in directing their attention to the most relevant passages to their topic of interest. Idea: conceptually highlight passages and keywords that are related to user search keywords. Ed H. Chi VAST 2006 Copyright 2004 PARC 8
  • 9. Conceptually highlight any relevant User first type search keywords: passages and keywords “anthrax symptoms” Draw user attention Ed H. Chi VAST 2006 Copyright 2004 PARC 9
  • 10. Biohazard •  by Ken Alibek •  non-fiction retelling of his experiences working on biological weapons in the former Soviet Union. •  13 index pages in two columns, consisting 829 entries Ed H. Chi VAST 2006 Copyright 2004 PARC 10
  • 11. Associated Entries underlined in red Exact Matches in red Ed H. Chi VAST 2006 Copyright 2004 PARC 11
  • 12. Ed H. Chi VAST 2006 Copyright 2004 PARC 12
  • 13. Ed H. Chi VAST 2006 Copyright 2004 PARC 13
  • 14. Ed H. Chi VAST 2006 Copyright 2004 PARC 14
  • 15. Ed H. Chi VAST 2006 Copyright 2004 PARC 15
  • 16. Goal: Compare how users find, compare, and comprehend information using the ScentIndex and 3Book versus the physical book. It’s not clear to us that the ScentIndex would be better, because: •  Unfamiliarity with 3Book Interface (page turning, clicking on page numbers, use of search box) •  Inability to grasp the ScentIndex concept (reorganization might be confusing, harder to read the index page on screen) •  Readability of the Screen (hard to read a large number of pieces of text) •  Users might be very good at using the physical book index. Ed H. Chi VAST 2006 Copyright 2004 PARC 16
  • 17. Study Design: •  Within-subject •  Interface condition (ScentIndex vs. Physical Book Index), and •  Task Type (find, compare, comprehend), •  with the order of the interface used and expertise level as the between-subject variables. Subjects: 16 subjects (8 experts on the content, 8 novices) Materials: subjects used PC with two LCD monitors, and the physical Alibek book. Ed H. Chi VAST 2006 Copyright 2004 PARC 17
  • 18. Overall, the ScentIndex eBook performed better against the physical Book. Faster Speed: •  Subjects using the ScentIndex were faster in completing their tasks no matter whether they were experts or novices, F(1,12)=12.96, p<.01. More Accurate: •  Answers that they provided while using ScentIndex interface were more accurate, F(1,12)=3.991, p=.06. Ed H. Chi VAST 2006 Copyright 2004 PARC 18
  • 19. Ed H. Chi VAST 2006 Copyright 2004 PARC 19
  • 20. The difficulty seems to be, not so much that we publish unduly in view of the extent and variety of present-day interests, but rather that publication has been extended far beyond our present ability to make real use of the record.” --- V. Bush Indeed, our goal is to enhance current Browsing Interfaces for more productive reading. Ed H. Chi VAST 2006 Copyright 2004 PARC 20
  • 21. The user study portion of this research has been funded in part by contract #MDA904-03-C-0404 to Stuart K. Card and Peter Pirolli from the Advanced Research and Development Activity, Novel Intelligence from Massive Data program. We thank Jock Mackinlay for some fruitful conversation about the interaction of the eBook; Michelle Gumbrecht and Tan Lee for running some of our experiments; Pam Desmond for help in the data analysis, and Brian Tramontana for the video production. Contact: Ed H. Chi (chi@acm.org) Ed H. Chi VAST 2006 Copyright 2004 PARC 21
  • 22. Ed H. Chi VAST 2006 Copyright 2004 PARC 22
  • 23. Ed H. Chi VAST 2006 Copyright 2004 PARC 23
  • 24. Ed H. Chi VAST 2006 Copyright 2004 PARC 24
  • 25. Ed H. Chi VAST 2006 Copyright 2004 PARC 25
  • 26. Page Textures sample Renderer scan Page Images Words + extract Locations OCR Word Association Text compute Matrix Sentence Scent parse Structure Highlights Ed H. Chi VAST 2006 Copyright 2004 PARC 26
  • 27. Ed H. Chi VAST 2006 Copyright 2004 PARC 27
  • 28. Early proposal of an indexing system: Memex [Bush45] Electronic Books: Rocket eBook, SoftBook Reader, DigiPaper [Huttenlocher00], DjVu [DjVu Zone03], PDF [Adobe03], MS Reader [MS03]. 3D Electronic Books: SGI Demo Book [SGI93], WebBook [Card96], British Library Turning the Pages [BL03], 3Book [Card03]. Computer help search systems such as Apple or Microsoft. Google or AltaVista provide highlighting and searching Automatic Indexing in IR: use noun-phrases and parsers to create indexes [Wacholder01, Nevill-Manning99]. Scatter/Gather [Cutting91]. Concept similar to: Word Co-occurrence [Schuetze99], InfoScent and Spreading Activation [Chi01, Chi00]. Ed H. Chi VAST 2006 Copyright 2004 PARC 28
  • 29. HyperText Book Systems: SuperBook [Remde87] provides a dynamic TOC with fisheye DOI. Ed H. Chi VAST 2006 Copyright 2004 PARC 29
  • 30. Two issues: •  M is calculated using a 40 word window •  Caveat: Exact word matches do not always show up. •  Solution: Insert large values onto the diagonal Ed H. Chi VAST 2006 Copyright 2004 PARC 30
  • 31. An experimenter without prior knowledge of how the ScentIndex system works devised a total of 12 tasks. The tasks were divided into two groups of six tasks each. •  Between the two sets of questions, half of the subjects received one set first; the other received the other set first. •  Tasks from one group were designed to be one-to-one equivalents of the other group. •  Of these six tasks, two were Simple Fact Retrieval questions, two were Dispersed Comparison questions, and two were Comprehension questions. Ed H. Chi VAST 2006 Copyright 2004 PARC 31
  • 32. Initial Survey (on computing and search experiences.) 4 expert and 4 novice subjects used the Book interface first, and the other eight used the ScentIndex first. Subjects were trained to use the ScentIndex immediately before they needed to use it. Each task was given on a separate sheet of paper. Read, understand each question completely before they start the task. •  time limit for each task (simple retrieval=2min, comparison=4min, comprehension=6min) with one minute warnings. •  For each interface, subjects performed the simple fact retrievals first, the dispersed comparisons second, and the comprehension questions last. Post Survey (comments, preferences). Ed H. Chi VAST 2006 Copyright 2004 PARC 32
  • 33. Simple Fact Retrieval: •  The last natural occurring case of WHICH virus occurred in Somalia in 1977? •  Who received a state award for developing a Q fever weapon? Dispersed Comparision: •  What is the death rate of smallpox and tularemia? Which virus has a higher death rate? •  What year did Russia open negotiations with Iraq for large fermentation vessels? What year did Vladimir Kryuchkov become chairman of the KGB? Which occurred first? Comprehension: •  Pasechnik’s defection to the West had grave implications for the Soviet biowarfare program. Match the person with the fact that describes how they’re involved: •  Persons: Frolov, Chernyayev, Karpov, Vinogradov •  Facts: A. First told Alibek about Pasechnik’s defection. B. Deputy minister who refused to sign formal diplomatic reply. C. Given demarche that said US have “new information”, presumably given by Pasechnik. D. Told American visitors that Pasechnik’s jetstream milling machine was for “salt”. Ed H. Chi VAST 2006 Copyright 2004 PARC 33
  • 34. Time to Completion 6 Participants using the ScentIndex interface performed ln(completion time) in seconds 5.5 tasks faster than those using the 5 Book, F(1,12)=12.96, p<.01. Many tasks not completed in 4.5 the time allotted using the Book interface. 4 Book •  6/7 for simple retrieval, ScentIndex 3.5 •  7/8 for comparison, •  3/5 for comprehension. 3 Simple Dispersed Comprehension Ed H. Chi VAST 2006 Copyright 2004 PARC 34
  • 35. Natural log transformation on the completion time 6 As predicted, experts performed ln(completion time) in seconds tasks faster than novices overall 5 5.435 •  (Expert Mean=4.85, S.D.=. 4 4.987 212, Novice Mean=4.58, S.D.=.212, F(1,12)=17.7, p<. 3 3.722 01.) 2 •  There were no interactions. Simple Retrieval < Dispersed 1 Comparison < Comprehension, F 0 (2,24)=204, p<.01. Sim ple Dispersed Com prehension Ed H. Chi VAST 2006 Copyright 2004 PARC 35
  • 36. Converted the scores for each task to a percentage. (measured in Simple Dispersed Comprehen points) Retrieval Comparison sion ScentIndex M=1.88 M=1.88 M=1.77 eBook S.D.=.342 S.D.=.269 S.D.=.284 Book M=1.75 M=1.58 M=1.84 SD=.447 S.D.=.516 S.D.=.259 We found that users performed better using the ScentIndex, reaching marginal significance F(1,12)=3.991, p=.06. We found no difference between experts and novices. Ed H. Chi VAST 2006 Copyright 2004 PARC 36
  • 37. Users overwhelmingly preferred the ScentIndex interface (15/16) •  “can search using keyword combinations” •  “clicking on page number to navigate” •  “highlighting enables faster scanning and skimming” •  “easier to compare index entries because it’s all on 1 page.” Some users mentioned that they prefer paper for extensive reading Potential Future work: •  Compare with search engines (organize results by relevancy). •  Understand difference between this technique and keyword finding. •  Limit the page number list of each relevant index entry to only the pages that are relevant to the keywords specified. Ed H. Chi VAST 2006 Copyright 2004 PARC 37