User research for thedevelopment of search systems        Max Kemman MSc        Martijn Kleppe MA              ESHCC      ...
www.axes-project.eu
www.axes-project.eu
How do we search for images?       By using words! (your own & someone else’s)     New possibilities:Speech- & Image reco...
Developing tools providing    new engaging ways to interact    with audiovisual libraries…Not just search:              Br...
... for a multitude of end users...           Broadcast professionals           Academics           Home users            ...
www.axes-project.eu
www.axes-project.eu
PoliMedia research questionWhat choices do different media make in the coverageof people and topics while reporting on deb...
Issues with current approach       Media analysis      +              =   Too much                           work         ...
PoliMedia approach PoliMedia                            Newspapers   Portal                               KB              ...
Presentation UB - 11-12-2012                               www.axes-project.eu
Our research agenda• Possibilities of eHumanities for research and  education• How and what do people search in our digita...
User research• Understanding the user  – Acceptance  – Performance  – Capabilities  – Weaknesses• Goal  – Creating systems...
Who are the users?• We can distinguish groups, using  classifications of  – Roles (e.g., by occupation)  – Demographic cha...
Getting to know the users• Prior research  – StatCounter, other literature• Asking potential users  – Interviews  – Focus ...
The more, the merrier?                   PhD student, History, 24       Presentation UB - 11-12-2012                      ...
The more, the merrier?“If there is an easier way, I willdo it another way. So I won’tgo on the Internet to search.There’s ...
Academic user research• Interviews• Survey  – Netherlands and Belgium  – Online  – N=294  – 15-20 minutes  – Lots of data ...
Who did the survey?• Position   – Largest group: PhD student   – But good distribution over     other positions• Age   – L...
Research questions1. To what extent are online databases used?2. Which subdisciplines use digital sources   more and which...
Which of the following digital data do          you use professionally?Very often    Often                                ...
Differences between disciplines‘Traditional’ digital data                        Modern digital dataImages: Literature, Hi...
Don’t                                                                        Sometimes                                    ...
What about sEURch?• 31 respondents from EUR                 Presentation UB - 11-12-2012                                  ...
When do you trust a search engine or                                    database?                         250Number of par...
Research questions1. To what extent are online databases used?   – Mostly text-based data usage, followed by images   – Go...
Search techniques                             1. Keywords 4,75                             2. Advanced search 3,36        ...
(Audiovisual) search behaviour• Academics are “surfers” with a non-specific  goal in mind• Academics are positively confid...
Research questions1. To what extent are online databases used?2. Which subdisciplines use digital sources   more and which...
Survey conclusion• We see that  – Text is the dominant medium  – Google is the dominant search system  – Google dominates ...
State of affairs?    Presentation UB - 11-12-2012                                   www.axes-project.eu
Consequences• How do we use this information in our search  system development?• How to make new search systems desirable?...
Desirable?             PhD student, History, 34 Presentation UB - 11-12-2012                                www.axes-proje...
Influence of usability“I think it’s veryimportant to have a goodand workable interfacefor these objects. That’swhy I used ...
Consequences• Making new search systems desirable by  incorporating user-centred design• Basing system design on findings ...
AXES search wireframe                                     • Clear and                                       immediate     ...
PoliMedia search wireframe                                        • Clear and                                          imm...
AXES browsing wireframe                                      • Keyword-search                                        remai...
PoliMedia browsing wireframe                                  • Keywords-search                                    remains...
Consequences• Making new search systems desirable by  incorporating user-centred design• Basing system design on findings ...
Thank you for you attention!          Martijn Kleppe       kleppe@eshcc.eur.nl              L2-47          Max Kemman     ...
Upcoming SlideShare
Loading in …5
×

User research for the development of search systems

521 views
470 views

Published on

Presentation at Erasmus University Library 11-12-2012.
For the most part a combination of slides from previous presentations, mostly from http://www.slideshare.net/MaxKemman/mapping-the-use-of-digital-sources-amongst-humanities-scholars-in-the-netherlands

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
521
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
1
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Limited: not everything is in it, but more importantly no mark-up or pages
  • User research for the development of search systems

    1. 1. User research for thedevelopment of search systems Max Kemman MSc Martijn Kleppe MA ESHCC www.axes-project.eu
    2. 2. www.axes-project.eu
    3. 3. www.axes-project.eu
    4. 4. How do we search for images?  By using words! (your own & someone else’s) New possibilities:Speech- & Image recognition AXES:Access to Audiovisual Archives www.axes-project.eu
    5. 5. Developing tools providing new engaging ways to interact with audiovisual libraries…Not just search: Browse Explore Experience www.axes-project.eu
    6. 6. ... for a multitude of end users... Broadcast professionals Academics Home users www.axes-project.eu
    7. 7. www.axes-project.eu
    8. 8. www.axes-project.eu
    9. 9. PoliMedia research questionWhat choices do different media make in the coverageof people and topics while reporting on debates in theDutch parliament since the first televised evening news in 1956 until 1995? www.axes-project.eu
    10. 10. Issues with current approach Media analysis + = Too much work Limited material + = and different systems www.axes-project.eu
    11. 11. PoliMedia approach PoliMedia Newspapers Portal KB Staten- Browse: Generaal Television debate and Digitaal Sound and Vision date KB- Search: debate and Radio person KB www.axes-project.eu
    12. 12. Presentation UB - 11-12-2012 www.axes-project.eu
    13. 13. Our research agenda• Possibilities of eHumanities for research and education• How and what do people search in our digital heritage collections? – Requirements – Testing Presentation UB - 11-12-2012 www.axes-project.eu
    14. 14. User research• Understanding the user – Acceptance – Performance – Capabilities – Weaknesses• Goal – Creating systems that are intuitive and helpful to the users www.axes-project.eu
    15. 15. Who are the users?• We can distinguish groups, using classifications of – Roles (e.g., by occupation) – Demographic characteristics (e.g., age, gender)• In AXES, we distinguish by role – Broadcast Professional – Academics (teachers, researchers, students) – Home users www.axes-project.eu
    16. 16. Getting to know the users• Prior research – StatCounter, other literature• Asking potential users – Interviews – Focus groups – Surveys• Experiments www.axes-project.eu
    17. 17. The more, the merrier? PhD student, History, 24 Presentation UB - 11-12-2012 www.axes-project.eu
    18. 18. The more, the merrier?“If there is an easier way, I willdo it another way. So I won’tgo on the Internet to search.There’s so much, there is somuch information. So you canbetter call someone whoknows than search it foryourself.” Presentation UB - 11-12-2012 www.axes-project.eu
    19. 19. Academic user research• Interviews• Survey – Netherlands and Belgium – Online – N=294 – 15-20 minutes – Lots of data (250 variables in SPSS) Presentation UB - 11-12-2012 www.axes-project.eu
    20. 20. Who did the survey?• Position – Largest group: PhD student – But good distribution over other positions• Age – Largest group: 25-34 – But good distribution over other age groups• Discipline – Largest group: History – Other disciplines: Social Studies, Mass Communications, Linguistics, Literature, Philosophy Presentation UB - 11-12-2012 www.axes-project.eu
    21. 21. Research questions1. To what extent are online databases used?2. Which subdisciplines use digital sources more and which less?3. Which search techniques are applied? Presentation UB - 11-12-2012 www.axes-project.eu
    22. 22. Which of the following digital data do you use professionally?Very often Often Text (books, news, etc.) Scholarly publications Regularly Statistical dataSometimes Numerical Digitized objects Images Never Audio Video data Presentation UB - 11-12-2012 www.axes-project.eu
    23. 23. Differences between disciplines‘Traditional’ digital data Modern digital dataImages: Literature, History Video: Mass CommunicationsDigitized objects: Literature, History, Audio: LinguisticsPhilosophy Numerical data: Social Studies, Linguistics Statistical data: Social Studies, Mass Communications, LinguisticsOverall: Literature, History, Philosophy Overall: Mass Communications, Linguistics, Social Studies Presentation UB - 11-12-2012 www.axes-project.eu
    24. 24. Don’t Sometimes Very often Often Regularly Never know it Google Google Images Google Scholar YouTube JSTOR Uitzending Gemist KB Flickr EBSCO Presentation UB - 11-12-2012 Nationaal Archief Web of Knowledge Yahoo! Bing Academia.nl or databases do you use? Europeana Scopus Microsoft Academic Search Which of the following search engines EUscreenwww.axes-project.eu Arkyves
    25. 25. What about sEURch?• 31 respondents from EUR Presentation UB - 11-12-2012 www.axes-project.eu
    26. 26. When do you trust a search engine or database? 250Number of participants 200 150 100 50 0 Experienced Expertise High quality Understand Heard about Broad range Read about it behind it selection its inner it of results it online workings Presentation UB - 11-12-2012 www.axes-project.eu
    27. 27. Research questions1. To what extent are online databases used? – Mostly text-based data usage, followed by images – Google dominant in every way – Trust is based mostly on experience2. Which subdisciplines use digital sources more and which less? – ‘Traditional’ digital data (Images, Digitized objects): • Literature, History and Philosophy – ‘Modern’ digital data (Video, Audio, Statistical and Numerical data) • Social studies, Mass Communications, Linguistics3. Which search techniques are applied? Presentation UB - 11-12-2012 www.axes-project.eu
    28. 28. Search techniques 1. Keywords 4,75 2. Advanced search 3,36 3. Related terms 2,52 4. Boolean 2,42 5. Browsing subject categories 2,29 6. Filters 2,19 7. Thesaurus 1,87 8. Visualization 1,22 Presentation UB - 11-12-2012 www.axes-project.eu
    29. 29. (Audiovisual) search behaviour• Academics are “surfers” with a non-specific goal in mind• Academics are positively confident in their ability to use search tools – Respondents below 45 years are more confident than those above 45 years Presentation UB - 11-12-2012 www.axes-project.eu
    30. 30. Research questions1. To what extent are online databases used?2. Which subdisciplines use digital sources more and which less?3. Which search techniques are applied? – Most important are ease and speed – Search behaviour: younger academics (below 45 years) are more confident in their use of (audiovisual) search tools Presentation UB - 11-12-2012 www.axes-project.eu
    31. 31. Survey conclusion• We see that – Text is the dominant medium – Google is the dominant search system – Google dominates search techniques – Trust in a search engine or database is based primarily on experience Presentation UB - 11-12-2012 www.axes-project.eu
    32. 32. State of affairs? Presentation UB - 11-12-2012 www.axes-project.eu
    33. 33. Consequences• How do we use this information in our search system development?• How to make new search systems desirable? Presentation UB - 11-12-2012 www.axes-project.eu
    34. 34. Desirable? PhD student, History, 34 Presentation UB - 11-12-2012 www.axes-project.eu
    35. 35. Influence of usability“I think it’s veryimportant to have a goodand workable interfacefor these objects. That’swhy I used a papercatalogue, because theinterface and the way tosearch these objects isnot ideal” Presentation UB - 11-12-2012 www.axes-project.eu
    36. 36. Consequences• Making new search systems desirable by incorporating user-centred design• Basing system design on findings from user studies Presentation UB - 11-12-2012 www.axes-project.eu
    37. 37. AXES search wireframe • Clear and immediate keyword-search • Separate advanced-search • Support for Booleans Presentation UB - 11-12-2012 www.axes-project.eu
    38. 38. PoliMedia search wireframe • Clear and immediate keyword-search • Separate advanced-search • Support for Booleans and (some) Google- search operators Presentation UB - 11-12-2012 www.axes-project.eu
    39. 39. AXES browsing wireframe • Keyword-search remains prominent • Engaging user by:  Suggesting query terms  Suggesting related videos Presentation UB - 11-12-2012 www.axes-project.eu
    40. 40. PoliMedia browsing wireframe • Keywords-search remains prominent • Easy access to related material Presentation UB - 11-12-2012 www.axes-project.eu
    41. 41. Consequences• Making new search systems desirable by incorporating user-centred design• Basing system design on findings from user studies• Relevancy for UB? Presentation UB - 11-12-2012 www.axes-project.eu
    42. 42. Thank you for you attention! Martijn Kleppe kleppe@eshcc.eur.nl L2-47 Max Kemman kemman@eshcc.eur.nl L2-47 www.axes-project.eu www.polimedia.nl Presentation UB - 11-12-2012 www.axes-project.eu

    ×