dr. Tanja MERČUN
prof. dr. Maja ŽUMER
University of Ljubljana, Slovenia
Faculty of Arts
Department of Library and Informat...
MOTIVATION & CONTEXT
Why is this topic important and relevant?
2
1. Current systems & their problems
• difficult to use, unintuitive, frustrating, time consuming
• finding tool: work best...
1. Current systems & their problems
• despite improvements toward better functionality and
search experience
– basic displ...
2. Changing information environment
• growing use of semantic web technologies
• connecting information: from data to know...
3. Towards a new generation
• outdated systems + new concepts, services, and evolving
technologies
 major transformation ...
FUTURE LIBRARY INFORMATION
SYSTEMS
design & evaluation
7
• user interface display, bibliographic data, system features
– what kind of display would be most effective
– what types ...
• FRBR = highly theoretical conceptual model & one of the
possible roads towards the „new generation“
potential:
− transfo...
• new concept of a user interface
• the need to involve users in the evaluation and the
design of future (FRBR-inspired) l...
FRBRVIS
case study
11
design
12
Objectives
1. propose a FRBR-based model that would improve
• the current presentation of bibliographic records
• navigati...
• interactive exploration of three main types of
relationships:
– editions and versions of the work
– works related to a g...
15
16
MM M M M M M M M
E E EE EEEE
W W W W W W
17
novel
literary criticism
TV documentary
W
W
W
W W
W
W
W
W
W
W W
W
W
WW
W
W
W
W
W
W
W
WW
W
W
W
W
W
W W
novel
motion picture...
W A
byW
W
W
W
W
W
W
WW
W
W
W
W
W
W W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
author about
W
W
W
W
biographies
TV
documen...
• information visualization
– interactive exploration of bibliographic data
– enables the display different structures
– v...
• 4 hierarchical layouts
Interface design
21
Interface design
22
evaluation
23
„is there a difference in performance and user satisfaction“
₋ classical : FRBR-visual approach
₋ 4 FRBR-visual designs
₋ ...
• required a baseline system for comparison
Research question
25
• 2 usability studies (December 1, 2011 – January 15, 2012)
• formal experiment : 120 subjects
• observational experiment ...
• evaluate and compare different prototype designs
• task based usability measures
₋ efficiency (task time & ease of use)
...
• insight into user‘s understanding of displays and navigation
• prototype exploration & comparison
₋ describing the syste...
results
29
Performance measures
EXPERIMENT 1. Task time, task success, optimal use of navigation, ease of use
30
Self-reported measures
LINEARLIST RADIAL CIRCLEPACK HIERARCHY SUNBURST
AVERAGE 2.08 1.82 1.74 1.56 1.63
0.00
1.00
2.00
3.0...
Self-reported measures
EXPERIMENT 1. Rank
0%
20%
40%
60%
80%
100%
LINEARLIST RADIAL CIRCLEPACK HIERARCHY SUNBURST
uservote...
Self-reported measures
0 10 20 30 40 50
organised
useful
quick to understand
complex
logical
efficient
clumsy
time consumi...
Self-reported measures
0 10 20 30 40 50 60
quick to understand
innovative
easy to use
logical
organised
transparent
intere...
• FrbrVis layouts: results similar to formal experiment
• Basline vs. FrbrVis: baseline perceived more favourably than
in ...
• users could intuitively and successfully work with FrbrVis
prototype
• feedback on different layouts: indication on whic...
Conclusions
37
• to establish what works and what does not
• to understand how people percieve the presented
information
• to see how the...
• cooperation between academics and practitioners:
– academic environment:
• analyse current problems, foresee future need...
• vision of what we wish to achieve – the „big picture“
• leading instead of following the changes in the
information land...
tanja.mercun@ff.uni-lj.si
thank you
41
Upcoming SlideShare
Loading in …5
×

Towards designing and evaluating future library information systems example of FrbrVis prototype

832 views

Published on

Presented at i3 conference -27th June 2013 in Aberdeen

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
832
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Towards designing and evaluating future library information systems example of FrbrVis prototype

  1. 1. dr. Tanja MERČUN prof. dr. Maja ŽUMER University of Ljubljana, Slovenia Faculty of Arts Department of Library and Information Science and Book Studies Towards designing and evaluating future library information systems example of FrbrVis prototype I3 conference, 27th June 2013
  2. 2. MOTIVATION & CONTEXT Why is this topic important and relevant? 2
  3. 3. 1. Current systems & their problems • difficult to use, unintuitive, frustrating, time consuming • finding tool: work best for known-item search • bypassed by users 3
  4. 4. 1. Current systems & their problems • despite improvements toward better functionality and search experience – basic display of records and bibliographic information remains unchanged – some of the main objectives still unfulfilled 4
  5. 5. 2. Changing information environment • growing use of semantic web technologies • connecting information: from data to knowledge • new cataloguing rules based on entity-relationship model FRBR (Functional Requirements for Bibliographic Records) 5
  6. 6. 3. Towards a new generation • outdated systems + new concepts, services, and evolving technologies  major transformation of bibliographic information systems in the near future 6
  7. 7. FUTURE LIBRARY INFORMATION SYSTEMS design & evaluation 7
  8. 8. • user interface display, bibliographic data, system features – what kind of display would be most effective – what types of questions should they be able to answer – how will they move beyond known-item search and support exploration – what attributes & relationships are needed at what points – … Questions to be answered 8
  9. 9. • FRBR = highly theoretical conceptual model & one of the possible roads towards the „new generation“ potential: − transforming linear lists of records by grouping and structuring data problem: − does not prescribe how its implementation should look like FRBR generation user interfaces? 9
  10. 10. • new concept of a user interface • the need to involve users in the evaluation and the design of future (FRBR-inspired) library information system displays FRBR generation user interfaces? 10
  11. 11. FRBRVIS case study 11
  12. 12. design 12
  13. 13. Objectives 1. propose a FRBR-based model that would improve • the current presentation of bibliographic records • navigation within bibliographic information systems 2. create an interface design for implementing the model 3. evaluate the usability of the model and interface design 13
  14. 14. • interactive exploration of three main types of relationships: – editions and versions of the work – works related to a given work – works by/about the author 1st increment: work family display 14
  15. 15. 15
  16. 16. 16
  17. 17. MM M M M M M M M E E EE EEEE W W W W W W 17
  18. 18. novel literary criticism TV documentary W W W W W W W W W W W W W W WW W W W W W W W WW W W W W W W W novel motion picture musical picture book play illustrations foreward 18
  19. 19. W A byW W W W W W W WW W W W W W W W W W W W W W W W W W W W W W W W W W W W W author about W W W W biographies TV documentariesW W reference works literary criticism W W W novels fairy tales plays poems poems children‘s stories essays 19
  20. 20. • information visualization – interactive exploration of bibliographic data – enables the display different structures – various presentation possibilities – quick overviews and efficient interaction with large data sets Interface design 20
  21. 21. • 4 hierarchical layouts Interface design 21
  22. 22. Interface design 22
  23. 23. evaluation 23
  24. 24. „is there a difference in performance and user satisfaction“ ₋ classical : FRBR-visual approach ₋ 4 FRBR-visual designs ₋ different dataset complexities Research question 24
  25. 25. • required a baseline system for comparison Research question 25
  26. 26. • 2 usability studies (December 1, 2011 – January 15, 2012) • formal experiment : 120 subjects • observational experiment : 32 subjects • participants • volunteers: undergraduate and graduate students • 65% humanities, social sciences & education • 35% science and engineering Study design 26
  27. 27. • evaluate and compare different prototype designs • task based usability measures ₋ efficiency (task time & ease of use) ₋ effectiveness (task success, use of navigation,...) ₋ satisfaction (rank, reaction cards) • design: – participant  3 testing conditions  10 tasks per condition – counterbalancing: layouts, work families, displays, tasks Formal experiment 27
  28. 28. • insight into user‘s understanding of displays and navigation • prototype exploration & comparison ₋ describing the system and what they learned ₋ comparing the different approaches • design: – participant  2 conditions  baseline + FRBR-visual design Observational experiment 28
  29. 29. results 29
  30. 30. Performance measures EXPERIMENT 1. Task time, task success, optimal use of navigation, ease of use 30
  31. 31. Self-reported measures LINEARLIST RADIAL CIRCLEPACK HIERARCHY SUNBURST AVERAGE 2.08 1.82 1.74 1.56 1.63 0.00 1.00 2.00 3.00 averagetaskdifficulty (1-veryeasy,5-verydifficult) EXPERIMENT 1. Task difficulty BASELINE 31
  32. 32. Self-reported measures EXPERIMENT 1. Rank 0% 20% 40% 60% 80% 100% LINEARLIST RADIAL CIRCLEPACK HIERARCHY SUNBURST uservotesby% 3rd 2nd 1st BASELINE 32
  33. 33. Self-reported measures 0 10 20 30 40 50 organised useful quick to understand complex logical efficient clumsy time consuming easy to use transparent convenient non-transparent appealing interesting advanced innovative baseline 0 10 20 30 40 50 informative innovative interesting non-transparent advanced easy to use convenient attractive appealing radial 0 10 20 30 40 50 quick to understand innovative useful easy to use informative advanced efficient attractive non-transparent circlepack EXPERIMENT 1. Reaction cards 33
  34. 34. Self-reported measures 0 10 20 30 40 50 60 quick to understand innovative easy to use logical organised transparent interesting informative advanced useful convenient fun appealing attractive efficient complex sunburst 0 10 20 30 40 50 60 organised quick to understand informative easy to use transparent logical useful convenient efficient appealing innovative interesting advanced attractive hierarchy EXPERIMENT 1. Reaction cards 34
  35. 35. • FrbrVis layouts: results similar to formal experiment • Basline vs. FrbrVis: baseline perceived more favourably than in formal experiment • participants felt they got a better overview of items and found out more about the author uşing FrbrVis, especially when working with more complex examples Self-reported measures EXPERIMENT 2. Perception of the prototypes 35
  36. 36. • users could intuitively and successfully work with FrbrVis prototype • feedback on different layouts: indication on which should be further developed in the future • identification of usability issues • ideas for new, additional features • ideas for further studies (individual differences, etc.) Outcome 36
  37. 37. Conclusions 37
  38. 38. • to establish what works and what does not • to understand how people percieve the presented information • to see how they interact with information • to identify possible issues and needs Designing and evaluating with users 38
  39. 39. • cooperation between academics and practitioners: – academic environment: • analyse current problems, foresee future needs, developments • as the driving force of ideas for the future • knowledge and resources to test different ideas, solutions – practitioners: • lack of time for research • need verified guidelines to work with • have the power to put the theory into practice Designing and evaluating in cooperation 39
  40. 40. • vision of what we wish to achieve – the „big picture“ • leading instead of following the changes in the information landscape Designing and evaluating with vision 40
  41. 41. tanja.mercun@ff.uni-lj.si thank you 41

×