0
Student’s World: Photo Diary Study




           Nicole Hennig
    Web Manager & Usability Specialist

  Computers in Lib...
www.hennigweb.com/presentations/cil2007/




                                           2
Outline
• why we did it
• methodology
• findings
• what we plan to do because of it
• further resources


                 ...
you hear all these theories:



            “students are using google”

         “they aren’t coming to libraries”

“they...
We wanted to find out what our students REALLY do.




                                                    5
So we studied their information-seeking
   habits in the context of real life.




                                       ...
In the past we’ve done usability tests
     of our systems, one by one.




                                         7
8
But that only tells us how each system works or doesn’t
 work for them, once they’ve found it and are using it.




      ...
usability testing is good for:
   fixing an existing system



contextual research is good for:
    deciding what to build
...
So we turn to...
 anthropology




                   11
An anthropologist on staff




         Thanks, University of Rochester!



                                            12
If you can’t follow the users around all day in their
environment, instead you can give them a “cultural probe.”




     ...
14
We called it the “photo/diary study.”




                                        15
students volunteered to track
their information-seeking behavior
           for one week




                             ...
in-depth interviews




                      17
16 undergrads + 16 grad students




                                   18
19
20
21
22
students from:

 1    art & architecture
14    engineering
 6    science
 5    school of humanities & social sciences
 3  ...
Their photos, screen shots, and a diary of what they did
            helped them tell us the story.




                  ...
the interviews




                 25
a total of 8 librarians
      teams of 2 people

- one to conduct the interview
      - one to take notes




            ...
warm up questions

1. what department are you in?

2. how long have you been at MIT?

3. about how many times a month do y...
they tell the story of their week

               (about an hour)

use the photos and diary to jog their memories




    ...
questions we used to guide the interviews

     what were you looking for?
  where were you doing your research?
      whe...
questions we used to guide the interviews

    what devices did you use?
          what worked?
   what problems did you h...
we repeated back what we heard along the way:


    “let me get this right, you’re saying that....”

          “so in othe...
useful book for interviewing techniques:




Understanding Your Users: A Practical Guide to User
  Requirements Methods, T...
results




          33
Everyone was busy
     and sleep-deprived.

    Especially undergrads.




Students rarely asked for help.




           ...
goals, tasks, methods




                        35
we used cards to write down each
method used for each task they were working on




                                      ...
we wanted to understand their goals
so that we could find ways to help them meet those goals




                          ...
grad student goals
- research & thesis 64%
- conference presentations & publishing   14%
- current awareness     9%

- hel...
undergrad student goals
- complete their course work         75%
- research 7%
- current awareness 7%


- participate in M...
tasks - most frequent

- search for a known item

- search for information on a topic

- find facts

- search for a partial...
tasks - most frequent

- complete class assignment

- check web sites or email or RSS feeds for current
awareness

- study...
methods - most frequent

- search Google
- go directly to a known URL
- use the library catalog to browse or search
- sear...
methods - most frequent
- read textbooks
- consult with other students
- consult with faculty or guest lecturers
- search ...
none of that is surprising.

so we asked ourselves...




                              44
- were they successful from their own
points of view?


- were they efficient and effective from
our point of view as libra...
- were they successful from their own
points of view?

grads:      yes, for 86% of their tasks
undergrads: yes      93%


...
- were they efficient and effective from
our point of view as librarians?


grads:      yes for 77% of their tasks
undergra...
For all tasks

                      successful          efficient
                 (in their own view)   (in our view)


 ...
- you might say... well librarians will
always think of better ways

  but...




                                        ...
- we also broke it down by type of task
and that’s where we saw a big difference




                                     ...
Searching for information on a topic

                      successful          efficient
                 (in their own vi...
Searching for information on a topic


This is an area where we can help.




                                       52
They had a tendency to start with
 sources they were familiar with.




                                    53
54
and with sources recommended by
   a trusted network of people.




                                  55
advisors

              colleagues
friends

             family

      roommates




                           56
not librarians




                 57
also common:

- figuring it out themselves




                              58
that’s the culture at MIT




                            59
This is another area where we can help.

          Help them make connections
Include our expert librarians in those conne...
They used a wide variety of sources.




                                       61
Google        e-books           Google Print



               Google Scholar
textbooks
                                Go...
Google            e-books             Google Print

         library databases
                    Google Scholar
textbook...
Google            e-books             Google Print

         library databases           old course notes
                ...
usually began with Google.




                             65
Almost everyone had a few favorite resources.




                                                66
They tended to reuse their favorites,
     rather than try new ones.




                                        67
Many students did their TOPICAL discovery
          in non-library sources.




                                          ...
then came to the library to look up the items they found.




                                                        69
Looking up known items in our systems
         usually worked well.




                                        70
71
Undergrads mainly looked for
information related to courses they were taking.




                                        ...
Often a few good sources were enough.




                                        73
In those cases, people get what’s convenient to get.




                                                       74
Graduate students
looked for information related to their research.




                                                  ...
- more difficult and time-consuming.

          - highly-specific topics.

- needed more depth and comprehensiveness.




  ...
1. trouble with knowing where to look


2. trouble with effectively searching the
 sources they used




                 ...
They spent a large amount of time
 with varying degrees of success.




                                    78
Sometimes used “brute force” methods.

        VERY time consuming




                                        79
from our point of view as librarians

“if they had only known about X!”

         (fill in the blank)




                 ...
from their point of view:

They often thought that’s how it had to be.




                                              81
Another area where we can help.




                                  82
Also:

They used a wide variety of methods
  for “personal info management.”

  i.e., organizing what they found




     ...
They spend a lot of time
doing this (in creative ways).




                                 84
Most students are suffering from information overload.




                                                         85
Refworks, Endnote, Zotero,
  CiteULike, Connotea....

 a big interest in this area.




                                86
By the way:

We conducted a large library survey this past year
        with a very high response rate.




              ...
users want us to simplify search,
        felt there were
    too many starting points




                               ...
Too many starting points




                           89
Too many starting points




                           90
Too many starting points




                           91
Too many starting points




                           92
Another finding:

They are not aware of many of the services we provide
              beyond the obvious ones.




        ...
A picture of their culture

- sleep deprived
- stressed, in a hurry
- do it yourself
- focused on getting course work done...
A picture of their culture

- not asking for help
- not knowing about all our services
- not knowing about all our researc...
so what are we going to do?




                              96
Priorities
• Make topical discovery easier and more
  effective.
• Incorporate community & trust
  features in our systems...
Faceted browsing improves discovery




                                      98
We’re looking at different solutions for a new catalog
         interface using faceted browsing:

                       ...
Not just our catalog, but also to include:

          - DSpace repository
           - Image collections
      - Future ar...
nearly 500 research databases




 aggregated search of our                       federated search of
local content with f...
And we’re building a better way for students to
       discover our research databases
(almost 500 of them), using federat...
community and trust features




                               103
So many community features available now:

- Social bookmarking
- Tagging
- Comments
- Reviews
- Rating
- Popularity ranki...
We want more of these features in our systems




                                                105
We’d like a
system where the
 MIT community
  can contribute
     reviews




                   106
We’d like a multi-university tagging sytem.

      (PennTags.. across academia).




                                     ...
putting ourselves where users are




                                    108
Where are the users?

- Google
- Google Scholar
- Google Print
- Wikipedia
- Amazon
- iTunes
- Facebook
- Stellar (course ...
We are already embedded in Google Scholar.




                                             110
111
112
Browser extensions


•   LibX - Firefox extension
    libraries.mit.edu/libx




                               113
114
LibX
adds our search to the browser toolbar




                                         115
Puts links to the libraries where the users
are:

-   such as Amazon, Barnes & Noble, Powell’s
-   New York Times book rev...
Links from Amazon to our catalog




                                   cue




                                         1...
Links from Amazon to our catalog




                                   118
LibX




links ISBNs to
  our catalog




Without LibX, the ISBN is not a link.

                                        1...
LibX

  right-click
sends selected
text on a page
      to
Google Scholar
  and more




                 120
libx.org




           121
Other ways to put ourselves where they are

- our news blog (Wordpress)

- RSS feeds for the blog and for new titles in ou...
We now have a “betas” page

... similar to Google Labs




                             123
libraries.mit.edu/betas




                          124
labs.google.com




                  125
126
127
Successful systems extend the users’ work practice.

                 - Karen Holzblatt




                              ...
Design works best when it models user behavior.
  -Joshua Porter




                                                  129
We should look for ways to understand and
    extend the practice of our users.




                                      ...
Useful resources

Understanding your users: a practical guide to user
requirements methods, tools, and techniques.
Morgan ...
www.hennigweb.com/presentations/cil2007/




                                           132
questions?




             133
Key reports with similar findings
     & recommendations to ours

Rethinking how we provide bibliographic services for the...
Upcoming SlideShare
Loading in...5
×

Student's World: Photo Diary Study

1,457

Published on

2006 ethnographic study of student information-seeking at MIT.

http://www.hennigweb.com/presentations/cil2007/index.html

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,457
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Student's World: Photo Diary Study"

  1. 1. Student’s World: Photo Diary Study Nicole Hennig Web Manager & Usability Specialist Computers in Libraries - April 16, 2007 1
  2. 2. www.hennigweb.com/presentations/cil2007/ 2
  3. 3. Outline • why we did it • methodology • findings • what we plan to do because of it • further resources 3
  4. 4. you hear all these theories: “students are using google” “they aren’t coming to libraries” “they use wikipedia, instead of scholarly resources” 4
  5. 5. We wanted to find out what our students REALLY do. 5
  6. 6. So we studied their information-seeking habits in the context of real life. 6
  7. 7. In the past we’ve done usability tests of our systems, one by one. 7
  8. 8. 8
  9. 9. But that only tells us how each system works or doesn’t work for them, once they’ve found it and are using it. 9
  10. 10. usability testing is good for: fixing an existing system contextual research is good for: deciding what to build 10
  11. 11. So we turn to... anthropology 11
  12. 12. An anthropologist on staff Thanks, University of Rochester! 12
  13. 13. If you can’t follow the users around all day in their environment, instead you can give them a “cultural probe.” 13
  14. 14. 14
  15. 15. We called it the “photo/diary study.” 15
  16. 16. students volunteered to track their information-seeking behavior for one week 16
  17. 17. in-depth interviews 17
  18. 18. 16 undergrads + 16 grad students 18
  19. 19. 19
  20. 20. 20
  21. 21. 21
  22. 22. 22
  23. 23. students from: 1 art & architecture 14 engineering 6 science 5 school of humanities & social sciences 3 management 3 undecided (roughly like the population at MIT) 23
  24. 24. Their photos, screen shots, and a diary of what they did helped them tell us the story. 24
  25. 25. the interviews 25
  26. 26. a total of 8 librarians teams of 2 people - one to conduct the interview - one to take notes 26
  27. 27. warm up questions 1. what department are you in? 2. how long have you been at MIT? 3. about how many times a month do you use the libraries in person? 4. about how many times a month do you use our electronic resources? 5. about how many times a month do you search for information NOT using the MIT libraries? 27
  28. 28. they tell the story of their week (about an hour) use the photos and diary to jog their memories 28
  29. 29. questions we used to guide the interviews what were you looking for? where were you doing your research? when were you doing this? what strategies did you use? what sources did you use? how did you learn about these sources? 29
  30. 30. questions we used to guide the interviews what devices did you use? what worked? what problems did you have? how often do you usually do this task? 30
  31. 31. we repeated back what we heard along the way: “let me get this right, you’re saying that....” “so in other words you did x?” 31
  32. 32. useful book for interviewing techniques: Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and Techniques Catherine Courage & Kathy Baxter 32
  33. 33. results 33
  34. 34. Everyone was busy and sleep-deprived. Especially undergrads. Students rarely asked for help. 34
  35. 35. goals, tasks, methods 35
  36. 36. we used cards to write down each method used for each task they were working on 36
  37. 37. we wanted to understand their goals so that we could find ways to help them meet those goals 37
  38. 38. grad student goals - research & thesis 64% - conference presentations & publishing 14% - current awareness 9% - helping the lab function - networking - job search 38
  39. 39. undergrad student goals - complete their course work 75% - research 7% - current awareness 7% - participate in MIT clubs & social activities 39
  40. 40. tasks - most frequent - search for a known item - search for information on a topic - find facts - search for a partially known item (don’t have complete citation) - take notes & organize information 40
  41. 41. tasks - most frequent - complete class assignment - check web sites or email or RSS feeds for current awareness - study for a class - learn about a software program - do course readings 41
  42. 42. methods - most frequent - search Google - go directly to a known URL - use the library catalog to browse or search - search our licensed citation databases - use course web sites - review class notes - search our finding tool for e-resources (Vera) 42
  43. 43. methods - most frequent - read textbooks - consult with other students - consult with faculty or guest lecturers - search Google Scholar - search our licensed full text databases - physically browse a collection in the library - refer to books in their personal library - use Wikipedia 43
  44. 44. none of that is surprising. so we asked ourselves... 44
  45. 45. - were they successful from their own points of view? - were they efficient and effective from our point of view as librarians? 45
  46. 46. - were they successful from their own points of view? grads: yes, for 86% of their tasks undergrads: yes 93% (we always asked: did you find what you were looking for?) 46
  47. 47. - were they efficient and effective from our point of view as librarians? grads: yes for 77% of their tasks undergrads: yes 85% (we subjectively rated each task) 47
  48. 48. For all tasks successful efficient (in their own view) (in our view) grads 86% 77% undergrads 93% 85% percentage of tasks done during the week of the study 48
  49. 49. - you might say... well librarians will always think of better ways but... 49
  50. 50. - we also broke it down by type of task and that’s where we saw a big difference 50
  51. 51. Searching for information on a topic successful efficient (in their own view) (in our view) grads 80% 40% undergrads 82% 64% percentage of tasks done during the week of the study 51
  52. 52. Searching for information on a topic This is an area where we can help. 52
  53. 53. They had a tendency to start with sources they were familiar with. 53
  54. 54. 54
  55. 55. and with sources recommended by a trusted network of people. 55
  56. 56. advisors colleagues friends family roommates 56
  57. 57. not librarians 57
  58. 58. also common: - figuring it out themselves 58
  59. 59. that’s the culture at MIT 59
  60. 60. This is another area where we can help. Help them make connections Include our expert librarians in those connections. 60
  61. 61. They used a wide variety of sources. 61
  62. 62. Google e-books Google Print Google Scholar textbooks Google maps MIT Open Courseware Amazon 62
  63. 63. Google e-books Google Print library databases Google Scholar textbooks Google maps personal libraries MIT Open Courseware web sites of other universities social science data sets Amazon 63
  64. 64. Google e-books Google Print library databases old course notes Google Scholar textbooks Google maps lab notebooks personal libraries MIT Open Courseware web sites of other universities social science data sets Amazon personal contacts (people) 64
  65. 65. usually began with Google. 65
  66. 66. Almost everyone had a few favorite resources. 66
  67. 67. They tended to reuse their favorites, rather than try new ones. 67
  68. 68. Many students did their TOPICAL discovery in non-library sources. 68
  69. 69. then came to the library to look up the items they found. 69
  70. 70. Looking up known items in our systems usually worked well. 70
  71. 71. 71
  72. 72. Undergrads mainly looked for information related to courses they were taking. 72
  73. 73. Often a few good sources were enough. 73
  74. 74. In those cases, people get what’s convenient to get. 74
  75. 75. Graduate students looked for information related to their research. 75
  76. 76. - more difficult and time-consuming. - highly-specific topics. - needed more depth and comprehensiveness. 76
  77. 77. 1. trouble with knowing where to look 2. trouble with effectively searching the sources they used 77
  78. 78. They spent a large amount of time with varying degrees of success. 78
  79. 79. Sometimes used “brute force” methods. VERY time consuming 79
  80. 80. from our point of view as librarians “if they had only known about X!” (fill in the blank) 80
  81. 81. from their point of view: They often thought that’s how it had to be. 81
  82. 82. Another area where we can help. 82
  83. 83. Also: They used a wide variety of methods for “personal info management.” i.e., organizing what they found 83
  84. 84. They spend a lot of time doing this (in creative ways). 84
  85. 85. Most students are suffering from information overload. 85
  86. 86. Refworks, Endnote, Zotero, CiteULike, Connotea.... a big interest in this area. 86
  87. 87. By the way: We conducted a large library survey this past year with a very high response rate. 87
  88. 88. users want us to simplify search, felt there were too many starting points 88
  89. 89. Too many starting points 89
  90. 90. Too many starting points 90
  91. 91. Too many starting points 91
  92. 92. Too many starting points 92
  93. 93. Another finding: They are not aware of many of the services we provide beyond the obvious ones. 93
  94. 94. A picture of their culture - sleep deprived - stressed, in a hurry - do it yourself - focused on getting course work done or doing research - current awareness important - relying on trusted network of friends and colleagues 94
  95. 95. A picture of their culture - not asking for help - not knowing about all our services - not knowing about all our research databases - using “brute force” methods - struggling with organizing the info they have already found - need to share the info with colleagues 95
  96. 96. so what are we going to do? 96
  97. 97. Priorities • Make topical discovery easier and more effective. • Incorporate community & trust features in our systems. • Embed ourselves & our systems where users are. • Build our expertise into the systems, create systems that you learn from by using. 97
  98. 98. Faceted browsing improves discovery 98
  99. 99. We’re looking at different solutions for a new catalog interface using faceted browsing: - Endeca - Siderean - Univ. of Rochester XC - build our own using Solr & Lucene - Worldcat Local 99
  100. 100. Not just our catalog, but also to include: - DSpace repository - Image collections - Future archival collections - Pages from our web site all with different types of metadata: MARC, Dublin Core,VRA, etc. 100
  101. 101. nearly 500 research databases aggregated search of our federated search of local content with faceted licensed e-content browsing DSpace image archives catalog repository collections 101
  102. 102. And we’re building a better way for students to discover our research databases (almost 500 of them), using federated searching. (to be released next fall) 102
  103. 103. community and trust features 103
  104. 104. So many community features available now: - Social bookmarking - Tagging - Comments - Reviews - Rating - Popularity rankings (circ stats and e-resource usage stats) 104
  105. 105. We want more of these features in our systems 105
  106. 106. We’d like a system where the MIT community can contribute reviews 106
  107. 107. We’d like a multi-university tagging sytem. (PennTags.. across academia). 107
  108. 108. putting ourselves where users are 108
  109. 109. Where are the users? - Google - Google Scholar - Google Print - Wikipedia - Amazon - iTunes - Facebook - Stellar (course management system at MIT) - MIT departmental web sites - faculty web sites at other institutions 109
  110. 110. We are already embedded in Google Scholar. 110
  111. 111. 111
  112. 112. 112
  113. 113. Browser extensions • LibX - Firefox extension libraries.mit.edu/libx 113
  114. 114. 114
  115. 115. LibX adds our search to the browser toolbar 115
  116. 116. Puts links to the libraries where the users are: - such as Amazon, Barnes & Noble, Powell’s - New York Times book reviews - Google - and more. 116
  117. 117. Links from Amazon to our catalog cue 117
  118. 118. Links from Amazon to our catalog 118
  119. 119. LibX links ISBNs to our catalog Without LibX, the ISBN is not a link. 119
  120. 120. LibX right-click sends selected text on a page to Google Scholar and more 120
  121. 121. libx.org 121
  122. 122. Other ways to put ourselves where they are - our news blog (Wordpress) - RSS feeds for the blog and for new titles in our catalog - podcasts in iTunes - “insideMIT” portal under development, build library widgets for that - work on extending library systems inside course management systems 122
  123. 123. We now have a “betas” page ... similar to Google Labs 123
  124. 124. libraries.mit.edu/betas 124
  125. 125. labs.google.com 125
  126. 126. 126
  127. 127. 127
  128. 128. Successful systems extend the users’ work practice. - Karen Holzblatt 128
  129. 129. Design works best when it models user behavior. -Joshua Porter 129
  130. 130. We should look for ways to understand and extend the practice of our users. 130
  131. 131. Useful resources Understanding your users: a practical guide to user requirements methods, tools, and techniques. Morgan Kaufmann, 2004. Catherine Courage and Kathy Baxter. http://www.amazon.com/Understanding-Your-Users-Requirements-Technologies/dp/1558609350/ Contextual design: a customer-centered approach to systems designs. Morgan Kaufmann, 1997. Hugh Beyer and Karen Holtzblatt. http://www.amazon.com/Contextual-Design-Customer-Centered-Interactive-Technologies/dp/ 1558604111 131
  132. 132. www.hennigweb.com/presentations/cil2007/ 132
  133. 133. questions? 133
  134. 134. Key reports with similar findings & recommendations to ours Rethinking how we provide bibliographic services for the University of California. Bibliographic Services Task Force. December 2005. http://libraries.universityofcalifornia.edu/sopag/BSTF/Final.pdf Perceptions of Libraries and Information Resources. OCLC Reports, 2005. http://www.oclc.org/reports/2005perceptions.htm The Changing Nature of the Catalog and Its Integration with Other Discovery. Library of Congress, 17 March 2006. http://www.loc.gov/catdir/calhoun-report-final.pdf 134
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×