Search Strings Presentation IASL 2007

682 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
682
On SlideShare
0
From Embeds
0
Number of Embeds
33
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Search Strings Presentation IASL 2007

  1. 1. Using Transaction Logs to Characterize User Needs Search Analysis of a Digital Library for Educators Marcia A. Mardis, EdD Assistant Professor Wayne State University
  2. 2. Michigan Teacher Network (MTN) <ul><li>Founded in 1998 </li></ul><ul><li>Moved and renamed Michigan Educators’ Resources in Feb. 2007 </li></ul><ul><li>~8500 professional learning, support, and curriculum resources </li></ul><ul><li>Received additional funding from the NSF </li></ul><ul><li>500,000 (summer)-700,000(fall/winter) page views per day </li></ul><ul><li>750 (summer)-1000 (fall/winter) search strings per day </li></ul><ul><li>Governed by privacy policies </li></ul>
  3. 3. What do search logs say? <ul><li>Characterize user sessions </li></ul><ul><ul><li>Time of day (and length of time) </li></ul></ul><ul><li>Characterize of user queries </li></ul><ul><ul><li>Number of revisions </li></ul></ul><ul><ul><li>Number of terms used </li></ul></ul><ul><ul><li>Use of logic and advanced search techniques </li></ul></ul><ul><li>Characterize search terms </li></ul><ul><ul><li>Topic </li></ul></ul><ul><ul><li>Frequency of terms </li></ul></ul>
  4. 4. Key IS Researchers & Studies <ul><li>Spink, Jansen, Wolfram & Saracevic (2000, 2001, 2002) </li></ul><ul><ul><li>Excite studies </li></ul></ul><ul><ul><li>Not always naughty stuff </li></ul></ul><ul><ul><li>Frameworks, general taxonomies </li></ul></ul><ul><li>Bryan-Kinns & Blandford (2000, 2001) </li></ul><ul><ul><li>Digital library Transaction Log Analysis (vs. Web TLA) </li></ul></ul><ul><ul><li>Interactional traps (misunderstanding; premature bailout) </li></ul></ul>
  5. 5. Key IS Researchers & Studies <ul><li>Silverstein & Henziger (1999) </li></ul><ul><ul><li>AltaVista search logs </li></ul></ul><ul><ul><li>Comparison of theoretical vs. actual user models </li></ul></ul><ul><li>Abbas (2002) </li></ul><ul><ul><li>ARTEMIS </li></ul></ul><ul><ul><li>Influence of metadata and search scaffolds </li></ul></ul>
  6. 6. Consistent IS Study Findings <ul><li>Few queries use advanced search features like boolean; that do contain mistakes </li></ul><ul><li>Highly skewed use of terms : few used a lot, a lot used infrequently </li></ul><ul><li>Distribution of terms bears no relation to distribution of content </li></ul><ul><li>Users look mainly at the first page of results </li></ul><ul><li>Most sessions contain only 1 query </li></ul><ul><li>Most queries are very short ( 3 terms or less ) </li></ul>
  7. 7. Key TE Researchers & Studies <ul><li>Teacher concerns </li></ul><ul><ul><li>Classroom discipline </li></ul></ul><ul><ul><li>Serving diverse learning needs and special education </li></ul></ul><ul><ul><li>Teaching out of immediate field </li></ul></ul><ul><li>Teacher use of technology </li></ul><ul><ul><li>Access  Use </li></ul></ul><ul><ul><li>Teachers need scaffolding </li></ul></ul>
  8. 8. Key TE Researchers & Studies <ul><li>Teachers’ information seeking behaviors </li></ul><ul><ul><li>Teachers ask colleagues (D. Williams, 2004) </li></ul></ul><ul><ul><li>Teachers search like humans search (D. Williams, 2004; Hanson & Carlson, 2004; T. Williams, 2005) </li></ul></ul><ul><ul><li>Teachers seek specific kinds of information (Mardis & Hoffman, 2005, 2006; Fitzgerald, et al 2004, etc.) </li></ul></ul><ul><ul><ul><li>Local </li></ul></ul></ul><ul><ul><ul><li>Curriculum linked </li></ul></ul></ul><ul><ul><ul><li>Classroom management ( Mardis, in press ) </li></ul></ul></ul><ul><ul><li>Teachers adapt and adopt resources </li></ul></ul>
  9. 9. Research Questions <ul><li>What do MTN users look for? To what extent do their search behaviours reflect what we know about user behaviors and teachers’ concerns? </li></ul><ul><li>When do MTN users look for information? What might this tell us about who is using the digital library? </li></ul>
  10. 10. Characteristics of 2003 Windows 2:38pm 2:40pm 3:12pm Mean time of search 1.99 1.97 2.48 Mean words per string 420 530 776 Mean Searches per day 5896 7462 10870 N 4/29-5/14 3/26-4/9 2/19-3/5
  11. 11. Top Searches 2/19-3/5/2003 58 technology education 64 elementary education 64 economics 71 social studies 90 lesson plans 102 sciences 72 history 91 states 87 sound 300 classroom management
  12. 12. Top Searches 3/26-4/9/2003 31 life science 34 music 35 rainforests 39 weather 41 private schools 47 fractions 51 educational assessment 55 lesson plans 60 physical education 304 classroom management
  13. 13. Top Searches 4/29- 5/14/2003 27 organizational skills 28 plagiarism 30 plants 31 zookeeper 37 private schools 41 lesson plans 48 science 109 poetry of kids 146 listserv(s) 180 classroom management
  14. 14. 1 3 2
  15. 15. Characteristics of 2004 Windows
  16. 16. Top Searches 2/19-3/5/2004 n=4048 19 ancient Japan 21 science 22 background clipart 22 teacher certification 25 Reading Strategies 31 clipart 36 core democratic values 51 lesson plans 72 solar system 179 classroom management
  17. 17. Top Searches 3/26 - 4/9/2004 N=2083 15 state curriculum guides 16 Michigan Educational Assessment Program 16 school districts 19 public schools 22 &quot;choices and consequences&quot; 22 solar system 24 teacher certification 31 colleges careers 33 lesson plans 163 classroom management
  18. 18. Top Searches 4/29-5/14/2004 N=12723
  19. 19. Characteristics of 2005 1:14 pm 2.5 385 1/1-12/31/2005 Mean time of day Mean words per string Mean Searches per day N=140012
  20. 20. Top Searches 1/1/05-12/31/05 378 Michigan Educational Assessment Program 401 Reference services 532 MEAP 647 Teachers reference 682 Clearinghouses 727 Teacher certification 881 Lesson plans 991 Educational research 2823 Education 4885 Classroom management
  21. 21. Characteristics of 2006 3 Mean words per search 1:07 pm Mean time of day 557 1/1-12/31/2006 Mean searches per day N=178,730
  22. 22. Cumulative Frequencies 04-06 2352 special education and special needs students 3969 teachers reference and teacher certification 1055 Michigan 1275 MEAP 1472 Internet 1659 reference services 1854 lesson plans 3062 clearinghouses 4170 educational research 8586 education 9073 classroom management
  23. 23. Curriculum areas 2004-2006
  24. 24. Implications <ul><li>Searches do reflect educational trends </li></ul><ul><li>Most searches do not occur late at night (disagree) </li></ul><ul><li>Users search very broad or very narrow concepts with few terms (agree) </li></ul><ul><li>Classroom management, MEAP, and job searching queries dominated, then the focus shifts </li></ul><ul><li>Science is a high need curriculum area </li></ul>
  25. 25. Implications <ul><li>Users repeatedly type in the same search </li></ul><ul><li>Many users exhibit “Search Engine Habits” </li></ul><ul><ul><li>Search for items beyond scope of collection </li></ul></ul><ul><ul><li>Seem to be expecting to search site text, not metadata </li></ul></ul><ul><ul><li>Perform searches on browsing categories </li></ul></ul>
  26. 26. On his way to use MTN/MER? -During & After school -Management issues -Science teaching - New teacher
  27. 27. Users search the strangest things… <ul><li>Ozzy Osbourne </li></ul><ul><li>cell phone etiquette </li></ul><ul><li>pet dandruff </li></ul><ul><li>guys </li></ul><ul><li>worry </li></ul><ul><li>Marriage counseling </li></ul>
  28. 28. Caveats <ul><li>Are the exhibited searching characteristics due to some unique aspect of the system or collection? </li></ul><ul><li>Are the user samples from the same or different populations? </li></ul><ul><li>Are the searching characteristics really applicable to all teacher users? </li></ul><ul><li>What are the implications of users who do not understand what they are searching (metadata vs. full-text) </li></ul>
  29. 29. Caveats <ul><li>Are the topic classification schemes used consistently? </li></ul><ul><li>Is the data collection tool accurate? </li></ul>

×