Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Crowd sourcing art history


Published on

A presentation at the ESF-COST symposium on the Networked Humanities - July 2010.

  • Be the first to comment

Crowd sourcing art history

  1. 1. CROWD SOURCINGART HISTORY<br />Research and Applications of <br />Social Tagging in Museums<br />Robert Stein, Indianapolis Museum of Art<br />Robert Stein<br />Chief Information Officer<br />Indianapolis Museum of Art<br /><br />@rjstein<br />
  2. 2. MUSEUMS HAVE<br />A PROBLEM<br />
  3. 3. COLLECTIONS <br />ARE TOO BIG TO <br />BROWSE<br />(even for small collections)<br />Flickr Credit ~andrewhowson<br />
  4. 4.
  5. 5. 54,000 objects in collection<br />2,242 objects on display (4%)<br />26,268 objectimages (48%)<br />
  6. 6. Unless you know what to look for…<br />Descriptive meta-data to support browsing is limited<br />Browsing is hard<br />Flickr Credit ~andercismo<br />
  7. 7. Search-only interfaces inhibit discovery<br />
  8. 8. From: J. P.<br />Date: Sat, 26 Aug 2006 11:24:43 -0700<br />To:<br />Subject: Looking for a paintingPlease help:I have been looking on and off for years for this painting. The painting is of a very well dressed renaissance man standing in a room (a library) in front of him on a table is a large hour glass. The painting has very rich colors. I have talked to a lot of people and they have said they have seen this painting but can't remember its name or the name of the artist.Could you please use your resources to find this painting?<br />
  9. 9.
  10. 10.
  11. 11. What “J. P.” knows:<br />painting<br />Renaissance<br />standing<br />man<br />very well dressed<br />library<br />hourglass<br />table<br />rich colors<br />What a Met curator knows:<br />Portrait of a Man, ca. 1520–25Morettoda Brescia (Alessandro Bonvicino) (Italian, Brescian, born about 1498, died 1554)Oil on canvas; 34 1/4 x 32 in. (87 x 81.3 cm)Rogers Fund, 1928 (28.79)<br />Provenance: Maffei, Brescia (by 1760, as "Ritrattod'uomo con carta in mano, edOrologio, diCallistoda Lodi"); by descent to contessa Beatrice ErizzoMaffeiFenaroli Avogadro, Palazzo Fenaroli, Brescia (by 1853–at least 1857, as by Moretto); her daughter, contessa Maria LiviaFenaroli Avogadro, later marchesaFassati, Brescia (in 1862); her son, marcheseIppolitoFassati, Milan (by 1878–at least 1912); [EliaVolpi, Florence, by 1915–16; sold to Knoedler]; [Knoedler, New York, 1916–28; sold to MMA]<br />
  12. 12. How can visitors<br />take part in powering <br />their own experience?<br />source ~ mindcaster-ezzolicious<br />
  13. 13. Visitors As Data<br />Visitors Havethe BrainPower We<br />Want<br />Credit: Benedict Campbell<br />
  14. 14. Unfortunately, they aren’t<br />clones who will do <br />our bidding<br />source ~donsolo<br />
  15. 15. MUSEUM<br />IMPACT<br />VISITOR <br />ENGAGEMENT<br />Can we create a virtuous circle with visitors that clearly expresses the value and impact of their participation?<br />source ~m-louis<br />
  16. 16. Steve.Museum<br />Exploring Applications of Social Tagging for Museums<br />Founded in 2005<br />2006 Institute for Museum and Library Services (IMLS) National Leadership Research Grant<br />2008 IMLS NLG Steve In Action<br />2008 IMLS NLG Research Grant T3: Text, Tags, Trust<br />Open Source software supporting tagging in museums<br />
  17. 17. Steve.Museum<br />Exploring Applications of Social Tagging for Museums<br />Founded in 2005<br />2006 Institute for Museum and Library Services (IMLS) National Leadership Research Grant<br />2008 IMLS NLG Steve In Action<br />2008 IMLS NLG Research Grant T3: Text, Tags, Trust<br />Open Source software supporting tagging in museums<br /><br />
  18. 18.<br />Why study social tagging?Every participant had a different answer<br />Can tagging help users find art more easily?<br />Can tagging change the way users look at and engage with art?<br />Can tagging help museums understand what visitors see and understand?<br />
  19. 19. Uncomfortable<br />
  20. 20. John Singleton Copley, Portrait of Paul Revere<br />
  21. 21. Tag: Jack Black<br />
  22. 22. Winslow Homer, The Gulf Stream<br />Tag: dolphins, leisure<br />
  23. 23. Winslow Homer, The Gulf Stream<br />Tag: dolphins, leisure<br />Accuracy?<br />
  24. 24. 2006-08 Research ResultsDownload the 10MB file at<br />
  25. 25. Some stats from the research <br />11 Participating Museums<br />1,784 Works of Art in the Research <br />93,380 Tags collected* <br />2,275 Users who tagged*<br />*Derived from the sum of statistics from single and multi-institutional deployments<br />
  26. 26. A Few Highlights<br />Museum professionalsfound most tags useful<br />88% of tags were useful<br />If you found this work using this term would you be surprised?<br />
  27. 27. A Few Highlights<br />Tags are different than museum documentation<br />86% of all tags not found in label copy<br />62% of distinct tags not in AAT<br />85% of distinct tags not in ULAN<br />
  28. 28. A Few Highlights<br />Tags are almost always useful when they are assigned two or more times<br />
  29. 29. A Few Highlights<br />Institutional Affiliation Matters<br />Users invited to tag by a single institution were 4 times as productive<br />Multi-Institution Tagger: 22 tags / user<br />Single-Institution Tagger: 82 tags / user<br />
  30. 30. Photo Credit ~warzauwynn (Flickr)<br />Putting Steveto work<br />
  31. 31. Steve in Action<br />Funded in 2008 by the IMLS<br />A Few Project Goals<br />Make Social Tagging Easy<br />Generalize to all object collections<br />Abstract Data for Tags<br />Develop Innovative NewInterfaces<br />Facilitate Cross-CollectionSearch / Browsing<br />
  32. 32. Current Tagger Stats<br />18 Participating Institutions<br />65,708 Objects in the Tagger<br />427,624 Tags collected <br />4,159 Users who tagged<br />
  33. 33. Steve in Action Features<br />Simple Import (CSV, CDWA, Scraping)<br />Hosted and Themable Data Collection Platform<br />Powerful API Access<br />Cut-n-Paste Tagging Widgets for Easy Integration<br />
  34. 34.
  35. 35.
  36. 36. Drive user-experience with tagging<br />Can tagging be fun?<br />
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45. LOST IN A CLOUD<br />
  46. 46. Finding a Needle in the Haystack<br />
  47. 47.
  48. 48. Funded in 2008 by IMLS<br />With the University of Maryland, and collaborative of museum partners<br />Studying the relationships between social tags, scholarly text and resources, and the application of trust networks to improve access to museum collections.<br />
  49. 49. Heirarchy for Tags<br />
  50. 50. Can we create hierarchy automatically?<br />Supporting Semantic Analysis…<br />
  51. 51. TermProcessing Framework<br />TokenStreamProcessor<br />Performs an operation on a token<br />TokenStreamPipeline<br />A sequence of TokenStreamProcessors to apply in order<br />Taggers<br />A special TokenStreamProcessor that adds metadata to a term.<br />
  52. 52. Token Stream<br />May contain one or more terms/tags in sequence<br />The<br />fox<br />jumped<br />over<br />the<br />lazy<br />dog.<br />quick<br />brown<br />INPUT<br />Tokenize<br />
  53. 53. Token Stream<br />Remove capitalization and punctuation<br />the<br />fox<br />jumped<br />over<br />the<br />lazy<br />dog<br />quick<br />brown<br />INPUT<br />Tokenize<br />RemoveCapitals<br />RemovePunctuation<br />
  54. 54. Token Stream<br />Remove Stop Words<br />fox<br />jumped<br />over<br />lazy<br />dog<br />quick<br />brown<br />INPUT<br />Tokenize<br />RemoveCapitals<br />RemovePunctuation<br />RemoveStopWords<br />
  55. 55. Token Stream<br />Tag Part of Speech and Normalize morphology<br />fox<br />jump<br />over<br />lazy<br />dog<br />quick<br />brown<br />INPUT<br />ADJ<br />ADJ<br />N<br />V<br />ADV<br />ADJ<br />N<br />Tokenize<br />RemoveCapitals<br />RemovePunctuation<br />RemoveStopWords<br />PartOfSpeechTagger<br />MergeMorphology<br />TermContexts<br />
  56. 56. Sample Chain<br />import steve.proccessing as sp<br />filter1 = sp.filters.RemoveWhiteSpace()<br />filter2 = sp.filters.LowerCase()<br />filter1.setInput([‘Star Wars’, ‘lightsaber’])<br />filter2.setInput(filter1)<br />Filter2.getOutput()<br />Result = [[‘starwars’], [‘lightsaber’]]<br />
  57. 57. Raw Tags<br />
  58. 58. Morphy Normalization<br />Naïve Normalization yields 105,547 distinct terms<br />Morphological Normalization yields 70,295 distinct terms<br />33% Reduction in the corpus<br />
  59. 59. Multi-Word Tags<br />Approximately 20% of tags contain more than one word (46% distinct)<br />fox<br />jump<br />over<br />lazy<br />dog<br />quick<br />brown<br />ADJ<br />ADJ<br />N<br />V<br />ADV<br />ADJ<br />N<br />?<br />quickbrownfoxjumpoverlazydog<br /><ul><li>Detect noun and verb phrases, proper nouns – split or merge?</li></li></ul><li>Lexical Tag Analysis (2 word tags)<br />58% NOUN-NOUN<br />“lotus flower”<br />36% ADJ-NOUN<br />“beautiful headdress”<br />6% AVD / VERB Combinations<br />*Klavans and Golbeck, 2010<br />
  60. 60. What About “New England”<br />Idioms / lexicalize phrases are more difficult<br />Heuristic comparison to Wikipedia Titles matched 46% (30% distinct) of multiword tags<br />i.e. “Grapes of Wrath”, “Irish Wolfhound”, “Franco-Prussian War”<br />*Klavans and Golbeck, 2010<br />
  61. 61. Thank You!<br />Questions?<br />