Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Preliminary Findings:  A

Comparative Study of User- and
lnde><er—assigned Subject Terms

Hannah Marie Marshall
Metadata L...
Outline

 

- Background & Context

- Research Design & Methodology
- Aims
- Research Questions
- Design
- Study Populatio...
Background & Context
The Arts & Sciences Images for

T aching Collection

 

- 44,000 images
- 30.000 works

- College of Arts & Sciences (Depa...
Subject Analysis of Images

"lmage indexing is a complex
socio-cognitive process that
involves processing sensory
input th...
Research Questions &

Methodology
Aims

 

Determine retrieval rates for subject based searching

Analyze the search utility of subject metadata for images
...
Research Questions

 

What is the level of correspondence between the existing
subject terms for these images and the par...
Research Question # 1

 

What is the level of E
correspondence between the  Do users search for images using
existing sub...
What is the level of correspondence 
in the types of subject terms '
assigned by users and those in the 

existing metadat...
Research Question # 3

 

Does providing users with a
framework for analyzing the

Ptuxvw-Iwltunrrngeirinthcu

subject of ...
- Image description surveys
0 Qualtrics
- Free text responses
- 10 images per survey

- 9-10 terms per image
- Existing me...
Study Population

 

- Fall 2014, Spring 2015

- All undergraduate students enrolled in courses in
the departments of Art ...
Timeline

 

ilnitial proposal submitted to Institute for Research
Design in Librarianship (IRDL)

January 2014

 
  
 
  ...
The Institute for Research

Design in Librarianship (IRDL)

- Write effective research questions and hypotheses

- Choose ...
A Comparative Study of Cataloger- and User-assigned Subject Terms

Iovman Mann In-v. n.: n llcmv: l)1.1 Lmrman ma Irma C-z...
Retrieval Rates

 

Subject-based image retrieval

Types of terms
rates

     

0 Primary Terms 0 Secondary Terms
0 Unsucc...
Primary Terms

- objects and elements are identified and named

- “What is the image oi’7"/ "What does the image incIude7”...
Existing Metadata

Participants (all)

Control Group

Variable Group

Comparison of Subject and
Non-subject Terms

as.  25...
Types of Sub'ect Terms

Primary Terms

- objects and elements are identilied and named
I00‘/ u
- “What is the image of? "/...
10000-i. 

75 00°»

25 O0‘. 

0 00-. 

Images of 2D vs.  3D Works

Images of 2D Works

 

 

‘ i9 an
I 5 3" D
11.8%
20.2“ ...
Images of 2D vs.  3D Works

 

Percentages of Primary and Non-Subject
Terms Assigned by Participants to images
of 2D and 3...
C Conclusions & Applications
Conclusions

Conclusion

Primary terms yield the greatest 

search utility

High numbers of non-subject
terms applied to i...
Areas of Further Research

 

~ Replicating this study in a faceted
search environment rather than a
free text search envi...
Thank you!
Sunyla2015 preliminary findings
Upcoming SlideShare
Loading in …5
×

Sunyla2015 preliminary findings

154 views

Published on

This presentation presents the complete findings of the first round of data collection in a study to determine the retrieval rates and search utility of subject metadata for images of works of art.

  • Be the first to comment

  • Be the first to like this

Sunyla2015 preliminary findings

  1. 1. Preliminary Findings: A Comparative Study of User- and lnde><er—assigned Subject Terms Hannah Marie Marshall Metadata Librarian for Image Collections Cornell University
  2. 2. Outline - Background & Context - Research Design & Methodology - Aims - Research Questions - Design - Study Population - Timeline - Findings - Conclusions & Applications
  3. 3. Background & Context
  4. 4. The Arts & Sciences Images for T aching Collection - 44,000 images - 30.000 works - College of Arts & Sciences (Departments of Art History and Classics) - Images of art, architecture, and cultural objects - Delivered in Artstor as a Shared Shelf Institutional Collection - Cataloged using a local MS Access database - VRA Core 4.0 - Cataloging Cultural Objects (CCO) - LOC & Getty vocabularies - Full descriptive and subject cataloging for all images
  5. 5. Subject Analysis of Images "lmage indexing is a complex socio-cognitive process that involves processing sensory input through classifying, abstracting, and mapping sensory data into concepts and entities often expressed through socially-defined and culturally- justified linguistic labels and identifiers" (Heidorn, 1999) "Concept—based indexing has the advantage of providing higher- level analysis of the image content but is expensive to implement and suffers from a lack of inter-indexer consistency due to the subjective nature of image interpretation" (Chen, Rasmussen, 1999)
  6. 6. Research Questions & Methodology
  7. 7. Aims Determine retrieval rates for subject based searching Analyze the search utility of subject metadata for images Test, through the use of a variable group, a potential framework for providing visual literacy outreach
  8. 8. Research Questions What is the level of correspondence between the existing subject terms for these images and the participant- assigned terms? What is the level of correspondence in the types of subject terms assigned by users and those in the existing metadata? Does providing users with a framework for analyzing the subject of an image change the nature and content of the subject terms they choose to assign to that images?
  9. 9. Research Question # 1 What is the level of E correspondence between the Do users search for images using existing subject terms for these the same terms we use to images and the participant— describe them? assigned terms E dead nature still life movement Elsie setting Fruit Vessels l Clocks ’ Roerners l Lemons
  10. 10. What is the level of correspondence in the types of subject terms ' assigned by users and those in the existing metadata? Primary Terms objects and elements are identified and named "What is the image of? "/“What does the image include? “ Secondary Terms objects and elements are interpreted; characters are identified, facial expressions and emotional states are asserted. and gestures are ascribed meaning “What is the image about? " : Do users search for images using the same types of terms we use to describe them? Tertiary Terms synthesis of the primary and secondary levels infused with an awareness of the work or image as an expressive output and a product of a time. place, and culture “What is the image a good example of? " / “How does the image communicate? "
  11. 11. Research Question # 3 Does providing users with a framework for analyzing the Ptuxvw-Iwltunrrngeirinthcu subject of an image change the : ;"$_: :;“; “:; ;'. T'; "” nature and content of the $t'°. n“, ;'; ‘2'. :‘. ‘°. .,e. .m. .,. s subject terms they choose to C§i. '.fl. §.T. ‘3‘. ‘?n“m". »‘T£"°"°'"”" assign to that images l5i’2‘. f.I2§: ""m: Z.1i, , ziwi-amzrunaqugmu zmflfle M7 Can we improve retrieval rates by aligning the interpretative practices of users and catalogers?
  12. 12. - Image description surveys 0 Qualtrics - Free text responses - 10 images per survey - 9-10 terms per image - Existing metadata - System for categorizing subject terms - Primary Secondary Tertiary Non-subject - Variable group Deoserevnnlh-¢Iv¢IlI: vhwiuim in-umuymm-wasunzoemixi aim-vi-sninsv-aesuuvm: vwmnmadnqornesirumuvnniot ucnmge. uegmeme«n; me exlsbwunmim » maxuaienqgnrv tmcm-. mn-room. » 1 ilIrn‘zis: i-erioguguuaalnu-ed’
  13. 13. Study Population - Fall 2014, Spring 2015 - All undergraduate students enrolled in courses in the departments of Art History and Classics within the College of Arts and Sciences ; *“t-21-'i"’e. Comell University Comell Univers' I of Classics artment of Hisiio of Art and Visual Studies I _ Departmen ry
  14. 14. Timeline ilnitial proposal submitted to Institute for Research Design in Librarianship (IRDL) January 2014 _ Rétirii H; ‘; iL1ai"; i;; i'g}}; "i ii’; r‘; iL; ;;"; ;i‘i; ii; ’Br'; Ii; ;{iH; ’ . .Jf. 'Y. ... §?. ?7?T? ?7.397f Early October 2014 Mid October 2014 IRB approval Round 1 data collection begins December 2014 Round 1 data collection ends Round 2 data collection begins May 2015 Elilound 2 data collection ends
  15. 15. The Institute for Research Design in Librarianship (IRDL) - Write effective research questions and hypotheses - Choose an appropriate research design for a library science study - Conduct a literature review i - Explain the conceptual logic behind various data collection approaches and describe the rationale for selection of specific methods - Access and participate in the Institute virtual community and related networks for support during the research process - Identify appropriate sampling strategies for research projects - Use and apply commonly used qualitative data collection methods - Assess and apply different qualitative data analysis options - Design and implement a survey - Understand survey data management - Explain various analytic options for surveys - Understand basic principles of mixed methods research design - Choose an appropriate research dissemination forum - Write and disseminate an effective research report
  16. 16. A Comparative Study of Cataloger- and User-assigned Subject Terms Iovman Mann In-v. n.: n llcmv: l)1.1 Lmrman ma Irma C-zwlbtlnns <. £:r'1eI Jnr. u:', nun __. .. ""' ' ' u. .. or. ’ gdnbh-. ' , ,. -0.. ... .-
  17. 17. Retrieval Rates Subject-based image retrieval Types of terms rates 0 Primary Terms 0 Secondary Terms 0 Unsuccessful retrieval 0 Successful retrieval Tertiary Teims ‘ N0“‘5UbieCt Terms
  18. 18. Primary Terms - objects and elements are identified and named - “What is the image oi’7"/ "What does the image incIude7” Secondary Terms ' obiects and elements are Interpreted characters are identified tacial expressions and emotional states are asserted, and gestures are ascribed meaning ' “What is the image about? ‘ Tertiary Terms - synthesis at the primary and secondary levels infused with an awareness oi the work or image as an CXDVOSSIVC output and a product oi a lime place, and Cultute ' “What is the image a good example of? " I ‘How does the image corrmunicate’7‘ Non-Subject Terms ' descriptive terms that do not address the subject at the work or image 100'» 75-. 50°. 25!. D“-. Comparison of types of terms -us in <5!» , _i ¥-'i -''. '!i it»: ‘W LIN? Existing Metadata Pamcapanis (all) Control Group Vunabie Group Category Axis T Primary Terms 7 Secondary Terms Tertiary Terms 7" Non—Subject Terms
  19. 19. Existing Metadata Participants (all) Control Group Variable Group Comparison of Subject and Non-subject Terms as. 253. Subject Terms Worktype Value Judgments Value Style/ Period fl Materials/ Techniques 50% Non»Subject Terms 75:. 100-. Location Culture Creator Condition Types of Non-Subject Terms 11% 15.0% 52% 30.0% 45.0% 60.0%
  20. 20. Types of Sub'ect Terms Primary Terms - objects and elements are identilied and named I00‘/ u - “What is the image of? "/“What does the image include? “ 75'/ . Secondary Terms - objects and elements are interpreted: characters are identified, lacial expressions and emotional states are asserted. and gestures are ascribed meaning 5% - “What is the image about? " 25% Tertiary Terms - synthesis of the primary and secondary levels infused with an awareness of the work or image as an expressive output and a product of a time, place. and culture W’ - “What is the image a good example ct? “ / “How does the image communicate? ‘ Types of Subject Terms Existing Metadata Participants (all) Control Group Variable Group I Primary Terms I Secondary Terms Tertiary Terms
  21. 21. 10000-i. 75 00°» 25 O0‘. 0 00-. Images of 2D vs. 3D Works Images of 2D Works ‘ i9 an I 5 3" D 11.8% 20.2“ V 71 7~, 49 an 40 B‘ e Calaloger Control omup Variable Group Respondents tall] I Primary Terms Tertiary Terms I Secondary Terms I Non-Subject Terms ion 00°. 75 00°. 50 00°. 0 oo= . Images of 3D Works B .2“ 9 A 7 2‘ . ~ 26 4”" Catalogs! Control Group Variable Group Raspandonls (all) I Primary Terms Tertiary Terms I Secondary Terms I Nonvsubiect Terms
  22. 22. Images of 2D vs. 3D Works Percentages of Primary and Non-Subject Terms Assigned by Participants to images of 2D and 3D Works Images of 2D works Image of 3D works I Primary Terms I Non-Subject Terms
  23. 23. C Conclusions & Applications
  24. 24. Conclusions Conclusion Primary terms yield the greatest search utility High numbers of non-subject terms applied to images of 3D works suggest that subject metadata is a weak access point for 3D works Further research into identifying and capitalizing on instances of Poor retrieval rates seem to be due to problems of vocabulary Potential Application Focus cataloging resources on assigning high numbers of primary terms Forego subject cataloging for images of 3D works, focusing instead on other descriptive access points conceptual correspondence
  25. 25. Areas of Further Research ~ Replicating this study in a faceted search environment rather than a free text search environment - Investigating instances of 3.. conceptual correspondence - Analyzing the degrees and strength of literal correspondences ‘ / - Look at the data according to the o __ _ 1. type of image (ie. Western v. Non- _~ . t_. ... ,_. g. ~ Western, Abstraction vs. Representation, etc. )
  26. 26. Thank you!

×