Connaway, Lynn Silipigni, William Harvey, Vanessa Kitzie, and Stephanie Mikitish. 2017. “Academic Library Impact: Improving Practice and Essential Areas to Research.” Presented at the Update on Value of Academic Libraries Initiative (ACRL) at the ALA Annual Conference, Chicago, Illinois, June 25.
Changing Tack: A Future-Focused ACRL Research AgendaLynn Connaway
Connaway, Lynn Silipigni, William Harvey, Vanessa Kitzie, and Stephanie Mikitish. 2017. “Changing Tack: A Future-Focused ACRL Research Agenda.” Presented at the ACRL 2017 Conference, Baltimore, Maryland, March 23.
Lecture presented by Rhea Rowena U. Apolinario at PAARL's Summer Conference on the theme "Library Analytics: Data-driven Library Management, held at Pearl Hotel, Manila on 20-22 April 2016
Action-Oriented Research Agenda on Library Contributions to Student Learning ...Lynn Connaway
Connaway, Lynn Silipigni, William Harvey, Vanessa Kitzie, and Stephanie Mikitish. 2017. “Action-Oriented Research Agenda on Library Contributions to Student Learning and Success.” Presented at the ALA Midwinter Meeting, Atlanta, Georgia, January 22.
Lecture presented by Fernan R. Dizon at PAARL's Summer Conference on the theme "Library Analytics: Data-driven Library Management", held at Pearl Hotel, Manila on 20-22 April 2016
Lecture presented by Vivian Praxedes D. Sy at PAARL's Summer Conference on the theme "Library Analytics: Data-driven Library Management", held at Pearl Hotel, Manila on 20-22 April 2016
This presentation was provided by Rachel Lewellen of Harvard University during the NISO event, Assessment Practices and Metrics in the 21st Century Training Session Four held on Friday, November 9th.
Changing Tack: A Future-Focused ACRL Research AgendaLynn Connaway
Connaway, Lynn Silipigni, William Harvey, Vanessa Kitzie, and Stephanie Mikitish. 2017. “Changing Tack: A Future-Focused ACRL Research Agenda.” Presented at the ACRL 2017 Conference, Baltimore, Maryland, March 23.
Lecture presented by Rhea Rowena U. Apolinario at PAARL's Summer Conference on the theme "Library Analytics: Data-driven Library Management, held at Pearl Hotel, Manila on 20-22 April 2016
Action-Oriented Research Agenda on Library Contributions to Student Learning ...Lynn Connaway
Connaway, Lynn Silipigni, William Harvey, Vanessa Kitzie, and Stephanie Mikitish. 2017. “Action-Oriented Research Agenda on Library Contributions to Student Learning and Success.” Presented at the ALA Midwinter Meeting, Atlanta, Georgia, January 22.
Lecture presented by Fernan R. Dizon at PAARL's Summer Conference on the theme "Library Analytics: Data-driven Library Management", held at Pearl Hotel, Manila on 20-22 April 2016
Lecture presented by Vivian Praxedes D. Sy at PAARL's Summer Conference on the theme "Library Analytics: Data-driven Library Management", held at Pearl Hotel, Manila on 20-22 April 2016
This presentation was provided by Rachel Lewellen of Harvard University during the NISO event, Assessment Practices and Metrics in the 21st Century Training Session Four held on Friday, November 9th.
Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Rese...Lynn Connaway
Connaway, Lynn Silipigni. 2017. “Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Research Agenda for Student Learning and Success.” Presented at the University of Macau, Macau, April 6.
Strategic Metrics, presented at the Leadership Seminar on Strategy, Assessment and Service Development. University of Lund, Sweden. 19th September 2012. Presentation by Selena Killick, Cranfield University. Presentation discusses the need for assessment of Library Strategies and some of the techniques available to achieve this.
Lecture presented by Dr. Reinabelle C. Reyes at PAARL's Summer Conference on the theme "Library Analytics: Data-driven Library Management, held at Pearl Hotel, Manila on 20-22 April 2016
ACRL Value Update 2014, Annual Las Vegasmbowlesterry
An update on the work of the Value of Academic Libraries committee, presented at a Sunday afternoon forum at ALA Annual Conference in Las Vegas by Lynn Silipigni Connaway and Melissa Bowles-Terry.
A presentation on enhancing the usability of libraries delivered at the Ark Group's recent conference, "Management Strategies for Library and Information Service Centres", Sydney, 19-21 September 2007.
Changing Tack: A Future-Focused ACRL Research AgendaOCLC
Connaway, Lynn Silipigni, William Harvey, Vanessa Kitzie, and Stephanie Mikitish. 2017. “Changing Tack: A Future-Focused ACRL Research Agenda.” Presented at the ACRL 2017 Conference, Baltimore, Maryland, March 23.
Action-Oriented Research Agenda on Library Contributions to Student Learning ...OCLC
Connaway, Lynn Silipigni, William Harvey, Vanessa Kitzie, and Stephanie Mikitish. 2017. “Action-Oriented Research Agenda on Library Contributions to Student Learning and Success.” Presented at the ALA Midwinter Meeting, Atlanta, Georgia, January 22.
Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Rese...OCLC
Connaway, Lynn Silipigni. 2017. “Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Research Agenda for Student Learning and Success.” Presented at the University of Macau, Macau, April 6.
Where are We Going and What Do We Do Next? Demonstrating the Value of Academi...OCLC
Connaway, Lynn Silipigni. 2017. “Where are We Going and What Do We Do Next? Demonstrating the Value of Academic Libraries in Time of Uncertainty.” Presented at the RLUK Conference 2017, London, United Kingdom, March 9.
Where are We Going and What Do We Do Next? Demonstrating the Value of Academi...Lynn Connaway
Connaway, Lynn Silipigni. 2017. “Where are We Going and What Do We Do Next? Demonstrating the Value of Academic Libraries in Time of Uncertainty.” Presented at the RLUK Conference 2017, London, United Kingdom, March 9.
Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Rese...Lynn Connaway
Connaway, Lynn Silipigni. 2017. “Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Research Agenda for Student Learning and Success.” Presented at the University of Macau, Macau, April 6.
Strategic Metrics, presented at the Leadership Seminar on Strategy, Assessment and Service Development. University of Lund, Sweden. 19th September 2012. Presentation by Selena Killick, Cranfield University. Presentation discusses the need for assessment of Library Strategies and some of the techniques available to achieve this.
Lecture presented by Dr. Reinabelle C. Reyes at PAARL's Summer Conference on the theme "Library Analytics: Data-driven Library Management, held at Pearl Hotel, Manila on 20-22 April 2016
ACRL Value Update 2014, Annual Las Vegasmbowlesterry
An update on the work of the Value of Academic Libraries committee, presented at a Sunday afternoon forum at ALA Annual Conference in Las Vegas by Lynn Silipigni Connaway and Melissa Bowles-Terry.
A presentation on enhancing the usability of libraries delivered at the Ark Group's recent conference, "Management Strategies for Library and Information Service Centres", Sydney, 19-21 September 2007.
Changing Tack: A Future-Focused ACRL Research AgendaOCLC
Connaway, Lynn Silipigni, William Harvey, Vanessa Kitzie, and Stephanie Mikitish. 2017. “Changing Tack: A Future-Focused ACRL Research Agenda.” Presented at the ACRL 2017 Conference, Baltimore, Maryland, March 23.
Action-Oriented Research Agenda on Library Contributions to Student Learning ...OCLC
Connaway, Lynn Silipigni, William Harvey, Vanessa Kitzie, and Stephanie Mikitish. 2017. “Action-Oriented Research Agenda on Library Contributions to Student Learning and Success.” Presented at the ALA Midwinter Meeting, Atlanta, Georgia, January 22.
Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Rese...OCLC
Connaway, Lynn Silipigni. 2017. “Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Research Agenda for Student Learning and Success.” Presented at the University of Macau, Macau, April 6.
Where are We Going and What Do We Do Next? Demonstrating the Value of Academi...OCLC
Connaway, Lynn Silipigni. 2017. “Where are We Going and What Do We Do Next? Demonstrating the Value of Academic Libraries in Time of Uncertainty.” Presented at the RLUK Conference 2017, London, United Kingdom, March 9.
Where are We Going and What Do We Do Next? Demonstrating the Value of Academi...Lynn Connaway
Connaway, Lynn Silipigni. 2017. “Where are We Going and What Do We Do Next? Demonstrating the Value of Academic Libraries in Time of Uncertainty.” Presented at the RLUK Conference 2017, London, United Kingdom, March 9.
Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Rese...OCLC
Connaway, Lynn Silipigni. 2017. “Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Research Agenda for Student Learning and Success.” Presented at the University of Hong Kong, Hong Kong, April 7.
Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Rese...Lynn Connaway
Connaway, Lynn Silipigni. 2017. “Demonstrating the Value of Academic Libraries in Times of Uncertainty: A Research Agenda for Student Learning and Success.” Presented at the University of Hong Kong, Hong Kong, April 7.
Academic library impact: Improving practice and essential areas to researchOCLC
Connaway, L. S. (2018). Academic library impact: Improving practice and essential areas to research. Presented at Bar-Ilan University, March 11, 2018, Ramat Gan, Israel.
Academic library impact: Improving practice and essential areas to researchLynn Connaway
Connaway, L. S. (2018). Academic library impact: Improving practice and essential areas to research. Presented at Bar-Ilan University, March 11, 2018, Ramat Gan, Israel.
Communicating Library Impact Beyond Library Walls: Findings from an Action-or...OCLC
Connaway, Lynn Silipigni, and Alan Carbery. 2017. “Communicating Library Impact Beyond Library Walls: Findings from an Action-oriented Research Agenda.” Presented at the ACRL Leadership Council at the ALA Annual Conference, Chicago, Illinois, June 23.
Communicating Library Impact Beyond Library Walls: Findings from an Action-or...Lynn Connaway
Connaway, Lynn Silipigni, and Alan Carbery. 2017. “Communicating Library Impact Beyond Library Walls: Findings from an Action-oriented Research Agenda.” Presented at the ACRL Leadership Council at the ALA Annual Conference, Chicago, Illinois, June 23.
NISO Virtual Conference: Expanding the Assessment Toolbox: Blending the Old and New Assessment Practices
Keynote Address: The Value of Library-Provided Content: Assessing Usage and Demonstrating Impact
Megan Oakleaf, Associate Professor of Library and Information Science, iSchool at Syracuse University
Lecture presented by Fe Angela M. Verzosa at PAARL’s One-Day Seminar on the theme "Revitalized Library Networking: PAARLNet at the Frontline" (Epifanio de los Santos Auditorium, The National Library, T.M. Kalaw, Manila, 25 November 2010)
Data-Informed Decision Making for Libraries - Athenaeum21Megan Hurst
Athenaeum21 presents three case studies of assessment and evaluation programs in libraries--one past, one current, and one future. The cases use three different modes of data gathering and analysis to show the power of understanding user needs and how well your organization is meeting them.
Data-Informed Decision Making for Digital ResourcesChristine Madsen
This session will provide three case studies of assessment and evaluation programs in libraries--one past, one current, and one future. The cases use three different modes of data gathering and analysis and show the power of understanding user needs and how well your organization is meeting them.
Similar to Academic Library Impact: Improving Practice and Essential Areas to Research (20)
Communicating library impact beyond library walls: Findings from an action-or...OCLC
Connaway, L. S. (2018). Communicating library impact beyond library walls: Findings from an action-oriented research agenda: A collaborative effort. Presented at Universidad Javeriana, October 1, 2018, Bogota, Colombia.
"You can just tell whether a website looks reliable or not." People's modes o...OCLC
Connaway, L. S. (2018). "You can just tell whether a website looks reliable or not." People's modes of online engagement. Keynote presented at Universidad Javeriana, October 2, 2018, Bogota, Colombia.
Factors influencing research data management programs.OCLC
Connaway, L. S., & Matusiak, K. (2018). Factors influencing research data management programs. Workshop presented at the Università di Teramo, October 15, 2018, Teramo, Italy.
Teaching research methods in LIS programs: Approaches, formats, and innovativ...OCLC
Connaway, L. S., Dickey, T., Hartel, J., Kendall, L., Rebmann, K., Rang, T., & Yontz, E. (2018). Teaching research methods in LIS programs: Approaches, formats, and innovative strategies. Presented at ALISE 2018 Conference, February 9, 2018, Denver, Colorado.
OCLC ALISE Library & Information Science Research Grant ProgramOCLC
Connaway, L. S. (2018). OCLC ALISE Library & Information Science Research Grant Program. Presented at ALISE 2018 Conference, February 8, 2018, Denver, Colorado.
Investing in library users and potential users: The Many Faces of Digital Vi...OCLC
Connaway, L. S. (2018). Investing in library users and potential users: The Many Faces of Digital Visitors and Residents. Presented at SLA-AGLA, March 6, 2018, Muscat, Oman.
Studying information behavior: The Many Faces of Digital Visitors and ResidentsOCLC
Connaway, L. S. (2018). Studying information behavior: The Many Faces of Digital Visitors and Residents. Presented at Bar-Ilan University, March 11, 2018, Ramat Gan, Israel.
Online engagement and information literacy: The Many Face of Digital Visitors...OCLC
Connaway, L. S. (2018). Online engagement and information literacy: The Many Face of Digital Visitors & Residents. Presented at the Bibliostar Conference, March 15, 2018, Milan, Italy.
People's mode of online engagement: The Many Faces of Digital Visitors and R...OCLC
Connaway, L. S. (2018). People's mode of online engagement: The Many Faces of Digital Visitors and Residents. Presented at the iConference, March 26, 2018, Sheffield, United Kingdom.
Applying research methods: Investigating the Many Faces of Digital Visitors &...OCLC
Connaway, L. S. (2018). Applying research methods: Investigating the Many Faces of Digital Visitors & Residents. Presented at the American University, March 29, 2018, Rome, Italy.
Using Qualitative Methods for Library Evaluation: An Interactive WorkshopOCLC
Connaway, Lynn Silipigni, and Marie L. Radford. 2016. "Using Qualitative Methods for Library Evaluation: An Interactive Workshop." Presented at the Libraries in the Digital Age (LIDA) Conference, Zadar, Croatia, June 14.
Visitors and Residents: The Hows and Whys of Engagement with TechnologyOCLC
Connaway, Lynn Silipigni. 2016. “Visitors and Residents: The Hows and Whys of Engagement with Technology.” Presented at the Library Association of the Republic of China (LAROC) Annual Meeting, Taiwan, December 10.
Visitors and Residents: Interactive Mapping Exercise WorkshopOCLC
Connaway, Lynn Silipigni, and William Harvey. 2017. “Visitors and Residents: Interactive Mapping Exercise Workshop.” Presented at the ASIS&T Regional Meeting, Dublin, Ohio, March 3.
Research Methods in Library and Information Science: Trends and Tips for Rese...OCLC
Connaway, Lynn Silipigni. 2017. "Research Methods in Library and Information Science: Trends and Tips for Researchers, Students, & Professionals." Presented at the University of Hong Kong, Hong Kong, March 31, 2017.
Capturing the Behaviors of the Elusive User: Strategies for Library EthnographyOCLC
Connaway, Lynn Silipigni. 2017. "Capturing the Behaviors of the Elusive User: Strategies for Library Ethnography." Presented at the IFLA World Library and Information Congress 2017, 83rd IFLA General Conference and Assembly, Wrocław, Poland, August 22.
Digital Literacy in the Era of Fake News: Key Roles for Information Professio...OCLC
Connaway, Lynn Silipigni, Heidi Julien, Michael Seadle, and Alex Kasprak. 2017. "Digital Literacy in the Era of Fake News: Key Roles for Information Professionals." Panel presented at ASIS&T 2017, 80th Annual Meeting of the Association for Information Science and Technology, Washington, DC, October 30.
'Is it a journal title, or what?' Mitigating Microaggressions in Virtual Refe...OCLC
Radford, Marie L., Vanessa Kitzie, Lynn Silipigni Connaway, and Diana Floegel. 2017. "'Is it a journal title, or what?' Mitigating Microaggressions in Virtual Reference." Presented at ALA/RUSA’s New Discoveries in Reference: The 23rd Annual Reference Research Forum, ALA Annual Conference, Chicago, Illinois, June 22-27.
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Normal Labour/ Stages of Labour/ Mechanism of LabourWasim Ak
Normal labor is also termed spontaneous labor, defined as the natural physiological process through which the fetus, placenta, and membranes are expelled from the uterus through the birth canal at term (37 to 42 weeks
Biological screening of herbal drugs: Introduction and Need for
Phyto-Pharmacological Screening, New Strategies for evaluating
Natural Products, In vitro evaluation techniques for Antioxidants, Antimicrobial and Anticancer drugs. In vivo evaluation techniques
for Anti-inflammatory, Antiulcer, Anticancer, Wound healing, Antidiabetic, Hepatoprotective, Cardio protective, Diuretics and
Antifertility, Toxicity studies as per OECD guidelines
This slide is special for master students (MIBS & MIFB) in UUM. Also useful for readers who are interested in the topic of contemporary Islamic banking.
Safalta Digital marketing institute in Noida, provide complete applications that encompass a huge range of virtual advertising and marketing additives, which includes search engine optimization, virtual communication advertising, pay-per-click on marketing, content material advertising, internet analytics, and greater. These university courses are designed for students who possess a comprehensive understanding of virtual marketing strategies and attributes.Safalta Digital Marketing Institute in Noida is a first choice for young individuals or students who are looking to start their careers in the field of digital advertising. The institute gives specialized courses designed and certification.
for beginners, providing thorough training in areas such as SEO, digital communication marketing, and PPC training in Noida. After finishing the program, students receive the certifications recognised by top different universitie, setting a strong foundation for a successful career in digital marketing.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
A workshop hosted by the South African Journal of Science aimed at postgraduate students and early career researchers with little or no experience in writing and publishing journal articles.
Academic Library Impact: Improving Practice and Essential Areas to Research
1. Value
of Academic Libraries
ALA | 25 June 2017
Academic Library Impact:
Improving Practice and Essential Areas to Research
Lynn Silipigni Connaway, Senior Research Scientist & Director of User Research, OCLC
William Harvey, Consulting Engineer, OCLC
Vanessa Kitzie, Assistant Professor, University of South Carolina
Stephanie Mikitish, Librarian, Rutgers University
3. Value
of Academic Libraries
The Value of Academic
Libraries
• ACRL Goal-area committee
– Part of Plan for Excellence
• Goal: Academic libraries
demonstrate alignment with &
impact on institutional
outcomes
– Promote impact & value of
libraries to higher ed. community
4. Value
of Academic Libraries
Selected Presentations
• Library Assessment
Conference (Nov 2016)
– Results of literature review
• ALA Midwinter (Jan 2017)
– Results of librarian and provosts
interviews
• ACRL (March 2017)
– Priority areas and preliminary
research questions
– Visualization tool
5. Value
of Academic Libraries
Agenda
• Priority areas
– Effective practices from
literature and interviews
– Research questions
– Exemplar studies for
content and research
design
• Visualization tool
• Final steps
6. Value
of Academic Libraries
Priority Areas
1. Communication
2. Mission strategy & alignment
3. Learning analytics
4. Student success
5. Teaching & learning
6. Collaboration
8. Value
of Academic Libraries
Communicate the Library’s Contributions
Host town halls that address the
role of libraries (PP04) and work
to dismantle “pigeonholed”
perceptions
Emphasize the library’s ability to
have “impacts beyond borders”
(PP01) when making funding
requests
Give provosts a “different style of
data” (PP11)
9. Value
of Academic Libraries
Match Library Assessment to Institution’s Mission
Be open to adopting less traditional
roles, such as creating 3 minute
research video competitions
(LM03 and PP03)
Be aware of student demographics,
and accommodate students with
low SES to demonstrate the
importance of the “university as a
gateway to a better life” (PP08)
10. Value
of Academic Libraries
Include Library Data in Institutional Data Collection
Consider collecting holistic student
data before and after graduation
(PP03, PP11)
Refer to data collected by independent,
for-profit institutions (PP14) for
suggestions
Engage senior leadership in dialogue
about the institution’s data
environment (LM13)
11. Value
of Academic Libraries
Quantify the Library’s Impact on Student Success
Measure, collect, analyze and report
data collected from multiple
systems within the academic
institution and statistically analyzed
to predict student success.
Select student success indicators that
consider the diverse experiences of
students.
12. Value
of Academic Libraries
Enhance Teaching and Learning
Collaborate across departments to
leverage library services, spaces,
collections, and expertise to assist
students, perhaps in first-year
experience programs (LM06, PP08)
Keep abreast of higher education
trends, such as fostering the critical
competency skills (PP03, PP10)
needed to be “informed citizens”
(PP06, PP14)
13. Value
of Academic Libraries
Collaborate with Educational Stakeholders
Provide library space and expertise to
support activities such as
interdisciplinary work (PP03, PP08)
or career counseling (PP05)
Partner with institutions outside the
university or college, such as
government and commercial
institutions or community groups
(PP13)
15. Value
of Academic Libraries
• How are other units effectively
communicating with stakeholders?
• What factors influence librarian
communication between academic library
users and potential users?
• How can library administrators and staff
leverage social media to effectively
increase student engagement?
Research Questions:
Communication
16. Value
of Academic Libraries
• How do libraries fit into the broader array
of institutional resources and programs
(e.g., writing centers, tutoring)?
• How are budget constraints affecting the
support by library administrators and staff
of the institution’s mission and specific
goals related to student learning and
success outcomes?
Research Questions:
Mission strategy &
alignment
17. Value
of Academic Libraries
• How are other stakeholders in
higher education using
analytics to affect the areas of
teaching and learning and
student success, and how can
library administrators and staff
contribute to these efforts?
• Will library administrators and
staff need to begin collecting
different and additional data?
Research Questions:
Learning analytics
18. Value
of Academic Libraries
• How do library resources and programs
(i.e., courses, events, etc.) impact
indicators of student success?
• What are the effects of library instruction
on success outcomes for diverse student
populations, (e.g., military students, non-
US students, English language learners,
non-residential students, online-only
students, etc.)?
Research Questions:
Student Success
19. Value
of Academic Libraries
• What are the most common difficulties
faced by library administrators and
staff in measuring teaching and
learning outcomes?
• Where do students go to discover
information (e.g., answers to
questions, resources for their needs)?
If this is not library websites or online
catalogs, library discovery systems be
integrated into academic users’
workflows?
Research Questions:
Teaching & Learning
20. Value
of Academic Libraries
• How can library administrators and
staff collaborate with staff and faculty
members from other academic
institutions to increase student
learning and success?
• What types of collaboration are the
most effective in facilitating student
learning and success outcomes?
Research Questions:
Collaborate
22. Value
of Academic Libraries
Preliminary work
• Context
– Audience
– Prior knowledge,
experience, & bias
– Environment
• Topic/problem
– Past work
– Gaps in research
26. Value
of Academic Libraries
Write-up
• Discussion
– Clear, relevant
findings
– Reiterate & frame
– Compare to past
work
• Future work
– Limitations
– Implications
29. Value
of Academic Libraries
Overview
• Quick Recap of
Visualization
Component
• About Usability Testing
• How We Did It
• What We Learned
• New Features
30. Value
of Academic Libraries
Visualization Component
• Literature Search Tool
– “Zappos for scholarly articles”
– Navigate document database
– Build/save/distribute/curate
reference lists
• Charts and Graphs Tool
– “Tableau for scholarly articles”
– Generate graphics to share
with stakeholders
36. Value
of Academic Libraries
What Is Usability Testing?
“Ethnography is the social equivalent of usability
testing. Where usability is about how people directly interact
with a technology in the more traditional sense, ethnography
is about how people interact with each other. As UX
designers, we’re primarily concerned with how we can use
such research to solve a problem through the introduction or
revision of technology.”
-- Nathanael Boehm
37. Value
of Academic Libraries
Usability Testing Reveals Expectations
• A product owner has one set of
expectations
• A software dev team has another set
• N users experiencing the software for the
first time have N sets
• M experienced users have another M sets
• The software itself has another set
38. Value
of Academic Libraries
Usability Testing Reveals Expectations
• A product owner has one set of
expectations
• A software dev team has another set
• N users experiencing the software for the
first time have N sets
• M experienced users have another M sets
• The software itself has another set
39. Value
of Academic Libraries
User Expectations
• Impedance mismatch
– What {users, X} want
– What {users, X} think they
need
– What users really need
– What can be delivered
• Minimize frustration (“It just
works!”) = maximize usability
40. Value
of Academic Libraries
Usability Testing Is Amazing
• The OCLC Usability Lab is a
fantastic resource
• Ethnography is valuable
• Usability Testing is valuable
42. Value
of Academic Libraries
Usability Study Participants
Gender Institution Participant Type Experience with AiA website
F Kent State
Library and Information
Science (student)
None
F
Adams County
Public Library
System
Library and Information
Science Researcher
None
F
California State
University
Dominguez Hills
Practicing Academic Librarian None
F
University of
Dallas
Academic Library
Administrator
Has used a few times
F
University of
Tennessee at
Chattanooga
Practicing Academic Librarian Never
M
Wayne State
University
Library and Information
Science Researcher
Never
43. Value
of Academic Libraries
Usability Tasks
1. Literature search task
only requiring facets
2. Literature search task
only requiring keywords
3. Literature search
combining (1) and (3).
4. Generate a graph
involving aggregation.
45. Value
of Academic Libraries
Literature Search Tasks
• Locating facets was a challenge
– Allow users to search through facets
• Users unsure of what is in the database
– Provide search-as-you-type to close the
loop
• Users want seamless Boolean/keyword
search
– Augment interface so both can coexist
51. Value
of Academic Libraries
Charts and Graphs Tool
• All users requested some help
materials/videos
– We are working on them
• Some users just are not familiar with
creating charts
– Perhaps training materials + practice
will help
• Familiarity is important
– Give users an escape hatch to Excel
53. Value
of Academic Libraries
Novel Features Added
• Sandboxed user uploads
– Log in with ALA account
– Upload new articles (visible only to user)
– Annotate articles, add new annotations
– Save data
• Broad redesign/reflow of Charts and Graphs
tool
56. Value
of Academic Libraries
Thank you to the following people for their contributions to this project:
Erin M. Hood, OCLC; Brittany Brannon, OCLC; Marie L. Radford, Rutgers
University; ACRL Board; ACRL VAL Committee; Advisory Group Members
57. Value
of Academic Libraries
References
Association of College and Research Libraries. Forthcoming. Academic Library Impact: Improving
Practice and Essential Areas to Research. Researched by Lynn Silipigni Connaway, William Harvey,
Vanessa Kitzie, and Stephanie Mikitish. Chicago: Association of College and Research Libraries.
Association of College and Research Libraries. 2010. Value of Academic Libraries: A Comprehensive
Research Review and Report. Researched by Megan Oakleaf. Chicago: Association of College and
Research Libraries. http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_report.pdf.
Bloom, Nathanael. 2010. “Ethnography in UX.” UX Matters, June 21.
http://www.uxmatters.com/mt/archives/2010/06/ethnography-in-ux.php.
Brown-Sica, Margaret. 2013. “Using Academic Courses to Generate Data for Use in Evidence Based
Library Planning.” Journal of Academic Librarianship 39, no. 3: 275-87. doi:10.1016/j.acalib.2013.01.001.
Connaway, Lynn Silipigni, and Marie L. Radford. 2017. Research Methods in Library and Information
Science, 6th ed. Santa Barbara, CA: Libraries Unlimited.
58. Value
of Academic Libraries
References
Fister, Barbara. 2010. “Critical Assets: Academic Libraries, A View from the Administration Building.”
Library Journal 135, no. 8: 24-27.
Hess, Amanda Nichols. 2015. “Equipping Academic Librarians to Integrate the Framework into
Instructional Practices: A Theoretical Application.” Journal of Academic Librarianship 41, no. 6: 771-76.
doi:10.1016/j.acalib.2015.08.017.
Jantti, M., and Heath, J. 2016. “What Role for Libraries in Learning Analytics?” Performance
Measurement and Metrics 17, no. 2: 203-10.
Lombard, E. 2012. “The Role of the Academic Library in College Choice.” Journal of Academic
Librarianship 38, no. 4: 237-41.
Wikipedia, s.v. “Usability Testing.” Last modified June 13, 2017.
https://en.wikipedia.org/wiki/Usability_testing.
59. Value
of Academic Libraries
Image Attributions
Slide 3: Image: http://bit.ly/2tF1Mtx (https://www.flickr.com/photos/stiwwe/6238265844/) by Steven Wolf / CC BY-NC 2.0
Slide 4: Image: http://bit.ly/2sPfwFv (https://www.flickr.com/photos/gsfc/3468651131/) by NASA Goddard Space Flight
Center / CC BY 2.0
Slide 5: Image: http://bit.ly/2sivd6R (https://www.flickr.com/photos/scottfeldstein/437129442/) by scott Feldstein / CC BY
2.0
Slide 6: Image: https://www.flickr.com/photos/parksdh/11340519505/ by Daniel Parks / CC BY-NC 2.0
Slide 8: Image: http://bit.ly/2rfLdrx (https://www.flickr.com/photos/corono/2767343921/) by Gareth Davies / CC BY 2.0
Slide 9: Image: http://bit.ly/2srChPm (https://www.flickr.com/photos/collinanderson/2349441498/) by Collin Anderson / CC
BY 2.0
Slide 10: Image: http://bit.ly/2mHdL6q (https://www.flickr.com/photos/pong/2404940312/) by Rob Pongsajapan / CC BY 2.0
Slide 11: Image: http://bit.ly/2scx4ul (https://www.flickr.com/photos/elvissa/505162199/) by Tricia / CC BY 2.0
Slide 12: Image: http://bit.ly/2m9i39Y (https://www.flickr.com/photos/stnorbertcollege/11839410166/) by stnorbert / CC BY-
NC-ND 2.0
Slide 13: Image: http://bit.ly/2rPulGQ (https://www.flickr.com/photos/hoffnungsschimmer/17264753128/) by Lena / CC BY
2.0
Slide 15: Image: http://bit.ly/2s0oo8r (https://www.flickr.com/photos/nseika/8880813114/) by Seika / CC BY 2.0
60. Value
of Academic Libraries
Image Attributions
Slide 16: Image: http://bit.ly/2msQ06a (https://www.flickr.com/photos/governordayton/5572098001/) by Governor Mark
Dayton / CC BY 2.0
Slide 20: Image: http://bit.ly/2scDnhu (https://www.flickr.com/photos/jblyberg/2185724080/) by John Blyberg / CC BY 2.0
Slide 22: Image: http://bit.ly/2rwaKbU (https://www.flickr.com/photos/jeanbaptisteparis/6301697857/) by jeanbaptisteparis /
CC BY-SA 2.0
Slide 23: Image: http://bit.ly/2tFkRMn (https://www.flickr.com/photos/jeanbaptisteparis/6301701751/) by jeanbaptisteparis /
CC BY-SA 2.0
Slide 24: Image: http://bit.ly/2rN05ZK (https://www.flickr.com/photos/beachmobjellies/33241097510/) by beachmobjellies /
CC BY-SA 2.0
Slide 25: Image: http://bit.ly/2sKOzmL (https://www.flickr.com/photos/question_everything/5104987433/) by Let Ideas
Compete / CC BY-NC-ND 2.0
Slide 26: Image: http://bit.ly/2tEVHxh (https://www.flickr.com/photos/kome8/3599392372/) by 8 Kome / CC BY-NC-ND 2.0
Slide 27: Image: http://bit.ly/2tFnNbL (https://www.flickr.com/photos/sakura-harusame/3828276414/) by
PhotographyBySakura / CC BY-NC-ND 2.0
Slide 29: Image: http://bit.ly/2sHxkSr (https://www.flickr.com/photos/marcinignac/7211176930/) by Marcin Ignac / CC BY-
NC-ND 2.0
Slide 30: Image: http://bit.ly/2rILa7G (https://www.flickr.com/photos/hello-sebastian/8207490946/) by Sebastian Sikora /
CC BY 2.0
61. Value
of Academic Libraries
Image Attributions
Slide 36: Image: http://bit.ly/2rNJt4a (https://www.flickr.com/photos/l-i-n-k/3654390818/) by Thomas Link / CC BY-SA 2.0
Slide 38: Image: http://bit.ly/2tFB04a (https://www.flickr.com/photos/taspicsvns/15262863623/) by Vern / CC BY-NC-ND 2.0
Slide 39: Image: http://bit.ly/2tFB04a (https://www.flickr.com/photos/taspicsvns/15262863623/) by Vern / CC BY-NC-ND 2.0
Slide 40: Image: http://bit.ly/2sHVr3v (https://www.flickr.com/photos/47259373@N02/5170090189/) by Mark Strobl / CC BY
2.0
Slide 41: Image: http://bit.ly/2rNqgQ3 (https://www.flickr.com/photos/75905404@N00/2220590528/) by OZinOH / CC BY-
NC 2.0
Slide 44: Image: http://bit.ly/2ssMcSr (https://www.flickr.com/photos/romanboed/20932834149/) by Roman Boed / CC BY
2.0
Slide 52: Image: http://bit.ly/2sHK7EC (https://www.flickr.com/photos/themeowverlord/13941607659/) by Amy Aletheia
Cahill / CC BY-SA 2.0
Slide 54: Image: http://bit.ly/2sPkLoK (https://www.flickr.com/photos/pedrosz/33385238882/) by Pedro Szekely / CC BY-SA
2.0
Slide 55: Image: http://bit.ly/2rNiAgJ (https://www.flickr.com/photos/monkeypainter/8322729745/) by David Ramos / CC
BY-NC-ND 2.0
Slide 57: Image: http://bit.ly/2rNjU3j (https://www.flickr.com/photos/jenosaur/4051305996/) by jen Collins / CC BY-NC-ND
2.0
Image: http://bit.ly/2tF1Mtx (https://www.flickr.com/photos/stiwwe/6238265844/) by Steven Wolf / CC BY-NC 2.0
This project relates to ACRL’s Plan for Excellence.
A major goal for this plan is to help academic libraries demonstrate their value to their parent institutions through their impact on institutional outcomes.
A major institutional outcome is to promote student learning and success.
With decreasing support, the competition for resources becomes more intense and the need for demonstrating value becomes more critical.
Association of College and Research Libraries. (2010). Value of Academic Libraries: A Comprehensive Research Review and Report, researched by Megan Oakleaf. Chicago, IL: Association of College and Research Libraries. Retrieved from: http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_report.pdf
Image: http://bit.ly/2sPfwFv (https://www.flickr.com/photos/gsfc/3468651131/) by NASA Goddard Space Flight Center / CC BY 2.0
Institutions, analysis of individual interviews with provosts from the advisory group’s institutions, priority
Image: http://bit.ly/2sivd6R (https://www.flickr.com/photos/scottfeldstein/437129442/) by scott Feldstein / CC BY 2.0
Image: https://www.flickr.com/photos/parksdh/11340519505/ by Daniel Parks / CC BY-NC 2.0
Analysis of all three data sources informed the identification of six priority areas for future research.
Communication: Communicate with those outside of library & at different levels within the institution
Collaboration: Understand different types & levels of collaboration & consider reviewing literature from related fields to see what is said about libraries & common ground
Mission strategy & alignment: Go outside of library to collect data & seek possible collaborators for common issues; Inform students, faculty, & administrators of how the academic library contributes to the institutional mission and goals.
Teaching & learning: Engage with faculty & students for librarian inclusion in developing academic & everyday life support services for students; Develop educated & informed citizens
Student success: Identify quantifiable student attainment indicators; Work with academic services and faculty
Learning analytics: Measure, collect, analyze & report “data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” ; Include library data with institutionally collected data to predict student success
Brown-Sica, Margaret. “Using Academic Courses to Generate Data for Use in Evidence Based Library Planning.” Journal of Academic Librarianship 39, no. 3 (2013): 275–87. doi:10.1016/j.acalib.2013.01.001.
Fister, Barbara. “Critical Assets: Academic Libraries, A View from the Administration Building.” Library Journal 135, no. 8 (2010): 24–27.
Hess, Amanda Nichols. “Equipping Academic Librarians to Integrate the Framework into Instructional Practices: A Theoretical Application.” Journal of Academic Librarianship 41, no. 6 (2015): 771–76. doi:10.1016/j.acalib.2015.08.017.
Jantti, M. and Heath, J. (2016). What role for libraries in learning analytics? Performance Measurement and Metrics 17(2), 203-210.
Lombard, E. (2012). The role of the academic library in college choice. Journal of Academic Librarianship 38(4), 237–41.
Priority areas are based on findings of:
the selected literature review
the advisory group interview
the provost semi-structured individual interviews
Areas are intentionally broad to foster discussion and input among librarians and others in higher ed.
Effective practices = represent actions library administrators and staff should take to align with and impact institutional goals and objectives related to student learning and success
They are specific and contextual because they came from the three data sources, were further categorized for application in a variety of contexts
Image: http://bit.ly/2rfLdrx (https://www.flickr.com/photos/corono/2767343921/) by Gareth Davies / CC BY 2.0
• Consider how different groups of stakeholders envision effective communication emanating from various forms of media and position the library accordingly. Communication is contextual. A mode of communication that might be successful in one context might fail in another. For instance, one advisory group member noted how social media works well with students, but not with faculty (Advisory Group Member LM09). Regardless of context, library administrators and staff must consider how higher education administrators prefer their information disseminated (Advisory Group Member LM08). Library administrators and staff might consider assigning one librarian to be responsible for communication (Advisory Group Member LM09). Provosts recommend being innovative and giving them a “different style of data” (Provost PP11). This innovation could include guerilla marketing, such as making short videos (Provost PP01).
• Work to dismantle “pigeonholed” perceptions of the library (e.g., as just providing collections) by thinking out of the box and providing new services, spaces, and collections. As denoted by the provost interviews, key stakeholders within their academic institutions hold several myths about the library. As a result, provosts indicated the importance of library administrators and staff communicating “how they bring people in touch with knowledge in various ways” (Provost PP02). One provost suggested that library administrators and staff could dispel these myths by inviting these stakeholders into the library. Specifically, administrators and staff could hold a town hall to address the question, “What is the role of the library today?” Attendees can formulate an institution-specific response to this issue, which can yield buy-in from all those attending (Provost PP04).
• Take advantage of the fact that, unlike other departments, the library serves all students and majors. One of the questions the team asked provosts was what library administrators and staff would need to do to enhance the possibility of the provost accepting a modest funding request from the library. Four provosts stated that the library was at an advantage when making this request. As provost said, the library is in an advantageous position when it comes to funding requests since it has “impacts beyond borders” (Provost PP01). Library administrators and staff can take advantage of this impact by articulating how they benefit key institutional stakeholders when making a funding request.
Image: http://bit.ly/2srChPm (https://www.flickr.com/photos/collinanderson/2349441498/) by Collin Anderson / CC BY 2.0
• Be open to adopting less traditional roles for services, collections, spaces, and staff to fulfill the strategic mission of the university. While specific activities will vary based on institutional context, one advisory group member recounted an example of how the administrators and staff at her library began sponsoring three-minute research video competitions among graduate students (Advisory Group Member LM03). This action increased the research mission of the university and drew attention to the library’s role in advancing this mission because students and faculty shared the videos. The provost from this advisory group member’s institution also brought up this initiative as exemplifying an effective practice (Provost PP03).
• Be aware of student and faculty demographics and respond to their needs and characteristics. Provost PP09 denotes the importance of library administrators and staff knowing their customer. She suggests that library administrators and staff marshal resources and design events that interest people. To develop interest, they must uncover a need and demonstrate how the library can satisfy this need. Provosts identified several approaches that library administrators and staff could take to exercise this awareness and response. Provost PP04 suggests that library administrators and staff offer hours in the library space where students can come and work with a tutor. In the context of the provost’s institution, this action assists the students in the state that has a prevalence of six-year, rather than four-year graduation rates. Provost PP08 identifies the importance of diversity and inclusivity within higher education by contending that library administrators and staff should accommodate students with low socioeconomic statuses to demonstrate the importance of the “university as a gateway to a better life.”
Image: http://bit.ly/2mHdL6q (https://www.flickr.com/photos/pong/2404940312/) by Rob Pongsajapan / CC BY 2.0
Pull the sort of data to track user behaviors that library administrators and staff have shied away from in the past. Provosts suggest that library administrators and staff must collect data on the holistic student experience including longitudinal metrics related to the student experience during and after graduation (Provosts PP03, PP11). One provost noted that library administrators and staff could be informed of new metrics to collect by referring to those obtained by independent, for-profit institutions (Provost PP14). An advisory group member suggested that library administrators and staff can initiate conversation with provosts about the types of user data they can collect categorized by departments (Advisory Group Member LM14).
Use the data environment employed by senior leadership. This practice was suggested by an advisory group member (Advisory Group Member LM13). It ties into a key finding from the team’s review of all three data sources that a gap exists between how library administrators and staff versus provosts communicate the library’s value. Implications of this finding for library administrators and staff are to ensure that they are using the same terminology employed by their provosts. This effective practice builds on this implication by suggesting that non-verbal forms of communication, such as quantitative data, must also be presented by library administrators and staff in a way that is recognized by provosts.
Image: http://bit.ly/2scx4ul (https://www.flickr.com/photos/elvissa/505162199/) by Tricia / CC BY 2.0
Measure, collect, analyze and report “data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” Library data should be included in the volumes of data collected from multiple systems within the academic institution and statistically analyzed to predict student success. In addition to the exemplar study in this priority area and the previous priority area, several other research studies from the fields of LIS and higher education exist that discuss privacy and confidentiality, and how to implement policies for both.
Select student success indicators that consider the diverse experiences of students. Library administrators and staff need to be creative in identifying outcomes that address the needs of non-traditional students. For example, a possible outcome might include the use of library services, collections, and spaces by non-traditional students, or the cost savings identified for students by the development of a LibGuide for open access resources.
Image: http://bit.ly/2m9i39Y (https://www.flickr.com/photos/stnorbertcollege/11839410166/) by stnorbert / CC BY-NC-ND 2.0
Support student engagement with library services, spaces, and collections by aligning with related programs that require them. Library administrators and staff should collaborate across departments to leverage library services, spaces, collections, and expertise to assist students in first-year experience programs (Advisory Group Member LM06, Provost PP08). Advisory group members and provosts mentioned the importance of programs, such as first-year learning experiences, to advancing the institutional mission and goals. Given that these programs are cross-departmental, the library can provide centralized grounds for students in these programs to learn and collaborate.
Keep abreast of higher education trends. As mentioned in the Discussion section, one way that library staff and administrators can contextualize their contributions to student learning and success is by how these contributions address emerging trends and issues in higher education and libraries. Some of the trends identified by provosts include: fostering critical competency skills (Provosts PP03, PP10), supporting continuing learning and educating students to become “informed citizens” (Provosts PP06, PP14), and changing people’s perceptions of the library as a storehouse for collections (Provost PP02).
Image: http://bit.ly/2rPulGQ (https://www.flickr.com/photos/hoffnungsschimmer/17264753128/) by Lena / CC BY 2.0
Identify needs of students and partner with related departments to meet them - think outside of the box. As advised by one provost, library administrators and staff must “reach across a variety of disciplinary areas and identify those particular areas that might benefit from a cooperative activity because the library must work with everyone. I think that’s the biggest challenge - to reach across a variety of disciplinary areas” (Provost PP10). Several provosts gave examples of how library administrators and staff could accomplish this cooperative activity. One example identified by the team as one that also exhibits thinking outside the box is to have library administrators and staff provide or support efforts such as career counseling (Provost PP05).
Use library space to provide central meeting grounds for programs across departments. As suggested by one provost in the Discussion section, library staff and administrators should not engage in “turf wars,” but rather promote the programmatic integration of other departments into the library space (Provost PP01). This integration can be accomplished by housing a cross-disciplinary program and allow spaces for students to learn and work (Advisory Group Member LM03), creating a teaching commons for faculty to engage in programs to improve instruction (Advisory Group Member LM13), and engaging in interdisciplinary work by bringing different departments into the space (Provosts PP03, PP08), among other activities.
Partner with institutions outside the university or college, such as government and commercial institutions. It not only is important for library administrators and staff to collaborate with those internal to the institution, but also bring in those external to it. As an example, one provost suggested that library administrators and staff allow community groups outside the institution to come into the library and give them space in which to engage (Provost PP13). This practice also aligns with a higher education trend identified by provosts, which relates to the growing decentralization of students from their physical institutions due to factors such as the increase in distance learning and the rising importance of continuing education. For these reasons, integrating the outside community into the institution can engender buy-in from those who not geographically linked to the institution.
Examples of RQs for each priority area are shown below.
Image: http://bit.ly/2s0oo8r (https://www.flickr.com/photos/nseika/8880813114/) by Seika / CC BY 2.0
Image: http://bit.ly/2msQ06a (https://www.flickr.com/photos/governordayton/5572098001/) by Governor Mark Dayton / CC BY 2.0
Image: http://bit.ly/2scDnhu (https://www.flickr.com/photos/jblyberg/2185724080/) by John Blyberg / CC BY 2.0
Image: http://bit.ly/2rwaKbU (https://www.flickr.com/photos/jeanbaptisteparis/6301697857/) by jeanbaptisteparis / CC BY-SA 2.0
Context
Who is the audience?
What knowledge, experience, and possible bias might the researcher/ bring to the research?
How might the institutional and library environments affect the research?
Topic/Problem
What does past work/literature say about the topic?
What gaps in the research exist on this subject and how could this research design bridge those gaps?
Image: http://bit.ly/2tFkRMn (https://www.flickr.com/photos/jeanbaptisteparis/6301701751/) by jeanbaptisteparis / CC BY-SA 2.0
Design
What elements of qualitative and quantitative research can help answer the research questions?
How can triangulation using mixed or multiple methods create a more complete picture of the phenomenon studied?
Is there a clear explanation as to why the researcher chose the selected approach instead of the alternatives?
Image: http://bit.ly/2rN05ZK (https://www.flickr.com/photos/beachmobjellies/33241097510/) by beachmobjellies / CC BY-SA 2.0
Data Sampling
Who would be most willing and able to participate in the study and provide the most useful data?
How can potential library users, who might not be using the collection, space, or service being considered by the researcher, be recruited?
Are there any gatekeepers that would be able and willing to forward the recruitment notice to relevant populations?
Data Collection
What are the possible ways that the data collected can be triangulated to give a more complete picture of what phenomenon the researcher is studying?
Is the rationale for the type of data collected clearly explained and justified?
How does one ensure the collection of relevant and critical data?
Image: http://bit.ly/2sKOzmL (https://www.flickr.com/photos/question_everything/5104987433/) by Let Ideas Compete / CC BY-NC-ND 2.0
Data Analysis
Do the results of the analysis answer the research questions? If not, which aspect of the research design need to be modified and repeated?
Are the data analysis methods appropriate for the qualitative and quantitative data collected?
Does the data analysis reporting meet the necessary (e.g., qualitative, quantitative, statistical) standard?
Image: http://bit.ly/2tEVHxh (https://www.flickr.com/photos/kome8/3599392372/) by 8 Kome / CC BY-NC-ND 2.0
Discussion
Are the most relevant findings clearly stated?
What is the best way to reiterate and frame the findings?
How do the findings of the study compare and relate to past work/literature?
Future work
How can the limitations of the study be clearly stated?
How can the implications of the work be connected to suggestions for future research?
Image: http://bit.ly/2tFnNbL (https://www.flickr.com/photos/sakura-harusame/3828276414/) by PhotographyBySakura / CC BY-NC-ND 2.0
Reporting
What non-LIS venues should researchers consider for sharing LIS research?
Is the information clearly reported when sharing the research?
Is the information being shared relevant to the audience?
Image: http://bit.ly/2sHxkSr (https://www.flickr.com/photos/marcinignac/7211176930/) by Marcin Ignac / CC BY-NC-ND 2.0
Usability Testing played a crucial role in the design and development of the visualization component of this project. [Read from slides.]
Image: http://bit.ly/2rILa7G (https://www.flickr.com/photos/hello-sebastian/8207490946/) by Sebastian Sikora / CC BY 2.0
This is the basic layout of the literature search tool. Documents are annotated using various attributes such as, e.g., Year Published or Geographic Location of Institution, and the facets (left) allow users to quickly find documents based on these attributes. A keyword/Boolean search tool (top) indexes document metadata (titles, abstracts, attributes) and shows a real-time summary of what’s in the database. A breadcrumb metaphor (bottom) allows a user to refine a query and explore the database interactively. Easy access to saving and sharing references lists is done with the green button.
The Charts and Graphs tool is inspired by the Tableau software for creating data visualizations. The interface is Drag-and-Drop.
The left column lists all of the available document attributes. The user drags attributes from this left panel on to the center panel…
…to create a chart (shown in the right panel). Little checkboxes work in the same way that the “Filter” options do in Excel.
Once a chart is created, it can be exported as an image file (easy to email/share), shared via permalink, or saved for editing later.
In the event that the user prefers using tools such as Microsoft Excel to do charting or number crunching, the “Export as CSV” button will allow the user to open the chart’s data in Excel (or any other CSV-capable tool) and work with charting/analysis in those tools if desired.
Context menus on the chart area allow customization of the chart’s visual attributes that are independent of the data.
Image: http://bit.ly/2rNJt4a (https://www.flickr.com/photos/l-i-n-k/3654390818/) by Thomas Link / CC BY-SA 2.0
Usability Testing is a methodology used to evaluate a product by observing users as they interact with it. Just as an ethnographer would observe the interactions of people with their environment, usability testers observe the interactions of users with a product.
Bloom, Nathanael. 2010. “Ethnography in UX.” UX Matters. Retrieved from: http://www.uxmatters.com/mt/archives/2010/06/ethnography-in-ux.php
Connaway, Lynn Silipigni & Marie L. Radford. 2017. Research Methods in Library & Information Science (6th ed., pp. 277-282). Santa Barbara, CA: Libraries Unlimited.
Why is usability testing such an important and beneficial activity?
Well, it turns out that human-computer interaction is a lot like human-human interaction.
Expectations: When we get along (with each other, or with our software), it is usually the case that we are aware of each others’ expectations, and we choose not to violate those expectations. Or, we are unaware of each other’s expectations but make the right assumptions. We can predict how each other are going to behave, and we make an effective team.
Confrontation: When things take a turn for the worse, someone gets frustrated and a confrontation can arise. This is usually because the offending party violated the expectations of the offended party, either deliberately or accidentally.
Resolution: To move forward with life, we have to communicate/negotiate our expectations to each other (social contracts) and agree to honor them by not violating each others’ expectations.
Avoiding confrontation: By clearly spelling out our expectations to each other, we can reduce the probability of frustration/confrontation, and increase the probability of working as an effective team.
Have you ever yelled at your computer? I know I have.
Has it ever yelled at you? (Pop-up error messages when you’re trying to do something you think is perfectly reasonable?)
Image: http://bit.ly/2tFB04a (https://www.flickr.com/photos/taspicsvns/15262863623/) by Vern / CC BY-NC-ND 2.0
Designing a usable software application may seem like a triviality. But in reality there are myriad sets of expectations that mold a piece of software. Additionally, there are hard constraints (budget, expertise, schedule, physics/thermodynamics) at play here too.
Here is just a small sample of the possible sets of expectations at play. Which set(s) ultimately matter?
Image: http://bit.ly/2tFB04a (https://www.flickr.com/photos/taspicsvns/15262863623/) by Vern / CC BY-NC-ND 2.0
Users, users, users. The user must be the center of the universe. The software exists for them. Software in the life of the user.
How often does an actual user get any say in how a piece of software works? If they did have a say, do you think they would be able to accurately guess the most effective design?
Image: http://bit.ly/2sHVr3v (https://www.flickr.com/photos/47259373@N02/5170090189/) by Mark Strobl / CC BY 2.0
The trouble is that (1) what users want, (2) what they think they need, (3) what they really need, and (4) what can be delivered are often subject to an impedance mismatch.
Thus, usability studies focus on successful completion of tasks to determine what it is that users really need to minimize their own frustration. Software is molded to meet the expectations that users exhibit based on how they interact with the software on their own terms.
The “It just works!” factor is an example of successful minimization of user frustration. When a design is super effective, it is effortless to use. When a design needs improvement, the friction manifests as user frustration.
Image: http://bit.ly/2rNqgQ3 (https://www.flickr.com/photos/75905404@N00/2220590528/) by OZinOH / CC BY-NC 2.0
The members of the research project have worked in tandem with the OCLC Usability Lab on quite a few projects. The contribution that the OCLC Usability Lab has made to our research is substantial.
We understand the value of ethnography, and it is only natural to use those same kinds of principles in the form of usability testing to mold our software and serve our users.
How many users should there be in a usability study? From Wikipedia (https://en.wikipedia.org/wiki/Usability_testing#Number_of_test_subjects):
On Jakob Nielson of Sun Microsystems: “Once it is found that two or three people are totally confused by the home page, little is gained by watching more people suffer through the same flawed design. "Elaborate usability tests are a waste of resources. The best results come from testing no more than five users and running as many small tests as you can afford." Nielsen subsequently published his research and coined the term heuristic evaluation.”
Wikipedia, s.v. “Usability Testing,” last modified June 13, 2017, https://en.wikipedia.org/wiki/Usability_testing.
Six users were recruited using purposive (non-probability) sampling. Only one user in the sample had any prior experience with the AiA website that features a similar literature search feature.
Image: http://bit.ly/2ssMcSr (https://www.flickr.com/photos/romanboed/20932834149/) by Roman Boed / CC BY 2.0
We had a total of four usability tasks. I used to teach CSE 101 and CSE 200 (Excel/Access), and I KNEW that #4 was going to be a doozy. The challenge would be to come up with an interface that would be obvious to Tableau users, and simple enough to be learnable when combined with some modest training materials otherwise.
Generating graphs with an unfamiliar tool is a difficult challenge (e.g., Excel pivot tables, MS Access visual query tool, Tableau, SQL queries, LINQ, Python list comprehensions, etc.). We try to provide the most learnable tool while giving users an escape hatch to chart data in Excel or another tool if they get stuck. Familiarity wins.
From the literature search tool usability tasks, we learned some unexpected things. [Read from slide.]
Here is the literature search interface during early usability testing.
We move the search bar from the upper-left corner…
…to up top, front and center.
Once doing this, we completely overhauled the way that the search box works. The end result elegantly satisfies all of the usability pain points:
All of the facets appear when you type text in the search bar. This works very much like the Amazon search bar.
The ”shape” of the database is a lot more discoverable with the real-time search feedback. This has the effect of helping to discover facet names, and reducing the time it takes a user to recover from a dead-end search that yields zero results.
Boolean search and keyword search functionalities coexist seamlessly. A keyword search is implicitly converted to a Boolean AND query. Explicitly using Boolean syntax (AND, OR, NOT, quotes for phrases) works as expected.
HOWEVER, just because we found an elegant solution does not mean that it was easy to think of it. We tried several different designs and worked with an OCLC web designer to engineer the solution.
AND, sometimes a seemingly small change makes a huge difference in usability. In this case, moving the location of the search bar.
Many of our users were confused about what they were seeing. In our prototype, names such as “Data Fields” and “Chart Fields” appeared out of the blue, and had no relationship to the literature search tool. Changing the names of these widgets to “Document Attributes” and “Chart Parameters” closed the loop. Also, terms such as “rows” and “columns” as shown were not clear – changing them to “Breakout X By” and “Breakout Y By” are less ambiguous and will make sense provided some additional documentation.
Image: http://bit.ly/2sHK7EC (https://www.flickr.com/photos/themeowverlord/13941607659/) by Amy Aletheia Cahill / CC BY-SA 2.0
Image: http://bit.ly/2sPkLoK (https://www.flickr.com/photos/pedrosz/33385238882/) by Pedro Szekely / CC BY-SA 2.0
The final version of the software allows users to upload and annotate their own articles (kept sandboxed in their own ”local copy” of the database). To implement this feature, we completely overhauled the data structures and algorithms that power the literature search tool, as well as the internal organization of the document database.
Also, the Charts and Graphs tool layout and functionality was heavily revised to take into account the findings from the usability study. Additionally, features have been added such as charting other user data that can be imported into the interface. This additional functionality is largely made possible by the underlying Vega Lite visualization engine. We are grateful to the University of Washington Interactive Data Lab for providing this excellent tool to the community.
Image: http://bit.ly/2rNiAgJ (https://www.flickr.com/photos/monkeypainter/8322729745/) by David Ramos / CC BY-NC-ND 2.0
Image: http://bit.ly/2rNjU3j (https://www.flickr.com/photos/jenosaur/4051305996/) by jen Collins / CC BY-NC-ND 2.0