This presentation was provided by Rachel Lewellen of Harvard University during the NISO event, Assessment Practices and Metrics in the 21st Century Training Session Four held on Friday, November 9th.

1. Formulating, Interpretation and
Presentation of Data
Rachel Lewellen
Head of Assessment and Program Management
NISO Webinar November 9, 2018

2. Introduction
• Data visualization
• Correlational and causation
• Benchmarking
• Data discomfort

3. Starr Hoffman – October 26, 2018

4. Frankie Wilson - November 2, 2018

5. Sample Visualizations
• Categories
• Maps
• Time
• Numbers

6. Context Matters
• Relative to total
• Relative to campus
• Relative to peers
• Relative to capacity (spaces)
• Relative to other factors (per student, per
faculty)

7. Categories
• Who
– Undergraduates
– Graduates
– Faculty
– Staff
– School, college or
department
– Other demographic
categories
• What
– Books
– Digital collections
– Images
– E-resources
– Equipment
– Rooms
– Services

8. Bar Charts

9. Stacked Bar

10. Pie, Donuts, and Treemaps

11. Mapping
• On-campus
– In library spaces
– In other spaces
• Off-campus
– Local
– Far away
• Collaborative or quiet study
• Open stacks, storage, or partner libraries

12. https://guides.lib.vt.edu/VTechWorksStatistics

13. Campus Map

14. https://visualibrarian.wordpress.com/2016/10/07/library-space-assessment-in-
tableau-a-step-by-step-guide-to-custom-polygon-maps-and-dashboard-actions/

15. Art Traffic at the Louvre
A study of visitors’ behavior using Bluetooth data
http://senseable.mit.edu/louvre/#
https://youtu.be/rRjWaH1QHJU

16. Time Series Data
• Time of day
• Day of week
• Week number
• Month
• By semester
• During extended hours
• By cyclical periods (exams, spring break, grant
periods)
• Calendar year/fiscal year
• Project tracking

17. Line Chart

18. Monthly and Running Total

19. https://library.buffalo.edu/aboutus/about-the-libraries/factbook/roombooking.html

20. Highlight Table

21. Time of Day and Day of Week

22. Heat Map

23. Gantt Chart

24. Numerical
• Total (sum) – items,
dollars, terabytes,
images, feet
• Difference
• Unique counts
• Mean (average)
• Median (middle)
• Mode (most frequently
occurring)
• Percent change
• Percent of total
• Year over year
• Percent capacity
• Distance to goal
• Cost per use
• Other ratios
• Key performance
indicators (KPI)

25. Bar-in-Bar

26. Table Data

27. KPIs

29. Survey Data
• Satisfaction
• Importance
• Interest
• Skill
• Learning
• Value

30. Survey Results
https://about.library.ubc.ca/assessment/

31. Survey Results

32. Small Multiples

33. Satisfaction and Importance

34. https://extremepresentation.typepad.com/blog/2006/09/choosing_a_good.html

35. https://www.techprevue.com/wp-content/uploads/2016/12/abela-chart-chooser.jpg

36. Know your audience
Delivering Business Intelligence with Microsoft SQL Server 2016, Fourth Edition by Brain Larson.
Page 16.

37. Operational Manager

38. Middle Management

39. Executive Management

40. Correlation and Causation
xkcd – a webcomic of romance, sarcasm, math and language
https://xkcd.com/552/

41. The danger of mixing up causality and correlation: Ionica Smeets at TEDxDelft
https://youtu.be/8B271L3NtAw

42. Evidence to Establish Causality
• Association
– Is there a relationship, is there a correlation?
– Is x related to y?
• Direction of influence
– Temporal factors (chicken and the egg)
– Does x come first?
• Absence of alternative explanations
– Non-spuriousness

43. Randomized Controlled Studies
https://hsl.lib.umn.edu/biomed/help/understanding-research-study-designs

44. Randomized Double Blind
Controlled Studies

46. Benchmarking
• Benchmarking is a comparison of performance
measures between similar entities and/or
against recognized standards.
• Library benchmarks are typically comparisons
of numerical (quantitative) statistics such as
circulation, visits, and revenues.
https://ivygroup.com/blog/benchmarking-library-performance/

47. How Benchmarking Works
• Select an aspect, service or process to benchmark
• Identify the key performance metrics
• Choose comparison group to benchmark
• Collect data on performance and practices
• Analyze the data and identify opportunities for
improvement
• Adapt and implement the best practices, setting
reasonable goals and ensuring organizational
acceptance

48. Examples of Library Benchmarking
• Resource Comparison
– Funding levels
– % of budget on collections or format
– Size and type of staff
• Service models
– Reference service models
– Writing center collaborations
– Learning Commons
– Copyright services
• Reorganization
– Administrative structure
– Department alignments
• Best Practices
– Recruitment and retention
– Improve printing
– Staff training programs

49. Comparison Groups
• Institutional defined peer group
• Demographics
• Within consortia
• Other characteristics
• Leaders or by reputation

50. Sources of Benchmarking Data
• Integrated Postsecondary Education Data
System (IPEDS)
• The Association of College and Research Libraries
(ACRL)
• Association of Research Libraries (ARL)
• Standardized surveys (LibQUAL, MISO, Ithaka,
National Survey of Student Engagement (NSSE),
Common Data Set (CDS)
• Email, phone calls and site visits

51. Data Discomfort

52. Build a Culture of Assessment
• Takes time
• Transparency with data
• Communication about data use
• Highlight examples of data use
• Build trust
• Data as information not as a judgement

53. Practices
• Explicitly acknowledge when data is used
– Highlight positive outcomes big and small
• Support data allies and champions
– Honor and recognize data practitioners and
practices
• Normalize everyday data use
– Include data competencies and practices in all job
descriptions
• Communication and transparency

54. Interpersonal Effectiveness
• Validate and understand concerns
• Acknowledge limitations of data when
appropriate
• Be trustworthy
– Professional credibility
– Personal credibility
• Be reasonable
• Be a leader
• Best change management strategies

55. Thank you!