Joan Stein, Carnegie Mellon University and Stephen Town, University of York. Delivered at the 10th Northumbria International Conference on Performance Measurement in Libraries and Information Services, 22-24 July 2013, York, UK.

1. Ten Northumbrias:
contribution and
celebration
Joan Stein,
Carnegie Mellon University
Stephen Town,
University of York

2. Northumbria at Longhirst
Photo CCL
Jim Brodie

3. The origins (1995)
• Northumbria University
• SCONUL Advisory Committee on Performance
Indicators
– “The Effective Academic Library”
• Inspired by the first Board:
– Ian Winkworth
– Geoffrey Ford
– Dick Hartley
– Roswitha Poll

4. Going south …
• Ancient Bernicia to
Deira (or Ebrauc?)
• 1st-3rd Conferences at
Longhirst Hall
• 5th & 6th Conferences at
Collingwood College
• 9th & 10th Conferences
at the University of York

5. Atlantic crossings …

6. Chasing IFLA …

7. Research questions?
From the papers and proceedings …
• Is there a progressive statistics-quality-value development?
• Do some ideas in library performance measurement die?
• Are new ideas taken up and new collaborations formed
subsequently?
• Does performance measurement generate higher level
attention?
• Does any of this have value for, or impact on, library
practice and performance?

8. Growth or continuum?

9. Papers by subject

10. Decline of the quantitative?

11. Service quality product growth

12. Impact and value

13. 1st Northumbria Conference: Word Cloud

14. There from the start …
• “A decade of experience …” (Willemse)
• Organisational effectiveness
– Climate, morale & change
– Staff issues
• Quality, including models and frameworks
• Benchmarking
• Electronic measures
• Stakeholder influences

15. Early conferences
Proceedings of the 2nd Conference:
“ … concern within the profession that
quantitative methods need to be complemented
by interpretation and strategic vision”
“the next conference *will move+ firmly into the
field of strategic management”

16. Authority … or just grumpy?
Wilfrid Lancaster keynote (2nd Conference)
Evaluating the electronic library
“I have witnessed a very considerable decline in the
service ideal within libraries”
“we have gone to far extremes in embracing technology
and have forgotten basic values”
“our best chance of survival as a profession lies in the
opposite direction”

17. 5th Northumbria Conference : Word Cloud

18. LibQUAL+ and successors
2001 (Pittsburgh) 4th
2003 (Collingwood) 5th
Colleen Cook onto the Board
Parasuraman keynote
2005 (Collingwood) 6th
• The first papers
• The first consortium
outside North America
and the first reports of
other languages
• ‘One size does not fit
all’ (Creaser)

19. 10th Northumbria Conference : Word Cloud

20. Recent conferences, and this one …
• Value and impact
– Big(ger) data
– Correlation
– Advocacy, rather than ROI
• Quality and culture
– Link to mission
– Organisational and ethnographic analysis
– Long term sustainability

21. JOAN’S REFLECTIONS & THEMES

22. Initial impressions
• Approached venue on narrow, one-lane secluded paths
– abduction by taxi?
• Overwhelmed by stately home and grounds – nothing
else for miles; yikes! Should I just return to the airport?
• Warm welcome by gracious hosts of conference –
maybe I hadn’t been marooned after all?
• Lively interactions between attendees carried on into
the night and into the bar – data junkies could party!
• I loved performance measurement! (whatever it was)

23. Personal Reflections
• First Northumbria: the Americans – two keynote
speakers and me (what have I done?)
• Serendipity – from ILL to performance measurement
• No concept of performance measurement; my
presentation was loosely related
• Focused my interest, changed my trajectory
• Returned to work ready to apply new ideas
• Unique experience, particularly in comparison

24. In comparison:
My first ALA Conference
• Over 10,000 people
• Giant convention center
• Never meet your hosts
• People wander in and out of
presentations
• People are scattered around
the city and have to plan to
meet each other
• Topics all over the map
1st Northumbria Conference
• @120 people
• Beautiful stately home on 75
acres of parkland
• Small group of gracious hosts
(who bought you a drink!)
• Presentation rooms doors
closed at the start; everyone
stayed and asked probing,
challenging questions after
• Attendees spent non-conference
time together in the Sticky
Wicket bar or out walking
• Focused

25. Broadening the reach:
• Slow to spread to the United States; not marketed there
in early days; relied on word of mouth
• Fourth Northumbria Conference on Performance
Measurement in Libraries and Information Services
held in Pittsburgh, PA in 2001
– Joint venture between the Northumbria Conference board,
me (as local organizer), and ARL
– Helped raise awareness and change the balance of attendees
from mostly European to a mix of nationalities

26. 4th Northumbria in Pittsburgh
• Brought new perspectives of researchers & practitioners in the
United States (i.e. Steve Hiller, Jim Self; Libqual papers by
Colleen Cook, et.al.)
• Tried to maintain the close atmosphere of Longhirst conferences
• Notable in particular for dinner speech by Stephen Town
– “Charmed, I’m sure.”
– Sex, politics, and death
– Novel perspective on performance measures in use when library-
directorships were male-dominated

27. Influential early key-notes
• Focusing on early presentations: reflected growth and
change in the field
• Those that showed us a way forward or were predictive
(and right!)
• Worth briefly reviewing key points to see where we
came from
• Think about these in terms of what we know and do
now in the field

28. Nancy Van House
• Well-known for book on measuring inputs, outputs, goal
attainment in public libraries in America
• First Northumbria: “Organizational Politics & Performance
Measurement” or “On Not the Mistaking the Finger for the
Moon”
• Main points:
– Change
– Competition
– Evaluation as how we construct sense in an organization (narrative;
making meaning)
– Context: library, larger organization, the user

29. Nancy Van House
• Discusses the value of the library, user-centered measures, the politics
of performance measurement and the power of stories
• Empirical evidence & narrative = quantitative & qualitative
• From inputs/outputs to user-centered measures or risk irrelevancy
• Competition for turf – our competitive advantage is understanding
users
• Evaluation highly political because it involves allocation of resources
• Context is crucial – library, parent organization, users
• Evaluation: an iterative process of sense-making; how we show what is
important & explain our importance; telling the story
• Avoid mistaking the measurement for the performance, the library for
the information

30. Rowena Cullen
• Researcher and professor from New Zealand
• Second Northumbria: “Does Performance Measurement
Improve Organizational Effectiveness? A Post-Modern
Analysis”; deconstructs discourse of evaluation
• If we really knew what an effective library was and could
measure it, we wouldn’t keep coming to these conferences
talking about different measures.
• If performance measures had value, shouldn’t they be easier to
define and adopt; wouldn’t they validate our assumptions?
• Performance measurement rests on the concept of “library”
which is changing/evolving; this uncertainty leads us to try to
create meaning with our measures

31. The Matrix
• Tore down existing paradigm and proposed a new one
• Placed measures into a conceptual framework:
values/focus/purpose matrix
• To position your library on matrix, must understand multi-
dimensional nature of performance & measurement; this would
ensure that performance measurement led to library
effectiveness
• Factors that produce positive outcome for measures:
– Recognize political nature and measure what stakeholders value
– Conduct multi-dimensional measures to understand needs
– Rewards and incentives drive measurement

32. Peter Brophy
• 4th Northumbria: “Performance Measures for 21st Century
Libraries”
• Networked environment fundamentally changed our business
• Proposed five models of future libraries:
– A: traditional library - B: memory institution - C: learning center
– D: library as community resource -* invisible intermediary
• Proposed measures for each
• May require new interpretation of “library goodness”; what does
it mean to be a “good” library in the networked world? More is
not better any longer.

33. Peter Brophy
• 7th Northumbria keynote: “Telling the story: qualitative
approaches to measuring the performance of emerging library
services”
• Proposed that libraries must meet and support users where their
life and research occurs, surfacing support embedded in their:
– Workflows - Learnflows - Leisureflows -Lifeflows
• To gain users’ trust: requires deep understanding of their
language and habits; know their terminology, concepts, how they
share meaning
• Academic disciplines are social systems; need to studied as such
• Requires ethnographic approaches and expert assessment
• Use results to “paint rich pictures”; to tell the story. Narrative
again (constructing meaning from complexity)!

34. STEPHEN’S CONCLUSIONS

35. Some conceptual conclusions …
• Is there a paradigm …?
– “The Library” … for good or ill
– Scientific method … rather than humanistic
• Libraries are still organisms …
• Keynotes and plenaries are all very well, but …
• Research is all very well, but …
• “The future is correlation”

36. Northumbria strengths
• A real community of practice, and the first in library
performance measurement
• Research input from Library and Information Schools,
balanced by practitioner developments and
involvement
• International and cross sectoral perspectives
• All levels of organisational involvement
• High percentage of contributors amongst attendees
• Trust in sharing data creates social capital and
enhances collaboration

37. Conference value
• An important and relevant idea (and still ahead of
other professions?)
• Libraries survive, because of evidence based
change
• A strong brand, with a growing set of followers
• A diverse, curious, and visible “invisible college”
• “Just people who make you feel less alone”