Presentation given in Glasgow on June 15 at the workshop 'Moving from Impact to Value', organised by the Scottish Network on Digital Cultural Resources Evaluation
2. Europeana Business Plan 2017 Workshop / May 17, 2016
writing a sustainability plan = asking
for crutches
3. Europeana Business Plan 2017 Workshop / May 17, 2016
assessing impact = systematically
understanding the positive change
that you are evoking in the lives of
people
The title of my presentation is a paraphrase of a book I highly recommend: Culture eats strategy for lunch (by Curt Coffman). It implies that what is eaten for lunch just doesn’t resonate anymore. There is a new predator who is bigger than you, your time is up!
Is it like in fashion where such a line pops up every season? Is it as banale as saying: ‘orange is the new black’?
If say you write a sustainability plan these days, you are essentially asking for support of what you are doing, without necessarily telling why it is worth supporting. And that doesn’t cut the mustard anymore.
While if you say you are doing an impact assessment these days, people may think you actually know what you are talking about ;-). You are indicating that you are prepared to dive deep into the matter and are concerned with the real results.
To me it comes down to the same thing: you are in the unfortunate position that you are very passionate about what you do, but you have to compete for resources with others who may be just as passionate as you. So you better have a decent story to tell!
So how do we approach this issue at Europeana? An interesting starting point to lead you into this, is to show you a short film. Armed with a lightweight camera and some pencils we asked asked a couple of people in our network point blank: what is your dream? So sit back and relax….
(end of film). You may have recognised Nick Poole in this story, then chairman of our network.
We have translated Nick’s dream, and the dreams of many others, into one mission statement: “We believe that we can transform the world with culture”. Quite a statement for what is essentially a metadata shop….
So how do we know that we are making progress against that big aim?
[Like any self-respecting organisation, we have developed a strategic plan which provides a pathway towards a beautiful world a couple of years ahead, that we will reach if we keep ourselves strictly to the plan. Or so we think. Because evidence shows that in the 21st century, strategy is nothing more than a pattern in a stream of decisions (Mintzberg). ]
We felt that we needed to approach this methodologically if we want to have any chance at succeeding. So we went shopping. We were lucky that such a model was being developed right around that time by Dr. Simon Tanner of Kings College, to whom we owe a lot. With a little adaptation we could make it our own…
Balanced we interpreted as not ‘singular’, balancing the expectations from our various (often political) stakeholders that their investment should have impact on on a social/cultural level, an economic level and on our ability to make our industry more innovative, apt to deliver on the challenges of the future.
Values are the things that drive everything that we do, the culture we nurture that make us different from everyone else. At Europeana are driven by making digital culture as usable as possible, in a reliable way and to the mutual benefit of the cultural institutions that take care of our heritage and the people who want to use that heritage for work, learning or pleasure. What this means in practice is that we are obsessed with making data available as openly as is legally possible but with correct attribution and and that we provide statistics to all contributing institutions.
This is in my opinion the biggest contribution from Tanner: he made us think beyond the mere output of our activities, towards what we believe the outcome of these activities will have, the changed behaviour on the receiving end. It made us aware that it wasn’t the digitised data that we collected about the first world war, it was the stories that connected people and the deeper understanding that this activity led to that provides the evidence of our success.
So this is how we started to look at our activities. When we lobby for copyright reform, will that result in stronger, more adaptive heritage institutions? When we make openly licensed material available to the creative industries, will this result in new businesses and economic innovation?
We we decided to put our framework to the test on one of our most successful projects, the 1914-1918 collection days. This project has been running since 2013, and since we have collected almost two hundred thousands postcards, diaries, photographs helmets from thousands of people in over 20 European countries.
But how do you do that? What questions do you ask and how do you evaluate the results? Tanner’s model gave us some clues but his notions are- and he will hate me for saying this - not easily translatable to questions you can ask normal people. So we refactored them a little ;-)
On the one hand we had the two roadshows we had planned in May in Prague and in Poznan (Poland). In video you see Ms. Elsbieta Sobiak, who brought to us photographs of her grandfather who got injured in Verdun. She recalled that she remembers him vividly, crying while singing German songs when at work. Nothing particularly remarcable or unique from a historic perspective. But as you can see she was clearly very moved when she shared her story. It meant a lot to her, and she had gone through a lot of trouble to make sure that her story would become part of the larger European story that we want to tell.
This is the human interest side of the story, still familiar territory for most of us. But we also wanted hard data, evidence that would go beyond the soft side of the story. Now this is a completely different ballgame. We are now entering the realm of statistics, ruled by data crunchers. So we engaged a band of professionals, created a survey, and started interviewing some of the contributors. We are still digesting the data, but I am going to share some of the preliminary results with you.
We sent the survey out to about 5.000 people on our mailing list and shared it widely on our social media channels. We asked people to help us understand what the service means to them so that we can improve it.
In total we got 1,517 responses- not a bad results by any standard.
After radical sanitation (we took out what is called ‘speeders’, people who complete a survey far faster than average) we were left with 393 what we consider ‘valid’ responses.
Note that because we reached 68.357 people on our site this sample represents 0.58% of users. This means our sample is considered indicative but not fully representative of our users.
I interpret this as a good result.
This is interesting, as it suggests that while the respondents believe it is very important for them, they think it is much less so for their friends who may have less of an interest in the subject.
Apparently the respondents that it is crucial for future generations to take lessons from this tragedy. They value it even higher than for themselves!
We need to improve the product!
This is perceived value. I interpret this question as: how much money do you think we should invest as taxpayers in making this service available?
People who identified themselves as users of the service valued the fact that the service exists higher than the ones who don’t use the service, as one would expect. But non-users also put a decent value mark on it. It is like saying: how much is it worth to you that the Scottish National Library exists. You may never go there but value the fact that it is there.
What have we learned?
It is difficult to be scientific about this and give very hard evidence. In the end I don’t believe we can or should try to translate everything that is meaningful to us into numbers. It would be like asking to value on a scale of 1-10 how much you love your partner or children. But I do think that being more systematic in your approach and to be in direct contact with your customers is vital to understand what you are actually achieving. This is like asking your partner if she loves you. You’ll be surprised to hear what she has to say…