Presentation at The Internet Librarian International, 18 October 2016, Olympia Conference Centre.
About the presentation: On behalf of the National Library of Norway, the Buskerud County Library developed an open source solution that allows every library in Norway to measure and compare the performance of library websites. The system is based on Piwik, the leading open source solution for web analysis.
Hello and good afternoon to you all. My name is Jannicke Røgler and I will present a Norwegian pilot project for measuring the impact of websites. I work as a library adviser and my main areas of work are ICT, statistics, collection development and training of library staff.
I’m thrilled to be able to talk to you about a project that I hope have the potential to be an important service for Norwegian libraries.
This is a brief overview of what I will talk about. I will talk a little about the need for consistent measuring, the background for the project and the inspiration we got from Denmark. I will also tell you about why we chose Piwik and not Google Analytics as out platform for gathering traffic data. Then I will introduce you to the library index. I will also say a few words about our results so far. The presentation will conclude with possibilities and wishes for a phase two for the library index.
Why should we measure the impact of websites? Libraries are spending both money and time on our web services. In order to gain return on investment we have to measure, analyse and act on our findings. The library web pages are meant to be dynamic. They are work in progress.
Why measure at the national level? The national library statistics collect traffic data from library web sites once a year. The data they collect are: Number of visits, number of page views and the number of unique users. A yearly report for this kind of data is not very useful if you want to develop your services or compare yourselves with other similar libraries. We are good at collecting statistics in Norway, but we are not very good at using it.
The starting point for this project was the revision of the national library statistics in Norway. The group started it’s work in 2014 and finished their work last summer. I was a member of the national working group and took a particular interest in reporting libraries digital services.
The main recommendation of the working group was: to develop a library index reporting libraries digital services for a consistent measuring of online performance using free software and open source. As a member of the working group I volunteered my regional library as project owner in developing a library index. Our library has in the recent years developed several digital services, including a dedicated service for lending ebooks.
The whole project took only about two months to develop. We used off the shelf components when possible. Bootstrap is a free and open-source front-end web framework that was used for designing the website. Google charts was used for the line chart. And of course, the service has a responsive web design. The library index is a pilot, in this phase one we measure library web pages only. The pilot is running through the autumn of 2016. Report and recommendations will be delivered to the National Library this winter. Hopefully the library index will be a regular service from the spring of 2017.
When we need inspiration in Norway, we often look to Denmark. The Danish Library Index started in 2008 and is updated on a weekly basis. The index form the basis for reporting to Statistics Denmark. Sadly the Danish solution use proprietary software and is a very costly solution. The Buskerud regional library is a group of competitive people and we set ourselves a hard goal of trying to make an even better service than the Danish for the Norwegian libraries at a fraction of the cost. That goal could only be met using free software.
We already knew that the Danish solution was too expensive for Norwegian libraries. Our choice was between Google Analytics and Piwik. The lack of ownership to our data in the free version of Google analytics made the choice easy. It had to be Piwik. Piwik is the leading open-source analytics platform and is used by more than 1.1 million websites in 150 countries.
We also did a test to compare the two, Piwik and Google Analytics. The conclusion after several tests were that both measurement tools have discrepancies but Piwik scores slightly better in the test. Google Analytics had the greatest deviation in the test.
The service consists of the following parts: The index main page and admin page. Piwik is used for collecting data and as the analysis tool for the participating libraries. I have also started developing help pages with text and video's. The services has no cost for the libraries.
This is the main page of the index. The index measures the following variables:
Unique visitors, Visitors, Pageviews, Average visit duration, Bounce rate, Population of the municipality, Visitors per 1000 inhabitants, Pages per visits and the percentage change from last week
The main page is split into two sections, on top you have the line chart in week view. Below the line chart you have the week view in tabular form.
The solution also has the possibility to compare different libraries. You just tick the boxes for the libraries you want to compare and then a line chart is produced on the fly. All data except population data is updated on a weekly basis. You can chose data by week, month or year. On this print screen I have chosen to show total visits per week for four different libraries.
Every library has it’s own page with the same variables. This example show data from our largest public library, the Deichmanske public library in Oslo.
A simple administration site has been developed. As admin you can add new libraries, admin users and change data for each library such as url and population data. Population data are to be updated annually with numbers from Statistics Norway.
Every participating library has their own Piwik site with all standard statistics reports. The reports are classified into four main analytics report categories – Visitors, Actions, Referrers and Goals. If you already is familiar with the dashboard of Google Analytics you will see that the dashboard is very similar.
I have just started making help pages. One part is for the index and one part is for using Piwik. The goal is to make instruction video's that explain the most important features of the services.
Our regional library is the project owner of Webloft. Webloft delivers websites and plugins to Norwegian libraries. These libraries are the main data source for the index in the pilot period. Webloft is getting new libraries all the time, and as a consequence the library index also expands.
Universal design was an important element when developing the pilot. In Norway every new web site now have to be universally designed.
My hope is that we have managed to develop a versatile solution that can be useful for a variety of institutions. Hopefully other sectors like museums and archives will see the potential the solution has in measuring performance.
The results so far: 50 libraries are participating in the test The index has so far proved to be a stable solution and Piwik has been a stable environment for collecting traffic data. What we need to do now is to start a professional debate about the collected data and what they signify. We don’t know how many visits we can expect on a library website and what is seen as a high number of visitors. The library index make it possible to start comparing our traffic data with other libraries in Norway and maybe also with Danicj libraries since the two countries have a similar solution.
Hopefully there will be a phase 2 where we among other things will be able to: Include other web services like Social media, databases, ebooks… We also need to separate the site by library type, public and academic We would love to test out having real time data from an ILS like the open source library system Koha And of course we would like to disseminate our solution to other sectors and countries.
Thank you for listening and feel free to test the library index and visit our page at GitHub.
Measuring the impact of websites: an open source, national solution
MEASURING THE IMPACT OF WEBSITES :
AN OPEN SOURCE, NATIONAL SOLUTION
PRESENTATION AT INTERNET LIBRARIAN INTERNATIONAL 18 OCTOBER 2016
OLYMPIA CONFERENCE CENTRE
Project Manager Jannicke Røgler, Buskerud Regional Library, Norway
› The need for a uniform and consistent
measuring of websites
› Inspiration from Denmark
› Piwik versus Google Analytics
› Norwegian library index,
› The role of Piwik analytics
› Further possibilities
Measuring the impact of websites
› Measure – monitor over time
› Analyse – understand your users and their
› Act – improve your sites and services
Why measure at the national level?
› Creating awareness
› Return of Investment
› Comparisons with other similar libraries
› Measuring effects of joint campaigns
› Development over time
› Professionalization of digital services
Starting point: Revision of the national statistics
› A working group was set up in 2014, finished 2015
› Three areas were emphasized:
• reporting libraries digital services and collections
• reporting activity and events in libraries
• reporting statistics for combination (school and public)
Statistics for libraries digital services
› The main recommendation of
the working group was:
• to develop a library index
reporting libraries digital services
for a consistent measuring of
online performance using free
software and open source.
• Buskerud Regional Library as project owner of pilot
• Specifications and tender spring 2016
• Technical development by Ignitas May – July.
• Cost technical development - NOK 150 000, Euro 16 700
• Phase 1 – MVP, minimum viable product =
Library home pages only
• Pilot running autumn 2016
• Report and recommendations to The National Library
• Regular service spring 2017 with The National Library as
Google Analytics versus Piwik
› Google Analytics
› Google Analytics is a widely used
› Google Analytics store your data
› The functionality is good
› Free Google Analytics can not be
recommended as a solution
because of the lack of ownership
to the data.
› Premium edition of Google
Analytics allows local storage of
data but costs
$ 150,000 per year (figure from
Piwik is the leading open-
source analytics platform.
The functionality is good.
100% data ownership.
User privacy protection.
Results after 3 months
› 50 libraries are participating in the test
› The index has so far proved to be a stable
› Piwik a stable environment for our input data
› Starting a professional
debate about the impact
of our library websites
Further development – Phase 2
› Include other web services
• Social media, databases, ebooks…
› Separate the site by library type, public and academic
› Include ILS like Koha
› Develop a community of users
› Dissemination to other
sectors and countries