Audience analysis and modelling: the case of CIBER and Deep Log Analysis David Nicholas CIBER University College London [email_address] www.ucl.ac.uk/slais/research/ciber

Background CIBER has been working in the media, health, publishing and education fields to help them understand what has happened (and happening) as the result of the digital transition CIBER has done this by portraying, in detail , what goes on in the anonymous and volatile virtual environment CIBER has done this through providing robust and undisputed evidence of what people do in the virtual environment

CIBER has been working in the media, health, publishing and education fields to help them understand what has happened (and happening) as the result of the digital transition

CIBER has done this by portraying, in detail , what goes on in the anonymous and volatile virtual environment

CIBER has done this through providing robust and undisputed evidence of what people do in the virtual environment

CIBER’s methodology: DLA Turn digital activity or ‘noise’ into powerful evidence-based audience (consumer) information Digital noise (as identified in logs) turned into information behaviour which then enables identification of diversity and establishment of best practice and crude levels of satisfaction. In conjunction with demographic/user information (subscriber or survey) usage data transformed into user data. Then – and only then - can identify real satisfaction, impacts and outcomes (The Holy Grail).

Turn digital activity or ‘noise’ into powerful evidence-based audience (consumer) information

Digital noise (as identified in logs) turned into information behaviour which then enables identification of diversity and establishment of best practice and crude levels of satisfaction. In conjunction with demographic/user information (subscriber or survey) usage data transformed into user data. Then – and only then - can identify real satisfaction, impacts and outcomes (The Holy Grail).

It is absolutely essential Findings drive system change - hand and glover with user dynamics. No digital concrete. DLA part of system design In a disintermediated environment, where information & content is ubiquitous and users have complete choice digital services will not stay the course unless kept in constant touch with the user. Digital visibility a key concept

Findings drive system change - hand and glover with user dynamics. No digital concrete. DLA part of system design

In a disintermediated environment, where information & content is ubiquitous and users have complete choice digital services will not stay the course unless kept in constant touch with the user.

Digital visibility a key concept

Particular importance of outcome/impact data Access card has run its course Have to move beyond that warm feeling What is poor or productive and profitable information behaviour? How do we know 24/7 provision is helping us? Are there obvious outcomes associated with it? The car park question!

Access card has run its course

Have to move beyond that warm feeling

What is poor or productive and profitable information behaviour?

How do we know 24/7 provision is helping us?

Are there obvious outcomes associated with it?

The car park question!

Benefits of DLA Size and reach . Massive numbers, no need to take a sample Direct & immediately available record of what people have done: not what they say they might, or would, do; not what they were prompted to say, not what they thought they did. Perceptions of time in surveys very subjective and lags behind the data and people leave their memories behind them in cyberspace. Data are unfiltered and provide a reality check sometimes missing from questionnaire and focus group Data real-time and continuous Creates a fantastic digital lab environment for the monitoring of change Raises the questions that need to be asked by questionnaire and interview. Enables level-playing field platform comparisons

Size and reach . Massive numbers, no need to take a sample

Direct & immediately available record of what people have done: not what they say they might, or would, do; not what they were prompted to say, not what they thought they did. Perceptions of time in surveys very subjective and lags behind the data and people leave their memories behind them in cyberspace.

Data are unfiltered and provide a reality check sometimes missing from questionnaire and focus group

Data real-time and continuous Creates a fantastic digital lab environment for the monitoring of change

Raises the questions that need to be asked by questionnaire and interview.

Enables level-playing field platform comparisons

The digital information footprint Information Seeking Characteristics Activity Metrics User Characteristics 1. Number of pages viewed 2. Number of full-text downloads 3. Number of sessions conducted 4. Site penetration 5. Time spent viewing a page 6. Time spent on a session 7. Number of searches undertaken in session 8. Number of repeat visits made 9. Number of sources used 10. Number of views per source 1. Subject/ discipline 2. Job status 3. Geographical location 4. Name of organisation 5. Type of organization used to access the service 6. User demographics: gender, age etc (if available) A. Type of content viewed 1. Number of sources used in a session 2. Names of sources used/not used 3. Subject of source 5. Age of source used 6. Type of material viewed 7. Type of full-text view 8. Size of source used 9. Publication status of article B. Searching style 1.Search approach adopted 2. Number of searches conducted in a session 3. Number of search terms used in search 4. Form of navigation 5. From where users arrive from 29 Key Features

Snippets from CIBER research Evaluating the usage and impact of e-journals in the UK. RIN, 2008- UK National E-Books Observatory. JISC, 2008-2009 Digital Lives, (with British Library). AHRC, 2007-2009 Behaviour of the Researcher of the Future (Google Generation). British Library and JISC, 2007 Authors as users: a deep log analysis of ScienceDirect. Elsevier; 2005-2006; Maximizing Library Investments in Digital Collections Through Better Data Gathering and Analysis. US Institute of Museum and Library Services; 2005-2007; The digital health consumers of BBC. Department of Health; 2004-2005; E Learning and the World Wide Web – accessibility and participation for people with cognitive disabilities. ESRC, 2004-2006; The Web, the kiosk, digital TV and the changing and evolving face of consumer health information provision: a national impact study. Department of Health; 2000 – 2004; An evaluation of pilot projects exploring the health applications of digital interactive television. Department of Health; 2001–2002.

Evaluating the usage and impact of e-journals in the UK. RIN, 2008- UK National E-Books Observatory. JISC, 2008-2009 Digital Lives, (with British Library). AHRC, 2007-2009

Behaviour of the Researcher of the Future (Google Generation). British Library and JISC, 2007 Authors as users: a deep log analysis of ScienceDirect. Elsevier; 2005-2006; Maximizing Library Investments in Digital Collections Through Better Data Gathering and Analysis. US Institute of Museum and Library Services; 2005-2007; The digital health consumers of BBC. Department of Health; 2004-2005; E Learning and the World Wide Web – accessibility and participation for people with cognitive disabilities. ESRC, 2004-2006; The Web, the kiosk, digital TV and the changing and evolving face of consumer health information provision: a national impact study. Department of Health; 2000 – 2004; An evaluation of pilot projects exploring the health applications of digital interactive television. Department of Health; 2001–2002.

Robots! Best kept secret Around half of all visitors to a scholarly site are robots In case of some AHRC funded sites account for 90% of visitors Makes you realize how things have changed

Best kept secret

Around half of all visitors to a scholarly site are robots

In case of some AHRC funded sites account for 90% of visitors

Makes you realize how things have changed

Navigators Navigating towards content in very large digital spaces a major human activity. People spend half their time viewing content, rest of the time they are trying to find there way to it (or out of it). So many possible routes to content people get lost (excited)

Navigating towards content in very large digital spaces a major human activity.

People spend half their time viewing content, rest of the time they are trying to find there way to it (or out of it).

So many possible routes to content people get lost (excited)

Diversity National differences: Germans the most ‘successful’ searchers and most active information seekers. Canadians and Australians more interested in older content Age differences : older users more likely to come back, and view abstracts. Elderly users had most problems searching – two thirds of searches obtained zero returns! Gender differences : women more likely to view articles in HTML and return to a site (less promiscuous!)

National differences: Germans the most ‘successful’ searchers and most active information seekers. Canadians and Australians more interested in older content

Age differences : older users more likely to come back, and view abstracts. Elderly users had most problems searching – two thirds of searches obtained zero returns!

Gender differences : women more likely to view articles in HTML and return to a site (less promiscuous!)

Brand, don’t go there, there are big problems Difficult in cyberspace: responsibility/authority almost impossible in a digital environment – so many players Also what you think is brand is not what other people think. TESCO! And then there is cool . Facebook!

Difficult in cyberspace: responsibility/authority almost impossible in a digital environment – so many players

Also what you think is brand is not what other people think. TESCO!

And then there is cool . Facebook!

Characteristic information behaviour in the virtual space In broad terms scholarly behaviour can be portrayed as being active , bouncing, navigating, checking and viewing. It is also promiscuous, diverse and volatile Does this constitute a dumbing down?

In broad terms scholarly behaviour can be portrayed as being active , bouncing, navigating, checking and viewing. It is also promiscuous, diverse and volatile

Does this constitute a dumbing down?

Dumbed down information seeking? Study confirms what many are beginning to suspect: that the web is having a profound impact on how we conceptualise, seek, evaluate and use information. What Marshall McLuhan called 'the Gutenberg galaxy' - that universe of linear exposition, quiet contemplation, disciplined reading and study - is imploding, and we don't know if what will replace it will be better or worse. But at least you can find the Wikipedia entry for 'Gutenberg galaxy' in 0.34 seconds

Study confirms what many are beginning to suspect: that the web is having a profound impact on how we conceptualise, seek, evaluate and use information. What Marshall McLuhan called 'the Gutenberg galaxy' - that universe of linear exposition, quiet contemplation, disciplined reading and study - is imploding, and we don't know if what will replace it will be better or worse. But at least you can find the Wikipedia entry for 'Gutenberg galaxy' in 0.34 seconds

Think big, think observatory: JISC National E-Book Observatory A geographical plot of IP addresses of participating universities Survey ran between 18 Jan and 1 March 2008, over which period 22,437 responses were received from 120+ universities. Logs – November to May. All page views to MyiLibrary titles - 3,600,000. JISC titles: views 337,500; sessions 32,800; separate IPs 8,871

A geographical plot of IP addresses of participating universities

Survey ran between 18 Jan and 1 March 2008, over which period 22,437 responses were received from 120+ universities.

Logs – November to May. All page views to MyiLibrary titles - 3,600,000. JISC titles: views 337,500; sessions 32,800; separate IPs 8,871

Number of page views to 26 JISC books

JISC books – subject comparison (page views)

Site penetration (JISC v other books)

Ranked JISC book usage 12.0 11.3 10.2 9.3 8.9 8.6 6.5 5.0 4.2 3.1 2.7 2.1 1.9 1.8 1.6 1.5 1.4 1.2 1.2 1.1 1.1 1.1 .7 .6 .6 .4 40490 38043 34279 31270 30097 28902 21800 16838 14199 10372 8989 7080 6544 6032 5369 5120 4823 4189 4118 3872 3679 3615 2198 2111 1956 1494 Media Gender and Identity Integrated Marketing Communication Marketing Strategy & Competitive Positioning Management Concepts & Practices Organisational Behaviour and Analysis: An Integrated Cinema Studies: The Key Concepts Chemical Engineering Volume 6: Chemical Engineering D Power without Responsibility The Dynamics of Employee Relations Engineering Materials 1 Public Relations Handbook Structural and Stress Analysis Media Institutions and Audiences Chemical Engineering Volume 2 Engineering Materials 2 Aerodynamics for Engineering Students Measurement and Instrumentation Principles Modern Structural Analysis Fundamentals of Wireless Communication English for Journalists Writing for Journalists A Short Course in Foundation Engineering A Short Course in Soil and Rock Slope Engineering Better Places to Live Conceptual Structural Design Better Places to Work 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 % Use JISC E-books ranked by usage Rank

Type of page viewed: all books

Top ten universities viewing Integrated Marketing Communication 2.6 Bournemouth 2.6 Stirling 3.4 Hertfordshire 4.3 St Andrews 4.7 Strathclyde 5.1 Sheffield 5.5 Glamorgan 5.7 Worcester 6.5 Coventry 7.2 Leeds Percentage of all use University

Plug for book http:// www.facetpublishing.co.uk/index.shtml

http:// www.facetpublishing.co.uk/index.shtml