Surveying Our Landscape From Top to Bottom


Published on

Slides for a talk on "Surveying Our Landscape From Top to Bottom" given by Brian Kelly at a workshop on "Metrics and Social Web Services: Quantitative Evidence for their Use and Impact" held at the Open University, Milton Keynes on 11 July 2011.


Note the slides were also used at the SOLO 2011 conference - see

Published in: Education, Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Surveying Our Landscape From Top to Bottom

  1. 1. Metrics and Social Web Services: Quantitative Evidence for their Use & Impact Surveying Our Landscape From Top To Bottom Brian Kelly UKOLN University of Bath Bath, UK UKOLN is supported by: This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence (but note caveat) Acceptable Use Policy Recording this talk, taking photos, discussing the talk using Twitter, blogs, etc. is permitted but please try to minimise distractions to others. Twitter: #ukolneim Blog: Twitter: @briankelly @ukwebfocus
  2. 2. You are free to: copy, share, adapt or re-mix; photograph, film or broadcast; blog, live-blog or post video of this presentation provided that: You attribute the work to its author and respect the rights and licences associated with its components. Idea from Cameron Neylon Slide Concept by Cameron Neylon, who has waived all copyright and related or neighbouring rights. This slide only CCZero . Social Media Icons adapted with permission from originals by Christopher Ross. Original images are available under GPL at: Off the record questions and comments should be flagged Evidence & comments
  3. 3. <ul><li>Services being considered: Twitter, Facebook, YouTube, iTunes, Technorati, Wikio & Slideshare </li></ul>
  4. 4. Twitter: Personal Trends <ul><li>I thought I used Twitter regularly ever since signing up in March 2007. </li></ul><ul><li>Evidence from Tweetstats: </li></ul><ul><ul><li>Significant use began in Jan 2008 (100+ tweets) </li></ul></ul><ul><ul><li>No use in Mar-Jun 2008 </li></ul></ul><ul><li>Reality: </li></ul><ul><ul><li>Significant use began in Apr 2008 at MW 2008 conf </li></ul></ul><ul><ul><li>Needed community in order to gain benefits </li></ul></ul><ul><li>Implications: </li></ul><ul><ul><li>Our memories may be incorrect </li></ul></ul><ul><ul><li>Data may be wrong (due to Twitter downtime??) </li></ul></ul>
  5. 5. Institutional Use of Twitter <ul><li>Questions: </li></ul><ul><ul><li>What can commercial social media analytics services tell us about institutional Use of Twitter? </li></ul></ul><ul><ul><li>What can we learn from the approaches the services take? </li></ul></ul><ul><ul><li>What are their strengths and weaknesses? </li></ul></ul><ul><ul><li>Can we / should we develop alternatives? </li></ul></ul>
  6. 6. Social Media Analytic Summaries <ul><li>Social media analytic summaries for Russell Group Universities </li></ul>Outliers may be interesting
  7. 7. Peerindex Comparisons (1) <ul><li>Peerindex’s ‘topic fingerprint’ for Oxford / Cambridge: </li></ul><ul><ul><li>Similar profiles </li></ul></ul><ul><ul><li>Oxford covers news, politics & history </li></ul></ul><ul><ul><li>Cambridge covers technologies </li></ul></ul>Peerindex and Klout group comparisons for Russell Group Unis available (data used in this talk is available freely for reuse under CC0 licence)
  8. 8. Peerindex Comparisons (2) <ul><li>Peerindex’s ‘topic fingerprint’ for: </li></ul><ul><ul><li>@psychemedia (blue blob) & @mweller (grey outline) </li></ul></ul>Do such comparisons tell us anything useful? Example used of four established bloggers / Twitterers
  9. 9. Peerindex For Participants <ul><li>Peerindex table of rankings for attendees. Is this: </li></ul><ul><ul><li>A self-fulfilling echo-chamber / a clique? </li></ul></ul><ul><ul><li>Easily gamed (e.g. by signing up to service)? </li></ul></ul>Scenario 1 : Research funding determined by flawed metrics but Unis still headhunt researchers with valuable RAE scores Scenario 2 : Social Media metrics flawed but Unis headhunt marketing / outreach people with valuable scores Scenario 3 : “IMAGINE a world in which we are assigned a number that indicates how influential we are. This number would help determine whether you receive a job …It’s not science fiction. It’s happening to millions”
  10. 10. Slideshare: Personal Use <ul><li>Steve Wheeler tweet on Aug 2010: </li></ul><ul><ul><li>Ironically there were 15 people in my audience for this Web 3.0 slideshow but >12,000 people have since viewed it </li></ul></ul><ul><li>In Dec 2010 blog post reported: </li></ul><ul><ul><li>~16K views &1.5K embedded views </li></ul></ul><ul><li>Blog post suggested: </li></ul><ul><ul><li>Value of embeddable resources (Slideshare sets OERs free) </li></ul></ul><ul><ul><li>Ways of comparing value of experiences of 15 local audience & 22.8K (now) remote viewers </li></ul></ul>
  11. 11. Slideshare at Events (1 of 2) <ul><li>IWMW events used Slideshare since 2006. </li></ul><ul><li>Statistics show: </li></ul><ul><ul><li>Overall nos of views (<1,000 live audience) </li></ul></ul><ul><li>and suggest: </li></ul><ul><ul><li>Potential for amplification of parallel sessions </li></ul></ul><ul><ul><li>Potential value of aggregation of slides </li></ul></ul><ul><li>But does decline since 2008 peak indicate: </li></ul><ul><ul><li>Slideshare is past it (use Slideboom, …)? </li></ul></ul><ul><ul><li>Viewing slides is passe </li></ul></ul><ul><ul><li>Benefits of longstanding availability? </li></ul></ul>
  12. 12. Slideshare at Events (2 of 2) <ul><li>Statistics of most popular slides suggest: </li></ul><ul><ul><ul><li>Popularity of non-HE speakers’ slides </li></ul></ul></ul><ul><ul><ul><li>Popularity of slides not in corporate house style </li></ul></ul></ul>This has some positive correlation with experiences of local audience Could this inform selection of speakers in future?
  13. 13. Slideshare: Lessons <ul><li>Lessons: </li></ul><ul><ul><li>Benefits of providing embeddable resources to enhance access </li></ul></ul><ul><ul><li>Informing policy: encouraging workshop facilitators to upload their slides </li></ul></ul><ul><ul><li>Challenging orthodoxy: value of house style </li></ul></ul><ul><ul><li>Apparent decline in numbers may reflect longevity of availability of resources </li></ul></ul><ul><li>Implications: </li></ul><ul><ul><li>What can the evidence and reflections tell us about teaching and learning repositories? </li></ul></ul>Slideshare Premium account provides richer statistics (trend analysis?) indicates value of business intelligence for $19/month.
  14. 14. Metrics For Personal Blogs <ul><li>Technorati gives authority & ranking for 1.2 M blogs </li></ul><ul><li>Authority measures blog's standing & influence on scale of 0-1000 (high good). Ranking given for Technorati Authority of all sites (low good) </li></ul><ul><li>Rank of 3,045 means top 0.2% of all blogs. </li></ul><ul><li>Technology rank of 376 (of 30,133) means top 1.2% </li></ul>Metadata for use in comparison with similar blogs Top 3% of all blogs Top 13% of technology blogs Or top 0.2% of all blogs Top 1.3% of technology blogs Or
  15. 15. Metrics For Personal Blogs <ul><li>For blogs with a “JISC” keyword: </li></ul><ul><li>MASHe: </li></ul><ul><li>Top 0.8% </li></ul><ul><li>Top 2% technology </li></ul><ul><li>JISC AM Team: </li></ul><ul><li>Top 6% </li></ul><ul><li>May be value in: </li></ul><ul><ul><li>Comparisons (learning) </li></ul></ul><ul><ul><li>Aggregation (value of HE) </li></ul></ul><ul><li>But … </li></ul>Note usefulness of keywords Why the drop?
  16. 16. Wikio <ul><li>Wikio also provides metrics for (registered) blogs </li></ul><ul><li>List of top technology blogs shown </li></ul>MASHe also ranked highly but not listed as Technology blog (so 4 of top 100 blogs here!) Together with display of trends for ranking, nos. of posts, links and backlinks, OUseful had peak at #18 in Nov 2008 What happened from Jun-Sep 2010?
  17. 17. Discussion <ul><li>Criticisms: Flawed Approaches </li></ul><ul><ul><li>Blogs have variety of purposes so inappropriate to reduce to 1 dimension </li></ul></ul><ul><li>Response: </li></ul><ul><ul><li>A degree can be seen as I/2.1/2.2 or 3 </li></ul></ul><ul><ul><li>Some blogs have similar purpose and can be compared </li></ul></ul><ul><li>Criticisms: Technical </li></ul><ul><ul><li>Results can be volatile & may be skewed by blog technologies; migration of blogs; … </li></ul></ul><ul><li>Response: </li></ul><ul><ul><li>Could improve </li></ul></ul><ul><ul><li>Not significantly different from University rating? </li></ul></ul>
  18. 18. Facebook Trends in UK HE <ul><li>Development since 2007: </li></ul><ul><li>Blog post in 2007 reported 7 UK Unis on Facebook: Aston, Cardiff, Kent and the University of Central Lancashire (UCLan) </li></ul><ul><li>Jun 2008 survey reported on 8 most popular UK HE pages: OU top with 7,539 fans </li></ul><ul><li>Aug 2010 survey showed growth over 2 years (OU 380%; Cardiff 13,000% and others from 400-700%) </li></ul>Result of decision to be pro-active in Fb marketing? If so, did it provide an ROI?
  19. 19. Facebook <ul><li>Survey of Russell Group Unis Fb use: </li></ul><ul><li>Branded Fb URL: 7 </li></ul><ul><li>Fb page: 7 </li></ul><ul><li>Fb group: 1 </li></ul><ul><li>No easily found Fb presence: 5 </li></ul><ul><li>Neglected or unofficial Fb presence: 1 </li></ul><ul><li>Range of ‘likes’:  2,047 – 137,395 </li></ul><ul><li>Recent report on ‘ROI of Fb advertising’ suggested “ 1 Facebook fan = 20 additional visits to your website ” </li></ul>How would we know if Cardiff‘s 20,035 Likes have provided an ROI?
  20. 20. YouTube Edu <ul><li>Survey of YouTube Edu accounts: </li></ul><ul><ul><li>1M+ views at Coventry Uni </li></ul></ul><ul><ul><li>Lack of comments </li></ul></ul>Example of stats available
  21. 21. iTunes Edu <ul><li>Numeric data not readily obtained from iTunes Edu </li></ul><ul><li>Is this: </li></ul><ul><ul><li>To be welcomed as an escape from bean-counting? </li></ul></ul><ul><ul><li>Of concern since gathering qualitative evidence can be costly? </li></ul></ul>Blog post analysed how services marketed: “ lots of content ”; “ free content ”; “ available on iPods ”; …
  22. 22. Conclusions <ul><li>To conclude quantitative evidence: </li></ul><ul><ul><li>Can inform policy decisions </li></ul></ul><ul><ul><li>Can influence individual decision-making </li></ul></ul><ul><li>The data: </li></ul><ul><ul><li>Needs to be gathered! </li></ul></ul><ul><ul><li>May be flawed and should be questioned </li></ul></ul><ul><li>The data analysis: </li></ul><ul><ul><li>May inform (and surprise) </li></ul></ul><ul><ul><li>May be flawed </li></ul></ul><ul><li>But: </li></ul><ul><ul><li>Links with value & financial ROI not yet established </li></ul></ul>
  23. 23. Questions <ul><li>Any questions or comments? </li></ul>