Professor Andrew Prescott, Theme Leader FellowAHRC Digital TransformationsStrategic ThemeBig Data: Some Initial Reflections
• The Met Office currently generates about 20TB ofdata each day• ‘The problems which confront the meteorologisttoday will be faced by the humanities scholar withinten years’
• Large Hadron Collider: 600 million ‘collision events’ persecond• One million jobs run by servers each day, with over 10GB of data per second transferred at peak times• Approx. 20 petabytes of data produced annually• Over 70 universities involved in processing the data
Whole brain imaging of neurone activity in a zebra fish, made usinglight sheet microscopy by Misha Ahrens and neuroscientists at theHoward Hughes Medical Institute. Each image comprises over 1terabyte of data.Link:http://www.youtube.com/watch?feature=player_embedded&v=KE9mVEimQVU
• Some working definitions of big data• Big data exceeds the capacity of existingdesktop machines and networks: you needhelp to deal with it• Data that is so large that existing methodsof analysis simply don’t work: you have tochange your methodology (probably tosomething quantitative)• Gartner definition: “Big data” is high-volume, -velocity and –variety informationassets that demand cost-effective,innovative forms of information processingfor enhanced insight and decision making.
Examples of everyday bigdata of research value• Retail data generated by supermarkets• Online retail data: Amazon• Transport information: Oyster card• Hospital data• Data from utility companies• Social media
Visualisation of languages used in tweets in London inSummer 2012: Centre for Advanced Spatial Analysis, UCL:http://mappinglondon.co.uk/2012/londons-twitter-tongues/
Wolphram Alpha analytics of my Facebook friends
Letter of Gladstone toDisraeli, 1878: BritishLibrary, Add. MS. 44457, f.166The political and literarypapers of Gladstonepreserved in the BritishLibrary comprise 762volumes containingapprox. 160,000documents.
George W. Bush Presidential Library:200 million e-mails4 million photographs
A Thousand Words: Advanced Visualisation in theHumanitiesTexas Advanced Computing CenterLink: http://www.youtube.com/watch?v=kvOuJ2RwBTA
‘Big data’ has alreadybeen an issue forlinguists for many years
Another familiar example of big data in thehumanities: censuses
Moving images and sound present some of the mostchallenging big data issue for arts and humanities
Archives and library catalogues as big data:Visible Archive browser: visiblearchive.blogspot.com
Visualisation by Jon Orwant of Google of Library ofCongress subject categorisations of books publishedbetween 1600 and 2010: winedarksea.org
Commons Explorer: experimental interface to allowexploration of large quantities of images in FlickrCommons: http://mtchl.net/cex/
The Anglo-American Legal Tradition: web siteholding seven million images of medievallegal records in the National Archives:www.aalt.law.uh.edu
Fabio Lattanzi Antinori,The Obelisk (2012): Open DataInstitute: http://www.theodi.org/culture/obelisk-2012
Asia Trend Map:predicting popularity ofgames, manga andanime:www.asiatrendmap.jp
Some Big Data Issues• Research has historically been hypothesis-driven; is a more data-driven researchrequired?• How valid are predictive and probabilistictechniques in arts and humanities research?• Data quality issues: do we lose a sense of thecontext and stratigraphy of the data?• Danger of thinking that data=truth
Digital Transformation theme andBig Data• Theme seeks to promote new researchmethods: using digital tools and materials todevelop completely new type of scholarship• Additional funding of £4m has been allocatedto work on big data• Following this workshop, call for big dataprojects will be issued• Smaller projects (up to £100k)• Larger projects (up to £600k)
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.