Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Brief History Of Data

1,613 views

Published on

My presentation from QCON London 2014 entitled "A Brief History Of Data"

Published in: Data & Analytics, Technology
  • Be the first to comment

A Brief History Of Data

  1. 1. Copyright © 2014 Splunk Inc. A brief history of data Damien Dallimore Worldwide Developer Evangelist @ Splunk
  2. 2. Who Am I 2 Worldwide Developer Evangelist @ Splunk I code I talk about coding Community, Collaboration, Open
  3. 3. From Aotearoa (New Zealand) 3
  4. 4. Where did the “BIG DATA” come from 4
  5. 5. 5 Let’s go on a journey
  6. 6. 20,000 BC Arithmetic 6 We start to count things
  7. 7. 15,000 BC Cave Painting 7 We start to record data visually
  8. 8. 3,500 BC Written Language 8 We start to record and “transmit” knowledge
  9. 9. 2,500 BC Sumerian Calendar 9 We start to organize and track time
  10. 10. 1,250 BC Library at Thebes 10 We start to store data in mass
  11. 11. 1,150 BC Egyptian Maps 11 The origin of Google Maps
  12. 12. 1000 BC Era Math, Computation, Logic 12 We start to develop understanding through numbers 500 BC Pythagoras 300 BC Euclid And how numbers can be used to compute data 250 BC Archimedes 100 BC Antikythera Mechanism We start to classify objects and use logic to derive insights 350 BC Aristotle
  13. 13. 0 - 1600 13 78 Pliny : “all” the world’s knowledge captured 105 Paper : bulk recording of data 340 Codices : making data browseable with sections and indexes 1350 Nicole Oresme : turning data into picture 1453 Guttenberg : mass distribution of data
  14. 14. 1600 - 1900 14 1640 Napier : before logs there were logarithms 1662 Graunt : father of statistics 1796 Watt : recording data with a machine 1801 Jacquard : programming !! 1830 Babbage : the first mechanical and programmable computer 1844 Morse : data encodings 1850 Reuter : first “WAN” (the CSMA/CD was a bit messy) 1876 Dewey : data classification
  15. 15. 1900 - 2000 15 1930’s Fisher : modern statistics 1936 Turing : the universal computer 1950’s Programming Languages : Fortran et al 1962 Tomlinson : first standard for Geo Data 1963 ASCII : a standard for representing letters and numbers 1969 ARPANET and other Protocols 1970’s RDBMS : ETL , BI , Data Warehouses , I am your father 1970’s/ 80’s Personal Computing : the foundations for alot of today’s data 1982 TCP/IP standardized and the Internet came to be 1989 The Web and HTML blink tags 1991 Unicode : all languages captured
  16. 16. To infinity and beyond 16 Web 2.0 Google Social Networking IOT
  17. 17. 17 Data Today ?
  18. 18. 18
  19. 19. 19
  20. 20. 20
  21. 21. 21
  22. 22. 22
  23. 23. 23
  24. 24. 24
  25. 25. 25
  26. 26. 26
  27. 27. Data Stats Candy 27 Every day 2.5 quintillion bytes of data (1 followed by 18 zeros) are created A full 90 percent of all the data in the world has been generated over the last two years 2.7 Zetabytes of data exist in the digital universe today. Facebook stores, accesses, and analyzes 30+ Petabytes of user generated data Akamai analyzes 75 million events per day to better target advertisements. Decoding the human genome originally took 10 years to process; now it can be achieved in one week. Data production will be 44 times greater in 2020 than it was in 2009
  28. 28. Data Characteristics 28 VOLUME VARIETY VERACITY VELOCITY
  29. 29. 29 Data Tomorrow ?
  30. 30. 30
  31. 31. 31 DATA UNDERSTANDING INSIGHTS ACTIONS
  32. 32. 32
  33. 33. 33 How are we going wrangle this data ?
  34. 34. Release the Developers 34
  35. 35. New approaches to data platforms are needed 35 Traditional ETL / Data Warehouses = schema at write time To cope with data today = schema at read time A Data language , the new SQL Platforms that support an ecosystem of developers , content creators , data knowledge sharing Open and easily extensible to cope with a variety of data sources and use cases, API oriented Elasticity
  36. 36. 36 Make machine data accessible, usable and valuable to everyone.
  37. 37. Spelunking 37
  38. 38. Platform for Machine Data Any Machine Data HA Indexes and Storage Search and Investigation Proactive Monitoring Operational Visibility Real-time Business Insights Commodity Servers Online Services Web Services Servers Security GPS Location Storage Desktops Networks Packaged Applications Custom ApplicationsMessaging Telecoms Online Shopping Cart Web Clickstreams Databases Energy Meters Call Detail Records Smartphones and Devices RFID
  39. 39. Powerful Platform for Enterprise Developers 39 REST API Build Splunk Apps Extend and Integrate Splunk Simple XML JavaScript Django Web Framework Java JavaScript Python Ruby C# PHP Data Models Search Extensibility Modular Inputs SDKs
  40. 40. Enough talking DEMO TIME !!
  41. 41. The Developer Opportunity in Data 41 It’s fun to make cool things Get a job , build a business , make money ! Promote yourself , Promote your company Get involved in community projects Do Good Think of new data sources and tap into them Democratize data Discover new things & drive society forward We talk alot about the how , what , where and who ….. but what about the WHY
  42. 42. Contact Me ddallimore@splunk.com @damiendallimore http://dev.splunk.com http://blogs.splunk.com/dev 42
  43. 43. Thankyou !

×