Downloaded 35 times
Uploaded byTim Willoughby
Big Data, Open Data, Big Costs - tim willoughby
The document discusses the implications and opportunities of big data, open data, and cloud technologies in various sectors, particularly in government and retail. It highlights the vast amounts of data generated by individuals and organizations, emphasizing the need for transparency and innovative applications of data to enhance government efficiency and empower citizens. The document also touches on the evolution of technology, including mobile and social media's impact on communication and data usage.
More Related Content
Similar to Big Data, Open Data, Big Costs - tim willoughby
Big Data, Open Data, Big Costs - tim willoughby
- 1. Tim Willoughby Big Data Open Data Big Costs? ISA 24th October
- 2. Gartner Top 10for 2012 - • Released today by Forbes
- 3. What has Cloudever done for us? Apart from Scale, Speed, Agility, Low Cost, Enterprise Adoption, Enterprise Mapping, Open Data, Standards, Google, API’s, etc Open Street Maps, Map Servers, GIS - More than just Location, Spatial Analysis and wider adoption now possible 3
- 4. How Much Datado you create?
- 5. How much datato you create?
- 6. Retailers.. • Assign eachshopper a unique code - keeps tabs on everything they buy. – If you use a credit card or a coupon, – or fill out a survey, – or mail in a refund, – or call the customer help line, – or open an e-mail we’ve sent you – or visit our Web site, – we’ll record it and link it to your ID,” • “We want to know everything we can.” • “Cool is one click away from Creepy” (Fjord)
- 7. Your Demographic TheyHave or can get • You o your age, o whether you are married, or got divorced o and have kids, Your ethnicity, • Where you live , the year you bought (or lost) your house, whether you’ve moved recently, • The number of cars you own. • Distance to nearest Outlet, • Worth • Your estimated salary, if you’ve ever declared bankruptcy, Charity Giving • What credit cards you carry in your wallet • Education, where you went to college, • Career - job history, • Online -what Web sites you visit., the magazines you read, reading habits, • The topics you talk about online, • Likes- certain brands of coffee, cereal etc, political leanings,
- 8. Big Data goeswith… More Devices More Access More Apps More Data…
- 11. 4 Global Themes Social DATA Mobile Cloud
- 13. Radical Change… Social has overtaken mobile …. Mobile Browsing has overtaken desktop…
- 14. Change… With or Withoutthe Owners / Shareholders
- 16. So far ICThas not fundamentally changed government • 1990s: lCT expected to make government more transparent, efficient and user oriented • 2005+: disillusion as bureaucracy still in existence • Can Mobile / Data /Cloud Help? Jane E. Fountain – Gov 1.0 – Just Replicating the Silos on the Internet
- 17. Goverment dont alwaysUnderstand What the people Want?
- 18. What governments oftenDeliver
- 19. Open Government? “If peopledon’t know what you’re doing, they don’t know what you’re doing wrong” “Jim Hacker” Open Government (I980)
- 20. The World haschanged…
- 23. Every day -you come across multiple… • - Devices • - applications • - access points • - Networks • - Locations • - Servers • - Data types • - Data caches • - Service providers • - Operating systems • - Storage locations • - Encryption / security protocols..
- 24. My Smart Deviceis…. Very smart Source : Peter Cochrane
- 25. It knows more aboutme than my partner… Source : Peter Cochrane
- 27. Security is changing •Security has to be appropriate • Security has to be measured • Can have things so secure that they are unusable.
- 28. Citizen Expectation Changed? Social Media
- 29. The More datawe have… the less we (have time to) Understand it
- 32. Government as aPlatform?
- 33. Mobile Development Spendis rapidly overtaking PC Development Spend
- 34. BIG DATA? Massivenetwork of services: water, sewage, drains … Need to know asset location for planning and maintenance Many databases, varying accuracy and provenance Context Ongoing street openings p.a. Safety!
- 35. Typical Individual effort manyhours, one map
- 36.
- 39. Our Lives areDifferent… • We communicate on — Facebook, Chat, Twitter, MMS • We Research Information — Blogs, eNews, Wiki, YouTube • We Buy — eBay, Amazon, Dell, Deal Done, etc… • We Travel - Tripadvisor, Flickr, RyanAir, Hotels, • We Meet / Retate - Linkedln, FaceBook, Friends • We Play – xBox, Playstation, Online games, ¡TV • We expect the Government to fit into these paradigms too…
- 42. What is BigData? • Size beyond the ability of commonly used software tools • Data growing challenges • volume increasing • velocity (speed of i/o) • variety (data type/source) • Examples: computer tracking of shipments, sales, suppliers, cust omers, email, web traffic, social network From McKinsey Report
- 43. Why Now?. Whatis the Catalyst? • People – Web sites with 300+ million unique visitors/month • Facebook • Google • YouTube • Access – 1.2 Billion active mobile broadband subscriptions • Commodity (community) Hardware – possibility for cost effective processing
- 44. Facts! • “Data isa vital raw material of the information economy, much as coal and iron ore were in the Industrial Revolution.” • “Mining and analyzing these big new data sets can open the door to a new wave of innovation, accelerating productivity and economic growth.” • We can exploit Internet-scale data sets to uncover new businesses and predict consumer behavior and location data market shifts. sales business data social network data
- 45. Think… • NY Traffic – They have mountains of data – More Daily – Don’t Understand it, can’t use it – What is the True Cost of Data Mining – Systems, Software, Computers, Analysts, STORAGE – Are we collecting, collating, etc the Right Data – People being removed daily, replaced by machines – Need to Stand Back….
- 46. How to dealwith Lots of data • 7TB a Day for Twitter… • 10,000 CD’s • 5m Floppy • 225GB during this talk… • How do you Store that… • HD Write Speed- 80mb/s… • 24.3 Hrs to write 7TB…. • Government Thinking is not in this space.. Yet..
- 48. Irish Government • Decision– “All Government Invoices Published” – Election Manifesto – Election Promises – Programme for Government • Action – None… – Fear, Bad Reactions, Data Quality,
- 54. Stakeholders • Private citizens • Activists • Coders • Entrepreneurs • NGO’s • Academics • Civil Servants • Gov Institutions
- 55. Which Apps? • Government– Do not try to second guess what are useful applications! • No one can predict what data to make open based on potential Apps. • There are a huge number of apps available for smartphones. Impossible to guess a few years back. • Put Data out there. Data as a platform for others to build on, as infrastructure. • Apps you will like and hate, but it’s not up to you.
- 56.
- 57. Where is OpenData delivering? • Participation / empowerment • Transparency, democratic control • New and better Government products and services • Impact measurement • Efficient and Effective Government • Data Journalism (Guardian Services) • New knowledge / real innovation
- 58. The Internet ofThings Cars, Planes, Cameras, Roads, Pipes, Public Cameras, Traffic Lights, Car Parks, Pharma and Farms – over 1 Trillion
- 59.
- 60. Mass increase innew net devices • Move to digital controls Wireless integration • Network standards – TCP/IP communications – Wireless communications 60
- 61. Or Wizards GIS isnow Mainstream
- 62. Water Meters • Readthe Meter • View or Pay the Bill • Compare the Usage to the Local or National Averages • Look for areas of Savings
- 63. Roadworks • Link toSystem – Online Road Works Control (OLRWC) • License for Road Opening • Cost of Reinstatement (Local Authority Usage) • License for any Street usage.
- 64. Planning Applications • View Applications • Get notified of Changes • Make Comments • Make a Submission • Notify Local Authority of Issues?
- 65. Sports • What is available now • How do I get there • What if I have to Cancel • Pay for it now
- 66. Parking • Select County •Select Zone • Select Price willing to pay? • Where is the Cheapest
- 67. open source /open data / cloud the fix your street model
- 68.
- 69. But all ofthese are static….. But… would you cross the road based on 5 min old information
- 70. Why Open Data •Because the Data already belongs to everyone! • Because there is Law, Inspire Directive - PSI • It helps change the incentives, from Fail Big, Fail Slow to Fail Small, Fail Fast, and Fail Forward (Innovation – More eyes) • Open Data will allow issues/problems to be seen, making them less likely to be repeated (learning from other people’s failures) • Open Data use can allow Incremental Innovation: - stand on the shoulders of giants. Don’t reinvent the wheel — use or improve someone else’s • Open Data quality can be increased over time — use and reuse by other people and combining with other data identifies (and helps solve) quality issues far more quickly and cheaply than internal checking
- 71. How do wemeasure Success? • Usage and Apps • Critical Mass of Companies • Search Results (Bing and Google Interested) • Business Committed to OD • Laws (Prison for breaking them) • If Open Data doesn’t cause difficulty for the Publisher to seek excellence then it isn’t working
- 72. Where next -Augmented Reality Bring together Big Data, Visualisation, GIS, GPS - Bringing Cloud, Social, Data and Mobile together…
- 73. What is AugmentedReality?
- 76.
- 77.
- 79. Understand what arethe Driving Forces… • Government Focus – Understanding, Efficiency, Accountability • Technology Innovation Focus – Data as a Platform, Semantic Web • Reward Focus – Profit, Recognition • Digitising Government Focus – Computerisation / Technology Drive • Problem Solving Focus – New Skills needed to work on new Challenge • Social / Public Sector / Enterprise – More Focussed Services
- 80. Conclusions • Change is constant • Cloud is the Future • Social is a Reality • Mobile is already taking over • Data has to keep up…
- 81. References • We livein “Flat Land” – There is danger in making representations more seductive than the truth • Envisioning Information - Tufte