Big data tokyo (extended version)

953 views
815 views

Published on

Presentation given to the Nikkei BP event on Big Data and Analytics in Tokyo, Japan on April 9, 2013.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
953
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Big data tokyo (extended version)

  1. 1. DATA ANDDISILLUSIONMENTSOLVEforINTERESTINGOTHERWISE LIFE IS DULL.
  2. 2. Volume (the “big” part) Pick any Velocity two Variety(the “fast” (the part) “anything” part)
  3. 3. Big Data is the Third Age of computing Computing Networking Big Data Automate Interconnect Predict & change things things things (Jim Stodgill of O’Reilly Radar said this.)
  4. 4. Enterprises expect Big Data to deliver betterdecisions and improved customer experiences What tangible benefits do you hope to achieve through your big data initiatives? NewVantage Partners LLC www.newvantage.com
  5. 5. (And apparently Hadoop is winning) What data management approaches are you considering? NewVantage Partners LLC www.newvantage.com
  6. 6. Therelationaldatabaseis a general-purposetool.
  7. 7. A library is a database optimized for retrievalPhoto by cybrgrrl (http://www.flickr.com/photos/cybrgrl/1295482521/) on Flickr
  8. 8. A changecounter is adatabaseoptimized forinsertion
  9. 9. An example:eventualconsistency
  10. 10. “End of Day Balance will only appear for dates previous tothe last 2 business days.”“Transactions from today are reflected in your balance, butmay not be displayed on this page if you recently updatedyour bankbook, if a paper statement was recently issued, orif a transaction is backdated. These transactions will appearin your history the following business day.”
  11. 11. Relational BIG Statistical
  12. 12. http://www.flickr.com/photos/jenny-pics/3239638494/sizes/l/ Breadcrumb trail
  13. 13. The average enterprise has 178 socialmedia accounts (According to @setlinger and the Altimeter group.)
  14. 14. Ward off disease. Pinpoint disasters.A force Reveal corruption.for good. Make cities smarter. Improve how we teach.
  15. 15. Big healthcare
  16. 16. Big philanthropy
  17. 17. Big commuting
  18. 18. Erode our privacy. Justify prejudices.A force Polarize groups.for bad. Leak private truths.
  19. 19. Big prejudice
  20. 20. “…nobody notices offers they do notget. And if these absent opportunitiesstart following certain social patterns(for example not offering them tocertain races, genders or sexualpreferences) they can have a deep civilrights effect.” Anders Sandberg, Oxford University
  21. 21. Personalization looks a lot like prejudice.
  22. 22. Big radio
  23. 23. Times a song in “heavy rotation”is played each day30 Every 55m15 Every 4h0 2007 2012
  24. 24. Humans are bad at data.
  25. 25. We prefer false positives.
  26. 26. Wooly mammothhttp://www.flickr.com/photos/pong/172438102/sizes/o/
  27. 27. Sun templehttp://www.flickr.com/photos/30787002@N02/3298693694/sizes/l/
  28. 28. Some proof.
  29. 29. It’s really hard to find people who can thinkabout data well How challenging is it to source data scientists? NewVantage Partners LLC www.newvantage.com
  30. 30. Mistake correlation for causalitySeek truthiness rather than factFind patterns where they don’t existEasily swayed by toneSide with our tribesDig in and ignore new evidence
  31. 31. Athenian swimming pools
  32. 32. Volume BigVariety Data Good dataVelocityVeracity
  33. 33. 525,000 state & local officersUnder 25 officers per precinct130 million incident reports200,000 uses of force31% keep computer files
  34. 34. Evidence.com
  35. 35. Hard drive
  36. 36. Big Data is not about data.
  37. 37. Big Data is about truth,auditability, and the ability to analyze data on a level playing field. It’s about analysis for everyone.
  38. 38. Alistair Croll @acroll www.solveforinteresting.comTHANKS! alistair@solveforinteresting.comSOLVEforINTERESTINGOTHERWISE LIFE IS DULL.

×