Big data journey_to_value_v5_john_sing

2. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 2 John Sing  32 years of experience in enterprise servers, storage, and software – 2015: IBM Product Manager – Spectrum Scale Storage – 2014: Director of Technology, 4cube – Infrastructure for Tomorrow – 2009 – 2013: IBM Executive IT Consultant: IT Strategy and Planning, Enterprise Large Scale Storage, Internet Scale Workloads and Data Center Design, Big Data Analytics, HA/DR/BC – 2002-2008: IBM IT Data Center Strategy, Large Scale Systems, Business Continuity, HA/DR/BC, IBM Storage – 1998-2001: IBM Storage Subsystems Group – Worldwide Marketing, Technical Support, Product Planner, Product Manager – Before that: • IBM Hong Kong, IBM China, IBM USA  john@johnsing.us  Follow me on Twitter: http://twitter.com/john_sing  Follow me on Slideshare.net: – http://www.slideshare.net/johnsing1  Blog: – http://johnsing.technology  LinkedIn: – http://www.linkedin.com/in/johnsing

3. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 3 Big Data’s Journey to Value Data + Analytics = Information Insight Desired Outcomes Information + Context = Insight + Actions =

5. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 5 You know how to analyze Big Data Goal: Analyze *all* the data real time Original source: Wikibon.org, “Big Data”, http://wikibon.org/blog/ten-%E2%80%9Cbig-data%E2%80%9D-realities-and-what-they-mean-to-you/ Very large Loosely structured Often incomplete Sampling not strategically competitive

6. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 6 Time ComputingPowerGrowth Traditional business “sensemaking” capability Available data for observation ContextEnterprise Amnesia What “Big Data” solves: Chart by: Jeff Jonas/Las Vegas/IBM, Chief Scientist, Context Computing: http://jeffjonas.typepad.com/

7. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 7 Enterprise Amnesia, definition A defect in memory, resulting in missed opportunity, wasted resources, lower revenues, unnecessary fraud losses, and other bad news.

8. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 8 Time ComputingPowerGrowth Traditional business “sensemaking” capability Available data for observation ContextEnterprise Amnesia Enterprise Amnesia examples….. Chart by: Jeff Jonas/Las Vegas/IBM, Chief Scientist, Context Computing: http://jeffjonas.typepad.com/

9. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 9 Time ComputingPowerGrowth Data + Analytics = “Information” Traditional business “sensemaking” Available Observation Space Context Big Data acquisition = New, Useful InformationAdd: Analytics What comes after “Information”?

11. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 11 Context More about Jeff Jonas, IBM Chief Scientist, Context Computing: http://bit.ly/1g3z9ZQ Jeff Jonas, IBM Chief Scientist Context Computing

15. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 15 Information in Context … = Insights Top 200 Customer Job Applicant Identity Thief Criminal Investigation scrila34@msn.com

16. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 16 The Puzzle Metaphor: what we mean by “Context”  Imagine an ever-growing pile of puzzle pieces of varying sizes, shapes and colors  What it represents is unknown – there is no picture on hand  Is it one puzzle, 15 puzzles, or 1,500 different puzzles?  Some pieces are duplicates, missing, incomplete, low quality, or have been misinterpreted  Some pieces may even be professionally fabricated lies  Until you take the pieces to the table and attempt assembly, you don’t know what you are dealing with

17. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 17 Here’s a “context” example…….. “Puzzling” 270 pieces 90% 200 pieces 66% 150 pieces 50% 6 pieces 2% (pure noise) 30 pieces 10% (duplicates)

26. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 26 Incremental Context – Incremental Discovery 6:40pm START 22min “Hey, this one is a duplicate!” 35min “I think some pieces are missing.” 37min “Looks like a bunch of hillbillies on a porch.” 44min “Hillbillies, playing guitars, sitting on a porch, near a barber sign … and a banjo!”

28. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 28 Incremental Context – Incremental Discovery 47min “We should take the sky and grass off the table.” 2hr “Let’s switch sides, and see if we can make sense of this from different perspectives.” 2hr10m “Wait, there are three … no, four puzzles.” 2hr17m “We need a bigger table.” 2hr18m “I think you threw in a few random pieces.”

31. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 31 Context Accumulates….. Into “Insights”  With each new observation … one of three assertions are made: – 1) Un-associated; – 2) placed near like neighbors; or – 3) connected  New observations sometimes reverse earlier assertions  Some observations produce new discovery  As the working space expands, computational effort increases  Given sufficient observations, there can come a tipping point. Thereafter, confidence improves while computational effort decreases!

33. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 33 Big Data [in context] = Insights. More data: better the predictions – Lower false positives – Lower false negatives More data: bad data … good – Suddenly glad your data was not perfect More data: less compute

35. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 35 Now that I create Insights..…. how do I take Action? Chart in public domain: IEEE Massive File Storage presentation, author: Bill Kramer, NCSA: http://storageconference.org/2010/Presentations/MSST/1.Kramer.pdf:

36. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 36 Answer: build actionable systems that use the insights Chart in public domain: IEEE Massive File Storage presentation, author: Bill Kramer, NCSA: http://storageconference.org/2010/Presentations/MSST/1.Kramer.pdf: n d Actionable Systems

37. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 37 Projected traffic Insights •10 minute-ahead volume forecast (blue) vs. actual value (black) •10 minute-ahead speed forecast (blue) vs. actual value (black). Black line: actions via signals = desired outcome Stockholm: http://www.youtube.com/watch?v=rfMylzF4lv8  Actionable traffic signals Blue line: analytics prediction 10 minutes in advance

38. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 38 Insights based on crime  actions: where to deploy of officers  Blue CRUSH predictive analysis for officer deployment & risk management generated easy-to-read crime maps every four hours  Richmond, VA: Violent crime decreased in the first year by 32%, another 40% thereafter, moving Richmond from #5 on the list of the most dangerous US cities to #99 Memphis Blue CRUSH MapMemphis Blue CRUSH Map Police videos: http://www.youtube.com/watch?v=8SJQtn4RO7I Play video https://www.youtube.com/watch?v=_xsffIAHY3I

39. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 39 Local Applications: Big Data’s Journey to Value Data + Analytics = Information Insight Desired Outcomes Information + Context = Insight + Actions =

43. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 43 Quiz: in following Futuristic video see if you can identify: Data + Analytics = Information Information + Context = Insight Insight + Actions = Desired Outcomes

45. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 45 Final Quiz: Big Data’s Journey to Value Data + Analytics = Information Insight Desired Outcomes Information + Context = Insight + Actions =

46. © 2015 John Sing – All Rights Reserved University of South Florida - Spring 2015 46 Thank You Merci Grazie ObrigadoDanke Japanese Hebrew English French Russian German Italian Brazilian Portuguese Arabic Traditional Chinese Simplified Chinese Hindi Tamil Korean Thai TesekkurlerTurkish