The Future of Advance Analytics

783 views

Published on

Presented at InnoTech Austin on November 8, 2012. All rights reserved.

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
783
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

The Future of Advance Analytics

  1. 1. 11/7/2012 The Future of Advance AnalyticsDavid SmithChief Executive Officer, HBMG Inc.dsmith@hbmginc.comRoom: 11Volume, variety, and velocity are changing us in our companies, governmentagencies and at home. How do BI, Social/Business media, mobility, Devices and BigData drive business decisions? The success is driven by the use of advanceanalytics. Business analytics facilitates realization of business objectives throughreporting of data to analyze trends, creating predictive models for forecasting andoptimizing business processes for enhanced performance. This is important not onlyin business but the military and government as well.This presentation will look at the current and future trends in analytics and how theywill impact each of us. Special emphasis will be given to the new trend of embeddedanalytics. As the world moves faster toward real-time the analytics must move aswell. Attendees will leave the session with a understanding of the future directions ofadvance analytics. The Future of Advance Analytics David Smith CEO  HBMGInc. dsmith@HBMGINC.com 1
  2. 2. 11/7/2012 Definition for Advanced AnalyticsAnalysis is the examination process itself where  analytics is the supporting technology and  associated tools. BI is quite synonymous to  analytics in IT context. Advanced Analytics,  Business Analytics, Data Analytics, Analytics  Software, Analytics Technology are almost  always marketing pleonasms (redundant  always marketing pleonasms (redundant expressions) and can be safely substituted by  just ‘analytics’ Definition for Advanced AnalyticsAnalysis is a pretty old, well understood term  and essentially means “breaking down” or  d ti ll “b ki d ” “decomposition”. More accurately –the  process of decomposing complex entity into  simpler components for easier  comprehension. 2
  3. 3. 11/7/2012 Definition for Advanced AnalyticsAdvanced analytics provides algorithms for complex analysis of  either structured or unstructured data. It includes  sophisticated statistical models, machine learning, neural  networks, text analytics, and other advanced data mining  techniques. Among its many use cases, it can be deployed to  find patterns in data, prediction, optimization, forecasting,  and for complex event processing/analysis. Examples include  predicting churn, identifying fraud, market basket analysis, or  understanding website behavior. Advanced analytics does not  understanding website behavior Advanced analytics does not include database query and reporting and OLAP cubes. America gets more than half its economic  growth from industries that barely existed  a decade ago—such is the power of  innovation, especially in the information  and biotechnology industries. —The Economist Copyright, 2011 © HBMG, Inc. 3
  4. 4. 11/7/2012 Business ProblemMore than half of business and IT executives, 56  percent, report they feel overwhelmed by the  amount of data their company manages.  Many report they are often delayed in making  important decisions as a result of too much  information. Surprisingly, 62 percent of C‐level  respondents whose time is considered the respondents – whose time is considered the  most valuable in most organizations – report  being frequently interrupted by irrelevant  incoming data.  4
  5. 5. 11/7/2012 Delivering business value is hard…• “Of the work executed: “Many (possibly  most) organizations lose as much as 45%  of their total revenues due to costs  associated with low quality associated with low quality” – Six Sigma• “Some 75 percent of most large‐scale J2EE  projects fail by missing both time and  budget projections …” – Mark Driver, Gartner• “64% of features actually delivered are 64% of features actually delivered are  either rarely or never used”  – Jim Johnson, Standish Group Why? • Technological innovation is now the most  important driver for competitive success – Many firms earn over one‐third of sales on  products developed within last five years • Product life cycles ( time between product  introduction to market and its withdrawal) – Software 4‐12 months – Computer hardware 12‐24 months – Large home appliances 18‐36 months Copyright, 2011 © HBMG, Inc. 5
  6. 6. 11/7/2012 Business, Knowledge, and Innovation Landscape• Typically 80% of the key knowledge (and value) is held by 20% of the people  we need to get it to the held by 20% of the people – we need to get it to theright people• Only 20% of the knowledge in an organization is typically used (the rest being undiscovered or under‐utilized)• 80 90% f th 80‐90% of the products and services today will be  d t d i t d ill bobsolete in 10 years – companies need to innovate & invent faster Copyright 2012@ HBMG Inc. Tapping into the Data• Data Storage• Reporting Utilized data• Analytics• Advanced Analytics – Computing with Unutilized data big datasets is a g that can be available t il bl to fundamentally business different challenge than doing “big compute” over a small dataset 6
  7. 7. 11/7/2012 Innovation: ‘Innovation = ‘The real voyage of discovery consists creative idea and not in seeing new lands, implementation’ but in seeing with new eyes’ (Source: Glossary of Electronics) (Source: Marcel Proust) ‘Innovation: change that creates a new dimension of ‘A new method, idea, performance’ product, etc’ (Source: Peter Drucker) (Source: Oxford English Dictionary) ‘Value innovators look for‘An innovation to be effective what customers value in has to be simple and it has common’ to be focused’ (Source: Kim & Mauborgne) (Source: Peter Drucker) ‘Firms need to manage steady state ‘Innovation is the process by which new products or innovation and radical change because methods of production are introduced, including all continuous improvement is no longer the steps from the inventor’s idea to bringing the enough’ new item to market’ (Source: Tom Peters) (Source: Baumol, Economics: Principles & Policy) “Big Data” and it’s close relatives “Cloud  Computing”, “Social Media” and  "Mobile"  are the new frontier of innovation. Driven by Advance Analytics 7
  8. 8. 11/7/2012 Big Data and It’s BrothersVolumeVarietyVelocity……….. Volume Volume is increasing at incredible rates.  With more people using high speed  h l h h d internet connections than ever, plus  these people becoming more proficient  at creating content and just more  people in general contributing  information are combined forces that  information are combined forces that are causing this tremendous increase in  Volume.  8
  9. 9. 11/7/2012 Variety Next in breaking down Big Data into easily digestible bite‐ size chunks is the concept of Variety. Take your personal  experience and think about how much information you  experience and think about how much information you create and contribute in your daily routine. Your  voicemails, your e‐mails, your file shares, your TV  viewing habits, your Facebook updates, your LinkedIn  activity, your credit card transactions, etc.  Whether you consciously think about it or not the Variety  Whether you consciously think about it or not the Variety of information you personally create on a daily basis  which is being collected and analyzed is simply  overwhelming.  VelocityThe speed at which data enters organizations  these days is absolutely amazing. With mega  internet bandwidth nearly being common  place anymore in conjunction with the  proliferation of mobile devices, this simply  gives people more opportunity than ever to  contribute content to storage systems. contribute content to storage systems.  9
  10. 10. 11/7/2012 VELOCITY Worldwide digital content will  double in 18 months, and  every 18 months thereafter.   IDC Mobile Inventory CRM Data GPS Emails Planning Demand Tweets Instant Messages Opportunities SpeedVOLUME Customer Velocity VARIETYIn 2005, humankind  Things 80% of enterprise data created 150 exabytes of  Service Calls will be unstructured, information.  In 2011,  Sales Orders spanning traditional and over 1,200 exabytes was  Transactions non traditional sources.created. Gartner The Economist But I Believe there are Four  V4 10
  11. 11. 11/7/2012• Volume:Gigabyte(109), Terabyte(1012),  Petabyte(1015), Exabyte(1018),  Zettabytes(1021)• Variety: Structured,semi‐structured, unstructured;  Text, image, audio, video, record• Velocity(Dynamic, sometimes time‐varying)• BUT needs to add and create Value! BUT needs to add and create Value! Trends driving data management – The volume of data has never been greater and is  growing exponentially – The value of data has never been better understood The value of data has never been better understood – The capabilities for processing data have never been  better • Higher processor performance and density are enabling  advanced processing on commodity hardware • Software enhancements designed to make best use of  processing performance and scalable architecture i f d l bl hit t • Advanced and in‐database analytics bring processing to the  22 data, reducing latency and improving efficiency – The data deluge problem is also a big data opportunity 11
  12. 12. 11/7/2012 From http://geekandpoke.typepad.com Copyright, 2011 © HBMG, Inc.Next Generation• Cloud Computing (e.g. “Blue Cloud”) – The “network becomes the computer” – D bd i Dumb devices ?????? ??????• Pervasive Computing – Monitoring and tracking almost anything – The Internet is Free• Continuous Services• The Cloud + Pervasive Computing – Smart buildings Smart buildings – Sensor nets – “Invisible computing” or “ubiquitous computing” – Semantic Interoperability – Ad hoc networking Copyright 2012@ HBMG Inc. 12
  13. 13. 11/7/2012 Advance Analytics as a strategic asset“The future belongs to companies and people  that turn data into products.”   Mike Loukides, O’Reilly 25 Advance Analytics as a strategic asset“85% of eBay’s analytic workload is new and  unknown. We are architected for the unknown.”   Oliver Ratzesberger, eBay• Data exploration – data as the new oil  The exploration for data, rather than the exploration of data  Uncovering pockets of untapped data  Processing the whole data set, without sampling  eBay’s Singularity platform combines transactional data with  26 behavioral data, enabled identification of top sellers, driving  increased revenue from those sellers 13
  14. 14. 11/7/2012 Advance Analytics as a strategic asset“Groupon will not be the first or last organization to  compete and win on the power of data. It s happening  compete and win on the power of data It’s happening everywhere.”   Reid Hoffman and James Slavet Greylock PartnersData harnessing – data as renewable energy  H Harnessing naturally occurring data streams i t ll i d t t  Like harnessing raw energy to be converted into usable energy 27  Conversion of raw data into usable data  Facebook 28 14
  15. 15. 11/7/2012BIG DATAREAL TIMEPREDICTIVEENABLED BYADVANCE ANALYTICS As the world gets smarter, infrastructure  demands will grow Smart Intelligent oil Smart Smart Smart traffic field food energy Smart retail healthcare systems technologies systems grids Smart Smart water Smart Smart Smart Smart supply management countries weather regions cities chains 15
  16. 16. 11/7/2012 DataDesktopKnowledge 16
  17. 17. 11/7/2012 Copyright 2012@ HBMG Inc.The Nature of Communications Has Been Changing... People to People to Things to People Things Things TOL L 15.62 Gallons 27.33 Dollars Thank You! Copyright, 2011 © HBMG, Inc. 17
  18. 18. 11/7/2012 Mainframe/Terminal Ubiquitous COSTS Client Server Mini/Terminal USAGE Server/PC Mainframe ASP/Appliance Distribution of Computing Power Virtualization Ubiquitous Improved Access/Convenience Improved Ease of Use Large Organizations + Small Organizations + Departments + BYOD Entrepreneurs Middle Class (PCs) + General Population Satellite Radio Wireless TV Receivers Wireless Cameras Monitors PDAs Digital Cameras Digital Video AdaptersDigital MusicAdapters Smart Phones Game Consoles Desktop PCs Smart Displays Networked Storage Centers Laptop PCs Networked DVD Player Wireless Gaming Mobile Gaming Adapters Devices “Fourth Generation” Movies-on-Demand Set-top Boxes Receivers 802.11 Speakers Digital Media Receivers Personal Video MP3 Players Recorders Copyright, 2012 © HBMG, Inc. 18
  19. 19. 11/7/2012The Global Grid Copyright 2012@ HBMG Inc.To Every Sensor is a Server Phone -PDA Smart Dust Processor Data Storage Microstorage Communications (Areal density 100x’s CD) Rich variety of sensors Microphone Embedded Biofluidics Chip Robot Copyright, 2011 © HBMG, Inc. 19
  20. 20. 11/7/2012Growth at the Edge of the Network 4,000 3,500 • Mobile • Device to Device 3,000Petabytes/Day Global • Sensors • Entertainment 2,500 • Smart Home • Distributed Industrial 2,000 • Autos/Trucks • Smart Toys 1,500 Converged 1,000 Content 500 Traditional Computation 0 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 Year Copyright 2012@ HBMG Inc. DOD Example DOD Example 20
  21. 21. 11/7/2012 Operation Trends Counterinsurgency operations are complex  increased emphasis on:  Information and analysis at lowest levels  Shortened decision making time-scales g  Wider array of information sources Continued growth in volume of data, especially informal information Source: TTI Vanguard Conference - Psydex with limited structure  must transform disparate info to knowledge Processing power and storage capacity increasing faster than communications capacity  must smartly position data and services within networks Increased use of commercial cellular networks  hybrid networks that exploit and interoperate with commercial wireless comms is key Enhancing coalition decision making depends on secure communications and information networks  must address end-to- end problem of data-to-decision (coalition) Info in War Revolution Technology—Information—Organization “Recce” P-38 “Recce” P-38 RF-101 Voodoo I ? I ISR S R S R 0-2 Bird Dog B-17 Spotter Corps B-52 0-2 Bird Dog <10 Minutes 14 Days 75 Days Number f Weapons N b of W Number of Sensors N b fS Required to Target Required to Target 1943 2009 Evolution of Technology, Information, and Culture Enabled Move from  Segregation of Ops and Intel to Integration of Ops and Intel… 42 21
  22. 22. 11/7/2012 Today’s Driving Forces Few Intel Dependencies ‐ Past Many Intel Dependencies ‐ Today Scan Schedules Based on  Single Integrated Intel/Information Current Threat Data Database at Squadron Level ISR Analysts Combined Intel/Ops Center to  Maintain Signature Data g Database/Sensor Engineer Data / g GI&S Datum Models DPPDB MIDB Intel Data Provided  Intel Data  to Operator; Little  MEPED Integrated Into  to No Integration Weapon System Threat Assessments Threat Assessments High‐fidelity ELINT Parametric DataCountermeasures Low Fidelity ELINT Data  Low‐Fidelity ELINT Data Order of Battle Od f B ttl Emitter‐to‐Platform Fit Data E itt t Pl tf Fit D tPre‐mission folders Datum Models Countermeasure Techniques Characteristic & Performance Data Paper Charts Platforms Feed Intel Specific Emitter Identification Data  Increasing Intelligence Needs and Integration Low‐Tech SmartPlatforms/Weapons Platforms/Weapons Intel Time 43 21st Century Challenges: Precision and Information Synergy Strategic Strategic Strategic CYBE CYBE SPAC SPAC Strategic AIR National Operational Operational Operational p Operational R CE CE ER ER Tactical Tactical Tactical Tactical Tactical DESERT 1999 2001 2009 INFO AGE STORM ALLIED ENDURING AF/PAK & Iraq WARFARE1991 & Prior FORCE FREEDOM TODAY TOMORROWSegregated Ops … I + S&R ISR GLOBALIntel & Ops + Intel INTEGRATED ISR OPS Kandahar  Runway NTI Multi‐Domain Real‐Time Fusion INTEL Pod Recce Fusion 22
  23. 23. 11/7/2012 Dimensions of ISR… “More of Everything”More CollectorsBetter Sensors More Data• More Storage• More Comms Better Intel• More Tools• More Analysts• More Linguists …All on an Operationally Responsive Timeline Sensor Data Volume How do we handle all this data? “Rebalancing Collection & PED may be Necessary” 23
  24. 24. 11/7/2012 Persistent Surveillance Data RatesGeneral Norton A. SchwartzAir Force’s chief of staff “If automation can provide a cue for our people that “If i id f l h would make better use of their time, that would help us  significantly,” NY Times.Lieutenant General David Deptula p y f f ff f qFormer Deputy Chief of Staff for ISR, Headquarters, US Air  Force“We’re going to find ourselves in the not too distant future  swimming in sensors and drowning in data” 24
  25. 25. 11/7/2012Advance Analytics• Advanced Analytics and Big Data are two of the  most active areas of innovation in the Tech  sector • legacy infrastructures and government policies are  increasingly at odds with the realities of the analytic  landscape • Certain forms of analysis is no longer possible within  an encrypted environment. Rules that require data  to be encrypted, both while in transit and at rest,  also introduce performance penalties that make it  difficult if not impossible to process large datasets in  an acceptable timeframeTodays CycleWhere is Real Time? 25
  26. 26. 11/7/2012 Advance Analytics • The time to use the output is increasingly getting  shorter – Real Time is becoming very common • Li it d Limited available human resources, and performance is  il bl h d f i often unreliable due to human fatigue and distraction.  Therefore, automated real‐time sensor processing  techniques are required to reliably detect and  discriminate targets of interest • Limited automated processing and tagging tools • – Still NOT enough Advance Analytics• The time to use the output is increasingly getting shorter – Real  Time is becoming very common• Limited available human resources, and performance is often  p unreliable due to human fatigue and distraction. Therefore,  automated real‐time sensor processing techniques are required to  reliably detect and discriminate targets of interest – Still NOT  enough• Need to move to the sensor/collector• Needs to be embedded in the  the sensor 26
  27. 27. 11/7/2012 Autonomous Systems Agents dynamically adapt Agents coordinate to and learn about and negotiate to achieve their environment common goals SocialIntelligent Adaptive Cooperative Personality Information Agents Agents Autonomous Mobile Interoperate Agents are goal directed Agents interoperate Agents move and act on their with humans, other, to where they own performing legacy systems, and are needed tasks on your behalf information sources HBMG Inc. Copyright 2012  Autonomic Networks Self-configuring : Adapt Self-healing: automatically to the Discover, diagnose, dynamically changing and react to environments of link and disruptions from node failures. Self- Self- Self- Self- catastrophes and attacks. Configuring Healing Self-optimizing: Monitor Self- Self- Self- Self- Self-protecting: and tune resources Anticipate, detect, automatically during an Optimizing Protecting identify, and protect attack to minimize its against attacks from attack during and in the anywhere (safety ) (safety.) aftermath. HBMG Inc. Copyright 2012  27
  28. 28. 11/7/2012 Numbers• How many data in the world? – 800 Terabytes, 2000 – 160 Exabytes, 2006 – 500 Exabytes(Internet), 2009 – 2.7 Zettabytes, 2012 – 35 Zettabytes by 2020• How many data generated ONE day? – 7 TB, Twitter Big data: The next frontier for innovation, competition, and productivity McKinsey Global Institute 2011 – 10 TB, Facebook 1 million  illi transactions during this presentation 28
  29. 29. 11/7/2012If You Liked ____, You’ll Love ___ ! 1 billi billion  clicks during this presentation 29
  30. 30. 11/7/2012Gartner Hype Cycle 2012 30
  31. 31. 11/7/2012 2012 Business Intelligence, Analytics and Information  Management Survey from InformationWeek ReportsA few insights from the report:•58% of those surveyed are “very interested” in advanced analytics y y y•Advanced analytics is the No. 1 leading-edge technology•Cloud analytics systems are hot because they are easier on thepocketbook; yet 63% of users have privacy concerns•Data pros just can’t get good data – data quality still ranks as the topbarrier to adopting BI products throughout the company•25% of those surveyed are mobilizing their data analytics withdashboards and data visualizations•40% of d t pros are struggling t stay above th bi d t wave 40% f data t li to t b the big data 31
  32. 32. 11/7/2012 Conclusion Data is one the major factors driving infrastructure computing The growing volume of data is a problem, but it is also an opportunity Don’t worry about ‘big data,’ worry about your data y g , y y Take a Total Data approach to data management • Think pragmatically about data storage and analysis • Attempt to capture and analyze any data that might be relevant,  regardless of where it resides ‘Datastructure’ will become increasingly valuable, not only as a source of  data but also as a source of intelligence The rise of the ‘data cloud’ and the PaaS data layer will encourage a  more flexible approach to data management and analytics The companies that win will be those that think about data as a strategic  asset and implement the technology to monetize it Conclusion The World is moving to Real Time Advanced Analytics is the Key y y Advanced Analytics Must be embedded in the  collectors and sensors• Think about where the data comes from• Attempt to capture and analyze any data that  might be relevant, regardless of where it resides• Realize collaboration is the key in Advance  Analytics just as it is in Business 32
  33. 33. 11/7/2012 If we don’t change our don tdirection, we’ll end up exactly where we are headed. —Ancient Chinese Proverb In Parting: Be Paranoid •“Sooner or later, something  fundamental in your business world  fundamental in your business world will change.” • Andrew S. Grove, Founder, Intel “Only the Paranoid Survive” Only the Paranoid Survive Copyright @2008 HBMG Inc. 33
  34. 34. 11/7/2012Framework 34

×