IBM Solutions Connect 2013 - Getting started with Big Data


Published on

You've heard of Big Data for sure. But what are the implications of this for your organisation? Can your organisation leverage Big Data too? If you decide to go ahead with your Big Data implementation where do you start? If these questions sound familiar to you then you've stumbled upon the right presentation. Go through the presentation to:
a. Learn more on Big data
b. How Big data can help you outperform in your marketplace.
c. How to proactively manage security and risk
d. How to create IT agility to underpin the business

Also, learn about IBM's superior Big Data technologies and how they are helping today's organisations take smarter decisions and actions.

Published in: Business, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • This slide shows two examples of just how instrumented our world has become. On the left is a Brasilian clothing retailer who has linked up smart hangers with Facebook, when you touch it, the Like factor gets incremented and your RFID-enabled card adds what you like to your personal wish list. On the right is the new Nike LeBron James basketball shoes, it’s instrumented such that it can tell you how far you ran, how high you jumped, and so on.
  • Why act now? We’ve set the groundwork of what big data & analytics can do to make you a smarter enterprise. There needs to be a compelling reason to act. The top three reasons to act now: To outperform your industry To manager risk To create IT agility to underpin the business
  • Shinsegae Mall - A leading retailer in South Korea gathers deep insights into consumer behavior and runs targeted online marketing campaigns for greater profitability and loyalty when it implements a solution based on IBM Unica Campaigns software, IBM Netezza Data Warehouse software, IBM InfoSphere software, IBM Cognos software, IBM SPSS Modeling software and IBM Power 570 systems running IBM AIX 6 ( Energy Investment Co. - A renewable wind energy utility company in Beijing improves forecasting accuracy by 15 percent and boosts power-producing capabilities by 10 percent when it engages IBM China Research Lab and deploys a power output-forecasting solution based on a HyREF solution; IBM business analytics and IBM Information Management software; and IBM BladeCenter, IBM System Storage and IBM System x technology ( Life Insurance - A major Korean insurance company estimates gaining nearly USD1.4 billion in profits, reducing liability fees by 12 percent, accelerating the fraud detection process by 50 percent and increasing employee productivity by 60 percent when it engages IBM Business Partner KSTEC to implement a fraud detection solution based on KSTEC SmartWorks FDS software as well as IBM DB2 Enterprise data server, IBM SPSS Modeler, IBM WebSphere ILOG and IBM WebSphere Application Server applications running on IBM System p, IBM System x and IBM System z servers and validated on the IBM Insurance Industry Framework ( Bank of India –A global bank based in India analyzes credit risk in near-real time and cuts reporting time by 92 percent when it taps IBM Global Services - Global Business Services and IBM SPSS Lab Services to implement IBM Business Analytics and IBM Information Management software supported by an IBM WebSphere solution to help it proactively manage risk and comply with Basel II recommendations ( Thiess Pty. Ltd - A mining company in Australia reduces heavy equipment maintenance costs and improves productivity when it works with IBM Global Services - Global Business Services and IBM Research to pilot an advanced condition-monitoring solution based on IBM SPSS software and an IBM DB2 data server (
  • Key PointsWe’re all familiar with the 3 V’sVolume is about rising volumes of data in all of your systems – which presents a challenge for both scaling those systems and also the integration points among themVariety is about managing many types of data, and understanding and analyzing them in their native form.Velocity is about ingesting data in real time and in-motionAnd veracity deals with the certainty, or truthfulness of big data. Veracity is a big issue – and one that directly relates to confidence. In fact, as the complexity of big data rises (the first 3 Vs grow), it actually becomes harder to establish veracity.
  • Key PointsResearch shows that data uncertainty is rising along with the volume of data, and we’re relatively early in the cycle. Why is uncertainty rising?One reason is that we are tapping into external data more than ever before. When combining external data, sometimes from uncertain sources, the overall level of uncertainty rises.Another reason are the various inputs – there are more sources of data. More fragmented records that need to be reconciled.Look at the statistics on the right1/3 make decisions on untrustworthy data. That’s from 2012. What is it like today? Or in 2014? 1/2 lack information and want more, yet 60% have too much data. That’s a paradox. We want more, but we can’t handle it. The answer isn’t making data “smaller”It isn’t ignoring new sources of big data and insight.And it isn’t making the data perfectly certain – that’s a fools errand.It’s about understanding the level of uncertainty, or confidence and acting despite that uncertainty. It’s about making the data good enough that you’re comfortable to act. That’s the new role for Information Integration and Governance.Client Stories & Anecdotes An insurer was gathering data in Hadoop for a telematics use case. They dumped in location data based on a device in your car – which was then used to calculate a potential monthly premium discount based on your actual driving history. But it wasn’t long before the marketing department was asking other questions. What are the household driving patterns? Who was driving the car? How long did they stay at particular locations? The issue of confidence came to the front – and it exposed that they weren’t confident in the data without combining it with other sources (such as master customer data records). Their first step was to better classify and understand that data – using enterprise metadata.Catchy StatementMore data = more uncertainty – yet everyone wants even more data. How will they cope?
  • Key PointsThe value of IBM’s big data & analytics platform is it’s breadth. We have the broadest set of capabilities for big data of any vendor.The whole becomes greater than the sum of the parts once we start integrating those components. And we’ve done that. Our data warehouses and Hadoop systems are well integrated with our IIG capabilities. Our analytic solutions such as Cognos and SPSS are integrated with the relevant components of the big data platform. Our industry solutions teams built industry specific and horizontal solutions based upon big data and analytics. Watson is one such example.Our security, systems, storage, and cloud are optimized for this And our partners build around this platform of capabilities with highly tailored and specific solutions for their clients.At the center of our BD&A offering is our Big Data Platform and Analytics capabilities. Information Integration and Governanace, Content Management, Hadoop, Stream Computing and Data Warehousing. Everything you need to manage and govern your data.The Analytics layer contains BI, Predictive, PM, Risk Analytics, Decision Management and Content Analytics.Solutions that leverage the platform to address specific Ithat address fraud, social media analytics, information lifecycle management
  • Key PointsIn our experience big data is best governed in zones. There are many sources of data – which you see on the left. The era of big data is about exploiting ALL available data – streaming, at-rest structured data, video and images, you name it. So the first requirement of the big data platform is that it can handle all available data.The big data platform must be capable of ingesting information, performing real-time analytics, persisting and analyzing data in warehouses and marts, and applying governance and security throughout each of the other zones. We’ll drill into this in more detail on the next slide.The big data platform enables advanced analytics and new insights. Cognitive capabilities – to learn dynamically and discover further insights. Predictive – to harness the power of big data to predict things your competitors do not see. A whole new set of analytic capabilities enables advanced applications. Watson is a good example – of cognitive and prescriptive capabilities based on a huge volume of big data. New automated processes will emerge – ones that are better informed by data and insight. All to support decisions
  • Takeaway – Choice. Openness. Most comprehensive IM portfolio with broadest set of SW and HW delivery options available. In most large enterprises today, it is common to see a mix of deployment options as clients choose the best deployment strategy to meet the unique requirements of a particular part of the business. For example, a client might use System z to support their manufacturing processes, and use private cloud services to support marketing, HR and finance, and PureData System for various analytic applications. When an area of the business needs: Highest qualities of service (i.e., security, availability, performance, efficiency), System z solutions are a good fit, . Ensure critical data is always available across the enterprise, making it accessible in new ways so that actionable insights can be derived from advanced and operational analyticsProvide ultimate security, ensuring the integrity of critical data while mitigating risk and providing enhanced complianceMost flexibility (i.e., need to run a particular application on a particular set of hardware and/or middleware), multi-platform software as part of custom-built solutions provide highest levels of flexibility for delivering and managing data services. Appliance Simplicity – PureData Systems are pre-integrated & workload-optimized systems that simplify data deployment and management. The Systems include server, storage, network and data management capabilities, pre-built and tuned for specific data workloads: For OLTP workloads: PureData System for Transactions For Operational analytic workloads: PureData System for Operational Analytics For Reporting and Analytic workloads: PureData System for Analytics Cloud Agility – Cloud services offer agility and speed time to market for delivering and managing data services. IBM offers options to provide cloud services like Database as a Service in both private and public cloud environments.
  • Big Data Platform – April 2013DB2 with BLU AccelerationSpeed of Thought Analytics8-25x faster reporting and analytics 10x storage space savings seen during beta testNoindexes, aggregates, tuning, or SQL / schema changesBig Data PlatformPlatform advances in consumability and performanceBig SQLstandard ANSI SQL access to data in BigInsights – standard SWL access to HadoopGPFS-FPO with POSIX compliance and enhanced security2-10xfaster Streams operations using bounded lists & maps - SpeedPureData System of HadoopExplore and analyze more data with appliance simplicity8x faster deployment than custom-built solutionsFirst appliancewith built-in analytics accelerator Only Hadoop system with built-in archiving toolsClaims based on product specs, IBM lab tests or client / partner beta test experience. Detailed footnotes on distribution version.
  • Key PointsWe’ve increased our momentum year after yearYou can see it in the growth in clients – with thousands of big data and analytics engagements we have the breadth of that experience. That experience drives our product roadmaps, our innovation, and our services people who know how to implement big data quickly.Over 100,000 registrants in big data university – that’s an incredible accomplishment. Our goal is to raise the market’s education level on big data and analytics and this has been an tremendous success, along with our developer days and hackathons. In addition to the 500+ big data platform partners, we have 2215 partners across big data and analytics. That’s the multiplying effect of the platform – those partners augment our technology with unique solutions that add value to specific markets. Catchy StatementOur momentum has been strong – but we think it will get stronger still with today’s announcement.Confidence in big data makes organizations confident that they can start this journey – and they can start it with a partner who will make them successful.
  • I'm pleased and excited to announce that IBM has jumped over Informatica in the Leader quadrant of the just published2013 Gartner Data Integration Tools MQ! In this report, IBM made the greatest positive movement of any vendor evaluated as part of the MQ. Additionally, with this ranking, IBM is #1 in two of the most recent Gartner reports -- the 2013 Data Integration Tools Magic Quadrant and the December 2012 Gartner Critical Capabilities for Data Integration report. Not only did we outperform Informatica in 2012 in the Data Integration Tools Magic Quadrant, our customer references also showed notable improvement in satisfaction with their overall customer experience relative to prior years. Keep those references coming in team, so that we can see more improvement next year! As part of the MQ, Gartner lauds IBM for our breadth of functionality, installed based and diversity of use, and alignment with information infrastructure and enterprise information management (EIM) trends. Here's a summary of our strengths, according to Gartner:Breadth of functionality Reference customers routinely cite [as strengths and their reasons for selecting IBM] the sheer breadth of functionality of the vendor's product set across …data integration styles, the degree of integration between the components …via common metadata and the scalability they can achieve in the face of high-volume requirements.Installed base and diversity of use IBM's tools are often deployed as an enterprise-wide standard. The scope and scale of the implementations is often large ... The customer base shows very heavy usage in BI/analytics and data warehousing scenarios, [but] reference customers also show diversity across a range of application types (including MDM, data migration and operational application integration). During 2012, IBM [achieved] solid growth of its data integration tool business, reflecting strong execution. Alignment with information infrastructure and enterprise information management (EIM) trendsCustomers view the broad and deep metadata management functionality as critical to the early stages and ongoing value in their EIM programs, and believe this enables them to derive greater value from IBM's data integration tools. Version 9.1 of Information Server introduced deeper support for Hadoop and the new InfoSphere Data Click functionality aimed at enabling power users to perform self-service data preparation for analytics purposes.
  • Why is Big Data so cool? Big Data provides objective information about people’s behaviors. Not their belief or morals, not what they want their behavior to be, or what they tell the world their behavior is, but honest to goodness unedited actions: their clicks on Web sites, comments on l Media Convl media classes and so on. Scientists can tell an enormous amount about you because of this data, more than the best surveys and research focus groups, or a Dr.’s interview.Consumability is really important, think about it for a moment. In a 20 person start-up, it’s easy enough for everyone to learn Hadoop and such in a month or so and start using it. But that’s just not the case in a large enterprise.
  • IBM Solutions Connect 2013 - Getting started with Big Data

    1. 1. Getting Started with Enterprise Big Data – From Concept to Reality
    2. 2. Four Technologies Help Define the Smarter Enterprise CLOUD COMPUTING ENTERPRISE MOBILITY BIG DATA ANALYTICS SOCIAL BUSINESS Client-centric, digitally savvy in its use of cloud, mobile, social and big data platforms to transform Embraces data in all forms to apply analytics, unlock insight, and make fact-based decisions Creates value in new ways by forging deeper relationships with clients and between employees Constantly adapts to changing market dynamics, buyer demands and disruptive technologies
    3. 3. The number of organizations who see analytics as a competitive advantage is growing. 63% 2010 business initiative 2011 2012 BUSINESS IMPERATIVE IQ
    4. 4. What’s Changing?: Big Data & Analytics Is Expanding Quickly Data is the world’s newest resource Decision-making extends from few to many As data value grows, current systems won’t keep pace
    5. 5. 5
    6. 6. Why Act Now? To outperform in your industry To proactively manage security and risk To create IT agility to underpin the business
    7. 7. Examples of Outstanding Performance Driven by Big Data and Analytics Traditional Approach One size fits all marketing å Manual weather forecasting Slow claims processing Transformational Outcomes Personalized, realtime marketing offers Real-time, automated weather prediction Intelligent & accelerated fraud detection Monthly risk management Real-time Risk Analysis Just in time maintenance Predictive maintenance & improved uptime
    8. 8. To Manage Risk and Create Agility: Embrace All Data ….Uncertainty of New Information is Growing Alongside its Complexity Volume Data at Scale Terabytes to petabytes of data Variety Data in Many Forms Structured, unstructured, text, multimedia Velocity Veracity Data in Motion Data Uncertainty Analysis of streaming data to enable decisions within fractions of a second. Managing the reliability and predictability of inherently imprecise data types.
    9. 9. The Big Data Conundrum  The economies of deletion have changed…. • Leading us into new opportunities and challenges • The percentage of available data an enterprise can analyze is decreasing proportionately to the available to that enterprise • Quite simply, this means as enterprises, we are getting “more naive” about our business over time • Just collecting and storing “Big Data” doesn’t drive a cent of value to an organization’s bottom line Data AVAILABLE to an organization Data an organization can PROCESS
    10. 10. By 2015, 80% of All Available Data Will Be Uncertain 1 in 3 9000 7000 90 80 6000 70 5000 60 50 4000 40 3000 30 20 2000 Aggregate Uncertainty % Global Data Volume in Exabytes 8000 10 0 Rising Uncertainty = Declining Confidence 1 in 2 Lack the information that they need We are here. Sensors Internet of things Social media 10 Video, Audio and Text 1000 0 Make decisions on untrustworthy data VoIP Enterprise Data Multiple sources: IDC, Cisco 2005 2015 2010 60% Have too much data
    11. 11. IBM Big Data and Analytics: Helps You Outperform, Manage Risk and Create IT Agility CONSULTING and IMPLEMENTATION SERVICES SOLUTIONS Sales Marketing Finance Risk IT Operations HR Watson and Industry Solutions ANALYTICS Content Decision Analytics Managemen t Business Intelligence and Predictive Analytics Performance Management Risk Analytics BIG DATA PLATFORM Content Management Hadoop System Stream Data Computin Warehouse g Information Integration and Governance SECURITY, SYSTEMS, STORAGE AND CLOUD Scale Management Parallel Processing Low Latency Resources Data Optimization The Whole is Greater Than the Sum of the Parts  Broadest set of capabilities across big data and analytics  Pre-integrated components accelerate value  Pre-built industry and horizontal solutions  Integration and optimization with storage and infrastructure  Delivered in multiple forms: software, appliance, and cloud  World-class consulting and implementation drives innovation and value
    12. 12. Big Data and Analytics Solutions Across Industries Banking Insurance Telco Energy & Utilities Media & Entertainment  Optimizing Offers and Cross-sell  360˚ View of Domain or Subject  Pro-active Call Center  Smart Meter Analytics  Business process transformation  Customer Service and Call Center Efficiency  Catastrophe Modeling  Network Analytics  Distribution Load Forecasting/Scheduli ng  Audience & Marketing Optimization  Fraud & Abuse  Location Based Services  Condition Based Maintenance Retail  Actionable Customer Insight  Customer Analytics & Loyalty Marketing  Merchandise Optimization  Predictive Maintenance Analytics  Dynamic Pricing Automotive  Advanced Condition Monitoring  Data Warehouse Optimization Consumer Products Travel & Transport Chemical & Petroleum  Operational Surveillance, Analysis & Optimization  Data Warehouse Consolidation, Integration & Augmentation Government Healthcare  Shelf Availability  Civilian Services  Promotional Spend Optimization   Defense & Intelligence  Measure & Act on Population Health Outcomes Merchandising Compliance  Tax & Treasury Services  Engage Consumers in their Healthcare Aerospace & Defense Electronics  Uniform Information Access Platform  Customer/ Channel Analytics  Data Warehouse Optimization  Advanced Condition Monitoring Life Sciences  Increase visibility into drug safety and effectiveness
    13. 13. Harvest Business Value via Key Business-Driven Use Cases Enrich Your Information Base with Big Data Exploration Reduction In Time Required For Analysis Help Reduce Risk and Prevent Fraud with Security and Intelligence Extension 1,100 99% Improve Customer Interaction with Enhanced 360 View of the Customer 42TB Association Publishing Partnerships Big Data Exploration Find, visualize, understand all big data to improve business knowledge Real-time Acoustic Data Analyzed Enhanced 360o View of the Customer Security/Intelligence Extension Achieve a true unified view, incorporating internal and external sources Lower risk, detect fraud and monitor cyber security in real-time Optimize Infrastructure and Monetize Data with Operations Analysis 60K Metered Customers in Five States Gain IT Efficiency and Scale with Data Warehouse Augmentation 40X Gain in Analysis Performance Operations Analysis Data Warehouse Augmentation Analyze a variety of machine data for improved business results Integrate big data and data warehouse capabilities to increase operational efficiency
    14. 14. Big Data & Analytics Reference Architecture Cognitive Computing Real-time Analytics Data in Motion Information Ingestion and Operational Information Landing Area, Analytics Zone and Archive Exploration, Integrated Warehouse, and Mart Zones Data at Rest Information Governance, Security & Business Continuity Data in Many Forms Real-time Analytics & Decision Management DecisionMaking Planning & Forecasting Predictive Analytics & Content Analytics Reporting, Analysis & Dashboards Business Processes Data Discovery & Visualization Security, Systems, Storage and Cloud Point of Interaction
    15. 15. Infrastructure Matters to Support New Big Data & Analytics Architecture Core infrastructure capabilities deliver speed and confidence Data Optimization Low Latency Parallel Processing Scalability An efficient and agile infrastructure balances the needs of different analytics workloads Optimal Infrastructure Predictive Analytics Data Warehouse SCM* Cores Text Analytics Hadoop Workloads Optimization Sensitivity Analysis Network Storage * SCM-Storage Class Memory
    16. 16. Delivering Workload Optimized Performance System for Transactions System for Analytics For apps like Order Management For apps like Sales Analysis Database cluster services optimized for transactional throughput and scalability Data warehouse services optimized for high-speed, peta-scale analytics and simplicity System for Operational Analytics System for Hadoop For apps like Real-time Fraud Detection For apps like Big Data Exploration Operational data warehouse services optimized to balance high performance analytics and real-time operational throughput Hadoop services optimized for exploration of large volumes of data with any type of structure; and as a queryable archive to augment traditional data warehousing
    17. 17. Complementary Analytics Traditional Approach New Approach Structured, analytical, logical Creative, holistic thought, intuition Data Warehouse Hadoop and Streams Multimedia Transaction Data Web Logs Internal App Data Mainframe Data Social Data Structured Repeatable Linear Unstructured Exploratory Dynamic Sensor data: images OLTP System Data ERP Data RFID Traditional Sources 17 Text Data: emails New Sources
    18. 18. A Year of Innovation for Big Data & Analytics AGILE GOVERNANCE FOR ALL DATA Single point Find and protect sensitive data 80% faster monitoring of security for traditional, NoSQL, and big data PERFORMANCE MANAGEMENT and BUSINESS INTELLIGENCE Cognos TM1 with Mobile contribution Deploy on Cloud, zLinux, on premise. Integrated metrics and scorecarding Native mobile on iOS and Android INFRASTRUCTURE Analytics on POWER 7-14x lower TCO X-86 innovation – 40% better perf efficiency System x – open analytics on Linux IBM Flash Systems for low latency analytics. Real-time compression to access all relevant data
    19. 19. IBM Big Data & Analytics Momentum 40,000 Members 1550 30,000 1040 1100 730 170 Big data Clients Business Partners Big Data Clients 85 Info Agenda Engagements 2010 Big Data Clients 860 Info Agenda Engagements 2011 10,000 Big Data University Enrollments Big Data Clients 9th Analytics Solution Center Opens in Ohio GBS Information and Analytics Engagements 1640 2215 Business Partners 2,300 Info Agenda Engagements 40,000 Big Data University Enrollments Business Partners 3,810 Info Agenda Engagements 2012 Source: IBM. Note: All numbers used are cumulative. 3/31/2013 101,000 Big Data University Enrollments 2013
    20. 20. 2013 Gartner Magic Quadrant – IBM Jumps Ahead 20
    21. 21. IBM Is Helping Address the Analytics Skills Gap  New technologies designed for business users  IBM AnalyticsZone to download and experiment with software  Big Data University with robust curriculum  Big Data Stampede for accelerated value  Partnering with major universities globally  On-line resource centers & books written by IBM thought leaders
    22. 22. How to Get Started 1. Build a culture that infuses analytics everywhere Develop a curiosity-driven and evidence-inspired workforce 2. Be proactive about privacy, security and governance Forward-thinking approaches to maximize impact while balancing risk 3. Invest in a Big Data & Analytics platform Build to master plan: all data, all analytics, full range of business outcomes
    23. 23. NO OTHER VENDOR can make this statement IBM delivers a governable, consumable Big Data platform that’s steeped in analytics for data in-motion and data at-rest.
    24. 24. © Copyright IBM Corporation 2013 All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Cognos, the Cognos logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.