Big Data: Transform Industries, Institutions, and Societies with Watson

Uploaded on

Paul Yip, Big Data Platform Technical Leader on the power of Watson and IBM's Cognitive Technologies Group with the ability to analyze incredibly complex systems and produce actionable solutions for a …

Paul Yip, Big Data Platform Technical Leader on the power of Watson and IBM's Cognitive Technologies Group with the ability to analyze incredibly complex systems and produce actionable solutions for a smarter world.

Presented at the #SmarterBiz Summit in Vancouver, BC on June 11, 2014.

For more information, contact YPaul[at]

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. 1 #SmarterBiz Transform Industries, Institutions, and Societies with Watson Paul Yip Big Data Platform Technical Leader IBM Canada
  • 2. 2 © 2014 International Business Machines Corporation “Data is the New Oil” “We have for the first time an economy based on a key resource [Information] that is not only renewable, but self-generating. Running out of it is not a problem, but drowning in it is.” – John Naisbitt Harvesting any resource requires Mining, Refining and Delivering Big Data is the next Natural Resource
  • 3. 3 #SmarterBiz © 2014 IBM Corporation3 How to you find the needle(s) in a haystack?
  • 4. 4 #SmarterBiz • Data is a Natural Resource • Survive or Thrive: Drive insights from that data • Infuse analytics into everything Transform Industries, Institutions, and Societies with Watson
  • 5. 5 #SmarterBiz What is Watson? Watson Foundations IBM Portfolio of solutions – Information Infrastructure – Big Data Governance & Security – Extract value and capability from any source of data – Allows us to better understand the data and take the best actions, plan for the future IBM Watson Group – Cognitive Technologies – Process information more like a human – Cognitive Ecosystem
  • 6. 6 © 2014 International Business Machines Corporation Dublin City Centre; Robust and efficient citywide traffic awareness system, enhance rapid action on incidents Need • A budget effective solution to improve traffic awareness system. • To bring accuracy in event detection, inferring traffic condition (road speed) and prediction of bus arrival. • Challenge is to rightly analyze GPS data, which is typically high data throughput and difficult to capture Benefits • Monitor arrival times of 1000 buses across 150 routes daily • Analyzes 50 bus location updates per second , using InfoSphere Streams • Collects, processes, and visualizes location data of all public transportation vehicles 66 Home
  • 7. 7 #SmarterBiz Municipal Data – City of Dublin Data Source Description Approx. Scale Bus System GPS Data Speed Data Stop Data Fare Data 1000 Buses Location reported every 20 seconds Traffic Flow Data Traffic Light Data XML 24 Intersections CCTV monitoring Real-time video Stream across the city Road Weather Conditions CSV 54 Stations Road Works and Maintenance CSV Dublin Event Data Small Attendee Events Large Attendee Events
  • 8. 8 #SmarterBiz Optimization Opportunity Examples Understand and Reduce Traffic Congestion – Look at all available localized data and determine possible causes – Improve public transportation Optimize Energy Consumption Understand how people use the city’s infrastructure – + Water management data – + Utilities data – + Environmental monitoring data – + Consumption data (energy, water, waste) – + Environmental impact (noise pollution)
  • 9. 9 #SmarterBiz Information Challenges in Big Data Need to Integrate all different types of data • Many formats • Structured, unstructured • Data at rest, real-time streaming data Data is noisy & erroneous – GPS data example • Bus moving at 300km/hr • Bus located in the middle of a river • Data may be incomplete
  • 10. 10 #SmarterBiz Volume Variety Velocity Veracity Data at Scale Terabytes to petabytes of data Data in Many Forms Structured, unstructured, text, multimedia Data in Motion Analysis of streaming data to enable decisions within fractions of a second. Data Uncertainty Managing the reliability and predictability of inherently imprecise data types. Velocity IS the game changer: It’s NOT just how fast data is produced or changed, BUT the speed at which it must be received, understood, and processed.
  • 11. 11 #SmarterBiz Information Integration & Governance Systems Security On premise, Cloud, As a service Storage IBM Big Data & Analytics Infrastructure New/Enhanced Applications All Data What action should I take? Decision management Cognitive Fabric Landing, Exploration and Archive data zone EDW and data mart zone Operational data zone Real-time Data Processing & Analytics What is happening? Discovery and exploration Why did it happen? Reporting and analysis What could happen? Predictive analytics and modeling Deep Analytics data zone IBM Watson Foundations
  • 12. 12 #SmarterBiz City of Fresno, CA – “Shrink Your Bubble” Tackling Rubbish and Illegal Dumping in Alleyways + Municipal Code Offence Data + School location Data + Engage Journalism Students – Blogs & Social Media + Challenge communities to clean up their part of the city + Drive ward level policy changes
  • 13. 13 #SmarterBiz A customer is a puzzle made up of many pieces Professional Life Employers, professional groups, certifications … Legal/Financial Life Property, credit rating, vehicles, … Contact Information Name, address, employer, marital… Business Context Account number, customer type, purchase history, … Leisure Hobbies, interests … Every interaction requires someone to piece together parts of the puzzle Social Media Social network, affiliations, network … Etc…. Information about your customers is dispersed, forcing your employees to extract it piece- by-piece
  • 14. 14 #SmarterBiz Individual silos can answer typical questions, one-by-one Wiki Who is best able to help this customer? Experts What is her view of our company? Social Media Fulfillment What issues has this customer had in the past? Support Ticketing Where else has she worked? External Sources Who is this customer? CRM What is available inventory? Supply Chain Email How is her company using our products? Content Mgt. What products has she purchased? DBMS … but an enhanced 360º view provides answers in one application
  • 15. 15 #SmarterBiz Enhanced 360º View answers questions that require multiple systems WikiExperts What should I know before calling her for renewal? Social Media What marketing materials should I send? Support Ticketing What’s going on with this customer TODAY? External Sources What products can I upsell this customer? CRM How can we increase engagement with her? Supply Chain Email How can we get more customers like her? Content Mgt. What impact will inventory have on her? DBMS Fusion of data from multiple systems enables deeper insights—not just facts Fulfillment
  • 16. 16 © 2014 International Business Machines Corporation List of past purchases by this contact from order tracking system Recent conversations from multiple sources: e.g., CRM, e-mail, etc. Product offers based on past purchases and conversations Contact information from CRM Consolidated list of products owned based on account affiliation
  • 17. 17 © 2014 International Business Machines Corporation Making each investor feel like they’re #1 Large brokerage and financial services firm Leading provider of workplace and individual retirement savings plans, mutual funds and other financial products for 20 million customers Secure access to over 30 different silos to empower agents to engage high- value customers to promote up-selling and cross-selling 17
  • 18. 18 #SmarterBiz Enhanced 360º View is complemented by master data management Unified View of Party’s Information CRM J Robertson Pittsburgh, PA 15213 35 West 15th Name: Address: Address: ERP Janet Robertson Pittsburgh, PA 15213 35 West 15th St. Name: Address: Address: Legacy Jan Robertson Pittsburgh, PA 15213 36 West 15th St. Name: Address: Address: SOURCE SYSTEMS BigInsights Streams Warehouse Master Data Management Janet 35 West 15th St Pittsburgh Robertson PA / 15213 F 48 1/4/64 First: Last: Address: City: State/Zip: Gender: Age: DOB: 360 View of Party Identity Unified View of Party’s Information
  • 19. 19 #SmarterBiz Volume Variety Velocity Veracity Data at Scale Terabytes to petabytes of data Data in Many Forms Structured, unstructured, text, multimedia Data in Motion Analysis of streaming data to enable decisions within fractions of a second. Data Uncertainty Managing the reliability and predictability of inherently imprecise data types. Velocity IS the game changer: It’s NOT just how fast data is produced or changed, BUT the speed at which it must be received, understood, and processed.
  • 20. 20 #SmarterBiz BigInsights: Value Beyond Open Source Enterprise Capabilities Administration & Security Workload Optimization Connectors Open source components Advanced Engines Visualization & Exploration Development Tools IBM-certified Apache Hadoop and related projects Key differentiators • Built-in text analytics • Accelerators • Enterprise software integration • Industry Best SQL support • Spreadsheet-style analysis • Security Enhancements • GPFS option • World-class support • Full open source compatibility Business benefits • Faster time-to-value due • Reduced operational risk • Enhanced business knowledge with flexible analytical platform • Leverages and complements existing software
  • 21. 21 © 2014 International Business Machines Corporation Constant Contact Transforming Email Marketing Campaign Effectiveness with IBM Big Data Need • Analyze 35 billion annual emails to guide customers on best dates & times to send emails for maximum response Benefits • 40 times improvement in analysis performance • 15-25% performance increase in customer email campaigns • Analysis time reduced from hours to seconds 2121 Home
  • 22. 22 © 2014 International Business Machines Corporation Asian Health Bureau reduces diagnostic errors Need • Telemedicine imaging diagnostics service to improve rural healthcare • Automatically sifts and analyzes large collections looking for anomalies and disease Benefits • Makes it possible for radiologists and Pathologists to analyze 1000s of patient images • Reduction in diagnostic errors • Improved outcomes by leveraging physicians treating similar cases “Over 80% of healthcare data is medical imaging”2222 Home
  • 23. 23 #SmarterBiz Achieve scale: By partitioning applications into software components By distributing across stream-connected hardware hosts Infrastructure provides services for Scheduling analytics across hardware hosts, Establishing streaming connectivity Transform Filter / Sample Classify Correlate Annotate Where appropriate: Elements can be fused together for lower communication latency  Continuous ingestion  Continuous analysis InfoSphere Streams for Real Time Analytics © 2013 IBM Corporation
  • 24. 24 #SmarterBiz 7200+ hearts/hour 2000+ breaths/hour 3600 blood oxygenation readings on screen Only 1 record/hour for heart rate respiratory rate blood oxygenation Big Data Saves Lives 12-24 hours before any sign of trouble, almost undetectable changes begin to appear in the vital signs of infants who have contracted an infection University of Ontario Institute of Technology (UOIT)
  • 25. 25 #SmarterBiz How to Find a Needle in a Haystack?
  • 26. 26 #SmarterBiz On February 14, 2011 made history
  • 27. 27 #SmarterBiz Then… Tabulating Systems Era Programmable Systems Era Cognitive Systems Era 1900 1950 2011
  • 28. 28 #SmarterBiz Understands natural language and human style communication Adapts and learns from training, interaction, and outcomes Generates and evaluates evidence-based hypothesis 1 2 3 Watson understands me. Watson engages me. Watson learns and improves over time. Watson helps me discover. Watson establishes trust. Watson has endless capacity for insight. Watson operates in a timely fashion. Watson combines transformational capabilities to deliver a new world experience
  • 29. 29 #SmarterBiz The customer experience lacks personalization, timeliness, & trust “We have recently connected with the IBM team to analyze potential opportunities to deploy Watson’s intelligence to better enable USAA to broadly serve and advise our membership during complex life experiences.” Wayne Peacock EVP Member Experience USAA • Impersonal – scripted speech • Process driven – not personal • Circumspect – not trusting You don’t know me You’re not connecting with me • Insulting – not engaging • Confusing – conflicting instructions • Maddening – lack of resolution It’s too hard • Time Consuming – long waits • Inconvenient – hours of operation, channels supported • Frustrating – repeating it over and over
  • 30. 30 #SmarterBiz IBM Watson Engagement Advisor Demonstrating an uncanny understanding of the customer in context Delivering personalized, trusted customer interactions regardless of the channel Empowering customers to take action in an intuitive, fast, accurate, and consistent fashion Continuously listens to, and learns from customers Fostering trust and respects customer privacy Be persistent and understanding how we have already interacted Know Me Empower Me Engage Me
  • 31. 31 #SmarterBiz Watson has the potential to transform how industries and individuals interact and make every day decisions. Watson helps make individuals, enterprises and society smarter by engaging the collective knowledge and insight of the market. •IBM Watson Group •2000+ professionals •$1B+ invested over next few years •New Watson HQ – NY (Silicon Alley) •New Watson Offerings •$100M equity investment in the Watson Ecosystem IBM is investing in Watson’s future – January, 2014 announce
  • 32. 32 #SmarterBiz Memorial Sloan Kettering (MSK) IBM Watson for Evidence Based Cancer Treatment
  • 33. 33 #SmarterBiz Watson Engagement Advisor May 21st Watson Engagement Advisor announced Cross-industry and cross- geography play January April July October First Two Commercial Offerings February 8th First 2 healthcare products were announced with MSK and WellPoint 2014 Watson Ecosystem November 14th  Watson Ecosystem announced  Driving innovation and fostering new possibilities Watson Healthcare Update Oct. 18 MD Anderson launches Expert Oncology Advisor powered by Watson Watson used to bridge clinical practice and medical research The journey so far
  • 34. 34 #SmarterBiz 34 The Watson Ecosystem empowers development of “Powered by IBM Watson” applications. Partners are building a community of organizations who share a vision for shaping the future of their industry through the power of cognitive computing. What is the Watson Ecosystem?
  • 35. 35 #SmarterBiz Fluid Inc., Powered by IBM Watson
  • 36. 36 #SmarterBiz Engage with the Customer
  • 37. 37 #SmarterBiz Expert shopping assistant continues the conversation in natural language,…
  • 38. 38 #SmarterBiz Brings together sources of free and for-fee data, including general knowledge, industry-specific content, and subject matter expertise to help power Powered by Watson apps. Watson Developer Cloud Watson Content Store Watson Talent Hub + + Tools and APIs enabling ISVs with self-service training, development, and testing of their cognitive application. Helps to bridge ISVs’ resource gaps by providing a marketplace for skills like linguistics, natural language processing, machine learning, user experience design, and analytics. engaged innovators in first 60 days 1000+ Delivering the promise of cognitive to the world million equity investments $100 subject matter experts 500+
  • 39. 39 #SmarterBiz Get Started: Talk to me today! • Watson Foundations • Watson Explorer • Watson EcoSystem
  • 40. 40 #SmarterBiz