Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Data Science and Smart Systems: Creating the Digital Brain

2,587 views

Published on

Big Data technologies enable us to build the digital brain of smart systems. I will illustrate with examples how we build a digital brain by collecting data from a large number of sensors and using the brain to find value in that data. We build a Data Lake using cutting edge technology from Pivotal and use it to store large amounts of sensor and other data. Then we can find patterns in that data by applying the Data Science methodology using sophisticated machine learning and statistical algorithms customized to run on big data within the Data Lake. Armed with these patterns the system can detect anomalies and respond in an appropriate manner. Data Science combined with sensors and actuators can make a system smart!

Published in: Technology
  • Be the first to comment

Data Science and Smart Systems: Creating the Digital Brain

  1. 1. A NEW PLATFORM FOR A NEW ERA
  2. 2. Creating the Digital Brain Kaushik Das Senior Principal Data Scientist © Copyright 2014 Pivotal. All rights reserved. 2
  3. 3. TODAY We have analytics on Big Data …and we have smart machines. © Copyright 2014 Pivotal. All rights reserved. 3
  4. 4. But That Does not Help Us Prevent Accidents Like the Macondo Disaster © Copyright 2014 Pivotal. All rights reserved. 4
  5. 5. WHAT IF We Could Prevent Disasters Like This ? © Copyright 2014 Pivotal. All rights reserved. 5
  6. 6. We Have All the Ingredients of a Smart System Sensors © Copyright 2014 Pivotal. All rights reserved. Actuators Decision Support Tools 6
  7. 7. But Where is the Brain? Sensors © Copyright 2014 Pivotal. All rights reserved. ? Actuators 7
  8. 8. The Brain Brings it All Together The Brain: 1.takes in the input from the eyes 2.analyzes it to compute the trajectory of the ball 3.tells the body what action to take to hit the ball © Copyright 2014 Pivotal. All rights reserved. 8
  9. 9. Let’s Put in a Digital Brain and Make Systems Smart © Copyright 2014 Pivotal. All rights reserved. 9
  10. 10. Let’s Build a Digital Brain The brain brings it all together 1.takes in the input from a large number of sensors 2.Builds a model and uses it to analyze incoming data 3.tells the actuators what action to take Over a network of machines © Copyright 2014 Pivotal. All rights reserved. 10
  11. 11. And Now Let’s Prevent Disasters Input: Action: Shut down to prevent blowout Sensors in drill The Digital Brain: Extracts patterns and flags outliers/anomalies © Copyright 2014 Pivotal. All rights reserved. 11
  12. 12. Smart System = Sensors + Digital Brain + Actuators Data Lake Sensors & actuators Data Science for Building Models
  13. 13. Data Lake Architecture Unified Sources Centralized Management System monitoring Real-time ingestion System management Unified Data Management Tier Data mgmt. services Flexible Actions MDM RDM Audit and policy mgmt. Real-time insights Workflow management Micro batch ingestion Processing Tier In-memory Interactive insights MPP database Batch ingestion Distillation Tier Batch insights HDFS storage Unstructured and structured data © Copyright 2014 Pivotal. All rights reserved. 13
  14. 14. The Digital Brain: Making a Smart Grid Smarter! Input: Data from smart meters Action: Where to send trucks and when, preventive maintenance The Digital Brain: Uses Fourier transform extracts patterns and flags outliers/anomalies © Copyright 2014 Pivotal. All rights reserved. 14
  15. 15. The Data Science that Goes into the Digital Brain Identify patterns based on frequency © Copyright 2014 Pivotal. All rights reserved. Detect anomalies 15
  16. 16. Now the Power Company Knows a Tree has Fallen Even Before the Residents Complain © Copyright 2014 Pivotal. All rights reserved. 16
  17. 17. “Zero unplanned downtime is a key goal for GE’s use of the Industrial Internet” Jeff Immelt, GE © Copyright 2014 Pivotal. All rights reserved. 17
  18. 18. WHAT IF We can take this even further – what about zero unplanned outages, zero industrial accidents and zero environmental disasters? © Copyright 2014 Pivotal. All rights reserved. 18
  19. 19. FOR MORE DETAILS http://blog.gopivotal.com/ features/creating-the-digital-brain © Copyright 2014 Pivotal. All rights reserved. 19
  20. 20. A NEW PLATFORM FOR A NEW ERA

×