Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Xanadu for Big Data + IoT + Deep Learning + Cloud Integration Strategy

2,909 views

Published on

Big Data in IoT & Deep Learning
Challenges of IoT Big Data Analytics Applications
Challenges of Cloud-based IoT Platform
Cloud-based IoT Platform Use Case: GE Predix for Smart Building Energy Management
Fog/Edge Computing & Micro Data Centers
Deep Learning for IoT Big Data Analytics Introduction
Deep Learning for IoT Big Data Analytics Use Case
Distributed Deep Learning
Big Data + IoT + Cloud + Deep Learning Insights from Patents
Big Data + IoT + Cloud + Deep Learning Strategy Development
Designing Data-Intensive Applications
Xanadu Functionality
Xanadu Use Case
Xanadu + Deep Learning + Hadoop Integration

Published in: Data & Analytics
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Xanadu for Big Data + IoT + Deep Learning + Cloud Integration Strategy

  1. 1. ©2017 Xanadu Big Data, LLC All Rights Reserved www.xanadubigdata.com Xanadu for Big Data + IoT + Deep Learning + Cloud Integration Strategy July 25, 2017 Alex G. Lee (alexglee@xanadubigdata.com)
  2. 2. ©2017 Xanadu Big Data, LLC All Rights Reserved Index -Big Data in IoT & Deep Learning -Challenges of IoT Big Data Analytics Applications -Challenges of Cloud-based IoT Platform -Cloud-based IoT Platform Use Case: GE Predix for Smart Building Energy Management -Fog/Edge Computing & Micro Data Centers -Deep Learning for IoT Big Data Analytics Introduction -Deep Learning for IoT Big Data Analytics Use Case -Distributed Deep Learning -Big Data + IoT + Cloud + Deep Learning Insights from Patents -Big Data + IoT + Cloud + Deep Learning Strategy Development -Designing Data-Intensive Applications -Xanadu Functionality -Xanadu Use Case -Xanadu + Deep Learning + Hadoop Integration
  3. 3. ©2017 Xanadu Big Data, LLC All Rights Reserved Big Data in IoT & Deep Learning Big Data IoT Deep Learning Volume: Size of Data Very large volumes of data from sensors and connected devices Increase in algorithm complexity and computing time Velocity: Data Processing Speed Real time streaming data of very shot time scales, high frequencies, high ingestion rates Difficulty in data learning process in a timely manner Variety: Different Types of Data Heterogeneous datasets from geographically distributed diverse IoT sensors and connected devices Increase in data processing diversity for different data characteristics and behavior Veracity: Truthfulness of Data Noisy & incomplete data which are characterized by uncertainty; Data security issues Significant increase in pre-processing of data
  4. 4. ©2017 Xanadu Big Data, LLC All Rights Reserved How to provide interoperability among heterogenious IoT data streams in formats, velocities, and semantics? How to provide data reliability by taking into account noisy and incomplete nature of IoT data streams? How to process nearly real-time streaming IoT data? How to deal with temporal and special dependences of IoT data? How to comply with security and privacy requirements of IoT data? Challenges of IoT Big Data Analytics Applications
  5. 5. ©2017 Xanadu Big Data, LLC All Rights Reserved A cloud-based IoT platform should provide a dynamic and flexible analytics resource sharing platform delivering IaaS, PaaS, and SaaS. A cloud-based IoT platform should provide optimized data management and processing across multiple geographically distributed datacenters. A cloud-based IoT platform should provide required quality of service guarantees regarding data storage management and security and privacy of sensitive data. A cloud-based IoT platform should provide low latency end-to-end processing of IoT data. Challenges of Cloud-based IoT Platform
  6. 6. ©2017 Xanadu Big Data, LLC All Rights Reserved Cloud-based IoT Platform Use Case: GE Predix for Smart Building Energy Management GE Current demo@GE Predix Boston (Industrial Internet) Meetup
  7. 7. ©2017 Xanadu Big Data, LLC All Rights Reserved Fog/Edge computing extends the cloud computing and services to near the IoT sensors and connected devices. Fog/Edge computing is similar to the cloud computing in data storage and management, data processing and application services, but significantly different in terms of its short geographical distance from edgy devices, its dense geographical distribution, and its support for mobility. Fog/Edge computing requires micro data centers that are scalable, flexible and agile data centers co-located with distributed IoT data sources. Fog/Edge Computing & Micro Data Centers
  8. 8. ©2017 Xanadu Big Data, LLC All Rights Reserved Deep Learning for IoT Big Data Analytics Introduction Convolutional Neural Networks (CNNs): image/video analysis Source: Deep Learning A-Z™: Hands-On Artificial Neural Networks by SuperDataScience@udemy Recurrent Neural Networks (RNNs):time series analysis Reinforcement Learning (RL): automatically determine the ideal behaviour within a specific context, in order to maximize its performance for a specific goal Source: Reinforcement Learning in Python by Lazy Programmer Inc.@udemy
  9. 9. ©2017 Xanadu Big Data, LLC All Rights Reserved Deep Learning for IoT Big Data Analytics Use Case CNN + RNN Source: SoftPoint Consultores S.L. Deep RL Source: Mobileye Source: ODSTCEAST 2017
  10. 10. ©2017 Xanadu Big Data, LLC All Rights Reserved Distributed Deep Learning Issues with IoT big data in deep learning: High resolution/scale microscopic images 81,025 pixels by 86,273 pixels (roughly 6.99 gigapixels) requires 78.12 GB memory to store Trained on 1.28 M images and evaluated on 50 K images took 2 – 3 weeks using 8 GPU machine in MS ResNet Distributed training: In model parallelism, different machines in the distributed system are responsible for the computations in different parts of a single network - for example, each layer in the neural network may be assigned to a different machine. In data parallelism, different machines have a complete copy of the model; each machine simply gets a different portion of the data, and results from each are somehow combined. For details: Skymind web: http://engineering.skymind.io
  11. 11. ©2017 Xanadu Big Data, LLC All Rights Reserved US20170175645 (GE: IIoT Application) Deep learning enables automatically learning actionable information relevant to a desired operation of a gas turbine in industrial plants from seemingly uncorrelated massive amounts of sensor and controller data. Big Data + IoT + Cloud + Deep Learning Insights from Patents
  12. 12. ©2017 Xanadu Big Data, LLC All Rights Reserved US20170169358 (Samsung: Fog Computing) Decentralized deep learning system on the local IoT data using edge storage network that is configure to store and process streaming IoT data. Big Data + IoT + Cloud + Deep Learning Insights from Patents
  13. 13. ©2017 Xanadu Big Data, LLC All Rights Reserved US20170060574 (Foghorn Systems: IIoT + Fog Computing) A real-time edge IoT analytics system that can handle the large amounts of data generated by industrial machines and provides intelligent edge computing platform. Big Data + IoT + Cloud + Deep Learning Insights from Patents
  14. 14. ©2017 Xanadu Big Data, LLC All Rights Reserved US20160196527 (FALKONRY : Smart Logistics) Cyber-physical supply chain logistics transportation system for predictive estimation of QoS across supply chains using condition monitoring and predictive analytics. Big Data + IoT + Cloud + Deep Learning Insights from Patents
  15. 15. ©2017 Xanadu Big Data, LLC All Rights Reserved Source: Prof. Pierre Azoulay@MIT Sloan Executive Education Big Data + IoT + Cloud + Deep Learning Strategy Development
  16. 16. ©2017 Xanadu Big Data, LLC All Rights Reserved Chapter 2 Data Models Chapter 3 Storage and Retrieval Chapter 5 Replication Chapter 6 Partitioning Chapter 7 Transactions Chapter 8 Trouble with Distributed Systems Chapter 9 Consistency and Consensus Chapter 10 Batch Processing Chapter 11 Stream Processing Designing Data-Intensive Applications
  17. 17. ©2017 Xanadu Big Data, LLC All Rights Reserved Xanadu enables competitive big data management in the Clouds or Enterprises Xanadu Functionality K AnyTypes&SizeofData NoSQL Database Data De-duplication Data Replication Massively Scalable Fault Tolerance Data Store NoSQL Database ACID Compliance High Throughput Low Latency Data Access
  18. 18. ©2017 Xanadu Big Data, LLC All Rights Reserved Xanadu Performance BMT
  19. 19. ©2017 Xanadu Big Data, LLC All Rights Reserved Xanadu Fault Tolerance Test Demo
  20. 20. ©2017 Xanadu Big Data, LLC All Rights Reserved Xanadu provides a composable architecture that can be integrated with other big data systems Xanadu Use Case + Total Integration Big Data Applications Big Data IT Infrastructure Xanadu Data Management Platform GPS / GLONASS WCDMA / LTE +
  21. 21. ©2017 Xanadu Big Data, LLC All Rights Reserved Xanadu Commodity Storage System Use Case
  22. 22. ©2017 Xanadu Big Data, LLC All Rights Reserved Xanadu Cloud Computing Use Case
  23. 23. ©2017 Xanadu Big Data, LLC All Rights Reserved AlexNet CNN architecture in DeepLearning4J (DL4J) Distributed deep leaning on CPUs/GPUs (+Hadoop/Spark) Xanadu file system used to store images and load directly into DL4J Local Machines & AWS Clusters Xanadu + Deep Learning + Hadoop Integration Source: researchgate.net. Data Source: https://www.kaggle.com/c/diabetic-retinopathy-detection
  24. 24. ©2017 Xanadu Big Data, LLC All Rights Reserved Thank you Xanadu Big Data, LLC

×