IoT and BigData
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

IoT and BigData

on

  • 605 views

An introduction to BigData and how it can be used in combination with the internet of things.

An introduction to BigData and how it can be used in combination with the internet of things.

Statistics

Views

Total Views
605
Views on SlideShare
605
Embed Views
0

Actions

Likes
0
Downloads
38
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

IoT and BigData Presentation Transcript

  • 1. WhoAmI Entrepreneurial geek Belgian Community Micro-clusters for BigData enthousiasts BigData Services
  • 2. Agenda Part 1 : BigData Part 2 : IoT and BigData Part 3 : Discussions
  • 3. BigData Part 1
  • 4. The Buzz Started by Google ● ● GFS Paper (2003) MapReduce Paper (2004) Storing and processing “Big” Data ~ multiple petabytes
  • 5. The Need Large amount of data IOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOI IOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOI IOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOI OIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOI OIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOII IOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOI OOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIOIIOIOIOIIIOIOO OOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOO OIOIIIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIO IOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOI OIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOII OIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOII OIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOI OIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOI OIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIOIIOIOIOIII OIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOI OOOOIOIIIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOI OIOIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOI OIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOI OIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOI OIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOI OIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOII OIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIOII OIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOI OIOIIIOIOOOOIOIIOIOIIOIIOIOIOIIIOIOOOOI
  • 6. The Need OIII IOII IOII OO OIII OIII High Speed at which IOII data becomes available IOII OO OIII OIII IOII IOII OO OIII
  • 7. The Need IOII IOII The Different ways data is structured IOII IOII IOII IOII IOIIIOIIIOII IOII IOII IOII IOII IOII IOII IOII IOII IOII IOII IOIIIOII IOII IOII IOII IOII IOII IOII IOII IOII
  • 8. Technologies
  • 9. Technologies Batch Processing: Realtime Processing: STORM
  • 10. Hadoop Platform for Batch Processing Yahoo! Doug Cutting Mike Cafarella Fetching and processing webpages
  • 11. Storage: HDFS
  • 12. Processing: YARN
  • 13. Processing: MapReduce
  • 14. Apache Storm Platform for Realtime Processing Looks like Complex Event Processing (CEP) Backtype Nathan Marz Acquired by Twitter Like “ever-running” MapReduce jobs
  • 15. Apache Storm
  • 16. Apache Storm
  • 17. Apache Storm
  • 18. Apache Storm
  • 19. Apache Storm
  • 20. Myths “BigData is all about storing data” “BigData is only for big companies having lots of data” “There are no BigData use cases in Belgium” “BigData is only being used for analytics”
  • 21. Dangers People are hard to find Technologies are “different” Too much technologies You need hardware Incorrect information
  • 22. IoT & BigData Part 2
  • 23. Iot as datasource Use data from “Things” As raw as possible No matter which format Store everything
  • 24. Iot as datasource
  • 25. Iot as datasource Adapt the processing flow based on the data flowing through it: - Disable faulty sensors - Take processing shortcuts
  • 26. Iot as datasource Control other “Things” based on the data flowing through: - Rain predicted => disable sprinklers - Check-out at work, trip home will take 60 min => start heating in 30 min Would be great to link with a rule-engine (Drools e.g.)
  • 27. Iot as datasource Store data for more complex processing: - Pattern or behaviour recognition - Forecasting - Prevention
  • 28. Iot as datasource Storage: Every event is a record Every event is immutable Every event has: a source a timestamp a type a value
  • 29. Iot as datasource
  • 30. Iot as Platform - Idea Quantity > Quality One platform to Store Process Nodes can be Anything with a controller Anything that can store data
  • 31. Iot as Platform - Risks Complex Resource Management Exploit Capabilities Complex Data Management Fragmented Data Complex Security Model Obfuscate Data
  • 32. Iot as Platform - Gains Keep your friends close, but your data closer. Delegate processing Use resources in the best way possible
  • 33. What do you think? Part 3