SlideShare a Scribd company logo
1 of 14
Internet OfThings & Big Data
As Service
About me
Sriram Reddy
Architect
www.Ammeon.com
Software Architect with a passion for
• Linked Devices
• Device Multi Tenancy
• Low Power Wide Area Networks
• Distributed Device Management
• Low Power Device Security
• Distributed Data Persistence
• Distributed Data Processing
• Analytics
Index
1. Introduction
2. Internet Of Things
• Micro Controller & Processor
• Power Management
3. IOT Networks
• Long Range
• Short Range
4. IOT Data Types
• Binary (MQTT & Proto buff)
• Jason
5. IOT Data Acquisition
• MQTT
• CoAP
• Kafka
5. IOT Data processing
• Spark
• Nifi
• StreamSets
6. Data Persistence
• Hadoop
• Cassandra
• OrentDB
• Titan
• Mongo
7.Reporting
• BIRT Reporting
• Eclipse RCP
8. Analytics
9. Market Vertices
10. Q&A
Introduction
Nothing is new.
There are devices, there is network and there is data.
Challenge is Lot Of Devices, Network Everywhere, Lot Of Data
IOTTriad
Enablement
HW-Clinet
Container
Modeling
Dashboard
Ethernet
GSM/3G/4G/MB-IOT
BLE/LORA /WIFI
AVR/PIC/Psoc (MCU)
SOC(MCP)
Boards (Pis)
MQTT
CoAP
HTTP
KAFKA
Hadoop
Cassandra
Ignite
Kafka
Spark/Flink/Drill
NIFI/Streamsets/Camel
An IOT deployment architecture demands right device design, right
communications and data management & analytics.
IOT Devices
Microprocessor Based Devices
Microcontroller Based Devices
Raspberry Pi, Orange Pi, Banana Pi, Nano Pi,
Bagel Bone, Panda board, Onion, Chip etc.
• SOCs from Boradcom, Rock chip, All winner, Intel etc
• Runs full Linux/Windows
• Powerful
• Power-hungry
Arduino, Wiring, Sypris etc
• Micro-chip, Atmel, Nordic etc
• SOC with memory and other peripherals
• Low power
• Less power consumption
IOT Communications
Long Range
Short Range
Networks to span full metros to countries
• Traditional operator provided 2G/3G/4G
• LORA
• SIGFOX
With in 100 meter range
• WIFI
• BLE
IOT DataTypes
Binary Data
JSON
Binary data provides size advantage but adds more
processing.
• MQTT
• Protobuffs
JSON or any markup payloads offer human readability
and existing processing infrastructure.
IOT Data Acquisition
MQTT
CoAP
Kafka
• Lot of queues (millions)
• Light Weight
• Open source brokers
• Well Integrated
• Well supported
• HTTP like
• Matured with telco industry
• Human readable
• Light server
• Not very heavy client
• Client controlled
• Fast
• Easy to scale
• Easy to replicate
IOT Data Processing
Spark Ignite
Nifi
StreamSets
Beam
IOT Data Persistence
Hadoop Cassandra
Oriented DB
IOT Market
IOT Market
Q&A

More Related Content

What's hot

What's hot (20)

Logicalis IoT & Smart Cities (Use Case)
Logicalis IoT & Smart Cities (Use Case)Logicalis IoT & Smart Cities (Use Case)
Logicalis IoT & Smart Cities (Use Case)
 
Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace
Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a PalaceInternet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace
Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace
 
Introduction to Microsoft Azure IoT
Introduction to Microsoft Azure IoTIntroduction to Microsoft Azure IoT
Introduction to Microsoft Azure IoT
 
Watson IoT Platform Sizing & Pricing - Sept 2016
Watson IoT Platform Sizing & Pricing - Sept 2016Watson IoT Platform Sizing & Pricing - Sept 2016
Watson IoT Platform Sizing & Pricing - Sept 2016
 
Intro to Big Data and Apache Hadoop by Dr. Amr Awadallah at CLOUD WEEKEND '13...
Intro to Big Data and Apache Hadoop by Dr. Amr Awadallah at CLOUD WEEKEND '13...Intro to Big Data and Apache Hadoop by Dr. Amr Awadallah at CLOUD WEEKEND '13...
Intro to Big Data and Apache Hadoop by Dr. Amr Awadallah at CLOUD WEEKEND '13...
 
Architect Your IoT Platform for Success
Architect Your IoT Platform for SuccessArchitect Your IoT Platform for Success
Architect Your IoT Platform for Success
 
IoT and Big Data - Iot Asia 2014
IoT and Big Data - Iot Asia 2014IoT and Big Data - Iot Asia 2014
IoT and Big Data - Iot Asia 2014
 
AI as a Catalyst for IoT
AI as a Catalyst for IoTAI as a Catalyst for IoT
AI as a Catalyst for IoT
 
Internet of Things (IoT) and Big Data
Internet of Things (IoT) and Big DataInternet of Things (IoT) and Big Data
Internet of Things (IoT) and Big Data
 
CL2015 - Datacenter and Cloud Strategy and Planning
CL2015 - Datacenter and Cloud Strategy and PlanningCL2015 - Datacenter and Cloud Strategy and Planning
CL2015 - Datacenter and Cloud Strategy and Planning
 
Accelerating the AIoT @ the EDGE
Accelerating the AIoT @ the EDGE Accelerating the AIoT @ the EDGE
Accelerating the AIoT @ the EDGE
 
Phoenix Data Conference - Big Data Analytics for IoT 11/4/17
Phoenix Data Conference - Big Data Analytics for IoT 11/4/17Phoenix Data Conference - Big Data Analytics for IoT 11/4/17
Phoenix Data Conference - Big Data Analytics for IoT 11/4/17
 
Powering the Intelligent Edge: HPE's Strategy and Direction for IoT & Big Data
Powering the Intelligent Edge: HPE's Strategy and Direction for IoT & Big DataPowering the Intelligent Edge: HPE's Strategy and Direction for IoT & Big Data
Powering the Intelligent Edge: HPE's Strategy and Direction for IoT & Big Data
 
AIoT and edge computing solutions
AIoT and edge computing solutionsAIoT and edge computing solutions
AIoT and edge computing solutions
 
Xanadu for Big Data + Deep Learning + Cloud + IoT Integration Strategy
Xanadu for Big Data + Deep Learning + Cloud + IoT Integration StrategyXanadu for Big Data + Deep Learning + Cloud + IoT Integration Strategy
Xanadu for Big Data + Deep Learning + Cloud + IoT Integration Strategy
 
Cloud Computing, SDN, Big Data and Internet of Everything - Lew Tucker
Cloud Computing, SDN, Big Data and Internet of Everything - Lew TuckerCloud Computing, SDN, Big Data and Internet of Everything - Lew Tucker
Cloud Computing, SDN, Big Data and Internet of Everything - Lew Tucker
 
Internet of Things and Big Data
Internet of Things and Big DataInternet of Things and Big Data
Internet of Things and Big Data
 
IoT at the Edge - Greengrass and More - AWS PS Summit Canberra 2017
IoT at the Edge - Greengrass and More - AWS PS Summit Canberra 2017IoT at the Edge - Greengrass and More - AWS PS Summit Canberra 2017
IoT at the Edge - Greengrass and More - AWS PS Summit Canberra 2017
 
Operational information processing: lightning-fast, delightfully simple
Operational information processing: lightning-fast, delightfully simpleOperational information processing: lightning-fast, delightfully simple
Operational information processing: lightning-fast, delightfully simple
 
IoT – The reality of real world solutions
IoT – The reality of real world solutions IoT – The reality of real world solutions
IoT – The reality of real world solutions
 

Viewers also liked

Viewers also liked (7)

Hadoop Self-Service Data Prep Fuels Analytics
Hadoop Self-Service Data Prep Fuels AnalyticsHadoop Self-Service Data Prep Fuels Analytics
Hadoop Self-Service Data Prep Fuels Analytics
 
Introduction to Kafka
Introduction to KafkaIntroduction to Kafka
Introduction to Kafka
 
Change Data Capture using Kafka
Change Data Capture using KafkaChange Data Capture using Kafka
Change Data Capture using Kafka
 
Python in telecommunications (in 7 minutes)
Python in telecommunications (in 7 minutes)Python in telecommunications (in 7 minutes)
Python in telecommunications (in 7 minutes)
 
Ericsson commond list, BSS+NSS=OSS
Ericsson commond list, BSS+NSS=OSSEricsson commond list, BSS+NSS=OSS
Ericsson commond list, BSS+NSS=OSS
 
The Business Advantage of Hadoop: Lessons from the Field – Cloudera Summer We...
The Business Advantage of Hadoop: Lessons from the Field – Cloudera Summer We...The Business Advantage of Hadoop: Lessons from the Field – Cloudera Summer We...
The Business Advantage of Hadoop: Lessons from the Field – Cloudera Summer We...
 
Distributed Computing with Apache Hadoop: Technology Overview
Distributed Computing with Apache Hadoop: Technology OverviewDistributed Computing with Apache Hadoop: Technology Overview
Distributed Computing with Apache Hadoop: Technology Overview
 

Similar to IoT Data as Service with Hadoop

ch5-Fog Networks and Cloud Computing
ch5-Fog Networks and Cloud Computingch5-Fog Networks and Cloud Computing
ch5-Fog Networks and Cloud Computing
ssuser06ea42
 

Similar to IoT Data as Service with Hadoop (20)

Nodemcu and IOT.pptx
Nodemcu and IOT.pptxNodemcu and IOT.pptx
Nodemcu and IOT.pptx
 
the connection of iot with lora pan which enable
the connection of iot with lora pan which enablethe connection of iot with lora pan which enable
the connection of iot with lora pan which enable
 
IoT and Its Application
IoT and Its ApplicationIoT and Its Application
IoT and Its Application
 
GDG Meetup Jakarta - Low Power IoT
GDG Meetup Jakarta - Low Power IoTGDG Meetup Jakarta - Low Power IoT
GDG Meetup Jakarta - Low Power IoT
 
Introduction to IoT Technologies - The need to know basics
Introduction to IoT Technologies - The need to know basicsIntroduction to IoT Technologies - The need to know basics
Introduction to IoT Technologies - The need to know basics
 
Internet of Things Architecture / Topology
Internet of Things Architecture / TopologyInternet of Things Architecture / Topology
Internet of Things Architecture / Topology
 
Global Azure boot camp 2015 - Microsoft IoT Solutions with Azure
Global Azure boot camp 2015 - Microsoft IoT Solutions with AzureGlobal Azure boot camp 2015 - Microsoft IoT Solutions with Azure
Global Azure boot camp 2015 - Microsoft IoT Solutions with Azure
 
Internet of things
Internet of thingsInternet of things
Internet of things
 
Atal io t introduction
Atal io t introductionAtal io t introduction
Atal io t introduction
 
IoT RF Protocols
IoT RF ProtocolsIoT RF Protocols
IoT RF Protocols
 
ch5-Fog Networks and Cloud Computing
ch5-Fog Networks and Cloud Computingch5-Fog Networks and Cloud Computing
ch5-Fog Networks and Cloud Computing
 
Null mumbai-iot-workshop
Null mumbai-iot-workshopNull mumbai-iot-workshop
Null mumbai-iot-workshop
 
IoT Control Units and Communication Models
IoT Control Units and Communication ModelsIoT Control Units and Communication Models
IoT Control Units and Communication Models
 
An infrastructual secure wireless sensing and actuating solution
An infrastructual secure wireless sensing and actuating solutionAn infrastructual secure wireless sensing and actuating solution
An infrastructual secure wireless sensing and actuating solution
 
INTERNET OF THINGS.pptx
INTERNET OF THINGS.pptxINTERNET OF THINGS.pptx
INTERNET OF THINGS.pptx
 
Building the Internet of Things with Thingsquare and Contiki - day 1, part 1
Building the Internet of Things with Thingsquare and Contiki - day 1, part 1Building the Internet of Things with Thingsquare and Contiki - day 1, part 1
Building the Internet of Things with Thingsquare and Contiki - day 1, part 1
 
Building a Citizen IoT Network on Microsoft Azure
Building a Citizen IoT Network on Microsoft AzureBuilding a Citizen IoT Network on Microsoft Azure
Building a Citizen IoT Network on Microsoft Azure
 
Link Labs LPWA Webinar
Link Labs LPWA WebinarLink Labs LPWA Webinar
Link Labs LPWA Webinar
 
Real-time DeepLearning on IoT Sensor Data
Real-time DeepLearning on IoT Sensor DataReal-time DeepLearning on IoT Sensor Data
Real-time DeepLearning on IoT Sensor Data
 
From Device to Data Center to Insights: Architectural Considerations for the ...
From Device to Data Center to Insights: Architectural Considerations for the ...From Device to Data Center to Insights: Architectural Considerations for the ...
From Device to Data Center to Insights: Architectural Considerations for the ...
 

More from Quantified Self Dublin

More from Quantified Self Dublin (20)

Quantified Wellness and Assisted Living
Quantified Wellness and Assisted LivingQuantified Wellness and Assisted Living
Quantified Wellness and Assisted Living
 
Quantified Brain Health
Quantified Brain HealthQuantified Brain Health
Quantified Brain Health
 
Nutrigenomics, Biomarkers & Health
Nutrigenomics, Biomarkers & HealthNutrigenomics, Biomarkers & Health
Nutrigenomics, Biomarkers & Health
 
Gut hormones - Appetite, Mood and Health
Gut hormones - Appetite, Mood and HealthGut hormones - Appetite, Mood and Health
Gut hormones - Appetite, Mood and Health
 
Self Quantifying Health
Self Quantifying HealthSelf Quantifying Health
Self Quantifying Health
 
Biohacking Sleep
Biohacking SleepBiohacking Sleep
Biohacking Sleep
 
Data Mining Sleep Data
Data Mining Sleep DataData Mining Sleep Data
Data Mining Sleep Data
 
Quantified Sleep - Science behind the sleep trackers
Quantified Sleep - Science behind the sleep trackersQuantified Sleep - Science behind the sleep trackers
Quantified Sleep - Science behind the sleep trackers
 
Future Trends in Wearables & Big Data
Future Trends in Wearables & Big DataFuture Trends in Wearables & Big Data
Future Trends in Wearables & Big Data
 
Quantifying and Treating Stress
Quantifying and Treating StressQuantifying and Treating Stress
Quantifying and Treating Stress
 
Evolution of Wearables - Past, Present, Future
Evolution of Wearables - Past, Present, FutureEvolution of Wearables - Past, Present, Future
Evolution of Wearables - Past, Present, Future
 
The Future of Quantified Self in Healthcare
The Future of Quantified Self in HealthcareThe Future of Quantified Self in Healthcare
The Future of Quantified Self in Healthcare
 
Quantified Self & Personal Healthcare
Quantified Self & Personal HealthcareQuantified Self & Personal Healthcare
Quantified Self & Personal Healthcare
 
Kenkodo Quantified Self Dublin June 2016
Kenkodo Quantified Self Dublin June 2016Kenkodo Quantified Self Dublin June 2016
Kenkodo Quantified Self Dublin June 2016
 
Food and Activity Detection in Life Logging Images
Food and Activity Detection in Life Logging ImagesFood and Activity Detection in Life Logging Images
Food and Activity Detection in Life Logging Images
 
Food and Activity Detection in Life Logging Images
Food and Activity Detection in Life Logging ImagesFood and Activity Detection in Life Logging Images
Food and Activity Detection in Life Logging Images
 
Improving Mental Focus
Improving Mental Focus Improving Mental Focus
Improving Mental Focus
 
Introduction to HRV
Introduction to HRVIntroduction to HRV
Introduction to HRV
 
Introduction to Google Fit
Introduction to Google FitIntroduction to Google Fit
Introduction to Google Fit
 
How Lightbulbs Made Alarm Clocks
How Lightbulbs Made Alarm ClocksHow Lightbulbs Made Alarm Clocks
How Lightbulbs Made Alarm Clocks
 

Recently uploaded

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Recently uploaded (20)

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdf
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 

IoT Data as Service with Hadoop

  • 1. Internet OfThings & Big Data As Service
  • 2. About me Sriram Reddy Architect www.Ammeon.com Software Architect with a passion for • Linked Devices • Device Multi Tenancy • Low Power Wide Area Networks • Distributed Device Management • Low Power Device Security • Distributed Data Persistence • Distributed Data Processing • Analytics
  • 3. Index 1. Introduction 2. Internet Of Things • Micro Controller & Processor • Power Management 3. IOT Networks • Long Range • Short Range 4. IOT Data Types • Binary (MQTT & Proto buff) • Jason 5. IOT Data Acquisition • MQTT • CoAP • Kafka 5. IOT Data processing • Spark • Nifi • StreamSets 6. Data Persistence • Hadoop • Cassandra • OrentDB • Titan • Mongo 7.Reporting • BIRT Reporting • Eclipse RCP 8. Analytics 9. Market Vertices 10. Q&A
  • 4. Introduction Nothing is new. There are devices, there is network and there is data. Challenge is Lot Of Devices, Network Everywhere, Lot Of Data
  • 5. IOTTriad Enablement HW-Clinet Container Modeling Dashboard Ethernet GSM/3G/4G/MB-IOT BLE/LORA /WIFI AVR/PIC/Psoc (MCU) SOC(MCP) Boards (Pis) MQTT CoAP HTTP KAFKA Hadoop Cassandra Ignite Kafka Spark/Flink/Drill NIFI/Streamsets/Camel An IOT deployment architecture demands right device design, right communications and data management & analytics.
  • 6. IOT Devices Microprocessor Based Devices Microcontroller Based Devices Raspberry Pi, Orange Pi, Banana Pi, Nano Pi, Bagel Bone, Panda board, Onion, Chip etc. • SOCs from Boradcom, Rock chip, All winner, Intel etc • Runs full Linux/Windows • Powerful • Power-hungry Arduino, Wiring, Sypris etc • Micro-chip, Atmel, Nordic etc • SOC with memory and other peripherals • Low power • Less power consumption
  • 7. IOT Communications Long Range Short Range Networks to span full metros to countries • Traditional operator provided 2G/3G/4G • LORA • SIGFOX With in 100 meter range • WIFI • BLE
  • 8. IOT DataTypes Binary Data JSON Binary data provides size advantage but adds more processing. • MQTT • Protobuffs JSON or any markup payloads offer human readability and existing processing infrastructure.
  • 9. IOT Data Acquisition MQTT CoAP Kafka • Lot of queues (millions) • Light Weight • Open source brokers • Well Integrated • Well supported • HTTP like • Matured with telco industry • Human readable • Light server • Not very heavy client • Client controlled • Fast • Easy to scale • Easy to replicate
  • 10. IOT Data Processing Spark Ignite Nifi StreamSets Beam
  • 11. IOT Data Persistence Hadoop Cassandra Oriented DB
  • 14. Q&A