Downloaded 22 times
Uploaded byQuantified Self Dublin
IoT Data as Service with Hadoop
The document discusses the integration of the Internet of Things (IoT) and Big Data as a service, highlighting the architecture required for effective device management, communication, and data processing. It covers various IoT technologies, including data acquisition methods like MQTT and CoAP, and processing frameworks such as Spark and Nifi. Additionally, it addresses the challenges posed by numerous devices and the need for efficient data management and analytics.
More Related Content
Similar to IoT Data as Service with Hadoop
More from Quantified Self Dublin
IoT Data as Service with Hadoop
- 1. Internet OfThings &Big Data As Service
- 2. About me Sriram Reddy Architect www.Ammeon.com SoftwareArchitect with a passion for • Linked Devices • Device Multi Tenancy • Low Power Wide Area Networks • Distributed Device Management • Low Power Device Security • Distributed Data Persistence • Distributed Data Processing • Analytics
- 3. Index 1. Introduction 2. InternetOf Things • Micro Controller & Processor • Power Management 3. IOT Networks • Long Range • Short Range 4. IOT Data Types • Binary (MQTT & Proto buff) • Jason 5. IOT Data Acquisition • MQTT • CoAP • Kafka 5. IOT Data processing • Spark • Nifi • StreamSets 6. Data Persistence • Hadoop • Cassandra • OrentDB • Titan • Mongo 7.Reporting • BIRT Reporting • Eclipse RCP 8. Analytics 9. Market Vertices 10. Q&A
- 4. Introduction Nothing is new. Thereare devices, there is network and there is data. Challenge is Lot Of Devices, Network Everywhere, Lot Of Data
- 5. IOTTriad Enablement HW-Clinet Container Modeling Dashboard Ethernet GSM/3G/4G/MB-IOT BLE/LORA /WIFI AVR/PIC/Psoc (MCU) SOC(MCP) Boards(Pis) MQTT CoAP HTTP KAFKA Hadoop Cassandra Ignite Kafka Spark/Flink/Drill NIFI/Streamsets/Camel An IOT deployment architecture demands right device design, right communications and data management & analytics.
- 6. IOT Devices Microprocessor BasedDevices Microcontroller Based Devices Raspberry Pi, Orange Pi, Banana Pi, Nano Pi, Bagel Bone, Panda board, Onion, Chip etc. • SOCs from Boradcom, Rock chip, All winner, Intel etc • Runs full Linux/Windows • Powerful • Power-hungry Arduino, Wiring, Sypris etc • Micro-chip, Atmel, Nordic etc • SOC with memory and other peripherals • Low power • Less power consumption
- 7. IOT Communications Long Range ShortRange Networks to span full metros to countries • Traditional operator provided 2G/3G/4G • LORA • SIGFOX With in 100 meter range • WIFI • BLE
- 8. IOT DataTypes Binary Data JSON Binarydata provides size advantage but adds more processing. • MQTT • Protobuffs JSON or any markup payloads offer human readability and existing processing infrastructure.
- 9. IOT Data Acquisition MQTT CoAP Kafka •Lot of queues (millions) • Light Weight • Open source brokers • Well Integrated • Well supported • HTTP like • Matured with telco industry • Human readable • Light server • Not very heavy client • Client controlled • Fast • Easy to scale • Easy to replicate
- 10. IOT Data Processing SparkIgnite Nifi StreamSets Beam
- 11. IOT Data Persistence HadoopCassandra Oriented DB
- 12.
- 13.
- 14.