How to maximize profit
from Internet of Things
using modern
Data Platform?
Author: Albert Lewandowski
• Big Data DevOps Engineer in Getindata
• Member of the IoT working group in the Ministry of
Digital Affairs
• Consultant for startups
• Focused on Industry 4.0, Smart City and Big Data
sectors
Who am I?
Ever-growing network of smart, connected devices,
objects and appliances.
- Gathering and transferring data
- Access to to the internet
What is Internet of Things?
- temperature and
humidity sensors,
- pressure sensors,
- proximity sensors,
- smoke sensors,
- level sensors,
What is Internet of Things?
- accelerometers,
- gyroscopes,
- gas sensors,
- infrared sensors,
- optical sensors,
- image sensors (like
CCTV)
- Digital signage
- In-store offering & promotions
- Supply chain
- Smart ordering & payment
- Vending machines
Retail
- Adherence & support
- Clinical
- Virtual care
- Wellness & prevention
Healthcare
- Health and life insurance
- Home insurance
- Industrial insurance
- Vehicle insurance
- Cross
Insurance
- Assisted & autonomous driving
- Fleet management
- In-vehicle infotainment
- Shared mobility
- Smart navigation
Connected cars
- Construction
- Education
- Energy
- Environmental
- Roads, traffic & transport
- Social & Security
Smart Cities & Energy
- Agriculture
- Mining
- Oils & gas
Natural resources
- Connected field
- Digital factory
- Product design & engineering
- Smart maintenance
Connected Industry
© Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data-based
company
1. Do you process data?
2. What kind of services do you use?
3. Do they support additional data sources?
4. Do you take advantage of being data-based
company?
5. What is your data strategy?
6. What about security?
7. How much data do you already have?
Questions
© Copyright. All rights reserved. Not to be reproduced without prior written consent.
First steps
with IoT
- Based on the network connection
- Wireless
- Wired
- Based on the data processing
- Edge
- Central
- Based on the sensor types
Types of IoT
- Multiple data sources
- Heavy-loading data processing if we want to
process all information (especially video)
- Network limitations
Data analysis based on IoT
- How can we process the data on the edge?
- How much data should we transfer?
- Big Data solutions for your needs
Data enrichment
© Copyright. All rights reserved. Not to be reproduced without prior written consent.
IoT Data Analytics
In Your Company
1. Define problems.
2. Analyze your current IT stack and used solutions.
3. Prepare plan for next 6-8 weeks to create PoC.
How to start?
- Which processes can be automated?
- What are the targets of our project?
- What do we plan to do in the future?
Understanding problem
- Which structure of IoT devices is the most suitable for
us?
- How do we need to collect information? Do we need
to send them to the central platform?
Which type of IoT?
© Copyright. All rights reserved. Not to be reproduced without prior written consent.
Let’s build
Data Platform
© Copyright. All rights reserved. Not to be reproduced without prior written consent.
Perception
Business
logic
CI/CD
Idempotency
Reprocessing
Explainability
Monitoring
Testing
Serving
Infrastructure
Data Ingestion
Security
© Copyright. All rights reserved. Not to be reproduced without prior written consent.
Reality
Business
logic
CI/CD
Idempotency
Reprocessing
Explainability
Monitoring
Testing
Serving
Infrastructure
Data Ingestion
Security
- Network
- Security
- Performance
Which type of IoT?
- Multiregional setup
- Private vs. public cloud
- Backup and disaster
recovery
High Availability
- Audit features
- Permissions management
- Encrypt traffic between IoT devices
- Update software and manage devices from one place
Security
- Storage
- Data ingestion
- Data processing
- ML / AI services
- BI tools
- Data visualisation
- Compute resources
- Managed or non-managed services
Big Data Services
Big Data Services
- Public cloud or own infrastructure?
- Managed or non-managed services
- How to store data?
- Avoid vendor-lock - is it important?
IT infrastructure
© Copyright. All rights reserved. Not to be reproduced without prior written consent.
Great services
for IoT
- Start with simple pipelines.
- Start with the simplest and most important use
case for the company
- Create flexible data infrastructure to add new
components easily
Data pipelines
Depends on:
- amount of data
- required time to deliver results
- costs and complexity
Central or Edge processing
Streaming or Batch
Streaming Batch
Process data continuously Process data in batch
We process piece-by-piece Once data is collected, it’s sent
for processing
Fast results and react to defined
triggers
Great for processing large
amount of data
- Do you need ML/AI?
- What business value can it add to the project?
- Edge AI
Machine Learning
Observability
- Do we have in-house skills?
- How big team do we need?
- How to manage the project?
Human skills
SRE & DevOps
If a human operator needs to touch your
system during normal operations, you have
a bug. The definition of normal changes as
your systems grow.
Carla Geisser, Google SRE
SRE & DevOps
If you think of DevOps like an interface in a
programming language, class SRE implements DevOps.
SLI, SLA & SLO
• SLA - Service Level Agreements
• SLI - Service Level Indicators
• SLO - Service Level Objectives
Measuring Service Risk
Time-based availability is traditionally calculated based on
the proportion of system uptime.
Aggregate availability shows how this yield-based metric is
calculated over a rolling window (i.e., proportion of successful
requests over a one-day window).
CICD pipelines
Besides black art, there is only automation and mechanization.
Federico García Lorca (1898–1936), Spanish poet and playwright
Source: AWS
© Copyright. All rights reserved. Not to be reproduced without prior written consent.
Stories
from life
https://getindata.com/blog/success-story-breastfeeding-s
upported-with-modern-iot-app-features-bigquery-cloud
Mamava
- Public-cloud based platform
- Detect failures from devices in factories
- Predictive maintenance
- Real-time data streaming for analyzing
information
Industry 4.0, startup
- Private-cloud based platform
- Real-time data streaming for analyzing
information
- Aggregating and analyzing data sent from
edge devices
Telecommunication, IoT data
- Public cloud-based startup
- Security of the employees in the construction
sites
- Edge data processing
Industry 4.0, startup
Whitepaper
https://getindata.com/blog/white-paper-big-data-analytics
-industrial-internet-things
© Copyright. All rights reserved. Not to be reproduced without prior written consent.
Q&A
Thank you
for your attention!

How to maximize profit from IoT by using data platform - Albert Lewandowski, GetInData

  • 1.
    How to maximizeprofit from Internet of Things using modern Data Platform? Author: Albert Lewandowski
  • 2.
    • Big DataDevOps Engineer in Getindata • Member of the IoT working group in the Ministry of Digital Affairs • Consultant for startups • Focused on Industry 4.0, Smart City and Big Data sectors Who am I?
  • 3.
    Ever-growing network ofsmart, connected devices, objects and appliances. - Gathering and transferring data - Access to to the internet What is Internet of Things?
  • 4.
    - temperature and humiditysensors, - pressure sensors, - proximity sensors, - smoke sensors, - level sensors, What is Internet of Things? - accelerometers, - gyroscopes, - gas sensors, - infrared sensors, - optical sensors, - image sensors (like CCTV)
  • 5.
    - Digital signage -In-store offering & promotions - Supply chain - Smart ordering & payment - Vending machines Retail
  • 6.
    - Adherence &support - Clinical - Virtual care - Wellness & prevention Healthcare
  • 7.
    - Health andlife insurance - Home insurance - Industrial insurance - Vehicle insurance - Cross Insurance
  • 8.
    - Assisted &autonomous driving - Fleet management - In-vehicle infotainment - Shared mobility - Smart navigation Connected cars
  • 9.
    - Construction - Education -Energy - Environmental - Roads, traffic & transport - Social & Security Smart Cities & Energy
  • 10.
    - Agriculture - Mining -Oils & gas Natural resources
  • 11.
    - Connected field -Digital factory - Product design & engineering - Smart maintenance Connected Industry
  • 12.
    © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Data-based company
  • 13.
    1. Do youprocess data? 2. What kind of services do you use? 3. Do they support additional data sources? 4. Do you take advantage of being data-based company? 5. What is your data strategy? 6. What about security? 7. How much data do you already have? Questions
  • 14.
    © Copyright. Allrights reserved. Not to be reproduced without prior written consent. First steps with IoT
  • 15.
    - Based onthe network connection - Wireless - Wired - Based on the data processing - Edge - Central - Based on the sensor types Types of IoT
  • 17.
    - Multiple datasources - Heavy-loading data processing if we want to process all information (especially video) - Network limitations Data analysis based on IoT
  • 18.
    - How canwe process the data on the edge? - How much data should we transfer? - Big Data solutions for your needs Data enrichment
  • 19.
    © Copyright. Allrights reserved. Not to be reproduced without prior written consent. IoT Data Analytics In Your Company
  • 20.
    1. Define problems. 2.Analyze your current IT stack and used solutions. 3. Prepare plan for next 6-8 weeks to create PoC. How to start?
  • 21.
    - Which processescan be automated? - What are the targets of our project? - What do we plan to do in the future? Understanding problem
  • 22.
    - Which structureof IoT devices is the most suitable for us? - How do we need to collect information? Do we need to send them to the central platform? Which type of IoT?
  • 23.
    © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Let’s build Data Platform
  • 24.
    © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Perception Business logic CI/CD Idempotency Reprocessing Explainability Monitoring Testing Serving Infrastructure Data Ingestion Security
  • 25.
    © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Reality Business logic CI/CD Idempotency Reprocessing Explainability Monitoring Testing Serving Infrastructure Data Ingestion Security
  • 26.
    - Network - Security -Performance Which type of IoT?
  • 27.
    - Multiregional setup -Private vs. public cloud - Backup and disaster recovery High Availability
  • 28.
    - Audit features -Permissions management - Encrypt traffic between IoT devices - Update software and manage devices from one place Security
  • 29.
    - Storage - Dataingestion - Data processing - ML / AI services - BI tools - Data visualisation - Compute resources - Managed or non-managed services Big Data Services
  • 30.
  • 31.
    - Public cloudor own infrastructure? - Managed or non-managed services - How to store data? - Avoid vendor-lock - is it important? IT infrastructure
  • 33.
    © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Great services for IoT
  • 34.
    - Start withsimple pipelines. - Start with the simplest and most important use case for the company - Create flexible data infrastructure to add new components easily Data pipelines
  • 35.
    Depends on: - amountof data - required time to deliver results - costs and complexity Central or Edge processing
  • 36.
    Streaming or Batch StreamingBatch Process data continuously Process data in batch We process piece-by-piece Once data is collected, it’s sent for processing Fast results and react to defined triggers Great for processing large amount of data
  • 37.
    - Do youneed ML/AI? - What business value can it add to the project? - Edge AI Machine Learning
  • 38.
  • 39.
    - Do wehave in-house skills? - How big team do we need? - How to manage the project? Human skills
  • 40.
    SRE & DevOps Ifa human operator needs to touch your system during normal operations, you have a bug. The definition of normal changes as your systems grow. Carla Geisser, Google SRE
  • 41.
    SRE & DevOps Ifyou think of DevOps like an interface in a programming language, class SRE implements DevOps.
  • 42.
    SLI, SLA &SLO • SLA - Service Level Agreements • SLI - Service Level Indicators • SLO - Service Level Objectives
  • 43.
    Measuring Service Risk Time-basedavailability is traditionally calculated based on the proportion of system uptime. Aggregate availability shows how this yield-based metric is calculated over a rolling window (i.e., proportion of successful requests over a one-day window).
  • 44.
    CICD pipelines Besides blackart, there is only automation and mechanization. Federico García Lorca (1898–1936), Spanish poet and playwright Source: AWS
  • 45.
    © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Stories from life
  • 46.
  • 47.
    - Public-cloud basedplatform - Detect failures from devices in factories - Predictive maintenance - Real-time data streaming for analyzing information Industry 4.0, startup
  • 48.
    - Private-cloud basedplatform - Real-time data streaming for analyzing information - Aggregating and analyzing data sent from edge devices Telecommunication, IoT data
  • 49.
    - Public cloud-basedstartup - Security of the employees in the construction sites - Edge data processing Industry 4.0, startup
  • 50.
  • 52.
    © Copyright. Allrights reserved. Not to be reproduced without prior written consent. Q&A
  • 53.