A session by Richard Winter, Founder, Wintercorp on the topic of 'IoT Data -- Like No Data We have Ever Seen' at InterCon USA 2019, held at Caesars Palace, Las Vegas on 18-20 June, 2019.

1. W I N T E R C O R P O R A T I O N
E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
IoT Data:
Like No Other Data We Have Ever Seen
Richard Winter, CEO

2. 2 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
In Just One Year
Petabytes of raw data,
trillions of records
340,000 flights for just
the 787 fleet globally

3. 3 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
Why Do They Need All That Data?
Wind, pressure,
temperature,
radiation, turbulence
Cabin air,
pressure,
humidity,
temperature
Video, GPS,
altitude, tilt, velocity,
Cabin doors
& baggage
Engine controls,
health, fuel, speed,
temperature, vibration,
pressure
Electric &
aux power
Wing & tail flaps
Landing gear,
wheels, brakes

4. 4 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
Operating Heavy Equipment
• Situation
– Digging for metals | Rebuilding
dirt roads
• Problem
– 400-ton machines | Safety,
efficiency, costs
• Solution
– Data lake + data warehouse
• Results
– Monitoring location, shifting,
strut pressures | fewer repairs
Photo by Dominik Vanyi on Unsplash

5. 5 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
Smart
buildings
1st
responders
Waste
mgmt
Connected
vehicles
Infrastructure
services
Energy
grid
Rail &
freight
Health &
pollution
Smart Cities
Photo by Jonathan Roger on Unsplash

6. W I N T E R C O R P O R A T I O N
E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
What Makes this So Difficult?
What Capabilities are Needed?

7. 7 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
Time-Series Sensor Data — Sawtooth Waves
SensorID
A2
P4
P5
X9
L3
High volume: date-time + sensor ID + numeric array

8. 8 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
Challenge: Sensor Data Integration
• Data preparation
– No standard data formats
– Missing data, noise
– Sensors that lie
– Network disconnects/reconnects
• Petabyte scale
– 100s of concurrent streams
– Continuous processing
– Distributed storage planning
Sensor X day 1
Sensor X day 2
measurements
Missing data?

9. 9 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
Challenge: Sensor Data
Analysis
• Correlating different range scales
– Sliding time window
• Finding the signal in the noise
– Many measures are not important
– Find measures above normal limits
• Complex algorithms
– Fast fourier, shaplets, Arima, Dickey–
Fuller
– Algorithms crash on missing data
– Data scientists required
Heat
limit
Time
correlating
2 sensors
Sliding
window
Not an
event
Pressure
limit
Time
2120
1000
5000psi
1000psi

10. W I N T E R C O R P O R A T I O N
E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
An IoT Architecture for Analytics

11. “This new product data is valuable by itself, yet its value
increases exponentially when it is integrated with other data, such as service
histories, inventory locations, commodity prices, and traffic patterns...”
Dr. Michael Porter, Harvard

12. 12 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
IoT General Architecture – Executive View
© 2016 Teradata
The Edge
Administration, Security, Analytics, & Governance
Operations & Analytics
Sensors&Actuators
Control
Systems
Gateways
Data Lake
Data
Warehouse
AnalyticsSecurity
IoT data
streams
Policies
Data Mgmt.
Application
s
Device
Mgmt.

13. W I N T E R C O R P O R A T I O N
E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
Doing It Right

14. 14 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
Boeing’s Big Data
Analytics Pipeline
• Data lake
– Petabytes of sensor data
– Data preparation
– R&D, government regulations
• Data warehouse
– Temporal compression
– Integrated data
– Parts, supply chain
– Designs, schedules
– Business user analysis Data Lake
Data Warehouse
Temporal SQL Queries
Temporal fact/
dimension tables
Data Warehouse
Product & Support
Parallel decode &
data normalization Hive
table

15. 15 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
Benefits
• 80X sensor data compression
– Discard repetitive measurements
– Saves storage costs
• Up to 600X faster query analysis
– Saves money on compute
• So What?
– More analytics per hour
– Increased accuracy
– Organizational agility
After:
1000 flights
analyzed in
17 minutes
Before:
200 hours to
analyze 100
flight hours

16. 16 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
• The IoT Future is Exciting
• Visionaries and Fast Followers
are reaping ROI
• Build a solid analytic
architecture
– Scalability
– Security
Summary

17. 17 W I N T E R C O R P : E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019

18. W I N T E R C O R P O R A T I O N
E X P E R T S I N A N A L Y T I C D A T A M A N A G E M E N T A T S C A L E
© WinterCorp LLC, Tyngsboro MA 2019
The End
Richard Winter
richard@wintercorp.com
617-849-7030