Les données de séries chronologiques sont de plus en plus au cœur des applications modernes: pensez à l'IoT, aux transactions sur actions, aux flux de clics, aux médias sociaux, etc. Avec le passage des systèmes batch aux systèmes temps réel, la capture et l'analyse efficaces des données de séries chronologiques peuvent permettre aux entreprises de mieux détecter et réagir aux événements en avance sur leurs concurrents ou d'améliorer l'efficacité opérationnelle pour réduire les coûts et les risques. Travailler avec des données de séries chronologiques est souvent différent des données d’application classiques et vous devez observer les meilleures pratiques. Cette conférence couvre: Composants communs d'une solution IoT Les défis liés à la gestion de données chronologiques dans les applications IoT Différentes conceptions de schéma et leur incidence sur l'utilisation de la mémoire et du disque sont deux facteurs déterminants dans les performances des applications. Comment interroger, analyser et présenter les données de séries chronologiques IoT à l'aide de MongoDB Compass et MongoDB Charts À la fin de la session, vous aurez une meilleure compréhension des meilleures pratiques clés en matière de gestion des données de séries chronologiques de l'IoT avec MongoDB.

1. #MDBlocal
PARIS
Best practices:
Working with IoT and Time-Series Data
Julien Contarin, Sr Solutions Architect, MongoDB

2. #MDBLocal

3. #MDBLocal
Agenda
Define time-series data
Review typical IoT Architecture
Designing time-series schemas
Query and visualize time-series data

4. #MDBLocal
What is time series data?

5. Data in an IoT Architecture

6. #MDBLocal
Common IoT Architecture

7. #MDBLocal
Common IoT Architecture

8. #MDBLocal
Common IoT Architecture

9. #MDBLocal
Common IoT Architecture

10. #MDBLocal
Common IoT Architecture

11. #MDBLocal
Common IoT Architecture

12. #MDBLocal
MongoDB IoT Reference Architecture

13. Data requirements in IoT
solution

14. #MDBLocal
Data Platform Requirements
Agility Schema Flexibility
Real-time Data Archiving
Global Presence Data Protection
IoT data is a Big Data problem - Volume, Velocity, Variety

15. Designing time-series schemas

16. #MDBLocal
Breaking the relational mindset….

17. #MDBLocal
MongoDB Schema Design
for Time Series data

18. #MDBLocal
Effects on Document count
Per second vs Per Minute

19. #MDBLocal
Effects on data storage
Per second vs Per Minute

20. #MDBLocal
Effects on memory
Per second vs Per Minute

21. #MDBLocal
Sample code to add to the size based bucketMongoDB Schema Design
for Time Series data

22. #MDBLocal
Case Study:
https://www.ampcontrolgroup.com/
Leaders in industrial electrical and electronic
engineering
Selected MongoDB for Gas-Detection Mining Application

23. #MDBLocal
Case Study:

24. #MDBLocal
Case Study:
One Document per Device per Parameter per Hour
Pre-aggregated data updated for every update
After 3 months samples array removed
Data archived for 7 years

25. #MDBLocal
Keys to successful time series applications
Know your application requirements
Writes Reads Data Retention Security

26. #MDBLocal
What to do with old data
• Pre-aggregation
• Offline archival
• db.remove
• TTL Indexes
• Dropping the collection
• Online archival
• Queryable backups
• Sharding archival data
• Offload to data warehouse / S3 (Atlas Data Lake)

27. How to query and visualize
time-series data

28. #MDBLocal
Real-time Analytics
One database for operational and analytical
data
Easy to query unstructured data
MongoDB Query Language (MQL)
Aggregation Framework

29. #MDBLocal
Using MQL Demo
https://github.com/julienmongodb/mongodb-iot

30. #MDBLocal
Key takeaways
Data Platform
Document Model
Aggregation Framework

31. #MDBLocal
Best practices summary
Know your requirements
1 sensor reading per document works but not the most efficient
Data retention policy
Data access requirements
For very large data consider multiple collections
Faster to archive/drop collections than db.remove()
Recommended design fixed-based schema
https://www.mongodb.com/collateral/time-series-best-practices
Archive data to S3 / Atlas Data Lake for long term storage and analysis

32. Why MongoDB for IoT and Time-
Series Data?

33. THANK YOU

35. Extra Slides

36. #MDBLocal
Real-time Analytics
Easily query unstructured data with Aggregation Framework

37. #MDBLocal
What is the largest value for a given sensor id?
db.SensorData.aggregate( [
{$match: { "sensorid":3003 }},
{$unwind: { path: '$samples' }},
{$group: { _id: ‘$date.day', MaxValue: {
$max: '$samples.val' } }},
{$sort: { "id": 1 }} ])

38. #MDBLocal
What is the largest value for a given sensor id?
db.SensorData.aggregate( [
{$match: { "sensorid":3003 }},
{$unwind: { path: '$samples' }},
{$group: { _id: '$day', MaxValue: { $max:
'$samples.val' } }},
{$sort: { "id": 1 }} ])

39. #MDBLocal
What is the largest value for a given sensor id?
db.SensorData.aggregate( [
{$match: { "sensorid":3003 }},
{$unwind: { path: '$samples' }},
{$group: { _id: '$day', MaxValue: { $max:
'$samples.val' } }},
{$sort: { "id": 1 }} ])

40. #MDBLocal
What is the largest value for a given sensor id?
db.SensorData.aggregate( [
{$match: { "sensorid":3003 }},
{$unwind: { path: '$samples' }},
{$group: { _id: '$day', MaxValue: { $max:
'$samples.val' } }},
{$sort: { "id": 1 }} ])

41. #MDBLocal
Cloud – Real-time Analytics
What is the largest value for a given sensor id?
db.SensorData.aggregate( [
{$match: { "sensorid":3003 }},
{$unwind: { path: '$samples' }},
{$group: { id: '$day', MaxValue: { $max:
'$samples.val' } }},
{$sort: { "id": 1 }} ])
{ "_id" : ISODate("2018-12-10T00:00:00Z"), "MaxValue" : 40.4 }
{ "_id" : ISODate("2018-12-11T00:00:00Z"), "MaxValue" : 40.43 }
{ "_id" : ISODate("2018-12-12T00:00:00Z"), "MaxValue" : 40.09 }
{ "_id" : ISODate("2018-12-13T00:00:00Z"), "MaxValue" : 39.7 }
…

42. #MDBLocal
Cloud – Visualization
MongoDB Charts

43. #MDBLocal
Cloud – Visualization
MongoDB BI Connector

44. #MDBLocal
Integration with