Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

FIWARE Tech Summit - Quantum Leap - A FIWARE Ttme-series DB

225 views

Published on

Presentation by Tomas Aliaga
Sr. Software Developer, Martel Innovate

FIWARE Tech Summit
28-29 November, 2017
Malaga, Spain

Published in: Technology
  • Be the first to comment

  • Be the first to like this

FIWARE Tech Summit - Quantum Leap - A FIWARE Ttme-series DB

  1. 1. QuantumLeap: a FIWARE time-series DB Tomas Aliaga tomas.aliaga@martel-innovate.com
  2. 2. Some background 1
  3. 3. 2 Orion: Review ▪ Main NGSI Concepts: ▪ Entity Id ▪ Entity Type ▪ Attributes ▪ By design, Orion CB persists only the last pushed value of each attribute. { https://github.com/Fiware/dataModels/blob/master/Weather/ WeatherObserved/example.json "id": "Spain-WeatherObserved-Valladolid-2016-11-30T07:00:00.00Z", "type": "WeatherObserved", "address": { "addressLocality": "Valladolid", "addressCountry": "ES" }, "barometricPressure": 938.9, "dataProvider": "TEF", "dateObserved": "2016-11-30T07:00:00.00Z", "location": { "type": "Point", "coordinates": [ -4.754444444, 41.640833333 ] }, "precipitation": 0, "pressureTendency": 0.5, "relativeHumidity": 1, "source": "http://www.aemet.es", "stationCode": "2422", "stationName": "Valladolid", "temperature": 3.3, "windDirection": -45, "windSpeed": 2, "refDevice": "device-0A3478" }
  4. 4. 3 Comet: Review ▪ Comet brings Historical Data to FIWARE in two forms: • Raw Queries http://fiware-sth-comet.readthedocs.io/en/latest/raw-data-retrieval/index.html http://<sth-host>:<sth-port>/STH/v1/ contextEntities/type/<entityType>/id/<entityId>/attributes/<attrName>? hLimit=3&hOffset=0&dateFrom=2016-01-01T00:00:00.000Z&dateTo=2016-01-31T23:59:59.999Z • Aggregated Queries http://fiware-sth-comet.readthedocs.io/en/latest/aggregated-data-retrieval/index.html http://<sth-host>:<sth-port>/STH/v1/ contextEntities/type/<entityType>/id/<entityId>/attributes/<attrName>? aggrMethod=sum&aggrPeriod=second&dateFrom=2016-02101T00:00:00.000Z&dateTo=2016-01- 01T23:59:59.999Z
  5. 5. 4 Comet: What’s missing ▪ Support for NGSI v2. • Discussed in GH #323 ▪ Support for Cross-Entities Queries. • E.g: Avg temperature of all weather station meters in Madrid? • Discussed in GH #430 • /STH/v1/contextEntities/type/<entityType>/id/<entityId>/attributes/<attrName>? ▪ Support for other than MongoDB. Why? • Difficult to scale it properly. • Schemaless -> no joins -> not easy cross-document queries. • Many new options for efficient and powerful TimeSeries data handling.
  6. 6. QuantumLeap’s Approach 5
  7. 7. 6 Typical Scenario ▪ 1: NGSI Subscription ▪ 2: NGSI Insert/Update ▪ 3: Orion Notification (/notify) ▪ 4: Client Queries ▪ 5: Client Visualisations
  8. 8. 7 1: NGSI v2 Subscription
  9. 9. 8 3: The API proposal: INPUT
  10. 10. 9 4: The API proposal: QUERIES
  11. 11. 10 4: The API proposal: PARAMETERS Query parameter Example Value type WeatherObserved, id station01,station02, idPattern .* attrs temperature,pressure Query parameter Example Value aggrMethod avg aggrPeriod month options see NGSIv2 Query parameter Example Value from 2016-01- 01T00:00:00.000Z to 2016-03- 01T00:00:00.000Z lastN 100 limit 100
  12. 12. 11 The API proposal: META /config endpoint to bring optimization opportunities through the customization of: ▪ Number of replicas ▪ Number of shards ▪ Partitioning by index
  13. 13. Live Example 12
  14. 14. How to get started 13
  15. 15. 14 Docs Available! DOCS: https://smartsdk.github.io/ngsi-timeseries-api/ REPO : https://github.com/smartsdk/ngsi-timeseries-api
  16. 16. 15 Report Issues! https://github.com/smartsdk/ngsi-timeseries-api/issues Don’t forget to include QL’s output log. ▪ When Deployed as a Service • $ docker service ls • Get ID of QuantumLeap service • $ docker service logs QL_SERVICE_ID ▪ When Deployed with Compose • $ docker ps • Get ID of container • $ docker logs QL_CONTAINER_ID
  17. 17. Thank you! http://fiware.org Follow @FIWARE on Twitter 16 https://smartsdk.github.io/ngsi-timeseries-api/ Don’t miss Today! - Docker Swarm secrets … (14:30 @ ROOM 1) - Mastermind ... (15:00 @ ROOM 1)
  18. 18. 18
  19. 19. 19
  20. 20. 20
  21. 21. 21
  22. 22. Thank you! http://fiware.org Follow @FIWARE on Twitter

×