Successfully reported this slideshow.
Your SlideShare is downloading. ×

apidays LIVE Helsinki & North 2022_Apps without APIs

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 16 Ad

apidays LIVE Helsinki & North 2022_Apps without APIs

Download to read offline

apidays LIVE Helsinki & North: API Ecosystems - Connecting Physical and Digital
March 16 & 17, 2022

Apps without APIs - Leveraging the stack that we all use, but never think about
Sampo Savolainen, CTO at Spatineo

apidays LIVE Helsinki & North: API Ecosystems - Connecting Physical and Digital
March 16 & 17, 2022

Apps without APIs - Leveraging the stack that we all use, but never think about
Sampo Savolainen, CTO at Spatineo

Advertisement
Advertisement

More Related Content

Similar to apidays LIVE Helsinki & North 2022_Apps without APIs (20)

More from apidays (20)

Advertisement

Recently uploaded (20)

apidays LIVE Helsinki & North 2022_Apps without APIs

  1. 1. apidays Helsinki & North 16.3.2022 Apps without APIs leveraging the stack that we all use, but never think about
  2. 2. Sampo Savolainen CTO Spatineo focuses on data flows and the intersection of Data APIs, Cloud technologies and Geospatial We have a SaaS platform for API monitoring & analytics for geospatial APIs: Spatineo Monitor
  3. 3. complexity reliability Complexity Complexity has grown over time: - frontend tech - load balancers - middleware - api gateways - cloud platforms & third party APIs - containers - infrastructure as code - IaC middleware / helpers - serverless functions - specialized server implementations - … Our usual tendency is to only add complexity. We must actively think about this and attempt to reduce it when possible t
  4. 4. Tuulituhohaukka 💨💥🦅 Aalto, Forest Centre & FMI are developing an AI model for predicting wind damages in forests Model based on: - Sentinel satellite imagery - Weather data - Forest data, terrain models - Wind damage declarations Image credits: https://www.copernicus.eu/en/media/images/meeting-waters, https://www.copernicus.eu/en/media/images/sentinel-3b-being-mated-rockot-adapter
  5. 5. 2022 SERIES OF EVENT New York JULY (HYBRID) Australia SEPTEMBER (HYBRID) Singapore APRIL (VIRTUAL) Helsinki & North MARCH (VIRTUAL) Paris DECEMBER (HYBRID) London OCTOBER (HYBRID) Hong Kong AUGUST (VIRTUAL) JUNE (VIRTUAL) India MAY (VIRTUAL) APRIL (VIRTUAL) Dubai & Middle East JUNE (VIRTUAL) Check out our API Conferences here Wa nt t o t a lk a t one of our conference? Apply t o spea k here
  6. 6. The App Spatineo built a tool for the project to study Tuulituhohaukka model output Allows comparing model and source data Supports any number of maps side by side, showing the same area and time but different data sets Visual comparison & time series for changes over time .. with lots of data: Sentinel satellites produce 20 TB per day. FMI hosts more than a petabyte of data, growing each day
  7. 7. Getting data in the browser? API: WMS, WMTS or other standards Expensive: 1+ PB of storage on a virtual machine for an application geared towards occasional research use Still requires non-trivial computing power for application server (multi-gigabyte files) Latency when accessing non-cached data No API, direct access to resources and processing offloaded to clients Application server with a local copy? No full mirror, only caching Without an application server altogether Application Server (API) Local copy of data sets index Data set master storage Application Server (API) Local cache of data sets index Data set master storage 💨💥🦅 💨💥🦅 💨💥🦅 Data set master storage index
  8. 8. Standards based Solution for Tuulituhohaukka Cloud optimized GeoTIFFS (COG) Satellite imagery and other data sets in a file format that provides efficient access using capabilities of HTTP STAC Catalogue Interlinked JSON-files describing data sets and organizing the data into collection and subcollections providing the client with an efficient index on available data sets, time periods for which data is available https://stacspec.org/ https://www.cogeo.org/
  9. 9. STAC When new satellite images and other data assets are received, they are analyzed to produce STAC items After items are built, they are indexed into catalogues These catalogues are subdivided into per-month catalogues Clients can index assets via the catalogue hierarchy The Catalogue Asset Item Time Dataset Image credits European Union, Copernicus Sentinel-1 imagery
  10. 10. Cloud optimized GeoTIFF HTTP GET Range queries - read internal index and metadata (tiles, overviews, georeferencing) - data access can be target to only the required tiles at required resolution Allow clients of efficiently retrieve only the parts of the file they need Specialized TIFF files Include georeferencing information Data in an image pyramid: - data structured in tiles - overviews (resampled) Internal index and metadata (georeferencing) Level 0 tiles: full resolution Level 1 tiles L2 3 GeoTIFF Image pyramid illustration credits: https://climserv.ipsl.polytechnique.fr/documentation/idl_help/Image_Tiling.html Client App 💨💥🦅
  11. 11. Metrics Actual download size Session with Chrome Dev Tools cache entirely disabled: Transferred: 250 Mb (includes the browser re-downloading data during this session) Amount of source data Maps: ● Sentinel 1: 2x 10Gb ● Sentinel 2: 3x 1Gb ● Model output: 1x 5 Mb ● x2 with comparison tool The time series: ● 3x 20 x 1Gb images Total: 100+ Gb
  12. 12. #1 #2 #3 #4 #5 #6 Key Takeaways Keep it simple, understand the basics, use what you already have in your tech stack - you can do a lot with just HTTP! No application server or “serverless” required: less maintenance, less code, less cost, one fewer point of failure Very fast - but does require more bandwidth than with a server pre-processing the data (WMS/WMTS) No data replication or cache management - access data directly as they are stored in the master data store Standards based solution helps tech work together: COGs work as-is in openlayers, and time series simple to produce with geotiff.js Because of standards, catalogue and data usable for other applications as well
  13. 13. 16.3.2022 Sampo Savolainen sampo.savolainen@spatineo.com https://www.spatineo.com/ PS. We are hiring…

×