Downloaded 24 times
Uploaded byCARTO
Think Spatial: Don't Ignore Location in your Models! [CARTOframes]
The document outlines Carto's capabilities in location intelligence, including data enrichment, analysis, and model integration for spatial data. It highlights the importance of considering spatial dependence in models to avoid biased results and presents various types of spatial data and modeling approaches. Additionally, it discusses the Carto Analysis Framework and API for executing user-defined analyses and emphasizes the need for appropriate data resolution and model scalability.
More Related Content
![Unlock the power of spatial analysis using CARTO and python [CARTOframes]](https://cdn.slidesharecdn.com/ss_thumbnails/unlockthepowerofspatialanalysisusingcartoandpython-190927095046-thumbnail.jpg?width=640&height=640&fit=bounds)
![The Sum of our Parts: the Complete CARTO Journey [CARTO]](https://cdn.slidesharecdn.com/ss_thumbnails/recordedwebinarthesumofourpartsthecompletecartojourneydeck-190424092526-thumbnail.jpg?width=640&height=640&fit=bounds)
![CARTO en 5 Pasos: del Dato a la Toma de Decisiones [CARTO]](https://cdn.slidesharecdn.com/ss_thumbnails/cartoen5pasosdeldatoalatomadedecisionesrecordedwebinar-190508103941-thumbnail.jpg?width=640&height=640&fit=bounds)
![How to Use Spatial Data Science in your Site Planning Process? [CARTOframes]](https://cdn.slidesharecdn.com/ss_thumbnails/webinardeckhowtousespatialdatascienceinyoursiteplanning-190828105208-thumbnail.jpg?width=640&height=640&fit=bounds)
![Tips and tricks for Working with Demographic Data [CARTOframes & Python]](https://cdn.slidesharecdn.com/ss_thumbnails/webinar-tipsandtricksforworkingwithdemographicdata-190319103145-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Think Spatial: Don't Ignore Location in your Models! [CARTOframes]
![Spatial_Data_Analysis_with_open_source_softwares[1]](https://cdn.slidesharecdn.com/ss_thumbnails/8db4d971-8e8c-4fd8-8682-b20e5d6cd65f-161221072847-thumbnail.jpg?width=640&height=640&fit=bounds)
![Hacking-Uncovered-How-People-Get-Hacked-and-How-to-Stay-Safe[1].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/hacking-uncovered-how-people-get-hacked-and-how-to-stay-safe1-260130170011-4883a9c7-thumbnail.jpg?width=640&height=640&fit=bounds)
Think Spatial: Don't Ignore Location in your Models! [CARTOframes]
- 1. FOLLOW @CARTO ONTWITTER
- 2. The Sum ofOur Parts Data Scientist Content Marketing Manager
- 3. CARTO is theplatform to build powerful Location Intelligence apps with the best data streams available.
- 4. CARTO Customers Pioneers in LocationIntelligence 1,200 End-users 300K Team members 100+
- 5. The Complete Journey 1.Data 2. Enrichment 3. Analysis 4. Solutions 5. Integration
- 6. The Complete Journey 1.Data 2. Enrichment 3. Analysis 4. Solutions 5. Integration
- 7. Enrichment Augment any data withdemographic data from around the globe with easeData Observatory Develop robust ETL processes and update mechanisms so your data is always enriched Mastercard Human Mobility POI
- 8. The Journey -Analysis Bring CARTO maps and data into your data science workflows and the Python data science ecosystem to work with Pandas, PySal,PyMC3, scikit-learn, etc. CARTOFrames Use the power of PostGIS and our APIs to productionalize analysis workflows in your CARTO platform. PostGIS by CARTO SQL API Python SDK
- 9. John Snow’s mapof cholera cases in London 1854. Red circles indicate locations of cholera cases and blue circles indicate locations of water pumps. ………………………………………………………………………………………………………….………………….…...
- 10. “Everything is relatedto everything else, but near things are more related than distant things.” (Tobler, 1970) ………………………………………………………………………………………………………….………………….…...
- 11. Modelling dependence oncovariates and the spatial correlation structure ● Estimation of underlying model parameters ● Prediction at unsampled locations ● Change of support (downscaling/upscaling) ………………………………………………………………………………………………………….………………….…...
- 12. Failure to includespatial dependence in your model can lead to biased statistical results and erroneous conclusions. How can CARTO help me with my spatial models? ➢ Types of spatial data ➢ Spatial modelling ➢ Demos ………………………………………………………………………………………………………….………………….…...
- 14. ● GPS tracking ●Fixed measuring devices ● High resolution satellites Geostatistical data We are thinking of a continuous spatial field
- 15. ● Census data ●Region-based counts ● Coarse resolution satellites Region-based data We are observing a discrete spatial field, but what are we thinking of?
- 16. ● Census data ●Region-based counts ● Coarse resolution satellites Region-based data We are observing a discrete spatial field, but what are we thinking of?
- 17. ● Locations ofoccurrences of some event ● Locations of trees ● UFO sightings Point patterns We are thinking of occurrences of events
- 19. ● We needa complex function based on the coordinates to adequately describe the effect of the location ● Regression models using the location's coordinates as predictors do not work well! ● More natural to explicitly model the variations of the process considering that it may be similar at nearby locations ………………………………………………………………………………………………………….………………….…...
- 20. What we aretrying to model (or the response variable) This is modelling! ………………………………………………………………………………………………………….………………….…...
- 21. The mean structure e.g.some function of some covariates The residual (or what is not explained by the mean structure) What we are trying to model (or the response variable) ………………………………………………………………………………………………………….………………….…...
- 22. But the numberof ways we could construct a model for the spatial process is unlimited! Spatially continuous models ➢ Gaussian processes (GP) Spatially discrete models ➢ Gaussian Markov Random Fields (GMRF) ………………………………………………………………………………………………………….………………….…...
- 23. ➢ A GPis parameterized by a mean function and covariance function ➢ as ↑ then ↓ ➢ depends on some parameters e.g.: the exponential covariance: The joint distribution of a finite number of outputs is a Gaussian! ………………………………………………………………………………………………………….………………….…...
- 24.
- 25.
- 26.
- 27.
- 28. ➢ How canwe properly account for the uncertainty in the spatial dependence structure? THINK BAYESIAN! DATA LEVEL (LIKELIHOOD) PROCESS LEVEL PRIOR LEVEL ………………………………………………………………………………………………………….………………….…...
- 29. ➢ THE BIGPROBLEM: computations scale as O(N3 ), for more than a few thousand points this is intractable! Construct a DISCRETE APPROXIMATION of the continuous field Figure from Cameletti et al. (AStA, 2013) ………………………………………………………………………………………………………….………………….…...
- 30. ➢ Based onneighbourhood structures 𝑖-th area first-order neighbours second-order neighbours ➢ Markov means conditional independence ………………………………………………………………………………………………………….………………….…...
- 31. Under the Markovianproperty, the elements in the precision matrix (the inverse of the covariance) are non-zero only for neighbours ➢ Fast computations due to a sparse precision matrix! ➢ Difficult to construct reasonable dependence structures 0.1% of non-zero elements! ………………………………………………………………………………………………………….………………….…...
- 32.
- 33. ➢ Compare revenuesfrom each travel agency to market performance ➢ We can use data from credit cards from purchases in the travel sector ………………………………………………………………………………………………………….………………….…...
- 34. Travel agencies Credit carddata ………………………………………………………………………………………………………….………………….…...
- 35. ➢ Compare revenuesfrom each travel agency to market performance ➢ We can use data from credit cards from purchases in the travel sector ➢ BUT… credit card data get anonymized in many locations ………………………………………………………………………………………………………….………………….…...
- 36. 1 month ofdata 5 months of data 12 months of data ………………………………………………………………………………………………………….………………….…...
- 37. ➢ Compare revenuesfrom each travel agency to market performance ➢ We can use data from credit cards from purchases in the travel sector ➢ BUT… credit card data get anonymized in many locations ………………………………………………………………………………………………………….………………….…... CAN WE PREDICT AT LOCATIONS WHERE THERE ARE NO DATA? R package: mgcv, Wood (2011, Journal of the Royal Statistical Society: Series B)
- 38. CAN WE PREDICTAT LOCATIONS WHERE THERE ARE NO DATA? ➢ Compare revenues from each travel agency to market performance ➢ We can use data from credit cards from purchases in the travel sector ➢ BUT… credit card data get anonymized in many locations ………………………………………………………………………………………………………….………………….…... R package: mgcv, Wood (2011, Journal of the Royal Statistical Society: Series B)
- 39. w/ GRMF smoothw/oGRMF smooth PREDICTED ORIGINAL ORIGINAL ………………………………………………………………………………………………………….………………….…...
- 40.
- 41.
- 42.
- 43. ➢ Upload yourdata to CARTO and viz it using CARTOframes First we need to define the aggregation or zoom level. At CARTO we use QuadKeys ………………………………………………………………………………………………………….………………….…...
- 44. NUMBER OF TRANSACTIONSWHERE WE WANT TO PREDICT ➢ Upload your data to your CARTO account and plot it using CARTOframes ………………………………………………………………………………………………………….………………….…...
- 45. ➢ Before modelling,enrich your data with CARTO DATA OBSERVATORY (DO) ………………………………………………………………………………………………………….………………….…...
- 46. ➢ Before modelling,enrich your data with CARTO DATA OBSERVATORY (DO) ………………………………………………………………………………………………………….………………….…...
- 47. ➢ Before modelling,enrich your data with CARTO DATA OBSERVATORY (DO) ………………………………………………………………………………………………………….………………….…...
- 48. NUMBER OF TRANSACTIONSDATA WE WANT TO USE AS COVARIATES e.g. POPULATION ➢ Before modelling, viz with CARTOframes ………………………………………………………………………………………………………….………………….…...
- 49. NUMBER OF TRANSACTIONSDATA WE WANT TO USE AS COVARIATES e.g. NUMBER OF FOOD POIs ➢ Before modelling, viz with CARTOframes ………………………………………………………………………………………………………….………………….…...
- 50. PRIORS HYPER PRIORS PROCESS DATA ………………………………………………………………………………………………………….………………….…... NUMBER OFTRANSACTIONS R package: R-INLA, Lindgren and Rue (2015, JSS)
- 51. PREDICTED NUMBER OFTRANSACTIONS (MEAN) NUMBER OF TRANSACTIONS ………………………………………………………………………………………………………….………………….…...
- 52. PREDICTED NUMBER OFTRANSACTIONS (STANDARD DEVIATION) NUMBER OF TRANSACTIONS ………………………………………………………………………………………………………….………………….…...
- 53. Population # POIfood # POI entertainment Income# POI transport # employees ………………………………………………………………………………………………………….………………….…...
- 54. RANDOM SPATIAL EFFECT (MEAN) RANDOMSPATIAL EFFECT (STANDARD DEVIATION) SPATIAL DOMAIN ………………………………………………………………………………………………………….………………….…...
- 55. PREDICTED NUMBER OFTRANSACTIONS (MEAN) ………………………………………………………………………………………………………….………………….…...
- 56. ➢ Think carefullywhat is the problem you are trying to solve and get the right data at the right spatial resolution CARTO Data Observatory ………………………………………………………………………………………………………….………………….…...
- 57. ➢ Think carefullywhat is the problem you are trying to solve and get the right data at the right spatial resolution ➢ Choose a scalable model and a flexible implementation CARTO Data Observatory ………………………………………………………………………………………………………….………………….…... CARTO Analysis Framework and API A Framework for provisioning, orchestrating, executing and monitoring of analyses (processes) An API to define, register, schedule and execute user-defined analysis written in virtually any language
- 58. ➢ Think carefullywhat is the problem you are trying to solve and get the right data at the right spatial resolution ➢ Choose a scalable model and a flexible implementation ➢ The estimates we construct come from a complicated interaction of the model and the computational method: visualization (and other metrics) are essential CARTO Data Observatory ………………………………………………………………………………………………………….………………….…... CARTO Analysis Framework and API CARTOframes
- 59. Request a demoat CARTO.COM Data Scientist // giulia@carto.com Content Marketing Manager // sisaac@carto.com