SlideShare a Scribd company logo
1 of 32
Working with Space-time Data in ArcGIS
Aileen Buckley
This session
• Visualizing temporal data
• Managing temporal data
• Analyzing temporal data
• Best practices are highlighted
with a check mark
•
• ArcUser article – Spring 2018
has a high-level overview

DEMO
Visualizing space-time data
Managing space-time data
Storing temporal data
• Store time stamps in a date field
- A field type that stores dates, times, or dates
and times
- Especially for large data
- Most efficient format for query / display
performance
- Supports more sophisticated database queries

ArcGIS supports
a wide range of
standard formats
and custom
formats
Storing range data
• Store data in a string or numeric fields
• For data that is not in “time” format
- Sequence, attributes, etc...
- Millions of years ago (ArcGIS display
date covers AD100 to AD10,000)
• Use numeric data in integer or float
format
• Set aliases for values
- 3rd day of week (value = 3), set alias to
“Tuesday”
- 20 MYA (value = 20), set alias to [field] &
“MYA”

• Ecological Marine Units – water depth
As separate features in a single feature class
• When the shape and/or
location of each feature
changes over time
• Store separate features
Moving
features
Features that move
over space
• Airplanes, boats,
vehicles
• People, animals
• Storm centers
Discrete
events
Events that happen at
specific locations
• Crimes
• Accidents
• Earthquakes,
lightening strikes,
volcanic events
Change /
growth
Features that change
condition over time
• Demographics
• Fire perimeters
• Flood extents
As features joined to a table
• When the shape and/or location of each feature is constant,
but attribute values change over time
• Store the changing attributes in a separate
(one-to-many) joined table
Stationary
recorders
Features that stay in one
place and record changes
• Weather stations
• Traffic sensors
• Stream gauges
Index temporal and range data
• Index the date or range field for faster query
performance
• Indexes are a property of the data that you can
set in Catalog
• Use the Add Attribute Index GP tool to add an
index to a field

When temporal data is not in date format
• Convert data to a date field type
• Use the Convert Time Field GP tool
- Converts Text/Number fields into a new Date field
- “July 09, 2016”  07/09/2016  MM/DD/YYYY
- Does not require you to add a new field first
The Date field can have a
custom format (e.g.,
MM dd, yyyy HH:mm:ss)
Change Output Time Type to
Text to do this
Convert the custom date to
a real date in a date field


When point in time data needs to be duration data
• Create an end time from the exiting start times
• Use the Calculate End Time GP tool
- Populates an end time field with the next record’s start time
- The last record will have
- For the last value, manually edit the end time if you know
what it should be

blah, blah, blah

When temporal data is stored in multiple columns
• Store temporal data in a row format (one time stamp per row)
- That is, duplicate features with different time stamps in separate rows
• Use the Transpose Fields GP tool
- Shifts data stored in columns into data stored in rows

When temporal data is in a separate table
• Create a one-to-one, many-to-one, or one-to-many join
• Use the Add Join GP tool
• For some GP tools, you will need to export the features (Copy Features GP tool) to
create a new feature class
- For example, Spatial Statistics and Space-Time Pattern Mining tools


When temporal data is in different time zones
• ArcGIS integrates data in different layers
across different time zones
- Layers in map can be in different time zones
• Convert data for same layer into the
same time zone
• Use the Convert Time Zone GP tool
- Converts time values recorded in a date field from
one time zone to another time zone

New York Los Angeles
When temporal data is stored in daylight savings time
• Store temporal data in standard time
- Multiple problems with DST
- Storing the time values in standard time
prevents loss or overlaps of data
- Prevents ambiguity in visualizations and
errors in analyses
• Use the Calculate GP tool to add or
subtract time
• Standardize on UTC1 or GMT2
- 1 Coordinated Universal Time
- 2 Greenwich Mean Time

• Regional differences
• Rules and boundaries change frequently
• Some DST zones adjust less than an hour
• 30 minute offset
• 45 minute offset

When you need to add or subtract from temporal data
• Use the Calculate GP tool to add or subtract time
- Python 3: !timefield!.replace(year=!timefield!.year + 10)
- Arcade: DateAdd( date, addValue, units)
https://docs.python.org/3/library/datetime.html
https://developers.arcgis.com/arcade/function-reference/date_functions/

Section Subhead
Analyzing space-time data
Space-time data analysis in ArcGIS – 3 ways it is done
1st approach
• Data for each time step is analyzed separately
• Results are presented as a single time enabled layer or a set of layers, one for each time
step
1st approach – 3 ways it is done
1. All GP tools honor the time settings
- For time-enabled layers, only those features that fall temporally within the time extent set
in the time slider will be processed
- Similar to a selection or definition query
- Results are then displayed as a single layer on a map
1st approach – 3 ways it is done
1. All GP tools honor the time settings
2. With some GP tools, the time attribute can be used as the case field to repeat the
analysis for each time step
- The case field is used to calculate statistics separately for each unique attribute value
(that is, each time step)
- Results are presented as a single layer, and time can be enabled on the case field in the
output feature class
- e.g., Mean Center and Directional Distribution
1st approach – 3 ways it is done
1. All GP tools honor the time settings
2. With some GP tools, the time attribute can be used as the case field to repeat the
analysis for each time step
3. A model or script with an iterator to specifies that the analysis should be repeated
for each time step
- Results are presented as separate outputs, one for each time step
- The separate outputs can then be combined so that the final result can be time enabled
and visualized using the time slider control
- For feature classes, use Merge or Append
- For rasters, use a mosaic dataset
2nd approach
• Data is analyzed using space-time constraints
• Results are presented as a single layer
The second approach
• Certain GP tools use a spatial weights matrix
• This defines feature relationships in terms of a
space-time window
- Specified critical distance and fixed interval in time
• Tools:
- Hot Spot Analysis – creates a map of clusters in both space
and time of significantly high (hot) or low (cold) values
- Cluster and Outlier Analysis – identifies the similarity (as the
spatial clustering of either high or low values) or the
dissimilarity (as spatial outliers) of features
- Spatially Constrained Multivariate Clustering – finds
spatially contiguous clusters of features based on a set of
feature attribute values and optional cluster size limits
3rd approach
• Data for all time steps is analyzed both spatially and temporally
• Results are presented as a single layer or a space-time cube
3rd approach
• Space-Time Pattern Mining Tools
- Require that the data be in a space-time cube
format
- Emerging Hot Spot Analysis – identifies hot and cold
spots and determines if they have a temporal trend
(new versus persistent versus historic hot spots)
- Local Outlier Analysis – determines if features have
belonged in clusters or been outliers over the entire
time extent of the dataset
- Time Series Clustering – partitions a collection of
time series, stored in a space-time cube, based on
the similarity of time series characteristics
3rd approach
• Space-Time Pattern Mining Tools - Utilities
- Working with space-time cubes
- Create the cube from points or polygons
- Visualize the cube in 2D or 3D
- Fill Missing Values (Utilities toolset) – replaces
missing values with estimates based on spatial
neighbors, space-time neighbors, or time-series
values
• Space Time Cube Explorer Add-In can also be
used for visualization
More traditional time series analysis
• Take advantage of the extensibility of the ArcGIS platform
• Capabilities to easily transfer data to open-source analytical software
- Use ArcPy utility functions to analyze the data in Python
- For example, FeatureClassToNumPyArray
- Use the R-ArcGIS Bridge to analyze the data using R
Aileen Buckley
Mark Gilbert
Thank you!

More Related Content

What's hot

Best practices for highly available and large scale SolrCloud
Best practices for highly available and large scale SolrCloudBest practices for highly available and large scale SolrCloud
Best practices for highly available and large scale SolrCloudAnshum Gupta
 
Analyzing and mapping space-time data
Analyzing and mapping space-time dataAnalyzing and mapping space-time data
Analyzing and mapping space-time dataAileen Buckley
 
Land Use/Land Cover Detection
Land Use/Land Cover DetectionLand Use/Land Cover Detection
Land Use/Land Cover DetectionWansoo Im
 
QGIS 기초
QGIS 기초 QGIS 기초
QGIS 기초 slhead1
 
Working with ArcGIS Online
Working with ArcGIS OnlineWorking with ArcGIS Online
Working with ArcGIS OnlineEsri
 
Introduction To PostGIS
Introduction To PostGISIntroduction To PostGIS
Introduction To PostGISmleslie
 
Mapserver vs. geoserver
Mapserver vs. geoserverMapserver vs. geoserver
Mapserver vs. geoserver鸣 饶
 
Geospatial data platform at Uber
Geospatial data platform at UberGeospatial data platform at Uber
Geospatial data platform at UberDataWorks Summit
 
Side by Side with Elasticsearch & Solr, Part 2
Side by Side with Elasticsearch & Solr, Part 2Side by Side with Elasticsearch & Solr, Part 2
Side by Side with Elasticsearch & Solr, Part 2Sematext Group, Inc.
 
WMS Performance Shootout 2010
WMS Performance Shootout 2010WMS Performance Shootout 2010
WMS Performance Shootout 2010Jeff McKenna
 
Apache Spark Performance Troubleshooting at Scale, Challenges, Tools, and Met...
Apache Spark Performance Troubleshooting at Scale, Challenges, Tools, and Met...Apache Spark Performance Troubleshooting at Scale, Challenges, Tools, and Met...
Apache Spark Performance Troubleshooting at Scale, Challenges, Tools, and Met...Databricks
 
Qgis tutorial compiled
Qgis tutorial compiledQgis tutorial compiled
Qgis tutorial compiledSARSIJ MISHRA
 
하이브 최적화 방안
하이브 최적화 방안하이브 최적화 방안
하이브 최적화 방안Teddy Choi
 
SOS: Optimizing Shuffle I/O with Brian Cho and Ergin Seyfe
SOS: Optimizing Shuffle I/O with Brian Cho and Ergin SeyfeSOS: Optimizing Shuffle I/O with Brian Cho and Ergin Seyfe
SOS: Optimizing Shuffle I/O with Brian Cho and Ergin SeyfeDatabricks
 
Join Algorithms in MapReduce
Join Algorithms in MapReduceJoin Algorithms in MapReduce
Join Algorithms in MapReduceShrihari Rathod
 
Satellite image processing
Satellite image processingSatellite image processing
Satellite image processingalok ray
 

What's hot (20)

Raster data ppt
Raster data pptRaster data ppt
Raster data ppt
 
Best practices for highly available and large scale SolrCloud
Best practices for highly available and large scale SolrCloudBest practices for highly available and large scale SolrCloud
Best practices for highly available and large scale SolrCloud
 
Analyzing and mapping space-time data
Analyzing and mapping space-time dataAnalyzing and mapping space-time data
Analyzing and mapping space-time data
 
Land Use/Land Cover Detection
Land Use/Land Cover DetectionLand Use/Land Cover Detection
Land Use/Land Cover Detection
 
QGIS 기초
QGIS 기초 QGIS 기초
QGIS 기초
 
Working with ArcGIS Online
Working with ArcGIS OnlineWorking with ArcGIS Online
Working with ArcGIS Online
 
Introduction To PostGIS
Introduction To PostGISIntroduction To PostGIS
Introduction To PostGIS
 
Introduction to GIS
Introduction to GISIntroduction to GIS
Introduction to GIS
 
Mapserver vs. geoserver
Mapserver vs. geoserverMapserver vs. geoserver
Mapserver vs. geoserver
 
Geospatial data platform at Uber
Geospatial data platform at UberGeospatial data platform at Uber
Geospatial data platform at Uber
 
GIS file types
GIS file typesGIS file types
GIS file types
 
Side by Side with Elasticsearch & Solr, Part 2
Side by Side with Elasticsearch & Solr, Part 2Side by Side with Elasticsearch & Solr, Part 2
Side by Side with Elasticsearch & Solr, Part 2
 
WMS Performance Shootout 2010
WMS Performance Shootout 2010WMS Performance Shootout 2010
WMS Performance Shootout 2010
 
Apache Spark Performance Troubleshooting at Scale, Challenges, Tools, and Met...
Apache Spark Performance Troubleshooting at Scale, Challenges, Tools, and Met...Apache Spark Performance Troubleshooting at Scale, Challenges, Tools, and Met...
Apache Spark Performance Troubleshooting at Scale, Challenges, Tools, and Met...
 
Qgis tutorial compiled
Qgis tutorial compiledQgis tutorial compiled
Qgis tutorial compiled
 
하이브 최적화 방안
하이브 최적화 방안하이브 최적화 방안
하이브 최적화 방안
 
SOS: Optimizing Shuffle I/O with Brian Cho and Ergin Seyfe
SOS: Optimizing Shuffle I/O with Brian Cho and Ergin SeyfeSOS: Optimizing Shuffle I/O with Brian Cho and Ergin Seyfe
SOS: Optimizing Shuffle I/O with Brian Cho and Ergin Seyfe
 
Join Algorithms in MapReduce
Join Algorithms in MapReduceJoin Algorithms in MapReduce
Join Algorithms in MapReduce
 
Satellite image processing
Satellite image processingSatellite image processing
Satellite image processing
 
Spatial databases
Spatial databasesSpatial databases
Spatial databases
 

Similar to Working with space time data in ArcGIS

Methods for analyzing and mapping temporal data
Methods for analyzing and mapping temporal dataMethods for analyzing and mapping temporal data
Methods for analyzing and mapping temporal dataAileen Buckley
 
Methods for Mapping Temporal Data
Methods for Mapping Temporal DataMethods for Mapping Temporal Data
Methods for Mapping Temporal DataAileen Buckley
 
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...Dawn Wright
 
BREEZE 3D Analyst for the Advanced AERMOD Modeler
BREEZE 3D Analyst for the Advanced AERMOD ModelerBREEZE 3D Analyst for the Advanced AERMOD Modeler
BREEZE 3D Analyst for the Advanced AERMOD ModelerBREEZE Software
 
Survey On Temporal Data And Change Management in Data Warehouses
Survey On Temporal Data And Change Management in Data WarehousesSurvey On Temporal Data And Change Management in Data Warehouses
Survey On Temporal Data And Change Management in Data WarehousesEtisalat
 
GRASS as a Temporal GIS - Sören Gebbert
GRASS as a Temporal GIS - Sören GebbertGRASS as a Temporal GIS - Sören Gebbert
GRASS as a Temporal GIS - Sören GebbertLuis_de_Sousa
 
Accumulo Summit 2016: GeoMesa: Using Accumulo for Optimized Spatio-Temporal P...
Accumulo Summit 2016: GeoMesa: Using Accumulo for Optimized Spatio-Temporal P...Accumulo Summit 2016: GeoMesa: Using Accumulo for Optimized Spatio-Temporal P...
Accumulo Summit 2016: GeoMesa: Using Accumulo for Optimized Spatio-Temporal P...Accumulo Summit
 
Informix partitioning interval_rolling_window_table
Informix partitioning interval_rolling_window_tableInformix partitioning interval_rolling_window_table
Informix partitioning interval_rolling_window_tableKeshav Murthy
 
Redshift Chartio Event Presentation
Redshift Chartio Event PresentationRedshift Chartio Event Presentation
Redshift Chartio Event PresentationChartio
 
Hadoop secondary sort and a custom comparator
Hadoop secondary sort and a custom comparatorHadoop secondary sort and a custom comparator
Hadoop secondary sort and a custom comparatorSubhas Kumar Ghosh
 
HP - Jerome Rolia - Hadoop World 2010
HP - Jerome Rolia - Hadoop World 2010HP - Jerome Rolia - Hadoop World 2010
HP - Jerome Rolia - Hadoop World 2010Cloudera, Inc.
 
Mine scheduling process
Mine scheduling processMine scheduling process
Mine scheduling processVR M
 
Teradata Tutorial for Beginners
Teradata Tutorial for BeginnersTeradata Tutorial for Beginners
Teradata Tutorial for Beginnersrajkamaltibacademy
 
Optimizing Geospatial Operations with Server-side Programming in HBase and Ac...
Optimizing Geospatial Operations with Server-side Programming in HBase and Ac...Optimizing Geospatial Operations with Server-side Programming in HBase and Ac...
Optimizing Geospatial Operations with Server-side Programming in HBase and Ac...DataWorks Summit
 
2017 PLSC Track: Using a Standard Version of ArcMap with External VRS Recieve...
2017 PLSC Track: Using a Standard Version of ArcMap with External VRS Recieve...2017 PLSC Track: Using a Standard Version of ArcMap with External VRS Recieve...
2017 PLSC Track: Using a Standard Version of ArcMap with External VRS Recieve...GIS in the Rockies
 
TCC14 tour hague optimising workbooks
TCC14 tour hague optimising workbooksTCC14 tour hague optimising workbooks
TCC14 tour hague optimising workbooksMrunal Shridhar
 
Sharing historical maps and atlases in web apps
Sharing historical maps and atlases in web appsSharing historical maps and atlases in web apps
Sharing historical maps and atlases in web appsAileen Buckley
 
K ingoldsby
K ingoldsbyK ingoldsby
K ingoldsbyNASAPMC
 

Similar to Working with space time data in ArcGIS (20)

Methods for analyzing and mapping temporal data
Methods for analyzing and mapping temporal dataMethods for analyzing and mapping temporal data
Methods for analyzing and mapping temporal data
 
Methods for Mapping Temporal Data
Methods for Mapping Temporal DataMethods for Mapping Temporal Data
Methods for Mapping Temporal Data
 
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
 
BREEZE 3D Analyst for the Advanced AERMOD Modeler
BREEZE 3D Analyst for the Advanced AERMOD ModelerBREEZE 3D Analyst for the Advanced AERMOD Modeler
BREEZE 3D Analyst for the Advanced AERMOD Modeler
 
Survey On Temporal Data And Change Management in Data Warehouses
Survey On Temporal Data And Change Management in Data WarehousesSurvey On Temporal Data And Change Management in Data Warehouses
Survey On Temporal Data And Change Management in Data Warehouses
 
GRASS as a Temporal GIS - Sören Gebbert
GRASS as a Temporal GIS - Sören GebbertGRASS as a Temporal GIS - Sören Gebbert
GRASS as a Temporal GIS - Sören Gebbert
 
Accumulo Summit 2016: GeoMesa: Using Accumulo for Optimized Spatio-Temporal P...
Accumulo Summit 2016: GeoMesa: Using Accumulo for Optimized Spatio-Temporal P...Accumulo Summit 2016: GeoMesa: Using Accumulo for Optimized Spatio-Temporal P...
Accumulo Summit 2016: GeoMesa: Using Accumulo for Optimized Spatio-Temporal P...
 
Informix partitioning interval_rolling_window_table
Informix partitioning interval_rolling_window_tableInformix partitioning interval_rolling_window_table
Informix partitioning interval_rolling_window_table
 
Redshift Chartio Event Presentation
Redshift Chartio Event PresentationRedshift Chartio Event Presentation
Redshift Chartio Event Presentation
 
Hadoop secondary sort and a custom comparator
Hadoop secondary sort and a custom comparatorHadoop secondary sort and a custom comparator
Hadoop secondary sort and a custom comparator
 
HP - Jerome Rolia - Hadoop World 2010
HP - Jerome Rolia - Hadoop World 2010HP - Jerome Rolia - Hadoop World 2010
HP - Jerome Rolia - Hadoop World 2010
 
Mine scheduling process
Mine scheduling processMine scheduling process
Mine scheduling process
 
Teradata Tutorial for Beginners
Teradata Tutorial for BeginnersTeradata Tutorial for Beginners
Teradata Tutorial for Beginners
 
Optimizing Geospatial Operations with Server-side Programming in HBase and Ac...
Optimizing Geospatial Operations with Server-side Programming in HBase and Ac...Optimizing Geospatial Operations with Server-side Programming in HBase and Ac...
Optimizing Geospatial Operations with Server-side Programming in HBase and Ac...
 
2017 PLSC Track: Using a Standard Version of ArcMap with External VRS Recieve...
2017 PLSC Track: Using a Standard Version of ArcMap with External VRS Recieve...2017 PLSC Track: Using a Standard Version of ArcMap with External VRS Recieve...
2017 PLSC Track: Using a Standard Version of ArcMap with External VRS Recieve...
 
Pig Experience
Pig ExperiencePig Experience
Pig Experience
 
TCC14 tour hague optimising workbooks
TCC14 tour hague optimising workbooksTCC14 tour hague optimising workbooks
TCC14 tour hague optimising workbooks
 
Sharing historical maps and atlases in web apps
Sharing historical maps and atlases in web appsSharing historical maps and atlases in web apps
Sharing historical maps and atlases in web apps
 
CLIM: Transition Workshop - Optimization Methods in Remote Sensing - Jessica...
CLIM: Transition Workshop - Optimization Methods in Remote Sensing  - Jessica...CLIM: Transition Workshop - Optimization Methods in Remote Sensing  - Jessica...
CLIM: Transition Workshop - Optimization Methods in Remote Sensing - Jessica...
 
K ingoldsby
K ingoldsbyK ingoldsby
K ingoldsby
 

More from Aileen Buckley

Vector tile style editor workshop
Vector tile style editor workshopVector tile style editor workshop
Vector tile style editor workshopAileen Buckley
 
Making the most of raster data from the arcgis living atlas of the world
Making the most of raster data from the arcgis living atlas of the worldMaking the most of raster data from the arcgis living atlas of the world
Making the most of raster data from the arcgis living atlas of the worldAileen Buckley
 
Ten telltale signs of novice cartography
Ten telltale signs of novice cartographyTen telltale signs of novice cartography
Ten telltale signs of novice cartographyAileen Buckley
 
Making the most of raster data from the arcgis living atlas of the world
Making the most of raster data from the arcgis living atlas of the worldMaking the most of raster data from the arcgis living atlas of the world
Making the most of raster data from the arcgis living atlas of the worldAileen Buckley
 
Compelling cartography with ArcGIS pro
Compelling cartography with ArcGIS proCompelling cartography with ArcGIS pro
Compelling cartography with ArcGIS proAileen Buckley
 
Atlas apps for online map collections
Atlas apps for online map collectionsAtlas apps for online map collections
Atlas apps for online map collectionsAileen Buckley
 
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018Aileen Buckley
 
Communicating spatial information visually
Communicating spatial information visuallyCommunicating spatial information visually
Communicating spatial information visuallyAileen Buckley
 
Learning to leverage the living atlas
Learning to leverage the living atlasLearning to leverage the living atlas
Learning to leverage the living atlasAileen Buckley
 
Atlas mapping in the hybrid age
Atlas mapping in the hybrid ageAtlas mapping in the hybrid age
Atlas mapping in the hybrid ageAileen Buckley
 
Compelling Cartography with ArcGIS
Compelling Cartography with ArcGISCompelling Cartography with ArcGIS
Compelling Cartography with ArcGISAileen Buckley
 
Maps that Engage, Inform, and Inspire
Maps that Engage, Inform, and InspireMaps that Engage, Inform, and Inspire
Maps that Engage, Inform, and InspireAileen Buckley
 
Atlas Mapping in the Hybrid Age
Atlas Mapping in the Hybrid AgeAtlas Mapping in the Hybrid Age
Atlas Mapping in the Hybrid AgeAileen Buckley
 
Dealing with incomplete data for mapping and spatial analysis
Dealing with incomplete data for mapping and spatial analysisDealing with incomplete data for mapping and spatial analysis
Dealing with incomplete data for mapping and spatial analysisAileen Buckley
 
Methods for mapping temporal data
Methods for mapping temporal dataMethods for mapping temporal data
Methods for mapping temporal dataAileen Buckley
 
Designing more engaging_maps_buckley_11-13-14
Designing more engaging_maps_buckley_11-13-14Designing more engaging_maps_buckley_11-13-14
Designing more engaging_maps_buckley_11-13-14Aileen Buckley
 

More from Aileen Buckley (20)

Vector tile style editor workshop
Vector tile style editor workshopVector tile style editor workshop
Vector tile style editor workshop
 
Making the most of raster data from the arcgis living atlas of the world
Making the most of raster data from the arcgis living atlas of the worldMaking the most of raster data from the arcgis living atlas of the world
Making the most of raster data from the arcgis living atlas of the world
 
Ten telltale signs of novice cartography
Ten telltale signs of novice cartographyTen telltale signs of novice cartography
Ten telltale signs of novice cartography
 
Making the most of raster data from the arcgis living atlas of the world
Making the most of raster data from the arcgis living atlas of the worldMaking the most of raster data from the arcgis living atlas of the world
Making the most of raster data from the arcgis living atlas of the world
 
Creative cartography
Creative cartographyCreative cartography
Creative cartography
 
Compelling cartography with ArcGIS pro
Compelling cartography with ArcGIS proCompelling cartography with ArcGIS pro
Compelling cartography with ArcGIS pro
 
Atlas apps for online map collections
Atlas apps for online map collectionsAtlas apps for online map collections
Atlas apps for online map collections
 
Geocart workshop
Geocart workshopGeocart workshop
Geocart workshop
 
Science at Esri
Science at EsriScience at Esri
Science at Esri
 
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018
 
Communicating spatial information visually
Communicating spatial information visuallyCommunicating spatial information visually
Communicating spatial information visually
 
Learning to leverage the living atlas
Learning to leverage the living atlasLearning to leverage the living atlas
Learning to leverage the living atlas
 
Atlas mapping in the hybrid age
Atlas mapping in the hybrid ageAtlas mapping in the hybrid age
Atlas mapping in the hybrid age
 
Compelling Cartography with ArcGIS
Compelling Cartography with ArcGISCompelling Cartography with ArcGIS
Compelling Cartography with ArcGIS
 
Maps that Engage, Inform, and Inspire
Maps that Engage, Inform, and InspireMaps that Engage, Inform, and Inspire
Maps that Engage, Inform, and Inspire
 
Atlas Mapping in the Hybrid Age
Atlas Mapping in the Hybrid AgeAtlas Mapping in the Hybrid Age
Atlas Mapping in the Hybrid Age
 
Dealing with incomplete data for mapping and spatial analysis
Dealing with incomplete data for mapping and spatial analysisDealing with incomplete data for mapping and spatial analysis
Dealing with incomplete data for mapping and spatial analysis
 
Methods for mapping temporal data
Methods for mapping temporal dataMethods for mapping temporal data
Methods for mapping temporal data
 
Mapping Flow Data
Mapping Flow DataMapping Flow Data
Mapping Flow Data
 
Designing more engaging_maps_buckley_11-13-14
Designing more engaging_maps_buckley_11-13-14Designing more engaging_maps_buckley_11-13-14
Designing more engaging_maps_buckley_11-13-14
 

Recently uploaded

Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Paola De la Torre
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...gurkirankumar98700
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 

Recently uploaded (20)

Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 

Working with space time data in ArcGIS

  • 1. Working with Space-time Data in ArcGIS Aileen Buckley
  • 2. This session • Visualizing temporal data • Managing temporal data • Analyzing temporal data • Best practices are highlighted with a check mark • • ArcUser article – Spring 2018 has a high-level overview 
  • 5. Storing temporal data • Store time stamps in a date field - A field type that stores dates, times, or dates and times - Especially for large data - Most efficient format for query / display performance - Supports more sophisticated database queries  ArcGIS supports a wide range of standard formats and custom formats
  • 6. Storing range data • Store data in a string or numeric fields • For data that is not in “time” format - Sequence, attributes, etc... - Millions of years ago (ArcGIS display date covers AD100 to AD10,000) • Use numeric data in integer or float format • Set aliases for values - 3rd day of week (value = 3), set alias to “Tuesday” - 20 MYA (value = 20), set alias to [field] & “MYA”  • Ecological Marine Units – water depth
  • 7. As separate features in a single feature class • When the shape and/or location of each feature changes over time • Store separate features Moving features Features that move over space • Airplanes, boats, vehicles • People, animals • Storm centers Discrete events Events that happen at specific locations • Crimes • Accidents • Earthquakes, lightening strikes, volcanic events Change / growth Features that change condition over time • Demographics • Fire perimeters • Flood extents
  • 8. As features joined to a table • When the shape and/or location of each feature is constant, but attribute values change over time • Store the changing attributes in a separate (one-to-many) joined table Stationary recorders Features that stay in one place and record changes • Weather stations • Traffic sensors • Stream gauges
  • 9. Index temporal and range data • Index the date or range field for faster query performance • Indexes are a property of the data that you can set in Catalog • Use the Add Attribute Index GP tool to add an index to a field 
  • 10. When temporal data is not in date format • Convert data to a date field type • Use the Convert Time Field GP tool - Converts Text/Number fields into a new Date field - “July 09, 2016”  07/09/2016  MM/DD/YYYY - Does not require you to add a new field first The Date field can have a custom format (e.g., MM dd, yyyy HH:mm:ss) Change Output Time Type to Text to do this Convert the custom date to a real date in a date field  
  • 11. When point in time data needs to be duration data • Create an end time from the exiting start times • Use the Calculate End Time GP tool - Populates an end time field with the next record’s start time - The last record will have - For the last value, manually edit the end time if you know what it should be  blah, blah, blah 
  • 12. When temporal data is stored in multiple columns • Store temporal data in a row format (one time stamp per row) - That is, duplicate features with different time stamps in separate rows • Use the Transpose Fields GP tool - Shifts data stored in columns into data stored in rows 
  • 13. When temporal data is in a separate table • Create a one-to-one, many-to-one, or one-to-many join • Use the Add Join GP tool • For some GP tools, you will need to export the features (Copy Features GP tool) to create a new feature class - For example, Spatial Statistics and Space-Time Pattern Mining tools  
  • 14. When temporal data is in different time zones • ArcGIS integrates data in different layers across different time zones - Layers in map can be in different time zones • Convert data for same layer into the same time zone • Use the Convert Time Zone GP tool - Converts time values recorded in a date field from one time zone to another time zone  New York Los Angeles
  • 15. When temporal data is stored in daylight savings time • Store temporal data in standard time - Multiple problems with DST - Storing the time values in standard time prevents loss or overlaps of data - Prevents ambiguity in visualizations and errors in analyses • Use the Calculate GP tool to add or subtract time • Standardize on UTC1 or GMT2 - 1 Coordinated Universal Time - 2 Greenwich Mean Time  • Regional differences • Rules and boundaries change frequently • Some DST zones adjust less than an hour • 30 minute offset • 45 minute offset 
  • 16. When you need to add or subtract from temporal data • Use the Calculate GP tool to add or subtract time - Python 3: !timefield!.replace(year=!timefield!.year + 10) - Arcade: DateAdd( date, addValue, units) https://docs.python.org/3/library/datetime.html https://developers.arcgis.com/arcade/function-reference/date_functions/ 
  • 18. Space-time data analysis in ArcGIS – 3 ways it is done
  • 19. 1st approach • Data for each time step is analyzed separately • Results are presented as a single time enabled layer or a set of layers, one for each time step
  • 20. 1st approach – 3 ways it is done 1. All GP tools honor the time settings - For time-enabled layers, only those features that fall temporally within the time extent set in the time slider will be processed - Similar to a selection or definition query - Results are then displayed as a single layer on a map
  • 21. 1st approach – 3 ways it is done 1. All GP tools honor the time settings 2. With some GP tools, the time attribute can be used as the case field to repeat the analysis for each time step - The case field is used to calculate statistics separately for each unique attribute value (that is, each time step) - Results are presented as a single layer, and time can be enabled on the case field in the output feature class - e.g., Mean Center and Directional Distribution
  • 22. 1st approach – 3 ways it is done 1. All GP tools honor the time settings 2. With some GP tools, the time attribute can be used as the case field to repeat the analysis for each time step 3. A model or script with an iterator to specifies that the analysis should be repeated for each time step - Results are presented as separate outputs, one for each time step - The separate outputs can then be combined so that the final result can be time enabled and visualized using the time slider control - For feature classes, use Merge or Append - For rasters, use a mosaic dataset
  • 23. 2nd approach • Data is analyzed using space-time constraints • Results are presented as a single layer
  • 24. The second approach • Certain GP tools use a spatial weights matrix • This defines feature relationships in terms of a space-time window - Specified critical distance and fixed interval in time • Tools: - Hot Spot Analysis – creates a map of clusters in both space and time of significantly high (hot) or low (cold) values - Cluster and Outlier Analysis – identifies the similarity (as the spatial clustering of either high or low values) or the dissimilarity (as spatial outliers) of features - Spatially Constrained Multivariate Clustering – finds spatially contiguous clusters of features based on a set of feature attribute values and optional cluster size limits
  • 25. 3rd approach • Data for all time steps is analyzed both spatially and temporally • Results are presented as a single layer or a space-time cube
  • 26. 3rd approach • Space-Time Pattern Mining Tools - Require that the data be in a space-time cube format - Emerging Hot Spot Analysis – identifies hot and cold spots and determines if they have a temporal trend (new versus persistent versus historic hot spots) - Local Outlier Analysis – determines if features have belonged in clusters or been outliers over the entire time extent of the dataset - Time Series Clustering – partitions a collection of time series, stored in a space-time cube, based on the similarity of time series characteristics
  • 27.
  • 28.
  • 29.
  • 30. 3rd approach • Space-Time Pattern Mining Tools - Utilities - Working with space-time cubes - Create the cube from points or polygons - Visualize the cube in 2D or 3D - Fill Missing Values (Utilities toolset) – replaces missing values with estimates based on spatial neighbors, space-time neighbors, or time-series values • Space Time Cube Explorer Add-In can also be used for visualization
  • 31. More traditional time series analysis • Take advantage of the extensibility of the ArcGIS platform • Capabilities to easily transfer data to open-source analytical software - Use ArcPy utility functions to analyze the data in Python - For example, FeatureClassToNumPyArray - Use the R-ArcGIS Bridge to analyze the data using R

Editor's Notes

  1. We hope you will agree that using the Living Atlas data can help make your raster analysis faster and easier than ever before.