SlideShare a Scribd company logo
1 of 36
Visual exploration and analysis of time series
earthquake data
Amit Chourasia1, Keith Richards-Dinger2, James Dietrich2 & Yifeng Cui1
1 SDSC/UC San Diego, 2 UC Riverside
PEARC 17, New Orleans, LA, Jul 12, 2017
RSQSim visualization : Amit Chourasia, SDSC
Presentation overview
• Background
• Challenges
• Implementation
 User interface design
 Visualization design + interaction
 Sample data analysis
• Conclusions
• Demonstration
Research motivation
Earthquake hazard estimation is an important problem.
Requires long term past records of earthquakes at all scales (magnitude,
space, time)
Problem: We neither have, nor could gather such data from the field
Solution: Generate synthetic records by method based on first principals
to bridge this critical gap of missing data
RSQSim visualization : Amit Chourasia, SDSC
RSQSim visualization : Amit Chourasia, SDSC
RSQsim
Generates synthetic quakes at all scales i.e. magnitude, space and time
Computation
• Code runs on O(1K) cores with O(100K) elements
• Future needs: O(1M) elements
• Memory usage scales as Nelem
2
Scaling and optimization work underway by Dmitry Pekurovsky (SDSC)
RSQSim visualization : Amit Chourasia, SDSC
Output data
Event (12 variables): csv file
Fault geometry (11 variables): csv file
Event action (8 variables + index) : binary files
The data is spread over 11 files (2 ASCII, 9 binary)
Sample data catalog
Size 2.4 GB
Files 2 ascii, 9 binary files
Catalog time duration From 50k to 90k years (40k years)
Number of events 5,970,621
Event variables 12
Number of fault patches 260,051
Fault patch variables 11
Number of event actions 19,127,461
Event action variables 8
RSQSim visualization : Amit Chourasia, SDSC
Early data exploration
• Number of events per year (frequency)
• Max magnitude for each year (maxM)
~ 40,000 records with year, frequency and maxM
RSQSim visualization : Amit Chourasia, SDSC
<500 events per year
Project motivation
• Scientist has a custom 3D interactive vis application
Lacks geographic context
Lacks multiuser capability
• Develop an interactive visualization in 2D with richer geographic
context, ideally usable from a web browser
RSQSim visualization : Amit Chourasia, SDSC
Challenges
Data wrangling
• Normalize index to 1 based
• Normalize variable names
• Translate 3D data to 2D
(triangle patches stored with vertex + 3D rotations)
(rectangle patches stored with center + length, width + 3D rotations)
• Add geographic projections from UTM to EPSG 4236
• Consider using raw data vs database
Visualization
• What to display?
• How to display? Which visualization idioms to use?
• Can it work in a web browser?
RSQSim visualization : Amit Chourasia, SDSC
Visualization goals
• Browse and show event of interest
• Show event chronology
• Show fault patches affected by an event
• Interactive
RSQSim visualization : Amit Chourasia, SDSC
Implementation
Developed a web app using the following
• Python
 Flask framework : server/client brokering
 Folium : map plotting
• JavaScript
 Leaflet : mapping
 C3JS : time series plots
 noUISliderJS : mobile friendly sliders
• HTML
RSQSim visualization : Amit Chourasia, SDSC
Implementation Phase 1
• Build an app with raw data
• Data wrangling
Format switched from ASCII to binary
Need to also support rectangular patch elements stored with different scheme
Problems
Slow start up 20 min (data pickling brought it to 2min)
Considerable memory use
Tedious development/debugging
RSQSim visualization : Amit Chourasia, SDSC
Implementation Phase 2
• Transform data to database
Which database to use?
Develop database schema?
Verify data translation to database
Verify and validate geographic projection transformations
• Retool application to use database
Switch search patterns to queries
Optimize query performance
RSQSim visualization : Amit Chourasia, SDSC
RSQSim visualization : Amit Chourasia, SDSC
Implementation Phase 3
• Develop user interface
Select event by ID ( ~ 6M)
Select event by time ( 40,000 years at second granularity)
Other filters
• Link map display with time series
• UI needs to be mobile friendly
User interface design
RSQSim visualization : Amit Chourasia, SDSC
User interaction
• Selection
• Event
• Time
• Filters
• Magnitude
• Number of patches
• Trail events
• Map
• Layers
• Markers pop-ups
• Link with time series plot
• Time series
• Linked with map plot
• Enable filter constraints
RSQSim visualization : Amit Chourasia, SDSC
UI must be mobile friendly
Visual exploration and analysis of time series earthquake data.
Visualization design + interaction
Visual exploration and analysis of time series earthquake data.
Show location of selected event with ID: 108632
Visual exploration and analysis of time series earthquake data.
5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
Popup info for selected event with ID: 108632
Visual exploration and analysis of time series earthquake data.
5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
Show patches affected by event with ID: 108632
Visual exploration and analysis of time series earthquake data.
5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
Show events previous to event with ID: 108632
Visual exploration and analysis of time series earthquake data.
5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
Show chronology of previous events to event with ID: 108632
Visual exploration and analysis of time series earthquake data.
5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
Show events next to event with ID: 108632
Visual exploration and analysis of time series earthquake data.
5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
Show chronology of next events to event with ID: 108632
Visual exploration and analysis of time series earthquake data.
5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
Visual exploration and analysis of time series earthquake data.
Final map with all layers
5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
Show time series adjacent to selected event (± 100 events)
Visual exploration and analysis of time series earthquake data.
Sample data analysis
Visual exploration and analysis of time series earthquake data.
MRI calculation
MRI = Mean recurrence interval
Average interval a quake occurs on a fault sub section
Faults are divided into named subsections
For all fault sub sections (~2600)
• With nucleation only: 1,785 sec
• With nucleation + participation: 12,613 sec
• Verify and validate calculations
Benchmark hardware
MacPro workstation
2x 2.26 Ghz Quad Core Intel Xeon processor
16 GB memory
Visual exploration and analysis of time series earthquake data.
Conclusions
Visual exploration and analysis of time series earthquake data.
Conclusions
• Transformed data to a database
Normalized data from mixed index in source data
Validated data transformations
Easier data management/sharing, only one file
Scalable, as data does not need to be loaded into memory
Fast and easy querying
• Developed interactive visualization web application
Enables quake exploration in rich geographic context for a large catalog
Can support multiple concurrent users
• Analysis
Calculated mean recurrence interval
With nucleation
With nucleation + participation
Streamlined data management, analysis and visualization process
Visual exploration and analysis of time series earthquake data.
Deliverables to research team
• Translation script to create SQLite database from raw data
• Web based visualization application
• Analysis script and other query examples
• Documentation
Visual exploration and analysis of time series earthquake data.
Potential for research team
• Use less code for analysis (no parsing needed)
• Maintain less error prone code for analysis (queries simplify manual filtering)
• Write output data to SQLite database from simulation instead of ascii+binary files
• Scientists will need to learn writing SQL queries
• Query optimization could be an issue
Visual exploration and analysis of time series earthquake data.
Ancillary findings
• Python modules and framework
Found a bug in Folium with geojson data
• JavaScript libraries
Found a bug in popular heatmapjs plugin for Leaflet
• Responsive user interface design for web is still cumbersome
Visual exploration and analysis of time series earthquake data.
Future work
RSQSim gateway
• Data service
Allow web based querying
Allow download of filtered data
• Light visualizations
Allow spatial filtering
Add heat map
Visual exploration and analysis of time series earthquake data.
Acknowledgments
This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is
supported by National Science Foundation grant number ACI-1053575. The National Science
Foundation grant number EAR-1135455 and Keck Foundation grant number 005590-00001 also
supported this work.
Demonstration
http://vis.sdsc.edu:5555
Visual exploration and analysis of time series earthquake data.

More Related Content

What's hot

CloudClustering: Toward an Iterative Data Processing Pattern on the Cloud
CloudClustering: Toward an Iterative Data Processing Pattern on the CloudCloudClustering: Toward an Iterative Data Processing Pattern on the Cloud
CloudClustering: Toward an Iterative Data Processing Pattern on the Cloud
Ankur Dave
 
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado BlascoDSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
Deltares
 

What's hot (9)

CloudClustering: Toward an Iterative Data Processing Pattern on the Cloud
CloudClustering: Toward an Iterative Data Processing Pattern on the CloudCloudClustering: Toward an Iterative Data Processing Pattern on the Cloud
CloudClustering: Toward an Iterative Data Processing Pattern on the Cloud
 
XGSN: An Open-source Semantic Sensing Middleware for the Web of Things
XGSN: An Open-source Semantic Sensing Middleware for the Web of ThingsXGSN: An Open-source Semantic Sensing Middleware for the Web of Things
XGSN: An Open-source Semantic Sensing Middleware for the Web of Things
 
MongoDB World 2016: The Best IoT Analytics with MongoDB
MongoDB World 2016: The Best IoT Analytics with MongoDBMongoDB World 2016: The Best IoT Analytics with MongoDB
MongoDB World 2016: The Best IoT Analytics with MongoDB
 
Oozma kappa
Oozma kappaOozma kappa
Oozma kappa
 
Real time data driven applications (SQL vs NoSQL databases)
Real time data driven applications (SQL vs NoSQL databases)Real time data driven applications (SQL vs NoSQL databases)
Real time data driven applications (SQL vs NoSQL databases)
 
View, Act, and React: Shaping Business Activity with Analytics, BigData Queri...
View, Act, and React: Shaping Business Activity with Analytics, BigData Queri...View, Act, and React: Shaping Business Activity with Analytics, BigData Queri...
View, Act, and React: Shaping Business Activity with Analytics, BigData Queri...
 
DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...
DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...
DSD-INT 2015 - Fast-data management toolbox using Open Earth building blocks ...
 
DSD-INT 2015 - Fast Tool -Gerrit Hendriksen
DSD-INT 2015 - Fast Tool -Gerrit HendriksenDSD-INT 2015 - Fast Tool -Gerrit Hendriksen
DSD-INT 2015 - Fast Tool -Gerrit Hendriksen
 
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado BlascoDSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
 

Similar to PEARC17: Visual exploration and analysis of time series earthquake data

Crowd sourcing gis for global urban area mapping
Crowd sourcing gis for global urban area mappingCrowd sourcing gis for global urban area mapping
Crowd sourcing gis for global urban area mapping
Hiroyuki Miyazaki
 
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
Dawn Wright
 
MongoDB for Time Series Data: Setting the Stage for Sensor Management
MongoDB for Time Series Data: Setting the Stage for Sensor ManagementMongoDB for Time Series Data: Setting the Stage for Sensor Management
MongoDB for Time Series Data: Setting the Stage for Sensor Management
MongoDB
 
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
Rudolf Husar
 

Similar to PEARC17: Visual exploration and analysis of time series earthquake data (20)

Crowd sourcing gis for global urban area mapping
Crowd sourcing gis for global urban area mappingCrowd sourcing gis for global urban area mapping
Crowd sourcing gis for global urban area mapping
 
Working with HDF and netCDF Data in ArcGIS: Tools and Case Studies
Working with HDF and netCDF Data in ArcGIS: Tools and Case StudiesWorking with HDF and netCDF Data in ArcGIS: Tools and Case Studies
Working with HDF and netCDF Data in ArcGIS: Tools and Case Studies
 
Il tempo vola: rappresentare e manipolare sequenze di eventi e time series co...
Il tempo vola: rappresentare e manipolare sequenze di eventi e time series co...Il tempo vola: rappresentare e manipolare sequenze di eventi e time series co...
Il tempo vola: rappresentare e manipolare sequenze di eventi e time series co...
 
Godiva2 Overview
Godiva2 OverviewGodiva2 Overview
Godiva2 Overview
 
OrientDB - Time Series and Event Sequences - Codemotion Milan 2014
OrientDB - Time Series and Event Sequences - Codemotion Milan 2014OrientDB - Time Series and Event Sequences - Codemotion Milan 2014
OrientDB - Time Series and Event Sequences - Codemotion Milan 2014
 
陸永祥/全球網路攝影機帶來的機會與挑戰
陸永祥/全球網路攝影機帶來的機會與挑戰陸永祥/全球網路攝影機帶來的機會與挑戰
陸永祥/全球網路攝影機帶來的機會與挑戰
 
Better Information Faster: Programming the Continuum
Better Information Faster: Programming the ContinuumBetter Information Faster: Programming the Continuum
Better Information Faster: Programming the Continuum
 
Big Data Visualization
Big Data VisualizationBig Data Visualization
Big Data Visualization
 
Introduction to Google Earth Engine .pptx
Introduction to Google Earth Engine .pptxIntroduction to Google Earth Engine .pptx
Introduction to Google Earth Engine .pptx
 
Visualization and Level-of-detail of Metadata for Interactive Exploration of ...
Visualization and Level-of-detail of Metadata for Interactive Exploration of ...Visualization and Level-of-detail of Metadata for Interactive Exploration of ...
Visualization and Level-of-detail of Metadata for Interactive Exploration of ...
 
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
Feature Geo Analytics and Big Data Processing: Hybrid Approaches for Earth Sc...
 
MongoDB for Time Series Data: Setting the Stage for Sensor Management
MongoDB for Time Series Data: Setting the Stage for Sensor ManagementMongoDB for Time Series Data: Setting the Stage for Sensor Management
MongoDB for Time Series Data: Setting the Stage for Sensor Management
 
Graphical display of statistical data on Android
Graphical display of statistical data on AndroidGraphical display of statistical data on Android
Graphical display of statistical data on Android
 
MongoDB for Time Series Data
MongoDB for Time Series DataMongoDB for Time Series Data
MongoDB for Time Series Data
 
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
 
0603 Esip Fed Wash Dc Tech Pres 060103 Esip Aq Tech Track
0603 Esip Fed Wash Dc Tech Pres 060103 Esip Aq Tech Track0603 Esip Fed Wash Dc Tech Pres 060103 Esip Aq Tech Track
0603 Esip Fed Wash Dc Tech Pres 060103 Esip Aq Tech Track
 
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
 
Re-Engineering PostgreSQL as a Time-Series Database
Re-Engineering PostgreSQL as a Time-Series DatabaseRe-Engineering PostgreSQL as a Time-Series Database
Re-Engineering PostgreSQL as a Time-Series Database
 
Performance of standardized web map servers for remote sensing Imagery
Performance of standardized web map servers for remote sensing ImageryPerformance of standardized web map servers for remote sensing Imagery
Performance of standardized web map servers for remote sensing Imagery
 
Kafka Migration for Satellite Event Streaming Data | Eric Velte, ASRC Federal
Kafka Migration for Satellite Event Streaming Data | Eric Velte, ASRC FederalKafka Migration for Satellite Event Streaming Data | Eric Velte, ASRC Federal
Kafka Migration for Satellite Event Streaming Data | Eric Velte, ASRC Federal
 

Recently uploaded

Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Choreo: Empowering the Future of Enterprise Software Engineering
Choreo: Empowering the Future of Enterprise Software EngineeringChoreo: Empowering the Future of Enterprise Software Engineering
Choreo: Empowering the Future of Enterprise Software Engineering
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 

PEARC17: Visual exploration and analysis of time series earthquake data

  • 1. Visual exploration and analysis of time series earthquake data Amit Chourasia1, Keith Richards-Dinger2, James Dietrich2 & Yifeng Cui1 1 SDSC/UC San Diego, 2 UC Riverside PEARC 17, New Orleans, LA, Jul 12, 2017
  • 2. RSQSim visualization : Amit Chourasia, SDSC Presentation overview • Background • Challenges • Implementation  User interface design  Visualization design + interaction  Sample data analysis • Conclusions • Demonstration
  • 3. Research motivation Earthquake hazard estimation is an important problem. Requires long term past records of earthquakes at all scales (magnitude, space, time) Problem: We neither have, nor could gather such data from the field Solution: Generate synthetic records by method based on first principals to bridge this critical gap of missing data RSQSim visualization : Amit Chourasia, SDSC
  • 4. RSQSim visualization : Amit Chourasia, SDSC RSQsim Generates synthetic quakes at all scales i.e. magnitude, space and time Computation • Code runs on O(1K) cores with O(100K) elements • Future needs: O(1M) elements • Memory usage scales as Nelem 2 Scaling and optimization work underway by Dmitry Pekurovsky (SDSC)
  • 5. RSQSim visualization : Amit Chourasia, SDSC Output data Event (12 variables): csv file Fault geometry (11 variables): csv file Event action (8 variables + index) : binary files The data is spread over 11 files (2 ASCII, 9 binary) Sample data catalog Size 2.4 GB Files 2 ascii, 9 binary files Catalog time duration From 50k to 90k years (40k years) Number of events 5,970,621 Event variables 12 Number of fault patches 260,051 Fault patch variables 11 Number of event actions 19,127,461 Event action variables 8
  • 6. RSQSim visualization : Amit Chourasia, SDSC Early data exploration • Number of events per year (frequency) • Max magnitude for each year (maxM) ~ 40,000 records with year, frequency and maxM
  • 7. RSQSim visualization : Amit Chourasia, SDSC <500 events per year
  • 8. Project motivation • Scientist has a custom 3D interactive vis application Lacks geographic context Lacks multiuser capability • Develop an interactive visualization in 2D with richer geographic context, ideally usable from a web browser RSQSim visualization : Amit Chourasia, SDSC
  • 9. Challenges Data wrangling • Normalize index to 1 based • Normalize variable names • Translate 3D data to 2D (triangle patches stored with vertex + 3D rotations) (rectangle patches stored with center + length, width + 3D rotations) • Add geographic projections from UTM to EPSG 4236 • Consider using raw data vs database Visualization • What to display? • How to display? Which visualization idioms to use? • Can it work in a web browser? RSQSim visualization : Amit Chourasia, SDSC
  • 10. Visualization goals • Browse and show event of interest • Show event chronology • Show fault patches affected by an event • Interactive RSQSim visualization : Amit Chourasia, SDSC
  • 11. Implementation Developed a web app using the following • Python  Flask framework : server/client brokering  Folium : map plotting • JavaScript  Leaflet : mapping  C3JS : time series plots  noUISliderJS : mobile friendly sliders • HTML RSQSim visualization : Amit Chourasia, SDSC
  • 12. Implementation Phase 1 • Build an app with raw data • Data wrangling Format switched from ASCII to binary Need to also support rectangular patch elements stored with different scheme Problems Slow start up 20 min (data pickling brought it to 2min) Considerable memory use Tedious development/debugging RSQSim visualization : Amit Chourasia, SDSC
  • 13. Implementation Phase 2 • Transform data to database Which database to use? Develop database schema? Verify data translation to database Verify and validate geographic projection transformations • Retool application to use database Switch search patterns to queries Optimize query performance RSQSim visualization : Amit Chourasia, SDSC
  • 14. RSQSim visualization : Amit Chourasia, SDSC Implementation Phase 3 • Develop user interface Select event by ID ( ~ 6M) Select event by time ( 40,000 years at second granularity) Other filters • Link map display with time series • UI needs to be mobile friendly
  • 15. User interface design RSQSim visualization : Amit Chourasia, SDSC
  • 16. User interaction • Selection • Event • Time • Filters • Magnitude • Number of patches • Trail events • Map • Layers • Markers pop-ups • Link with time series plot • Time series • Linked with map plot • Enable filter constraints RSQSim visualization : Amit Chourasia, SDSC
  • 17. UI must be mobile friendly Visual exploration and analysis of time series earthquake data.
  • 18. Visualization design + interaction Visual exploration and analysis of time series earthquake data.
  • 19. Show location of selected event with ID: 108632 Visual exploration and analysis of time series earthquake data. 5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
  • 20. Popup info for selected event with ID: 108632 Visual exploration and analysis of time series earthquake data. 5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
  • 21. Show patches affected by event with ID: 108632 Visual exploration and analysis of time series earthquake data. 5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
  • 22. Show events previous to event with ID: 108632 Visual exploration and analysis of time series earthquake data. 5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
  • 23. Show chronology of previous events to event with ID: 108632 Visual exploration and analysis of time series earthquake data. 5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
  • 24. Show events next to event with ID: 108632 Visual exploration and analysis of time series earthquake data. 5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
  • 25. Show chronology of next events to event with ID: 108632 Visual exploration and analysis of time series earthquake data. 5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
  • 26. Visual exploration and analysis of time series earthquake data. Final map with all layers 5 ≥ M < 6 6 ≥ M < 7 M ≥ 74 ≥ M < 5M < 4
  • 27. Show time series adjacent to selected event (± 100 events) Visual exploration and analysis of time series earthquake data.
  • 28. Sample data analysis Visual exploration and analysis of time series earthquake data.
  • 29. MRI calculation MRI = Mean recurrence interval Average interval a quake occurs on a fault sub section Faults are divided into named subsections For all fault sub sections (~2600) • With nucleation only: 1,785 sec • With nucleation + participation: 12,613 sec • Verify and validate calculations Benchmark hardware MacPro workstation 2x 2.26 Ghz Quad Core Intel Xeon processor 16 GB memory Visual exploration and analysis of time series earthquake data.
  • 30. Conclusions Visual exploration and analysis of time series earthquake data.
  • 31. Conclusions • Transformed data to a database Normalized data from mixed index in source data Validated data transformations Easier data management/sharing, only one file Scalable, as data does not need to be loaded into memory Fast and easy querying • Developed interactive visualization web application Enables quake exploration in rich geographic context for a large catalog Can support multiple concurrent users • Analysis Calculated mean recurrence interval With nucleation With nucleation + participation Streamlined data management, analysis and visualization process Visual exploration and analysis of time series earthquake data.
  • 32. Deliverables to research team • Translation script to create SQLite database from raw data • Web based visualization application • Analysis script and other query examples • Documentation Visual exploration and analysis of time series earthquake data.
  • 33. Potential for research team • Use less code for analysis (no parsing needed) • Maintain less error prone code for analysis (queries simplify manual filtering) • Write output data to SQLite database from simulation instead of ascii+binary files • Scientists will need to learn writing SQL queries • Query optimization could be an issue Visual exploration and analysis of time series earthquake data.
  • 34. Ancillary findings • Python modules and framework Found a bug in Folium with geojson data • JavaScript libraries Found a bug in popular heatmapjs plugin for Leaflet • Responsive user interface design for web is still cumbersome Visual exploration and analysis of time series earthquake data.
  • 35. Future work RSQSim gateway • Data service Allow web based querying Allow download of filtered data • Light visualizations Allow spatial filtering Add heat map Visual exploration and analysis of time series earthquake data.
  • 36. Acknowledgments This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1053575. The National Science Foundation grant number EAR-1135455 and Keck Foundation grant number 005590-00001 also supported this work. Demonstration http://vis.sdsc.edu:5555 Visual exploration and analysis of time series earthquake data.

Editor's Notes

  1. RSQSim incorporates rate-state constitutive properties. Rate State Quotient? RSQSim has unique capabilities to deterministically model short-term clustering together with long-term statistical properties of earthquakes [1, 2]; and to represent the different modes of slip observed in nature. Can perform repeated simulations of long earthquake catalogs (100K to 10M events) with outer scales of the dimensions of regional plate boundaries, and sufficiently resolved inner scales to permit detailed simulations of the evolution of system state through occurrence of frequent small earthquakes.
  2. RSQSim incorporates rate-state constitutive properties. Rate State Quotient? RSQSim has unique capabilities to deterministically model short-term clustering together with long-term statistical properties of earthquakes [1, 2]; and to represent the different modes of slip observed in nature. Can perform repeated simulations of long earthquake catalogs (100K to 10M events) with outer scales of the dimensions of regional plate boundaries, and sufficiently resolved inner scales to permit detailed simulations of the evolution of system state through occurrence of frequent small earthquakes.
  3. Data rarely will need to change (written once) Will be read many times for analysis Scientists don’t have DB experience, simpler solution is better DB needs to be portable across platform
  4. Map visual idiom is suitable - As it enables us to show geographic context. Color coded Red polygon shows the event and its location on surface Pop up reveals further information about the event
  5. Map visual idiom is suitable - As it enables us to show geographic context. Color coded Red polygon shows the event and its location on surface Pop up reveals further information about the event
  6. Map visual idiom is suitable - As it enables us to show geographic context. Color coded Red polygon shows the event and its location on surface Pop up reveals further information about the event
  7. Map visual idiom is suitable - As it enables us to show geographic context. Color coded Red polygon shows the event and its location on surface Pop up reveals further information about the event
  8. Map visual idiom is suitable - As it enables us to show geographic context. Color coded Red polygon shows the event and its location on surface Pop up reveals further information about the event
  9. Map visual idiom is suitable - As it enables us to show geographic context. Color coded Red polygon shows the event and its location on surface Pop up reveals further information about the event
  10. Map visual idiom is suitable - As it enables us to show geographic context. Color coded Red polygon shows the event and its location on surface Pop up reveals further information about the event
  11. Nucleation - When quake originates on a sub section Participation – When quake originates elsewhere, but affects a sub section
  12. On web things don’t work as would expect.
  13. Shout out to ECSS