SlideShare a Scribd company logo
1 of 46
Exploring Data Preparation and Visualization
           Tools for Urban Forestry



                  340 N 12th St, Suite 402
                  Philadelphia, PA 19107
                       215.925.2600
                    info@azavea.com
              www.azavea.com/opentreemap
About Us


    Deborah Boyer
    OpenTreeMap Project Manager
    dboyer@azavea.com
    215.701.7506




   Jeremy Heffner
   Product Manager
   jheffner@azavea.com
   215.701.7712
About Azavea

• Founded in 2000
• B Corporation
• 30+ people
• Based in Philadelphia
   – Boston office
• Geospatial + web + mobile
   – Software development
   – Spatial analysis services
   – User experience
Agenda


• The Ideal: Gathering Organized, Perfect Data

• The Reality: Cleaning and Preparing Your Data

• Adding Context

• Exploring, Preparing and Sharing Data Visualizations

• Questions
Gathering Data
An open source tree data management system
for collaborative, geography enabled urban tree inventory
Main Features

• Search and Explore Tree
  Data

• View Ecosystem Benefits

• Add New Trees

• Edit and Update Trees

• Upload Tree Photos

• Track Stewardship Activities
Data Quality Checks

• Remove duplicate
  trees during data
  upload

• Tree watch list

• Drop down lists

• User groups

• Reputation points
Cleaning and Preparing Data
Data Cleaning: Your Questions



• At what point in the data maintenance
  process do you find yourself cleaning data?



• Are there ways that you would like to
  improve the workflow?
Cleaning & Preparing Data

• Making sense of data starts at the point of collection
   – Define what you want to measure / track
      • Clearly define schema and fields
          – Have a shared meaning for values
          – Data validation on entry

   – Collect your data
   – Examine results
      • Are there common mistakes you could prevent?
      • Are there different interpretations of fields?

   – Close the feedback loop & iterate
Cleaning & Preparing Data

• Common data quality issues
   – Combined fields
      • Address: “340 N 12th St, Suite 402 , Philadelphia, PA 19107”

   – Invalid entries
      • ZIP code: 1234 (length check, is number)
      • Age: 204 (reasonable range check, is number)

   – Format variations
      • State: PA vs. Pennsylvania (drop down or scrubbing rules)

   – Duplicates
      • CRM: John Smith with old and new addresses
Cleaning & Preparing Data




    Not a reasonable option
What does this have to do with trees?

• We track things - tree inventories, potential planting
  sites, community groups, people who requested
  trees, etc .

• Data comes from lots of places - web forms,
  collected by various staff, submitted by community
  groups.

• None of it matches.

• Good data makes our lives easier.
Cleaning & Preparing Data

• Tools to clean tabular data
   – Excel (or open source equivalent)
      • Pros:
          – Broad features
          – Widely utilized / common skill
          – Formulas / sorting / flexible

      • Cons:
          – Doesn’t understand record concept
          – Mass changes can be tedious
Cleaning & Preparing Data

• Tools to clean tabular data
   – DataWrangler
      • http://vis.stanford.edu/wrangler/
      • Pros:
          – Focused on transforming data into relational format
          – Live previews

      • Cons:
          – Alpha quality version
          – Data size limits / online tool
          – Can be difficult to figure out what set of transforms are needed
Cleaning & Preparing Data

• Tools to clean tabular data
   – Google Refine
      • http://code.google.com/p/google-refine/
      • Pros:
          – Understands record concept
          – Formulas / Facets
          – Undo capability
          – Windows / Mac / Linux

      • Cons:
          – There is a learning curve
          – Unusual type of app
                » Download, unzip, run exe file, access through browser
Demo
Assembling Data and Building Context
Context: Your Questions

• What challenges have you faced putting your data
  in context?


• Are you struggling to identify what “context” means
  for your organization?


• Do you know what data you’d like to use, but have
  trouble finding it?
Your Data in Context

• Your data is essential!
• But it is more meaningful in context…
   – Ratios & rates
       • Service level
       • Market penetration

   – Indicators & trends
       • How you compare

   – Targeting
       • Key demographics                 Juice Analytics


       • Custom summaries
What does this have to do with trees?



• Trees don’t exist in a vacuum.

• Contextual data = more effective outreach.

• More info gives you new insights.
Making Sense of the Census

• American FactFinder
• http://factfinder2.census.gov
   – Decennial Census
      • Every 10 years
      • Full population survey
      • Just 10 questions

   – American Community Survey (ACS)
      • Monthly sample
      • Aggregated over different time periods (1-, 3- and 5-year)
      • Extremely detailed questions
      • Subject to sampling error
FactFinder Frustrations
Helpers: Social Explorer

• http://www.socialexplorer.com/

• Data Dictionary
   –   Survey
   –   Dataset
   –   Table
   –   Variable
   –   Formula
   –   Population
Helpers: Social Explorer

• Background
  – Key Terms
  – Collection Methodology
  – Uses & applications
Helpers: ACS Alchemist

•   https://github.com/azavea/acs-alchemist 
•   Retrieval of block group-level data
•   Custom variable selection
•   Delivery in spatial data format ready for mapping




This tool was developed by Azavea in collaboration with Jerry Ratcliffe and Ralph Taylor of Temple
University Center for Security and Crime Science. This project was supported by Award No. 2010-DE-BX-
K004, awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice.
Helpers: ACS Alchemist

As easy as 1-2-3
1.Create a document with your selected variables
Helpers: ACS Alchemist

As easy as 1-2-3
1.Create a document with your selected variables
2.Pick your geographies
Helpers: ACS Alchemist

As easy as 1-2-3
1.Create a document with your selected variables
2.Pick your geographies and geolevels
3.Retrieve your shapefiles
Other Sources

• Public data
   – Open Data Portals
      • Federal, state & local data

   – Political Data
      • Voter data
      • Legislative boundaries



• Commercial data
   – Population Projections
   – Consumer Data
Data Visualizations
Data Visualization: Your Questions

• Do you currently share data with your constituents?


• Where do you use data visualizations (e.g. annual
  report, embedded infographics, live data trackers)?


• Do you currently map your data?
What does this have to do with trees?

• Charts, graphs, maps, and photos help us
  tell a story.

• Show that trees are more than just leaves
  and branches.

• Explore the science without making
  people’s eyes glaze over.
Exploring Data

• Visualization tools
   – Tableau
       • http://www.tableausoftware.com/
       • Pros:
           – Flexible interface makes data exploration easy
           – Fast even on large data sets

       • Cons:
           – Easy to visualize something that doesn’t make sense to look at
           – Price (for desktop tool)
Demo
Exploring Data

• Visualization tools
   – GeoCommons (GeoIQ)
       • http://geocommons.com/
       • Pros:
           – Intuitive interface
           – Analysis tools
           – Geocoding for up to 5,000 records
           – Supports KML (Google Maps) import & export

       • Cons:
           – US-only geocoding
Exploring Data

• Desktop GIS: Proprietary
   – Esri ArcGIS
      • Pros:
          – Industry standard
          – Many tools
          – Extensive training materials
          – Customer support

      • Cons:
          – Windows only
          – Potentially expensive *


            *
Exploring Data

• Visualization tools
   – ArcGIS Explorer online
       • http://www.arcgis.com/explorer/
       • Pros:
           – Supports many data formats
           – Online digitizing
           – Integration with other Esri services
           – Presentation view / mobile app

       • Cons:
           – Can’t export geocoded results
           – Geocoding limited to 250 records
Demo
Exploring Data

• Desktop GIS: Open Source
– Quantum GIS (QGIS)
– GRASS
– uDig
         • Pros:
             – Free
             – Multi-platform (Windows, Mac OS, Linux)

         • Cons:
             – Limited functionality (for advanced users)
             – Community-based support
Questions?
Contact Us


     Deborah Boyer
     OpenTreeMap Project Manager
     dboyer@azavea.com
     215.701.7506




    Jeremy Heffner
    Product Manager
    jheffner@azavea.com
    215.701.7712
Exploring Data Preparation and Visualization
           Tools for Urban Forestry



                  340 N 12th St, Suite 402
                  Philadelphia, PA 19107
                       215.925.2600
                    info@azavea.com
              www.azavea.com/opentreemap

More Related Content

What's hot

Growing Your Urban Forest: Using the OpenTreeMap Bulk Uploader
Growing Your Urban Forest: Using the OpenTreeMap Bulk UploaderGrowing Your Urban Forest: Using the OpenTreeMap Bulk Uploader
Growing Your Urban Forest: Using the OpenTreeMap Bulk UploaderAzavea
 
Getting Started with OpenTreeMap Cloud
Getting Started with OpenTreeMap CloudGetting Started with OpenTreeMap Cloud
Getting Started with OpenTreeMap CloudAzavea
 
November 12, 2014 Webinar: Hackers, Beer Geeks, and Arborly Love - Reaching o...
November 12, 2014 Webinar: Hackers, Beer Geeks, and Arborly Love - Reaching o...November 12, 2014 Webinar: Hackers, Beer Geeks, and Arborly Love - Reaching o...
November 12, 2014 Webinar: Hackers, Beer Geeks, and Arborly Love - Reaching o...Azavea
 
Using New Tools to Analyze and Plan Your Urban Forest
Using New Tools to Analyze and Plan Your Urban Forest Using New Tools to Analyze and Plan Your Urban Forest
Using New Tools to Analyze and Plan Your Urban Forest Azavea
 
Sample presentation
Sample presentationSample presentation
Sample presentationcle Thompson
 
Sample presentation
Sample presentationSample presentation
Sample presentationsalazark
 
Custom Tools for Urban Forestry Nonprofits and Outreach July 12, 2016
Custom Tools for Urban Forestry Nonprofits and Outreach July 12, 2016Custom Tools for Urban Forestry Nonprofits and Outreach July 12, 2016
Custom Tools for Urban Forestry Nonprofits and Outreach July 12, 2016Chris Peiffer
 
Libby Hemphill, "Elected Officials and Social Media"
Libby Hemphill, "Elected Officials and Social Media"Libby Hemphill, "Elected Officials and Social Media"
Libby Hemphill, "Elected Officials and Social Media"summersocialwebshop
 

What's hot (10)

Growing Your Urban Forest: Using the OpenTreeMap Bulk Uploader
Growing Your Urban Forest: Using the OpenTreeMap Bulk UploaderGrowing Your Urban Forest: Using the OpenTreeMap Bulk Uploader
Growing Your Urban Forest: Using the OpenTreeMap Bulk Uploader
 
Getting Started with OpenTreeMap Cloud
Getting Started with OpenTreeMap CloudGetting Started with OpenTreeMap Cloud
Getting Started with OpenTreeMap Cloud
 
November 12, 2014 Webinar: Hackers, Beer Geeks, and Arborly Love - Reaching o...
November 12, 2014 Webinar: Hackers, Beer Geeks, and Arborly Love - Reaching o...November 12, 2014 Webinar: Hackers, Beer Geeks, and Arborly Love - Reaching o...
November 12, 2014 Webinar: Hackers, Beer Geeks, and Arborly Love - Reaching o...
 
Using New Tools to Analyze and Plan Your Urban Forest
Using New Tools to Analyze and Plan Your Urban Forest Using New Tools to Analyze and Plan Your Urban Forest
Using New Tools to Analyze and Plan Your Urban Forest
 
Sample presentation
Sample presentationSample presentation
Sample presentation
 
Sample presentation
Sample presentationSample presentation
Sample presentation
 
Custom Tools for Urban Forestry Nonprofits and Outreach July 12, 2016
Custom Tools for Urban Forestry Nonprofits and Outreach July 12, 2016Custom Tools for Urban Forestry Nonprofits and Outreach July 12, 2016
Custom Tools for Urban Forestry Nonprofits and Outreach July 12, 2016
 
CAPS GUIDELINES
CAPS GUIDELINESCAPS GUIDELINES
CAPS GUIDELINES
 
Lee rainie
Lee rainieLee rainie
Lee rainie
 
Libby Hemphill, "Elected Officials and Social Media"
Libby Hemphill, "Elected Officials and Social Media"Libby Hemphill, "Elected Officials and Social Media"
Libby Hemphill, "Elected Officials and Social Media"
 

Viewers also liked

Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist SoftServe
 
Reinventing the Modern Information Pipeline: Paxata and MapR
Reinventing the Modern Information Pipeline: Paxata and MapRReinventing the Modern Information Pipeline: Paxata and MapR
Reinventing the Modern Information Pipeline: Paxata and MapRLilia Gutnik
 
Business Research Methods. data collection preparation and analysis
Business Research Methods. data collection preparation and analysisBusiness Research Methods. data collection preparation and analysis
Business Research Methods. data collection preparation and analysisAhsan Khan Eco (Superior College)
 
Data Preparation vs. Inline Data Wrangling in Data Science and Machine Learning
Data Preparation vs. Inline Data Wrangling in Data Science and Machine LearningData Preparation vs. Inline Data Wrangling in Data Science and Machine Learning
Data Preparation vs. Inline Data Wrangling in Data Science and Machine LearningKai Wähner
 
Data Preparation and Processing
Data Preparation and ProcessingData Preparation and Processing
Data Preparation and ProcessingMehul Gondaliya
 

Viewers also liked (6)

Data Preparation for Data Science
Data Preparation for Data ScienceData Preparation for Data Science
Data Preparation for Data Science
 
Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist
 
Reinventing the Modern Information Pipeline: Paxata and MapR
Reinventing the Modern Information Pipeline: Paxata and MapRReinventing the Modern Information Pipeline: Paxata and MapR
Reinventing the Modern Information Pipeline: Paxata and MapR
 
Business Research Methods. data collection preparation and analysis
Business Research Methods. data collection preparation and analysisBusiness Research Methods. data collection preparation and analysis
Business Research Methods. data collection preparation and analysis
 
Data Preparation vs. Inline Data Wrangling in Data Science and Machine Learning
Data Preparation vs. Inline Data Wrangling in Data Science and Machine LearningData Preparation vs. Inline Data Wrangling in Data Science and Machine Learning
Data Preparation vs. Inline Data Wrangling in Data Science and Machine Learning
 
Data Preparation and Processing
Data Preparation and ProcessingData Preparation and Processing
Data Preparation and Processing
 

Similar to Exploring Data Preparation and Visualization Tools for Urban Forestry

NTEN Webinar - Data Cleaning and Visualization Tools for Nonprofits
NTEN Webinar - Data Cleaning and Visualization Tools for NonprofitsNTEN Webinar - Data Cleaning and Visualization Tools for Nonprofits
NTEN Webinar - Data Cleaning and Visualization Tools for NonprofitsAzavea
 
Exploring Collaborative Tree Inventory with OpenTreeMap
Exploring Collaborative Tree Inventory with OpenTreeMapExploring Collaborative Tree Inventory with OpenTreeMap
Exploring Collaborative Tree Inventory with OpenTreeMapAzavea
 
LoCloud Geolocation enrichment tools, Siri Slettvag, Asplan Viak Internet (Av...
LoCloud Geolocation enrichment tools, Siri Slettvag, Asplan Viak Internet (Av...LoCloud Geolocation enrichment tools, Siri Slettvag, Asplan Viak Internet (Av...
LoCloud Geolocation enrichment tools, Siri Slettvag, Asplan Viak Internet (Av...locloud
 
Implimenting and Mitigating Change with all of this Newfangled Technology
Implimenting and Mitigating Change with all of this Newfangled TechnologyImplimenting and Mitigating Change with all of this Newfangled Technology
Implimenting and Mitigating Change with all of this Newfangled TechnologyIndiana Online Users Group
 
Open Metrics for Open Repositories at OR2012
Open Metrics for Open Repositories at OR2012Open Metrics for Open Repositories at OR2012
Open Metrics for Open Repositories at OR2012Nick Sheppard
 
Bren - UCSB - Spooky spreadsheets
Bren - UCSB - Spooky spreadsheetsBren - UCSB - Spooky spreadsheets
Bren - UCSB - Spooky spreadsheetsCarly Strasser
 
Changing the Curation Equation: A Data Lifecycle Approach to Lowering Costs a...
Changing the Curation Equation: A Data Lifecycle Approach to Lowering Costs a...Changing the Curation Equation: A Data Lifecycle Approach to Lowering Costs a...
Changing the Curation Equation: A Data Lifecycle Approach to Lowering Costs a...SEAD
 
Software Programs for Data Analysis
Software Programs for Data AnalysisSoftware Programs for Data Analysis
Software Programs for Data Analysisunmgrc
 
Scoping call presentation_rev_4_mary
Scoping call presentation_rev_4_maryScoping call presentation_rev_4_mary
Scoping call presentation_rev_4_maryrhochambeau32
 
A Data Scientist Perspective on Data Curation in the Digital Era
A Data Scientist Perspective on Data Curation in the Digital EraA Data Scientist Perspective on Data Curation in the Digital Era
A Data Scientist Perspective on Data Curation in the Digital EraVicki Ferrini
 
Large scale computing
Large scale computing Large scale computing
Large scale computing Bhupesh Bansal
 
Ordering the chaos: Creating websites with imperfect data
Ordering the chaos: Creating websites with imperfect dataOrdering the chaos: Creating websites with imperfect data
Ordering the chaos: Creating websites with imperfect dataAndy Stretton
 
FAIRDOM data management support for ERACoBioTech Proposals
FAIRDOM data management support for ERACoBioTech ProposalsFAIRDOM data management support for ERACoBioTech Proposals
FAIRDOM data management support for ERACoBioTech ProposalsFAIRDOM
 
Sp meetup 17 slidedeck
Sp meetup 17 slidedeckSp meetup 17 slidedeck
Sp meetup 17 slidedeckRic Centre
 
Open data for_resilience
Open data for_resilienceOpen data for_resilience
Open data for_resilienceDami Sonoiki
 
CTOs Perspective on Adding Geospatial and Location-based Information
CTOs Perspective on Adding Geospatial and Location-based InformationCTOs Perspective on Adding Geospatial and Location-based Information
CTOs Perspective on Adding Geospatial and Location-based InformationBradley Brown
 

Similar to Exploring Data Preparation and Visualization Tools for Urban Forestry (20)

NTEN Webinar - Data Cleaning and Visualization Tools for Nonprofits
NTEN Webinar - Data Cleaning and Visualization Tools for NonprofitsNTEN Webinar - Data Cleaning and Visualization Tools for Nonprofits
NTEN Webinar - Data Cleaning and Visualization Tools for Nonprofits
 
Exploring Collaborative Tree Inventory with OpenTreeMap
Exploring Collaborative Tree Inventory with OpenTreeMapExploring Collaborative Tree Inventory with OpenTreeMap
Exploring Collaborative Tree Inventory with OpenTreeMap
 
DataUp at ACRL 2013
DataUp at ACRL 2013DataUp at ACRL 2013
DataUp at ACRL 2013
 
LoCloud Geolocation enrichment tools, Siri Slettvag, Asplan Viak Internet (Av...
LoCloud Geolocation enrichment tools, Siri Slettvag, Asplan Viak Internet (Av...LoCloud Geolocation enrichment tools, Siri Slettvag, Asplan Viak Internet (Av...
LoCloud Geolocation enrichment tools, Siri Slettvag, Asplan Viak Internet (Av...
 
Implimenting and Mitigating Change with all of this Newfangled Technology
Implimenting and Mitigating Change with all of this Newfangled TechnologyImplimenting and Mitigating Change with all of this Newfangled Technology
Implimenting and Mitigating Change with all of this Newfangled Technology
 
Open Metrics for Open Repositories at OR2012
Open Metrics for Open Repositories at OR2012Open Metrics for Open Repositories at OR2012
Open Metrics for Open Repositories at OR2012
 
Bren - UCSB - Spooky spreadsheets
Bren - UCSB - Spooky spreadsheetsBren - UCSB - Spooky spreadsheets
Bren - UCSB - Spooky spreadsheets
 
Changing the Curation Equation: A Data Lifecycle Approach to Lowering Costs a...
Changing the Curation Equation: A Data Lifecycle Approach to Lowering Costs a...Changing the Curation Equation: A Data Lifecycle Approach to Lowering Costs a...
Changing the Curation Equation: A Data Lifecycle Approach to Lowering Costs a...
 
Software Programs for Data Analysis
Software Programs for Data AnalysisSoftware Programs for Data Analysis
Software Programs for Data Analysis
 
Scoping call presentation_rev_4_mary
Scoping call presentation_rev_4_maryScoping call presentation_rev_4_mary
Scoping call presentation_rev_4_mary
 
A Data Scientist Perspective on Data Curation in the Digital Era
A Data Scientist Perspective on Data Curation in the Digital EraA Data Scientist Perspective on Data Curation in the Digital Era
A Data Scientist Perspective on Data Curation in the Digital Era
 
Hawaii Pacific GIS Conference 2012: LiDAR for Intrastructure and Terrian Mapp...
Hawaii Pacific GIS Conference 2012: LiDAR for Intrastructure and Terrian Mapp...Hawaii Pacific GIS Conference 2012: LiDAR for Intrastructure and Terrian Mapp...
Hawaii Pacific GIS Conference 2012: LiDAR for Intrastructure and Terrian Mapp...
 
About Scanning and Metadata Standards - NEMO 2010
About Scanning and Metadata Standards - NEMO 2010About Scanning and Metadata Standards - NEMO 2010
About Scanning and Metadata Standards - NEMO 2010
 
Open Data Presentation
Open Data PresentationOpen Data Presentation
Open Data Presentation
 
Large scale computing
Large scale computing Large scale computing
Large scale computing
 
Ordering the chaos: Creating websites with imperfect data
Ordering the chaos: Creating websites with imperfect dataOrdering the chaos: Creating websites with imperfect data
Ordering the chaos: Creating websites with imperfect data
 
FAIRDOM data management support for ERACoBioTech Proposals
FAIRDOM data management support for ERACoBioTech ProposalsFAIRDOM data management support for ERACoBioTech Proposals
FAIRDOM data management support for ERACoBioTech Proposals
 
Sp meetup 17 slidedeck
Sp meetup 17 slidedeckSp meetup 17 slidedeck
Sp meetup 17 slidedeck
 
Open data for_resilience
Open data for_resilienceOpen data for_resilience
Open data for_resilience
 
CTOs Perspective on Adding Geospatial and Location-based Information
CTOs Perspective on Adding Geospatial and Location-based InformationCTOs Perspective on Adding Geospatial and Location-based Information
CTOs Perspective on Adding Geospatial and Location-based Information
 

More from Azavea

7 misconceptions about predictive policing webinar
7 misconceptions about predictive policing webinar7 misconceptions about predictive policing webinar
7 misconceptions about predictive policing webinarAzavea
 
Forecasting Space-Time Events - Strata + Hadoop World 2015 San Jose
Forecasting Space-Time Events - Strata + Hadoop World 2015 San JoseForecasting Space-Time Events - Strata + Hadoop World 2015 San Jose
Forecasting Space-Time Events - Strata + Hadoop World 2015 San JoseAzavea
 
HunchLab 2.0 Getting Started
HunchLab 2.0 Getting StartedHunchLab 2.0 Getting Started
HunchLab 2.0 Getting StartedAzavea
 
Is it a Package or a Wrapper? Designing, Documenting, and Distributing a Pyth...
Is it a Package or a Wrapper? Designing, Documenting, and Distributing a Pyth...Is it a Package or a Wrapper? Designing, Documenting, and Distributing a Pyth...
Is it a Package or a Wrapper? Designing, Documenting, and Distributing a Pyth...Azavea
 
Your New Partners: Understanding Civic Hackathons, Why You Should be Involved...
Your New Partners: Understanding Civic Hackathons, Why You Should be Involved...Your New Partners: Understanding Civic Hackathons, Why You Should be Involved...
Your New Partners: Understanding Civic Hackathons, Why You Should be Involved...Azavea
 
HunchLab 2.0 Predictive Missions: Under the Hood
HunchLab 2.0 Predictive Missions: Under the HoodHunchLab 2.0 Predictive Missions: Under the Hood
HunchLab 2.0 Predictive Missions: Under the HoodAzavea
 
HunchLab 2.0 Preview Webinar - Place
HunchLab 2.0 Preview Webinar - PlaceHunchLab 2.0 Preview Webinar - Place
HunchLab 2.0 Preview Webinar - PlaceAzavea
 
Five Technology Trends Every Nonprofit Needs to Know
Five Technology Trends Every Nonprofit Needs to KnowFive Technology Trends Every Nonprofit Needs to Know
Five Technology Trends Every Nonprofit Needs to KnowAzavea
 
Data Philly Meetup - Big (Geo) Data
Data Philly Meetup - Big (Geo) DataData Philly Meetup - Big (Geo) Data
Data Philly Meetup - Big (Geo) DataAzavea
 
Fed Geo Day - Applying GeoTrellis at the US Army Corps
Fed Geo Day - Applying GeoTrellis at the US Army CorpsFed Geo Day - Applying GeoTrellis at the US Army Corps
Fed Geo Day - Applying GeoTrellis at the US Army CorpsAzavea
 
Fed Geo Day - GeoTrellis Intro
Fed Geo Day - GeoTrellis IntroFed Geo Day - GeoTrellis Intro
Fed Geo Day - GeoTrellis IntroAzavea
 
Fed Geo Day 2013 - Azavea Intro
Fed Geo Day 2013 - Azavea Intro Fed Geo Day 2013 - Azavea Intro
Fed Geo Day 2013 - Azavea Intro Azavea
 
Modeling Count-based Raster Data with ArcGIS and R
Modeling Count-based Raster Data with ArcGIS and RModeling Count-based Raster Data with ArcGIS and R
Modeling Count-based Raster Data with ArcGIS and RAzavea
 
How to Conquer your Post-Election Data Chaos with the Cicero API
How to Conquer your Post-Election Data Chaos with the Cicero APIHow to Conquer your Post-Election Data Chaos with the Cicero API
How to Conquer your Post-Election Data Chaos with the Cicero APIAzavea
 
Exploring Mobile Technology with OpenTreeMap Mobile
Exploring Mobile Technology with OpenTreeMap MobileExploring Mobile Technology with OpenTreeMap Mobile
Exploring Mobile Technology with OpenTreeMap MobileAzavea
 
10 Steps to Optimize Your Crime Analysis
10 Steps to Optimize Your Crime Analysis10 Steps to Optimize Your Crime Analysis
10 Steps to Optimize Your Crime AnalysisAzavea
 

More from Azavea (16)

7 misconceptions about predictive policing webinar
7 misconceptions about predictive policing webinar7 misconceptions about predictive policing webinar
7 misconceptions about predictive policing webinar
 
Forecasting Space-Time Events - Strata + Hadoop World 2015 San Jose
Forecasting Space-Time Events - Strata + Hadoop World 2015 San JoseForecasting Space-Time Events - Strata + Hadoop World 2015 San Jose
Forecasting Space-Time Events - Strata + Hadoop World 2015 San Jose
 
HunchLab 2.0 Getting Started
HunchLab 2.0 Getting StartedHunchLab 2.0 Getting Started
HunchLab 2.0 Getting Started
 
Is it a Package or a Wrapper? Designing, Documenting, and Distributing a Pyth...
Is it a Package or a Wrapper? Designing, Documenting, and Distributing a Pyth...Is it a Package or a Wrapper? Designing, Documenting, and Distributing a Pyth...
Is it a Package or a Wrapper? Designing, Documenting, and Distributing a Pyth...
 
Your New Partners: Understanding Civic Hackathons, Why You Should be Involved...
Your New Partners: Understanding Civic Hackathons, Why You Should be Involved...Your New Partners: Understanding Civic Hackathons, Why You Should be Involved...
Your New Partners: Understanding Civic Hackathons, Why You Should be Involved...
 
HunchLab 2.0 Predictive Missions: Under the Hood
HunchLab 2.0 Predictive Missions: Under the HoodHunchLab 2.0 Predictive Missions: Under the Hood
HunchLab 2.0 Predictive Missions: Under the Hood
 
HunchLab 2.0 Preview Webinar - Place
HunchLab 2.0 Preview Webinar - PlaceHunchLab 2.0 Preview Webinar - Place
HunchLab 2.0 Preview Webinar - Place
 
Five Technology Trends Every Nonprofit Needs to Know
Five Technology Trends Every Nonprofit Needs to KnowFive Technology Trends Every Nonprofit Needs to Know
Five Technology Trends Every Nonprofit Needs to Know
 
Data Philly Meetup - Big (Geo) Data
Data Philly Meetup - Big (Geo) DataData Philly Meetup - Big (Geo) Data
Data Philly Meetup - Big (Geo) Data
 
Fed Geo Day - Applying GeoTrellis at the US Army Corps
Fed Geo Day - Applying GeoTrellis at the US Army CorpsFed Geo Day - Applying GeoTrellis at the US Army Corps
Fed Geo Day - Applying GeoTrellis at the US Army Corps
 
Fed Geo Day - GeoTrellis Intro
Fed Geo Day - GeoTrellis IntroFed Geo Day - GeoTrellis Intro
Fed Geo Day - GeoTrellis Intro
 
Fed Geo Day 2013 - Azavea Intro
Fed Geo Day 2013 - Azavea Intro Fed Geo Day 2013 - Azavea Intro
Fed Geo Day 2013 - Azavea Intro
 
Modeling Count-based Raster Data with ArcGIS and R
Modeling Count-based Raster Data with ArcGIS and RModeling Count-based Raster Data with ArcGIS and R
Modeling Count-based Raster Data with ArcGIS and R
 
How to Conquer your Post-Election Data Chaos with the Cicero API
How to Conquer your Post-Election Data Chaos with the Cicero APIHow to Conquer your Post-Election Data Chaos with the Cicero API
How to Conquer your Post-Election Data Chaos with the Cicero API
 
Exploring Mobile Technology with OpenTreeMap Mobile
Exploring Mobile Technology with OpenTreeMap MobileExploring Mobile Technology with OpenTreeMap Mobile
Exploring Mobile Technology with OpenTreeMap Mobile
 
10 Steps to Optimize Your Crime Analysis
10 Steps to Optimize Your Crime Analysis10 Steps to Optimize Your Crime Analysis
10 Steps to Optimize Your Crime Analysis
 

Recently uploaded

Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfFIDO Alliance
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyUXDXConf
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomCzechDreamin
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsStefano
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...FIDO Alliance
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeCzechDreamin
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...CzechDreamin
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessUXDXConf
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...marcuskenyatta275
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераMark Opanasiuk
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandIES VE
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...FIDO Alliance
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekCzechDreamin
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceSamy Fodil
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlPeter Udo Diehl
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FIDO Alliance
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty SecureFemke de Vroome
 

Recently uploaded (20)

Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří Karpíšek
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 

Exploring Data Preparation and Visualization Tools for Urban Forestry

  • 1. Exploring Data Preparation and Visualization Tools for Urban Forestry 340 N 12th St, Suite 402 Philadelphia, PA 19107 215.925.2600 info@azavea.com www.azavea.com/opentreemap
  • 2. About Us Deborah Boyer OpenTreeMap Project Manager dboyer@azavea.com 215.701.7506 Jeremy Heffner Product Manager jheffner@azavea.com 215.701.7712
  • 3. About Azavea • Founded in 2000 • B Corporation • 30+ people • Based in Philadelphia – Boston office • Geospatial + web + mobile – Software development – Spatial analysis services – User experience
  • 4. Agenda • The Ideal: Gathering Organized, Perfect Data • The Reality: Cleaning and Preparing Your Data • Adding Context • Exploring, Preparing and Sharing Data Visualizations • Questions
  • 6. An open source tree data management system for collaborative, geography enabled urban tree inventory
  • 7. Main Features • Search and Explore Tree Data • View Ecosystem Benefits • Add New Trees • Edit and Update Trees • Upload Tree Photos • Track Stewardship Activities
  • 8.
  • 9. Data Quality Checks • Remove duplicate trees during data upload • Tree watch list • Drop down lists • User groups • Reputation points
  • 11. Data Cleaning: Your Questions • At what point in the data maintenance process do you find yourself cleaning data? • Are there ways that you would like to improve the workflow?
  • 12.
  • 13. Cleaning & Preparing Data • Making sense of data starts at the point of collection – Define what you want to measure / track • Clearly define schema and fields – Have a shared meaning for values – Data validation on entry – Collect your data – Examine results • Are there common mistakes you could prevent? • Are there different interpretations of fields? – Close the feedback loop & iterate
  • 14. Cleaning & Preparing Data • Common data quality issues – Combined fields • Address: “340 N 12th St, Suite 402 , Philadelphia, PA 19107” – Invalid entries • ZIP code: 1234 (length check, is number) • Age: 204 (reasonable range check, is number) – Format variations • State: PA vs. Pennsylvania (drop down or scrubbing rules) – Duplicates • CRM: John Smith with old and new addresses
  • 15. Cleaning & Preparing Data Not a reasonable option
  • 16. What does this have to do with trees? • We track things - tree inventories, potential planting sites, community groups, people who requested trees, etc . • Data comes from lots of places - web forms, collected by various staff, submitted by community groups. • None of it matches. • Good data makes our lives easier.
  • 17. Cleaning & Preparing Data • Tools to clean tabular data – Excel (or open source equivalent) • Pros: – Broad features – Widely utilized / common skill – Formulas / sorting / flexible • Cons: – Doesn’t understand record concept – Mass changes can be tedious
  • 18. Cleaning & Preparing Data • Tools to clean tabular data – DataWrangler • http://vis.stanford.edu/wrangler/ • Pros: – Focused on transforming data into relational format – Live previews • Cons: – Alpha quality version – Data size limits / online tool – Can be difficult to figure out what set of transforms are needed
  • 19. Cleaning & Preparing Data • Tools to clean tabular data – Google Refine • http://code.google.com/p/google-refine/ • Pros: – Understands record concept – Formulas / Facets – Undo capability – Windows / Mac / Linux • Cons: – There is a learning curve – Unusual type of app » Download, unzip, run exe file, access through browser
  • 20. Demo
  • 21. Assembling Data and Building Context
  • 22. Context: Your Questions • What challenges have you faced putting your data in context? • Are you struggling to identify what “context” means for your organization? • Do you know what data you’d like to use, but have trouble finding it?
  • 23. Your Data in Context • Your data is essential! • But it is more meaningful in context… – Ratios & rates • Service level • Market penetration – Indicators & trends • How you compare – Targeting • Key demographics Juice Analytics • Custom summaries
  • 24. What does this have to do with trees? • Trees don’t exist in a vacuum. • Contextual data = more effective outreach. • More info gives you new insights.
  • 25. Making Sense of the Census • American FactFinder • http://factfinder2.census.gov – Decennial Census • Every 10 years • Full population survey • Just 10 questions – American Community Survey (ACS) • Monthly sample • Aggregated over different time periods (1-, 3- and 5-year) • Extremely detailed questions • Subject to sampling error
  • 27. Helpers: Social Explorer • http://www.socialexplorer.com/ • Data Dictionary – Survey – Dataset – Table – Variable – Formula – Population
  • 28. Helpers: Social Explorer • Background – Key Terms – Collection Methodology – Uses & applications
  • 29. Helpers: ACS Alchemist • https://github.com/azavea/acs-alchemist  • Retrieval of block group-level data • Custom variable selection • Delivery in spatial data format ready for mapping This tool was developed by Azavea in collaboration with Jerry Ratcliffe and Ralph Taylor of Temple University Center for Security and Crime Science. This project was supported by Award No. 2010-DE-BX- K004, awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice.
  • 30. Helpers: ACS Alchemist As easy as 1-2-3 1.Create a document with your selected variables
  • 31. Helpers: ACS Alchemist As easy as 1-2-3 1.Create a document with your selected variables 2.Pick your geographies
  • 32. Helpers: ACS Alchemist As easy as 1-2-3 1.Create a document with your selected variables 2.Pick your geographies and geolevels 3.Retrieve your shapefiles
  • 33. Other Sources • Public data – Open Data Portals • Federal, state & local data – Political Data • Voter data • Legislative boundaries • Commercial data – Population Projections – Consumer Data
  • 35. Data Visualization: Your Questions • Do you currently share data with your constituents? • Where do you use data visualizations (e.g. annual report, embedded infographics, live data trackers)? • Do you currently map your data?
  • 36. What does this have to do with trees? • Charts, graphs, maps, and photos help us tell a story. • Show that trees are more than just leaves and branches. • Explore the science without making people’s eyes glaze over.
  • 37. Exploring Data • Visualization tools – Tableau • http://www.tableausoftware.com/ • Pros: – Flexible interface makes data exploration easy – Fast even on large data sets • Cons: – Easy to visualize something that doesn’t make sense to look at – Price (for desktop tool)
  • 38. Demo
  • 39. Exploring Data • Visualization tools – GeoCommons (GeoIQ) • http://geocommons.com/ • Pros: – Intuitive interface – Analysis tools – Geocoding for up to 5,000 records – Supports KML (Google Maps) import & export • Cons: – US-only geocoding
  • 40. Exploring Data • Desktop GIS: Proprietary – Esri ArcGIS • Pros: – Industry standard – Many tools – Extensive training materials – Customer support • Cons: – Windows only – Potentially expensive * *
  • 41. Exploring Data • Visualization tools – ArcGIS Explorer online • http://www.arcgis.com/explorer/ • Pros: – Supports many data formats – Online digitizing – Integration with other Esri services – Presentation view / mobile app • Cons: – Can’t export geocoded results – Geocoding limited to 250 records
  • 42. Demo
  • 43. Exploring Data • Desktop GIS: Open Source – Quantum GIS (QGIS) – GRASS – uDig • Pros: – Free – Multi-platform (Windows, Mac OS, Linux) • Cons: – Limited functionality (for advanced users) – Community-based support
  • 45. Contact Us Deborah Boyer OpenTreeMap Project Manager dboyer@azavea.com 215.701.7506 Jeremy Heffner Product Manager jheffner@azavea.com 215.701.7712
  • 46. Exploring Data Preparation and Visualization Tools for Urban Forestry 340 N 12th St, Suite 402 Philadelphia, PA 19107 215.925.2600 info@azavea.com www.azavea.com/opentreemap