First half on how to use Census Data. Presentation from the perspective of a data person in a Governmental Agency. Second part is about combined Census and an example of how I used ESRI's amazing Tapestry Data.
Dr. Peter Angelides details to the Pennsylvania chapter of the American Planning Association how the recent increase in Big Data collection can be used to make insights about the real estate sector.
July 21, 2021
NCompass Live - http://nlc.nebraska.gov/NCompassLive/
Introduction to U.S. Census Bureau Data Products and Tools, American Community Survey Concepts and Profiles, and new data access platform data.census.gov. The purpose of this informational data session is to acquaint organizations to Census data tools and data.census.gov. By the end of the presentation, participants will be able to access Quick Facts, American Community Survey (ACS) Narrative Profile, and Data Social/Economic Profiles, which provides quick and easy access to select statistics collected by the U.S. Census Bureau.
Presenter: Blanca E. Ramirez-Salazar, Partnership Specialist, Dallas Regional Census Center/Field Division/Denver Region, U.S. Census Bureau.
Dr. Peter Angelides details to the Pennsylvania chapter of the American Planning Association how the recent increase in Big Data collection can be used to make insights about the real estate sector.
July 21, 2021
NCompass Live - http://nlc.nebraska.gov/NCompassLive/
Introduction to U.S. Census Bureau Data Products and Tools, American Community Survey Concepts and Profiles, and new data access platform data.census.gov. The purpose of this informational data session is to acquaint organizations to Census data tools and data.census.gov. By the end of the presentation, participants will be able to access Quick Facts, American Community Survey (ACS) Narrative Profile, and Data Social/Economic Profiles, which provides quick and easy access to select statistics collected by the U.S. Census Bureau.
Presenter: Blanca E. Ramirez-Salazar, Partnership Specialist, Dallas Regional Census Center/Field Division/Denver Region, U.S. Census Bureau.
2017 Global Infrastructure Index: Public Satisfaction and PrioritiesIpsos Public Affairs
ccording to the Ipsos 2017 Global Infrastructure Index, nearly two thirds of Americans (62%) believe that the U.S. is not doing enough to meet its infrastructure needs. Frustration about the amount of attention given to infrastructure is higher in the U.S. than it is on average across the 28 countries surveyed by Ipsos (56%) and higher than in all economically advanced nations with the sole exception of Italy (63%). In contrast, only 23% in Japan, 40% in France, and 50% in Canada say their country is not doing enough. Yet, roughly three quarters of Americans think investing in infrastructure is vital to America’s future economic growth (73%). The release of this report coincides with the 9th North American Infrastructure Leadership Forum organized by CG/LA Infrastructure in partnership with Ipsos.
Immigration Research: Numbers and Findingsborderzine
by D'Vera Cohn, senior writer for the Pew Research Center
Special for the 2013 Specialized Reporting Institute on Immigration Reform.
http://immigrationreportingworkshop2013.borderzine.com/
With the Census in England and Wales taking place on 21 March 2021, we created a programme of webinars to showcase our plans for design and quality assurance. The series, which was carried out through November and December 2020, included a high-level introductory overview as well as 'In Focus' sessions that outlined specific aspects in more detail. These webinars gave attendees the opportunity to ask questions and provide feedback.
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
2017 Global Infrastructure Index: Public Satisfaction and PrioritiesIpsos Public Affairs
ccording to the Ipsos 2017 Global Infrastructure Index, nearly two thirds of Americans (62%) believe that the U.S. is not doing enough to meet its infrastructure needs. Frustration about the amount of attention given to infrastructure is higher in the U.S. than it is on average across the 28 countries surveyed by Ipsos (56%) and higher than in all economically advanced nations with the sole exception of Italy (63%). In contrast, only 23% in Japan, 40% in France, and 50% in Canada say their country is not doing enough. Yet, roughly three quarters of Americans think investing in infrastructure is vital to America’s future economic growth (73%). The release of this report coincides with the 9th North American Infrastructure Leadership Forum organized by CG/LA Infrastructure in partnership with Ipsos.
Immigration Research: Numbers and Findingsborderzine
by D'Vera Cohn, senior writer for the Pew Research Center
Special for the 2013 Specialized Reporting Institute on Immigration Reform.
http://immigrationreportingworkshop2013.borderzine.com/
With the Census in England and Wales taking place on 21 March 2021, we created a programme of webinars to showcase our plans for design and quality assurance. The series, which was carried out through November and December 2020, included a high-level introductory overview as well as 'In Focus' sessions that outlined specific aspects in more detail. These webinars gave attendees the opportunity to ask questions and provide feedback.
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
2. 101 Where does it come from?
o The American Community Survey
o Socio-economic characteristics of a population
o Administered by the U.S. Census Bureau funded by Housing and Urban
Development (HUD)
o 5 year estimates for Zip Codes, Census Tracts, and Block Groups
o Decennial Census
o Population Count
o Last data with socio-economic data – 2000
o 2010 and 2020 does/will not have social & economic data
o Population and Race
3. American Fact Finder = Data
o Download the data
o Or don’t and call the API
o Join by a geography identifier to the geographic areas
o Document table and year
o Block Group – smallest level of geography Census collects data on
o Highest margin of error
o Highest precision lowest accuracy
factfinder.census.gov
5. Policy : update [x] every 10
years based on Census Data
o Possibly a policy created when socio-economic data was collected every ten
(10) years with the decennial census ?
o Update [geographic.Areas] with [ x% of Poverty] with the
Decennial Census numbers
o This is not possible
o Can make calculations on areas each year a dataset comes out
o Calculations each year to identify key areas NOT to compare overlapping
data sets
7. CAUTION! Boundaries Change
The city grew in Population from {yyyy} to {yyyy}; == People are moving to City of {cityName};
FALSE
Did it really? Or did the city annex residential areas?
City Boundaries change every year (Census Boundary and Annexation Survey)
Define “city” - Numbers by City boundary or a Metro area?
Cities (City geography is called “Place” by the Census)
8. CAUTION! Boundaries Change
or don’t exist
Zip Codes
o Most familiar – easiest to comprehend by the public
o But Not a Census Geography
o Representations of USPS service areas
o The USPS service areas change multiple times throughout
the year
o Census makes a ZIP boundary shapefile every year
o 1st ACS data set with Zip Codes is the 2007 – 2011 release
9. Tracts and Block Groups
Census 2000
ACS 2005 - 2009
CAUTION! Boundaries Change
Census 2010
ACS 2006 - 2010
ACS 2007 - 2011
ACS 2008 - 2012
ACS 2009 - 2013
ACS 2010 - 2014
ACS 2011 - 2015
ACS 2012 - 2016
ACS 2013 - 2017
ACS 2014 - 2018
ACS 2015 - 2019
2010 to 2014 First NON-
overlapping 5 year ACS dataset
with ACS 2005 – 2009
2011 – 2015 First NON-
overlapping 5 year ACS dataset
of the SAME GEOGRAPHY with
ACS 2006 – 2010
Census 2020
ACS 2016 - 2020
ACS 2017 - 2021
ACS 2018 - 2022
ACS 2019 - 2023
ACS 2020 - 2024
ACS 2021 - 2025
ACS 2022 - 2026
ACS 2023 - 2027
ACS 2024 - 2028
ACS 2025 - 2029
10. CAUTION! Survey Questions Change
Understand there are limitations to the data
The data is what it is
make calculations and move on
CAUTION! People don’t respond to the survey
CAUTION! Statistical sampling margin of error
CAUTION!
CAUTION!
CAUTION!
CAUTION!
11. What the Client needs to know to make decisions
Example
Is [Address] in a qualified [BlockGroup] based on [ACS.DataConstraint]?
Extra data
Why YES
For reliability and
trust in result
YES
Exactly what Block Group
YES or not – the
main piece of
information
13. Combined Demographic Data
o Previous example combined two ACS data tables
o Can combine data tables, assign weighted values etc etc
14. o Social Vulnerability Index
o Aid in planning for populations to respond to hazardous events
o Agency for Toxic Substances & Disease registry
o Geography: Nationally by County and Tract
o Uses: 14 Factors
o Data Year: 2014
o ACS 2014 5 yr data and other source population data
o Download Shapefile or CSV file http://svi.cdc.gov/SVIDataToolsDownload.html
Pre-Combined Socio-economic Data
15. o Socioeconomic Demographic Clusters
o Ranks socioeconomic status from “higher” to “lower”
o GA Department of Public Health
o Geography: Georgia by Block Group
o Uses: 25 Factors
o Data Year: 2011 (2007-2011 ACS)
o View online map or printable poster
https://oasis.state.ga.us/gis/demographiccluster/DemoClusters2011.htm
o Request or Purchase data: http://dph.georgia.gov/phip-data-request
Pre-Combined Socio-economic Data
For GEORGIA
16. o Esri Tapestry Segmentation
o Help understand customers’ lifestyle choices – identify best customers and
underserved markets
o Environmental Systems Research Institute :P
o Geography: Nationally by all geographic levels
o Uses: 67 Factors ACS and many more data sources
o Data Year: 2016
o Yes 2016 read more if you like– but to view the map you need an ArcGIS Online account
https://www.arcgis.com/home/item.html?id=a422e35d395743089893f08c6f4325f6
o Accessible online map by Zip Code http://www.esri.com/landing-pages/tapestry
o Zip Code Boundaries and methodology created by HERE – not the same boundaries at the
Census Zip Codes!
Pre-Combined Socio-economic Data
17. Open Tapestry from ArcGIS Online in
ArcGIS for Desktop
o Not very useful
o Can only click and view the data
o Cannot use in Geoprocessing tools
o Cannot Export
o Cannot view Attribute Table
o Cannot even select!
Womp womp
18. So much data!
Predictive Demographics
(see every component of this vast
dataset with the Identify Tool)
19. Use Esri Maps for Office
http://www.esri.com/software/maps-for-office#mapsForOfficeDownload
ArcGIS Online comes with ArcGIS
for Desktop license
Need an ArcGIS Online
account to use Esri Maps for
Office
20. Esri Maps for Office
o If concerned about consuming credits – Enrich data with caution
o Scenario – I want to know Tapestry data for a bunch of addresses
o Enrich the addresses?
o But what geography level is going into these addresses?
o Response from chat with esri tech support “When using point data, the output values are
interpolated in some way based on the entire zip code”
o This lead me to learning all about Zip codes – conclusion not a good idea to use Zip Codes
o … A conversation for another time
o Instead – get a dataset I can work with longer term for addresses again and again
o Enrich by Tract or Block Group ID (or Zip Code too if you need to)
21. Esri Maps for Office
o Recommendation – Don’t enrich your point data
o Get base layer dataset then geocode and spatial join to the data
o Have control and knowledge of the geography
DEMO! YouTube: https://youtu.be/IKwgS0N9XSI
Tapestry Data for all the Tracts in Georgia Cost (two columns) 40 Credits
Data Enrichment costs 10 credits per 1,000 records
Which is only $4 ($100 = 1,000 credits)
22. Send me that excel sheet or
shapefile!
Absolutely NOT
Cannot send other organization’s data if
that organization charges for data
23. Case Study
o What insights can I gain about a client’s customers with just the address?
o Client – Homeowner Program for people that make less than $65k/year
o Customer – Homebuyer
o Data- Addresses for 5 years
24. Workflow
Get Tapestry Data
by enriching
polygon GEOID in
Esri Maps for
Office
Geocode
Addresses
Join to Shapefile
Spatial Join
Frequency (Count)
or don’t and use
Pivot Table in Excel
26. LifeMode 7 Ethnic Enclaves
o Established diversity—young, Hispanic homeowners with
families
o Multilingual and multigenerational households feature
children that represent second-, third- or fourth-
generation Hispanic families
o Neighborhoods feature single-family, owner-occupied
homes built at city's edge, primarily built after 1980
o Hard-working and optimistic, most residents aged 25 years
or older have a high school diploma or some college
education
o Shopping and leisure also focus on their children—baby
and children's products from shoes to toys and games and
trips to theme parks, water parks or the zoo
o Residents favor Hispanic programs on radio or television;
children enjoy playing video games on personal computers,
handheld or console devices
o Many households have dogs for domestic pets
LifeMode 4 Family Landscapes
o Successful young families in their first homes
o Non-diverse, prosperous married-couple families, residing
in suburban or semirural areas with a low vacancy rate
(second lowest)
o Homeowners (80%) with mortgages (second highest %),
living in newer single-family homes, with median home
value slightly higher than the U.S.
o Two workers in the family, contributing to the second
highest labor force participation rate, as well as low
unemployment
o Do-it-yourselfers, who work on home improvement
projects, as well as their lawns and gardens
o Sports enthusiasts, typically owning newer sedans or SUVs,
dogs, and savings accounts/plans, comfortable with the
latest technology
o Eat out frequently at fast food or family restaurants to
accommodate their busy lifestyle
o Especially enjoy bowling, swimming, playing golf, playing
video games, watching movies rented via Redbox, and
taking trips to a zoo or theme park