SlideShare a Scribd company logo
1 of 38
Download to read offline
SUMMER PROJECT:
DISTANCE-RELATED VARIABLES
AT BLOCK LEVEL IN NYC
PRESENTER: TIANYUAN LIU
INSTRUCTOR: MIN ZHU
08/2015
CONTENTS
• Project Description
• Methodology
• Takeaway
PROJECT
DESCRIPTION
APPROACH
• Exogeneity
• The attributes may be price-independent.
• Isolate the area-wide factors from property-dependent
factors.
• Hedonic
• Distances to certain facilities increase/decrease the value as
the level convenience of living increases/decreases
• Distances as attributes
• Distances to certain facilities contribute to the value of a block,
a lot, or a single property.
PLACE OF INTEREST (POI)
• Facilities that impact on surrounding area.
• POIs (in ArcGIS) present as points, lines, polygons, or raster.
• We select some facilities as POIs to test if the impact of each
POI is significant.
• We also summarize non-spatial factors as the zonal density of
noise as a POIs.
PROXIMITY(DISTANCE)
• Proximity:
• Attributes of each block
• Test the sensitivity of block-level scale.
• Measured as network distances
• Accessibility of facilities-dependent of road
network, such as walking distances
• Measured as Euclidean distances
• Externality of the facilities-independent from
road network
• Proximity to certain facilities may
positively/negatively impact on property values.
• Impacts diminish at certain rates as distances
increase.
• The diminishing rates may be non-linear.
https://en.wikibooks.org/wiki/Transportation_Geography_and_Network_Science/Circuity#/media/File:TGNS_NetworkDistance.png
https://en.wikibooks.org/wiki/Transportation_Geography_and_Network_Science/Circuity#/media/File:TGNS_EuclideanDistance.png
http://resources.arcgis.com/en/help/main/10.1/index.html#/Near/00080000001q000000/
METHODOLOGY
•Block shapefile of each borough
•Use block suffix to identify
block of the same block
•POI shapefile
Input
•Network Analysis
•Find closest facilities
•Calculate Network Distance
•Generate Near Table
•Calculate Euclidean Distance
•Rasterize non-spatial attributes
•Calculate the number of
facilities within certain distance
of a block
Interim
•Distance Table
•Distance-Dummy Table
•Zonal Attribute Table
Output
PROCESS
INPUT-POI PREPARATION
Name Selection Standard and Action Source Feature
SubwayStation
Copy and Paste DOITT points
Copy and Paste DOITT points
SelectedPark_5a Acreage>=217800 (5 acres) DOITT polygons
Rail_grd ROW_TYPE=Elevated, Surface, Open Cut Depression, Embankment,Viaduct DOITT polylines
Bridge_Tunnel RW_TYPE=Bridges (across shoreline), dissolve, DOITT polylines
PublicAccessibleWaterfront Merge PAWS.shp and NYC_Waterfront_Parks.shp BYTE of BIGAPPLE polygons
WasteManagement Copy and Paste BYTE of BIGAPPLE points
College_3K SubGroup Type=13, Capacity>=3000 BYTE of BIGAPPLE points
College_10K SubGroup Type=13, Capacity>=10000 BYTE of BIGAPPLE points
CulturalFacilities_Others FacType=1601, Capacity>0 BYTE of BIGAPPLE points
Library_300K FacType=1401 and 1402, Capacity>300000 BYTE of BIGAPPLE points
RailStation Copy and Paste DOITT points
Hospital FacType=3102,Capacity>0 BYTE of BIGAPPLE points
HistoricDistrict Status=Designated NYC OPEN DATA polygons
Noise_311 Complaint_Type Contains Noise,Display XY data NYC OPEN DATA points
Noise_Den_25 Point Density, cell size=25, mask=nybb NA raster
Pharmacy Selected by Location (nybb), Amenity=Pharmacy/Name=CVS, Duane Reade, WALGREENS, Rite Aid OpenStreetMap points
Shelter FacType=4401,4402,4411,4412,4414,Capacity>0 BYTE of BIGAPPLE points
INPUT-BLOCK PREPARATION
Identify each
Block
•Newbase table
containing bbl and
block suffix
•Select index lot from
each physical block
•Sort by Boro, Block,
Block Suffix, Lot
•Exclude lot of:
•Pid <0
•Land size=0
•BC=T*, U*, R*
Select block
•Digital Tax Map
containing tax lot
features
•Table containing bbl
and block suffix
•Join by lot BBL
•Lot Feature
containing Boro,
Block, and Block
Suffix.
Blocks with
blksuf
•Digital Tax Map tax
block feature
•Spatial Join the lot
feature with block
feature (get attributes)
•Dissolve to combine
the small block with
same block and suffix
number
•Generate centroid for
each block
PROCESS METHODS
•The accessibility of POI
relies on road network
•Active Access
•walking
•Driving
Network
Analyst
•The accessibility of POI
doesn’t rely on road
network
•Externality of
noise/pollution
•Passive Access
Nearest
Distance
•Summarize the non-
spatial variables
•Create spatial
distribution surfaces
Point
Density
Subway
Station
Rail
Stations
Universi
ties
Museu
m
Hospital
Shelter
Library
Pharmacy
Publicly
Accessible
Waterfront
Railroad
on the
ground
Park
Bridge
and
Tunnel
Waste
Manage
ment
Brownfield
Historic
District
Noise
METHOD LOGIC
If the POI should be
actively accessed from
each block…
Network Analyst
(5 nearest POIs)
Distance Table:
1st Nearest Distance
2nd Nearest Distance
3rd Nearest Distance
4th Nearest Distance
5th Nearest Distance
ArcGIS shapefile
If the POI should be
passively accessed
from each block…
Make Near Table
Nearest Distance Table
ArcGIS shapefile
If the non-spatial
attributes can be
presented
geographically…
Point Density/Raster/
Zonal Table
Zonal Table:
Non-spatial attributes
If the number of POIs
were to be
summarized at block
level…
Multiple Buffers/Spatial
Join
Count Table:
Numbers of POIs of each
block at distance_1
Numbers of POIs of each
block at distance_2
ArcGIS shapefile
INTERIM-NETWORK ANALYST
Incidents
-Blocks
•Block
centroid
shapefile
(OID)
•By boro
•Generate
IncidentID
•Reasonable
Check
Facilities -
POIs
•POI (Point
features only
•Generate
FacilityID
•From
incidents to
facilities
Use
Network
•Road
Network
• Generated
from CSCL
Centerline
(topology)
Solve
•Use incidents,
facilities, and
network feature
layers
•Find the Closest
Facility
•Number of POIs
to find=5
•Use trip length as
impedance
Save
results
•Save route
feature
class
•Save the 5
distance
values to
table
•Transpose
by incident
Join
Distance
back to
Block
•Distance
table with
IncidentID
•Blocks with
IncidentsID
•Blocks with
OID
• Input
• Tax block
• POI
• Subway
Station
Distance to the
1st nearest
Subway
Station
Distance to the
2nd nearest
Subway
Station
Distance to the
3rd nearest
Subway
Station
Distance to the
4th nearest
Subway
Station
Distance to the
5th nearest
Subway
Station
Mean Distance
INTERIM-GENERATE NEAR TABLE
Input feature
-block
•Block centroid
shapefile
•Add OID to identify
each block
•By boro
Near feature
-POIs
•Polylines
•Polygons
•Points
•Euclidean distance
Generate
Near Table
Join Distance
back to Block
•Distance Table for
each block
• Input
• Tax block
• POI
• Park
• Larger than 5
acres
Nearest Distance
INTERIM-
CAPTURE SPATIAL RELATED VARIABLES
Input feature
-block
•Block centroid
shapefile
•Add OID to identify
each block
•By boro
Create Raster
-POIs
•Polylines
•Polygons
•Points
•Attributes: density
Create zonal
table to
summarize
the raster
attributes
into each
block
• Sum
• Area
• Sum/Area
Join zonal
table back to
Block
•Spatial attributes
for each block
Noise Complaint Density
INTERIM-
GIS PROCESS-GENERATE DUMMY VARS
Buffer
•Block feature
•Generate OID for
each block
•Generate Multiple
Buffers for each
block
•0.3-mile buffer
•0.5-mile buffer
Calculate
numbers of
facilities within
buffers of each
block
•Spatial Join with
the point POI
feature
•Field summarize
the number of
facilities
•Save the table
Generate
Dummy
Variables
•If none of the facilities
fall in 0.3-mile buffer,
then dist_030_var0=1,
else=0
•If 1 facility falls in 0.3-
mile buffer, then
dist_030_var1=1,
else=0
• Input
• Tax block
• POI
• Subway
Station
Number of Subway
Stations within 0.3-mile
radius of each block
Number of Subway
Stations within 0.5-mile
radius of each block
Distance=0.3 mile
#=0 #=1 #=2 #=3 #=4 #>=5
Distance=0.5 mile
#=0 #=1 #=2 #=3 #=4 #>=5
TAKEAWAY
PROJECT DESCRIPTION
• Takeaway
• We create a pool of distance attributes for all blocks, and
distances will be classified into different groups based on future
modeling.
• The data can be collected at block/lot/property level.
• Reusable Python script tools enables distance calculation for
point/polyline/polygon POI feature classes.
• The next step may be creating an index based on areal attributes,
such as distance-value index system.
• The raw output as well as the index system can be input
variables for future models.
FILE SYSTEM-
ORIGINAL DATA RawInput
DCP DOITT OPENDATA OpenStreet Collected
workflow_d
ocumentati
on
NYC_PubliclyAccessibleWater
Front_2014
NYC_SelectedFacilities_
2015
TANK Borough_Bo
undaries
cscl_pub.gdb NYC_Planim
etrics_2010
Noise_311_
07012014_0
7012015
TANK remedsitebo
rders
new-
york_new-
york.osm-
point.shp
Potential
Materials
nyc_paws_2
014shp
nyc_waterfrontp
arks_2014shp
nyc_facilities2015_shp Potential
Materials
nybb_15b CSCL SubwayStati
on.shp
NYC_DOITT_
Planimetric_
Seamless_2
010.gdb
Potential
Materials
Remediatio
n_site_bord
ers
PAWS.shp NYC_Waterfront
_Parks.shp
Facilities - 01 -
Schools.lyr
nybb.shp Centerline.s
hp
RailStation.s
hp
NYCPlanime
tric
Remediation
_site_border
s.shp
Facilities - 02 -
Recreational & Cultural
Facilities.lyr
Rail.shp PARK.shp
Facilities - 04 - Nursing
Homes, Hospitals,
Hospices and
Ambulatory Services.lyr
Subway.shp
Facilities - 10 - Food
Programs & Residential
Facilities for Adults and
Families.lyr
Facilities - 12 - Waste
Management
Facilities.lyr
Table File
Shapefile or Layer File
Tools and Documentation
Folder or Geodatabase
FILE SYSTEM-
NETWORK ANALYSIS/NEARANALYSIS
NetworkAnalysis
POI_input POI_output Output_dist Tools_Python Tools_SAS
boroBD.gdb dtmblock.gdb POI.gdb RoadNetwork.gdb BridgeTunnel.txt POI* dist_mean_input dist_mean_outp
ut
NA_block_mean
dist
1_BlkSuf.py POI_MakeNear
Table
POI_NetworkAn
alyst
boro*_BD.shp boro*_blk.shp POI*.shp RoadNetwork College_3K.txt boro*_POI*_Cre
ationDate*.gdb
POI* POI* POI* 2_blksuf_cent_to
_poigdb.py
POI* (create
near table)
POI*(for
network
analysis)
nybb.shp boro*_blkcent.sh
p
POI*=BridgeTunnel,
Brownfield, College_3K,
College_10K,
CulturalFacilities_Others
, HistoricDistrict,
Hospital, Library_300K,
Noise, Pharmacy,
PublicAccessibleWaterfr
ont, Rail_grd,
RailStation,
SelectedPark_5a,
Shelter, SubwayStation,
WasteManagement
RoadNetwork_ND College_10K.txt boro*_POI*_Cre
ationDate*
(table)
blkcent_boro*_P
OI*_CreationDat
e*.dbf
boro*_POI*_Cre
ationDate*_mea
ndist.dbf
boro*_POI*_blk.
dbf (for raster)
3_NA_NF.py boro_macro.sa
s
boro_macro.sa
s
boro*=MH, BX,
BK, QN, SI
CulturalFacilities_O
ther.txt
blkcent-
boro*_POI*_Cre
ationDate*.shp
blkcent_boro*_P
OI*_CreationDat
e*.dbf
4_Blkcent_dist_jo
in.py
macrocall.sas macrocall.sas
Library_300K.txt 5_MakingNearTa
ble.py
Pharmacy.txt 6_near_Blkcent_
dist_join.py
PublicAccessibleWa
terfront.txt
9_raster_blk_join
.py
Rail_grd.txt
SelectedPark_5A.txt
Table File
Shapefile or Layer File
Tools and Documentation
Folder or Geodatabase
FILE SYSTEM-
CREATE DUMMY VARIABLE (BETA)
DistanceAnalysis
POI_buffer_inpu
t
POI_buffer_ouput table_input_Python
table_interim_S
AS
table_output_SAS table_tablejoin Tools_Python Tools_SAS
SubwayStation SubwayStation SubwayStation
condosuff_Subw
ayStation_count
.dbf
SubwayStation
condosuff_Subw
ayStation_count
.dbf
blk_boro*_Sub
wayStation_720
15_dummy.dbf
7_number_coun
t.py
buffer_count.sa
s
cdsuff_xy.csv
boro*_SubwaySt
ation_72015.gd
b
scratch.gdb
boro*_SubwayS
tation_72015_b
fct.dbf
boro*_SubwayStati
on_bfct.dbf
8_count_join.py
boro*_SubwaySt
ation_72015.shp
boro*_SubwaySt
ation_72015_bf
ct.shp
blk_boro*_Subw
ayStation_72015
_dummy.shp
Table File
Shapefile or Layer File
Tools and Documentation
Folder or Geodatabase
*FUTURE ACTIONS-
ADD POI
• Download original shapefiles in RawInput Folder
• Sort by the source of the files (DCP, DOITT, OPENDATA,
OpenStreetMap, or SelfCollection…)
• Put POI shapefiles in POI_inputPOI.gdb
• Select the Python Tools and SAS Tools to process
• Need to change POIs manually in each script
*FUTURE ACTIONS-
TOOLS AND RESULT TABLES…
• Network Analysis-
• Input
• POI_inputPOI.gdb
• POI_inputdtmblock.gdbblk(cent)
• Point Features only
• Tool_Python3_NA_NF.py
• dist_mean_inputPOI*dbf
• Tool_SASPOI_NetworkAnalystboro_macro
• dist_mean_outputPOI*dbf
• Tool_Python4_blkcent_dist_join
• NA_block_meandistPOI*dbf
• Generate Near Table-
• Input
• POI_inputPOI.gdb
• POI_inputdtmblock.gdbblk(cent)
• Point/Polyline/Polygon features
• Tool_Python5_make_near_table.py
• dist_mean_inputPOI*dbf
• Tool_SASPOI_MakeNearTable boro_macro
• dist_mean_outputPOI*dbf
• Tool_Python 6_near_Blkcent_dist_join.py
• NA_block_meandistPOI*dbf
*FUTURE ACTIONS-
SUMMARIZE THE RESULT
• Summarize the result in the master table of each boro
• Output_distDescriptiveboro*.xlsx
• Sort the result based on the method of distance calculation
• Near
• Sorted by ORIG_FID
• Network Analyst
• Sorted by ORIG_FID
• Mark the missing value with IncidentID
• Raster (Beta)
• Sorted by OID_12
• Mark the missing value with IncidentID

More Related Content

Viewers also liked

Staff Appreciation Day ~ AUGUST 6, 2015
Staff Appreciation Day ~ AUGUST 6, 2015Staff Appreciation Day ~ AUGUST 6, 2015
Staff Appreciation Day ~ AUGUST 6, 2015massarthr
 
Factors of production
Factors of productionFactors of production
Factors of productionShelando
 
Slide master gerund
Slide master gerund Slide master gerund
Slide master gerund dewikus
 
aplikom_UNSRI_1.Biodata dan Keunikan Matematika_Restie Amelia
aplikom_UNSRI_1.Biodata dan Keunikan Matematika_Restie Ameliaaplikom_UNSRI_1.Biodata dan Keunikan Matematika_Restie Amelia
aplikom_UNSRI_1.Biodata dan Keunikan Matematika_Restie AmeliaRestie Amelia
 
Villas in Whitefield Bangalore
Villas in Whitefield BangaloreVillas in Whitefield Bangalore
Villas in Whitefield BangaloreDaiwiksparsh
 
Basic of textile
Basic of textileBasic of textile
Basic of textilerahul dev
 
Rights, Diversity and Adult Learning
Rights, Diversity and Adult LearningRights, Diversity and Adult Learning
Rights, Diversity and Adult LearningShakeema Hughes
 

Viewers also liked (16)

Staff Appreciation Day ~ AUGUST 6, 2015
Staff Appreciation Day ~ AUGUST 6, 2015Staff Appreciation Day ~ AUGUST 6, 2015
Staff Appreciation Day ~ AUGUST 6, 2015
 
Factors of production
Factors of productionFactors of production
Factors of production
 
Dobner- Event Recap
Dobner- Event RecapDobner- Event Recap
Dobner- Event Recap
 
Ignite Presentation
Ignite PresentationIgnite Presentation
Ignite Presentation
 
Games 42-45 - MSU
Games 42-45 - MSUGames 42-45 - MSU
Games 42-45 - MSU
 
Slide master gerund
Slide master gerund Slide master gerund
Slide master gerund
 
BDAS Presentation
BDAS PresentationBDAS Presentation
BDAS Presentation
 
Curriculum Vitae sajan
Curriculum Vitae sajanCurriculum Vitae sajan
Curriculum Vitae sajan
 
Nawabi
NawabiNawabi
Nawabi
 
aplikom_UNSRI_1.Biodata dan Keunikan Matematika_Restie Amelia
aplikom_UNSRI_1.Biodata dan Keunikan Matematika_Restie Ameliaaplikom_UNSRI_1.Biodata dan Keunikan Matematika_Restie Amelia
aplikom_UNSRI_1.Biodata dan Keunikan Matematika_Restie Amelia
 
Copy of bills resume
Copy of bills resumeCopy of bills resume
Copy of bills resume
 
Villas in Whitefield Bangalore
Villas in Whitefield BangaloreVillas in Whitefield Bangalore
Villas in Whitefield Bangalore
 
NEW CV
NEW CVNEW CV
NEW CV
 
Analisis skripsi
Analisis skripsiAnalisis skripsi
Analisis skripsi
 
Basic of textile
Basic of textileBasic of textile
Basic of textile
 
Rights, Diversity and Adult Learning
Rights, Diversity and Adult LearningRights, Diversity and Adult Learning
Rights, Diversity and Adult Learning
 

Similar to Summary_presentation_TL

Building a Spatial Database in PostgreSQL
Building a Spatial Database in PostgreSQLBuilding a Spatial Database in PostgreSQL
Building a Spatial Database in PostgreSQLSohail Akbar Goheer
 
TYBSC IT PGIS Unit IV Spacial Data Analysis
TYBSC IT PGIS Unit IV  Spacial Data AnalysisTYBSC IT PGIS Unit IV  Spacial Data Analysis
TYBSC IT PGIS Unit IV Spacial Data AnalysisArti Parab Academics
 
Landmark-based instructions for pedestrian navigation systems using OSM
Landmark-based instructions for pedestrian navigation systems using OSMLandmark-based instructions for pedestrian navigation systems using OSM
Landmark-based instructions for pedestrian navigation systems using OSMAnita Graser
 
Stratio's Cassandra Lucene index: Geospatial use cases
Stratio's Cassandra Lucene index: Geospatial use casesStratio's Cassandra Lucene index: Geospatial use cases
Stratio's Cassandra Lucene index: Geospatial use casesAndrés de la Peña
 
Stratio's Cassandra Lucene index: Geospatial Use Cases (Andrés de la Peña & J...
Stratio's Cassandra Lucene index: Geospatial Use Cases (Andrés de la Peña & J...Stratio's Cassandra Lucene index: Geospatial Use Cases (Andrés de la Peña & J...
Stratio's Cassandra Lucene index: Geospatial Use Cases (Andrés de la Peña & J...DataStax
 
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...Safe Software
 
On Line Training of the Path-Loss Model in Bayesian WLAN Indoor Positioning
On Line Training of the Path-Loss Model in Bayesian WLAN Indoor PositioningOn Line Training of the Path-Loss Model in Bayesian WLAN Indoor Positioning
On Line Training of the Path-Loss Model in Bayesian WLAN Indoor Positioninglbruno236
 
NGSI: Geoqueries & Carto integration
NGSI: Geoqueries & Carto integrationNGSI: Geoqueries & Carto integration
NGSI: Geoqueries & Carto integrationFIWARE
 
GEOPROCESSING IN QGIS
GEOPROCESSING IN QGISGEOPROCESSING IN QGIS
GEOPROCESSING IN QGISSwetha A
 
Workshop_Indoor and outdoor mapping
Workshop_Indoor and outdoor mappingWorkshop_Indoor and outdoor mapping
Workshop_Indoor and outdoor mappingElisabet Adeva
 
City focus: A web-based interactive 2D and 3D GIS application to find the bes...
City focus: A web-based interactive 2D and 3D GIS application to find the bes...City focus: A web-based interactive 2D and 3D GIS application to find the bes...
City focus: A web-based interactive 2D and 3D GIS application to find the bes...Carolina Arias Muñoz
 
IRJET- Road Recognition from Remote Sensing Imagery using Machine Learning
IRJET- Road Recognition from Remote Sensing Imagery using Machine LearningIRJET- Road Recognition from Remote Sensing Imagery using Machine Learning
IRJET- Road Recognition from Remote Sensing Imagery using Machine LearningIRJET Journal
 
Agile2014 fhc25 using open street maps data and tools for indoor mapping in a...
Agile2014 fhc25 using open street maps data and tools for indoor mapping in a...Agile2014 fhc25 using open street maps data and tools for indoor mapping in a...
Agile2014 fhc25 using open street maps data and tools for indoor mapping in a...Guillermo Amat
 
A customized workflow for indoor CBRN dispersions analysis in subway stations
A customized workflow for indoor CBRN dispersions analysis in subway stationsA customized workflow for indoor CBRN dispersions analysis in subway stations
A customized workflow for indoor CBRN dispersions analysis in subway stationsFLUIDIAN
 
Apdm 101 Arc Gis Pipeline Data Model (1)
Apdm 101 Arc Gis Pipeline Data Model  (1)Apdm 101 Arc Gis Pipeline Data Model  (1)
Apdm 101 Arc Gis Pipeline Data Model (1)David Nichter, GISP
 

Similar to Summary_presentation_TL (20)

Zoning in ArcGIS
Zoning in ArcGISZoning in ArcGIS
Zoning in ArcGIS
 
Building a Spatial Database in PostgreSQL
Building a Spatial Database in PostgreSQLBuilding a Spatial Database in PostgreSQL
Building a Spatial Database in PostgreSQL
 
TYBSC IT PGIS Unit IV Spacial Data Analysis
TYBSC IT PGIS Unit IV  Spacial Data AnalysisTYBSC IT PGIS Unit IV  Spacial Data Analysis
TYBSC IT PGIS Unit IV Spacial Data Analysis
 
Landmark-based instructions for pedestrian navigation systems using OSM
Landmark-based instructions for pedestrian navigation systems using OSMLandmark-based instructions for pedestrian navigation systems using OSM
Landmark-based instructions for pedestrian navigation systems using OSM
 
Path planning all algos
Path planning all algosPath planning all algos
Path planning all algos
 
Gis Concepts 5/5
Gis Concepts 5/5Gis Concepts 5/5
Gis Concepts 5/5
 
SEI OSGIS presentation
SEI OSGIS presentationSEI OSGIS presentation
SEI OSGIS presentation
 
Stratio's Cassandra Lucene index: Geospatial use cases
Stratio's Cassandra Lucene index: Geospatial use casesStratio's Cassandra Lucene index: Geospatial use cases
Stratio's Cassandra Lucene index: Geospatial use cases
 
Stratio's Cassandra Lucene index: Geospatial Use Cases (Andrés de la Peña & J...
Stratio's Cassandra Lucene index: Geospatial Use Cases (Andrés de la Peña & J...Stratio's Cassandra Lucene index: Geospatial Use Cases (Andrés de la Peña & J...
Stratio's Cassandra Lucene index: Geospatial Use Cases (Andrés de la Peña & J...
 
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...
 
On Line Training of the Path-Loss Model in Bayesian WLAN Indoor Positioning
On Line Training of the Path-Loss Model in Bayesian WLAN Indoor PositioningOn Line Training of the Path-Loss Model in Bayesian WLAN Indoor Positioning
On Line Training of the Path-Loss Model in Bayesian WLAN Indoor Positioning
 
Openstreetmap
OpenstreetmapOpenstreetmap
Openstreetmap
 
NGSI: Geoqueries & Carto integration
NGSI: Geoqueries & Carto integrationNGSI: Geoqueries & Carto integration
NGSI: Geoqueries & Carto integration
 
GEOPROCESSING IN QGIS
GEOPROCESSING IN QGISGEOPROCESSING IN QGIS
GEOPROCESSING IN QGIS
 
Workshop_Indoor and outdoor mapping
Workshop_Indoor and outdoor mappingWorkshop_Indoor and outdoor mapping
Workshop_Indoor and outdoor mapping
 
City focus: A web-based interactive 2D and 3D GIS application to find the bes...
City focus: A web-based interactive 2D and 3D GIS application to find the bes...City focus: A web-based interactive 2D and 3D GIS application to find the bes...
City focus: A web-based interactive 2D and 3D GIS application to find the bes...
 
IRJET- Road Recognition from Remote Sensing Imagery using Machine Learning
IRJET- Road Recognition from Remote Sensing Imagery using Machine LearningIRJET- Road Recognition from Remote Sensing Imagery using Machine Learning
IRJET- Road Recognition from Remote Sensing Imagery using Machine Learning
 
Agile2014 fhc25 using open street maps data and tools for indoor mapping in a...
Agile2014 fhc25 using open street maps data and tools for indoor mapping in a...Agile2014 fhc25 using open street maps data and tools for indoor mapping in a...
Agile2014 fhc25 using open street maps data and tools for indoor mapping in a...
 
A customized workflow for indoor CBRN dispersions analysis in subway stations
A customized workflow for indoor CBRN dispersions analysis in subway stationsA customized workflow for indoor CBRN dispersions analysis in subway stations
A customized workflow for indoor CBRN dispersions analysis in subway stations
 
Apdm 101 Arc Gis Pipeline Data Model (1)
Apdm 101 Arc Gis Pipeline Data Model  (1)Apdm 101 Arc Gis Pipeline Data Model  (1)
Apdm 101 Arc Gis Pipeline Data Model (1)
 

Summary_presentation_TL

  • 1. SUMMER PROJECT: DISTANCE-RELATED VARIABLES AT BLOCK LEVEL IN NYC PRESENTER: TIANYUAN LIU INSTRUCTOR: MIN ZHU 08/2015
  • 2. CONTENTS • Project Description • Methodology • Takeaway
  • 4. APPROACH • Exogeneity • The attributes may be price-independent. • Isolate the area-wide factors from property-dependent factors. • Hedonic • Distances to certain facilities increase/decrease the value as the level convenience of living increases/decreases • Distances as attributes • Distances to certain facilities contribute to the value of a block, a lot, or a single property.
  • 5. PLACE OF INTEREST (POI) • Facilities that impact on surrounding area. • POIs (in ArcGIS) present as points, lines, polygons, or raster. • We select some facilities as POIs to test if the impact of each POI is significant. • We also summarize non-spatial factors as the zonal density of noise as a POIs.
  • 6. PROXIMITY(DISTANCE) • Proximity: • Attributes of each block • Test the sensitivity of block-level scale. • Measured as network distances • Accessibility of facilities-dependent of road network, such as walking distances • Measured as Euclidean distances • Externality of the facilities-independent from road network • Proximity to certain facilities may positively/negatively impact on property values. • Impacts diminish at certain rates as distances increase. • The diminishing rates may be non-linear. https://en.wikibooks.org/wiki/Transportation_Geography_and_Network_Science/Circuity#/media/File:TGNS_NetworkDistance.png https://en.wikibooks.org/wiki/Transportation_Geography_and_Network_Science/Circuity#/media/File:TGNS_EuclideanDistance.png http://resources.arcgis.com/en/help/main/10.1/index.html#/Near/00080000001q000000/
  • 8. •Block shapefile of each borough •Use block suffix to identify block of the same block •POI shapefile Input •Network Analysis •Find closest facilities •Calculate Network Distance •Generate Near Table •Calculate Euclidean Distance •Rasterize non-spatial attributes •Calculate the number of facilities within certain distance of a block Interim •Distance Table •Distance-Dummy Table •Zonal Attribute Table Output PROCESS
  • 9. INPUT-POI PREPARATION Name Selection Standard and Action Source Feature SubwayStation Copy and Paste DOITT points Copy and Paste DOITT points SelectedPark_5a Acreage>=217800 (5 acres) DOITT polygons Rail_grd ROW_TYPE=Elevated, Surface, Open Cut Depression, Embankment,Viaduct DOITT polylines Bridge_Tunnel RW_TYPE=Bridges (across shoreline), dissolve, DOITT polylines PublicAccessibleWaterfront Merge PAWS.shp and NYC_Waterfront_Parks.shp BYTE of BIGAPPLE polygons WasteManagement Copy and Paste BYTE of BIGAPPLE points College_3K SubGroup Type=13, Capacity>=3000 BYTE of BIGAPPLE points College_10K SubGroup Type=13, Capacity>=10000 BYTE of BIGAPPLE points CulturalFacilities_Others FacType=1601, Capacity>0 BYTE of BIGAPPLE points Library_300K FacType=1401 and 1402, Capacity>300000 BYTE of BIGAPPLE points RailStation Copy and Paste DOITT points Hospital FacType=3102,Capacity>0 BYTE of BIGAPPLE points HistoricDistrict Status=Designated NYC OPEN DATA polygons Noise_311 Complaint_Type Contains Noise,Display XY data NYC OPEN DATA points Noise_Den_25 Point Density, cell size=25, mask=nybb NA raster Pharmacy Selected by Location (nybb), Amenity=Pharmacy/Name=CVS, Duane Reade, WALGREENS, Rite Aid OpenStreetMap points Shelter FacType=4401,4402,4411,4412,4414,Capacity>0 BYTE of BIGAPPLE points
  • 10. INPUT-BLOCK PREPARATION Identify each Block •Newbase table containing bbl and block suffix •Select index lot from each physical block •Sort by Boro, Block, Block Suffix, Lot •Exclude lot of: •Pid <0 •Land size=0 •BC=T*, U*, R* Select block •Digital Tax Map containing tax lot features •Table containing bbl and block suffix •Join by lot BBL •Lot Feature containing Boro, Block, and Block Suffix. Blocks with blksuf •Digital Tax Map tax block feature •Spatial Join the lot feature with block feature (get attributes) •Dissolve to combine the small block with same block and suffix number •Generate centroid for each block
  • 11. PROCESS METHODS •The accessibility of POI relies on road network •Active Access •walking •Driving Network Analyst •The accessibility of POI doesn’t rely on road network •Externality of noise/pollution •Passive Access Nearest Distance •Summarize the non- spatial variables •Create spatial distribution surfaces Point Density Subway Station Rail Stations Universi ties Museu m Hospital Shelter Library Pharmacy Publicly Accessible Waterfront Railroad on the ground Park Bridge and Tunnel Waste Manage ment Brownfield Historic District Noise
  • 12. METHOD LOGIC If the POI should be actively accessed from each block… Network Analyst (5 nearest POIs) Distance Table: 1st Nearest Distance 2nd Nearest Distance 3rd Nearest Distance 4th Nearest Distance 5th Nearest Distance ArcGIS shapefile If the POI should be passively accessed from each block… Make Near Table Nearest Distance Table ArcGIS shapefile If the non-spatial attributes can be presented geographically… Point Density/Raster/ Zonal Table Zonal Table: Non-spatial attributes If the number of POIs were to be summarized at block level… Multiple Buffers/Spatial Join Count Table: Numbers of POIs of each block at distance_1 Numbers of POIs of each block at distance_2 ArcGIS shapefile
  • 13. INTERIM-NETWORK ANALYST Incidents -Blocks •Block centroid shapefile (OID) •By boro •Generate IncidentID •Reasonable Check Facilities - POIs •POI (Point features only •Generate FacilityID •From incidents to facilities Use Network •Road Network • Generated from CSCL Centerline (topology) Solve •Use incidents, facilities, and network feature layers •Find the Closest Facility •Number of POIs to find=5 •Use trip length as impedance Save results •Save route feature class •Save the 5 distance values to table •Transpose by incident Join Distance back to Block •Distance table with IncidentID •Blocks with IncidentsID •Blocks with OID
  • 14. • Input • Tax block • POI • Subway Station
  • 15. Distance to the 1st nearest Subway Station Distance to the 2nd nearest Subway Station Distance to the 3rd nearest Subway Station Distance to the 4th nearest Subway Station Distance to the 5th nearest Subway Station
  • 17.
  • 18. INTERIM-GENERATE NEAR TABLE Input feature -block •Block centroid shapefile •Add OID to identify each block •By boro Near feature -POIs •Polylines •Polygons •Points •Euclidean distance Generate Near Table Join Distance back to Block •Distance Table for each block
  • 19. • Input • Tax block • POI • Park • Larger than 5 acres
  • 21.
  • 22. INTERIM- CAPTURE SPATIAL RELATED VARIABLES Input feature -block •Block centroid shapefile •Add OID to identify each block •By boro Create Raster -POIs •Polylines •Polygons •Points •Attributes: density Create zonal table to summarize the raster attributes into each block • Sum • Area • Sum/Area Join zonal table back to Block •Spatial attributes for each block
  • 24.
  • 25. INTERIM- GIS PROCESS-GENERATE DUMMY VARS Buffer •Block feature •Generate OID for each block •Generate Multiple Buffers for each block •0.3-mile buffer •0.5-mile buffer Calculate numbers of facilities within buffers of each block •Spatial Join with the point POI feature •Field summarize the number of facilities •Save the table Generate Dummy Variables •If none of the facilities fall in 0.3-mile buffer, then dist_030_var0=1, else=0 •If 1 facility falls in 0.3- mile buffer, then dist_030_var1=1, else=0
  • 26. • Input • Tax block • POI • Subway Station
  • 27. Number of Subway Stations within 0.3-mile radius of each block Number of Subway Stations within 0.5-mile radius of each block
  • 28. Distance=0.3 mile #=0 #=1 #=2 #=3 #=4 #>=5
  • 29. Distance=0.5 mile #=0 #=1 #=2 #=3 #=4 #>=5
  • 30.
  • 32. PROJECT DESCRIPTION • Takeaway • We create a pool of distance attributes for all blocks, and distances will be classified into different groups based on future modeling. • The data can be collected at block/lot/property level. • Reusable Python script tools enables distance calculation for point/polyline/polygon POI feature classes. • The next step may be creating an index based on areal attributes, such as distance-value index system. • The raw output as well as the index system can be input variables for future models.
  • 33. FILE SYSTEM- ORIGINAL DATA RawInput DCP DOITT OPENDATA OpenStreet Collected workflow_d ocumentati on NYC_PubliclyAccessibleWater Front_2014 NYC_SelectedFacilities_ 2015 TANK Borough_Bo undaries cscl_pub.gdb NYC_Planim etrics_2010 Noise_311_ 07012014_0 7012015 TANK remedsitebo rders new- york_new- york.osm- point.shp Potential Materials nyc_paws_2 014shp nyc_waterfrontp arks_2014shp nyc_facilities2015_shp Potential Materials nybb_15b CSCL SubwayStati on.shp NYC_DOITT_ Planimetric_ Seamless_2 010.gdb Potential Materials Remediatio n_site_bord ers PAWS.shp NYC_Waterfront _Parks.shp Facilities - 01 - Schools.lyr nybb.shp Centerline.s hp RailStation.s hp NYCPlanime tric Remediation _site_border s.shp Facilities - 02 - Recreational & Cultural Facilities.lyr Rail.shp PARK.shp Facilities - 04 - Nursing Homes, Hospitals, Hospices and Ambulatory Services.lyr Subway.shp Facilities - 10 - Food Programs & Residential Facilities for Adults and Families.lyr Facilities - 12 - Waste Management Facilities.lyr Table File Shapefile or Layer File Tools and Documentation Folder or Geodatabase
  • 34. FILE SYSTEM- NETWORK ANALYSIS/NEARANALYSIS NetworkAnalysis POI_input POI_output Output_dist Tools_Python Tools_SAS boroBD.gdb dtmblock.gdb POI.gdb RoadNetwork.gdb BridgeTunnel.txt POI* dist_mean_input dist_mean_outp ut NA_block_mean dist 1_BlkSuf.py POI_MakeNear Table POI_NetworkAn alyst boro*_BD.shp boro*_blk.shp POI*.shp RoadNetwork College_3K.txt boro*_POI*_Cre ationDate*.gdb POI* POI* POI* 2_blksuf_cent_to _poigdb.py POI* (create near table) POI*(for network analysis) nybb.shp boro*_blkcent.sh p POI*=BridgeTunnel, Brownfield, College_3K, College_10K, CulturalFacilities_Others , HistoricDistrict, Hospital, Library_300K, Noise, Pharmacy, PublicAccessibleWaterfr ont, Rail_grd, RailStation, SelectedPark_5a, Shelter, SubwayStation, WasteManagement RoadNetwork_ND College_10K.txt boro*_POI*_Cre ationDate* (table) blkcent_boro*_P OI*_CreationDat e*.dbf boro*_POI*_Cre ationDate*_mea ndist.dbf boro*_POI*_blk. dbf (for raster) 3_NA_NF.py boro_macro.sa s boro_macro.sa s boro*=MH, BX, BK, QN, SI CulturalFacilities_O ther.txt blkcent- boro*_POI*_Cre ationDate*.shp blkcent_boro*_P OI*_CreationDat e*.dbf 4_Blkcent_dist_jo in.py macrocall.sas macrocall.sas Library_300K.txt 5_MakingNearTa ble.py Pharmacy.txt 6_near_Blkcent_ dist_join.py PublicAccessibleWa terfront.txt 9_raster_blk_join .py Rail_grd.txt SelectedPark_5A.txt Table File Shapefile or Layer File Tools and Documentation Folder or Geodatabase
  • 35. FILE SYSTEM- CREATE DUMMY VARIABLE (BETA) DistanceAnalysis POI_buffer_inpu t POI_buffer_ouput table_input_Python table_interim_S AS table_output_SAS table_tablejoin Tools_Python Tools_SAS SubwayStation SubwayStation SubwayStation condosuff_Subw ayStation_count .dbf SubwayStation condosuff_Subw ayStation_count .dbf blk_boro*_Sub wayStation_720 15_dummy.dbf 7_number_coun t.py buffer_count.sa s cdsuff_xy.csv boro*_SubwaySt ation_72015.gd b scratch.gdb boro*_SubwayS tation_72015_b fct.dbf boro*_SubwayStati on_bfct.dbf 8_count_join.py boro*_SubwaySt ation_72015.shp boro*_SubwaySt ation_72015_bf ct.shp blk_boro*_Subw ayStation_72015 _dummy.shp Table File Shapefile or Layer File Tools and Documentation Folder or Geodatabase
  • 36. *FUTURE ACTIONS- ADD POI • Download original shapefiles in RawInput Folder • Sort by the source of the files (DCP, DOITT, OPENDATA, OpenStreetMap, or SelfCollection…) • Put POI shapefiles in POI_inputPOI.gdb • Select the Python Tools and SAS Tools to process • Need to change POIs manually in each script
  • 37. *FUTURE ACTIONS- TOOLS AND RESULT TABLES… • Network Analysis- • Input • POI_inputPOI.gdb • POI_inputdtmblock.gdbblk(cent) • Point Features only • Tool_Python3_NA_NF.py • dist_mean_inputPOI*dbf • Tool_SASPOI_NetworkAnalystboro_macro • dist_mean_outputPOI*dbf • Tool_Python4_blkcent_dist_join • NA_block_meandistPOI*dbf • Generate Near Table- • Input • POI_inputPOI.gdb • POI_inputdtmblock.gdbblk(cent) • Point/Polyline/Polygon features • Tool_Python5_make_near_table.py • dist_mean_inputPOI*dbf • Tool_SASPOI_MakeNearTable boro_macro • dist_mean_outputPOI*dbf • Tool_Python 6_near_Blkcent_dist_join.py • NA_block_meandistPOI*dbf
  • 38. *FUTURE ACTIONS- SUMMARIZE THE RESULT • Summarize the result in the master table of each boro • Output_distDescriptiveboro*.xlsx • Sort the result based on the method of distance calculation • Near • Sorted by ORIG_FID • Network Analyst • Sorted by ORIG_FID • Mark the missing value with IncidentID • Raster (Beta) • Sorted by OID_12 • Mark the missing value with IncidentID