Embedding user generated content into oblique airborne photogrammetry based 3D city model

Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=tgis20
Download by: [Arizona State University Libraries] Date: 16 June 2016, At: 09:08
International Journal of Geographical Information
Science
ISSN: 1365-8816 (Print) 1362-3087 (Online) Journal homepage: http://www.tandfonline.com/loi/tgis20
Embedding user-generated content into oblique
airborne photogrammetry-based 3D city model
Jianming Liang, Shen Shen, Jianhua Gong, Jin Liu & Jinming Zhang
To cite this article: Jianming Liang, Shen Shen, Jianhua Gong, Jin Liu & Jinming Zhang
(2016): Embedding user-generated content into oblique airborne photogrammetry-
based 3D city model, International Journal of Geographical Information Science, DOI:
10.1080/13658816.2016.1180389
To link to this article: http://dx.doi.org/10.1080/13658816.2016.1180389
Published online: 29 Apr 2016.
Submit your article to this journal
Article views: 56
View related articles
View Crossmark data

Embedding user-generated content into oblique airborne
photogrammetry-based 3D city model
Jianming Liang a,b
, Shen Shena,c
, Jianhua Gonga,b
, Jin Liua,c,d
and Jinming Zhanga,e
a
State Key Laboratory of Remote Sensing Science, Institute of Remote Sensing and Digital Earth, Chinese
Academy of Sciences, Beijing, China; b
Zhejiang-CAS Application Center for Geoinformatics, Jiashan, China;
c
University of Chinese Academy of Sciences, Beijing, China; d
National Marine Data and Information Service,
Tianjin, China; e
Institute of Geospatial Information, Information Engineering University, Zhengzhou, China
ABSTRACT
Oblique airborne photogrammetry-based three-dimensional (3D)
city model (OAP3D) provides a spatially continuous representation
of urban landscapes that encompasses buildings, road networks,
trees, bushes, water bodies, and topographic features. OAP3D is
usually present in the form of a group of unclassified triangular
meshes under a multi-resolution data structure. Modifying such a
non-separable landscape constitutes a daunting task because
manual mesh editing is normally required. In this paper, we pre-
sent a systematic approach for easily embedding user-generated
content into OAP3D. We reduce the complexity of OAP3D mod-
ification from a 3D mesh operation to a two-dimensional (2D)
raster operation through the following workflow: (1) A region of
interest (ROI) is selected to cover the area that is intended to be
modified for accommodating user-defined content. (2) Spatial
interpolation using a set of manually controlled elevation samples
is employed to generate a user-defined digital surface model
(DSM), which is used to reform the ROI surface. (3) User-generated
objects, for example, artistically painted road textures, procedu-
rally generated water effects, and manually created 3D building
models, are overlaid onto the reformed ROI.
ARTICLE HISTORY
Received 25 November 2015
Accepted 14 April 2016
KEYWORDS
Oblique airborne
photogrammetry; 3D city
model; 2D editing;
user-generated content
1. Introduction
Modeling urban space in three dimensions is important for urban planning and urban
environmental analysis (Köninger and Bartel 1998). Data acquisition, data processing,
and visualization technologies for 3D geospatial data have enhanced the process of 3D
city modeling (Moser et al. 2010), which lays the foundation for a wide range of urban
applications including urban climate simulation (Strzalka et al. 2011), urban planning
(Wu et al. 2010), and visibility calculation (Yasumoto et al. 2011, 2012). An increasing
number of applications and systems incorporate 3D city model as an integral compo-
nent to serve urban planning and redevelopment, facility management, logistics, secur-
ity, telecommunications, disaster management, location-based services, real estate
portals as well as urban-related entertainment and education products (Döllner et al.
CONTACT Jianhua Gong gongjh@radi.ac.cn
INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE, 2016
http://dx.doi.org/10.1080/13658816.2016.1180389
© 2016 Informa UK Limited, trading as Taylor & Francis Group
Downloadedby[ArizonaStateUniversityLibraries]at09:0816June2016

2006). An urban 3D GIS can potentially be extended to a virtual geographic environ-
ment, which is a type of workspace for computer-aided geographic experiments and
geographic analyses with support for geo-visualization, geo-simulation, geo-collabora-
tion, and human participation (Lin et al. 2013).
Traditionally, labor-intensive computer-aided design (CAD) modeling (Figure 1(b)) is
employed to manually create realistic 3D cities (Liang et al. 2003). Building footprints from
cadastral survey can also be extruded into geometrically simple polyhedra (Figure 1(a)),
though with a lack of textural details. LiDAR has become a popular technology for
obtaining accurate 3D geometric information (Rottensteiner and Briese 2002), yet the
lack of textural information can be overcome only through aerial photography.
Oblique airborne photogrammetry is a recently developed solution for rapid and
accurate 3D city reconstruction at an acceptable cost. It has become an increasingly
mature means for 3D city acquisition with the rapid progress in data acquisition and
data processing technologies. Affordable unmanned airborne vehicles and sensors can
now be easily integrated into a robust system to acquire multi-angle oblique imagery
(Remondino et al. 2011). Compared to traditional image acquisition systems, unmanned
airborne vehicles have demonstrated many advantages such as low cost, high flexibility,
and fine image resolution. Photo-based automatic 3D city reconstruction, which is
driven primarily by computer vision research, has been standardized and streamlined
by leading industry solution providers. A commercial 3D reconstruction solution, such as
Acute3D’s Smart3D, Airbus’s Street Factory, or Skyline’s Photomesh, can now
Figure 1. Different sources of 3D city model: (a) building footprint extrusions, (b) CAD modeling, and
(c) OAP3D.
2 J. LIANG ET AL.

automatically generate a highly realistic 3D urban landscape (Figure 1(c)) from an
unorganized set of overlapping photos, without requiring any expert knowledge of
computer vision or GIS.
Several leading 3D reconstruction solution providers, such as Acute3D (acquired by
Bentley in January 2015), Airbus, and Skyline, have implicitly converged on a shared
multi-resolution data model for efficient storage, streaming, and rendering of OAP3D.
This data model is becoming a de facto industry standard widely accepted by relevant
solution providers and application developers. An OAP3D exported from any of the
three solutions can be imported directly into the open-source 3D rendering engine
OpenSceneGraph for interactive navigation with built-in support for out-of-core data
paging. Similarly, Skyline offers seamless integration of OAP3D to its commercial virtual
globe platform Skyline Globe. With these sophisticated data acquisition, data processing,
and data visualization solutions, a solid foundation has already been established for
OAP3D to be exploited in 3D urban GIS applications in fields such as urban planning,
traffic management, cadastral survey, and emergency management. These are key
components in developing next-generation smart cities (Tao 2013) to provide enhanced
urban services.
While the multi-resolution data model shared by the leading OAP3D solution provi-
ders has been highly optimized for data streaming and interactive visualization, it has
not yet been sufficiently prepared for use in real-world 3D urban GIS applications. A 3D
urban GIS application is usually built on the combination of multiple data sources, which
typically consist of imagery, elevation, road networks, stream networks, points of inter-
est, and other user-generated features. Integrating these data sources with an OAP3D
can be challenging, since mesh re-triangulation (Vivoni et al. 2005) or data conflation
issues (Kreveld and Silveira 2011) might be involved. To begin with, we introduce a few
common scenarios in which such data integration issues would be faced:
(1) Overlaying manually created 3D building models over an OAP3D (Figure 2). Urban
planners sometimes might need to replace a city block within an OAP3D with a
conceptual design exported from CAD. Urban residents who intend to build or
rebuild a house might also want to merge an architectural design into an OAP3D
background to help aesthetic quality assessment.
(2) Removing trees from an OAP3D (Figure 3). As the structure of a tree is geome-
trically very complex, it cannot normally be reconstructed using aerial photo-
grammetry alone at the same level of quality as a building is done. As a result, it
might sometimes be necessary to remove the areas covered by trees to make
room for artificially created objects of better visual quality.
(3) Merging user-generated roadways or structures into an OAP3D (Figure 3). Streets
may be partially hidden by tree canopies, leading to bumpy surfaces. Road
widening or new road construction work is not uncommon in cities that are
experiencing rapid urban development and population increase. Underground
civil engineering may also necessitate modification to OAP3D.
While robust industrial solutions for acquisition and distribution of OAP3D has been
established, directly utilizing such data in 3D urban GIS applications would still be
challenging. Hence, it is arguably necessary to develop a practical framework for easily
INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE 3

integrating OAP3D into 3D urban GIS applications with support for user-generated
content. This framework is intended to facilitate utilization of the increasingly large
volume of OAP3D data, which typically constitute the foundation of a 3D urban GIS
application.
This paper is organized as follows. Section 2 provides a detailed description of the
multi-resolution data structure shared by the leading industrial solution providers.
Section 3 presents the conceptual framework and its implementation for modifying
Figure 2. Potential need to replace individual building in OAP3D.
Figure 3. Potential need to remediate road surface that is covered by tree canopies in OAP3D.
4 J. LIANG ET AL.

OAP3D and incorporating user-generated content. Section 4 presents some application
examples to demonstrate the applicability of this framework. Section 5 summarizes the
contributions of the presented method.
2. The shared data model
Manual 3D modeling in CAD remains a mainstream strategy for 3D city reconstruction. A
manually created 3D city is typically composed of a group of building models accom-
panied with a variety of natural or artificial landscape features, for example, water
bodies, trees, and street lights. These 3D models are normally represented by an array
of triangular meshes associated with material colors or textures. Since a 3D city can
encompass thousands of buildings and a large number of accessories, a consumer-level
computer may not be able to accommodate the whole city for real-time rendering. To
overcome this bottleneck, the CityGML data model was introduced (Kolbe et al. 2005)
and later approved by the Open Geospatial Consortium. In a CityGML-based 3D city,
building objects can have multiple levels of details. With these multi-resolution building
models and a reasonable view-dependent data streaming strategy, users can smoothly
explore a large-scale 3D city on a computer of limited power. One of the major
disadvantages accompanying CityGML is that the multi-resolution data content of a
3D city requires a substantial amount of labor to create.
Unlike traditional manual 3D modeling which is labor intensive, OAP3D can reduce
human intervention to an affordable level by means of unmanned airborne vehicle-
based image acquisition and photo-based 3D reconstruction. A commercial 3D recon-
struction solution can automatically transform a set of non-georeferenced oblique
images into a spatially continuous triangular mesh that encompasses the ground,
buildings, plants, and everything else that is visible from an unmanned airborne vehi-
cle-borne camera.
Due to the huge volume of overlapping images captured from unmanned airborne
vehicles for 3D reconstruction, a large urban area usually needs to be spatially parti-
tioned into a grid of cells or sub-extents, each of which is utilized to extract a subset of
images to generate a continuous triangular mesh bounded by it, and the sub-extent
should be limited in space so that the data volume of the subset of images inside may
not overburden the 3D reconstruction software or the hardware system on which it is
running. As such a continuous 3D mesh are densely loaded with geometric and textural
information, a multi-resolution data model must be used in order to facilitate interactive
exploration. The CityGML data model, however, is not directly applicable to these
meshes because they cannot automatically be segmented into individual buildings,
which are fundamental elements comprising a CityGML city.
Across an OAP3D, a separate tree-based level-of-detail structure is used for each sub-
extent, in which a large continuous mesh is partitioned into a given number of sub-tiles
at each tree level, with each level hosting progressively downsampled geometric and
textural data from the bottom up (Figure 5). Structurally similar to an image pyramid,
this type of level-of-detail mesh actually functions like a ‘mesh pyramid’. Normally, the
top tree level is split into only one or a few sub-tiles that are in the most simplified form
across the whole range of tree levels. From the top down, each parent tile is further
divided in one or more sub-tiles with higher resolution meshes and textures present

(Figure 5). Consequently, the bottom level has the largest number of sub-tiles, which are
leaf nodes loaded with the most finely scaled meshes and textures commensurate with
the original data resolution.
This data model is highly optimized for real-time rendering and data streaming. A 3D
GIS application can even dynamically manage terabytes of OAP3D data for interactive
visualization without causing a significant decrease in rendering performance.
To demonstrate how this data model performs in 3D urban applications with massive
city models, a test was performed with an OAP3D that covers a downtown area of
45 km2
with a data volume of approximately 64.4 GB. This OAP3D was generated using
Skyline’ Photomesh with an image resolution of approximately 10–20 cm. The dataset
was loaded into a 3D interactive visualization system on a consumer-level desktop
computer, which has a 2.90-GHz quad-core Intel Core i52310 CPU with 4 GB RAM and
a NVIDIA GeForce GTS 450 graphics card with 1.0 GB RAM. The real-time rendering
performance and quality was evaluated at a progressively closer range of viewing
distance. As shown in Figure 4, the level-of-detail transition from far away to close up
was smooth and nearly seamless, and the rendering performance at any of the four
viewing positions was maintained between 90 and 140 frames per second with an
average of 120, which is sufficient for most 3D GIS applications to accommodate other
GIS-related functionalities.
The standardized data model may have been sufficient for serving a visualization-
centered 3D GIS application that utilizes OAP3D as a standalone data source, but it is not
well prepared for use in a 3D urban GIS application that requires integration of OAP3D
and other types of data sources.
Modifying OAP3D meshes in CAD is very complicated and thus time consuming.
Because an OAP3D comprises a grid of sub-extents with independent mesh pyramids, a
building may straddle multiple neighboring mesh pyramids across an OAP3D or multiple
Figure 4. Smooth level-of-detail transition of a massive OAP3D from far away to close up in real-
time rendering at an average of 120 frames per second.
6 J. LIANG ET AL.

neighboring mesh tiles within a single mesh pyramid (Figure 5). To edit a straddling
building in CAD, these neighboring meshes must be separately processed in a way that
they can be seamlessly stitched back together. To avoid this challenge that cannot be
efficiently managed in processing large-area or geometrically complex features, such as
roadways, streams, forested land, and buildings, we propose that a GIS-integrated 2D
editing approach be used.
3. Method
The shared data model of OAP3D is characteristic of both 2D DSMs and 3D triangulated
meshes in that, for a top-down view it resembles an urban height field overlaid with a
digital orthophoto map (DOM), while for a side view it becomes a 3D mesh model with
multi-angular textural information. Inspired by this understanding, we propose a new
method that reduces the complexity of OAP3D modification from 3D mesh editing to 2D
raster editing (Figure 6). The method is based on two assumptions: (1) an OAP3D mesh
can be reformed as a 2D height field by updating the height value of each vertex, as in
most cases we do not have to view a target area as a static 3D mesh. For example, to
replace an existing 3D building from an OAP3D with a newly designed model, we need
only to lower the height field of the ROI to the ground level; (2) user-defined objects, for
example, 3D buildings, trees, or road segments can be generated externally and then
seamlessly merged into the reshaped mesh area at a later stage.
3.1. Orthoimagery generation
To enable surface elevation and texture editing in a 2D graphic environment, for
example, in ArcGIS or Photoshop, an OAP3D (Figure 7(a)) needs to be transformed
Figure 5. An OAP3D is partitioned into a grid of mesh pyramids to support level-of-detail interactive
visualization.

into a DOM (Figure 7(b)) and a DSM (Figure 7(c)). In computer graphics, there is a
technique known as ‘Render to Texture’. A CAD software application, for example, 3ds
Max, may be able to utilize this technique to render 3D models onto a 2D image based
on a pre-conﬁgured virtual camera, and if the virtual camera is set to top-down view
Figure 6. Workﬂow for integrating user-generated content into OAP3D.
Figure 7. Generating DOM and DSM from OAP3D to facilitate 2D editing in a GIS.
8 J. LIANG ET AL.

with an orthographic projection, the 2D image mosaic produced may serve as a DOM.
There are several GIS-related issues, however, these are not considered in CAD software.
For example, CAD software cannot normally accommodate the level-of-detail data
structure of an OAP3D and thus may not be able to handle the large data volume
associated with such a 3D city. Moreover, CAD software such as 3ds Max is not able to
generate a DSM, because the ‘Render to Texture’ technique in CAD is intended only for
producing colored images. Inspired by the ‘Render to Texture’ technique used in CAD,
OpenSceneGraph is used here as a 3D rendering engine for transforming OAP3D into 2D
orthoimages including DOM (Figure 7(b)) and DSM (Figure 7(c))
The OpenSceneGraph engine is widely used by application developers in fields
such as visual simulation, games, virtual reality, scientific visualization, and modeling.
With an application developed under the framework of OpenSceneGraph, the geo-
metric and textural data are dynamically loaded from disk files or web streams into
system memory, and then transferred to video memory for rendering by graphics
processing units.
Similarly, a large urban scene also needs to be spatially partitioned into a grid of sub-
extents before it is rendered onto images due to texture size and video memory
limitations, although this space partitioning may be independent of the internal grid
structure of the OAP3D being processed. An image of a given width and height is
allocated for each sub-extent, and this image is defined as a sub-image, which covers a
sub-extent of the full scene area. For instance, given a scene area of 2 × 2 km2
, a spatial
resolution of 10 cm and a sub-image size of 1024 × 1024, approximately 200 × 200 sub-
images would be required to fill the grid of the scene.
Normally, OpenSceneGraph outputs its rendering results as 3-byte or 4-byte colors
to the screen for immediate display. Therefore, the Render Target technique and the
OpenGL shading language are employed to withhold the color and position data
from the graphics processing unit buffers for user-defined output. A Render Target
Texture is allocated to store the 3-byte color data for the DOM and an additional
floating point Render Target Texture is used for the DSM. The two Render Target
Textures are attached to an orthographic camera for storing the DOM and DSM. In
the shading fragment program, the color data are written to the DOM render target
(Figure 7(b)) and the floating point height data are written to the DSM render target
(Figure 7(c)).
3.2. ROI delineation
An ROI is a polygon which serves to define the boundary of an area that is intended to
be modified (Figure 8). For example, if a lake needs to be removed from an OAP3D, its
boundary will be manually delineated to provide reference for the following surface
elevation or texture editing.
3.3. DSM modification
OAP3D meshes can be reformed using a 2D approach, which can conveniently be
performed in a GIS environment. A DSM can be modified using two types of techniques,
which include:

(1) Surface elevation sampling and spatial interpolation. To erase an above-ground
object from an OAP3D, surface elevations at the exposed ground immediately
surrounding this object are sampled and then spatial interpolation is applied to
reconstruct the ground surface hidden under this object. To flatten the surface of a
water body, which was distorted due to lack of feature points in the process of 3D
reconstruction, elevation points that are considered within a reasonable range of
height values may be sampled for spatial interpolation. The values of these elevation
samples may also be interactively adjusted in GIS based on prior knowledge.
(2) Flattening, which refers to applying a uniform elevation value to form a planar
surface. In some cases, the surface that is covered by an ROI can be assumed to be
flat. For example, a lake is normally a planar surface. The flattening technique may
also be utilized to place a building on an undulating surface.
3.4. DOM modification
An OAP3D is a complex landscape comprising bare earth, buildings, water bodies, roads,
and vegetation, which are represented through a combination of geometric and textural
information. However, certain types of landscape elements can be sufficiently character-
ized by texture alone, for example, water surfaces. Therefore, modifying the DOM alone
through 2D texture editing may effectively change the way a landscape element
appears on an OAP3D. Image processing software such as Photoshop can be employed
to edit a texture cropped from a DOM using ROI masking.
Figure 8. Delineating ROI in a GIS based on the DOM and DSM generated from the OAP3D.
10 J. LIANG ET AL.

3.5. Overlaying user-generated 3D models on ROI
DSM modification can be conveniently performed in a GIS to reform an OAP3D mesh
from a 2D perspective. In real-world applications, however, a user may intend to overlay
onto an OAP3D externally sourced 3D models, for example, building models that are
manually created in CAD or procedurally generated using certain rules according to
urban designs.
3.6. Dynamic surface elevation and texture updating
The modified DOM and DSM will eventually need to be embodied in the OAP3D during
interactive rendering. Since these types of DOM and DSM are usually too large to fit into
video memory for real-time rendering, we propose that a DOM and DSM image pyramid
structurally identical to the mesh pyramid of each sub-extent be used. Upon loading a
mesh tile from a mesh pyramid, the corresponding DSM tile and DOM tile are retrieved
from the image pyramid. The DSM is utilized to change the height values of the mesh
tile and the DOM tile serves as an extra texture layer to override the original pixels
where modifications have been made.
Since an independent DOM and DSM image pyramid is used for dynamic updating,
no change needs to be made to the original OAP3D. Therefore, we can maintain
multiple versions of image pyramids to keep track of changes as well as for inter-
comparison, which may potentially benefit urban planners who are seeking optimized
solutions.
4. Application examples
4.1. Road surface remediation
In this OAP3D, some roads and streets are partially covered by dense tree canopies
(Figure 9). Tree removal, surface flattening, and texture painting were considered
necessary to create smoother and more realistic road surfaces, which are important for
certain purposes, for example, realistic simulation of vehicles or pedestrians navigating
the streets.
Figure 9. Road surface covered by tree canopies in the original OAPD3D.

ROI polygons were delineated in a GIS to include the road pixels that were
covered by tree canopies. Surface elevation samples were also extracted from a GIS
for the purpose of reconstructing the ROI surfaces. As the ground elevations beneath
an above-ground object are unknown, we assume that the ground elevation of a
point at the object can be estimated by a group of known ground points from the
exposed ground in the immediate vicinity. Elevation samples were evenly distributed
around the ROI polygons and restricted to flat road surfaces that were not covered
by trees. Inverse distance weighting interpolation was employed to produce a DSM
using these elevation samples. The DSM was processed using a moving average filter
to make the road surfaces smoother (Figure 10). The DOM was then imported into
Photoshop for texture editing. In Photoshop, the road pixels inside the ROI polygons
were erased and replaced with texture patterns that were visually consistent the
surrounding areas.
Figure 11 shows that the visual quality of the road surfaces have been considerably
improved due to the flattening of the areas that were previously bulging outwards
because of the presence of trees, and the realistic texture patterns that were artistically
painted over the tree pixels.
4.2. Replacement of above-ground objects for urban planning
When viewed from a distance, an OAP3D may appear highly realistic without noticeable
artifacts. When viewed at a closer range, however, buildings may begin to show bulges,
depressions, or other distortions. Moreover, buildings in OAP3D typically are represented
as a meshed shell without any internal structure, which can be important for certain
urban applications, for example, building information modeling, architectural design,
indoor mapping, and navigation. In an OAP3D, tree branches and leaves are barely
Figure 10. Smooth road surface reconstructed using non-tree elevation samples in a GIS.
12 J. LIANG ET AL.

separable and therefore a tree is usually rendered in the shape of a fully closed ellipsoid
with a bumpy surface.
Due to these technical imperfections and limitations facing OAP3D, users may
sometimes intend to locally replace these areas with externally sourced 3D models.
There is also a need from urban land redevelopment to reclaim old city blocks. The
workflow for integrating 3D models into OAP3D (Figure 12) are similar to that used
in the road surface remediation, the only difference is that DOM reconstruction using
spatial interpolation may be spared since the ROI is to be covered by externally
created 3D models.
Figure 11. Improved road surface with modified DSM and DOM from a GIS.
Figure 12. Workflow for replacing individual building: (a) original building, (b) ROI delineation and
DSM editing in GIS, (c) building removed, and (d) new building placed at the same location.

4.3. Surface modification for land repurposing
In Figure 13(a), a land lot on the edge of the urban area was defined using a polygon of
yellow borders. Elevation samples were selected in a GIS to reform the topography of
the polygonal area. Inside the polygon, elevation values at the sample points were
manually controlled to achieve a desired topography. Outside the polygon, elevation
values at the sample points were derived directly from the DSM in order to obtain a
gradual sloping effect. Spatial interpolation was performed using a combination of the
elevation samples from inside and outside the ROI polygon. In the first example, a
uniform height was assigned to the evenly distributed samples within the polygon to
flatten the ground surface, which was decorated with grass texture patterns to show
that the land lot has been repurposed for greenery (Figure 13(a)). In the second
example, several elevation samples were assigned significantly higher values to serve
as mountain peaks for the artificially generated mountainous area (Figure 13(b)). In the
third example, the land lot was repurposed as a water body, which can serve as a lake or
a reservoir. To approximate the topography of a water body, significantly lower eleva-
tion values were applied inside the polygon (Figure 13(c)).
5. Discussion and conclusion
We have presented a method for embedding user-generated content into OAP3D in
three steps, that is, ROI delineation, DSM modification, and external content embed-
ding. The technical framework described in Section 3 and the application examples
presented in Section 4 have shown that this method can: (1) support easy modifica-
tion of OAP3D’s geometric content through 2D GIS operations; (2) support easy
modification of OAP3D’s textural content through 2D image editing; and (3) dynami-
cally apply the modifications during real-time rendering without changing the origi-
nal OAP3D.
Figure 13. Land lot repurposing: (a) original land lot, (b) surface reconstruction in GIS, (c) repurpos-
ing for greenery, (d) repurposing as mountains, and (e) repurposing as a water body.
14 J. LIANG ET AL.

With the rapid development of unmanned airborne vehicle technology, OAP3D data
will increasingly be accumulated. Microsoft Bing Maps (http://www.bing.com/dev/en-us/
maps-preview-app) and Google Earth (https://www.google.com/earth/) have acquired
and published hundreds of OAP3D-based virtual cities for augmenting the present 3D
urban application, which traditionally relied mainly on high resolution imagery with few
3D building models. This paradigm shift in 3D urban application was likely driven by the
obvious advantages of OAP3D in terms of data accuracy, real-time rendering perfor-
mance, and data acquisition efficiency. Many smaller businesses have also built up the
capacity to operate unmanned airborne vehicles and acquired OAP3Ds for commercial
services. The method presented in this paper can be very helpful in developing OAP3D-
based 3D urban GIS applications, since it allows user’s ideas and needs to be embodied
in the static city models. With this method, the time and labor costs that would
potentially be required to bring user-generated content into an OAP3D can be signifi-
cantly reduced, because the 2D-3D mapping approach is as straightforward as drawing
on a canvas.
There are three implications of the presented method for application of OAP3D in
GIS. (1) Data integration is one of the most challenging issues in fully exploiting the
value of OAP3D in GIS. Our method offers a cost-effective solution to address this issue
and has the potential to promote the use of OAP3D in a broader and deeper manner. (2)
It is a systematically designed approach, which can potentially inspire researchers from
the academia and product developers from the industry to develop a fully integrated
solution. (3) User-generated content is a very important data source for augmenting
urban GIS applications. Although remotely sensed data can faithfully and accurately
capture the present state of a city, the past and future can be created only through the
knowledge and imagination of urban planners, artists, and decision-makers, who are in
need of an easy tool to help embed their ideas into OAP3D.
Acknowledgments
This research was supported and funded by the Foundation for Young Scientists of the State
Key Laboratory of Remote Sensing Science (15RC-08), the Key Knowledge Innovative Project
of the Chinese Academy of Sciences (KZCX2 EW 318), the National Key Technology R&D
Program of China (2014ZX10003002), and the National Natural Science Foundation of China
(41371387).
Disclosure statement
No potential conflict of interest was reported by the authors.
Funding
This research was supported and funded by the Foundation for Young Scientists of the State Key
Laboratory of Remote Sensing Science: [grant number 15RC-08]; the Key Knowledge Innovative
Project of the Chinese Academy of Sciences: [grant number KZCX2 EW 318]; the National Key
Technology R&D Program of China: [grant number 2014ZX10003002]; and the National Natural
Science Foundation of China: [grant number 41371387].

ORCID
Jianming Liang http://orcid.org/0000-0002-4043-6816
References
Döllner, J., Baumann, K., and Buchholz, H., 2006. Virtual 3D city models as foundation of complex
urban information spaces. In: Proceedings of CORP 2006 & Geomultimedia06, February, Vienna,
107–112. Available from: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.83.
4098&rep=rep1&type=pdf [Accessed April 2016].
Kolbe, T.H., Gröger, G., and Plümer, L., 2005. CityGML: interoperable access to 3D city models. In: P.
van Oosterom, S. Zlatanov, and E.M. Fendel, eds. Geo-information for disaster management.
Berlin: Springer-Verlag, 883–899.
Köninger, A. and Bartel, S., 1998. 3D-GIS for urban purposes. Geoinformatica, 2 (1), 79–103.
doi:10.1023/A:1009797106866
Kreveld, M.V. and Silveira, R.I., 2011. Embedding rivers in triangulated irregular networks with
linear programming. International Journal of Geographical Information Science, 25 (4), 615–631.
doi:10.1080/13658816.2010.488240
Liang, J., et al., 2015. An open-source 3D solar radiation model integrated with a 3D Geographic
Information System. Environmental Modelling & Software, 64, 94–101. doi:10.1016/j.
envsoft.2014.11.019
Lin, H., Chen, M., and Lu, G., 2013. Virtual geographic environment: a workspace for computer-
aided geographic experiments. Annals of the Association of American Geographers, 103 (3), 465–
482. doi:10.1080/00045608.2012.689234
Moser, J., Albrecht, F., and Kosar, B., 2010. Beyond visualisation–3D GIS analyses for virtual city
models. International Archives of the Photogrammetry, Remote Sensing and Spatial Information
Sciences, 38 (4), W15.
Remondino, F., et al., 2011. UAV photogrammetry for mapping and 3d modeling–current status
and future perspectives. International archives of the photogrammetry. Remote Sensing and
Spatial Information Sciences, 38 (1), C22.
Rottensteiner, F. and Briese, C., 2002. A new method for building extraction in urban areas from
high-resolution LIDAR data. International Archives of Photogrammetry Remote Sensing and
Spatial Information Sciences, 34 (3/A), 295–301.
Strzalka, A., et al., 2011. 3D city modeling for urban scale heating energy demand forecasting.
HVAC&R Research, 17 (4), 526–539. doi:10.1080/10789669.2011.582920
Tao, W., 2013. Interdisciplinary urban GIS for smart cities: advancements and opportunities. Geo-
spatial Information Science, 16 (1), 25–34. doi:10.1080/10095020.2013.774108
Vivoni, E.R., et al., 2005. Embedding landscape processes into triangulated terrain models.
International Journal of Geographical Information Science, 19 (4), 429–457. doi:10.1080/
13658810512331325111
Wu, H., He, Z., and Gong, J., 2010. A virtual globe-based 3D visualization and interactive framework
for public participation in urban planning processes. Computers, Environment and Urban
Systems, 34 (4), 291–298. doi:10.1016/j.compenvurbsys.2009.12.001
Yasumoto, S., et al., 2012. Virtual city models for assessing environmental equity of access to
sunlight: a case study of Kyoto, Japan. International Journal of Geographical Information Science,
26 (1), 1–13. doi:10.1080/13658816.2011.570268
Yasumoto, S., et al., 2011. The use of a virtual city model for assessing equity in access to views.
Computers, Environment and Urban Systems, 35 (6), 464–473. doi:10.1016/j.
compenvurbsys.2011.07.002
16 J. LIANG ET AL.

Embedding user generated content into oblique airborne photogrammetry based 3D city model

More Related Content

What's hot

Similar to Embedding user generated content into oblique airborne photogrammetry based 3D city model

Embedding user generated content into oblique airborne photogrammetry based 3D city model