In this paper, we present an approach for out-of-core dynamic streaming of a virtual scene. We use the traditional client/server architecture, where the main responsibility of the client application is visualization and interaction, and the main responsibility of the server is accepting and serving requests from the client application. The client will query geometry in the camera proximity to the server and the server will stream geometry (in CityGML format) to the client application. We implement the client application using Unity game engine. Performance comparison between traditional loading and our dynamic streaming are provided. We also show the scalability advantage of our work. Was Presented in Cyberworlds 2018, Singapore.

1. Cloud-Based Dynamic
Streaming and
Loading of 3D Scene
Budianto Tandianus1 , Hock Soon Seah1
Tuan Dat Vu2, Anh Tu Phan2
1. Multi-plAtform Game Innovation Centre,
Nanyang Technological University, Singapore
2. School of Information and Communication Technology,
Hanoi University of Science and Technology, Vietnam

2. Introduction
Source:
https://youtu.be/NYNRjcjL0eM

3. Introduction
• Visualization of very large scene using out-of-core approach
– Client application does not have enough resource to load all data
• Aim for scalability
• Cloud-based geometry loading approach
– Geometry is stored in cloud
– Geometry is streamed and loaded on-demand
• Other approaches:
– Cloud rendering : Deng et al. [1]
– Out-of-core CPU to GPU : Crassin et al. [3]

4. Introduction
• Basic client-server communication
– Client sends request to server to query for nearby buildings
• Information sent: position, location
– Server returns geometry
• Client receives the geometry and import it to the scene
– Client implemented in Unity
– Geometry in CityGML format
• Client discards geometry whose distance is greater than
specified radius

5. Contribution
• Scalable cloud-based urban simulation approach
– Can be used in various applications such as visualization and
games
– Designed to handle non-urban objects (e.g. vehicles and
humans), not only urban objects (e.g. buildings)
• We show the advantage of our approach compared to
traditional loading (in term of memory usage and rendering
performance)
– Able to handle very large scene (more than 1 million polygons)
without large drop in system performance.

6. Application
• Urban planning and development using Virtual Reality
– Visualize semantic and dynamic information
– Explore virtual world
• Seamless virtual simulation of an entire metropolis
• Beneficial for applications the require intuitive path planning and
interaction:
– Building and urban design visualization
– Evacuation planning
– Semantic crowd simulation
– Computer games

7. Related Work
• Local out-of-core storage approach
– Room subdivision : [4]
– Preprocessed assets such as in Unity and Unreal Engine [5-10]
• Adaptive loading
– based on current and previous state : He [19]
– based on geometry importance : Tian and AlREgib [11]
– based on viewing frustum : Zhi et al. [20]

8. Related Work
• Stream precomputed illumination and progressive mesh :
Pacanowski et al. [12]
• Regular scene subdivision : He [11] and Zhi et al. [15]
• Optimize texture streaming : Eu et al. [14] and Englert et al. [15]
• Attach geometry to other file format : Concolato et al. [16]
• Distributed P2P streaming: Wang et al. [17] and Jie et al. [18]

9. System Design
• Cloud Storage: store CityGML models
• Database: indexing and retrieving buildings
– Use PostgreSQL
– Future plan: non-urban objects such as animals and NPCs.
More flexibility in the future such as splitting objects.
• API Service: PHP and Yii2
• Internal Communication: RESTFUL APIs
– Send location to server and sever returns a set of unique IDs
of 3D objects within predefined radius
– IDs are used to retrieve geometry from cloud storage
• Renderer: Unity
– Use modified Jaeniche’s script for loading CityGML data

10. System Design
A cloud based system to
streamline data
management
Media
Transcoding & Streaming
Cross Platform Accessibility
Expandability via
Virtual User
Flexible User Role
Management
Efficient
Workflow
Management
Secure File
Sharing
Powerful Asset Management & Tracking Provide a more efficient
and collaborative platform

11. Experiment Setup
• Two datasets:
– Berlin : 17,122 polygons, 954 files, 38.2 MB
– Amsterdam : 1,221,379 polygons, 90,260 files, 906 MB
• Workstation specification:
– CPU : Intel(R) Xeon(R) CPU E5-2609 0 @ 2.4GHz 16 GB
RAM
– GPU : NVIDIA Quadro M4000 16GB
• Traditional Loading vs Dynamic Streaming
– Compared memory usage
– Frames-per-second
– Rendering time per-frame
Berlin Dataset
Amsterdam Dataset

12. Result
Berlin dataset
Traditional average: 43.78 MB
Dynamic average: 1.81 MB
Amsterdam dataset
Dynamic average: 1.62 MB

13. Result
Berlin dataset
Traditional average: 9.16 fps
Dynamic average: 30.54 fps
Amsterdam dataset
Dynamic average: 29.99 fps

14. Result
Berlin dataset
Traditional average: 103.16 ms
Dynamic average: 32.07 ms
Amsterdam dataset
Dynamic average: 32.83 ms

15. Result
Traditional Loading Dynamic Streaming

16. Result
Video
https://youtu.be/OPhbnDceDII

17. Conclusion
• Promising improvement using dynamic streaming
• Low memory usage
• High FPS

18. Future Work
• Asset allocation and deallocation based on prediction on player’s action
history
– Camera speed and orientation
• Storing precomputed illumination on hierarchical grid
• Perceptual-based optimization
– Progressive mesh refinement
– Texture streaming
• Large scene simulation with path planning
• Deploy to standalone VR device
– Oculus Go
– HTC Vive Focus

19. Acknowledgement
• This research is supported by the National
Research Foundation, Prime Minister’s Office,
Singapore under its IDM Futures Funding
Initiative.

20. The End