We present an efficient resource scheduling scheme for out-of-core dynamic streaming of a 3D scene. The entire scene is stored in the cloud and relevant scene data are streamed to a client mobile device in real time based on a user-selected path in the 3D scene. We analyze the path data in order to yield efficient streaming of 3D urban objects. We compare streaming scheduling based on the user-selected path and on only the user's current location (i.e. without path lookahead) in terms of {\color{black}number of loaded objects}, rendering performance, and storage. The client application is implemented in Unity game engine and we perform experiment on an Android mobile device. Was presented in IWAIT-IFMIA 2019.

1. Efficient Scheduling for
Dynamic Streaming of 3D
Scene for Mobile Devices
Budianto Tandianus1 , Hock Soon Seah1
Tuan Dat Vu2, Anh Tu Phan2
1. Multi-plAtform Game Innovation Centre,
Nanyang Technological University, Singapore
2. School of Information and Communication Technology,
Hanoi University of Science and Technology, Vietnam

2. Introduction
• Visualization of very large scene using out-of-core approach
– Client application does not have enough resource to load all data
• Aim for scalability
– Support for mobile devices
• Cloud-based geometry loading approach
– Geometry is stored in cloud and it is streamed to client based on
proximity to camera
– However, objects with large file size may delay streaming of
other objects

3. Introduction
• Optimize streaming by analyzing the user-selected path
• Basic client-server architecture
– Client (mobile devices) sends starting and finishing points to server
– Server (storage and compute units) returns path and geometries along
the path
– The geometries are streamed according to the determined schedule
• Client receives the geometry and import it to the scene
– Client implemented in Unity
– Geometry in FBX format

4. Contribution
• Scalable and efficient scheduling scheme for cloud-based urban
simulation approach
– Support very large scene in mobile devices
– Can be used in various applications such as visualization and games
– Designed to handle non-urban objects (e.g. vehicles and humans), not only
urban objects (e.g. buildings)
• Realistic mobile walkthrough applications
– Car navigation
– Historical site walkthrough
– Large complex (e.g. university) navigation application for visitors

5. Related Work
• Precomputation approach
– Precompute geometry importance : Tian and AlREgib [1]
– Precompute illumination : Pacanowski et al. [2]
• Regular scene subdivision Wang et al. [3]
• Optimize texture streaming : Eu et al. [4] and Englert et al. [5]
• Network approach
– Attach geometry to other file format : Concolato et al. [6]
– Distributed P2P streaming: Wang et al. [7] and Jie et al. [8]

6. Proposed Method
• Data preprocessing :
– Each object (e.g. building) is a separate FBX file and it is processed into
an asset bundle
– Also obtain position and bounding box of each object
– The bounding boxes are grouped
• Path finding : Djikstra’s algorithm
– The grouped bounding boxes are treated as obstacles

7. Bounding Box Processing
Extracted bounding boxes. Grouped bounding boxes.

8. Path Finding
Path 1. Path 2.

9. Streaming scheduling
• Sample points at regular interval in the
path
• Define a rectangular loading area
centered at each point
• 3D objects fall into the loading area are
organized into a set and they are sorted
from largest to smallest in term of file
size
• The sets are combined sequentially

10. System Design
• Cloud storage : Internally developed cloud storage
• Mobile client : developed by using Unity
• PHP web server : Yii2
• Python service : Process bounding box by using Shapely. Also, use
pyvisgraph to create visibility graph for path finding
• RabbitMQ : message broker

11. System Design

12. Experiment Setup
• NTU dataset
– 9,629 bundles
– Total 932 MB
• Mobile device specification:
– Samsung Galaxy S7 Edge with Exynos 8890 CPU and 4GB RAM
• Compare with and without scheduling
– With scheduling: use RabbitMQ and scheduling algorithm
– Without scheduling: do not use RabbitMQ and scheduling algorithm.
Assets are transferred directly from server to mobile client. Client
always requests to server objects within vicinity.

13. Result

14. Result

15. Result
With scheduling. Without scheduling.

16. Result
Video
https://youtu.be/aW6h5uDWPpg

17. Conclusion
• Scalable
• Mitigate late-loading of 3D models
• Rendering time advantage
• Scheduling method will consume larger memory only
during earlier traversal

18. Future Work
• Consider varying camera speed and orientation
• Consider perceptual factor
• Progressively stream vertices based on mesh refinement method
[2].
• Optimize texture by using perceptual-guided refinement strategy [4]
• Upscale the test data and using more robust path-planning method,
e.g. subregion graph [13]
• Test on standalone VR devices
– HTC Vive Focus
– Oculus Quest

19. Acknowledgement
• This research is supported by the National
Research Foundation, Prime Minister’s Office,
Singapore under its IDM Futures Funding
Initiative.

20. The End