Successfully reported this slideshow.

Video Fields: Fusing Multiple Surveillance Videos into a Dynamic Virtual Environment

0

Share

1 of 83
1 of 83

Video Fields: Fusing Multiple Surveillance Videos into a Dynamic Virtual Environment

0

Share

Download to read offline

Paper and videos: http://www.videofields.com
Another interesting project: http://socialstreetview.com

Video Fields system fuses multiple videos, camera-world matrices from a calibration interface, static 3D models, as well as satellite imagery into a novel dynamic virtual environment. Video Fields integrates automatic segmentation of moving entities during the rendering pass and achieves view-dependent rendering in two ways: early pruning and deferred pruning. Video Fields takes advantage of the WebGL and WebVR technology to achieve cross-platform compatibility across smart phones, tablets, desktops, high-resolution tiled curved displays, as well as virtual reality head-mounted displays. See the supplementary video at http://video-fields.com.

Paper and videos: http://www.videofields.com
Another interesting project: http://socialstreetview.com

Video Fields system fuses multiple videos, camera-world matrices from a calibration interface, static 3D models, as well as satellite imagery into a novel dynamic virtual environment. Video Fields integrates automatic segmentation of moving entities during the rendering pass and achieves view-dependent rendering in two ways: early pruning and deferred pruning. Video Fields takes advantage of the WebGL and WebVR technology to achieve cross-platform compatibility across smart phones, tablets, desktops, high-resolution tiled curved displays, as well as virtual reality head-mounted displays. See the supplementary video at http://video-fields.com.

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Video Fields: Fusing Multiple Surveillance Videos into a Dynamic Virtual Environment

  1. 1. Video Fields: Fusing Multiple Surveillance Videos into a Dynamic Virtual Environment Ruofei Du, Sujal Bista, Amitabh Varshney The Augmentarium| UMIACS | University of Maryland, College Park {ruofei, sujal, varshney} @ cs.umd.edu www.VideoFields.com
  2. 2. image courtesy: university of maryland, college park Introduction Surveillance Videos - Monitoring
  3. 3. image courtesy: www.icsc.org Introduction Surveillance Videos – Shopping Centers
  4. 4. image courtesy: wikipedia Introduction Surveillance Videos - Airports
  5. 5. image courtesy: wikipedia Introduction Surveillance Videos – Train stations
  6. 6. image courtesy: university of maryland, college park Introduction Surveillance Videos - Campuses
  7. 7. image courtesy: university of maryland, college park Introduction Surveillance Videos - Conventional
  8. 8. image courtesy: theimaginativeconservative.org Introduction Surveillance Videos – Cognitive Burden
  9. 9. image courtesy: university of maryland, college park Introduction Surveillance Videos – Fusing & Interpreting
  10. 10. Related Work Fusing Multiple Static Photographs
  11. 11. Related Work Fusing Multiple Static Photographs
  12. 12. Related Work Fusing Multiple Static Photographs
  13. 13. Related Work Fusing Multiple Static Photographs
  14. 14. Related Work Fusing Multiple Static Photographs
  15. 15. Related Work Fusing Multiple Dynamic Videos
  16. 16. Related Work Fusing Multiple Dynamic Videos RGB
  17. 17. Related Work Fusing Multiple Dynamic Videos RGB RGBD
  18. 18. Related Work Fusing Multiple Dynamic Videos
  19. 19. Related Work Fusing Multiple Dynamic Videos
  20. 20. Related Work Fusing Multiple Dynamic Videos
  21. 21. Related Work Fusing Multiple Dynamic Videos
  22. 22. Related Work Fusing Multiple Dynamic Videos
  23. 23. Related Work Fusing Multiple Dynamic Videos
  24. 24. Related Work Fusing Multiple Dynamic Videos
  25. 25. Related Work Fusing Multiple Dynamic Videos
  26. 26. Related Work Fusing Multiple Dynamic Videos SIGGRAPH 2016 Wednesday, 3:30-4:00 PM
  27. 27. Related Work Fusing Multiple Dynamic Videos
  28. 28. Related Work Fusing Multiple Dynamic Videos
  29. 29. Related Work Fusing Multiple Dynamic Videos
  30. 30. Our Approach?
  31. 31. Video Fields
  32. 32. Video Fields
  33. 33. Introduction Video Field
  34. 34. Introduction Video Field
  35. 35. Conception, architecting & implementation Video Fields A mixed reality system that fuses multiple surveillance videos into an immersive virtual environment,
  36. 36. Integrating automatic segmentation of moving entities Video Fields Rendering Real-time fragment shader processing
  37. 37. Two algorithms to fuse multiple videos Early & deferred pruning These methods use voxels and meshes respectively to render moving entities in the video fields
  38. 38. Achieving cross-platform compatibility by WebGL + Three.js smartphones, tablets, desktop, high-resolution large-area wide field of view tiled display walls, as well as head-mounted displays.
  39. 39. System Overview
  40. 40. Architecture Video Fields Flowchart
  41. 41. Architecture Video Fields Flowchart
  42. 42. Architecture Video Fields Flowchart
  43. 43. Architecture Video Fields Flowchart
  44. 44. Background Modeling Motivation • Provide a background texture for each camera • Identify moving entities in the rendering stage • Reduce the network bandwidth requirements
  45. 45. Background Modeling Gaussian Mixture Models (GMM)
  46. 46. Background Modeling Advantages [Stauffer and Grimson] More adaptive with: • different lighting conditions, • repetitive motions of scene elements, • moving entities in slow motion
  47. 47. Architecture Video Fields Flowchart
  48. 48. Segmentation Moving Entities
  49. 49. Background Modeling Gaussian Mixture Models (GMM)
  50. 50. Architecture Video Fields Flowchart
  51. 51. Visibility Test Plus Opacity Modulation
  52. 52. Architecture Video Fields Flowchart
  53. 53. Video Fields Mapping Overview
  54. 54. Video Fields Mapping Challenges 1. Vertex in the 3D models -> Pixel in the texture space 2. Pixel in the texture space -> Vertex on the ground • The second is useful for projecting a 2D segmentation of a moving entity to the 3D world
  55. 55. Video Fields Mapping Projection Mapping
  56. 56. Video Fields Mapping Perspective correction
  57. 57. Video Fields Mapping Depth Map / Hashing Function
  58. 58. Early Pruning for Rendering Moving Entities Voxels
  59. 59. Deferred Pruning for Rendering Moving Entities Billboards
  60. 60. Visual Comparison Early Pruning vs. Deferred Pruning
  61. 61. View-dependent Rendering
  62. 62. View-dependent Rendering
  63. 63. View-dependent Rendering
  64. 64. View-dependent Rendering
  65. 65. Experimental Results Early Pruning vs. Deferred Pruning
  66. 66. Experimental Results Early Pruning vs. Deferred Pruning
  67. 67. Experimental Results Early Pruning vs. Deferred Pruning
  68. 68. Visual Comparison Early Pruning vs. Deferred Pruning
  69. 69. Future Work Scale Up - Hundreds of cameras
  70. 70. Future Work Bandwidth Problem
  71. 71. Future Work Holoportation with RGB cameras
  72. 72. Acknowledgement Augmentarium Lab | GVIL | UMIACS
  73. 73. Acknowledgement NSF | Nvidia | MPower | UMIACS
  74. 74. Video Fields www.Video-Fields.com Thank you! Questions or comments? Ruofei Du and Amitabh Varshney Augmentarium Lab | GVIL | UMIACS Web3D 2016

×