Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CrowdMap: Accurate Reconstruction of Indoor Floor Plan from Crowdsourced Sensor-Rich Videos

645 views

Published on

paper pdf download link: http://ubisec.cse.buffalo.edu/files/crowdmap.pdf

CrowdMap: Accurate Reconstruction of Indoor Floor Plan from Crowdsourced Sensor-Rich Videos

  1. 1. CrowdMap: Accurate Reconstruction of Indoor Floor Plans from Crowdsourced Sensor-Rich Videos Si Chen, Muyuan Li, Kui Ren, Chunming Qiao Department of Computer Science and Engineering University at Buffalo – State University of New York
  2. 2. Page  2 Outline  1. Introduction  2. System Architecture and Design Details  3. Implementation and Evaluation  4. Future Work
  3. 3. Page  3 Outdoor Maps Google Map OpenStreetMap
  4. 4. Page  4 Indoor Maps (Floor Plans)  Unlike outdoor environment, acquiring digital indoor floor plan information is very challenging.  The state-of-the-art Google Indoor Maps only have 10,000 locations available worldwide, which is not in a position to compete with the total number of indoor environments around the world.
  5. 5. Page  5 Indoor Maps  The complexity of the indoor environment is the major obstacle to achieve ubiquitous coverage.  Existing centralized collection and on-site calibration techniques demand professional devices and multi-party coordination, which are time consuming, inconvenient and costly. Google Trekker http://www.navvis.lmt.ei.tum.de/about/
  6. 6. Page  6 Crowdsourcing http://cdn.thephonebulletin.com/wp-content/uploads/2013/01/smartphone-sensors.jpg Recently, the wide availability of smartphones and wearable devices (e.g. google glasses) equipped with built-in visual, acoustic and inertial sensors makes the mobile user easier than ever to devote themselves to contribute mobile data.
  7. 7. Page  7 Crowdsourcing http://senda.uab.es/node/15
  8. 8. Page  8 Indoor Floor Plan Reconstruction by Crowdsourcing There have been several studies trying to explore the possibility of using crowdsourced inertial sensory data to generate an indoor floor plan automatically. Alzantot, Moustafa, and Moustafa Youssef. "Crowdinside: automatic construction of indoor floorplans." Proceedings of the 20th International Conference on Advances in Geographic Information Systems. ACM, 2012.
  9. 9. Page  9 Drawback of Inertial Sensor Only Methods  However, current crowdsourcing floor plan reconstruction systems are not able to produce accurate enough results. – Most of existing indoor floor plan reconstruction systems heavily rely on inertial data. The premise of their work is that users would be able to move across all edges and corners in an indoor environment. • the edge of an indoor scene is usually blocked by furniture or other objects, • some restricted areas in an indoor environment are also inaccessible for most of the users Visual information preserve more context information for an unknown indoor environment, such as the geometric information, color information, lighting conditions and text information. [1].http://www.sigmobile.org/mobicom/2014/talks/slides_6_3.pdf [1]
  10. 10. Page  10 CrowdMap: Key Idea  CrowdMap: An accurate indoor floor plan reconstruction system based on sensor-rich videos.  Key idea: leverage the spatio-temporal relationship between each consecutive frame of the crowdsourced video.
  11. 11. Page  11 CrowdMap: Floor Plan Basic elements of building floor plan
  12. 12. Page  12 Outline  1. Introduction  2. System Architecture and Design Details  3. Implementation and Evaluation  4. Future Work
  13. 13. Page  13 CrowdMap: System Architecture mobile front-end cloud back-end i) crowdsourced data collection module ii) indoor path modeling module iii) room layout modeling module iv) floor plan modeling module
  14. 14. Page  14 CrowdMap: Crowdsourced Data Collection Module We assume that users actively get involved in the data collecting tasks. Example: a user opens our mobile application and inputs the floor number, starts capturing the room environment by spinning his/her body (SRS task); then, walks towards the hallway (SWS task). The walking distance |AB| is calculated by the step counting method. In addition, the direction change of each step ∆𝜔 is calculated by jointly using compass, gyroscope and accelerometer.
  15. 15. Page  15 CrowdMap: Crowdsourced Data Collection Module By using the inertial sensor data, we are able to reconstruct the trajectory of the user when they perform the SWS task.
  16. 16. Page  16 CrowdMap: Indoor Path Modeling Module How to aggregate multiple user trajectories and reconstruct the path of the floor?
  17. 17. Page  17 CrowdMap: Indoor Path Modeling Module [1].Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool. "Surf: Speeded up robust features." Computer vision–ECCV 2006. Springer Berlin Heidelberg, 2006. 404-417. We select the state-of-the-art SURF [1] algorithm to precisely match two candidate key-frames
  18. 18. Page  18 CrowdMap: Indoor Path Modeling Module Correct Matching
  19. 19. Page  19 CrowdMap: Indoor Path Modeling Module Incorrect Matching
  20. 20. Page  20 CrowdMap: Indoor Path Modeling Module  We use multiple key-frames to determine whether the two user trajectories can be merged.  If there is a match between the two trajectories generated in the same floor, there should be a common path between them in a high probability. Hence, we use the longest common subsequence to capture this notion. Where Ta and Tb are the two user trajectories with length of i and j, respectively. Parameter 𝛿 represents the maximum length difference between two user trajectories and 𝜀 is the distance threshold.
  21. 21. Page  21 CrowdMap: Indoor Path Modeling Module
  22. 22. Page  22 CrowdMap: Room Layout Modeling Module We utilize crowdsourced images to create the panorama for each place, and then use computer vision techniques to process the panorama, and thereby, generate the room layout.
  23. 23. Page  23 CrowdMap: Room Layout Modeling Module
  24. 24. Page  24 CrowdMap: Room Layout Modeling Module
  25. 25. Page  25 CrowdMap: Floor Plan Modeling Module
  26. 26. Page  26 CrowdMap: Floor Plan Modeling Module
  27. 27. Page  27 CrowdMap: Floor Plan Modeling Module
  28. 28. Page  28 Outline  1. Introduction  2. System Architecture and Design Details  3. Implementation and Evaluation  4. Future Work
  29. 29. Page  29 CrowdMap: Implementation
  30. 30. Page  30 CrowdMap: Implementation Screenshot for CrowdMap Mobile Frontend
  31. 31. Page  31 Microsoft Azure Platform
  32. 32. Page  32 CrowdMap: Evaluation
  33. 33. Page  33 CrowdMap: Evaluation
  34. 34. Page  34 Compare with Structure from Motion (SfM)
  35. 35. Page  35 Outline  1. Introduction  2. System Architecture and Design Details  3. Implementation and Evaluation  4. Future Work
  36. 36. Page  36 CrowdMap: Future Work We will focus on further processing of the room panorama to extract more context information of the room: • object detection • object recognition. We also plan to further study several issues: • user incentive mechanism • privacy preservation mechanism.
  37. 37. Page  37

×