Crowdsourcing the Acquisition and Analysis of Mobile Videos for Disaster Response
•Download as PPTX, PDF•
1 like•466 views
Efficient and thorough data collection and its timely analysis are critical for disaster response and recovery in order to save people's lives during disasters. However, access to comprehensive data in disaster areas and their quick analysis to transform the data to actionable knowledge are challenging. With the popularity and pervasiveness of mobile devices, crowdsourcing data collection and analysis has emerged as an effective and scalable solution. This paper addresses the problem of crowdsourcing mobile videos for disasters by identifying two unique challenges of 1) prioritizing visual data collection and transmission under bandwidth scarcity caused by damaged communication networks and 2) analyzing the acquired data in a timely manner. We introduce a new crowdsourcing framework for acquiring and analyzing the mobile videos utilizing fine granularity spatial metadata of videos for a rapidly changing disaster situation. We also develop an analytical model to quantify the visual awareness of a video based on its metadata and propose the visual awareness maximization problem for acquiring the most relevant data under bandwidth constraints. The collected videos are evenly distributed to off-site analysts to collectively minimize crowdsourcing efforts for analysis. Our simulation results demonstrate the effectiveness and feasibility of the proposed framework. Links: http://infolab.usc.edu/DocsDemos/to_ieeebigdata2015.pdf http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7363814
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Crowdsourcing the Acquisition and Analysis of Mobile Videos for Disaster Response
Similar to Crowdsourcing the Acquisition and Analysis of Mobile Videos for Disaster Response (20)
More from University of Southern California
More from University of Southern California (10)
Recently uploaded
Recently uploaded (20)
Crowdsourcing the Acquisition and Analysis of Mobile Videos for Disaster Response
- 1. 1 Crowdsourcing the Acquisition and Analysis of Mobile Videos for Disaster Response IEEE Big Data 2015, October 31, 2015 Presented by Hien To Integrated Media Systems Center University of Southern California Dr. Seon Ho Kim Prof. Cyrus Shahabi
- 2. 2 GeoQ by National Geospatial-Intelligence Agency (NGA) GeoQ is a crowdsourcing platform to analyze disaster dataEach project contains a set of jobsEach job contains a set of work cells, which are assigned to analystsAnalysts evaluate mobile videos within their assigned work cells
- 3. 3 Motivation Obtain real-time information to enhance situational awareness during and after disasters Live streaming of mobile video, e.g., YouTube, Periscope Analyze collected data in timely manner Prioritize data collection with limited transmission
- 4. 4 Haiti Earthquake Disasters Assess damage across large geographical area Cable Cuts in Vietnam Disrupted communication networks or data overload! Haiti earthquake, 85% Haitians could still access to their mobile phones but 70% cell towers were destroyed
- 5. 5 Crowdsourcing Disaster Response Early disaster data collection focus on VGI Disaster surveillance and response system Processing and integration of disaster data by leveraging Linked Open Data [Goodchild et al. 2010] [Chu et al. 2012] [Ortmann et al. 2011] Combine human and machine intelligence [Schulz et al. 2012] Focuses on both data collection and data analysis Collects huge amount of videos concerning disasters Improves resilience and responsiveness of computer systems Our study: Infrastructure damage caused by Earthquake [Kobayashi 2014] Resilient communication technologies, e.g., drones [Nemoto and Hamaguchi 2014]
- 6. 6 Crisis Response Efforts Ushahidi USGS ShakeMap Did You Feel It? NGA GeoQAI for DR Planetary Response Network USC MediaQ Highlight GeoQ and MediaQ
- 7. 7 MediaQ and GeoQ MediaQ [1]: mobile crowdsourcing platform to collect videos/images FOV Metadata: p: camera location d: camera direction α: viewable angle R: viewable distance t: timestamp 𝑁 α 𝑑 p R Field Of View (FOV) GeoQ [2]: crowdsourcing platform to analyze disaster data [1] By USC Integrated Media System Center [2] By US National Geospatial-Intelligence Agency (NGA) GeoQ’s Work Cells used in Nepal Earthquake 2015 MediaQ workers are on-site people; GeoQ workers are off-site analysts. MediaQ provides visual data to GeoQ. Data Collection Data Analysis Work cells are assigned to analysts
- 8. 8 Proposed Techniques Coverage Map Video metadata Upload metadata (FOV) first then identify videos with high visual awareness to be uploaded Partition space based on video locations Video content Collect data fast because time is critical Analyze the collected data effectively
- 9. 9 Proposed Framework 0 0 0 1 3 2 5 2 0 0 4. Evaluate work cells 3. Assign videos to analysts 1. Crowdsource videos from community Urgency Map Analyst Server 2. Upload relevant videos Coverage Map Space Decomposition Video content Video metadata
- 10. 10 1. Acquisition of Video Metadata and Content Real-time acquisition of metadata via Internet, SMS, WLAN Size of metadata is thousands times smaller than content Hold video in the device and have server ping the device when it wants the owner to upload the video Real-time analysis on the uploaded metadata, e.g., visualization Support data governance, preserve privacy and strict access control [Dey and Chinchwadkar, ICDE 2015]
- 11. 11 2. Visual Awareness Maximization Visual Awareness (VA) of a Video f1 f2 f3 VA(v) =Urgency(w) area(v) area(w) Selects videos that maximize total VA without exceeding budget (i.e., constrained bandwidth) This problem is 0-1 Knapsack. Video location Work cell A video has high VA if Enclosing work cell is urgent Aggregate video coverage is large 3-seconds video within a work cell
- 12. 12 3. Assign Videos to Analysts Each analyst handles equal number of videos Terminates when #workcells > #analysts-3 Quadtree (Gaussian dist.)Kd-tree (Gaussian dist.) GeoQ uses of grid-based partition Some analysts may be overloaded Space partitioning algorithm based on Quadtree and Kd-tree
- 13. 13 Minimum Redundant Coverage Upload key frames and metadata to server and minimize redundant coverage Select frames that maximizes total Visual Awareness without exceeding budget This problem is Maximum Coverage Problem. f2 f1 f3 FOV overlap f4
- 14. 14 Statistics Value Spatial distributions Uniform, Gaussian and Zipfian #videos 1000 with varying length Region size 10 × 10 square km in Los Angeles area Unit grid cells 500×500 FOV # per second 1 Viewable angle 60 degrees Visible distance 200 meters Synthetic Datasets Simulation Results 1. Partitioning techniques: Grid, Quadtree, Kd-tree 2. Visual awareness at video level vs. frame level [Ay et. Al. SIGSPATIAL’2010]
- 15. 15 Visual Awareness Maximization at Video Level Varying the number of analysts Uniform Dataset 0 0.5 1 1.5 2 2.5 3 3.5 16 25 36 49 64 VisualAwareness Analyst count Grid Quadtree Kd-tree Kd-tree increases visual awareness by up to 300% 0.1 1 10 100 1000 Grid Quadtree Kd-tree Variance(logscale) Uniform Gaussian Zipfian Variance of #videos per analyst Kd-tree produces similar #videos per work cell Gaussian Dataset
- 16. 16 Frame Level 0 20 40 60 80 100 120 16 25 36 49 64 VisualAwareness Analyst count Grid Quadtree Kd-tree Obtained visual awareness is an order of magnitude higher in comparison to video-level opt Runtime Measurements 5.2 7 7.4 0 1 2 3 4 5 6 7 8 Grid Quadtree Kd-tree Constructiontime(ms) Construction times are small and the differences are insignificant Uniform Dataset Uniform Dataset O(n) O(nlogn)O(logAn)
- 17. 17 Conclusion Proposed crowdsourcing framework for collection and analysis of video data under disaster situations Introduced more effective spatial decomposition to assign videos to analysts Developed an analytical model to quantify the visual awareness of a particular video or frame Introduced two variants visual awareness maximization problem: uploading the entire videos and individual frames Introduced a mechanism to enable time-sensitive acquisition and analysis of spatial metadata
- 18. 18 Q/A Hien To University of Southern California hto@usc.edu
- 19. 19 0 1 2 3 4 5 6 4 8 16 25 36 49 64 VisualAwareness Analyst count Grid Quadtree Kd-tree Experiment on MediaQ Dataset
- 20. 20 Location/Region Entropy Location entropy (LE) – Measures the diversity of unique visitors of a location – High if many users were observed at the location with equal proportion Cranshaw, J., et al., (2010). Bridging the gap between physical locations and online social networks. UBICOMP, 119- 128 1u 2u 1l 2l 1000 1 500 500 LE 0.35 0.69 lOlFreq )( luOlUserCount ,)( )(log)()( uPuPlLE lUu ll l lu l O O uP , )(where : total number of visits : number of visits by worker u
- 21. 21 Work Cell Urgency Urgency(w)= Priority(w)a +RegionEntropy(w.r)b Compute urgency value of a work cell Region Entropy – geosocial factor of a region. RE is high if the region are visited by many people many times, e.g., school, hospital. USC Campus The priority of an area, e.g., nuclear plant areas are more important than residence areas. Nuclear plant
- 22. 22 Future Work 1. Use user-generated videos from MediaQ 2. Use near-real-time ShakeMap from USGS
- 23. 23 Data Analysis Work Cell Analysis Analysts measure the severity of their assigned work cells and set urgency values to them Urgency of the work cells can be automatically calculated
Editor's Notes
- Collecting disaster data efficiently and analyzing these data effectively are important for disaster response because it can save people lives. There are many kinds of disaster-related data, text, audio, in which videos and images are most effective in understanding the disaster situation because they can be watched and easily understood by people, who may not know the local language. However, collecting big video data is challenging because communication network can be damaged during disaster. Also, having the analysts to process the videos quickly is important In this study, we focus on crowdsourcing mobile videos during and after disaster and solving such 2 unique challenges
- The application we are interested assessing damage across large geographical area of disaster. There are many kinds of disaster, natural disaster or man-made disaster. In all of these disasters either communication network is disrupted or we may have the issue of having too many data. For example, in Haitii earthquake, 85% Haitians could still access to their mobile phones but 70% of cell towers were destroyed. In such a case, uploading a lot of user-generated videos to the server is challenging.
- Crowdsourcing disaster response is a growing research area. Crowdsourcing has been shown to be a cost-effective and time-efficient way to acquire disaster data. There have been many studies on this topic, just to name a few here. However, these studies focus on either data collection or data analysis while our study aims to deal with both issues. We will propose a framework that collects a huge amount of videos concerning disasters. The framework improves resilience and responsiveness of computer systems to facilitate realtime data collection.
- There are lots of efforts in disaster response. We now introduce two existing platforms that inspire our study, one for data acquisition and one for data analysis.
- MediaQ is mobile crowdsourcing platform to collect video metadata and content. The geospatial metadata are transparently associated with each video frame. We represent a video as a sequence of frames, each frame is modeled as a field of view or FOV. An FOV includes camera location, direction, viewable angle and viewable distance and timestamp. On the other hand, GeoQ is another crowdsourcing platform that allows analysts to assess disaster damage across a large area. The idea is to divide disaster area into small regions namely work cells and assign them to the analysts. The role of the analysts is to evaluate the available data sources in their allocated work cells to identify relevant incident, for example a building on fire, or road block. Two reasons MediaQ and GeoQ are compliment to each others.
- We identify two challenges with MediaQ and GeoQ. The first one is to collect the data fast because in disaster time is critical. Also bandwidth is limited during and after disaster, so we may not be able to upload all videos. So there is a need of uploading metadata first. Then, from the metadata, we identify videos to be collected. We prefer the videos with high visual awareness. The second challenge is to analyze the collected data effectively. GeoQ makes use of grid-based partition. However, the shortcoming of such approach is that depending on the data distribution, some analysts may be assigned the workcells that containing a lots of videos while the others are assigned zero videos. So we propose to partition space based on video locations. We now introduce a unified framework that adopt such techniques.
- The framework has 4 steps. First, videos can be collected voluntarily from community or on-demand manner. When users record a video, its metadata is automatically captured and uploaded to the server but not the actual content. Then, basing on the uploaded metadata, the server uploads the potentially relevant videos, given a bandwidth constraint. The server will then assign the collected data to the analysts by partitioning the space into small regions namely work cells. Finally, the analysts watch the videos in their workcells and assign an importance value to the workcell. The higher the value, the more urgent the situation in the cell. The urgency values can also be automatically calculated. However, we will not discuss here, you can find details in our paper. I will talk about steps 1,2,3 in details.
- We propose the idea of holding the video in the device, and having a server ping the device when it wants the owner to upload the video. Being small in size, the metadata can be can be transmitted in realtime through various communication channels such as Internet, SMS and WLAN. To illustrate, the size of metadata of a ten-seconds video is only 1-3KB with a sampling rate of one FOV per second. However, the size of a video is typically a few MBs. This idea has various applications, for example showing the coverage map of the videos from the collected metadata. Other reasons for handling metadata separately are shown in ICDE 1015 paper, including supporting data governance as metadata often lives longer than its content, preserving privacy and strict access control.
- Based on the collected metadata, we will identify the potentially relevant videos to be uploaded. We define visual awareness as the importance of a video. It is important to compute visual awareness based on the geospatial metadata only because analyzing the actual content may be expensive in terms of human or computational resources. Urgency(w) is the urgency value of the workcell, between 0 and 5 and given by the analyst. The aggregated the coverage area of the video wrt the containing work cell is a value between 0 and 1. This equation means that the visual awareness of a video is high if the workcell is urgent OR the coverage area of the video is large. We formulate the problem of selecting a set of videos with the maximum total visual awareness under bandwidth constraints. We prove that this problem is 0-1 knapsack. So, we use dynamic programming to optimally solve it.
- When the videos are uploaded, they are distributed to analysts. GeoQ use equal-size grid, w soorkload distribution to analysts are not even. Some analysts may be assigned a lots of videos while the others are assigned zero video. Because human analysts are the most expensive resource, to facilitate timely evaluation on the acquired data, we propose an algorithm for space partitioning based on Quadtree and Kd-tree. The algorithm adaptively decomposes the space based on the spatial distribution of the videos. We modify the traditional point quadtree or kd-tree such that the algorithm terminates when the number of workcells is larger than the number of analysts. By doing so, each workcell can be assigned to at least one analyst. As the result, each analyst has similar number of videos to analyze and no one becomes the bottleneck in collective crowdsourcing efforts.
- Our last technique is to upload individual frames rather than entire video. The reason is that, many times several frames from a video are enough to understand the situation. Also, time is critical. So, we propose to upload only keyframes to the server. At the same time, we also minimize redundant coverage of these frames to reduce the bandwidth usage. In this example, if we can select two FOVs to maximize total coverage, those would be f1 and f3. Our goal now is to select a set of frames that maximizes total VA given a budget constraint. We formally define the problem and prove that it is NP-hard by reduction from the MCP problem. Details can be found in the paper. So, we adopt the greedy algorithm, which is the best-possible polynomial time approximation algorithm for MCP.
- More analysts look at the same amount of video VA increases More analysts work cells become smaller VA increases The improvement of kd-tree over uniform grid is clear because Grid does not adapt to the distribution of the data The improvement over quadtree is because kd-tree use median split while quadtree partition based on space The improvement is even bigger in Gaussian and Zipfian distributions
- Key observation VA of all techniques is higher by 1 order of magnitude because we can avoid a lot of redundant coverage in the case of uploading entire videos. Construction time in memory is small construction time is not our main concern. Take away messages: Uploading keyframes provide much higher VA when compared to uploading entire videos Good to go with kd-tree because the overhead is small when compared with the benefits