Moving objects media data computing(2019)
•
1 like•884 views
o Moving objects and media data computing - Spatial Big Data Computing - Moving Objects and Media Data Computing - Integrating Spatial Media and Deep Learning
Recommended
More Related Content
Similar to Moving objects media data computing(2019)
Similar to Moving objects media data computing(2019) (20)
More from Kwang Woo NAM
More from Kwang Woo NAM (20)
Recently uploaded
Recently uploaded (20)
Moving objects media data computing(2019)
- 1. Moving Objects and Media Data Computing (이동체 미디어 데이터 컴퓨팅 연구 동향) 2019. 07. 14 Kwang Woo NAM kwnam@kunsan.ac.kr Kunsan National University 이 연구는 국토해양부의 국토해양부의 국토정보연구 프로그램(14NSIP‐B080144‐01)과 한국연구재단 중견연구사업(No.2018R1A2B6007982), 한국국토정보공사의 산학협력 R&D 지원 사업에 의해 지원되었습니다. 2019.07. 15 Pitta Soft http://infolab.kunsan.ac.kr
- 2. 2 Who Professor/Kunsan National University ‐ KSNU, School of Computer, Information, and Communications Engineering, Professor (2004‐now) ‐ Future Vehicle R&D Training Project (2017‐now) 미래형자동차 R&D 전문인력양성사업단(2017‐) : { KEA, 군산대, 한양대, 충북대, 인하대, 부품연 } ‐ U. of Minnesota, Visiting Scholar(2015‐2016) ‐ ETRI, Telematics Division, Senior Researcher(2001‐2004) Projects • 한국연구재단, 지오미디어 스트림 데이터 컴퓨팅 및 공간 딥러닝 질의 지원(18‐21) : Deep Learning+Spatial Media • 한국국토정보공사, 도시상태 탐지를 위한 도시 환경 GeoAI 허브 기반 기술 개발(18‐19) : Deep Learning+Spatial • 국토해양부, 공간정보 S/W활용을 위한 오픈소스 가공기술 개발(14‐19) : PostTrajectory, PostGeoMedia • NRF, 소셜 미디어 스트림 기반 공간 지식의 연속 마이닝(13‐16): SKSpark, SpatialTajo • ETRI, 시멘틱 공간정보 서비스 프레임워크 및 구조 분석(10) • 국토해양부, U‐GIS 국토정보 컨텐츠 서비스 요소 기술 연구(07‐11) • NRF, geoSensor 센서 스트림 기반 점진적 온라인 패턴 마이닝 기법 연구(09‐12) • NRF, GeoSensor Network를 위한 효율적인 공간 센서 스트림 죠인 및 집계처리 기법 연구(06‐07) • NRF, 상황정보 기반의 Networked GeoVision 기술 개발(08‐11) : GeoVIsion
- 3. Introduction Spatial Data Spatial Media and Contents Spatial Information Spatial Data Spectrum 2D Vector RS Data 2.5D Data DEM Data LBS/TelematicsGeoERPWeb GIS MultiSensor Smart Phone Real 3D Data 360 Panorama Video+GPS (blackbox) Game,Edu,… Closed Box(Public Sector) Open Box( Enterprise Sector) Shared Universe(People Sector) Sony Camera (GPS+Pano) iWatch Google Glass GoPro • Evolutions of Spatial Data/Information
- 4. Introduction • 일반적인 공간좌표 및 POI 등의 공간 정보와 웹 문서, 도서, 사진, 동영상, 음성 등의 미디어가 결합된 공간정보 컨텐츠들의 등장 • 예) GoPro, PointCloud, 차량 블랙박스, 360 파노라마, 구글 글래스 GeoTagged Photo(Panoramio) 360 Panorama(StreetView) Car Blackbox Video PointCloud(LiDAR) by Self Driving Cars
- 5. 5 • Converging into Spatial Data Introduction GNSS - 무선데이터통신망을 이용한 정밀위치정보제공 - 차량항법, 교통정보 제공 GIS SIIS ITS - 정밀 기준점 위치정보에 의한 영상정보 정확도 향상 - GPS 기지국 구축적지 선정 - 차량용 네비게이션 정보제공 - 교통주제도 구축 및 계량적 분석 기술제공 - 교통시설물 관리 및 IT기반 운영기술 제공 - 대용량 공간정보 자동수정 갱신 - 영상지도 제작 - 측량기술 및 관련기술 (위성삼각측량, WGS84)제공 - 실시간 Mobile GIS응용기술 - 영상기반 교통정보 수집기술제공 - 고 정밀 영상검지 시스템 구축 출처 : ETRI
- 6. 공간정보 서비스 개발 플랫폼
- 11. 11 • Lidar and Imagery from Self Driving Cars • Google( Waymo ) Self Driving Car Spatial Media Services https://waymo.com/ Video : https://waymo.com/tech/
- 13. 13 • Data Sets • Deep Learning Annotations • Lidar • BDD100k : https://bair.berkeley.edu/blog/2018/05/30/bdd/ Self‐Driving Car Dataset
- 16. 16 • Spatial Queries • top‐K • Spatial operation • Point : kNN (k‐nearest neighbor), distance • Line : buffer, inside, intersect • Polygon : inside, outside Spatial Data Model K=4 kNN N distance buffer inside / intersect inside / outside
- 17. 17 • R‐Tree Spatial Data Indexing a b c d a b c d R S R S root root R-tree 1 2 3 4 5 6 7 8 9 Pointers to geometries 4 5 6 7 8 91 2 3 m = 2; M = 3
- 18. 18 • PostGIS/PostgreSQL : Stonebraker/Berkeley Spatial Database Systems Oracle Ingres PostgresSybaseIngres Illustra Postgres95 SQL Server Relational Theory Ingres Informix (IUS) 1980 1970 1990 2000 PostgreSQL
- 19. 19 • PostGIS • PostgreSQL Spatial Extension • SQ1: Retrieve all roads crossing Highway #1 • SELECT gid, name FROM bc_roads WHERE ST_Crosses(geom, ST_GeomFromText($Highway#1, 3005) ); • SQ2: Retrieve total length of all roads in the seoulRoad tables • SELECT Sum( ST_Length( geom ) ) / 1000 AS km_roads FROM seoulRoad; Spatial Database Systems ST_Intersects(G1,G2) ST_Contains(G1,G2) ST_Within(G1,G2) ST_Touches(G1,G2) ST_DWithin(G1,G2,D) 경부고속도로
- 20. 20 • pgRouting • Routing function extension to PostGIS • SQ3 : Calculate the shortest path between A and B • Quebec Snow Mobile Trail Map • http://fcmq.viaexplora.com/carte‐motoneige/indexen.html Spatial Database Systems SELECT * FROM pgr_dijkstra(' SELECT gid AS id, source, target, cost_s / 3600 * 100 AS cost, reverse_cost_s / 3600 * 100 AS reverse_cost FROM ways', 13009, 3986);
- 21. 21 • New York Taxi Data • 1.1 Billion NYC Taxi and Uber Trips • PostGIS + pgRouting Spatial Applications : NY Taxi http://toddwschneider.com/posts/analyzing‐1‐1‐billion‐nyc‐taxi‐and‐uber‐trips‐with‐a‐vengeance/
- 24. 24 • HDFS block system : Append Only • Default block size : 64MB • Default replications : 3 Spatial Big Data : Micro Blogs 블록1 블록2 블록3 블록4 블록5 320MB 파일 블록1 블록3 블록4 블록2 블록3 블록4 블록1 블록3 블록5 블록2 블록4 블록5 블록1 블록2 블록5 HDFS 파일저장
- 26. 26 • Spatial Social Data : Twitter • Square Union Model Spatial Big Data : Micro Blogs Person Locati on Time WordsTweet ① Dimension 2 ex) Tweet + Person ② Dimension 3 ex) Tweet + Person + Location ③ Dimension 4 ex) Tweet + Person + Location+Time ④ Dimension5
- 27. 27 • Real‐world Social Data • (lat, long) => key value Spatial Big Data : Micro Blogs o1 o2 o5 o7 o4 o8 o9 o12 o10 o11 o15 o6 o13 o14 o16 o17o18 o19 o3 o20 a b c e d f oid wordset o1 Italian, restaurant, expensive o2 coffee, expensive, restaurant o3 Italian, pizza, expensive o4 restaurant, pizza, expensive o5 Italian, pizza, restaurant, expensive o6 coffee, restaurant, low-priced o7 Italian, coffee, low-priced, restaurant, pizza o8 coffee, restaurant, expensive o9 expensive, restaurant o10 pasta, pizza, expensive o11 pasta, low-priced, restaurant o12 Italian, restaurant, expensive o13 pizza, low-priced o14 tea, expensive, restaurant o15 Italian, restaurant o16 pasta, restaurant, expensive o17 pizza, restaurant, low-priced o18 Italian, pizza, restaurant o19 Italian, pasta, restaurant, expensive o20 pasta, expensive (b) Social Database (a) social data in real world (lat, long)
- 28. 28 • Simplified Tweet Data Model • Simple MapReduce Example : Counting Tweets by User Spatial Big Data : Micro Blogs tweet : (mid, userid, x, y, time, text) tweet : (mid, userid, x, y, time, {word,…}) (mid, userid, x, y, time, {word,…}) Map (userid1, mid1) (userid2, mid2) (userid3, mid3) (userid2, mid4) (userid3, mid5) (userid4, mid6) shuffle Reduce (userid1, {mid1}) (userid3, {mid3,mid5}) (userid2, {mid2,mid4}) (userid4, mid6) (userid1, 1) (userid3, 2) (userid2, 2) (userid4, 1)
- 29. 29 • GeoHash : Z‐Ordering Partition Spatial Big Data : Micro Blogs 0 1 4 5 16 17 20 21 2 3 6 7 18 19 22 23 8 9 12 13 24 25 28 29 10 11 14 15 26 27 30 31 32 33 36 37 48 49 52 53 34 35 38 39 50 51 54 55 40 41 44 45 56 57 60 61 42 43 46 47 58 59 62 63 o1 o2 o5 o7 o4 o8 o9 o12 o10 o11 o15 o6 o13 o14 o16 o17o18 o19 o3 o20 (b) Spatial social data and grid space oid wordset geo o1 Italian, restaurant, expensive 13 o2 coffee, expensive, restaurant 18 o3 Italian, pizza, expensive 12 o4 restaurant, pizza, expensive 60 o5 Italian, pizza, restaurant, expensive 12 o6 coffee, restaurant, low-priced 35 o7 Italian, coffee, low-priced, restaurant, pizza 44 o8 coffee, restaurant, expensive 62 o9 expensive, restaurant 15 o10 pasta, pizza, expensive 12 o11 pasta, low-priced, restaurant 13 o12 Italian, restaurant, expensive 11 o13 pizza, low-priced 44 o14 tea, expensive, restaurant 15 o15 Italian, restaurant 35 o16 pasta, restaurant, expensive 15 o17 pizza, restaurant, low-priced 18 o18 Italian, pizza, restaurant 7 o19 Italian, pasta, restaurant, expensive 60 o20 pasta, expensive 62 (c) Spatial Wordset Transaction Database Spatial (a) social data in real world Single Level Grid Approach
- 30. 30 • GeoHash • Z‐Ordering ID : Array of Bits • Characterizing by Base32 Encoding Spatial Big Data : Micro Blogs Grid Level 1 Grid Level 2 Grid Level 3 Grid Level 4 00 01 10 11 00 01 10 11 Miami, Florida의 ID? 11 11 10 = geohash : 7 Base32 Table
- 31. 31 • GeoHash as a Row Key • Pros. ① Easy to calculate ② Prefix can be used to find nearest neighbors • Cons. • Accuracy and Boundary Problem Spatial Big Data : Micro Blogs
- 32. 32 • Spatial Aggregation : Counting Spatial‐Tweets • GeoHash Optimization Spatial Big Data : Micro Blogs (mid, userid, x, y, time, {word,…}) (geoid, mid) Map Shuffle (geoid, {mid, mid, …}) Reduce (geoid, n) (‘aaaaa’, mid1) (‘aaaab’, mid2) (‘aaaac’, mid3) (‘aaaad’, mid4) (‘aaaae’, mid5) (‘aaaba’, mid6) (‘aaabb’, mid7) (‘aaaa’, mid1) (‘aaaa’, mid2) (‘aaaa’, mid3) (‘aaaa’, mid4) (‘aaaa’, mid5) (‘aaab’, mid6) (‘aaab’, mid7) original data Approach 1 (‘aaaa’, mid1) (‘aaaa’, mid2) (‘aaaa’, mid3) (‘aaaa’, mid4) (‘aaab’, mid6) (‘aaab’, mid7) Approach 2 (‘aaae’, mid5)
- 33. 33 • Finding Home/Visitor • Finding Spatial Words Spatial Big Data : Micro Blogs (mid, userid, x, y, time, {word,…}) (<userid,geoid>, mid) Map Shuffle (<userid,geoid>, {mid, mid, …}) Reduce (<userid,geoid>, n) (mid, userid, x, y, time, {word,…}) (<geoid, word>, mid) (<geoid, word>, mid) (<geoid, word>, mid) Map Shuffle (<geoid, word>,{mid, mid, …}) Reduce (<geoid, word>, n)
- 34. 34 • SpatialHadoop • Hadoop Extension with Spatial Operators • Components • MapReduce‐based Operations • Two‐level Spatial Indexes • Global index : Grid File • Local index : R‐tree Spatial Big Data : Systems Ahmed Eldawy, Mohamed F. Mokbel: SpatialHadoop: A MapReduce framework for spatial data. ICDE 2015: 1352‐1363
- 35. 35 • Hadoop‐GIS • HiveSP: Spatial Data Warehouse System • Spatial Partitioning • Spatial Query Engine • Two‐level Spatial Indexes Spatial Big Data : Systems Ablimit Aji, Fusheng Wang, Hoang Vo, Rubao Lee, Qiaoling Liu, Xiaodong Zhang, Joel H. Saltz: Hadoop‐GIS: A High Performance Spatial Data Warehousing System over MapReduce. PVLDB 6(11): 1009‐1020 (2013)
- 36. 36 • Spark : In‐memory Computing • GeoSpark • Spatial Extension of Aparch Spark • SpatialRDD with Spatial Data Types • Memory‐level Spatial Local Index Spatial Big Data : Systems Jia Yu, Jinxuan Wu, Mohamed Sarwat: GeoSpark: a cluster computing framework for processing large‐scale spatial data. SIGSPATIAL/GIS 2015: 70:1‐70:4
- 37. 37 • SpatialSpark • Spatial Extension of Aparch Spark • GPGPU accelerated Spatial Operators • JTS Library • LocationSpark • Query Scheduler • Spatial Partitioning and Scheduling • Query Executor • Query Evaluation and Index Cost Estimation Spatial Big Data : Systems SpatialSpark : http://simin.me/projects/spatialspark/ LocationSpark : MingJie Tang, Yongyang Yu, Qutaibah M. Malluhi, Mourad Ouzzani, Walid G. Aref: LocationSpark: A Distributed In‐Memory Data Management System for Big Spatial Data. PVLDB 9(13): 1565‐1568 (2016)
- 38. 38 • SKSpark : Spatial Keyword Spark by KSNU • First Spatial Keyword Big Data System • BRQ : Boolean Spatial Range Query • 𝑄 𝜓, 𝑟 • 𝜓 = 질의 단어 • 𝑟 = 질의 범위 • 𝐵𝑅𝑄 𝑄. 𝑟 ∩ 𝑜. 𝑟 ∈ 𝐷 ∩ 𝑄. 𝜓 ∩ 𝑜. 𝜓 ∈ 𝐷 • BkQ : Boolean k‐Nearest Queries • 𝑄 𝜓, 𝑟, 𝑘 • Spatial‐Keyword In‐memory Indexing Spatial Big Data : Systems
- 39. 39 • SKSpark Spatial Big Data : Systems SpatialWebRDD Layer GeoPartitionedRDD Layer SKIndexedRDD Layer Apache Spark Cluster on HDFS Spark Layer Query Layer 0 5 10 15 20 25 1 2 3 4 5 6 검 색 시 간 ( m s ) 검색 키워드 수 DIR-tree QP-tree 0 5 10 15 20 25 30 35 40 45 50 1 2 3 4 5 6 검 색 시 간 ( m s ) k값 DIR- tree BRQ BkQ
- 41. 41 • Queries for Static Object/Static Map • Shortest path query • Range query : Find the restaurants • k-NN/top-K queries • Queries for Moving Object/Static Map • Continuous range queries • Location-based k-NN/top-K map queries • Queries for Moving Objects/Moving Objects • Internet connected cars • k-NN/top-K moving objects queries • Continuous moving objects queries MO and Spatial Queries
- 42. 42 Trajectory Queries • Queries • Spatial Queries • Which cars within 1 km from here? • Spatio‐temporal Queries • Which cars were within 1 km from here 7am to 8am today? • Moving Objects Queries • Which cars left from the Everland 7am to 8am today? • Which cars were stayed in Everland more than 4 hours • MO Mining Queries • What cars are moving together for more than 10 minutes?
- 44. 44 • Spatiotemporal Relationship Operations • Enters • Leaves • Crosses • StayIn • Bypass Data Model cross leave stayinenter bypass t x polygon r x x x
- 45. 45 • Slice Operations • Temporal Slicing • Spatial Slicing • Spatiotemporal Slicing Data Model t polygon r temporal slicing spatial slicing spatiotemporal slicing mpoint slice(mpoint, period) mpoint slice(mpoint, geometry) mpoint slice(mpoint, geometry, period) mpoint slice(mpoint, period) mpoint slice(mpoint, geometry) mpoint slice(mpoint, geometry, period)
- 46. 46 • Projection Operations • Spatial Projection • Temporal Projection Data Model LineString sproject( mpoint ) periodtproject( mpoint) LineString sproject( mpoint ) periodtproject( mpoint) temporal projection spatial projection t LineString start timestamp end timestamp
- 47. 47 Trajectory Queries • Range Queries • k‐NN Queries Yu Zheng, Trajectory Data Mining : An Overview, ACM Trans. on Intelligent System and Technology, Sept. 2015
- 48. 48 Trajectory Refinement • Noise Filtering • Mean(or Median) Filter • Kalman and Particle Filter • Heuristics‐based Outlier Detection • v > 300km/h
- 49. 49 Trajectory Refinement • Stay Point Detection • Semantic Filtering • 예 : 고속도로 평균 속도/시간 계산시에 삭제/보정 • Another Example
- 50. 50 Trajectory Segmentation • Reasons of Trajectory Segmentation • Too large MBR : false hit in Indexing • Semantical segmentation for mining City Highway Marios Hadjieleftherius, et. al, Efficient Indexing of Spatiotemporal Objects, EDBT 02.
- 51. 51 Trajectory Segmentation • Methods of Trajectory Segmentation • Time interval segmentation • Turning points segmentation
- 52. 52 Trajectory Segmentation • Methods of Trajectory Segmentation • Key shape segmentation • Stay point segmentation Yu Zheng, Trajectory Data Mining : An Overview, ACM Trans. on Intelligent System and Technology, Sept. 2015
- 53. 53 Trajectory Segmentation • Methods of Trajectory Segmentation • Semantical change point segmentation Yu Zheng, Trajectory Data Mining : An Overview, ACM Trans. on Intelligent System and Technology, Sept. 2015
- 54. 54 Trajectory Queries • k‐NN Trajectory Queries • ex ) Car Pool App • Find similar trajectory cars with my trajectory Yu Zheng, Trajectory Data Mining : An Overview, ACM Trans. on Intelligent System and Technology, Sept. 2015
- 55. 55 Moving Objects Indexing • Indexing Moving Objects • 3DR‐Tree for Historical Trajectory Data • Historical trajectories are represented by their three‐dimensional MBB(Minimum Bounding Box) Time Mohamed F. Mokbel, Thanaa M. Ghanem, Walid G. Aref: Spatio‐Temporal Access Methods. IEEE Data Eng. Bull. 26(2): 40‐49 (2003) Long‐Van Nguyen‐Dinh, Walid G. Aref, Mohamed F. Mokbel: Spatio‐Temporal Access Methods: Part 2 (2003 ‐ 2010). IEEE Data Eng. Bull. 33(2): 46‐55 (2010)
- 56. 56 Moving Objects Indexing • Indexing Moving Objects • Multi‐version Indexing for Historical Trajectories • Maintain an R‐tree for each time instance • R‐tree nodes that are not changed across consecutive time instances are linked together Timestamp 1Timestamp 0 3D‐R‐tree Mohamed F. Mokbel, Thanaa M. Ghanem, Walid G. Aref: Spatio‐Temporal Access Methods. IEEE Data Eng. Bull. 26(2): 40‐49 (2003) Long‐Van Nguyen‐Dinh, Walid G. Aref, Mohamed F. Mokbel: Spatio‐Temporal Access Methods: Part 2 (2003 ‐ 2010). IEEE Data Eng. Bull. 33(2): 46‐55 (2010)
- 57. 57 Moving Objects Indexing • TB‐Tree : Trajectory Bundle Tree • Hybrid index structure which preserves trajectories • A predominant trajectory index structure for Euclidean spaces • Fast access to the trajectory information of moving objects Dieter Pfoser, Christian S. Jensen, Yannis Theodoridis: Novel Approaches to the Indexing of Moving Object Trajectories. VLDB 2000: 395‐406
- 58. 58 Update Memo Spatio‐temporal Queries Raw answer set Final answer set • RUM‐tree : R‐tree with Update Memo • Memo : R‐tree의 최근 update 포함 • update 될 때 Memo에 적어놓음 Moving Objects Indexing Access Xiong, Xiaopeng, and Walid G. Aref. "R-trees with update memos." Data Engineering, 2006. ICDE'06. Proceedings of the 22nd International Conference on. IEEE, 2006.
- 59. 59 Moving Objects Indexing • The Time‐parameterized R‐tree (TPR‐tree) • Minimum bounding rectangles (MBR) • Velocity bounding rectangles (VBR) • MBR&VBR은 차량의 속도가 유지되는 한 동일한 MBR 안에 유지 될 수 있음을 보장함으로서 update를 최소화 Simonas Saltenis, Christian S. Jensen, Scott T. Leutenegger, Mario A. López: Indexing the Positions of Continuously Moving Objects. SIGMOD Conference 2000: 331‐342
- 60. 60 Moving Objects Indexing Christian S. Jensen, Dalia Tiesyte, Nerius Tradisauskas: Robust B+‐Tree‐Based Indexing of Moving Objects. MDM 2006: 12 • Bx‐Tree • B+ tree 를 기반으로 time/spatial id를 key로 갖도록 변형 • spatial id는 Hilbert Key를 사용 • Bdual‐Tree, BBx‐tree 등으로 계속 확장
- 61. 61 Moving Objects Indexing • Indexing on Road Network • FNR‐Tree : Fixed Network R‐tree • MON‐Tree : Moving Objects on Network FNR‐Tree : E. Frentzos. Indexing objects moving on fixed networks. SSTD, 2003. MON‐tree : Victor Teixeira de Almeida, et. al, Indexing the Trajectories of Moving Objects in Networks. GeoInformatica 9(1), 2005 R‐tree for Network R‐tree for MO
- 62. 62 Moving Objects Patterns Fosca Giannotti, Mirco Nanni, Fabio Pinelli, Dino Pedreschi: Trajectory pattern mining. KDD 2007: 330‐339 • Region of Interest(ROI) • T‐Patterns • ST Sequence Patterns BA 20 min C 3 min 3 min 6 min 2 min BA 20 min C 3 min
- 63. 63 Moving Objects Patterns • Region of Interest(ROI) : Static ROI • Fixed pre‐defined regions • Find frequent visited regions A B C time
- 64. 64 Moving Objects Patterns • Region of Interest(ROI) : Dynamic ROI • Spatial Clustering and Labeling • Find frequent visited regions cluster(x1,y1) cluster(x2,y2) time
- 65. 65 Moving Objects Patterns • Moving Together Patterns • Flock • 특정 Region r내에서 m 개체이상이 같은 방향으로 움직임 • Leadership • 가장 먼저 움직인 객체 • Convergence • m 개체 이상이 같은 방향으로 • Encounter • r내에서 만나는 지역 [37] J. Gudmundsson and M.V. Kreveld. 2006. Computing longest duration flocks in trajectory data. In Proceedings of the 14th Annual ACM International Symposium on Advances in Geographic Information Systems. ACM, 35‐42. [38] J. Gudmundsson, M.V. Kreveld, and B. Speckmann. 2004. Efficient detection of motion patterns in spatio‐temporal data sets. In Proceedings of the 12th Annual ACM International Symposium on Advances in Geographic Information Systems. ACM, 250–257.
- 66. 66 Moving Objects Patterns • Moving Together Patterns • Convoy • min consecutive timestamps together H. Jeung, M. Yiu, X. Zhou, C. Jensen, and H. Shen. 2008. Discovery of convoys in trajectory databases. Proceedings of the VLDB Endowment 1, 1, 1068–1080.
- 67. 67 Moving Objects Patterns • Moving Together Patterns • Swarm • min timestamps together Z. Li, B. Ding, J. Han, and R. Kays. 2010. Swarm: Mining relaxed temporal moving object clusters. Proceedings of the VLDB Endowment 3, 1-2, 723–734.
- 68. 68 Moving Objects Patterns • Trajectory Clustering • A Partition‐and‐Group Framework J. G. Lee, J. Han, and K. Y. Whang. 2007. Trajectory clustering: A partition‐and‐group framework. In Proceedings of ACM SIGMOD Conference on Management of Data. ACM, 593‐604
- 69. 69 Moving Objects Patterns • Trajectory Clustering • A Partition‐and‐Group Framework J. G. Lee, J. Han, and K. Y. Whang. 2007. Trajectory clustering: A partition‐and‐group framework. In Proceedings of ACM SIGMOD Conference on Management of Data. ACM, 593‐ 604
- 70. 70 Moving Objects Patterns • Trajectory Class • TraClass Algorithm • Classify subtrajectories instead of whole trajectories Jae‐Gil Lee, Jiawei Han, Xiaolei Li, and Hector Gonzalez, “TraClass: Trajectory Classification Using Hierarchical Region‐Based and Trajectory‐Based Clustering”, Proc. 2008 Int. Conf. on Very Large Data Base (VLDB'08), Auckland, New Zealand, Aug. 2008
- 71. 71 Moving Objects Patterns • Travel Recommendation • Find the interesting locations and travel sequences from trajectories Y. Zheng, L. Zhang, Z. Ma, X. Xie, W.‐Y. Ma. 2011. Recommending friends and locations based on individual location history. ACM Transaction on the Web 5, 1, 5‐44. Y. Zheng, L. Zhang, X. Xie, W.‐Y. Ma. 2009. Mining interesting locations and travel sequences from GPS trajectories. In Proceedings of the 18th International Conference on World Wide Web. ACM, 791‐800.
- 73. 73 • New Era of Spatial Data/Information • GPS Everywhere! Introduction images from Apple https://www.wired.com/2009/06/black‐box‐for‐the‐car/ http://www.bayareabikeshare.com CarPhone Bike and, life log for human
- 75. GeoMedia : GeoPhoto GeoPhoto GPS Tagged Photo (x, y, z) Geophoto KML Camera, Lookat + Field of View Geophoto GeoPhoto Field of View(FoV)
- 77. 77 • Real Example : GeoVideo/Trajectories in Car Blackbox • Moving Point and Moving Double GeoMedia :GeoVideo *.mp4 : video data *.gps : gps data(NMEA) *.3gf : acceleration sensor
- 78. GeoVideo : Moving Objects • GPS Trajectory and GeoVideo의 분석과 모니터링 차량 3대 센서 데이터 비교 분석
- 79. 79 • MediaQ by USC • GeoUGV : User‐Generated Mobile Video • Since 2014 • MediaQ web: http://mediaq.usc.edu Related Work Ying Lu, Hien To, Abdullah Alfarrarjeh, Seon Ho Kim, Yifang Yin, Roger Zimmermann, and Cyrus Shahabi,GeoUGV: User‐Generated Mobile Video Dataset with Fine Granularity Spatial Metadata, In the 7th ACM Multimedia Systems Conference (MMSys), Klagenfurt am Worthersee, Austria, May 10‐13, 2016 Seon Ho Kim, Ying Lu, Giorgos Constantinou, Cyrus Shahabi, Guanfeng Wang, and Roger Zimmermann,MediaQ: Mobile Multimedia Management System. In the conference of Multimedia Systems (MMSys), In the conference of Multimedia Systems (MMSys), pages: 224 ‐ 235, Singapore, March 19 ‐21, 2014
- 80. 80 • GeoVideoIndex by POSTECH • GeoVideo • MBTR : Minimum Bounding Tilted Rectangle Related Work Dongha Lee, Jinoh Oh, Woong-Kee Loh, Hwanjo Yu: GeoVideoIndex: Indexing for georeferenced videos. Inf. Sci. 374: 210-223 (2016)
- 81. 81 • Architecture PostGeoMedia Architecture 81 PostgreSQL DB PostGIS PostTrajectory Trajectory Type Trajectory Indexing PostGeoPhoto GeoPhoto Type GeoPhoto Indexing Geometry Type R-Tree on GiST BlackBox App for Cars (SkyBox) Apache/Tomcat Web Sever PostGeoVideo GeoVideo Type GeoPhoto Indexing Trajectory Storage Trajectory Queries Trajectory Analysis Trajectory Apps GeoV/P Storage GeoV/P Queries GeoV/P Analysis GeoV/P Apps Trajectory/GeoPhoto/GeoVideo Web User Geospatial Media Contents Management System(GeoCMS)
- 82. 82 • Data Types Data Model Type Description MPoint MPoint ((x1 y1) t1, (x2 y2) t2, … (xn yn) tn) MPolygon MPolygon (polygon1 t1, polygon2 t2, … polygonn tn) MLineString MLineString (line1 t1, line2 t2, … linen tn) MString MString (string1 t1, string2 t2, … stringn tn) MDouble MDouble (double1 t1, double2 t2, … doublen tn) MBool MBool (bool1 t1, bool2 t2, … booln tn) MDuration MDuration (duration1 t1, duration2 t2, … durationn tn) MInt MInt (int1 t1, int2 t2, … intn tn) MInstant MInstant (instant1, instant2, … instantn) MPeriod MPeriod ((fromTime1 toTime2), (fromTime2 toTime3), … (fromTimen‐1 t oTimen)) MMultiPoint MMultiPoint ((point1 []) t1, (point2 []) t2, … (pointn [])tn) *STPhoto STPhoto ((uri1 height1 width1 altitude exif fov point) t) *MPhoto MPhoto ((uri1 height1 width1 altitude1 exif1 fov1 point1) t1, (uri2 height2 width2 altitude2 exif2 fov2 point2) t2, … (urin heightn widthn altituden exi fn fovn pointn) tn) *MVideo MVideo ((uri1 altitude1 exif1 fov1 point1) t1, (uri2 altitude2 exif2 fov2 poi nt2) t2, … (urin altituden exifn fovn pointn) tn)
- 83. 83 • Moving Objects Examples • MPoint • ((40.67 ‐73. 83) 1000, (40.74, ‐73.99) 2000). • STPhoto • ('http://u‐gis.net/p1.jpg' (40.74, ‐73.99) 1000 60 13.15 10). • MPhoto • (('http://u‐gis.net/p1.jpg' (40.74, ‐73.99) 1000 60 13.15 10), ('http://u‐gis.net/p2.jpg' (40.75, ‐73.93) 2000 60 13.23 10)). • MVideo • ('http://u‐gis.net/mv1.mp4', ('t=1' (40.74, ‐73.99) 1000 60 13.17 10), ('t=2', (40.75, ‐73.93) 2000 60 12.59 10)). Storage Model
- 84. 84 • Tables Storage Model userVideos (id: int, name: text, mv: mvideo) carTrajs (id: int, carnumber: text, mt: mpoint) carSensors( id:int, carnumber:text, accX: mdouble, accY:mdouble, accZ:mdouble) city (id: int, name: text, geo: polygon)
- 85. 85 • Append • Query 1 : “Append current two moving points data into car 1’s trajectory in carTrajs table” • Remove GPS Trajectories Trajectory Queries UPDATE carTrajs SET mt = M_Append (mt, 'MPOINT ((40.67 ‐73. 83) 1000, (41.67 ‐73.81) 2000)') WHERE id = 1; UPDATE carTrajs SET mt = M_Remove( mt, 500, 550 ) WHERE id = 1;
- 86. 86 • Temporal Queries • Query 2 “Retrieve videos in userVideos table which are taken during ‘period (1100, 1200) • Query 3 “Retrieve videos in userVideos table which are taken during ‘period (1100, 1150 )’, and returns the sliced video files by ‘period (1100, 1200)’ ” Trajectory Queries SELECT id, mv FROM userVideos WHERE M_Overlaps (mv, 'Period (1100, 1150)'); SELECT id, M_Slice ( mv, 'Period (1100, 1200)') FROM userVideos WHERE M_Overlaps (mv, 'Period (1100, 1150)');
- 87. 87 • Spatial and Spatiotemporal Queries • Query 4 “Retrieve videos in userVideos table which have entered the city ‘New York’ ” • Query 5 “Retrieve the cars in userTrajs table that have left from city ‘New York’ during ‘period (1100, 1200 )’ ” Trajectory Queries SELECT id, name, mv FROM carTrajs a, city c WHERE M_Enters (a.mt, c.geo ) = true AND c.name=’New York’; SELECT id, carnumber FROM carTrajs a, city c WHERE M_Leaves(M_Slice(a.mt,'Period(1100, 1200)'), c.geo) = true AND c.name = ‘ New York’;
- 88. 88 • Query 5.2 “Retrieve the cars in userTrajs table that traveled up to 500m from central New York city” • Similarity Query • Query 6 “Return LCVS similarity distances between two videos in userVideos table.” Trajectory Queries SELECT id, carnumber FROM carTrajs a WHERE M_Max( M_Distance( a.mt, ‘POLYGON(40.74 ‐73.99, 40.74 ‐73.98, 40.73 ‐ 73.98, 40.73 ‐73.99, 40.74 ‐73.99 )’ ) < 500.0 SELECT a.id, b.id, M_LCVS (a.mv, b.mv) FROM userVideos a, userVideos b;
- 89. 89 • Join Queries • Query 7: “Retrieve videos whose FOV intersects each other” • Query 8: “Find the videos which could be taken 'Empire State’ building SELECT a.id, b.id FROM userVideos a, userVideos b WHERE M_ANY(M_Intersects( FOV(a.mv), FOV(b.mv))) ; SELECT a.id, b.id, a.mv FROM userVideos a, city c WHERE M_Intersects( FOV(a.mv), c.geo ) AND c.name =’Empire State’;
- 90. 90 • Simple Distance Queries • Distance in WHERE Trajectory Queries SELECT id, M_distance( mt, 'Point( 50 50 )' ), M_mindistance( mt, ‘Point( 50 50 )’ ), M_maxdistance(mt, 'Point( 50 50 )' ) FROM carTraj; SELECT id, carnumber FROM carTraj WHERE M_mindistance(traj, ‘point( 50 50 )') < 20;
- 93. 93 • Example : Finding GPS Photos in Flickr Related Work : Flickr 지역/시/주/우편번호 입력지역/시/주/우편번호 입력 컨텐츠의 지정 : face, bike, building, …
- 97. 97 • Example : TensorFlow Results Related Work : TensorFlow https://www.tensorflow.org/tutorials/
- 99. 99 • Solution : PostGeoMedia • PostgreSQL/PostGIS + OpenCV/Tensorflow Our Approach PostgreSQL/PostGIS with Media Processing Metadata Photos OpenCV/ TensorFlow OpenCV/ TensorFlow Geo Media Processing Apache/Tomcat
- 101. 101 • Deep Object Detection and Find in a Photo • Deep Object Count • Location Deep GeoPhoto Queries SELECT M_deepFind(‘http://u‐gis.net/files/attach/filebox/615/002/2615.jpg’ , ‘person, car’ , “mscoco”) SELECT M_deepCountAsJSON( ‘http://u‐gis.net/2615.jpg’, ‘person’); SELECT M_deepImgLoc( ‘http://u‐gis.net/2615.jpg');
- 102. 102 • Deep Object Queries in a Table • Spatial Deep Queries Deep GeoPhoto Queries SELECT id, img_url FROM tourphotos WHERE M_deepFind(img_url, ‘car,bus,truck'); SELECT id, img_url, user, FROM tourphotos WHERE ST_Contains( POLYGON (tour_area ), M_deepImgLoc(img_url ) )
- 103. 103 • Count the photos which contain some Person or car • Retrieve all photos which contain some person and car within Region A Deep GeoPhoto Queries SELECT count(*) FROM TourPhotos WHERE M_DeepCount(uri, 'person') >0 OR M_DeepCount( uri, ‘car’ ) > 0 ; SELECT M_GeoJSON( uri ) FROM TourPhotos WHERE M_DeepCount(uri, 'person') >0 AND M_DeepCount( uri, ‘car’ ) > 0 AND ST_WITHIN( geom, ST_MakeEnvelope(191232, 243117,191232, 243119,312) );
- 104. 104 • Join Deep GeoPhoto Queries SELECT count(*) FROM TourPhotos tp, cityLandmark c WHERE ST_WITHIN( tp.geo, ST_BUFFER( c.geo, 10 ) ) AND c.name = ‘Empire State’ AND M_DeepCount( tp.uri, ‘person) >= 1 ;
- 105. 105 • Detection and FoV features GeoCMS : with Deep Learning FoV will be supported if your pics were taken in your smart phone. Click
- 108. 108 • Embed in your web document • Example : http://deepapi.io GeoCMS : with Deep Learning Click & Paste it <img src="http://api.u‐ gis.net:8002/?command=detect&model=mscoco&class es=person&img=http://mcalab.kunsan.ac.kr:8080/files /attach/images/675/386/002/002cfda9daba2f27d0d13 f4e4baf325e.jpg"/>
- 109. Dynamic Generation of Road Damage Map using Pittasoft Blackbox KCC 2019. 06. 27
- 110. 110 • Road Damage Data Set • 차량 촬영 영상을 통한 딥러닝 기반 도로 파손 탐지 • 2018 IEEE Big Data Challenge 데이터 기반 딥러닝 : Yolo • https://bdc2018.mycityreport.net/overview/ Introduction
- 112. 112 • 차량용 도로 영상 취득 장치 • Blackvue DR900S‐2CH • GPS 내장형 블랙박스 • 온라인 비디오 스트리밍 기능 지원 • GPS 및 가속도 센서 데이터를 MP4 파일에 저장 Device : DR900S‐2CH
- 113. 113 Object Detection and Location t1 t2 t3 t4 o1 o2 o3 o4 o1 o2o1 o1 o1 o3 o4 (gps,camera)
- 114. 114 • Deep Model • Yolo‐based RCNN Model • Data Set • Tokyo University, Road Damage Detection Dataset • 9,053 images, 15,435 bbox RCNN‐based Detection Detail Type Class name Vertical Crack Wheel mark D00 Construction joint D01 Horizontal Crack Equal interval D10 Construction joint D11 Alligator Crack partial pavement, overal pavement D20 etc Potholes D40 Paint Crosswalk Paint Blur D43 LIne Paint Blur(White/yellow) D44
- 115. 115 도로 파손 지도 생성 알고리즘 mp4 GPS/ Camera FoVs RCNN 기반 Road Damage Detection 인식된 Road Damage의 location determination GeoJSON 기반의 Road Damage Map 생성 지도상에 Map의 가시화
- 116. 116 Conclusion • Trends • Spatial Big Data Computing • Moving Objects Media Data Computing • Our Systems • SKSpark • PostGeoMedia • PostGeoMedia with Deep Learning Features