本著作採用創用CC 「姓名標示」授權條款台灣3.0版         Intelligent   Video Surveillance and       Sousveillance        Wang, Yuan-Kai(王元凱)    ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   2                      Contents     1.     Image, Vision...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   3                 1. Image, Vision, and                 ...
Wang, Yuan-Kai(王元凱)        Video Surveillance and Sousveillance   p.   4                       Image, Vision, and         ...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance           p.   5       Camera Based Environment              ...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.   6                 2. Video Surveillance         Video ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance          p.   7           Video Surveillance Market       • CCT...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   8               Who’s Watching You?         UK has the ...
Wang, Yuan-Kai(王元凱)              Video Surveillance and Sousveillance                   p.   9          Crimes Breaking by...
Wang, Yuan-Kai(王元凱)                  Video Surveillance and Sousveillance          p.   10          CCTV v.s. Crime Breaki...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   11              Important Crime Cases         近年來運用路口監視...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.   12                      Case Study         94年5月17日台中市...
Wang, Yuan-Kai(王元凱)           Video Surveillance and Sousveillance               p.   13           CCTV Video Surveillance...
Wang, Yuan-Kai(王元凱)               Video Surveillance and Sousveillance   p.   14           Digital Video Surveillance     ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance          p.   15                 Technology Trend of           ...
Wang, Yuan-Kai(王元凱)           Video Surveillance and Sousveillance            p.   16                  Surveillance over I...
Wang, Yuan-Kai(王元凱)       Video Surveillance and Sousveillance   p.   17        Why Intelligent Surveillance         Too ...
Wang, Yuan-Kai(王元凱)        Video Surveillance and Sousveillance   p.   18                        Applications of          ...
Wang, Yuan-Kai(王元凱)        Video Surveillance and Sousveillance   p.   19                      Visual SurveillanceVisual S...
Wang, Yuan-Kai(王元凱)           Video Surveillance and Sousveillance                   p.     20                       Video...
Wang, Yuan-Kai(王元凱)          Video Surveillance and Sousveillance            p.   21                                IBM S3...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.   22                      ObjectVideo
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   23                              ITRI
王元凱                 以視覺為基礎之智慧型環境(Vision Based Intelligent Environment)           p. 24           經濟部學界科專計畫 (1/3)         ...
Wang, Yuan-Kai(王元凱)       Video Surveillance and Sousveillance   p.   25           經濟部學界科專計畫 (2/3)     智慧型建築         目標:...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   26            經濟部學界科專計畫 (3/3)     智慧型社區與城市            ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   27      Activities in the VBIE Project         參加國際展覽  ...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance   p.   28       A Proposed Architecture for            Police...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance   p.   29       A Proposed Architecture for            Police...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   30            Demos from ISLab@FJU         2.1 Mobile V...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.   31       2.1 Mobil Video Surveillance              Even...
Wang, Yuan-Kai(王元凱)        Video Surveillance and Sousveillance                    p.   32        Current Mobile Surveilla...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   33            Instant Alarm Messaging                  I...
Wang, Yuan-Kai(王元凱)           Video Surveillance and Sousveillance             p.   34            Our System Architecture ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   35             Our System: Web-based               Brows...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   36              Our System –      Event, Key Frame, Vide...
Wang, Yuan-Kai(王元凱)      Video Surveillance and Sousveillance     p.   37                      Mobile Interface           ...
Wang, Yuan-Kai(王元凱)                   Video Surveillance and Sousveillance                                      p.   38   ...
Wang, Yuan-Kai(王元凱)                                 Video Surveillance and Sousveillance           p.   39                ...
Wang, Yuan-Kai(王元凱)      Video Surveillance and Sousveillance   p.   40                      2.2 Night Vision         Nig...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   41            Moving Object Detection         Backgroun...
Wang, Yuan-Kai(王元凱)              Video Surveillance and Sousveillance                                p.   42           An ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   43
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance      p.   44      2.3 Reconfigurable Hardware      for Moving O...
Wang, Yuan-Kai(王元凱)                    Video Surveillance and Sousveillance                      p.   45             Backg...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   46         Parallelization by Hardware         Parallel...
Wang, Yuan-Kai(王元凱)      Video Surveillance and Sousveillance            p.   47                      Design & Results    ...
Wang, Yuan-Kai(王元凱)          Video Surveillance and Sousveillance          p.   48          Experimental Comparison      ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance               p.    49                 2.4 Super-resolution    ...
Wang, Yuan-Kai(王元凱)          Video Surveillance and Sousveillance        p.   50                The Proposed Method      ...
Wang, Yuan-Kai(王元凱)       Video Surveillance and Sousveillance                p.   51                      Experiments    ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   52                 Experimental Results
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   53              2.5 Camera Tampering                    ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance           p.   54          Tampering Examples (1/2)            ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance           p.   55          Tampering Examples (2/2)            ...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.   56                      The System         Sabotage   ...
王元凱                 以視覺為基礎之智慧型環境(Vision Based Intelligent Environment)           p. 57               2.6 System Integratio...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance        p.   58            Heterogeneous Cameras         We u...
Wang, Yuan-Kai(王元凱)        Video Surveillance and Sousveillance   p.   59                      Smart Building (1/2)      ...
Wang, Yuan-Kai(王元凱)        Video Surveillance and Sousveillance           p.   60                      Smart Building (2/2...
Smart Campus (1/2)                       13.機器人導引訪客導引  6.智慧型車牌偵測                             12.快速人臉辨識  7.行人交錯偵測與追蹤       ...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.   62                  Smart Campus (2/2)
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   63               3. Video Sousveillance       Environme...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance   p.   64                      Paradigm Shift         Evolut...
Wang, Yuan-Kai(王元凱)        Video Surveillance and Sousveillance       p.   65                      Desktop Computing      ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   66        Jurassic Mobile Computing         Goal: Stay i...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance   p.   67         Current Mobile Computing          Still ha...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance              p.   68                 Wearable Computing       ...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance                         p.   69                      The Hard...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.   70                  Input Sensor - Glove         Hand ...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance   p.   71                      Output Display         Optica...
Wang, Yuan-Kai(王元凱)       Video Surveillance and Sousveillance       p.   72                      Add Vision Sensor      ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   73          Why Wearable Vision (1/2)         Capture o...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   74          Why Wearable Vision (2/2)         Game assi...
Wang, Yuan-Kai(王元凱)             Video Surveillance and Sousveillance           p.   75        Where to Wear an Extra Eye  ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   76        Where to Wear an Extra Eye
Wang, Yuan-Kai(王元凱)            Video Surveillance and Sousveillance              p.   77           Steve Mann’s WearComp  ...
Wang, Yuan-Kai(王元凱)             Video Surveillance and Sousveillance       p.   78             US Armys Land Warrior      ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   79                 Demo of ISLab@FJU                    ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance     p.   80                      The X-Eye         Live photo ...
Wang, Yuan-Kai(王元凱)                Video Surveillance and Sousveillance        p.   81                      X-Eye Componen...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   82           1st Generation Prototype
Wang, Yuan-Kai(王元凱)         Video Surveillance and Sousveillance                       p.   83                   Embedded ...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance                                 p.   84              Output -...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance   p.   85                      Input - Camera
Wang, Yuan-Kai(王元凱)         Video Surveillance and Sousveillance   p.   86                        User Interface         ...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance             p.   87                      Photo Mode           ...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.   88                      Manage Mode                    ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance                p.   89       X-Eye Hardware Architecture       ...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   90       X-Eye Software Architecture
Wang, Yuan-Kai(王元凱)         Video Surveillance and Sousveillance                  p.   91                X-Eye Software St...
Wang, Yuan-Kai(王元凱)                        Video Surveillance and Sousveillance                                         p....
Wang, Yuan-Kai(王元凱)             Video Surveillance and Sousveillance   p.   93                        Algorithm (2/2)     ...
Wang, Yuan-Kai(王元凱)      Video Surveillance and Sousveillance        p.   94                      Smart Camera         Eg...
Wang, Yuan-Kai(王元凱)        Video Surveillance and Sousveillance              p.   95                 FJUCam1 - Hardware • ...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.   96                  FJUCam1 - Software         Develop...
Wang, Yuan-Kai(王元凱)       Video Surveillance and Sousveillance   p.   97         FJUCam1 - Face Detection         The Ada...
Wang, Yuan-Kai(王元凱)      Video Surveillance and Sousveillance   p.   98                       FJUCam2 (1/5)         Cons ...
Wang, Yuan-Kai(王元凱)               Video Surveillance and Sousveillance                   p.   99                          ...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance                      p.   100                      FJUCam2 (3...
Wang, Yuan-Kai(王元凱)      Video Surveillance and Sousveillance   p.   101                      FJUCam2 (4/5)           Alg...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance   p.   102                      FJUCam2 (5/5)         Possib...
Wang, Yuan-Kai(王元凱)     Video Surveillance and Sousveillance   p.   103                      4. Conclusions         Image...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.   104                      How to Study         Basic co...
Wang, Yuan-Kai(王元凱)   Video Surveillance and Sousveillance   p.   105                      Activities         中華民國影像處理與圖形...
Wang, Yuan-Kai(王元凱)        Video Surveillance and Sousveillance   p.   106                        The End                 ...
Wang, Yuan-Kai(王元凱)    Video Surveillance and Sousveillance   p.                      本簡報授權聲明      此簡報內容採用   Creative Com...
Upcoming SlideShare
Loading in …5
×

Intelligent Video Surveillance and Sousveillance

2,505 views

Published on

It is a presentation for acamedia talk about intelligent video surveillance and video sousveillance given in 2010. Some of our research results are also presented in this presentation.

Published in: Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,505
On SlideShare
0
From Embeds
0
Number of Embeds
17
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Intelligent Video Surveillance and Sousveillance

  1. 1. 本著作採用創用CC 「姓名標示」授權條款台灣3.0版 Intelligent Video Surveillance and Sousveillance Wang, Yuan-Kai(王元凱) Electrical Engineering Department,Fu Jen Univ., Taiwan (輔仁大學電機工程系) Email: ykwang@mail.fju.edu.tw URL: http://www.ykwang.tw 2010/11/06
  2. 2. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 2 Contents 1. Image, Vision and Intelligence 2. Intelligent Video Surveillance 3. Intelligent Video Sousveillance 4. Conclusions
  3. 3. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 3 1. Image, Vision, and Intelligence  Humans vision system
  4. 4. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 4 Image, Vision, and Intelligence in the Digital World  Digital vision system + Robust Computer Vision Algorithms
  5. 5. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 5 Camera Based Environment Person : Camera = 1 : N Outside-In : Video Surveillance, Human Computer Interface Inside-Out : Video Sousveillance, Egocentric Vision
  6. 6. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 6 2. Video Surveillance  Video surveillance  Use video camera to monitor an area for crime investigation  From video surveillance to visual surveillance  Impose video analytics by computer vision algorithm
  7. 7. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 7 Video Surveillance Market • CCTV has been a mass-product market • Since the 911 event, the market is continuously increasing(百萬美元) Source: JP Freeman
  8. 8. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 8 Who’s Watching You?  UK has the most CCTV cameras in Europe  4.2 million cameras which is  20% of the worlds CCTV  1 camera for every 14 people in UK  On average a person can be caught on camera 200-300 times a day
  9. 9. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 9 Crimes Breaking by CCTV 92年度 93年度前3季 統計 案類/年度 件數 人數 件數 人數 件數 人數 總計 610 689 720 796 1330 1485 竊盜 364 412 425 447 789 859 搶奪 91 69 104 104 195 173 強盜 44 53 39 65 83 118 殺人 18 36 13 20 31 56 擄人勒贖 5 15 1 2 6 17 重傷害 4 8 3 3 7 11 恐嚇取財 5 6 1 2 6 8 強制性交 5 5 3 3 8 8 其他 74 85 131 150 205 235
  10. 10. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 10 CCTV v.s. Crime Breaking  監視系統對破獲刑案的助益 120000 7000 監視器數量 100000 6000 因監視器破獲件數 5000 80000 4000 60000 3000 40000 2000 20000 1000 0 0 92 93 94 95 96 97 年度  監視器數量和監視器破獲件數兩者間呈現 正向關係
  11. 11. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 11 Important Crime Cases  近年來運用路口監視器 偵破社會矚目重大案件  白米炸彈客  汐止市殺警奪槍案  蠻牛千面人案  台南國道襲警奪槍案  新莊襲警奪槍案  英國倫敦地鐵爆炸案  台中角頭槍殺案
  12. 12. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 12 Case Study  94年5月17日台中市蠻牛千面人案, 造成全省恐慌  破案關鍵在於  幾個放置毒蠻牛 的超商監視器 錄到千面人身影  歹徒車號被 提款機監視器 清楚拍下  動員500警員 觀看6000小時的錄影資料
  13. 13. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 13 CCTV Video Surveillance Video Display & Record VCR / DVR Analog Multiplexer components Centralized Monitoring Video Capture analogue analogue analogue analogue
  14. 14. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 14 Digital Video Surveillance High scalibility IPCam + analog camera Network transmission Remote control Digital storgage digital Network digital digital analogue analogue analogue analogue
  15. 15. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 15 Technology Trend of Video Surveillance 資料來源:拓璞產業研究所,2008年5月
  16. 16. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 16 Surveillance over IP Paradigm shift of video surveillance  Role from security monitoring to the personalized video contents  Advent of the intelligent surveillance Changes in technology & desire 1. Network 2. Video compression 3. Live images Intelligent Surveillance IP Surveillance CCTV (DVR) CCTV (VCR) 1G 2G 3G
  17. 17. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 17 Why Intelligent Surveillance  Too many cameras, too few human guards  “After only 20 minutes, human attention to video monitors degenerates to an unacceptable level.” (Sandia National Laboratories)
  18. 18. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 18 Applications of Visual Surveillance
  19. 19. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 19 Visual SurveillanceVisual Surveillance = Digital CCTV + Video Analytics Smart/Intelligent Surveillance
  20. 20. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 20 Video Analytics影像擷取 相機異常偵測 人臉辨識 查詢、過濾、聯防 Video Image Object Object Object Behavior Capture Enhance /Event Tracking /Event Analysis Retrieval Detection Recognition 強光抑制 警戒線、 跌倒、人潮行為分析 路徑追蹤、流浪漢監控
  21. 21. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 21 IBM S3 Exploratory Computer Vision Group in IBM T.J. Watson Research Center. http://www.research.ibm.com/ecvg/
  22. 22. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 22 ObjectVideo
  23. 23. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 23 ITRI
  24. 24. 王元凱 以視覺為基礎之智慧型環境(Vision Based Intelligent Environment) p. 24 經濟部學界科專計畫 (1/3)  MOEA’s Technology Development Program for Academia  Name: Construction of Vision- Based Intelligent Environment  Phase I : 2004/5 ~ 2008/4  Phase II: 2008/11 ~ 2012/10  Involved people  29 professors from 18 universities  110 staffsFu Jen University Department of Electronic Engineering Yuan-Kai Wang
  25. 25. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 25 經濟部學界科專計畫 (2/3)  智慧型建築  目標:開發智慧型建築內部空間不可或缺的全 方位、主動式、機動性的智慧性視訊監控系統 A1 日夜活動式廣域安全監視 系統 A2 視訊監控中央管理系統 A3 室內突發事件分析系統  攝影機網路  感測網路
  26. 26. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 26 經濟部學界科專計畫 (3/3)  智慧型社區與城市  目標:開發戶外社區及城市大範圍區域之穩定、成熟而 具產品面向的智慧性視覺監控系統 B1 人車偵測與辨識系統 B2 都會區人物追蹤系統 B3 室外事件分析與搜尋系統
  27. 27. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 27 Activities in the VBIE Project  參加國際展覽  研究技術需  展示化:技術需能常駐展示  系統化:大型整合展示  指標化:技術有量化指標  市場化:建立產業鏈地圖、政策規劃  專利化:專利佈局、專利地圖分析  商品化:網路行銷百餘項技術  參與國際標準制訂(ONVIF)  引導業界投資  與警政機關合作
  28. 28. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 28 A Proposed Architecture for Police Office (1/2) 智慧型視訊監控技術在警政治安上之可行性研究, 詹毓青,中央警察大學資訊管理所碩士論文,2009
  29. 29. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 29 A Proposed Architecture for Police Office (2/2) 智慧型視訊監控技術在警政治安上之可行性研究, 詹毓青,中央警察大學資訊管理所碩士論文,2009
  30. 30. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 30 Demos from ISLab@FJU  2.1 Mobile Video Surveillance  2.2 Night Vision  2.3 Reconfigurable Hardware for Moving Object Detection  2.4 Super-resolution  2.5 Camera Tampering Detection  2.6 System Integration
  31. 31. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 31 2.1 Mobil Video Surveillance Event-driven Instant Messaging
  32. 32. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 32 Current Mobile Surveillance  Mobile phone directly connects to the camera Mobile networks (circuit-switched & packet-switched) Remote monitor Camera client Drawbacks: 1. Not event-driven 2. High communication cost
  33. 33. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 33 Instant Alarm Messaging Is Important
  34. 34. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 34 Our System Architecture Keyframe Web Selection Server PC Object 1. Object Info. Video Detection 2. Keyframe Streaming &Tracking 3. Video Clip Mobile Phone Video SMS Transcoding Messaging
  35. 35. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 35 Our System: Web-based Browsing Interface
  36. 36. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 36 Our System – Event, Key Frame, Video Clip
  37. 37. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 37 Mobile Interface SMS Message for Mobile Event Notification video streaming
  38. 38. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 38 Protocol 1.監控 1.監控 新竹 視訊 台北 視訊 分析 分析 3. Video Analytic 確認 Video Analytic 2 3. 1 確認 2.5G 5. 2.5G 5. 3GP 3GP 確認 確認 2. SMS 2. SMS 3G 4.關鍵 畫面 4.關鍵 3G 畫面 6.監控 視訊 6.監控 串流 視訊 串流 輔仁大學
  39. 39. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 39 Performance  Instant messaging  PC: Within 1 seconds  Mobile phone: ~ 5 seconds 上午 12:52 - 上午 12:00 Average time of an event 3.9 sec 上午 12:00 Face detection Skin detection 3.4 sec 1.7 sec 下午 11:45 - 上午 12:00 間隔描述 上午 11:54 - 下午上午 11:31 - 下午 2:24 2:02 上午 11:40 - 下午 2:15 Detection & Key frame MMS tracking Selection 3~5 sec 5.7 sec 5.1 sec Mobile Streaming (RTSP) 上午 12:52 - 上午 12:00 Transcoding 4.1 sec
  40. 40. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 40 2.2 Night Vision  Night vision means to detect moving objects from night images  < 1 lux  High-level noises  Night vision should be important because crimes usually happens at night
  41. 41. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 41 Moving Object Detection  Background subtraction is the most important method  However, it can not work at night  Bad image quality  Bad detection quality
  42. 42. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 42 An Example Environment  Entry of elevators in the hall  Two challenges  Low-light-level: Low contrast, strong noise  Drastic light change: When elevators open, strong light emits from the elevators 緊急出口警示燈 (微弱) 現場唯一光源(門外屋簷) IPCam架設位置 1 2 3 4 真實場景 by RICOH RX200 (Defaults Setting)
  43. 43. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 43
  44. 44. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 44 2.3 Reconfigurable Hardware for Moving Object Detection Background subtraction, ... • 2.8 GHz Intel CPU • Software: C/C++ FPGA • Frame rate: 10 fps for 1 channel
  45. 45. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 45 Background Subtraction Current Frame B k + 1 Background M k +1 ( x, y ) P k+1 Image Update Background Image = Pk +1 ( x, y ) − Bk ( x, y ) - Bk M k + 1 Bk +1 ( x, y ) = αBk ( x, y ) + (1 − α ) Pk +1 ( x, y ) Post Processing Motion Object Image Speed up by (1) Circuit design, (2) Parallelization
  46. 46. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 46 Parallelization by Hardware  Parallelism: 7-level pipeline  SIMD with stream processing
  47. 47. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 47 Design & Results  Hardware: Altera Cyclone II 2C35  Design: Verilog HDL with Quartus II Background New Frame Result Frame rate • Background module : 368 fps • Whole system : 51 fps
  48. 48. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 48 Experimental Comparison  PC: 2.8GHz CPU, C implementation  FPGA can speed up 500 times 2.8G 51 CPU FPGA 25M 10 Clock(Hz) FPS
  49. 49. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 49 2.4 Super-resolution  Object images are too small  Super-resolution is to  Construct a high-resolution image from low-resolution images  Challenges: Ill-posed problem Low-resolution High-resolution image image
  50. 50. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 50 The Proposed Method  Pervious methods  Interpolation, reconstruction  Feature-based learning  Dimension Expansion Y X Image Space Dimension • Maximum A Posteriori Reduction (MAP) Estimator Feature Space Dimension • Gibbs Prior Expansion x • 2D2 Locality High-resolution Preserving Projection Features
  51. 51. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 51 Experiments  Four-time scale  Low-resolution is 70×50  High-resolution is 140×100 Ground Truth Low-resolution Bicubic Our Method NN interpolation interpolation
  52. 52. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 52 Experimental Results
  53. 53. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 53 2.5 Camera Tampering Detection  Possible tampering  Spray-painting  Replacement  Hit/collision  Defocus  Blocking  ...
  54. 54. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 54 Tampering Examples (1/2) Before After Spray painting Defocus
  55. 55. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 55 Tampering Examples (2/2) Before AfterReplacement (Day)Replacement (Night)
  56. 56. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 56 The System  Sabotage detection before visual surveillance algorithms  Server-based solution for large-scale surveillance systems
  57. 57. 王元凱 以視覺為基礎之智慧型環境(Vision Based Intelligent Environment) p. 57 2.6 System Integration in the VBIE Project  Smart Building  Integrated monitoring within building  Tracking target: person  Smart Campus  Long range tracking within campus  Tracking target: car and personFu Jen University Department of Electronic Engineering Yuan-Kai Wang
  58. 58. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 58 Heterogeneous Cameras  We use various kinds of cameras 環場攝影機 PTZ攝影機 紅外線熱像攝影機 固定式攝影機 活動攝影機 活動攝影機畫面
  59. 59. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 59 Smart Building (1/2)  Characteristics  11 techniques are integrated by top- down design  It follows the CMMI software engineering method Documents of spec requirement, testing, ...  A long-term test site is built
  60. 60. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 60 Smart Building (2/2) NTSC一般攝影機 PTZ網路攝影機 魚眼攝影機
  61. 61. Smart Campus (1/2) 13.機器人導引訪客導引 6.智慧型車牌偵測 12.快速人臉辨識 7.行人交錯偵測與追蹤 11.人員連續追蹤 8.特定人員追蹤與特寫 10.軌跡式人員追蹤與檢索3.循園多攝影機轉場監控 9.多頻道視訊平滑轉場 2.多攝影機整合式視訊監控 4.多重解析度顯示及轉換 5.手勢指定物件 1.串聯式車牌辨識 14.整合平台及信號傳輸 61
  62. 62. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 62 Smart Campus (2/2)
  63. 63. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 63 3. Video Sousveillance  Environment cameras  Outside-in images  Body-worn cameras  Inside-out images
  64. 64. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 64 Paradigm Shift  Evolution of computing paradigm  Desktop computing  Mobile computing  Wearable computing  Evolution of camera technology  Desktop vision  Mobile vision  Wearable vision⇒ Egocentric vision
  65. 65. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 65 Desktop Computing No interaction with the environment User is focused on the system stationary
  66. 66. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 66 Jurassic Mobile Computing Goal: Stay in touch for mobile usage
  67. 67. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 67 Current Mobile Computing  Still has drawbacks  Hands are not free  Screen is too small  Need keyboard  Not proactive
  68. 68. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 68 Wearable Computing  Wear the computer on the body Smart glasses Smart clothing
  69. 69. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 69 The Hardware  Computer: Embedded system  Input  No keyboard & mouse Gestures  But sensors  Output Text Audio Special Purpose
  70. 70. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 70 Input Sensor - Glove  Hand Gesture Interaction
  71. 71. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 71 Output Display  Optical see-through HMD (Head Mounted Display)
  72. 72. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 72 Add Vision Sensor  Video see-through HMD Wearable Vision
  73. 73. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 73 Why Wearable Vision (1/2)  Capture of casual events
  74. 74. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 74 Why Wearable Vision (2/2)  Game assistant
  75. 75. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 75 Where to Wear an Extra Eye Mayol-Cuevas, Tordoff, Murray, "On the Choice and Placement of Wearable Vision Sensors," IEEE Trans. SMC A, March 2009.
  76. 76. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 76 Where to Wear an Extra Eye
  77. 77. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 77 Steve Mann’s WearComp S. Mann, "Humanistic Computing: WearComp as a New Framework and Application for Intelligent Signal Processing", Proc. of IEEE, vol. 86, no. 11, 1998.
  78. 78. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 78 US Armys Land Warrior  Helmet with OLED display to show map & troop locations http://en.wikipedia.org/wiki/Land_Warrior
  79. 79. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 79 Demo of ISLab@FJU The X-Eye
  80. 80. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 80 The X-Eye  Live photo management  with more comfortable display Demo
  81. 81. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 81 X-Eye Components 觸控面板 顯示器 移動電源 Camera 自製 USB 外殼 USB 筆電 連接線 Hub BeagleBoard 微投影機 SD卡 USB-WIFI卡 鍵盤 USB-RS232 讀卡機 控制線 滑鼠
  82. 82. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 82 1st Generation Prototype
  83. 83. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 83 Embedded Computer USER RESET OMAP3530 ProcessorPeripheral I/O •600MHz Cortex-A8 •NEON+VFPv3•USB Host •16KB/16KB L1 •256KB L2•JTAG •430MHz C64x+ DSP •32K/32K L1•DVI-D video out •48K L1D •32K L2•S-Video out •Power VR SGX GPU •64K on-chip RAM•SD/MMC+ POP Memory •256MB LPDDR RAM•Stereo in/out •256MB NAND flash•RS-232 serial1•Alternate power•USB 2.0 HS OTG 7.6 cm 83
  84. 84. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 84 Output - Pico Projector  Very small projector  Large screen for mobile usage  Screen: 15”~30”  Resolution: 480 x 320  Brightness: 7 lumens  Contrast ratio: 1000:1 2009~2018年微型投影機市場預估 (單位:百萬台) (來源:DisplaySearch Pocket Projector Technology and Market Forecast Report)
  85. 85. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 85 Input - Camera
  86. 86. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 86 User Interface  Gesture Recognition with Bare Hand  No keyboard & mouse Capture egocentric images
  87. 87. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 87 Photo Mode Capture Command: capture images Switch Command: Mode switch
  88. 88. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 88 Manage Mode Original Photos Next Command Previous Command Switch Command 2(to photo) 88
  89. 89. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 89 X-Eye Hardware Architecture 2010.04.25 89
  90. 90. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 90 X-Eye Software Architecture
  91. 91. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 91 X-Eye Software Stack Module Version Function Gesture Command 1.0 Gesture Recognition Module OpenCV 1.0 Image Processing FFMpeg 0.5.1 Audio/Video Codec QT 4.6.2 Windows Interface VMWare 6.5.3 Virtual OS Ubuntu (host) 9.04 Development OS Ubuntu (client) 9.10 Filesystem Kernel (client) 2.6.29 Linux Kernel 91
  92. 92. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 92 Algorithm (1/2)  Gaussian Mixture Model (GMM)  Model four colors 1 N 1 ( − ( x − mi )T Covi−1 ( x − mi )) p ( x | c) = ∑ ωi e 2 i =1 2π || Covi ||1/2  Expectation Maximization (EM)  Parameter estimation of GMM E Step M Step N 1 ∑ E(z N 1 ω p( x j | m , C ) ωit +1 = t +1 ) m = ∑ E(z t t t )x j E ( zij ) = Nω t +1 i i i ij i ij i N j =1 i j =1 ∑ ω tp p( x j | mtp , C tp ) =C t +1 1 t +1 ∑ N E ( zij )[( x j − mit +1 )( x j − mit +1 )T ] p =1 N ωi i j =1 92
  93. 93. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 93 Algorithm (2/2)  Color Identification  c ˆ = arg max P( x | c j ), j 1 ~ k cj  Performance optimization by Look Up Table (LUT) for real-time  Gesture Recognition  Four gestures: capture, switch, next, previous, 93
  94. 94. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 94 Smart Camera  Egocentric vision needs small and smart cameras FJUCam1 FJUCam2 Image Image Image Capturing Processing Recognition
  95. 95. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 95 FJUCam1 - Hardware • Weight: 35gm • Power sources: •Size: • 5V DC current 6 x 4.5 x 5 (cm) • 5V Cell Battery (W x H x D) • Power consumption: 1W Three Modules 1. Main board, 2. Lens module 3. Storage module
  96. 96. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 96 FJUCam1 - Software  Development environment  C Language  PC Windows + Cygwin + GCC  cc3 library (open source developed by CMU)
  97. 97. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 97 FJUCam1 - Face Detection  The Adaboost algorithm  Proposed by Viola and Jones in 2001  Cascaded weak classifiers(21 cascades)  Algorithm refinement  Reduced to 5 cascades  Fixed-point arithmetic  Stream processing for only 64KB memory utilization Image FJUCam Display Face Detection
  98. 98. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 98 FJUCam2 (1/5)  Cons of FJUCam1  Low speed, small memory  Low image resolution  Next-generation FJUCam  Adopts multicore technology CPU + DSP (MPSoC chip) 99/12 Image RAM 128MB Resolution FJUCam2 VGA 98/2 64KB FJUCam1 CIF Processor 60MHz 600MHz Speed
  99. 99. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 99 FJUCam2 (2/5)  TI OMAP3530@ Cortex-A8 600MHz @ DSP C64x+ 412MHz @ PowerVR SGX 530  Integrated L1 memory for ARM (16kB I- Cache, 16kB D-Cache, 256kB L2)  256MByte low power mobile DDR  512MByte NAND Flash  Mini HDMI interface  Serial port(UART/RS232) x 2  PCB dimensions  USB port (Host)  55mm × 55mm  USB port (Client)  Outside dimensions  Micro SD card slot (L×W×H)  USB power Back side – Expansion  63mm 59mm×13 mm  GPIO socket for extra peripherals  Weight : 20 grams
  100. 100. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 100 FJUCam2 (3/5) Software (on board)  Angstrom with Linux kernel 2.6.32 Camera Input using ARM  VCam Image/Video OpenCV+FFMpeg DSP processing demo Image/Video  Using DSP and the Processing integrated vision libraries GUI Output using QT  Other supported embedded OS: easy interface & powerful computing Android  We are compiling “VCam Laboratory Manual” which will be made publicly at the end of 2010
  101. 101. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 101 FJUCam2 (4/5)  Algorithms going to be developed for FJUCam2  Color tracking  Gesture recognition  Face tracking and recognition  Human/hand/finger detection  Event detection  Video summarization  Distributed vision processing
  102. 102. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 102 FJUCam2 (5/5)  Possible applications  Wearable computing  Augmented reality  Robotic vision  Visual surveillance  Medical applications for patient monitoring
  103. 103. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 103 4. Conclusions  Image processing and computer vision  is funny, is cool  Has been realistic, comes into real life  It is time to study it  Hardware is small, cheap, and wearable  More robust algorithms
  104. 104. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 104 How to Study  Basic courses  Programming skills  Engineering Mathematics  Signal and Systems  Digital Signal Processing  Advanced courses  Digital Image Processing  Computer Vision  Pattern Recognition  Artificial Intelligence
  105. 105. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 105 Activities  中華民國影像處理與圖形識別學會(IPPR)  電腦視覺、圖學暨影像處理研討會(CVGIP)  IEEE (國際電機電子學會)  Transactions on  Image Processing  Pattern Analysis and Machine Intelligence  Medical Imaging  Conferences  ICIP, ICME, ICPR, ...
  106. 106. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 106 The End Free for Any Questions
  107. 107. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 本簡報授權聲明  此簡報內容採用 Creative Commons 「姓名標示 - 非商 業性台灣 3.0 版」授權條款  歡迎非商業目的的重製、散布或修改本簡報的內容,但 請標明: (1)原作者姓名:王元凱; (2)圖標示:  簡報中所取用的部份圖形創作乃截取自網際網路,僅供 演講者於自由軟體推廣演講時主張合理使用,請讀者不 得對其再行取用,除非您本身自忖亦符合主張合理使用 之情狀,且自負相關法律責任。

×