A pyramid is a structure whose outer surfaces are triangular and converge to a single step at the top, making the shape roughly a pyramid in the geometric sense. The base of a pyramid can be trilateral, quadrilateral, or of any polygon shape. As such, a pyramid has at least three outer triangular surfaces. Wikipedia पिरामिड जैसे ज्यामितीय आकार से मिलती जुलती संरचनाओं को पिरामिड कहते हैं। विश्व में बहुत सी संरचनाएँ पिरैमिड के आकार की हैं जिनमें मिस्र के पिरामिड बहुत प्रसिद्ध हैं। पिरामिड आकार की संरचनाओं की सबसे बड़ी विशेषता यह है कि इसके भार का अधिकांश भाग जमीन के पास होता हैA pyramid is a structure whose outer surfaces are triangular and converge to a single step at the top, making the shape roughly a pyramid in the geometric sense. The base of a pyramid can be trilateral, quadrilateral, or of any polygon shape. As such, a pyramid has at least three outer triangular surfaces. Wikipedia पिरामिड जैसे ज्यामितीय आकार से मिलती जुलती संरचनाओं को पिरामिड कहते हैं। विश्व में बहुत सी संरचनाएँ पिरैमिड के आकार की हैं जिनमें मिस्र के पिरामिड बहुत प्रसिद्ध हैं। पिरामिड आकार की संरचनाओं की सबसे बड़ी विशेषता यह है कि इसके भार का अधिकांश भाग जमीन के पास होता है

1. Computer Vision
Mubarak Shah
shah@eecs.ucf.edu

2. Computer Vision
 The ability of computers to see.
 Image Understanding
 Machine Vision
 Robot Vision
 Image Analysis
 Video Understanding

3. A picture is worth a thousand
words.

4. A word is worth a thousand
pictures.
A HUNT

5. Image
 2-D array of numbers (intensity values,
gray levels)
 Gray levels 0 (black) to 255 (white)
 Color image is 3 2-D arrays of numbers
 Red
 Green
 Blue
 Resolution (number of rows and columns)
 128X128
 256X256
 512X512
 640X480

7. Image Formats
 TIF
 PGM
 PBM
 GIF
 JPEG

8. Video
 Sequence of frames
 30 frames per second
 Formats
 AVI
 MPEG
 Quick Time

9. Video Clip

10. Sequence of Images

11. Image Formation
 Light Source
 Camera (extrinsic and intrinsic
parameters)
 Scene (Surface reflectance, Surface
shape )

12. Perspective Projection (Pin
Hole)
(X,Y,Z)
Lens
Image Plane
image
Z
y
f
World
point

13. Orthographic Projection
(X,Y,Z)
Image Plane
image
y
World
point

14. Shape from X
 Recover 3-D shape from 2-D image(s)
 Stereo
 Motion
 Shading
 Texture
 Contours

15. Stereo

16. http://www.vision3d.com/stereo.html

17. Renault Stereo Pair

18. Depth Map

19. Shape from Shading

20. Lambertian Model
S=L, light
source
I=S.N

21. Vase
(1, 0, 1)
(-1, 1, 1)
(-1,-1, 1)

22. Shape from Texture

23. Visual Motion

24. Shape from Motion: Moving
Light Display

25. Shape from Motion

26. Photosynth

27. Sequence Raw Optical flow
27

28. Video Clip & Mosaic

29. Applications of Computer
Vision
 Face Recognition
 Object Recognition
 Video Surveillance and Monitoring
 Object detection, tracking and behavior analysis
 Remote Sensing: UAVs
 Robotics
 Computer Graphics

30. Object Recognition
Finding People in images
Problem 1: Given an image I
Question: Does I contain an image of a
person?

31. “Yes” Instances

32. “No” Instances

33. Localize People (Human Detection)

34. Human Detection

35. Airplanes

36. Motor Cycles

37. Face Recognition

38. FACIAL EXPRESSIONS
RAISE EYE BROWS SMILE

39. Detecting Driver Alertness

40. Lipreading

41. Video Surveillance and Monitoring
 Automated Surveillance System (Detection &
Tracking)
Object detection Object tracking
Object categorization
and classification
Event or Activities
Recognition

42. • Part of theWAS (wide area surveillance) project executed by the
Homeland Security Advanced Research Project Agency (HSARPA)
• Current Sensor
• 8 high-resolution cameras
• provide a 100 mega-pixel, 360°field of view
• frame rate: 5 frames per second
• Next Generation
• 48 cameras with significantly higher resolution
• smaller size
NONA: Project Overview
Current Sensor Next Generation

43. NONA Sysmtem—Airport
Sequence 1

44. UAV: Unmanned Aerial Vehicle

45. UAVs: Unmanned Aerial
Vehicles (Drones)
Global Hawk
Predator
Microdrone

46. KINGFISHER AEROSTAT BALLOON
8/21/2012 Computer Vision Lab, UCF 46

47. COCOA – System Flow
Ego Motion
Compensation
Feature based + Gradient
Based
Motion Detection
Accumulative Frame
Differencing + Background
Modeling + Object
Segmentation
Object Tracking
Kernel Tracking + Blob
Tracking + Occlusion Handling
Telemetry*
Aerial
Video
Registered Images Motion Detection Tracks
COCOA
Event Detection & Indexing

48. Registration Result ‐ I
Aerial Video - EO Mosaic
Alignment Mask

49. Registration Result ‐ II
Aerial Video - IR Mosaic
Alignment Mask

50. Detection Results

51. Tracking Results

52. Wide Area Surveillance

53. Wide Area Surveillance

54. Tracking Results

55. Robot Vision (Unmanned Ground
Vehicle)
UGV

56. UGV

57. Human Action Recognition

58. Events, Actions, Activities,
• Action
• Event
• Movement
• Activity
• Interaction
• Verb
• ….

59. • 10 actions
• 9 actors per action
Bend Jack Wave 2 hands Wave 1 hand Jump in place
Sidestep Hop Walk Skip Run
Weizmann Action Dataset

60. KTH Data Set
• Six Categories, 25 actors, 4 instances, 600. clips
Boxing Hand Clapping Hand Waving
Jogging Running Walking

61. UCF Sports Action Dataset
Bench Swing Dive Swing Run
Kick Lift Ride Golf Swing Skate
9 actions, 142 videos.

62. IXMAS Multi‐view Data Set
• 13 action categories, 4 camera views, 10 actors, 3 instances.
View 1 View 2 View 3 View 4
Check Watch
Get Up

63. UCF YouTube Action Dataset (UCF‐11)
Cycling Diving Golf Swinging Riding
Juggling Basketball Shooting Swinging Tennis Swinging
Volleyball Spiking Trampoline Jumping Walking Dog

64. Baseball Pitch Basketball Shooting Biking
Bench Press Billiards
Breaststroke Clean and Jerk Drumming
Diving Fencing
Golf Swing High Jump Horse Riding
Horse Race Hula Hoop
Javelin Throw Juggling Balls Jump Rope
Jumping Jack Kayaking
Lunges Military Parade Nun chucks
Mixing Batter Pizza Tossing
UCF50

65. Playing Guitar Playing Piano Playing Violin
Playing Tabla Pole Vault
Pommel Horse Pull Ups Push Ups
Punch Rock Climbing Indoor
Rope Climbing Rowing Skate Boarding
Salsa Spins Skiing
Ski jet Soccer Juggling TaiChi
Swing Tennis Swing
ThrowDiscus Trampoline Jumping Walking with a dog
Volleyball Spiking Yo Yo
UCF50

66. Microsoft Kinect sensor
• Data Captured using Microsoft Kinect sensor
• Approximately 50,000 gesture samples
RGB Camera
IR Camera 1 IR Camera 2

67. Gesture Lexicons
Gestures from Depth camera
Gestures from RGB camera
Diving Signals Referee Signals Nurse Gesture Music Notes

68. Discovered Primitives
Representative Motion Primitives (out of 136) for different batches
Left arm moving down
Left arm moving up
Left arm moving to body
Left hand moving forward
Left arm waving up
Left arm moving away from body
Right arm moving laterally up
Right arm moving away from body
Left shoulder moving to right
Left arm moving to right
Left arm moving down
Left shoulder moving down
Right arm moving up
Left arm moving up
Left arm moving down
Right arm moving down

69. Test case 1: Torso motion adds noise (devel 01– 10 gestures)
Instances of Successful Recognition
Instances of Failed Recognition

70. Instances of Successful Recognition
Instances of Failed Recognition
Test Case 2: Improvisations (devel 06 – 9 gestures)

71. Test Case 3: Subtle differences (devel 09 – 10 gestures)
Instances of Successful Recognition
Instances of Failed Recognition

73. High Density Crowded Scenes
Political Rallies Religious Festivals Marathons High Density
Moving Objects

74. Tracking in Crowds
 Average chip size 14 x 22 pixels
 492 Frames
 Selected 199 athletes for tracking
 Successfully tracked 143 athletes

75. Results

76. Experiment – 1

77. Experiment-3
 Average chip size 14 x 17 pixels
 453 Frames
 Selected 50 athletes for tracking

78. Experiment – 3

79. Behaviors in Crowded Scenes
Bottleneck Departure Lanes Arch/Rings Blockings

80. Where Am I?

81. “Where Am I?”
 Problem:
Accurate Image Localization
Input Output
Mere Visual Information(Images) Location in Terms of λ (Lon.) and φ (Lat.)
=40.4419, =-79.9986

82. Geospatial Trajectory Extraction

83. Computer Vision for Computer
Graphics

84. Video Completion

85. Layer Based Video Composition

86. Results of Doll

87. Results of Mom-Daughter

88. Multimedia: Segmentation of
Moving-Sounding Objects
88
Accepted in IEEE Transactions on
Multimedia

89. New York Times, August 19,2012
Robot arms like those at a Philips Electronics
factory in the Netherlands can perform the same
tasks as hundreds of low-skill workers.