1. Pratheba Selvaraju
( 4 Homework + 1 Project )
(University of Washington)
Real-Time Designing and Modeling
Conclusions
Idea:
Introduction
Real Time Designing and Modelling:
Everyday engagement in
Conversation over video chat
• It will be useful to present
a design idea on the spot without
Use of any special software.
• Fun to play games where you
Model your own character.
• Show the person in other end
of video conference a new
Idea for a car design.
Approach- Phase I
Probability Mask and Bounding Box:
• Skin-Color-Probability mask
•
•
•
~ 20 frames
• Optical-flow-Probability mask
•
• Region of Interest
Centroid and BoundingBox
Results :
•Due to high number of processing,
frame rate is low, which results in
not a very smooth drawing.
•Very high speed movements
could not be tracked.
• To design withoutadditional software
• No additional hardware
• Must work with lowresolution camera
• Must be real-time withlittle to no-preprocessing
• Use of Hand / FingerTip to draw
and design.
• Use of motion to detect which object
To track for drawing
• Get the drawing to create 3D model
• Augment it to the video
Requirement
System Overview
Use case
Meeting Presentation
Gaming Application
• Face is detected from each frame
• ROI of face region is averaged
• HSV range of skin color is obtained, (bin
size = range)
• Hue Range is Obtained from HSV
.
.
.
Averaged face
HSV range
Haar Cascade classifier
Viola-Jones
(Frontal face)
Hue Mask
References
•Araki, R. ; Waseda Univ., Tokyo, Japan ;
Gohshi, S. ; Ikenaga, T. Real-time both hands
tracking using CAMshift with motion mask and
probability reduction by motion prediction,
Signal & Information Processing Association
Annual Summit and Conference (APSIPA ASC),
2012 Asia-Pacific
•Dorin Comaniciu,, Mean Shift : A Robust
approach towards feature space, IEEE
transactions on Pattern Analysis and Machine
intelligence
•Jean-yves Bouguet, Pyramidal implementation
of the Lucas Kanade feature tracker, Intel
Corporation, Microprocessor Research Labs,
2000.
Previous
Frame
Current
Frame
Grid
Features
Where is Probability mask
and is the x and y flow
vector
Pyramidal Lucas Kanade
Dense Optical
Flow
FlowmaskDense Optical
Flow
Normalize to range
between 0 – 255
= threshold = 255
= threshold = 0
• Optical flow image is smoothed, filtered and dilated
(~ 3 iteration) to remove noise.
• Extract image from FlowMask with range 200-255
• Normalize the extracted image to 0-255
• Obtain Contours and convex hull
Three coin algorithm
• Ability to draw in real-time with
Only webcam (low resolution).
• Can be used to track any moving
With better accuracy than traditional
Camshift.
ContourImage Convex Hull
FlowmaskHue Mask
• Regular camshift uses only the color
probability, which might result in incorrect
tracking due to noise or object with same
color range
• With addition of flow mask, it correctly
identifies the object to be tracked.
• Depending upon speed of object,
threshold level is adjusted.
Bounding box andcentroid used for initial
setting initial region ofinterest for tracking
where, 0 ≤ α ≤ 1 and 0 ≤ β ≤ 1
• When object start moving, camshaft tracks the
object
• Previous and current tracking box positions are
obtained.
• Line is drawn between center of previous and
current box position
Future Tasks:
•Improve performance processing.
•Convert 2D drawing into 3D model
real-time
(either via classification of shapes
or by rendering exact 3D shape of
2D drawing)
•Augment the 3D model in video in
real-time
•Send it via network to be viewed
by other person in chat