Nikhil Mane -100731
Saurabh Shigwan -100753
Noel Francis -100674
• An Introduction
• Why Virtual Mouse is Needed
• Existing System
• Proposed System
• Flow Charts
• Hardware and Software Requirements
• Work Done
Table of Contents
• “Your mouse has moved. Windows must be
restarted for the change to take effect.”
<click on OK to continue>
SPATIAL – An Introduction
• There is no other more popular navigation/interaction device
than the mouse.
• Overcoming the restrictions a corporeal device has would
enrich the experience of interaction.
• Keeping this in mind, we present an idea of implementing a
virtual mouse system as an alternative.
Virtual Mouse is:
• User makes a specified hand gesture that is captured by a
• Object recognition techniques are used to extract
information from the capture.
• This is then translated to some meaningful event on the
Why VIRTUAL MOUSE is Needed
• Mouse is a physical device subject to mechanical wear
• It is not easy to adapt to different environments and is
often limited by environment.
• Limited functions even in present operational
• VIRTUAL MOUSE hopes to fulfill these requirements of
the user by overcoming normal physical barriers.
• The user rolls the ball with the thumb, fingers, or the palm
of the hand to move a cursor.
• Large tracker balls are common on CAD workstations for
• Before the advent of the touchpad, small trackballs were
common on portable computers.
• Usually not as accurate as a mouse.
• Ball mechanism of trackballs requires more frequent
cleaning than a mouse.
• Not very user friendly.
Mechanical Mouse :
• A single ball that could rotate in any direction.
• As part of the hardware package of the Xerox
• Detection of the motion of the ball was light based with
the help of chopper wheels.
• Cannot provide high precision performance.
• Has specific surface requirements to operate.
• Needs more desk space when compared with a
Optical Mouse :
• Uses a light-emitting diode and photodiodes to detect
movement relative to the underlying surface.
• Digital image correlation, a technology pioneered by the
defense industry for tracking military targets.
• Use image sensors to image naturally occurring texture
in materials such as wood, cloth, mouse pads and
• Image captures in continuous succession and
comparison to determine mouse movement.
• Special hardware required.
• Again, specific surface requirements.
Features of VIRTUAL MOUSE:
• Any new product should either make human life
more comfortable, more productive or more fun.
• Provides greater flexibility than the existing system.
• Can provide more functions depending on the choice
• Easy to modify and adapt
• Less prone to physical damage due to absence of a
fixed physical device.
• Avoid the mouse-related wrist damage like CTS & RSI.
• Also, there is a certain degree of fun & entertainment
associated with the whole idea.
It is divided into the following modules:
• Module 1 (Image Acquisition).
• Module 2 (Object Recognition).
• Module 3 (Object Tracing and Information Retrieval).
• Module 4 (Point Coordinate Calculation and Motion Analysis).
• Module 5 (Setting Cursor Position)
• Module 6 (Event Generation)
Flow charts (continued)
Final Flow chart
Hardware and Software Requirements
• Intel Pentium D processor 1.8 GHz or AMD Athlon X2 processor 1.8
GHz or higher
• 3 GB RAM
• 5 GB HDD space
• Peripheral webcam at least 30 frames/second, 640x480 resolution
• Windows XP x86 or higher (for x86 environment)
• Windows XP professional x64 or higher (for x64 environment)
• .NET framework 3.5 or higher
• Visual Studio 2008 professional
• EmguCV library (wrapper of OpenCV library for .NET framework)
• EmguCV library 64 bit binaries (for developing on x64 environment)
• Webcam drivers (device specific)
• Studied the existing systems and available information
• Hence, proposed a new system.
• Main purpose – to eliminate physical constraints and
• A literature survey in relation to the various contexts
• Implementation work has been started.
• VIRTUAL MOUSE is an idea of implementing an adaptable, multi-
functional navigation/interaction tool that overcomes physical
• The system will be ‘real’ enough to not affect the interaction much.
• Ease of use is the foremost concern.
• Availability, Adaptability and Ability
• Project goal will be to build a system that satisfies all three ideals
Future Enhancement :
• Development for specific objects.
• Creation of particular action areas for utility.
• More advanced and highly specific functionality.
• 1. Richard E. Woods, Rafael C. Gonzalez, “Digital Image Processing”, Pearson Education Asia, 3/E, 2008
• 2. Ming-Hsuan Yang, Narendra Ahuja, “Face detection and gesture recognition for human- computer interaction” O’Reilly, 2006.
• 3. Gary Bradski, Adrian Kaehler, “Learning OpenCV: Computer Vision with the OpenCV Library”, Shroff/O'Reilly, 2008
• 4. Stanley B. Lippman, Josee Lajoie, “A C++ Primer”, Addison-Wesley, 3rd Edition 2008
• 5. Ivor Horton, “Ivor Horton's Beginning Visual C++ 2008”, Wiley India Pvt. Ltd/Wrox, 2008
• 6. Jon Skeet, “C# in Depth”, Manning Publication, Second Edition, 2008
• 7. Kogent Solutions Inc. , “.NET 3.5 Programming Black Book: Covering .NET Framework, VB 2008, C# 2008, And ASP.NET 3.5” , Wiley India Pvt.
• 8. Christian Nagel, Bill Evjan, Jay Glynn, Morgan Skinner, Karli Watson, “Professional C# 2008”, Wiley, 2008
• 9. Chensheng Wang, Fei Wang “A Knowledge-based Strategy for Object Recognition and Reconstruction”, 2009
• 10. Soo Chahn Lee, Duck Hoon Kim, IlDongYun andSangUk Lee School of EECS, “How Can We Evaluate Object Recognition Algorithms” ,IEEE, 25
• 11. M. A. Moni, A B M Shawkat Ali, “HMM based Hand Gesture Recognition: A Review on Techniques and Approaches”, IEEE, 03 January 2006
• 12. Jacinto Nascimento, Jorge S. Marques,”New Performance Evaluation Metrics for Object Detection Algorithms”, ISR/IST, 2006.
• 13. http://opencv.willowgarage.com/wiki/
• 14. http://en.wikipedia.org/wiki/OpenCV’
• 15. http://www.emgu.com/wiki/index.php/OpenCV
• 16. http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html
• 17. http://msdn.microsoft.com/en-us/library/aa139615.aspx
• 18. http://www.youtube.com/user/badrepent
• 19. http://stackoverflow.com/questions/3652327/choice-between-win32-apis-and-net-framework
• 20. http://stackoverflow.com/questions/3762081/using-c-for-real-time-applications
• 21. http://stackoverflow.com/questions/3691198/selecting-an-appropriate-ide