Kinect Arabic Interfaced Drawing  Application By : Mohammed Hethnawi Mohammed Zaza   Subhi Mara’ba  Yasser Hisham
Contents <ul><li>Abstract </li></ul><ul><li>Introduction </li></ul><ul><li>Kinect Skeleton Gesture Capture </li></ul><ul><...
Abstract <ul><li>Introduces brief history about Kinect. </li></ul><ul><li>Defines Kinect sensors. </li></ul><ul><li>Presen...
Introduction <ul><li>Brief history about Kinect : </li></ul><ul><li>The technology behind Kinect was invented in 2005 by Z...
Introduction <ul><li>Kinect sensors : </li></ul><ul><li>The Kinect sensor is a horizontal bar connected to a small base wi...
Introduction <ul><li>Kinect Drivers : </li></ul><ul><ul><li>OpenNI ( Open Natural Interaction ) : </li></ul></ul><ul><li>I...
Introduction <ul><li>Kinect Drivers : </li></ul><ul><ul><li>Microsoft SDK beta 2 : </li></ul></ul><ul><li>It includes Wind...
Tutorial  <ul><li>This application allows the user to draw on his canvas using his body gestures. The user has the choice ...
Tutorial(continue) <ul><li>The user has to lower his left hand below his shoulder to start painting. </li></ul>
 
Bitmap implementation <ul><li>The drawing application provides four different brushes </li></ul><ul><li>With different bru...
Cursor implementation <ul><li>The first challenge we faced was that the Kinect lacks the feature of mouse click. </li></ul...
Kinect Curser <ul><li>Instead of using joints movement as indicators of certain actions we used joints coordinates. </li><...
Kinect Curser (continue) <ul><li>We also used the feature of coordinates as an action as we introduced left hand joint y c...
Kinect Curser (continue) <ul><li>It should also be mentioned that our application runs on kinect if it is available and if...
Further work <ul><li>We will divide our further work into three main fields. </li></ul><ul><li>1- Develop the drawing proj...
Developing the drawing application <ul><li>This application is a simple drawing app. However we are planning to add more u...
Getting things kinectified !! Why NOT     <ul><li>Here we introduce an interesting idea about getting any software kinect...
Deaf Language Translator ( DLT ) <ul><li>This software is not developed yet in any part of the world, the idea is to trans...
Summary and Conclusion <ul><li>Kinect technology is still under development which made our task harder to implement. </li>...
References <ul><li>Microsoft SDK samples </li></ul><ul><li>OpenNI samples </li></ul><ul><li>Microsoft SDK beta 2 documenta...
 
Upcoming SlideShare
Loading in …5
×

Kinect Arabic Interfaced Drawing Application

1,372 views

Published on

Kinect Arabic Interfaced Drawing Application

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,372
On SlideShare
0
From Embeds
0
Number of Embeds
91
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Kinect Arabic Interfaced Drawing Application

  1. 1. Kinect Arabic Interfaced Drawing Application By : Mohammed Hethnawi Mohammed Zaza Subhi Mara’ba Yasser Hisham
  2. 2. Contents <ul><li>Abstract </li></ul><ul><li>Introduction </li></ul><ul><li>Kinect Skeleton Gesture Capture </li></ul><ul><li>Bitmap Implementation </li></ul><ul><li>Combining Bitmap Drawer with Kinect Gesture Capture </li></ul><ul><li>Future Work </li></ul><ul><li>Summary and Conclusion </li></ul>
  3. 3. Abstract <ul><li>Introduces brief history about Kinect. </li></ul><ul><li>Defines Kinect sensors. </li></ul><ul><li>Presents a brief info about Kinect Drivers </li></ul><ul><ul><li>OpenNI </li></ul></ul><ul><ul><li>Microsoft SDK beta 2 </li></ul></ul><ul><li>Provides a brief explanation of our bitmap helper class implementation </li></ul><ul><li>Presents our techniques of getting the advantage of capturing user’s gestures for implementing our drawer </li></ul><ul><li>Reveal early thoughts of our future work </li></ul>
  4. 4. Introduction <ul><li>Brief history about Kinect : </li></ul><ul><li>The technology behind Kinect was invented in 2005 by Zeev Zalevsky, Alexander Shpunt, Aviad Maizels and Javier Garcia. </li></ul><ul><li>In November 2010, Adafruit Industries offered an open-source driver for Kinect . </li></ul><ul><li>In December 2010, PrimeSense, released their own open source drivers ( which includes OpenNI organisation ) along with motion tracking middleware called NITE. </li></ul><ul><li>On June 16, 2011, Microsoft announced its official release of its SDK for non-commercial use. </li></ul>
  5. 5. Introduction <ul><li>Kinect sensors : </li></ul><ul><li>The Kinect sensor is a horizontal bar connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or below the video display . </li></ul><ul><li>The device features an “ RGB camera, depth sensor and multi-array microphone running proprietary software ”, which provide full-body 3D motion capture, facial recognition and voice recognition capabilities . </li></ul>
  6. 6. Introduction <ul><li>Kinect Drivers : </li></ul><ul><ul><li>OpenNI ( Open Natural Interaction ) : </li></ul></ul><ul><li>It’s an industry-led, not for profit organization focused on certifying and improving interoperability of natural interaction devices. </li></ul><ul><li>Natural Interaction Devices are devices that capture body movements and sounds to allow for a more natural interaction of users with computers. </li></ul><ul><li>The OpenNI framework provides a set of open source APIs , which provide support for Voice and voice command recognition , Hand gestures , and Body Motion Tracking </li></ul>
  7. 7. Introduction <ul><li>Kinect Drivers : </li></ul><ul><ul><li>Microsoft SDK beta 2 : </li></ul></ul><ul><li>It includes Windows 7 compatible PC drivers for Kinect device. </li></ul><ul><li>It provides Kinect capabilities to developers to build applications with C++, C#, or Visual Basic by using Microsoft Visual Studio 2010. </li></ul><ul><li>It includes the following features : raw sensor streams, Skeletal tracking, Advanced audio capabilities. </li></ul>
  8. 8. Tutorial <ul><li>This application allows the user to draw on his canvas using his body gestures. The user has the choice to select a brush of his choice with a certain font size. The user also has the ability to choose the color he wants from a list of colors. The user can as well save his work and review it later. </li></ul><ul><li>The user has to raise his left hand above his shoulder and moves the cursor with his right hand. </li></ul>
  9. 9. Tutorial(continue) <ul><li>The user has to lower his left hand below his shoulder to start painting. </li></ul>
  10. 11. Bitmap implementation <ul><li>The drawing application provides four different brushes </li></ul><ul><li>With different brush font sizes: </li></ul><ul><li>1- eraser </li></ul><ul><li>2- airbrush </li></ul><ul><li>3- pen </li></ul><ul><li>4- brush; </li></ul><ul><li>We used a linear interpolation function to implement drawing bitmaps between line segments. </li></ul><ul><li>To destinguish between different brushes we used different number of steps in each brush interbolation function. </li></ul>
  11. 12. Cursor implementation <ul><li>The first challenge we faced was that the Kinect lacks the feature of mouse click. </li></ul><ul><li>Thus we used the principle of hovering over regions for a certain amount of time instead to implement mouse click. </li></ul><ul><li>We extended the C# given buttons and check lists classes with hovering methods to facilitate the use of Kinect </li></ul><ul><li>We introduced a focused regions for each button that hovering over enable it ( cause and event such as ischecked … etc) </li></ul>
  12. 13. Kinect Curser <ul><li>Instead of using joints movement as indicators of certain actions we used joints coordinates. </li></ul><ul><li>We implemented the mouse movements by replacing the mouse coordinate with the right hand joint coordintaes (x,y) thus making the cursor trace the right hand movements. </li></ul><ul><li>“ Nui.Vector nuiv = skeleton.Joints[Nui.JointID.HandRight].ScaleTo((int)ActualWidth, (int)ActualHeight, 0.50f, 0.30f).Position; </li></ul><ul><li>CursorPosition = new Point(nuiv.X, nuiv.Y);” </li></ul>
  13. 14. Kinect Curser (continue) <ul><li>We also used the feature of coordinates as an action as we introduced left hand joint y coordinate and the left shoulder joint y coordinate and compared between them as a threshold for virtiualized mouse click. </li></ul><ul><li>“ Nui.Joint lh = skeleton.Joints[Nui.JointID.HandLeft]; </li></ul><ul><li>Nui.Joint ls = skeleton.Joints[Nui.JointID.ShoulderLeft]; </li></ul><ul><li>bool isdown = lh.Position.Y < ls.Position.Y;” </li></ul>
  14. 15. Kinect Curser (continue) <ul><li>It should also be mentioned that our application runs on kinect if it is available and if not it can use the mouse for the cursor coordinates. </li></ul><ul><li>“ Loaded += (s, e) => </li></ul><ul><li>{ </li></ul><ul><li>if (MainWindow.Instance == null || MainWindow.Instance.NuiRuntime == null) </li></ul><ul><li>CompositionTarget.Rendering += (s2, e2) => OnUpdate(null, new Nui.SkeletonFrameReadyEventArgs()); </li></ul><ul><li>else </li></ul><ul><li>MainWindow.Instance.NuiRuntime.SkeletonFrameReady += OnUpdate; </li></ul><ul><li>};” </li></ul>
  15. 16. Further work <ul><li>We will divide our further work into three main fields. </li></ul><ul><li>1- Develop the drawing project and add more features to it. </li></ul><ul><li>2- Develop the keyboard simulator (get things kinectified !! Why NOT  ) </li></ul><ul><li>3- Introducing an advanced program to translate the Deaf language. </li></ul>
  16. 17. Developing the drawing application <ul><li>This application is a simple drawing app. However we are planning to add more utilities such as: </li></ul><ul><li>1- More brushes. </li></ul><ul><li>2- Auto complete and auto correct utility. </li></ul><ul><li>3- Simplify the drawing techniques. (two handed drawing). </li></ul><ul><li>4- Add more standard shapes. </li></ul>
  17. 18. Getting things kinectified !! Why NOT  <ul><li>Here we introduce an interesting idea about getting any software kinectified. </li></ul><ul><li>Keyboard simulator is a smart interfacing software that shows you a list of the running programs all you have to do is just picking one and start sending characters to it by Kinect gesture detector. </li></ul>
  18. 19. Deaf Language Translator ( DLT ) <ul><li>This software is not developed yet in any part of the world, the idea is to translate the speechless moves using the kinect gesture detector and then tells us what each move means. </li></ul><ul><li>Sounds good and Humanist but very hard to implement since we have to define each move and we cant use the already implemented moves !! </li></ul>
  19. 20. Summary and Conclusion <ul><li>Kinect technology is still under development which made our task harder to implement. </li></ul><ul><li>We faced many challenges during the process such as: </li></ul><ul><ul><li>Having to program using xml and C# which have no previous experience in. However we got the chance to get introduced to them which is beneficial for us as IT engineers. </li></ul></ul><ul><ul><li>Implementing the bitmap class was one of the hardest tasks. We used many resources and bitmap classes to implement. </li></ul></ul><ul><li>After all this was one interesting and fun to do project ( hopefully prebaring us for our graduation project). </li></ul>
  20. 21. References <ul><li>Microsoft SDK samples </li></ul><ul><li>OpenNI samples </li></ul><ul><li>Microsoft SDK beta 2 documentation </li></ul><ul><li>www.wikipedia.com </li></ul><ul><li>Our website : http://alrasam.0fees.net/about.html </li></ul>

×