Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Virtual Interaction Using Myo And Google Cardboard (slides)


Published on

This project focuses on developing a mobile application that integrates Google Cardboard and Myo Armband. The mobile application developed is an educational application that teaches users to write Japanese characters by getting users to trace the characters display on the phone screen. The users will air draw the letters while the Myo Armband will capture the gestures, send the data to the smartphones and display the drawn character on the screen.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Virtual Interaction Using Myo And Google Cardboard (slides)

  1. 1. VIRTUAL INTERACTION USING MYO AND GOOGLE CARDBOARD Name : ChongWee Nee ID : 1122702155 Supervisor : Dr Poo Kuan Hoong Co-Supervisor : IanTan Moderator : DrWongYa Ping
  2. 2. Project Objectives • To develop a Google Cardboard application and integrate the Myo Armband into the 3D environment. • To enable user to control the 3D environment using the Myo Armband. • To provide an immersive experience of learning Japanese characters
  3. 3. Project Scope • Use only 5 gestures • Positional tracking not included • Capture hand movement and transform the data to display air drawn characters in the virtual world. • The Japanese character will be limited to a few (Hiragana). • The application will be able to determine if the Japanese character drawn is correct or wrong.
  5. 5. Myo Armband • Reads muscle activity in forearm • Electromyography technique • EMG sensor & Motion sensor (9-axis IMU) • Detect 5 gestures • Connect to devices via Bluetooth
  6. 6. Google Cardboard
  8. 8. Virtual Reality Devices Oculus Rift HTC Vive Sony’s Project Morpheus Samsung Gear VR Host Device Windows PC Windows PC Play Station 4 Galaxy Note 4, S6, S6 Edge Field Of View 110° Approx. 100° Approx. 100° 96° Latency Less than 20ms 22ms Less than 18ms Less than 20ms Refresh rate 90Hz 90Hz 120Hz Mere 60Hz Positional Tracking Yes Yes (Lighthouse) Yes No
  9. 9. Gesture Recognition Devices Device Method of Gesture Recognition Kinect Depth Sensor Play Station Move Camera, Orb on the controller Wii Wii Remote, Sensor bar Leap Motion 2 cameras, 3 infrared LED
  10. 10. Requirements Hardware • Android phone (Android 4.3 and up & Bluetooth LE 4.0) • Google Cardboard • Myo Armband Software • Android Studio 1.0 or higher • Google Cardboard sdk • MYO Android sdk • Unity 5.0 • Blender
  11. 11. Flow Diagram User Myo Armband Phone Identify the Pose hand pose head movement raw EMG data gestural data gestural data gestural data manipulated classroom phone movement updated orientation ofVR world Plugin Identify the control assigned to the gesture Track Rotation &Track Position
  12. 12. FYP 1 Accomplishments • 3D classroom environment • Integrate Myo Armband into 3D environment • Created plugin to connect Android-Myo, Unity
  13. 13. FYP 2 Accomplishments • Capture hand movement and convert into drawing (gyroscope) • Included two Japanese characters for users to trace. • Added sound effect and background music. • Identify whether the strokes are drawn correctly (compare points).
  14. 14. Future works • Improve plugin (slow response) • Add gestures
  15. 15. Screenshot
  16. 16. Screenshot
  17. 17. Screenshot
  18. 18. Screenshot
  20. 20. Q&A