Your SlideShare is downloading. ×
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones

389

Published on

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
389
On Slideshare
0
From Embeds
0
Number of Embeds
26
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. MobAppDev Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones Vladimir Kulyukin www.vkedco.blogspot.com
  • 2. Outline ● ● ● ● Touchscreens Motion Events Development & Consumption of Touch Gestures Touch Gesture Recognition in Mobile Robot Control
  • 3. Touchscreens
  • 4. Motivation ● ● ● ● Touch gestures are a viable input alternative, especially on the go Touchscreen – is a touch gesture consumption surface Recognition accuracy for many (not all) users are higher for touch gestures than for speech Unlike speech utterances, touch gestures preserve privacy
  • 5. Touchscreen ● ● ● ● ● Touchscreen is a surface (e.g., an electronic visual display) for detecting touches within a given area Touchscreen is made up of special materials that capture pressure inputs and translate them into digital data Digital data (x, y, pressure, etc) are passed to software that process them Some touchscreens require the use of a stylus Touchscreens are becoming a must for many personal digital assistants and many video games
  • 6. Bits of History: Air Traffic & Music ● ● ● ● ● The concept of touchscreen is described by E.A. Johnson in 1965 Potential application in air traffic control systems proposed in 1968 First application developed at CERN (European Center for Nuclear Research) in 1973 In the 1980's several musical sampling and synthesis systems used light pen technology (Fairlight CMI, Fairlight CMI IIx) HP-150 built in 1983 is one of the first touchscreen computers
  • 7. Touchscreen Technologies ● Resistive:   ● touchscreen consists of two thin electric layers separated by thin space voltage is applied to top layer and sensed by bottom one Surface acoustic wave (SAW):   when surface is touched a portion of the wave is absorbed  ● utrasonic waves pass over touchscreen surface changes in waves register touch positions and send them to touch processor Capacitive:  Capacitance is the ability of a body to store electrical charge  Capacitive touchscreens consist of insulators (e.g., glass) coated with transparent conductors  Since human hand stores electrical charge, touching surface results in surface's electrostatic field distortions used to capture touch's data
  • 8. Motion Events
  • 9. MotionEvent ● ● ● On Android, digital data from touchscreens are captured as MotionEvent objects MotionEvent objects are created when the user touches the device's touchscreen MotionEvent objects are handled by the View.onTouchEvent() method
  • 10. MotionEvent Sequences ● ● ● When user places his/her finger, moves it (without lifting it up), and then lifts it up, a sequence of MotionEvent objects is generated Such sequences can be captured and used in touch gesture recognition Each MotionEvent object contains information on: 1) type of action captured (e.g., MotionEvent.ACTION_DOWN, MotionEvent.ACTION_UP, etc); 2) pressure value; 3) x and y coordinates; 4) time of event
  • 11. MotionEvent ● ● ● ● On Android, digital data from touchscreens as MotionEvent objects MotionEvent objects are created when the user touches the device's touchscreen As the bare minimum each MotionEvent object contains the x and y coordinates of the touch MotionEvent objects are handled by the View.onTouchEvent() method
  • 12. MotionEvent.getAction() ● ● MotionEvent.getAction() can be used to retrieve the type of action Examples:  MotionEvent.getAction() returns ACTION_DOWN when user touches screen  MotionEvent.getAction() returns ACTION_MOVE when user moves sideways  MotionEvent.getAction() returns ACTION_UP when user lifts his/her finger
  • 13. Interface View.OnTouchListener ● ● ● Classes the receive touchscreen events must implement View.OnTouchListener Two main methods to handle MotionEvents are onTouch(View, MotionEvent) and onTouchEvent(MotionEvent) onTouch(View, MotionEvent) is used when one OnTouchListener handles MotionEvents from multiple views
  • 14. Interface View.OnTouchListener ● ● If onTouch() or onTouchEvent() consumes a received MotionEvent and no other component should not about it, the method should return true If onTouch() or onTouchEvent() does not/cannot consume a received MotionEvent, it should return false
  • 15. Development & Consumption of Touch Gestures
  • 16. Development of Touch Gestures ● ● ● ● You create your own gesture recognizer that captures and interprets a library of touch gestures You can also use the Android GestureBuilder application to develop a library of touch gestures The built gestures are saved on sdcard Each gesture receives a name and can be subsequently deleted or refined
  • 17. Problem Use the Android GestureBuilder application to develop gestures for handwritten letters a, b, and c. Write an application that consumes the built gestures and toasts the names of recognized gestures. Source code BuiltGesturesConsumer of is here
  • 18. Develop & Save Touch Gestures 1. Start GestureBuilder 2. Draw a Gesture 3. Save it
  • 19. Consumption of Touch Gestures ● To implement a touch (BuiltGesturesConsumer)  Create a directory /res/raw gesture consumer  Place the gestures file you build with the Android GestureBuilder into /res/raw  The main layout of the gesture consumer must be android.gesture.GestureOverlayView  The class that consumes gestures must implement OnGesturePerformedListener
  • 20. BuiltGesturesConsumer: Main Layout <?xml version="1.0" encoding="UTF-8"?> <android.gesture.GestureOverlayView xmlns:android="http://schemas.android.com/apk/res/android" android:id="@+id/gestures" android:layout_width="fill_parent" android:layout_height="fill_parent" android:gestureStrokeType="multiple" android:eventsInterceptionEnabled="true" android:orientation="vertical"> <ListView android:id="@android:id/list" android:layout_width="fill_parent" android:layout_height="fill_parent" /> </android.gesture.GestureOverlayView>
  • 21. BuiltGesturesConsumer: Gesture Library // Class must implement OnGesturePerformedListener public class BuiltGesturesConsumerAct extends Activity implements OnGesturePerformedListener { // Gestures are saved in GestureLibrary private GestureLibrary mLibrary; }
  • 22. BuiltGesturesConsumer: Gesture Library @Override public void onCreate(Bundle icicle) { super.onCreate(icicle); this.setContentView(R.layout.main); // Load the gesture library from /res/raw/gestures mLibrary = GestureLibraries.fromRawResource(this, R.raw.gestures); if (!mLibrary.load()) { finish(); } GestureOverlayView gestures = (GestureOverlayView) findViewById(R.id.gestures); // Make this class to be the gesture listener gestures.addOnGesturePerformedListener(this); }
  • 23. BuiltGesturesConsumer: Gesture Consumption @Override public void onGesturePerformed(GestureOverlayView overlay, Gesture gesture) { // 1. get the predictions ArrayList<Prediction> predictions = mLibrary.recognize(gesture); // 2. make sure that there is at least one prediction if (predictions.size() > 0) { // 3. get the best prediction Prediction prediction = (Prediction) predictions.get(0); // 4. make sure the confidence score is > 1.0 if (prediction.score > 1.0) { // Show the letter Toast.makeText(this, prediction.name, Toast.LENGTH_SHORT).show(); }}}
  • 24. Touch Gesture Control of Mobile Robots from Android Smartphones
  • 25. References ● en.wikipedia.org/wiki/Touchscreen ● http://developer.android.com/reference/android/view/MotionEvent.html ● http://developer.android.com/reference/android/view/View.OnTouchListener.html ● http://en.wikipedia.org/wiki/Chess ● http://vkedco.blogspot.com/2012/12/mobappdev-2d-frame-animations-on-android.html

×