MobAppDev
Touchscreens, MotionEvents,
Development & Consumption of Touch
Gestures,
Touch Gesture Control of Mobile Robots ...
Outline
●
●
●
●

Touchscreens
Motion Events
Development & Consumption of Touch Gestures
Touch Gesture Recognition in Mobil...
Touchscreens
Motivation
●

●
●

●

Touch gestures are a viable input alternative,
especially on the go
Touchscreen – is a touch gesture...
Touchscreen
●

●

●

●
●

Touchscreen is a surface (e.g., an electronic visual display) for
detecting touches within a giv...
Bits of History: Air Traffic & Music
●

●

●

●

●

The concept of touchscreen is described by E.A. Johnson in
1965
Potent...
Touchscreen Technologies
●

Resistive:



●

touchscreen consists of two thin electric layers separated by thin space
vo...
Motion Events
MotionEvent
●

●

●

On Android, digital data from touchscreens are
captured as MotionEvent objects
MotionEvent objects ar...
MotionEvent Sequences
●

●

●

When user places his/her finger, moves it (without lifting it up),
and then lifts it up, a ...
MotionEvent
●

●

●

●

On Android, digital data from touchscreens as
MotionEvent objects
MotionEvent objects are created ...
MotionEvent.getAction()
●

●

MotionEvent.getAction() can be used to retrieve the
type of action
Examples:
 MotionEvent.g...
Interface View.OnTouchListener
●

●

●

Classes the receive touchscreen events must implement
View.OnTouchListener
Two mai...
Interface View.OnTouchListener
●

●

If onTouch() or onTouchEvent() consumes a received
MotionEvent and no other component...
Development & Consumption
of
Touch Gestures
Development of Touch Gestures
●

●

●
●

You create your own gesture recognizer that captures
and interprets a library of ...
Problem
Use the Android GestureBuilder application to develop
gestures for handwritten letters a, b, and c. Write an
appli...
Develop & Save Touch Gestures

1. Start GestureBuilder

2. Draw a Gesture

3. Save it
Consumption of Touch Gestures
●

To
implement
a
touch
(BuiltGesturesConsumer)
 Create a directory /res/raw

gesture

cons...
BuiltGesturesConsumer: Main Layout
<?xml version="1.0" encoding="UTF-8"?>
<android.gesture.GestureOverlayView
xmlns:androi...
BuiltGesturesConsumer: Gesture Library
// Class must implement OnGesturePerformedListener
public class BuiltGesturesConsum...
BuiltGesturesConsumer: Gesture Library
@Override
public void onCreate(Bundle icicle) {
super.onCreate(icicle);
this.setCon...
BuiltGesturesConsumer: Gesture Consumption
@Override
public void onGesturePerformed(GestureOverlayView overlay, Gesture ge...
Touch Gesture Control of Mobile
Robots from Android Smartphones
MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of M...
Upcoming SlideShare
Loading in...5
×

MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones

402

Published on

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
402
On Slideshare
0
From Embeds
0
Number of Embeds
26
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "MobAppDev (Fall 2013): Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones"

  1. 1. MobAppDev Touchscreens, MotionEvents, Development & Consumption of Touch Gestures, Touch Gesture Control of Mobile Robots from Android Smartphones Vladimir Kulyukin www.vkedco.blogspot.com
  2. 2. Outline ● ● ● ● Touchscreens Motion Events Development & Consumption of Touch Gestures Touch Gesture Recognition in Mobile Robot Control
  3. 3. Touchscreens
  4. 4. Motivation ● ● ● ● Touch gestures are a viable input alternative, especially on the go Touchscreen – is a touch gesture consumption surface Recognition accuracy for many (not all) users are higher for touch gestures than for speech Unlike speech utterances, touch gestures preserve privacy
  5. 5. Touchscreen ● ● ● ● ● Touchscreen is a surface (e.g., an electronic visual display) for detecting touches within a given area Touchscreen is made up of special materials that capture pressure inputs and translate them into digital data Digital data (x, y, pressure, etc) are passed to software that process them Some touchscreens require the use of a stylus Touchscreens are becoming a must for many personal digital assistants and many video games
  6. 6. Bits of History: Air Traffic & Music ● ● ● ● ● The concept of touchscreen is described by E.A. Johnson in 1965 Potential application in air traffic control systems proposed in 1968 First application developed at CERN (European Center for Nuclear Research) in 1973 In the 1980's several musical sampling and synthesis systems used light pen technology (Fairlight CMI, Fairlight CMI IIx) HP-150 built in 1983 is one of the first touchscreen computers
  7. 7. Touchscreen Technologies ● Resistive:   ● touchscreen consists of two thin electric layers separated by thin space voltage is applied to top layer and sensed by bottom one Surface acoustic wave (SAW):   when surface is touched a portion of the wave is absorbed  ● utrasonic waves pass over touchscreen surface changes in waves register touch positions and send them to touch processor Capacitive:  Capacitance is the ability of a body to store electrical charge  Capacitive touchscreens consist of insulators (e.g., glass) coated with transparent conductors  Since human hand stores electrical charge, touching surface results in surface's electrostatic field distortions used to capture touch's data
  8. 8. Motion Events
  9. 9. MotionEvent ● ● ● On Android, digital data from touchscreens are captured as MotionEvent objects MotionEvent objects are created when the user touches the device's touchscreen MotionEvent objects are handled by the View.onTouchEvent() method
  10. 10. MotionEvent Sequences ● ● ● When user places his/her finger, moves it (without lifting it up), and then lifts it up, a sequence of MotionEvent objects is generated Such sequences can be captured and used in touch gesture recognition Each MotionEvent object contains information on: 1) type of action captured (e.g., MotionEvent.ACTION_DOWN, MotionEvent.ACTION_UP, etc); 2) pressure value; 3) x and y coordinates; 4) time of event
  11. 11. MotionEvent ● ● ● ● On Android, digital data from touchscreens as MotionEvent objects MotionEvent objects are created when the user touches the device's touchscreen As the bare minimum each MotionEvent object contains the x and y coordinates of the touch MotionEvent objects are handled by the View.onTouchEvent() method
  12. 12. MotionEvent.getAction() ● ● MotionEvent.getAction() can be used to retrieve the type of action Examples:  MotionEvent.getAction() returns ACTION_DOWN when user touches screen  MotionEvent.getAction() returns ACTION_MOVE when user moves sideways  MotionEvent.getAction() returns ACTION_UP when user lifts his/her finger
  13. 13. Interface View.OnTouchListener ● ● ● Classes the receive touchscreen events must implement View.OnTouchListener Two main methods to handle MotionEvents are onTouch(View, MotionEvent) and onTouchEvent(MotionEvent) onTouch(View, MotionEvent) is used when one OnTouchListener handles MotionEvents from multiple views
  14. 14. Interface View.OnTouchListener ● ● If onTouch() or onTouchEvent() consumes a received MotionEvent and no other component should not about it, the method should return true If onTouch() or onTouchEvent() does not/cannot consume a received MotionEvent, it should return false
  15. 15. Development & Consumption of Touch Gestures
  16. 16. Development of Touch Gestures ● ● ● ● You create your own gesture recognizer that captures and interprets a library of touch gestures You can also use the Android GestureBuilder application to develop a library of touch gestures The built gestures are saved on sdcard Each gesture receives a name and can be subsequently deleted or refined
  17. 17. Problem Use the Android GestureBuilder application to develop gestures for handwritten letters a, b, and c. Write an application that consumes the built gestures and toasts the names of recognized gestures. Source code BuiltGesturesConsumer of is here
  18. 18. Develop & Save Touch Gestures 1. Start GestureBuilder 2. Draw a Gesture 3. Save it
  19. 19. Consumption of Touch Gestures ● To implement a touch (BuiltGesturesConsumer)  Create a directory /res/raw gesture consumer  Place the gestures file you build with the Android GestureBuilder into /res/raw  The main layout of the gesture consumer must be android.gesture.GestureOverlayView  The class that consumes gestures must implement OnGesturePerformedListener
  20. 20. BuiltGesturesConsumer: Main Layout <?xml version="1.0" encoding="UTF-8"?> <android.gesture.GestureOverlayView xmlns:android="http://schemas.android.com/apk/res/android" android:id="@+id/gestures" android:layout_width="fill_parent" android:layout_height="fill_parent" android:gestureStrokeType="multiple" android:eventsInterceptionEnabled="true" android:orientation="vertical"> <ListView android:id="@android:id/list" android:layout_width="fill_parent" android:layout_height="fill_parent" /> </android.gesture.GestureOverlayView>
  21. 21. BuiltGesturesConsumer: Gesture Library // Class must implement OnGesturePerformedListener public class BuiltGesturesConsumerAct extends Activity implements OnGesturePerformedListener { // Gestures are saved in GestureLibrary private GestureLibrary mLibrary; }
  22. 22. BuiltGesturesConsumer: Gesture Library @Override public void onCreate(Bundle icicle) { super.onCreate(icicle); this.setContentView(R.layout.main); // Load the gesture library from /res/raw/gestures mLibrary = GestureLibraries.fromRawResource(this, R.raw.gestures); if (!mLibrary.load()) { finish(); } GestureOverlayView gestures = (GestureOverlayView) findViewById(R.id.gestures); // Make this class to be the gesture listener gestures.addOnGesturePerformedListener(this); }
  23. 23. BuiltGesturesConsumer: Gesture Consumption @Override public void onGesturePerformed(GestureOverlayView overlay, Gesture gesture) { // 1. get the predictions ArrayList<Prediction> predictions = mLibrary.recognize(gesture); // 2. make sure that there is at least one prediction if (predictions.size() > 0) { // 3. get the best prediction Prediction prediction = (Prediction) predictions.get(0); // 4. make sure the confidence score is > 1.0 if (prediction.score > 1.0) { // Show the letter Toast.makeText(this, prediction.name, Toast.LENGTH_SHORT).show(); }}}
  24. 24. Touch Gesture Control of Mobile Robots from Android Smartphones
  25. 25. References ● en.wikipedia.org/wiki/Touchscreen ● http://developer.android.com/reference/android/view/MotionEvent.html ● http://developer.android.com/reference/android/view/View.OnTouchListener.html ● http://en.wikipedia.org/wiki/Chess ● http://vkedco.blogspot.com/2012/12/mobappdev-2d-frame-animations-on-android.html

×