• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content


Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Multitouching your apps



Multitouch on Linux, more specifically GTK+/GNOME

Multitouch on Linux, more specifically GTK+/GNOME



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Multitouching your apps Multitouching your apps Presentation Transcript

    • Multitouching your apps [email_address]
    • Multitouch
      • Touchscreen interaction with more than one appendage
      • Getting more common
    • And not only mobile devices...
    • Multitouch
      • Recognition of the shapes of pressure points
    • The good ol' times
      • Single mouse pointer
      • Single keyboard focus
      • Nothing new since the 80s
    • So what now?
      • Collaborative interfaces
        • Multiple users
        • Multiple pointer/keyboard pairs
      • Multitouch
        • Single user
        • Single keyboard
        • Multiple pointers
    • And any combination...
    • Device hierarchy
      • Master: Devices visible in the screen
      • Slave: Hardware devices
        • Attached: Control a Master device
        • Floating: Not attached to any Master device
    • Device hierarchy typedef enum { GDK_DEVICE_TYPE_MASTER, GDK_DEVICE_TYPE_SLAVE, GDK_DEVICE_TYPE_FLOATING } GdkDeviceType; GList * gdk_device_manager_list_devices (GdkDeviceManager *device_manager, GdkDeviceType type); GdkDevice * gdk_device_get_associated_device (GdkDevice *device); GdkDeviceType gdk_device_get_device_type (GdkDevice *device);
    • Device hotplugging
      • Signals in GdkDeviceManager
        • device-added
        • device-removed
        • device-changed
    • Event interaction
      • Events generated by input devices
        • Button press/release
        • Key press/release
        • Grab broken
      • These events will contain information about the device that generated them
    • GdkDevice* gdk_event_get_device (const GdkEvent *event); /* EXPERIMENTAL */ GdkDevice* gdk_event_get_source_device (const GdkEvent *event);
    • Enabling events
      • A GdkWindow only obeys Master devices by default
      • Slave devices need explicit events activation
      void gdk_window_set_device_events (GdkWindow *window, GdkDevice *device, GdkEventMask event_mask); /* EXPERIMENTAL */ void gdk_window_set_source_events (GdkWindow *window, GdkInputSource source, GdkEventMask event_mask);
    • Enter/Leave events
      • Emitted whenever a pointer enters or leaves a GdkWindow
      • For backwards compatibility, only a pair of events is emitted if several pointers are within the window area
        • First pointer entering -> GDK_ENTER_NOTIFY
        • Last pointer leaving -> GDK_LEAVE_NOTIFY
    • Enter/Leave events
      • Each pointer emits its own Enter/Leave events
      • Widgets need to take this into account if these functions are called
      void gdk_window_set_support_multidevice (GdkWindow *window, gboolean support_multidevice); void gtk_widget_set_support_multidevice (GtkWidget *widget, gboolean support_multidevice);
    • Where are thou, pointer?
      • The old calls are non deterministic
      • Basically, the position of a “favourite” pointer is obtained, the client pointer
    • void gdk_display_get_device_state (GdkDisplay *display, GdkDevice *device, GdkScreen **screen, gint *x, gint *y, GdkModifierType *mask); GdkWindow * gdk_display_get_window_at_device_position (GdkDisplay *display, GdkDevice *device, gint *win_x, gint *win_y); GdkWindow * gdk_window_get_device_position (GdkWindow *window, GdkDevice *device, gint *x, gint *y, GdkModifierType *mask);
    • Pointer and keyboard grabs
      • Capture events from the device, and confine them to a window
        • Menus
        • Drag and Drop
      • Blocks interaction with further applications
    • GdkGrabStatus gdk_device_grab (GdkDevice *device, GdkWindow *window, GdkGrabOwnership grab_ownership, gboolean owner_events, GdkEventMask event_mask, GdkCursor *cursor, guint32 time_); void gdk_device_ungrab (GdkDevice *device, guint32 time_);
    • Grab Ownership
        Defines how the device owns control over the grabbed window
        • None: Any other devices are free to interact
        • Window: The GdkWindow is effectively blocked for any other device
        • Application: The full application is blocked
    • Grab Ownership
      • Several devices may hold a grab at the same time, on the same or different windows
      • The device with most restrictive ownership blocks the others
    • GTK+ grabs
      • Similar to GDK grabs, but don't block interaction with other applications
        • Modal dialogs
    • void gtk_device_grab_add (GtkWidget *widget, GdkDevice *device, gboolean block_others); void gtk_device_grab_remove (GtkWidget *widget, GdkDevice *device); /* To be used in grab-notify handler */ gboolean gtk_widget_device_is_shadowed (GtkWidget *widget, GdkDevice *device);
    • Multitouch
      • Each finger gets a device
      • Devices emit individual events
      • There is a need to gather and correlate them
        • Obtain an event packet
        • Calculate the distance and relative angle between 2 events
        • GtkDeviceGroup and multidevice-event signal
    • /* EXPERIMENTAL */ GtkDeviceGroup * gtk_widget_create_device_group (GtkWidget *widget); void gtk_widget_remove_device_group (GtkWidget *widget, GtkDeviceGroup *group); void gtk_device_group_add_device (GtkDeviceGroup *group, GdkDevice *device); void gtk_device_group_remove_device (GtkDeviceGroup *group, GdkDevice *device); GList * gtk_device_group_get_devices (GtkDeviceGroup *group);
    • /* EXPERIMENTAL */ typedef enum { GTK_EVENT_DEVICE_ADDED, GTK_EVENT_DEVICE_REMOVED, GTK_EVENT_DEVICE_UPDATED } GtkMultiDeviceEventType; struct _GtkMultiDeviceEvent { GtkMultiDeviceEventType type; guint n_events; GdkEventMotion **events; GdkEventMotion *updated_event; GdkDevice *updated_device; };
    • gboolean gdk_events_get_distance (GdkEvent *event1, GdkEvent *event2, gdouble *distance); gboolean gdk_events_get_angle (GdkEvent *event1, GdkEvent *event2, gdouble *angle); gboolean gdk_events_get_center (GdkEvent *event1, GdkEvent *event2, gdouble *x, gdouble *y);
    • Gestures
      • Definition: “ The act of moving the limbs or body as an expression of though or emphasis”
        • Not really computable
      • Gestures are a set of finger movements over time, reduced to an expression with a meaning,
        • The logic to get an expression out of a gesture is highly heuristic
    • Gestures (2)
      • Gestures are something extending and complementing multi-touch, rather on top than part of it.
    • What's left?
      • Consolidating multidevice-event
      • Gesture support
      • Further multidevice support for stock widgets
    • Where's all this?
      • Now in GTK+ master
      • Methods marked as “experimental”:
      • git clone -b xi2-playground git://git.gnome.org/gtk+
    • Questions?
    • Thank you!