Multimodal Interfaces

1,494 views
1,300 views

Published on

Ignite Columbus presentation by Thomas Winningham
11/13/08
http://IgniteCbus.com

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,494
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
34
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Multimodal Interfaces

  1. Multimodal Interfaces <ul><li>November, 2008 </li></ul><ul><li>Thomas Winningham </li></ul>
  2. Multi ah-wha? <ul><li>Computer vision, mostly </li></ul><ul><li>The Wiimote 100hz camera </li></ul><ul><li>Capacitive surfaces (or lots of buttons)‏ </li></ul><ul><ul><li>IPhone & Mac touchpad </li></ul></ul><ul><ul><li>HP TouchSmart </li></ul></ul><ul><li>Tin Foil & an Arduino </li></ul><ul><li>Force transducers, the Wii Fit </li></ul><ul><li>Teaching new dogs old tricks </li></ul>
  3. Regions of interest <ul><li>At 30 fps or faster: </li></ul><ul><ul><li>Find a finger smudge (blobs)‏ </li></ul></ul><ul><ul><li>Dots of light </li></ul></ul><ul><ul><li>Triangulate based on closest data </li></ul></ul><ul><ul><li>Color keying </li></ul></ul><ul><ul><li>LEDs as two-way </li></ul></ul><ul><ul><li>Use IR for spectrum separation </li></ul></ul><ul><ul><li>Isolate features with laser </li></ul></ul><ul><ul><li>Map in 3D using Lidar </li></ul></ul>
  4. Better Software & Hardware <ul><li>DARPA Challenge </li></ul><ul><ul><li>Intel OpenCV </li></ul></ul><ul><li>Multiprocessor Machines </li></ul><ul><li>Home clusters of machines </li></ul><ul><li>GPU processors on videocards </li></ul><ul><li>Better web cams </li></ul><ul><li>Cheaper, more widely available materials </li></ul><ul><li>Easier access to information </li></ul>
  5. Fresh Ways to Interface with Computers <ul><li>It is not a panacea, but at least everyone is mostly invited. </li></ul>
  6. Leave it to the big boys? <ul><li>IPhone's </li></ul><ul><ul><li>And knock-offs </li></ul></ul><ul><li>Microsoft Surface & Windows 7 & Tablets </li></ul><ul><ul><li>Microsoft Research </li></ul></ul><ul><li>HP TouchSmart </li></ul><ul><li>Perceptive Pixel </li></ul><ul><li>NUI Group </li></ul><ul><ul><li>step 1. cut a hole in a box </li></ul></ul>
  7. Gesturing <ul><li>Combos! </li></ul><ul><li>Changes over time </li></ul><ul><li>Inferring about the placement of input in orientation to the real world </li></ul><ul><ul><li>Surface </li></ul></ul><ul><ul><li>Wii fit </li></ul></ul><ul><ul><li>You're holding a nail near tinfoil </li></ul></ul>
  8. Simple! <ul><li>Or at least, reasonably simple given the complexity of what is going on. </li></ul><ul><li>Small blocks of code in repition </li></ul><ul><li>Again, faster computers </li></ul>
  9. Web Cam in a Box <ul><li>Step #2 </li></ul><ul><ul><li>Put your junk in that box </li></ul></ul>
  10. Double true! <ul><li>Stackable cubes </li></ul><ul><li>Large walls </li></ul><ul><li>Small hobby projects </li></ul><ul><li>Multiple cameras </li></ul><ul><li>GPUs </li></ul><ul><li>Networked machines </li></ul><ul><li>Store Windows, floors , buildings </li></ul>
  11. Mostly flash <ul><li>(needs a lot of help from c)‏ </li></ul><ul><li>But also: </li></ul><ul><ul><li>Python </li></ul></ul><ul><ul><li>Processing </li></ul></ul><ul><ul><li>Max </li></ul></ul><ul><ul><li>VVVV </li></ul></ul><ul><ul><li>Pure Data </li></ul></ul><ul><ul><li>.NET </li></ul></ul><ul><ul><li>Firefox / Javascript (Lily)‏ </li></ul></ul>
  12. Rethinking a lot of things <ul><li>Gesture navigation </li></ul><ul><li>User Interface </li></ul><ul><li>Feedback </li></ul>
  13. Guerrilla Arm
  14. All kinds of things I tell ya! <ul><li>Sphere </li></ul><ul><li>Gesturing with your hands above the home row </li></ul><ul><li>Or just using your hands </li></ul><ul><li>Or moving your head, or eyes, or thoughts </li></ul><ul><li>This is an extra bullet point. Ha! </li></ul>
  15. Meanwhile, back at the bat cave... <ul><li>Home clustering </li></ul><ul><li>Semistructured web and public data </li></ul><ul><li>HDTV's, projectors, err... iphones </li></ul><ul><li>Social connectivity </li></ul><ul><ul><li>Except in the workplace where we'd generally like everyone to politely sod off </li></ul></ul><ul><li>Advanced modeling, free software for fluid dynamics and physics simulation </li></ul>
  16. Lack of standards? <ul><li>lack of patent free ideas, maybe </li></ul><ul><li>TUIO / OSC for ROI or Fiducials </li></ul>
  17. Romper room <ul><li>I see: </li></ul><ul><ul><li>NUI Group </li></ul></ul><ul><ul><li>Open Frameworks </li></ul></ul><ul><ul><li>Processing </li></ul></ul><ul><ul><li>Apple Developer forums </li></ul></ul><ul><ul><li>IPhone jailbreakers </li></ul></ul><ul><ul><li>Multi-pointer X-Windows, Compiz, and the whole open-source world. </li></ul></ul>
  18. Feedback loop <ul><li>Proliferation? Adoption? </li></ul><ul><li>Social implications </li></ul><ul><li>This will all spawn something else </li></ul><ul><li>Everybody's coming to the net </li></ul><ul><li>Voluntary mass hallucination (Gibson)‏ </li></ul>
  19. Come out and play! <ul><li>If yer under eighteen ya won't be doin any ty-ee-eye-ee-ime </li></ul>
  20. There's a place for us! <ul><li>More people playing! </li></ul><ul><ul><li>== better software </li></ul></ul><ul><li>Not as hard as it sounds, just steal code ;p </li></ul><ul><li>Who's getting laid anyway? </li></ul><ul><li>It is amazing what you can get done if you don't care who gets the credit. (jets ftw!)‏ </li></ul>

×