Kinect and Why You're A Prick - BarCamp Bournemouth 5
•Download as PPTX, PDF•
0 likes•505 views
Here's a talk I did at BarCamp Bournemouth 5. It's all about Kinect development and trying to come up with a design that pleases everyone, but by pleasing nobody in the process.
More Related Content
Viewers also liked
Recently uploaded
Recently uploaded (20)
Kinect and Why You're A Prick - BarCamp Bournemouth 5
- 1. KINECT AND WHY YOU‟RE A PRICK
- 2. MATT CROUCH
- 3. INFRARED RGB CAMERA PROJECTOR / SENSOR MOTORISED TILT MICROPHONE ARRAY
- 4. DEPTH STREAM RGB CAMERA SKELETON AND SENSOR DATA INFRARED PLAYER FACE TRACKING
- 11. BUT WHAT‟S A SWIPE?
- 12. NO STANDARDS
- 13. TAKE THEM BY THE HAND
- 14. MICROSOFT‟S WAY
- 17. VOICE NAVIGATION
- 18. “IF YOU SEE IT, SAY IT”
- 19. “SEARCH THE “FIND ME THE FIFA “GET THE MARKETPLACE FIFA 13 DEMO” 13 TRIAL” FOR THE FIFA 13 DEMO”
- 20. GET FIND FIFA PERFORM FIFA 13 SEARCH FIFA 13 A SEARCH LATEST LOCATE FETCH FIFA TRIAL TRYOUT DEMO DEMO “GET THE FIFA 13 DEMO”
- 21. CONCLUSION @STUPLER MATTCROUCH.NET
Editor's Notes
- Introduction to presentationAll about my experiences with Kinect, and why everybody not being the same can cause some rather big headaches.
- Matt Crouch is my name!Currently a web developer in PooleRecent graduate of Interactive Media Production course at Bournemouth UniFinal Major Project was Kinect project called MotoYear and a half ago, didn’t really know what the Kinect was about.
- Quick rundown of basics of the KinectRGB Camera – Like a normal webcamInfrared projector/sensor – Provides depth information. Sends out IR speckles in a 9x9 grid of intensity and crunches numbers based on the intensity of the light back and the results from a preprogrammed set depth to determine the distance each speckle is from the deviceMicrophone array – Four individual microphones can pick up, through intensity in each, where a sound is coming from in relation to the device. Allows player-specific voice commands and cancellation of background noise (also noise coming from program running on computer)Motorised tilt – Track tall and small people
- Demonstration of what the Kinect seesRGB Camera – Like a normal cameraIR Sensor – What it’s receiving. Note more intense dots to aid calibration of 9x9 gridDepth View – Kinect’s number crunching. Automatic designation of player data (and colouring)Face Tracking – Close views can pinpoint facial expressions and respond to happy and sad, for example.
- Moto – Kinect Music ExperienceAllow two people to play guitar, keyboard, drums or the triangle together. Similar concept to Guitar Hero (but with no end goal)Wall of Sound – Play samples with a 3D sample board in front of you from set soundboards, or record your own.http://www.mattcrouch.net/moto
- Ethno Tekh – Company specialising in installations and performances with body motion.Generating music on the fly using Kinect motion trackingOutputs skeleton data to Ableton Live, to control various tracks and sliders to generate music.Output data to Unity to create visuals for a performance.http://ethnotekh.com/
- Alex’s Place – Cancer and blood disorder treatment centre for children.Reduce fear of children, introduce them to technology and put them at ease.Added bonus of being able to observe a child’s motor and social skills externally without requiring any unnatural tests to be done.http://123kinect.com/alexs-place-cancer-clinic-uses-kinect-to-help-children/39349/
- Nipple Tracker, using Open Frameworks (Before the times of the official SDK)Silly app for tracking nipples on the torso which, when found, plays Hot Stuff on loop with a random selection of novelty nipple covers.Knows where the torso is, and guesses where it sees a heightened peak in between the shoulders that these are nipples, and attempts to cover them uphttp://vimeo.com/17095170
- The possibilities of KinectComputer system (Web development) – Stuck to the browser. Generally rely on keyboard and mouse/touch input. Hardware usability is already done, with conventions made. As long as a button gets pressed, that’s all you care about.Kinect – No physical controller to aid. No handle on what could be done, or how. Leads to problems with people unfamiliar with the system.
- Moto trailer, made from a playtest. Unseen fails.People not used to control mechanisms, or doing things you didn’t anticipate. Leads to a lot of problems.
- Ash had a problem with swiping using that menu.Swipe left to right would select the current item in the menu, but movement up or down would move the menu while you swiped. How do you detect a swipe before their arm moves down?What’s a swipe, anyway? Which way? How much? How quickly?
- Neilsen’s problem areas of Kinect – Where the problems lie.http://www.nngroup.com/articles/kinect-gestural-ui-first-impressions/Most of the time we’re relying on a user’s learned behavior. With controllers, for example Xbox, A is an ‘Accept’ button, while B is a ‘Back’ button. With Kinect, we’ve got people with different ideas on how to navigate, and it’s getting people confused.
- You literally take them by the handWe’re now in the realm of the gesture-based UI. Start afresh, and start from the basics.Dance Central was a Kinect Launch title, so had little idea what competitors were doing, and Harmonix weren’t able to conform to a standard, just do internal research as to what people found easy to use.Research was performed with people with different experiences, with more or less people in the room and saw how it reacted. Some playtests were performed drunk to see how it fared in a party setting.
- Microsoft’s way, the hover timer. Trying to emulate a ‘click’. Developing more of a standard, but a personal hatred of mine.Hold your hand in a certain position for a couple of seconds to lock into a particular selection.Xbox Dashboard uses it, along with a growing list of other games including Kinect Sports and Kinect Adventures.Purposeful delaying of navigation is never a good idea? What about power users?When are you selecting something? Is your arm by its side selecting something towards the bottom of the screen?When talking to someone, you might be selecting something if you’re not paying attention
- People (at this point) aren’t used to controlling with the Kinect. People were trying to mimic what they already knew from a computer/controller setting by attempting to ‘push’ buttons.Factoring in lag, sometimes people aren’t sure whether what they did has had the right effect on screen, leading to slower navigation through menus and less confident gameplayWe need a solution which doesn’t rely on assumed knowledge, that utilises a positive gesture of some description to allow movement through menus without accidental selection.
- Let the user know what they’re doing is actually doing something. Kinect Adventures has the players avatar (or a scanned guesstimation of the player, if a guest) mimic their movements in the background, to give them an idea what their actions could be doing.Dance Central gives an visual and audio cue when an item is being, or about to be, selected. Users are to continue the swipe until that sound completes.
- But gesture interaction isn’t the only way Kinect’s changed the way we navigate, we’ve also got a very good array microphone to tap into.But people aren’t going to be playing in silence. Kinect’s a social platform, so people are going to be talking, or play it at parties with music blaring loudly in the background.Thankfully Kinect can detect and ignore background noise, as well as cancel out any sounds coming from the computer it’s running on (or in-game on the Xbox)The Speech API will return a confidence based on what it thinks was said, so you can only act if the detection was good enough, avoiding false positives.
- Microsoft developed a pseudo-standard when Kinect launched, which invited players to use voice navigation – “If you see it, say it”.It’s found when navigating the dashboard. The microphone icon serves as a reminder. Say “Xbox”, then what you see. And it will do it.All about making it as easy as possible.
- But people don’t speak like robots. Depending on your situation, people could give voice commands in a natural language sense. “Search the Marketplace for the FIFA 13 Demo”“Get the FIFA 13 Demo”“Find me the FIFA 13 Trial”This is going to give the better user experience. Give them the visual prompts of what they *could* say, or what subject areas the project will respond to should you say something about it.There’s so many differing combinations, it’s going to be impossible for you to catch them all…
- Thankfully, the Microsoft Speech API (now bundled with the SDK downloaded) can help you build grammars to combat this.As long as you know the structures of sentences people are going to say, you can set predefined lists for different parts of a sentence, attach a meaning to them, and act accordingly when there’s a match
- We want to give the user the best interaction experience possible. This is only going to be possible if we keep testing new ideas out on a wide range of people. But, until we find a norm, we need to find what people can consider an obvious way to use the interfaces we provide to get what they want as quickly as they want, even if it means pandering to a few old conventions in the mean time. People are quite stuck in their ways, but I hope by opening your eyes to how complex some of these things can be, you can be less of a prick and start to learn new ways of interacting with a new piece of tech we’ve not yet experienced properly yet