Per c for unity devs
Upcoming SlideShare
Loading in...5

Per c for unity devs






Total Views
Views on SlideShare
Embed Views



1 Embed 44 44



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Per c for unity devs Per c for unity devs Presentation Transcript

  • Intel Confidential1Perceptual Computing– Pour l’instant destiné aux développeurs– Camera Creative Senz3D disponible au prochaintrimestre– Intégration dans des machines Intel prévuepour le second semestre 2014 !• Perceptual Computing SDK– C++– Passerelles C#, Unity, Processing, openframeworks…
  • Intel Confidential2Speech Recognition:Voice command and control, short sentencedictation, and text to speech synthesisSDK Usage Modes Today11 New usage modes may be added in the futureClose-range Depth Tracking (6 in. to 3 ft.):Recognize the positions of each of the user’shands, fingers, static hand poses and moving handgestures.Facial Analysis:Face detection and recognition (six and seven point landmark andattribution detection, including smiles, blinks, and age groups)Augmented Reality:Combine real-time images from the camera and close-rangetracking from the depth sensor with 2D or 3D graphical images.
  • Intel Confidential3Close-Range Finger Tracking• Geometric Node Tracking– 7-point tracking: fingertips, palm center and elbow.– Special usability points: fingertip and grabbingpoints.– Estimate positions, volumes, openness andhandedness.• Standardized Poses/Gestures– Poses: thumb up/down and peace– Gestures: swipe left/right/up/down, circle, andwave• Label map– Hand image and its parameters.
  • Intel Confidential4Face Tracking and Analysis• Multiple face detection/tracking• Landmark detection– 6/7-point detection including eyes, nose, andmouth• Facial attribute detection– Age-group including baby/youth/adult/senior– Gender detection– Smile/blink detection• Face recognition– Similarity among a set of faces
  • Intel Confidential5• Nuance* Voice Command and Control– Recognize within a list of predefinedcommands.• Nuance Voice Dictation– Recognize short sentences (<30 seconds).• Nuance Voice Synthesis– Text to speech for short sentencesVoice Recognition and Synthesis
  • Intel Confidential6• Track any 2D planer surfaces– Report position, orientation andother parameters• Track limited 3D objects– Based on 3D models• Track faces orientationAugmented Reality
  • Intel Confidential7SDK Usage H/W RequirementsSDK UsageModeSpeechCertified Dual-ArrayMicrophonesRGBWebcamCreative*CameraClose-rangeDepthtrackingXSpeechRecognitionX XFacialTrackingX XAugmentedRealityX X
  • Intel Confidential8PXCSession, PXCImage, PXCAudio, PXCCapture, PXCGesture, PXCFaceAnalysis, PXCVoiceUtilCapture, UtilPipelineC#PXCMSessionPXCMImagePXCMAudioPXCMCapturePXCMGesturePXCMFaceAnalysisPXCMVoiceUtilMCaptureUtilMPipelinepxcupipelineUnity*ProProcessingopenFrameworks*ApplicationsCore FunctionalitiesModule InteractionAdditional Language andFramework SupportSDK API Hierarchy*Other brands and trademarks may be claimed as the property of their respective owners
  • Intel Confidential9• Color streams RGB24 640x480 25fps, 30fps RGB24 640x360 25fps, 30fps RGB24 1280x720 25fps, 30fps• Depth streams (16-bit integer, 0-32000) 320x240 25fps, 30fps, 50fps, 60fps UVMAP (Depth  Color) Confidence Map (16-bit integer)• Vertices streams (real world coordinates in 3Dfixed-point integers)• Audio streams (At least 2-array MIC) 44.1KHz mono/stereo 48KHz mono/stereoCamera Streams
  • Intel Confidential10PXCImage: Depth & ColorIMAGE_TYPE_COLOR(of any color format)• planes[0]: colordata• Planes[1]: empty• planes[2]: empty• planes[3]: emptyIMAGE_FORMAT_DEPTH•planes[0]: depthmap•Planes[1]: confidencemap•planes[2]: uvmap•planes[3]: empty2-byte perpixel8-byte perpixelIMAGE_FORMAT_VERTICES•planes[0]: vertex (x,y,z)•Planes[1]: confidencemap•planes[2]: uvmap•planes[3]: empty12-byteper pixelOne real depth stream, two virtual
  • Intel Confidential11Programming Language andFramework Support• C++, C#, Java• Supported Frameworks– processing– openFrameworks– Unity– Havok– Total Immersion AR
  • Intel Confidential12Declare The SDK Object• Unityprivate PXCUPipeline pipeline;pipeline = new PXCUPipeline();
  • Intel Confidential13Select Features and Initialize• Select Features using PXCUPipeline.Modeenum• Use Bitwise OR (|) for Multiple Features• Use Init() To Set Features and Enable SDKAccesspipeline.Init(PXCUPipeline.Mode.COLOR_VGA|PXCUPipeline.Mode.DEPTH_QVGA|PXCUPipeline.Mode.GESTURE);
  • Intel Confidential14Capture A Frame• Poll For A Frame Using AcquireFrame(bool);– Can be blocking or non-blocking– AcquireFrame(true) is blocking,AcquireFrame(false) is non-blocking• Returns true If A Frame Is Availableif(pipeline.AcquireFrame(false)){}
  • Intel Confidential15Retrieve Image Data• Data Is Retrieved via Query*(<T>)– QueryRGB(), QueryLabelMapAsImage(), etc…• UnityTexture2D rgbTexture = newTexture2D(640,480,TextureFormat.ARGB32, false);pipeline.QueryRGB(rgbTexture);
  • Intel Confidential16Retrieve Geonodes Data• Data Is Retrieved via QueryGeoNode()– Use PXCGesture::GeoNode::Label andPXCGesture::GeoNode to retrieve data• UnityPXCMGesture.GeoNode node;pipeline.QueryGeoNode(PXCMGesture.GeoNode.LABEL_BODY_HAND_PRIMARY, out node);
  • Intel Confidential17Retrieve Gesture Data• Data Is Retrieved via QueryGesture()– UsePXCGesture::GeoNode::Label, PXCGesture::Gesture::Label and PXCGesture::Gesture to retrievedata• UnityPXCMGesture.Gesture gest;pipeline.QueryGesture(PXCMGesture.GeoNode.LABEL_BODY_HAND_PRIMARY, out gest);if(gest.label==PXCMGesture.Gesture.Label.LABEL_HAND_WAVE)…
  • Intel Confidential18Clean Up• Use ReleaseFrame() To “Free Up ThePipeline”pipeline.ReleaseFrame();
  • Intel Confidential19What next ?