Intel Confidential1Perceptual Computing– Pour l’instant destiné aux développeurs– Camera Creative Senz3D disponible au pro...
Intel Confidential2Speech Recognition:Voice command and control, short sentencedictation, and text to speech synthesisSDK ...
Intel Confidential3Close-Range Finger Tracking• Geometric Node Tracking– 7-point tracking: fingertips, palm center and elb...
Intel Confidential4Face Tracking and Analysis• Multiple face detection/tracking• Landmark detection– 6/7-point detection i...
Intel Confidential5• Nuance* Voice Command and Control– Recognize within a list of predefinedcommands.• Nuance Voice Dicta...
Intel Confidential6• Track any 2D planer surfaces– Report position, orientation andother parameters• Track limited 3D obje...
Intel Confidential7SDK Usage H/W RequirementsSDK UsageModeSpeechCertified Dual-ArrayMicrophonesRGBWebcamCreative*CameraClo...
Intel Confidential8PXCSession, PXCImage, PXCAudio, PXCCapture, PXCGesture, PXCFaceAnalysis, PXCVoiceUtilCapture, UtilPipel...
Intel Confidential9• Color streams RGB24 640x480 25fps, 30fps RGB24 640x360 25fps, 30fps RGB24 1280x720 25fps, 30fps• D...
Intel Confidential10PXCImage: Depth & ColorIMAGE_TYPE_COLOR(of any color format)• planes[0]: colordata• Planes[1]: empty• ...
Intel Confidential11Programming Language andFramework Support• C++, C#, Java• Supported Frameworks– processing– openFramew...
Intel Confidential12Declare The SDK Object• Unityprivate PXCUPipeline pipeline;pipeline = new PXCUPipeline();
Intel Confidential13Select Features and Initialize• Select Features using PXCUPipeline.Modeenum• Use Bitwise OR (|) for Mu...
Intel Confidential14Capture A Frame• Poll For A Frame Using AcquireFrame(bool);– Can be blocking or non-blocking– AcquireF...
Intel Confidential15Retrieve Image Data• Data Is Retrieved via Query*(<T>)– QueryRGB(), QueryLabelMapAsImage(), etc…• Unit...
Intel Confidential16Retrieve Geonodes Data• Data Is Retrieved via QueryGeoNode()– Use PXCGesture::GeoNode::Label andPXCGes...
Intel Confidential17Retrieve Gesture Data• Data Is Retrieved via QueryGesture()– UsePXCGesture::GeoNode::Label, PXCGesture...
Intel Confidential18Clean Up• Use ReleaseFrame() To “Free Up ThePipeline”pipeline.ReleaseFrame();
Intel Confidential19What next ?http://perchackparis.eventbrite.fr/https://perceptualchallenge.intel.com/
Upcoming SlideShare
Loading in …5
×

Per c for unity devs

529 views

Published on

Published in: Technology, Spiritual
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
529
On SlideShare
0
From Embeds
0
Number of Embeds
49
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Per c for unity devs

  1. 1. Intel Confidential1Perceptual Computing– Pour l’instant destiné aux développeurs– Camera Creative Senz3D disponible au prochaintrimestre– Intégration dans des machines Intel prévuepour le second semestre 2014 !• Perceptual Computing SDK– C++– Passerelles C#, Unity, Processing, openframeworks…
  2. 2. Intel Confidential2Speech Recognition:Voice command and control, short sentencedictation, and text to speech synthesisSDK Usage Modes Today11 New usage modes may be added in the futureClose-range Depth Tracking (6 in. to 3 ft.):Recognize the positions of each of the user’shands, fingers, static hand poses and moving handgestures.Facial Analysis:Face detection and recognition (six and seven point landmark andattribution detection, including smiles, blinks, and age groups)Augmented Reality:Combine real-time images from the camera and close-rangetracking from the depth sensor with 2D or 3D graphical images.
  3. 3. Intel Confidential3Close-Range Finger Tracking• Geometric Node Tracking– 7-point tracking: fingertips, palm center and elbow.– Special usability points: fingertip and grabbingpoints.– Estimate positions, volumes, openness andhandedness.• Standardized Poses/Gestures– Poses: thumb up/down and peace– Gestures: swipe left/right/up/down, circle, andwave• Label map– Hand image and its parameters.
  4. 4. Intel Confidential4Face Tracking and Analysis• Multiple face detection/tracking• Landmark detection– 6/7-point detection including eyes, nose, andmouth• Facial attribute detection– Age-group including baby/youth/adult/senior– Gender detection– Smile/blink detection• Face recognition– Similarity among a set of faces
  5. 5. Intel Confidential5• Nuance* Voice Command and Control– Recognize within a list of predefinedcommands.• Nuance Voice Dictation– Recognize short sentences (<30 seconds).• Nuance Voice Synthesis– Text to speech for short sentencesVoice Recognition and Synthesis
  6. 6. Intel Confidential6• Track any 2D planer surfaces– Report position, orientation andother parameters• Track limited 3D objects– Based on 3D models• Track faces orientationAugmented Reality
  7. 7. Intel Confidential7SDK Usage H/W RequirementsSDK UsageModeSpeechCertified Dual-ArrayMicrophonesRGBWebcamCreative*CameraClose-rangeDepthtrackingXSpeechRecognitionX XFacialTrackingX XAugmentedRealityX X
  8. 8. Intel Confidential8PXCSession, PXCImage, PXCAudio, PXCCapture, PXCGesture, PXCFaceAnalysis, PXCVoiceUtilCapture, UtilPipelineC#PXCMSessionPXCMImagePXCMAudioPXCMCapturePXCMGesturePXCMFaceAnalysisPXCMVoiceUtilMCaptureUtilMPipelinepxcupipelineUnity*ProProcessingopenFrameworks*ApplicationsCore FunctionalitiesModule InteractionAdditional Language andFramework SupportSDK API Hierarchy*Other brands and trademarks may be claimed as the property of their respective owners
  9. 9. Intel Confidential9• Color streams RGB24 640x480 25fps, 30fps RGB24 640x360 25fps, 30fps RGB24 1280x720 25fps, 30fps• Depth streams (16-bit integer, 0-32000) 320x240 25fps, 30fps, 50fps, 60fps UVMAP (Depth  Color) Confidence Map (16-bit integer)• Vertices streams (real world coordinates in 3Dfixed-point integers)• Audio streams (At least 2-array MIC) 44.1KHz mono/stereo 48KHz mono/stereoCamera Streams
  10. 10. Intel Confidential10PXCImage: Depth & ColorIMAGE_TYPE_COLOR(of any color format)• planes[0]: colordata• Planes[1]: empty• planes[2]: empty• planes[3]: emptyIMAGE_FORMAT_DEPTH•planes[0]: depthmap•Planes[1]: confidencemap•planes[2]: uvmap•planes[3]: empty2-byte perpixel8-byte perpixelIMAGE_FORMAT_VERTICES•planes[0]: vertex (x,y,z)•Planes[1]: confidencemap•planes[2]: uvmap•planes[3]: empty12-byteper pixelOne real depth stream, two virtual
  11. 11. Intel Confidential11Programming Language andFramework Support• C++, C#, Java• Supported Frameworks– processing– openFrameworks– Unity– Havok– Total Immersion AR
  12. 12. Intel Confidential12Declare The SDK Object• Unityprivate PXCUPipeline pipeline;pipeline = new PXCUPipeline();
  13. 13. Intel Confidential13Select Features and Initialize• Select Features using PXCUPipeline.Modeenum• Use Bitwise OR (|) for Multiple Features• Use Init() To Set Features and Enable SDKAccesspipeline.Init(PXCUPipeline.Mode.COLOR_VGA|PXCUPipeline.Mode.DEPTH_QVGA|PXCUPipeline.Mode.GESTURE);
  14. 14. Intel Confidential14Capture A Frame• Poll For A Frame Using AcquireFrame(bool);– Can be blocking or non-blocking– AcquireFrame(true) is blocking,AcquireFrame(false) is non-blocking• Returns true If A Frame Is Availableif(pipeline.AcquireFrame(false)){}
  15. 15. Intel Confidential15Retrieve Image Data• Data Is Retrieved via Query*(<T>)– QueryRGB(), QueryLabelMapAsImage(), etc…• UnityTexture2D rgbTexture = newTexture2D(640,480,TextureFormat.ARGB32, false);pipeline.QueryRGB(rgbTexture);
  16. 16. Intel Confidential16Retrieve Geonodes Data• Data Is Retrieved via QueryGeoNode()– Use PXCGesture::GeoNode::Label andPXCGesture::GeoNode to retrieve data• UnityPXCMGesture.GeoNode node;pipeline.QueryGeoNode(PXCMGesture.GeoNode.LABEL_BODY_HAND_PRIMARY, out node);
  17. 17. Intel Confidential17Retrieve Gesture Data• Data Is Retrieved via QueryGesture()– UsePXCGesture::GeoNode::Label, PXCGesture::Gesture::Label and PXCGesture::Gesture to retrievedata• UnityPXCMGesture.Gesture gest;pipeline.QueryGesture(PXCMGesture.GeoNode.LABEL_BODY_HAND_PRIMARY, out gest);if(gest.label==PXCMGesture.Gesture.Label.LABEL_HAND_WAVE)…
  18. 18. Intel Confidential18Clean Up• Use ReleaseFrame() To “Free Up ThePipeline”pipeline.ReleaseFrame();
  19. 19. Intel Confidential19What next ?http://perchackparis.eventbrite.fr/https://perceptualchallenge.intel.com/

×