• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
PL-4043, Accelerating OpenVL for Heterogeneous Platforms, by Gregor Miller
 

PL-4043, Accelerating OpenVL for Heterogeneous Platforms, by Gregor Miller

on

  • 369 views

Presentation PL-4043, Accelerating OpenVL for Heterogeneous Platforms, by Gregor Miller at the AMD Developer Summit (APU13) Nov. 11-13, 2013.

Presentation PL-4043, Accelerating OpenVL for Heterogeneous Platforms, by Gregor Miller at the AMD Developer Summit (APU13) Nov. 11-13, 2013.

Statistics

Views

Total Views
369
Views on SlideShare
367
Embed Views
2

Actions

Likes
0
Downloads
12
Comments
0

1 Embed 2

https://twitter.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    PL-4043, Accelerating OpenVL for Heterogeneous Platforms, by Gregor Miller PL-4043, Accelerating OpenVL for Heterogeneous Platforms, by Gregor Miller Presentation Transcript

    • OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU Gregor Miller and Sidney Fels Human Communication Technologies Laboratory University of British Columbia Vancouver, Canada 13th November 2013 AMD Developer Summit
    • Vision Everywhere OpenVL Source: AFDS Keynote by Phil Rogers, AMD Inc. (12/06/12) 2 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • What’s happening in the real world? OpenVL Quote from Hacker News about FastCV and OpenCV (28/10/11): “I hate how this (and to some extent OpenCV) expects you to understand computer vision to use the library. I sympathise with your frustration, but being in vision myself, I have an idea why this is the case: vision only barely works! What this means is that most things you would want to do are only barely possible, if at all, and it is highly application- and input-dependent. To get it to work the way you want, you have to understand some vision, so that you can tweak things the right way. I think once vision matures more and things become more ‘off-the-shelf’, you'll find more libraries aimed at non-experts.” Source: http://news.ycombinator.com/item?id=3167295 3 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Mainstream Developers are Key to Technology Adoption OpenVL WHAT  WILL  IT  TAKE  TO  HAVE  MORE  PROJECTS  LIKE  “CINEMADNG->OPENEXR CONVERSION”? HOW DO WE ATTRACT THE MAINSTREAM DEVELOPER? (Performance, Features, etc.) Developer Return GPU DEVELOPERS (NINJAs) ~100K GPU Coders ~200+ Apps Significant Niche Value MAINSTREAM DEVELOPERS ~10+M* CPU Coders ~4M+ Apps Good User Experiences Developer Investment (Effort, Time, New skills) *IDC Source: AFDS Keynote by Tom Malloy, Senior Vice President and Chief Software Architect, Adobe Systems Inc. (12/06/12) 4 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU 13 |    The  Promise  of  Parallel:  Today’s  State  of  Heterogeneous  Computing    |    June  12,  2012
    • Higher-Level Abstractions are Needed OpenVL SOFTWARE CONSIDERATIONS PLATFORMS (AND STANDARDS) Claim 3:  OpenCL is  a  valuable  standard  for  ninjas,  but  …  Higher level abstractions that provide performance portability needed for mainstream developers  May be evolutionary  See Khronos’  OpenCL Overview http://www.khronos.org/assets/uploads/developers/library/overview/opencloverview.pdf  …  or  it  may  be  something  completely  different Source: AFDS Keynote by Tom Malloy, Senior Vice President and Chief Software Architect, Adobe Systems Inc. (12/06/12) 5 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Current Computer Vision Frameworks OpenVL • OpenCV: Probably the most widely-used vision framework • Not aimed at mainstream developers: use face detection as an example • We do not consider it developer-friendly for three reasons: – The specific algorithm is used as the function name - cvHaarDetectObjects – The arguments are specific to the algorithm – No indication of the conditions under which it operates: e.g. how much of a rotation is allowed, what’s the smallest/largest face, etc. • Most common level of abstraction for vision, similar found in Matlab IPT/ CVT, ... 6 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • HCI for Vision, focussing on developers; Why? OpenVL • Example from FastCV: Qualcomm’s vision library optimised for mobile and embedded platforms – fcvCornerFast9InMaskScoreu8(const uint8_t *__restrict src, unsigned int srcWidth, unsigned int srcHeight, unsigned int srcStride, int barrier, unsigned int border, uint32_t *__restrict xy, uint32_t *__restrict scores, unsigned int nCornersMax, uint32_t *__restrict nCorners, const uint8_t *__restrict mask, unsigned int maskWidth, unsigned int maskHeight) – fcvNCCPatchOnCircle8x8u8_v2(const uint8_t *__restrict patch, const uint8_t *__restrict src, unsigned short srcWidth, unsigned short srcHeight, unsigned short search_center_x, unsigned short search_center_y, unsigned short search_radius, int filterLowVariance, uint16_t *best_x, uint16_t *best_y, uint32_t *bestNCC, int findSubPixel, float *subX, float *subY) – fcvTrackBMOpticalFlow16x16u8(const uint8_t *__restrict src1, const uint8_t *__restrict src2, uint32_t srcWidth, uint32_t srcHeight, uint32_t srcStride, uint32_t roiLeft, uint32_t roiTop, uint32_t roiRight, uint32_t roiBottom, uint32_t shiftSize, uint32_t searchWidth, uint32_t searchHeight, uint32_t searchStep, uint32_t usePrevious, uint32_t *numMv, uint32_t *locX, uint32_t *locY, uint32_t *mvX, uint32_t *mvY) – fcvDescriptor17x17u8To36s8(const uint8_t *__restrict patch, int8_t *__restrict descriptorChar, int32_t *__restrict descriptorNormSq) – fcvMserExtu8(void *mserHandle, const uint8_t *__restrict srcPtr, unsigned int srcWidth, unsigned int srcHeight, unsigned int srcStride, unsigned int maxContours, unsigned int *__restrict numContours, unsigned int *__restrict numPointsInContour, unsigned int *__restrict pointsArray, unsigned int pointsArraySize, unsigned int *__restrict contourVariation, int *__restrict contourPolarity, unsigned int *__restrict contourNodeId, unsigned int *__restrict contourNodeCounter) 7 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • A Case for the Problem Description Layer OpenVL Applications Consumer Computer Vision Problem Description Mainstream Developer Computer Vision Solutions Vision Ninja Computer Vision Frameworks Vision Researcher Hardware Acceleration API Physical Hardware Each of these layers plays a role for widespread adoption of computer vision. 8 Optimisation Specialists Users OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Higher-level abstraction for computer vision OpenVL ? Power Power Power Precision Precision Precision Performance Performance Performance 9 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Analysis using OpenVL: ‘What’ vs ‘How’ OpenVL • Analysis using OpenVL – High-level abstraction targeted towards mainstream developers • API based on problem description – The description is interpreted to select algorithm(s) to find the required solution • Expressive interface: ‘what’ vs. ‘how’ – Describe what the task is, not how to solve it • Example: face detection – Description: pose, 2D rotation, expression, age, occlusion, size quantity, species 10 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • OpenVL OpenVL • Input / Output • Operations – Images: input from developer (via VU) – Segments: basic unit; distinct regions within the image e.g. red circle – Matches/Detections: representation of matches and detections e.g. a face – Transforms: geometric representation e.g. vector of motion between segments • Descriptors Segment(constraints, images, segments) Detect(constraints, template, segments, detections) Match(constraints, variances, segments, matches) Solve(constraints, matches/detections, transforms) Select(constraints, conditions, segments, segments) • Constraints – Template: Description of well-known (and usually arbitrary) ‘object’ to find in images e.g. face, eyes, hand, body – Variances: Differences between segments such as intensity – Conditions: Property-based requirements such as colour == red 11 – – – – – – Operation-specific hints e.g. – Quantity of segments/detections/matches – Shape/regularity of segments – Type of match e.g. unique OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Using Segments to Abstract Pixels OpenVL Definition: A segment is a distinct continuously bounded region within a bounded 2D plane 12 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Segmentation using a Problem Description 13 OpenVL OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Image Registration Example OpenVL Segment - Match - Solve(Image) Input 14 Visualized Output OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Flexibility of Operation Sequences OpenVL • Segment - Match(variances) - Solve(Image) defines image registration – Match variances:  Intensity: HDR registration  Blur: Multi-focal registration • Segment - Match(variances) defines correspondence, as shown 15 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • OpenVL Matching Demo 16 OpenVL OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Optical Flow (Video Motion) OpenVL Segment - Match - Solve(Segment) • Change Solve(Image) to Solve(Segment) defines Optical Flow – Find the transform of each pair of matched segments 17 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Combined Example: Stitching and Detection 18 OpenVL OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Example accelerated back-end to OpenVL using HSA 19 OpenVL OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Summary and Future Work OpenVL • A problem description layer fills the gap between applications and algorithms – Critical for widespread adoption of computer vision – Allows replacement of implementation without impacting developer – High-level abstraction aimed at mainstream developers - not ninjas! • Expressive API (based on ‘What’, not ‘How’) • An abstraction for all platforms • Prototype reference implementations for CPU, GPU and HSA platforms 20 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • OpenVL has been released! (Image segmentation) OpenVL gregor@ece.ubc.ca | http://www.openvl.org 21 OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU
    • Acknowledgements 22 OpenVL OpenVL: A Developer-Friendly Abstraction for Computer Vision on APU, GPU and CPU