Open Standards for Gaming

  • 376 views
Uploaded on

Following our successful participation at SIGGRAPH Asia 2012 in Singapore, the Khronos Group is excited to demonstrate and educate about Khronos APIs at SIGGRAPH Asia 2013 in Hong Kong. This …

Following our successful participation at SIGGRAPH Asia 2012 in Singapore, the Khronos Group is excited to demonstrate and educate about Khronos APIs at SIGGRAPH Asia 2013 in Hong Kong. This presentation covers Open Standards for today's gaming industry, by Erik Noreke.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
376
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
10
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Open Standards for Today’s Gaming Industry Erik Noreke VP of Business Development, Khronos Group © Copyright Khronos Group 2013 - Page 1
  • 2. Gaming Evolution DESKTOP New high performance games continue to evolve on high-end platforms MOBILE Mobile is the new platform for game innovation. Mobile APIs unlock hardware acceleration and conserve battery life, making portable gaming possible Diverse platforms – mobile, TV, embedded – means HTML5 will become increasingly important as a universal gaming platform © Copyright Khronos Group 2013 - Page 3
  • 3. Courtesy of PC Gaming Alliance www.pcgamingalliance.org © Copyright Khronos Group 2013 - Page 4
  • 4. Courtesy of PC Gaming Alliance www.pcgamingalliance.org © Copyright Khronos Group 2013 - Page 5
  • 5. US$28.7B gaming revenues generated on platforms supporting Khronos native graphics and compute APIs Courtesy of PC Gaming Alliance www.pcgamingalliance.org © Copyright Khronos Group 2013 - Page 6
  • 6. OpenGL 3D API Family Tree ES3 is backward compatible so new features can be added incrementally Programmable vertex and fragment shaders Fixed function 3D Pipeline OpenGL ES 2.0 Content OpenGL ES 1.1 Content OpenGL ES 3.0 Content Mobile 3D WebGL 1.0 OpenGL ES 1.1 OpenGL ES 2.0 WebGL-Next OpenGL ES 3.0 ES-Next OpenGL ES 1.0 OpenGL 1.3 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.1 OpenGL 3.3 OpenGL 4.2 OpenGL 4.3 OpenGL 4.4 OpenGL 4.0 OpenGL 3.0 OpenGL 3.2 OpenGL 4.1 OpenGL 4.4 is a superset of DX11 Desktop 3D 2002 2003 2004 GL-Next 2005 2006 2007 2008 2009 2010 2011 2012 2013 © Copyright Khronos Group 2013 - Page 7
  • 7. Mobile OpenCL Shipping • Android ICD extension released in latest extension specification - OpenCL implementations can be discovered and loaded as a shared object • Multiple implementations shipping in Android NDK - ARM, Imagination, Vivante, Qualcomm, Samsung … © Copyright Khronos Group 2013 - Page 8
  • 8. Active Gamers by Platform Courtesy of PC Gaming Alliance www.pcgamingalliance.org CAMERA © Copyright Khronos Group 2013 - Page 9
  • 9. Sensors & Cameras • Single sensor RGB cameras are just the start of the mobile visual revolution - IR sensors – LEAP Motion, eye-trackers • Multi-sensors: Stereo pairs -> Plenoptic array -> Depth cameras - Stereo pair can enable object scaling and enhanced depth extraction • Hybrid visual sensing solutions - Different sensors mixed for different distances and lighting conditions Dual Camera Plenoptic Array Capri Structured Light 3D Camera LG Electronics Pelican imaging PrimeSense © Copyright Khronos Group 2013 - Page 10
  • 10. Typical Imaging Pipeline • Pre- and Post-processing can be done on CPU, GPU, DSP… • ISP controls camera via 3A algorithms Auto Exposure (AE), Auto White Balance (AWB), Auto Focus (AF) • ISP may be a separate chip or within Application Processor Lens, sensor, aperture control 3A Bayer Pre-processing CMOS sensor Color Filter Array Lens RGB/YUV Image Signal Processor (ISP) Postprocessing App Need for advanced camera control API! CAMERA © Copyright Khronos Group 2013 - Page 11
  • 11. Low Power Environment Scanning • Many sensor use cases would consume too much power to be running 24/7 - Environment aware use cases have to be very low power • ‘Scanners’ - very low power, always on, detect things in the environment - Trigger the next level of processing capability ARM 7 DSP GPU 1 MIP and accelerometers can detect someone in the vicinity Low power activation of camera to detect someone in field of view GPU acceleration for precision gesture processing © Copyright Khronos Group 2013 - Page 12
  • 12. StreamInput Sensor Fusion Stack Applications Platforms can provide increased access to improved sensor data stream – driving faster, deeper sensor usage by applications StreamInput implementations compete on sensor stream quality, reduced power consumption, environment triggering and context detection – enabling sensor subsystem vendors to increased ADDED VALUE OS Sensor OS APIs Middleware (E.g. Android SensorManager or iOS CoreMotion) Middleware engines need platformportable access to native, low-level sensor data stream (E.g. Augmented Reality engines, gaming engines) Low-level native API defines access to fused sensor data stream and context-awareness Sensor Sensor … Sensor Sensor Hub Hub Mobile or embedded platforms without sensor fusion APIs can provide direct application access to StreamInput Hardware transport interfaces are defined by each system, e.g. IIO or HID sensor © Copyright Khronos Group 2013 - Page 13
  • 13. Worldwide Game Users – 2012 (Active and Casual) WORLDWIDE GAME USERS BY REGION AND PLATFORM (IN MILLIONS) 500 400 300 200 100 0 N.America Europe Console PC Asia Dedicated Handheld Other Mobile Courtesy of PC Gaming Alliance www.pcgamingalliance.org © Copyright Khronos Group 2013 - Page 14
  • 14. Cross-OS, Cross Platform Portability HTML/CSS SDK C/C++ Dalvik (Java) HTML/CSS Objective C HTML/CSS HTML5 provides cross platform portability. GPU accessibility through WebGL available soon on ~90% mobile systems C# Preferred development environments not designed for portability DirectX Native code is portablebut apps must cope with different available APIs and libraries © Copyright Khronos Group 2013 - Page 15
  • 15. WebGL Availability in Browsers Much WebGL content uses three.js library: http://threejs.org/ - Microsoft – “where you have IE11, you have WebGL – turned on by default and working all the time” - Microsoft - WebGL also enabled for Windows applications - web app framework and web view - Apple - WebGL must be explicitly turned on MAC Safari and only exposed on iOS for iAds - Chrome OS - WebGL is the only cross-platform API to program the GPU - Google IO announcement - Chrome on Android will soon launch with WebGL © Copyright Khronos Group 2013 - Page 16
  • 16. WebCL – Parallel Computing for the Web • JavaScript bindings to hardware acceleration - Giving performance gaming in a web browser http://www.youtube.com/user/SamsungSISA#p/a/u/1/9Ttux1A-Nuc © Copyright Khronos Group 2013 - Page 17
  • 17. © Copyright Khronos Group 2013 - CapCom Remember Me image courtesy of Page 18
  • 18. 3D Needs a Transmission Format! • Compression and streaming of 3D assets becoming essential - Mobile and connected devices need access to increasingly large asset databases • 3D is the last media type to define a compressed format - 3D is more complex – diverse asset types and use cases • Needs to be royalty-free - Avoid an ‘internet video codec war’ scenario • Eventually enable hardware implementations of successful codecs - High-performance and low power – but pragmatic adoption strategy is key Audio Video Images 3D MP3 H.264 JPEG ? ! An effective and widely adopted codec ignites previously unimagined opportunities for a media type © Copyright Khronos Group 2013 - Page 19
  • 19. glTF Goals • Binary file format for efficient transmission for 3D assets - Reduce network bandwidth and minimize client processing overhead • Run-time neutral - DO NOT IMPLY OR MANDATE ANY RUN-TIME BEHAVIOR - Can be used by any app or run-time – usually WebGL accelerated • Scalable to handle compression and streaming - Though baseline format does not include compression • ‘Direct load efficiency’ for WebGL - Little or NO processing to drop glTF data into WebGL client • Carry conditioned data from any authoring format - Prototyping and optimizing efficient handling of COLLADA assets Authoring Playback A standards-based content pipeline for rich native and Web 3D applications © Copyright Khronos Group 2013 - Page 20
  • 20. Game On! • Gaming and Animation is a complex and expanding application domain and coherent multiple standards across multiple domains are needed to enable the market • Khronos family of standards allow you to focus your efforts on game development, not porting between platforms • Now is a great time to get involved with the standards initiatives that effect your business and game development • www.khronos.org • erik.noreke@khronos.org © Copyright Khronos Group 2013 - Page 21
  • 21. © Copyright Khronos Group 2013 - Page 22
  • 22. WebGL First Wave Application Categories • Maps and Navigation • Modeling Tools and Repositories • Games • 3D Printing • Visualization • Music Videos and Promotion • Education • Photo Editors • Music Visualizers • Vision/Video Processing © Copyright Khronos Group 2013 - Page 23
  • 23. COLLADA and glTF Open Source Ecosystem OpenCOLLADA Importer/Exporter and COLLADA Conformance Tests On GitHUB Tool Interop COLLADA2GLTF Translator Other authoring formats https://github.com/KhronosGroup/glTF Web-based Tools Pervasive WebGL deployment https://github.com/KhronosGroup/OpenCOLLADA https://github.com/KhronosGroup/COLLADA-CTS Three.js glTF Importer. Rest3D initiative © Copyright Khronos Group 2013 - Page 24
  • 24. Google Maps • All rendering (2D and 3D) in Google Maps uses WebGL © Copyright Khronos Group 2013 - Page 25
  • 25. WebCL – Parallel Computing for the Web • JavaScript bindings to OpenCL APIs - Enables initiation of Kernels written in OpenCL C within the browser • Bindings stay close to the OpenCL standard - Maximum flexibility to provide a foundation for higher-level middleware - Minimal language modifications for 100% security and app portability - E.g. Mapping of CL memory objects into host memory space is not supported • API definition underway – public draft released - https://cvs.khronos.org/svn/repos/registry/trunk/public/webcl/spec/latest/index.html • Compelling use cases - Physics engines for WebGL games - Image and video editing in browser River Trail JavaScript binding to OpenCL APIs to enable initiation of OpenCL C kernels Data-parallel language extensions to JavaScript for offload at run-time © Copyright Khronos Group 2013 - Page 26
  • 26. Leveraging Proven Native APIs into HTML5 • Khronos and W3C liaison - Leverage proven native API investments into the Web - Fast API development and deployment - Designed by the hardware community - Familiar foundation reduces developer learning curve HTML Canvas WebVX? Vision Processing Path Rendering Native APIs shipping or Khronos working group JavaScript API shipping, acceleration being developed or work underway WebCAM(!) WebStream? Sensor Fusion Camera control and video processing Camera Control JavaScript Native Possible future JavaScript APIs or acceleration © Copyright Khronos Group 2013 - Page 27
  • 27. OpenGL ES 3.0 Highlights • Better looking, faster performing games and apps – at lower power - Incorporates proven features from OpenGL 3.3 / 4.x - 32-bit integers and floats in shader programs - NPOT, 3D textures, depth textures, texture arrays - Multiple Render Targets for deferred rendering, Occlusion Queries - Instanced Rendering, Transform Feedback … • Make life better for the programmer - Tighter requirements for supported features to reduce implementation variability • Backward compatible with OpenGL ES 2.0 - OpenGL ES 2.0 apps continue to run unmodified • Standardized Texture Compression - #1 developer request! © Copyright Khronos Group 2013 - Page 32
  • 28. Accelerating OpenGL Innovation Bringing state-of-theart functionality to cross-platform graphics OpenGL 4.4 OpenGL 4.3 OpenGL 4.2 OpenGL 4.1 OpenGL 3.3/4.0 OpenGL 3.2 OpenGL 3.1 OpenGL 2.0 2004 OpenGL 2.1 2005 DirectX 9.0c 2006 2007 DirectX 10.0 OpenGL 3.0 2008 2009 DirectX 10.1 2010 DirectX 11 2011 2012 2013 DirectX 11.1 © Copyright Khronos Group 2013 - Page 33
  • 29. OpenGL 4.3 Compute Shaders • Execute algorithmically general-purpose GLSL shaders - Can operate on uniforms, images and textures • Process graphics data in the context of the graphics pipeline - Easier than interoperating with a compute API IF processing ‘close to the pixel’ • Standard part of all OpenGL 4.3 implementations - Matches DX11 DirectCompute functionality Physics AI Simulation Ray Tracing Imaging Global Illumination © Copyright Khronos Group 2013 - Page 34
  • 30. OpenGL Conformance Test Suite released!   Conformance submissions are required for GL 4.4 implementations  encouraged for earlier driver versions Shared codebase with OpenGL ES 3.0 CTS  additional desktop-specific tests  Core profile functionality  Enhancements underway to add more coverage © Copyright Khronos Group 2013 - Page 35
  • 31. OpenCL and OpenGL Compute Shaders • OpenGL compute shaders and OpenCL support distinctly different use cases - OpenCL provides a significantly more powerful and complete compute solution 1. Full ANSI C programming of heterogeneous CPUs and GPUs 2. Utilize multiple processors 3. Precisely defined IEEE accuracy 1. Fine grain compute operations inside OpenGL 2. GLSL Shading Language 3. Execute on single GPU only Enhanced 3D Graphics apps “Shaders++” Imaging Video Physics AI Pure compute apps touching no pixels Compute Shaders © Copyright Khronos Group 2013 - Page 36
  • 32. Gaming Evolution DESKTOP New high performance games still evolve on highend platforms MOBILE Mobile is the new platform for game innovation. Mobile APIs unlock hardware acceleration and conserve battery life, making portable gaming possible Diverse platforms – mobile, TV, embedded – means HTML5 will become increasingly important as a universal gaming platform © Copyright Khronos Group 2013 - Page 37