Khronos Overview 概述

Beijing China, April 2013 北京 中国, 2013年4月
Erik Noreke
Technology Visionary
Vice President Business Dev...
Topics 主题
•  Why open standards advance the industry
•  How Khronos creates open API standards for silicon acceleration
• ...
Why do we NEED Standards? 我们为什么需要标准?
•  Need standard INTERFACES for
INTEROPERABILITY …
… Widely adopted standard interfac...
Khronos Connects Software to Silicon 链接软件与芯片
•  Khronos APIs define processor acceleration capabilities
-  Graphics, video...
APIs BY the Industry FOR the Industry
业界为行业开发的API
•  Khronos defines APIs at the software
•  Khronos为软件芯片界面定义API
silicon i...
APIs BY the Industry FOR the Industry
业界为行业开发的API
•  Khronos is OPEN for any company
to join and participate
-  Standards ...
Apple

Over 100 members – any company
worldwide is welcome to join

Board of Promoters

© Copyright Khronos Group 2013 | P...
Khronos Standards
Unified Sensor and
Input Processing

统一传感器
和输入处理

15 Active Standards
15个现行标准

Plugin-free
3D Web Conten...
Additional Khronos Initiatives 其他活动
•  KITE
-  Connecting Industry and Educators

• KITE - Khronos培训教育学院
-  链接业界与学术界

•  C...
Khronos Working Groups

工作组

Academic Members 学术会员
-  Participation in WGs
-  工作组参与权

Contributor Members 贡献者会员

API Worki...
Academia is Responding Positively
学术界的积极参与
•  Nine New Academic Members joining in 2012 since announcing Kite!
•  自2012年推出...
API Standard Evolution API标准的演进
DESKTOP
桌面

New API
technology first
evolves on high-end
platforms
全新API技术首先进入高
端平台

MOBIL...
Technology Overview 技术概述
Beijing China, April 2013 北京 中国, 2013年4月
Erik Noreke
Technology Visionary
Vice President Business...
OpenGL 20th Birthday - Then and Now
OpenGL 20岁生日 – 过去与未来
Ideas in Motion - SGI

Launched OpenGL 4.3
at SIGGRAPH 2012
发布Ope...
OpenGL 4.3 Compute Shaders 计算着色器
•  Execute algorithmically general-purpose
GLSL shaders
-  Operate on uniforms, images an...
OpenGL ES

Fixed function
3D Pipeline
修复后功能3D管线

Programmable vertex and
fragment shaders
可编程顶点和碎片着色器

ES3 is backward
com...
OpenGL ES 3.0 Highlights 亮点
•  Better looking, faster performing
games and apps – at lower power
-  Incorporates proven fe...
OpenGL ES Deployment in Mobile 移动领域部署
Use of 3D APIs in Mobile Devices
Source: Jon Peddie Research
移动设备3D API的使用
来源:Jon Pe...
Processor Parallelism 处理器并行性
CPUs

Multiple cores driving
performance increases
多核驱动性能提高

Emerging
Intersection
新兴交叉点

Mul...
OpenCL – Heterogeneous Computing 异构计算
•  Native framework for programming diverse parallel
computing resources
-  CPU, GPU...
OpenCL 1.2 Update 更新
•  Create an OpenCL image from a OpenGL multi
-sampled texture
-  Provides more flexibility in intero...
Mobile Innovation Hot Spots 移动创新热点
•  New platform capabilities being driven by SILICON and APIs
•  芯片和API所带动的新平台性能
Consol...
OpenMAX - Media Acceleration 媒体加速
•  Family of royalty-free, cross-platform open API standards for video, image stream and...
OpenMAX AL Streaming Media Framework 流媒体框架
•  Enables key video, image stream and camera
use cases
-  Enables optimal hard...
OpenSL ES – Advanced Audio 高级音频
•  OpenSL ES does for audio what OpenGL ES
does for graphics
-  Advanced audio functionali...
EGLStream – Streaming Images 流图片
•  EGL – originally embedded version of WGL
-  Abstraction layer to window systems and me...
EGLStream – Streaming Images 流图片

OpenMAX AL Media Player is the
EGLStream “Producer” and controls
production of frames.
O...
Portable Access to Sensor Fusion 传感融合的便携式获取
Apps request semantic sensor information

Spec expected in 2013
预计2013年推出规范

S...
OpenVX
•  Vision Hardware Acceleration Layer
-  Enables hardware vendors to implement accelerated
imaging and vision algor...
HTML5 – Cross OS App Platform
跨操作系统应用平台
•  Increasing diversity of devices creates a
demand for a true cross OS programmin...
HTML5 – Cross OS App Platform
跨操作系统应用平台

Traditional Web-content 传统网页内容

Rich Experiential Processing
Multi-core CPUs
Rich...
Leveraging Proven Native APIs into HTML5
本地API到HTML5上的过渡
•  Leverage native API investments into the Web
-  Faster API dev...
WebGL – 3D on the Web – No Plug-in!
网页上的3D – 无插件!
•  Historic opportunity to bring accelerated
3D graphics to web
-  WebGL...
WebCL – Parallel Computing for the Web 网页并行计算
•  JavaScript bindings to OpenCL APIs
-  Enables initiation of Kernels writt...
API Adoption 采用

Mobile Operating Systems 手机操作系统

OpenGL ES 2.0
Shipping - Android 2.2
OpenSL ES 1.0
Shipping – Android 2....
Extended Native APIs on Android 安卓上扩展的本地API
•  Native APIs can be shipped as
NDK extensions before Google
Adoption
-  Do n...
In Summary 总结
•  APIs are key to enable compelling applications on advanced hardware – APIs developed
on high-end hardware...
Upcoming SlideShare
Loading in …5
×

2013 April Meetup Asia - Khronos Overview

1,094 views
893 views

Published on

Khronos Group Overview in Beijing China 2013 by Erik Noreke, Technology Visionary and Vice President Business Development of The Khronos Group http://www.khronos.org/

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,094
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

2013 April Meetup Asia - Khronos Overview

  1. 1. Khronos Overview 概述 Beijing China, April 2013 北京 中国, 2013年4月 Erik Noreke Technology Visionary Vice President Business Development, The Khronos Group © Copyright Khronos Group 2013 | Page 1
  2. 2. Topics 主题 •  Why open standards advance the industry •  How Khronos creates open API standards for silicon acceleration •  State of the art in visual computing APIs •  Mobile Multimedia and Advanced Audio •  Vision and sensor processing for Augmented Reality •  Advanced hardware acceleration in HTML5 • 开放标准是如何推动行业发展的 • Khronos是如何为芯片加速创建API标准的 • 视觉计算API的最新动态 • 移动多媒体和高级音频 • 增强现实的视觉和传感处理 • HTML5的高级硬件加速 © Copyright Khronos Group 2013 | Page 2
  3. 3. Why do we NEED Standards? 我们为什么需要标准? •  Need standard INTERFACES for INTEROPERABILITY … … Widely adopted standard interfaces can grow market opportunity •  需要为交互操作提供标准界面… …受到广泛采用的标准界面可以带动市场增长 •  Interoperability enables compelling user experiences … … to be built cheaply enough to build a mass market •  互操作性实现了更强的用户体验… •  Standards avoid fragmentation that adds no value … … Industries need to COOPERATE to build a market and then COMPETE •  标准避免了毫无价值的存储残片问题… •  The mobile industry is defined by standards -  GSM/EDGE, UMTS/HSPA, LTE, IEEE 802.11, Bluetooth, USB, HDMI … •  移动行业是由标准定义的 …以最低的成本构建巨大的市场 …业界需要合作建立市场然后再竞争 - GSM/EDGE, UMTS/HSPA, LTE, IEEE 802.11, Bluetooth, USB, HDMI … © Copyright Khronos Group 2013 | Page 3
  4. 4. Khronos Connects Software to Silicon 链接软件与芯片 •  Khronos APIs define processor acceleration capabilities -  Graphics, video, audio, compute, vision and sensor processing •  Khronos的API定义了处理加速性能 - 图形图像、视频、音频、计算、视觉和传感处理 APIs developed today define the functionality of platforms and devices tomorrow 今天开发的API定义着未来平台和设备的性能 © Copyright Khronos Group 2013 | Page 4
  5. 5. APIs BY the Industry FOR the Industry 业界为行业开发的API •  Khronos defines APIs at the software •  Khronos为软件芯片界面定义API silicon interface -  每个平台都需要的低水平“基础”性能 -  Low-level “Foundation” functionality needed on every platform •  Khronos standards have strong industry momentum -  100s of man years invested by industry experts -  Shipping on billions of devices across multiple operating systems -  Rigorous conformance tests for cross -vendor consistency •  Khronos标准拥有强大的行业意义 -  行业领先专家超过100人年的投入 -  在数十亿的设备和各种操作系统上使用 -  严格的符合性测试以保证各方面一致性 Silicon Software © Copyright Khronos Group 2013 | Page 5
  6. 6. APIs BY the Industry FOR the Industry 业界为行业开发的API •  Khronos is OPEN for any company to join and participate -  Standards are cooperative – one company, one vote -  Proven legal and IP framework for industry cooperation -  Khronos membership fees to cover expenses •  Khronos欢迎所有公司加入和参与 -  真正开放的标准 —— 一家公司一个投票权 -  为业界合作提供巩固的法律和知识产权框架 -  Khronos会员费仅仅用于组织运作费用 •  They are FREE •  全部是免费的 -  会员同意无需支付版税 -  Members agree to not request royalties Silicon Software © Copyright Khronos Group 2013 | Page 6
  7. 7. Apple Over 100 members – any company worldwide is welcome to join Board of Promoters © Copyright Khronos Group 2013 | Page 7
  8. 8. Khronos Standards Unified Sensor and Input Processing 统一传感器 和输入处理 15 Active Standards 15个现行标准 Plugin-free 3D Web Content Cross platform desktop 3D Camera, Images and Streaming Media 3D Digital Asset Exchange format 3D 数字资产交 换格式 照相机、图片、 流媒体 2000 2003 无插件3D网页 内容 并行计算 Hardware accelerated vision functions 硬件加速视觉功能 跨平台桌面 3D Web Compute Embedded and Mobile 3D 嵌入和移动3D Parallel Computing 网页计算 Advanced Audio Vector 2D 高级音频 矢量 2D 2004 2005 2006 2008 2009 2011 © Copyright Khronos Group 2013 | Page 8
  9. 9. Additional Khronos Initiatives 其他活动 •  KITE -  Connecting Industry and Educators • KITE - Khronos培训教育学院 -  链接业界与学术界 •  Chapters • 分会 -  Grass Roots Community for Khronos -  Khronos草根社群 – 相关活动 -related activities -  全球网络汇集志同道合的志愿者和 -  Global network of like minded volunteers 参与者 – 类似ACM and participants – similar to ACM -  当地社交活动,教育和沟通 -  Local networking, education and activities (Education) WebGL Meet-up San Francisco Sep11 WebGL旧金山线下活动,2011年9月 Chapters (Community) Industry (Technology) © Copyright Khronos Group 2013 | Page 9
  10. 10. Khronos Working Groups 工作组 Academic Members 学术会员 -  Participation in WGs -  工作组参与权 Contributor Members 贡献者会员 API Working Groups -  Participation and vote in WGs -  工作组参与权和投票权 (Industry and Academic) 工作组(企业与学术会员) Promoter Members 发起者会员 - Participation and vote in WGs -  Board seat for strategy, budget and spec ratification -  工作组参与权和投票权 -  董事会席位,参与战略、预算和规范审批 Conformance Tests and Adopters Program 符合性测试和采用者项目 Ratified Specifications 规范批准 SDKs, Sample, Ref Cards and Man Pages 软件开发包、示例、参考卡、 手册 KITE encourages academic participation in working groups to produce materials for educators and certification KITE鼓励学术会员参与到工作组中,为教育者 和认证方面创建素材 Educator Guidelines and Courseware 教育者指导和课件 Certification Materials 认证素材 Members 会员 Wider Community 更广泛的社群 Adopters 采用者 Build conformant implementation and products 建立符合性执行和产品 Developers 开发者 Develop applications using the APIs 使用API开发应用程序 Educators / Certifiers 教育者/认证者 Academic Courses 学术课件 Training and Certification 培训和认证 © Copyright Khronos Group 2013 | Page 10
  11. 11. Academia is Responding Positively 学术界的积极参与 •  Nine New Academic Members joining in 2012 since announcing Kite! •  自2012年推出KITE项目,已有9个新的学术会员加入! Khronos Academic Member 学术会员 Country 国家 Date Joined Khronos 加入日期 Beijing University of Posts and Telecommunications 北京邮电大学 China 中国 8 Apr 2013 Nihon University 日本大学 Japan 日本 18 Mar 2013 University of Bristol 布里斯托大学 Great Britain 英国 7 Aug 2012 ITRI 工业促进协会 Taiwan 台湾 5 July 2012 Oregon State University 俄勒冈州大学 USA 美国 29 May 2012 Politecnico di Milano 米兰理工大学 Italy 意大利 01 Mar 2012 Universita di Bologna 博洛尼亚大学 Italy 意大利 01 Mar 2012 France 法国 13 Feb 2012 Imperial College London 帝国理工大学 Great Britain 英国 17 Jan 2012 University of Toronto 多伦多大学 Canada 加拿大 31 May 2011 Australia 澳大利亚 14 May 2010 Kyungpook National University 庆北大学 Korea 韩国 23 Mar 2010 Seoul National University 首尔国立大学 Korea 韩国 26 May 2009 UMEÅ 于默奥大学 Sweden 瑞典 18 Apr 2008 Universite Claude Bernard Lyon University Western Australia 里昂第⼀一大学 西澳大学 © Copyright Khronos Group 2013 | Page 11
  12. 12. API Standard Evolution API标准的演进 DESKTOP 桌面 New API technology first evolves on high-end platforms 全新API技术首先进入高 端平台 MOBILE 移动 Mobile is the new platform for apps innovation. Mobile APIs unlock hardware and conserve battery life 手机是应用创新的新平台。移动AP I解脱了硬件和电池消耗的束缚 INTEROP, VISION AND SENSORS 交互操作,视觉和传感 WEB 网页 Apps embrace mobility’s unique strengths and need complex, interoperating APIs with rich sensory inputs e.g. Augmented Reality 各种应用涵盖了移动设备的特殊性能并需要 复合功能的实现,于是带来了拥有丰富传 感输入的内部操作API,如增强现实 Diverse platforms – mobile, TV, embedded – means HTML5 will become increasingly important as a universal app platform 各种平台– 手机、电视、嵌入式HTML 5将越来越重要 © Copyright Khronos Group 2013 | Page 12
  13. 13. Technology Overview 技术概述 Beijing China, April 2013 北京 中国, 2013年4月 Erik Noreke Technology Visionary Vice President Business Development, The Khronos Group © Copyright Khronos Group 2013 | Page 13
  14. 14. OpenGL 20th Birthday - Then and Now OpenGL 20岁生日 – 过去与未来 Ideas in Motion - SGI Launched OpenGL 4.3 at SIGGRAPH 2012 发布OpenGL4.3  Triangles  /  sec  (millions)   三角形/秒(百万)    Pixel  Fragments  /  sec  (millions)   像素碎片/秒(百万)    GigaFLOPS     Rage -id Software 1992   Reality  Engine   8  Geometry  Engines     4  Raster  Manager  boards   2012  Mobile   NVIDIA  Tegra  3   Nexus  7  Android   Tablet   2012  PC   NVIDIA     GeForce  GTX  680     Kepler  GK104     1     103  (x103)   1800  (x1800)   240     1040  (x4.3)   14,400    (x60)   0.64   15.6  (x25)   3090   (x4830)   1.5KW <5W © Copyright Khronos Group 2013 | Page 14
  15. 15. OpenGL 4.3 Compute Shaders 计算着色器 •  Execute algorithmically general-purpose GLSL shaders -  Operate on uniforms, images and textures •  在算法上执行整体目标GLSL着色器 -  在uniforms、图片、纹理上操作 •  Process graphics data in the context of the graphics pipeline -  Easier than interoperating with a compute API IF processing ‘close to the pixel’ •  在图形图像管线上下文中处理图形图像数据 -  比和一个计算API IF处理“close to the pixel”交互操作更容易 •  Standard part of all OpenGL 4.3 implementations -  Matches DX11 DirectCompute functionality •  全部OpenGL 4.3 执行的标准部分 -  和DX11 DirectCompute性能匹配 Physics 物理 AI Simulation AI 模拟 Ray Tracing 光线追踪 Imaging 成像 Global Illumination 全局光照 © Copyright Khronos Group 2013 | Page 15
  16. 16. OpenGL ES Fixed function 3D Pipeline 修复后功能3D管线 Programmable vertex and fragment shaders 可编程顶点和碎片着色器 ES3 is backward compatible with ES2 so new features can be added incrementally ES3是和ES2向后兼容的, 所以可以直接添加新功能 •  Streamlined subset of desktop OpenGL for embedded and mobile devices •  用于嵌入式和移动设备的桌 面OpenGL的流线化子集 OpenGL ES 1.1 Platforms Released OpenGL ES 1.0 Spec Released OpenGL ES 1.1 Spec Released OpenGL 1.5 OpenGL ES 2.0 Content OpenGL ES 1.1 Content OpenGL ES 3.0 Platforms Released OpenGL ES 2.0 Platforms Released OpenGL ES 2.0 Spec Released OpenGL 2.0 OpenGL ES 3.0 Content OpenGL ES 3.0 Spec Released GL4/DX11-class Capabilities come to ES ?? OpenGL 3.1 OpenGL 3.3 OpenGL 4.2 OpenGL 4.3 OpenGL 4.0 OpenGL 3.0 OpenGL 3.2 OpenGL 4.1 OpenGL 2.1 OpenGL 4.3 is a superset of DX11 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 © Copyright Khronos Group 2013 | Page 16
  17. 17. OpenGL ES 3.0 Highlights 亮点 •  Better looking, faster performing games and apps – at lower power -  Incorporates proven features from OpenGL 3.3 / 4.x -  32-bit integers and floats in shader programs -  NPOT, 3D textures, depth textures, texture arrays -  Multiple Render Targets for deferred rendering, Occlusion Queries -  Instanced Rendering, Transform Feedback … •  更好的外观、更高的游戏和应用性能 – 更低的能耗 -  融合经证实的OpenGL 3.3 / 4.x 的功能 -  着色器编程中32位的integer和float -  NPOT,3D纹理,深度纹理,纹理排列 -  延迟渲染、Occlusion Queries的多渲染器目标 -  实例渲染,转换反馈… •  让编程员的生活更好 -  对降低执行可变性的支持功能提出更高的要求 •  与OpenGL ES 2.0向后兼容 -  OpenGL ES 2.0 应用程序无需修改继续运行 •  标准化纹理压缩 -  开发者的首要需求! •  Make life better for the programmer -  Tighter requirements for supported features to reduce implementation variability •  Backward compatible with OpenGL ES 2.0 -  OpenGL ES 2.0 apps continue to run unmodified •  Standardized Texture Compression -  #1 developer request! © Copyright Khronos Group 2013 | Page 17
  18. 18. OpenGL ES Deployment in Mobile 移动领域部署 Use of 3D APIs in Mobile Devices Source: Jon Peddie Research 移动设备3D API的使用 来源:Jon Peddie Research 研究公司 On  PC  –  DirectX  is  used  for  most  apps.     On  mobile  the  situaVon  is  reversed   在PC上 – 大多数应用使用DirectX 在移动设备上确实恰恰相反的 OpenGL ES is the 3D API used in Android, iOS and almost every other mobile and embedded OS – other than Windows OpenGL ES是在Android,iOS和几乎每一个其他移动和嵌入式操作系统上使 用的3D API, 远远不只Windows © Copyright Khronos Group 2013 | Page 18
  19. 19. Processor Parallelism 处理器并行性 CPUs Multiple cores driving performance increases 多核驱动性能提高 Emerging Intersection 新兴交叉点 Multiprocessor programming – e.g. OpenMP 多处理器编程 – 如OpenMP Heterogeneous Computing 异构计算 GPUs Increasingly general purpose data-parallel computing 不断增长的整体目标数据并行计算 Graphics APIs and Shading Languages 图形图像API 和着色语言 OpenCL is a programming framework for heterogeneous compute resources OpenCL是一种异构计算资源的编程框架 © Copyright Khronos Group 2013 | Page 19
  20. 20. OpenCL – Heterogeneous Computing 异构计算 •  Native framework for programming diverse parallel computing resources -  CPU, GPU, DSP – as well as hardware blocks(!) OpenCL Kernel OpenCL Code OpenCL Kernel Code OpenCL Kernel Code Kernel Code •  Powerful, low-level flexibility -  Foundational access to compute resources for higher-level engines, frameworks and languages •  Embedded profile -  No need for a separate “ES” spec -  Reduces precision requirements CPU OpenCL Kernel OpenCL Code OpenCL Kernel Code OpenCL Kernel Code Kernel Code •  为多种并行计算资源编程的本地框架 -  CPU, GPU, DSP和硬件组块(!) •  强大的低水平灵活性 -  为高水平引擎、框架和语言基础接入到计算资源 •  嵌入式profile -  无需独立的”ES”规范 -  降低精确度要求 A cross-platform, cross-vendor standard for harnessing all the compute resources in an SOC 为巩固SOC内各种计算资源的一个跨平台,跨用户标准 CPU GPU GPU CPU One code tree can be executed on CPUs or GPUs 一个代码树可以在CPU和GPU上使用 © Copyright Khronos Group 2013 | Page 20
  21. 21. OpenCL 1.2 Update 更新 •  Create an OpenCL image from a OpenGL multi -sampled texture -  Provides more flexibility in interoperating 3D graphics and compute •  从OpenGL多示例纹理创建一个OpenCL图像 -  在交互操作3D图形图像和计算中提供更多的灵 活性 •  Create 2D images from an OpenCL buffer -  Process memory structures using the advanced properties of OpenCL images •  从OpenCL缓冲创建2D图像 -  使用OpenCL图像的高级性能处理存储结构 •  Security features for WebCL implementations •  WebCL执行分层的OpenCL安全特性 layered over OpenCL -  在核执行之前的本地初始化和独立存储 -  Initialize local and private memory before a kernel -  队列和API终止OpenCL文本,以确保长时间核 begins execution 运行不会影响系统稳定性 -  Query and API to terminate an OpenCL context to ensure a long running kernel does not affect system stability •  从一个SPIR载入OpenCL项目目标 •  Load an OpenCL program object from a -  不断提高的工具链灵活性并避免了商业应用中 Standard Portable Intermediate Representation 跳过核资源的需求 (SPIR) instance -  Increased tool chain flexibility and avoids the need to ship kernel source in commercial applications © Copyright Khronos Group 2013 | Page 21
  22. 22. Mobile Innovation Hot Spots 移动创新热点 •  New platform capabilities being driven by SILICON and APIs •  芯片和API所带动的新平台性能 Console-Class 3D Performance, Quality, Controllers and TV connectivity 遥控类3D 性能,质量,遥控器和电视连接 •  Vision - Camera as sensor Computational Photography Gesture Processing Augmented Reality 视觉 – 摄像机传感器 计算性相片 姿势处理 增强现实 •  Media and Image Streaming Heterogeneous Parallel Processing •  媒体和图像流 •  构并行处理 Sensor Fusion Devices become ‘magically’ context aware – location, usage, position 传感器融合 设备成为“神奇的”文本感应 – 地点,使用,位置 HTML5 Web Apps that can be discovered on the Net and run on any platform 网页应用可以在网页上开发并在任 何平台上运行 © Copyright Khronos Group 2013 | Page 22
  23. 23. OpenMAX - Media Acceleration 媒体加速 •  Family of royalty-free, cross-platform open API standards for video, image stream and camera processing •  免版税跨平台开放API标准家族,用于视频、图像流和相机处理 Applications 应用程序 “Application Layer” Portable access to native multimedia hardware acceleration 可移植的接入本地多媒体硬 件加速 Portable and flexible access to media components (codecs) 可移植的灵活的接入媒体组 件(编解码器) Native Multimedia Framework “应用程序层” 本地多媒体框架 Sophisticated apps that provide their own media framework processing can interface to codecs directly 为其提供媒体框架处理 的复杂的应用程序处理可 以直接和编解码器对接 “Integration Layer” Media component interfaces “整合层” 媒体组件界面 Audio Codecs e.g. MP3 Video Codecs e.g. 264 Image Codecs e.g. JPEG OpenMAX does not define CODECs – but provides codec interfaces OpenMAX不定义CODEC, 但是提供编解码器界面 © Copyright Khronos Group 2013 | Page 23
  24. 24. OpenMAX AL Streaming Media Framework 流媒体框架 •  Enables key video, image stream and camera use cases -  Enables optimal hardware acceleration with app portability •  Create Media Objects to play and process images and video with AV sync -  Connect to variety of input and output objects to PLAY and RECORD media •  Full range of video effects and controls -  Including playback rate, post processing, and image manipulation Sources OpenMAX AL includes sophisticated camera controls OpenMAX AL包 括复杂的照相机 控制 •  实现关键视频,图片流和相机使用案例 -  通过应用程序可移植性实现可选硬 件加速 •  实现媒体目标用于播放和图片处理以及 视频与AV同步 -  连接各种输入输出目标到播放和记 录媒体 •  完整的视频效果和控制 -  包括回放速率、后处理、图片管理 Efficient data routing for CPU processing 为CPU处理的高效数据 路由 Sinks EGLStream to GPU Data Tap to CPU Analog Radio Audio Mix Camera Audio Input URI Memory DSrc OpenMAX AL Media Object Processing DSnk Display Window URI EGLStreams extension enables efficient transfer of image stream to GPU texture memory EGLStreams扩展版实 现了高效图片流传输到 GPU纹理内存 Memory © Copyright Khronos Group 2013 | Page 24
  25. 25. OpenSL ES – Advanced Audio 高级音频 •  OpenSL ES does for audio what OpenGL ES does for graphics -  Advanced audio functionality from simple playback to full 3D positional audio •  OpenSL ES 对音频的作用就像OpenGL ES对图形图 像的作用 -  从简单回放到完整的3D定位音频的各种高级音 频功能 •  Object-based native audio API for simplicity and high performance -  Same object framework as OpenMAX AL -  Reduces development time •  目标为基础的本地音频API实现简化和高性能 -  像OpenMAX AL的相同目标框架 -  减少开发时间 •  Attractive alternative to open source frameworks -  Tightly defined specification with full conformance tests -  Robust application portability across platforms and OS •  有吸引力的交互的开源框架 -  严格定义完整的符合性测试 -  高效的应用程序可移植性可以跨平台和跨操作 系统 © Copyright Khronos Group 2013 | Page 25
  26. 26. EGLStream – Streaming Images 流图片 •  EGL – originally embedded version of WGL -  Abstraction layer to window systems and memory buffers •  Role has expanded to provide API interop – data and events -  EGLImages – single buffers that can be passed between APIs -  EGLStreams – provides stream of images – with AV sync -  Cross process EGLStreams – Producer and Consumer can be in different processes for performance or security – e.g. browser compositing process •  Android SurfaceTexture is a Java wrapper around EGLStreams -  Captures video decode or camera preview to OpenGL ES texture •  EGL – 来源于嵌入式版本的WGL -  窗口系统和内存缓冲的缩写层 •  功能扩展为提供API交互操作-数据和事件 -  EGLImages – 单独的缓冲可以在API间传递 -  EGLStreams – 提供图片流 – 通过AV同步 -  跨流程EGLStreams – 制造商和消费者可以在不同的功能或安全流程中 – 如浏览器编写流程 •  安卓SurfaceTexture是一个在EGLStreams周边的Java包装 -  捕捉视频解码或相机预览到OpenGL ES纹理 © Copyright Khronos Group 2013 | Page 26
  27. 27. EGLStream – Streaming Images 流图片 OpenMAX AL Media Player is the EGLStream “Producer” and controls production of frames. OpenMAX AL媒体播放器是 EGLStream“生产者”和产品框架控制 Camera File URL Etc. 相机文件 MEDIA PLAYER Object 媒体播放器目标 EGLStreams enables and hides details of video frame transport. Enables multiple buffering modes for different uses cases e.g.: FIFO and explicit latch/release EGLStreams实现并隐藏了视频框架传输细节。 实现了不用使用情况的多缓冲模式,如FIFO和 显式锁定/释放 EGLStream ? OpenGL ES GL_TEXTURE_EXTERNAL is the EGLStream “Consumer” and converts video format into RGB OpenGL ES texture OpenGL ES GL_TEXTURE_EXTERNAL 是EGLStream“消费者”,并将视频格 式转换成RGB OpenGL ES纹理 GL_TEXTURE_EXTERNAL © Copyright Khronos Group 2013 | Page 27
  28. 28. Portable Access to Sensor Fusion 传感融合的便携式获取 Apps request semantic sensor information Spec expected in 2013 预计2013年推出规范 StreamInput defines possible requests, e.g. “Provide Skeleton Position” “Am I in an elevator?” 应用程序需要语义的传感信息 StreamInput定义了可能的需求,如“提供骨骼位置”,“我是 ⼀一个电梯吗?” Standardized Node Intercommunication Advanced Sensors Everywhere RGB and depth cameras, multi-axis motion/position, touch and gestures, microphones, wireless controllers, haptics keyboards, mice, track pads 高级传感无处不在 RGB和深度相机,多轴动作/位置,触摸和 姿势,麦克风,无线遥控,触觉键盘, mice,track pad Universal Timestamps Input Device Input Device Filter Node Filter Node Filter Node Input Device App Processing graph provides sensor data stream Apps Need Sophisticated Access to Sensor Data Without coding to specific sensor hardware 应用程序需要复杂的传感数据接入 无需编码到具体的传感硬件 Utilizes optimized, smart, sensor middleware Apps can gain ‘magical’ situational awareness 处理图形提供传感数据流 使用优化的,只能传感中间件 应用程序可以拥有“神奇的”感应能力 © Copyright Khronos Group 2013 | Page 28
  29. 29. OpenVX •  Vision Hardware Acceleration Layer -  Enables hardware vendors to implement accelerated imaging and vision algorithms -  For use by high-level libraries or apps •  Focus on enabling real-time vision -  On mobile and embedded systems Application 应用 OpenCV open source library 开源库 Other higher-level CV libraries 其他更高水平CV库 •  Diversity of efficient implementations -  From programmable processors, through GPUS to dedicated hardware pipelines OpenVX does not duplicate OpenCV functionality – JUST provides essential acceleration OpenVX不是 制OpenCV的功能 – 只是提供必要的加速 •  视觉硬件加速层 -  使硬件供应商可以执行加速成像和视觉算法 -  用于高水平库和应用程序 •  关注实现实时视觉 -  在移动和嵌入式系统上 Open source sample implementation? 开源实例执行? Hardware vendor implementations 硬件供应商执行 •  高效执行的多样化 -  从可编程处理器,通过GPUS到专门的硬件管线 Dedicated hardware can help make vision processing performant and low-power enough for pervasive ‘always-on’ use 专 的硬件可以帮助是视觉处理实现性能,同时低能耗足 满足长期的持续使用 © Copyright Khronos Group 2013 | Page 29
  30. 30. HTML5 – Cross OS App Platform 跨操作系统应用平台 •  Increasing diversity of devices creates a demand for a true cross OS programming platform •  BUT need more than “more HTML” •  不断提高的设备多样性创造了对真正跨操作系 统编程平台的需求 •  但是我们所需要的“不仅仅是HTML” © Copyright Khronos Group 2013 | Page 30
  31. 31. HTML5 – Cross OS App Platform 跨操作系统应用平台 Traditional Web-content 传统网页内容 Rich Experiential Processing Multi-core CPUs Rich 2D and 3D GPU GPU Computing Multiple HD cameras Image and vision processing Video encode/decode Audio encode/decode Inertial and positional sensors 丰富的体验处理 多核CPU 丰富的2D和3D GPU GPU 计算 多个高清相机 图片和视觉处理 视频编码/解码 音频编码/解码 内部和位置传感 How can the Browser rapidly assimilate such diverse functionality? 浏览器如何可以迅速地吸收这些多样化的功能? © Copyright Khronos Group 2013 | Page 31
  32. 32. Leveraging Proven Native APIs into HTML5 本地API到HTML5上的过渡 •  Leverage native API investments into the Web -  Faster API development and deployment -  Familiar foundation reduces developer learning curve •  Will this enable web based AR apps? -  Access to necessary hardware acceleration is there! •  本地API到网页上的投资 -  比API开发和执行的更快 -  对基础的熟悉降低了开发者的学习曲线 •  这将实现网页基础的增强现实应用程序吗? -  必要的硬件加速已经推出! Canvas WebMAX? Camera control and video processing 相机控制 视频处理 WebSL? Easy to use JavaScript Audio 使用简便 WebVX? Vision Processing 视觉处理 Device and Sensor APIs Device Orientation Working Groups 设备和传感API 设备主导工作组 JavaScript Native Native APIs shipping or 本地API已推出或 working group underway 工作组正在进行中 JavaScript API已推出 JavaScript API shipping or working group underway 或工作组正在进行中 Possible future 未来可能的 JavaScript APIs JavaScript API © Copyright Khronos Group 2013 | Page 32
  33. 33. WebGL – 3D on the Web – No Plug-in! 网页上的3D – 无插件! •  Historic opportunity to bring accelerated 3D graphics to web -  WebGL defines JavaScript binding to OpenGL ES 2.0 •  历史性的机遇:将3D图形图像带入网页 -  WebGL定义了JavaScript绑定到OpenGL ES 2.0 •  Leveraging HTML 5 and uses <canvas> element -  Enables a 3D context for the canvas •  衡量HTML 5 并使用 <canvas> 元素 -  为canvas实现3D上下文 •  WebGL 1.0 Released at GDC March 2011 -  Mozilla, Apple, Google and Opera working closely with GPU vendors •  2011年3月在GDC上发布了WebGL 1.0 -  Mozilla, Apple, Google 和 Opera与GPU供应 商紧密合作 几乎在每个网页设 备上都可以获得的 OpenGL和 OpenGL ES Availability of OpenGL and OpenGL ES on almost every webcapable device JavaScript binding to OpenGL ES 2.0 Increasing JavaScript performance. HTML 5 Canvas Tag 不断提高的 JavaScript性能 © Copyright Khronos Group 2013 | Page 33
  34. 34. WebCL – Parallel Computing for the Web 网页并行计算 •  JavaScript bindings to OpenCL APIs -  Enables initiation of Kernels written in OpenCL C within the browser •  Bindings stay close to the OpenCL standard -  Maximum flexibility to provide a foundation for higher-level middleware -  Minimal language modifications for 100% security and app portability -  E.g. Mapping of CL memory objects into host memory space is not supported •  API definition underway – public draft released -  https://cvs.khronos.org/svn/repos /registry/trunk/public/webcl/spec/latest /index.html •  Compelling use cases -  Physics engines for WebGL games -  Image and video editing in browser •  JavaScript 绑定OpenCL API -  实现在浏览器上用OpenCL C编写核 •  绑定与OpenCL标准保持紧密联系 -  最大化灵活度为高水平中间件提供基础 -  最小化语言修改以实现100%安全和应用可移植性 -  如CL内存目标到主机内存空间mapping不支持 •  API正在定义中 – 公开草本发布 -  https://cvs.khronos.org/svn/repos/registry/trunk /public/webcl/spec/latest/index.html •  使用案例正在完成 -  WebGL游戏屋里引擎 -  浏览器上图片和视频编辑 River Trail JavaScript binding to OpenCL APIs to enable initiation of OpenCL C kernels Data-parallel language extensions to JavaScript for offload at run-time © Copyright Khronos Group 2013 | Page 34
  35. 35. API Adoption 采用 Mobile Operating Systems 手机操作系统 OpenGL ES 2.0 Shipping - Android 2.2 OpenSL ES 1.0 Shipping – Android 2.3 OpenMAX AL 1.0 Shipping - Android 4.0 EGL 1.4 Shipping under SDK -> NDK Chrome on ICS will have WebGL. Opera and Firefox WebGL now OpenGL 3.2 on MacOS OpenCL 1.1 on MacOS OpenGL ES 2.0 on iOS Can enable on MacOS Safari iOS5 enables WebGL for iAds Microsoft WinRT/Metro: -  Only Microsoft native APIs -  HTML5 but not yet WebGL -  只有微软本地API -  HTML5,但还没有WebGL © Copyright Khronos Group 2013 | Page 35
  36. 36. Extended Native APIs on Android 安卓上扩展的本地API •  Native APIs can be shipped as NDK extensions before Google Adoption -  Do not break/change existing Google APIs •  Khronos APIs have strong momentum in the silicon community -  Evangelize Google to adopt into standard platform •  Extended APIs can be used by: -  Bundled apps, Market apps with API selection -  Multiple APKs behind single multi -APK SKU •  在Google采用前,本地API可以作为NDK扩展进行 使用 -  不要破坏/修改已有Google的API •  Khronos API拥有强大的芯片行业动力 -  传道Google采用到其标准平台 •  扩展API可以使用在: -  捆绑附带应用程序,带有API选项的市场应用 程序 -  在单一多个APK SKU背后的多个APK Evangelize Google and ISVs to adopt enabling APIs Standard Android SDK and NDK Extended NDK for Key use cases E.g. video, camera, imaging, sensor and composition APIs © Copyright Khronos Group 2013 | Page 36
  37. 37. In Summary 总结 •  APIs are key to enable compelling applications on advanced hardware – APIs developed on high-end hardware are now enabling mobile devices •  APIs no longer exist alone – they interoperate and provide input AND output processing to form a complete platform for advanced content •  Significant cooperation underway between native and Web APIs to bring advanced visual computing to HTML5 •  Khronos is driving open standards for hardware acceleration Participate, change the industry AND get the inside edge for your products! •  API是在高级硬件上实现具有竞争力的应用的关键 – 在高端硬件上开发的API现在已经在移动设备 上实现 •  API不再单独存在了 – 其交互操作并提供输入和输出处理,以为高级内容搭建一个完整平台 •  本地和网页API直接的重大合作正在进行中,这将为HTML5带来高级视觉计算 •  Khronos带动着硬件加速开放标准,参与并改变整个行业,让你的产品真正进入到尖端市场吧! Connecting Software to Silicon © Copyright Khronos Group 2013 | Page 37

×