Download to read offline
![© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Video Texture – How to use?
Confirm if VideoTexture is supported by Context3D class
7
if( stage.stage3Ds.length > 0 )
{
var stage3D:Stage3D = stage.stage3Ds[0];
stage3D.addEventListener( Event.CONTEXT3D_CREATE, Context3DHandler );
stage3D.requestContext3D( );
}
function Context3DHandler ( event : Event ) : void
{
if (Context3D.supportsVideoTexture)
{
// We can create VideoTexture Object with Context3D object.
}
}](https://image.slidesharecdn.com/webinarpptcombined2-150601181332-lva1-app6891/85/Adobe-Session-on-Flash-Online-Conference-12-7-320.jpg)
Uploaded byFlash Conference
Adobe Session on Flash Online Conference #12
The document discusses the implementation of video textures in Stage3D, providing a new approach that allows direct access to textures from a netstream or camera, enhancing performance by eliminating the need for multiple bitmap copies. It outlines the steps for creating and using video textures in applications, including support verification and event handling. Additionally, the document addresses texture compression strategies, particularly the advantages of using ETC2 for better performance and reduced memory usage across platforms.
More Related Content
Similar to Adobe Session on Flash Online Conference #12
Adobe Session on Flash Online Conference #12
- 1. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Video and Camera Support for Stage3D using Video Texture Nimisha/Chandra Prakash | AIR Engineering
- 2. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Overview 2 How Stage3D fits within the Flash display model? Problem at hand: using video as a content in 3D scene Current method to fix the problem at hand New approach to solve the problem How to use the new approach? Use cases
- 3. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Stage3D: The stage behind the Stage 3 Set of special Stages that are specifically dedicated to 3D
- 4. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Demo 4
- 5. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. How to fix : Bitmap representation of video 5 Problems Requires multiple copying of bitmap Slow and Resource Intensive Video rendered in RGB on a Video ObjectVideo rendered in RGB on a Video Object Bitmap Representation of VideoBitmap Representation of Video Create a TextureCreate a Texture Update the texture with the bitmapUpdate the texture with the bitmap
- 6. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. New Approach - Video Texture 6 Directly access texture from netstream or cameraDirectly access texture from netstream or camera Advantages Source texture in stage 3D pipeline Fast and Hardware accelerated
- 7. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Video Texture – How to use? Confirm if VideoTexture is supported by Context3D class 7 if( stage.stage3Ds.length > 0 ) { var stage3D:Stage3D = stage.stage3Ds[0]; stage3D.addEventListener( Event.CONTEXT3D_CREATE, Context3DHandler ); stage3D.requestContext3D( ); } function Context3DHandler ( event : Event ) : void { if (Context3D.supportsVideoTexture) { // We can create VideoTexture Object with Context3D object. } }
- 8. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Video Texture – How to use? Create VideoTexture object and attach NetStream/Camera object to created VideoTexture object 8 var ns:NetStream; var context3D:Context3D; var texture:VideoTexture; texture = context3D.createVideoTexture(); texture.attachNetstream(ns); texture.addEventListener(Event.TEXTURE_READY, renderFrame); ns.play("http://www.adobe.com/AIR/videoTexture/demo.mp4");
- 9. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Demo 9
- 10. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Video Texture – How to use? Video frame corresponding to current audio playback Callback for an event Event.TEXTURE_READY is fired Use an updated VideoTexture object with currently available video frame 10 var context3D:Context3D; function renderFrame(e:Event):void { // Render on stage3D with VideoTexture }
- 11. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Video Texture – How to use? VideoTextureEvent object that works same with the StageVideoEvent event 11 texture.addEventListener(VideoTextureEvent.RENDER_STATE, onRenderState); function onRenderState (event:VideoTextureEvent) { if ( event.status == VideoStatus.SOFTWARE) { // Indicates software video decoding works. } if ( event.status == VideoStatus.ACCELERATED ) { // Indicates hardware-accelerated (GPU) video decoding works. } if ( event.status == VideoStatus.UNAVAILABLE ) { // Indicates Video decoding is not supported. } }
- 12. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Video Texture – Use Cases 12 Camera Input In-game video call with shader effects Efficient embedding of camera streams Video Ads In place Video Ads In game video streams containing streamed content or live content or commercials Credit: Waste Invaders
- 13. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Roadmap Implementation of video texture on iOS We have already signed off the feature for iOS and the developer community is using it Implementation of video texture on Android Feature is ready for release and is available in labs for developers to try out video texture on Android The feature would most probably be usable in a full fledged way in our next release 13
- 14. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Questions 14
- 15. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. ETC2 Texture Compression (Ericsson Texture Compression) Ridam Batra | AIR Engineering
- 16. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. GPU TEXTURE COMPRESSION 16 CPU GPU Upload of textures from CPU to GPU is a very slow operation GPU can understand block compressed texture formats like etc1,pvrtc,etc2,dxt etc. Compressed Textures Texture Compression is the ANSWER.
- 17. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. ADOBE TEXTURE FORMAT 17 • Container format for compressed images. • AIR or Flash Player automatically extracts the appropriate texture depending on platform.
- 18. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Problem 18 ETC1 (android) Size on CPU – 170KB Size after GPU upload - 170KB PVRTC (ios) Size on CPU – 170KB Size after GPU upload - 170 KB Original png image Size on CPU – 256KB Size after GPU upload - 1048KB DIFFERENT RENDERING RESULT ON IOS AND ANDROID !!!!
- 19. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. SOLUTION OpenGL ES 3.0 introduced high quality ETC2 compression as a standard feature. It works on ios and android both. 19 Original png image 1048 KB ATF with ETC1 + PVRTC 340 KB ATF with ETC2 170 KB SIMILAR RENDERING RESULT ON IOS AND ANDROIDSIMILAR RENDERING RESULT ON IOS AND ANDROID SIZE OF TEXTURES FURTHER REDUCED SIZE OF TEXTURES FURTHER REDUCED ANDAND
- 20. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Demo 20
- 21. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Comparing The Two Applications 21 Size of APK with PNG files - 37.7 MB Size of APK with ATF Files - 21.2 MB App With PNG Images GPU Memory for textures– 73,738 KB App With Atf Files GPU Memory for textures– 9,216 KB GPU MEMORY
- 22. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Questions 22
- 23. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. 64-bit Support & Parallel Compilation for iOS Abhinav Dhandh | AIR Engineering
- 24. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. 64-bit support for iOS applications Work on supporting 64-bit binary feature for iOS applications Support for newer A7+ processor devices: Additionally, Apple issued new guidelines for app submission iPhone 5s iPhone 6 iPhone 6+ iPad Air Mini iPad Air iPad Air 2
- 25. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Universal binary was the way.. SWFs ABC Files 32-bit Compilation and Linking Executable File IPA Combine ADT Additionally, stability of New compiler has been improved Major bug fixes for real world applications 64-bit Compilation and Linking
- 26. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. But.. Average Packaging Time Increase : 93%
- 27. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Parallel Compilation for packaging time reduction: 27 SWFs ABC Files ABC Files 32-bit Compilation and Linking Executable File IPA Combine ADT Multi-threaded Compilation Executable File Combine 32-bit Compilation and Linking 32-bit Compilation and Linking 64-bit Compilation and Linking 64-bit Compilation and Linking 64-bit Compilation and Linking
- 28. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Packaging time comparison Average reduction in packaging time : 44%
- 29. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Questions 29
- 30. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Standard Extended Profile Mayank Bhagya | AIR Engineering
- 31. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Features Enhanced register set Shader size 31 AGAL v2 AGAL v3 Vertex Attributes 8 16 Fragment Constants 64 200 AGAL v2 AGAL v3 Tokens 1024 2048
- 32. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. A sample shader 32
- 33. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Usecases 33 Complex shaders © Jojo Nicado Water © Dom Crossley Fog
- 34. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Demo 34
- 35. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Adobe Graphics Assembly Language - v3 Standard Extended Profile Mobile – AIR v17 Desktop – FP & AIR v18 Mini Assembler @ GitHub https://github.com/adobe-flash/graphicscorelib 35
- 36. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential. Questions 36
- 37. © 2015 AdobeSystems Incorporated. All Rights Reserved. Adobe Confidential.
Editor's Notes
- #6 Font Grey color Using video object
- #7 Using a VideoTexture object, all this work can be optimized internally by the GPU. As a result, the decode, conversion from YUV to RGB and texture loading can be completely moved to the GPU. Thus this feature will allow efficient use of Video textures with Stage3D. Again, with the use of this feature the stage3D content need not be removed and therefore it solves the problem of video transitions in games. One can use remote or Local camera in game (ie chatting with game mates), or applying shader effects to a video camera stream. This feature will allow efficient embedding of camera streams within Stage3D content. Apart from this, in game video streams: one can use video streams inside the content (for example inside a monitor in the scene) and can contain either local files, streamed content or live content, commercials etc. You can also apply special effects to video and still get full HW acceleration, even if it not as efficient as StageVideo, it will still be GPU accelerated. Now, let me show you a demo of an app running a video as a texture in a Stage 3D environment.
- #8 Now, let’s take a look at the concepts and definition of this feature. Before using VideoTexture object, we should confirm if Context3D class supports VideoTexture feature on current platform and product by checking a static property "supportsVideoTexture" of Context3D class.
- #9 In the next step, we create VideoTexture object and attach a NetStream/Camera object to the object. In action script, user can get a current video frame as a texture from the VideoTexture Object when a callback for an event Event.TEXTURE_READY is fired. Event.TEXTURE_READY is fired whenever there is an available video frame corresponding to current audio playback. Depending on audio playback time, frames past than current audio playback time would be dropped as normal video playback.
- #12 In addition, this feature provides VideoTextureEvent object that works exactly same with the StageVideoEvent. With this event, you can get to know status changes of hardware resources (hardware resources are prone to status changes or failures) Know status changes of hardware resources (hardware resources are prone to status changes or failures)
- #13 Using a VideoTexture object, all this work can be optimized internally by the GPU. As a result, the decode, conversion from YUV to RGB and texture loading can be completely moved to the GPU. Thus this feature will allow efficient use of Video textures with Stage3D. Again, with the use of this feature the stage3D content need not be removed and therefore it solves the problem of video transitions in games. One can use remote or Local camera in game (ie chatting with game mates), or applying shader effects to a video camera stream. This feature will allow efficient embedding of camera streams within Stage3D content. Apart from this, in game video streams: one can use video streams inside the content (for example inside a monitor in the scene) and can contain either local files, streamed content or live content, commercials etc. You can also apply special effects to video and still get full HW acceleration, even if it not as efficient as StageVideo, it will still be GPU accelerated. Now, let me show you a demo of an app running a video as a texture in a Stage 3D environment.
- #31 AGAL is the language for writing shaders. First version came along with Stage3D. V2 introduced features like MRT, differentials, half-floats Now V3.
- #32 Enhanced register set. Increased shader size
- #33 Problem People trying to do advanced stuff with agal fall short of registers. Gaming engine: Tangents, fog, binormals. Now with agal3, double per-vertex data. Hope you create more awesome stuff!
- #34 Problem People trying to do advanced stuff with agal fall short of registers. Gaming engine: Tangents, fog, binormals. Now with agal3, double per-vertex data. Hope you create more awesome stuff!
- #36 Work on devices that support the std extended profile On mobile in AIRv17 & on desktop in BETA v18. beta on labs and release 2 weeks time. MiniAssembler for compiling the new shaders on github.