Delivering the Best Flash HD Video by R Blank from Adobe MAX 2011

Delivering the Best Flash HD Video
R Blank
CTO

©2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Conﬁdential.

Please Fill Out the
Scorecard


Today...

Getting Outstanding HD
to Every Device w/Flash


Current Flash Platform Device Support


Stage Video Enhances
Video on Many Screens


In This Session...


In This Session...

1. StageVideo
■
What it is, how it Works, and its Limitations


In This Session...

1. StageVideo
■
2. Video
■
Optimizing Encoding for StageVideo


In This Session...

1. StageVideo
■
2. Video
■
3. OSMF & Strobe
■
Options to use StageVideo with no Code


In This Session...

1. StageVideo
■
2. Video
■
3. OSMF & Strobe
■
Options to use StageVideo with no Code
4. AS3 APIs
■
The ActionScript to Make StageVideo Work


Stage Video


Stage Video

■
Stage Video (FP 10.2+) is Just Better


Stage Video

■
■
Higher-Res


Stage Video

■
■
Higher-Res
■
Higher-Framerate


Stage Video

■
■
Higher-Res
■
Higher-Framerate
■
HD Video


Stage Video

■
■
Higher-Res
■
Higher-Framerate
■
HD Video
■
In Flash


Stage Video

■
■
Higher-Res
■
Higher-Framerate
■
HD Video
■
In Flash
■
Up to 85% Performance Savings!


Performance Demo


How?


How?

Overhead Savings


How?

Overhead Savings
& Hardware Acceleration


A Slimmer Video Player


Hardware Acceleration Means...


Tapping into the Power of
the Viewer’s Device


Tapping into the Power of
the Viewer’s Device
To Accelerate Playback

Hardware Acceleration Support in Flash Player

Player Support Added

9.0.115 H.264 and Full-Screen Hardware Acceleration

10 Added Hardware Acceleration in Normal View

10.1 Improved GPU acceleration

10.2 Stage Video

11 Stage 3D


Hardware Acceleration with Video


Hardware Acceleration with StageVideo


Hardware Acceleration Comparison

With Video With StageVideo

The decoder renders the video and Flash
Flash renders the video and all graphics
draws on top of it


Getting started with stage video | Adobe Developer Connection 3/14/11 12:56 PM

by making immersive HD video experiences on the web possible. Recent implementations of Flash Player

With Video
on mobile devices have brought new ideas to the Flash Player team at Adobe for how to display video in
Flash and always improve the user experience. Stage video is the result of those efforts. With StageVideo
The traditional method for rendering video in Flash Player uses the Video object. The Video object is
Getting started with stage video | Adobe Developer Connection 3/14/11 12:56 PM
treated the same as any other object on the stage, which gives developers an unprecedented amount of
creative control. For example, video can be displayed on each face of a spinning cube, or multiple videos Player display list but sits behind the stage instead. Figure 2 illustrates this design.
can be blended together with one another. The Video object is treated as any other DisplayObject. Figure
1 illustrates this idea.

Figure 2. The StageVideo object sits behind the Flash stage.
Figure 1. Multiple videos can be blended together using the Video object.
The performance benefits of stage video are especially pronounced for televisions, set-top boxes, and
To support that level of creative control, Flash Player must do a significant amount of processing for each mobile devices. These devices do not have CPUs as powerful as desktop computers, but they do have very

Flash renders the video and all
video frame. Depending on the power of the underlying device, this increased processing may decrease
the frame rate of the video, or it may increase the load that Flash Player places on the CPU.
The decoder renders the video
powerful video decoders capable of rendering high-quality video content with very little CPU usage.
However, even on the desktop, stage video will dramatically change video performance in Flash Player.

graphics
A new way to render video and Flash draws on top of it
As a developer, you must understand that stage video is the second step of enhancements related to video
GPU acceleration in Flash Player. The first step is encoding the video so as to take best advantage of the
hardware acceleration available on the target platform. Hence, to get the best experience possible, you
To mitigate the performance impact of rendering video in the Video object, Adobe has introduced stage will need to have the two steps enabled. The H.264 codec is stage video's best friend; using this will ensure
video as a new way to render video. This approach takes full advantage of the underlying video hardware. you get full GPU acceleration from video decoding to rendering. With this approach, no read-back
(sending the data from the GPU to the CPU) is required to composite the video frames in the display list
The result is a much lower load on the CPU, which translates into higher frame rates on less powerful
anymore. The YUV 4:2:0 formatted video frames are converted to RGB through a GPU shader (DirectX9 or
devices and also less memory usage. With stage video, the StageVideo object does not sit inside the Flash OpenGL) and blitted onscreen. As a result, you will see higher pixel fidelity and some reduction in CPU and
memory usage.
http://www.adobe.com/devnet/flashplayer/articles/stage_video.html Page 3 of 18

http://www.adobe.com/devnet/flashplayer/articles/stage_video.html Page 4 of 18


Stage Video Limitations



■
You Can’t...



■
You Can’t...
■
Have more than 8 concurrent videos on desktop



■
You Can’t...
■
■
Or more than 1 on mobile devices



■
You Can’t...
■
■
■
Freely rotate the video



■
You Can’t...
■
■
■
■
Can only rotate @ 0, 90, 180, 270 degrees



■
You Can’t...
■
■
■
■
■
Apply 3D, color or skewing transformations



■
You Can’t...
■
■
■
■
■
■
Apply alpha channel, blendMode, ﬁlter, mask, or scale9Grid
transformations



■
You Can’t...
■
■
■
■
■
■
transformations
■
Copy the video into BitmapData or Bitmap-cache the video



■
You Can’t...
■
■
■
■
■
■
transformations
■
■
Embed the video in an FLA/XFL (only NetStream playback)



■
You Can’t...
■
■
■
■
■
■
transformations
■
■
Embed the video in an FLA/XFL (only NetStream playback)
■
Remember: Stage Video is not always available

For Best Results

■
Maximize Cross-Browser & Device Consistency in the
Flash Embed Code
■
By Setting WMODE to
■
“gpu” (FP 10.2 or 10.3)
■
“direct” (FP 10.1, 10.2 or 10.3)


WMODE

<param name="wmode" value="direct" />


WMODE Settings

Setting Meaning

window Render SWF in its own window

opaque Render SWF opaquely, above other browser content

transparent Render SWF transparently, below other browser content

direct Use the fastest path to screen to render
Invoke hardware acceleration from the graphics card to
gpu render
Default since 10.1


And so...

■
You Shouldn’t...
■
Use Stage Video with transparent SWFs


Optimizing Encoding
for Stage Video


CODEC

Code that
COmpresses &
DECompresses Video

Codecs and Containers



■
Video is Compressed with a Codec



■
■
Then Stored in a File with a Format (or Container)



■
■
■
So the Same Type of Container Can Have Different
Codecs, E.g.:



■
■
■
Codecs, E.g.:
■
a .MOV May be Encoded with H.264 Codec or Animation Codec
or Others



■
■
■
Codecs, E.g.:
■
or Others
■
an .FLV May be Encoded with Spark Codec or VP6 Codec



■
■
■
Codecs, E.g.:
■
or Others
■
■
H.264 is Used in Many Formats, Including:



■
■
■
Codecs, E.g.:
■
or Others
■
■
■
.mp4 | .f4v | .m4v | .m4a | .mov | .3gp



■
■
■
Codecs, E.g.:
■
or Others
■
■
■
.mp4 | .f4v | .m4v | .m4a | .mov | .3gp
■
But, Not All Files of Those Types Use H.264

History of Video Support in Flash

Ver Codec Notes
4/5 SWF Motion JPG Variant from Wildform Flix
Sorensen Spark
6 Streaming only; MetaData & CuePoints
FLV
7 Added Progressive FLV Video

8 On2 VP6 FLV Improved Quality; Transparency; HD

H.264 (all Industry-standard codec; Full-Screen
9.0.115
filetypes) Hardware Acceleration

New format (F4V); HA out of Full-Screen; OSMF &
10
Strobe Introduced

10.2 Stage Video on Desktop & Mobile with mode=”direct”

H.264


H.264...


H.264...

■
Looks Good at Many Bitrates


H.264...

■
■
Encodes Relatively Quickly


H.264...

■
■
■
Is Not Owned/Controlled by Any One Company
■
http://www.mpegla.com


H.264...

■
■
■
■
■
Is Already Used by Many Firms in Their Products
■
e.g. Apple, Sony, Nokia, SanDisk, Blackberry and Microsoft


H.264...

■
■
■
■
■
Is Already Used by Many Firms in Their Products
■
e.g. Apple, Sony, Nokia, SanDisk, Blackberry and Microsoft
■
Is Pre-Installed on Tons of Devices
■
e.g., Blu-ray, PS3, XBox, iPhone, and all Video Mobile Devices


Encode With H.264 to
Deliver the Best Flash
Video

Video Rendering Paths

If you are using... Then Flash will use...
Object Codec Decoding Compositing
Video Non-Accelerated CPU CPU
Video Accelerated (H.264) GPU CPU
StageVideo Non-Accelerated CPU GPU
StageVideo Accelerated (H.264) GPU GPU
The “Direct Path” with best performance


Checking Your Codec


Checking Your Codec

■
on Mac with QuickTime ■
on PC with G-Spot
■
Load Video into Quicktime ■
Load any video; check Codec
Player ■
http://gspot.headbands.com
■
Load Info Window (Window>
Show Movie
Inspector)


Creating H.264 with Adobe Media Encoder

■
On the Main Screen, Choose ■
Or, in the Export Settings,
any F4V Preset (not FLV) click the “Format” tab, and
Select “F4V” (not FLV)


Creating H.264 with FFMPEGx

■
FFMPEG is a Multi-
Platform, Open-Source
Encoder
■
Get FFMPEGx for Mac
http://homepage.mac.com/
major4/
■
Select a Video Setting
That Has ‘H264’ in the
Name


Open Source Media
Framework
http://osmf.org

Delivery Platform


Creation Tools

Delivery Platform


Content

Creation
Tools

Delivery
Platform

Content

Code

Creation
Tools

Delivery
Platform

Content

Code

Application Framework

Creation
Tools

Delivery
Platform

Content

Code

Application Framework Media Framework

Creation
Tools

Delivery
Platform

OSMF Makes it Easy To...



■ Load,



■ Load,
■ Play



■ Load,
■ Play

■ & Display



■ Load,
■ Play

■ & Display

■ Media


Supported Flash Media Types



■ Streaming Audio
■
mp3, AAC, Speex, and
Nellymoser



■ Streaming Audio
■
Nellymoser
■ Streaming Video
■
FLV, F4V, MP4, MPEG-4: MP4,
M4V, F4V, 3GPP



■ Streaming Audio
■
Nellymoser
■ Streaming Video
■
M4V, F4V, 3GPP
■ Progressive Audio
■
mp3



■ Streaming Audio ■ Progressive Video
■
mp3, AAC, Speex, and ■
FLV, F4V, MP4, MP4V-ES, M4V,
Nellymoser 3GPP, 3GPP2, QuickTime
■ Streaming Video
■
M4V, F4V, 3GPP
■
mp3



■
■ Streaming Video ■ Images
■
FLV, F4V, MP4, MPEG-4: MP4, ■
PNG, GIF, or JPG
M4V, F4V, 3GPP
■
mp3



■
■ Streaming Video ■ Images
■
FLV, F4V, MP4, MPEG-4: MP4, ■
PNG, GIF, or JPG
M4V, F4V, 3GPP
■ SWF ﬁles
■
mp3


OSMF Optimizes
Video Delivery
On Its Own

Disabling Stage Video in OSMF



■
OSMFSettings.enableStageVideo = false;



■
OSMFSettings.enableStageVideo = false;

■
OSMFSettings.enableStageVideo = true;


Start Using OSMF


Start Using OSMF

■
Go to osmf.org & Download OSMF.swc


Start Using OSMF

■
Go to osmf.org & Download OSMF.swc
■
Copy OSMF.swc to
■
For Flash Pro
■
In Windows:
■
Program FilesAdobeAdobe Flash CS5CommonConfigurationActionScript 3.0libs
■
On Mac:
■
/Applications/Adobe Flash CS5/Common/Configuration/ActionScript 3.0/libs
■
For Flash Builder
■
place in project ‘libs’


OSMF is Only Code
No Design or GUI


OSMF is Only Code
No Design or GUI
What if I Need an Actual
Video Player?

Strobe Media Playback

http://osmf.org/strobe_mediaplayback.html

Start Using Strobe Media Playback

■
Download Strobe
■
Run setup.html


That’s Still Too Much


Flash Media Playback

■
Adobe-hosted Version ■
Use FMP Configurator
of Strobe ■
http://osmf.org/configurator/fmp/
■
http://adobe.com/products/
flashmediaplayback/


Stage Video in SMP & FMP

SMP & FMP Exploit Stage
Video Automatically


AS3 API


StageVideo Replaces
the Video Class


myVideo.attachNetStream ( ns ) ;


myVideo.attachNetStream ( ns ) ;

myStageVideo.attachNetStream ( ns ) ;


StageVideo Does NOT
Behave the Same as Video


StageVideo Objects

You Don’t Create
StageVideo Objects


StageVideo Objects

You Don’t Create
StageVideo Objects
FP Creates Them for You


StageVideo Objects

You Don’t Create
StageVideo Objects
FP Creates Them for You
If it Can


StageVideo Objects

Number of StageVideo
Objects Varies


StageVideo Objects

Objects Varies
Upto 8 on Desktop


StageVideo Objects

Objects Varies
Upto 8 on Desktop
1 on Mobile


StageVideo Objects

Objects Varies
Upto 8 on Desktop
1 on Mobile
0 if Unavailable

StageVideo Objects

ALWAYS Include a Video
Object as Backup


StageVideo Objects

ALWAYS Include a Video
Object as Backup
in Case StageVideo is Unavailable!


StageVideo Objects

StageVideo is NOT a
DisplayObject


StageVideo Objects

StageVideo is NOT a
DisplayObject
no addChild()


stageVideos Vector

■
StageVideo objects stored in a vector


stageVideos Vector

■
■
stage.stageVideos


stageVideos Vector

■
■
stage.stageVideos
■
stage.stageVideos.length Tells You How Many StageVideo
Instances Will be Supported


stageVideos Vector

■
■
stage.stageVideos
■
stage.stageVideos.length Tells You How Many StageVideo
Instances Will be Supported

availStageVideos:Vector.<StageVideo> = stage.stageVideos ;
if (availStageVideos.length > 0 )
stageVid:StageVideo = stage.stageVideos [ 0 ] ;


StageVideo Availability

■
StageVideo objects can Appear and Disappear



■
■
E.g., StageVideo Unavailable at Start, but User Enters FullScreen
Mode and StageVideo Becomes Available



■
■
■
Thus, Listen for Changes in StageVideo Availability



■
■
■
■
StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY



■
■
■
■
StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY

stage.addEventListener(StageVideoAvailabilityEvent.STA
GE_VIDEO_AVAILABILITY, onStageVideoState);


StageVideoAvailabilityEvent

■
Dispatched When...



■
Dispatched When...
■
Listener Added



■
Dispatched When...
■
Listener Added
■
StageVideo Availability Changes



■
Dispatched When...
■
Listener Added
■
■
Use ‘availability’ Property, Which Can Be



■
Dispatched When...
■
Listener Added
■
■
■
StageVideoAvailability.AVAILABLE



■
Dispatched When...
■
Listener Added
■
■
■
■
StageVideoAvailability.UNAVAILABLE



■
Dispatched When...
■
Listener Added
■
■
■
■
StageVideoAvailability.UNAVAILABLE
onStageVideoState(e:StageVideoAvailabilityEvent):void {
if (e.availability == StageVideoAvailability.AVAILABLE)
//stage video is available -- can use StageVideo
else
//stage video is unavailable -- use Video object
}

Playing Video

To See Your Video, Attach
the NetStream to the
StageVideo
stageVid.attachNetStream ( ns ) ;


Playing Video

Control Video Playback
via NetStream

ns.play ( ) ;


Putting it all Together

Simple Demo


Controlling Size & Position

■
viewport:Rectangle
■
The Visible Surface of the StageVideo Instance
■
A Rectangle Object ( x , y , width , height ) Must be Speciﬁed


Controlling Size & Position

■
viewport:Rectangle
■
The Visible Surface of the StageVideo Instance
■
A Rectangle Object ( x , y , width , height ) Must be Speciﬁed

stageVid.viewPort = new Rectangle ( 0 , 0 , 320 , 240 );


Render State


Render State

■
StageVideoEvent.RENDER_STATE Dispatched When
a NetStream is Attached to the StageVideo object


Render State

■
StageVideoEvent.RENDER_STATE Dispatched When
a NetStream is Attached to the StageVideo object

stage.addEventListener(StageVideoEvent.RENDER_S
TATE, onStateChange);


Accurate Sizing


Accurate Sizing

■ videoHeight:int
■
Native Video Height; Read-Only


Accurate Sizing

■ videoHeight:int
■
■ videoWidth:int
■
Native Video Width; Read-Only


Accurate Sizing

■ videoHeight:int
■
■ videoWidth:int
■

trace ( “Stage Video Dimensions: “ +
stageVid.videoWidth + “ x “ + stageVid.videoHeight ) ;


Accurate Sizing

■ videoHeight:int
■
■ videoWidth:int
■

trace ( “Stage Video Dimensions: “ +
stageVid.videoWidth + “ x “ + stageVid.videoHeight ) ;

stageVid.viewPort = new Rectangle ( 0 , 0 ,
stageVid.videoWidth , stageVid.videoHeight ) ;


Rendering Method

■
StageVideoEvent.status Tells How Frames are
Decoded
■
VideoStatus.SOFTWARE
■
Indicates hardware-accelerated (GPU) video decoding
■
VideoStatus.ACCELERATED
■
Indicates software video decoding
■
VideoStatus.UNAVAILABLE
■
Video decoding is not supported


Rendering Method

■
StageVideoEvent.status Tells How Frames are
Decoded
■
VideoStatus.SOFTWARE
■
Indicates hardware-accelerated (GPU) video decoding
■
VideoStatus.ACCELERATED
■
Indicates software video decoding
■
VideoStatus.UNAVAILABLE
■
Video decoding is not supported

myTextField.appendText ( "StageVideo is
rendering with " + evt.status );

Zooming StageVideo


Zooming StageVideo

■
zoom:Point


Zooming StageVideo

■
zoom:Point
■
A point, specifying an x (horizontal) and y (vertical) zoom
factor, with values between 1 (no zoom) and 16 (16x zoom)


Zooming StageVideo

■
zoom:Point
■
A point, specifying an x (horizontal) and y (vertical) zoom
factor, with values between 1 (no zoom) and 16 (16x zoom)

stageVid.zoom = new Point (8,8);


Panning Zoomed StageVideo



■
pan:Point



■
pan:Point
■
A point, with x and y pan values between -1 (left/up) and 1
(down/right), with default of 0 (center) that determines pan
positioning of zoomed video



■
pan:Point
■
A point, with x and y pan values between -1 (left/up) and 1
(down/right), with default of 0 (center) that determines pan
positioning of zoomed video

stageVid.pan = new Point (1,1);


More StageVideo Properties



■
depth:int



■
depth:int
■
The z-depth of the StageVideo object, for z-ordering among
multiple StageVideo objects



■
depth:int
■
■
colorSpaces:Vector.<String>



■
depth:int
■
■
colorSpaces:Vector.<String>
■
A vector list of the color spaces available on the playback device


Putting it all Together

More Complex Demo


This Talk in 10 Seconds...



■
Play Outstanding, Pixel-Drenched Video



■
■
In Flash Player



■
■
In Flash Player
■
On Every Device



■
■
In Flash Player
■
On Every Device
■
To Optimize the Experience:



■
■
In Flash Player
■
On Every Device
■
■
Use StageVideo



■
■
In Flash Player
■
On Every Device
■
■
Use StageVideo
■
& H.264 Video



■
■
In Flash Player
■
On Every Device
■
■
Use StageVideo
■
& H.264 Video
■
Under the Hood...



■
■
In Flash Player
■
On Every Device
■
■
Use StageVideo
■
& H.264 Video
■
Under the Hood...
■
Use the new AS3 APIs



■
■
In Flash Player
■
On Every Device
■
■
Use StageVideo
■
& H.264 Video
■
Under the Hood...
■
Use the new AS3 APIs
■
or use OSMF or Strobe to forget about the code!


Please Fill Out the
Scorecard


Thank You

Personal Blog | http://rblank.com
Online Training | http://richmediainstitute.com

Delivering the Best Flash HD Video by R Blank from Adobe MAX 2011

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

Delivering the Best Flash HD Video by R Blank from Adobe MAX 2011

Editor's Notes