Web DU Mobile Meow
Upcoming SlideShare
Loading in...5
×
 

Web DU Mobile Meow

on

  • 998 views

This is the story of the glory and struggle of bringing a high quality YouTube experience to the mobile web. Once upon a time there was a web developer who wanted to play videos on the web. So he ...

This is the story of the glory and struggle of bringing a high quality YouTube experience to the mobile web. Once upon a time there was a web developer who wanted to play videos on the web. So he filmed a cat and wrote a Flash and a HTML5 player. He spent many hours making it work on his favorite desktop browsers and even the one his grandfather still used. People could watch his cat video, he smiled, and the world was good. Then one day someone put a browser in a phone and soon there were many phones with many different browsers. This new set of environments were even harder to develop for and had a slew of new terrifying bugs. The web developer was miserable knowing people couldn’t watch his cat video. With much time and effort he figured out many of the secrets needed to combat the evils of the different mobile platforms. Once again people could watch his cat video, he smiled, and the world was good. The end.

Statistics

Views

Total Views
998
Views on SlideShare
987
Embed Views
11

Actions

Likes
2
Downloads
2
Comments
0

2 Embeds 11

http://localhost 9
http://lanyrd.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Web DU Mobile Meow Web DU Mobile Meow Presentation Transcript

  • Mobile Meow Bring YouTube to the Mobile WebBy YouTubes Greg Schechter and Matt Ward
  • Greg Schechter Matt WardThe Web Warrior Seek Bar Engineerschechter@google.com mattward@google.com
  • Video
  • Video
  • A Brief History
  • Video
  • Flash
  • <video>
  • <video> <video> <video> IE Chrome Firefox <video> <video> Opera Safari
  • Video
  • Playback Stats
  • Flash Flash
  • Flash Native <video>
  • FlashFlash == Flash
  • Native
  • Native ○ For fallbacks, use rtsp:// protocol (serving .3pg) if the device wont support HTML5 ○ Use custom protocol / URL scheme to launch your own Android, iOS native app (no Windows Phone yet) ■ youtube://video_id
  • <video>
  • The Different Browsers Safari Android
  • The Different Browsers OperaSafari Android Chrome Silk Mini Firefox Opera IE
  • The Different Browsers OperaSafari Android Chrome Silk Mini Firefox Opera IE AMP LE S
  • Fragmentation
  • By brownpau at http://www.flickr.com/photos/brownpau/533267369/
  • How do we start <video>
  • The Source <video src="funny_cat_video">
  • The Source<video src="funny_cat_video.webm/mp4/ogg">
  • Mobile Formats Support Chrome Safari Firefox Opera IE AndroidH.264WebM Android 4+ Android 4+ Android 4+HLS Android 3+ Platform Versions and Distribution
  • The Source <video> <source src="funny_cat_video.mp4"> <source src="funny_cat_video.webm"> </video>
  • HLS = HTTP Live Streaming <video src="playlist.m3u8">
  • HLS video-0-1.ts format-0.m3u8 video-0-2.ts ... video-1-1.ts index.m3u8 format-0.m3u8 video-1-2.ts ... video-2-1.ts format-0.m3u8 video-2-2.ts ...
  • HLSBig Features ○ Live and Adaptive Streaming ○ Built in CDN failover ○ Metadata + Cue PointsApple App Requirements ○ > 10 minutes in duration or 5 mb in 5 minutes ○ at least one 64 kbps or lower stream requiredTools and more information on Apples website:Apples Using HTTP Live Streaming
  • Power
  • Power Consumption: H264 vs WebM **fullscreen flash **windowed flashResearch done by Yossi Oren For more information visit http://iss.oy.ne.ro/
  • Stick it in a page
  • Basic video tagSafari ○ Renders a thumbnail poster and play button (flaky) ○ Background is set to black and cannot be overwritten ○ If a parent element has display:none set at any point the video fails to playback properlyOpera ○ Gives video dimentions black background if the information is available ○ Has issues with stretching WebMIE and Android ○ Play button/film icon for all video tags regardless if browser can play the format ○ Background is set to black/grey and cannot be overwritten
  • Give it a poster and click it
  • Poster AttributeUse the poster attribute to get a thumbnailSafari, Chrome, Android, and Opera ○ Preserve aspect ratio of poster imageIE and Firefox ○ Stretch image to size of video tagIE ○ Keeps poster as paused stateAll Others ○ Replace video with the paused frame
  • Basic video tagChrome ○ Default click to toggle play/pause ○ Remove with an empty onclick handlerSafari ○ Will launch any supported video with or without an onclick handlerAndroid and IE ○ Shows click interactions but does not play with empty onclick
  • Basic video tagPro tips: ○ Create your own cued state to convey you can play the video ○ Set the background color to black for continuity across platforms
  • Controls <video controls>
  • Lets get some controls
  • Controls Chrome Safari Firefox Opera IE AndroidPlay/pauseButtonVolume Only in Mute toggle Dont work inControls Fullscreen only Android 4Seek barFullscreen Button Gesture ButtonbuttonPlayback Both Fullscreen Inline only Determined Fullscreen BothType only by hardware only
  • Fullscreen
  • requestFullScreen
  • requestFullScreen● Desktop Only ○ Firefox, Safari, and Chrome ○ Vendor Prefixed
  • Fullscreenvar elem = document.getElementById("my-element");document.onwebkitfullscreenchange = function() { // The fullscreen element: // document.webkitFullScreenElement; console.log ("We went fullscreen!");};elem.webkitRequestFullScreen();
  • What about Mobile?
  • Isnt it already fullscreen?
  • Yes (sort of)
  • Fullscreen● Desktop ○ requestFullScreen● Mobile
  • Fullscreen● Desktop ○ requestFullScreen● Mobile ○ Open New Tab
  • Fullscreen● Desktop ○ requestFullScreen● Mobile ○ Open New Tab ○ webkitEnterFullscreen ● Webkit only ● Video element only ● metadata must be loaded
  • Fullscreen● Desktop ○ requestFullScreen● Mobile ○ Open New Tab ○ webkitEnterFullscreen ● Webkit only ● Video element only ● metadata must be loaded● Pro Tip: Remember size context changes so use viewport to scale icons and controls
  • @viewport { width: device-width; }
  • Custom Controls Lets get some custom controls
  • Custom Controls● Preserve your brand● Unified experience across platforms and browsers● Allows us to expand the set of controls and add our own ○ annotations ○ playlist ○ captions ○ more
  • Custom Controls Pro Tips● User expect to be able to drag the progress bar ○ Need to remender to prevent scroll on touchstart● Volume cant be set everywhere and users are accustomed to using device controls ○ So dont build controls for it
  • Custom Controls Pro Tips● Fullscreen ○ The browsing context is always fullscreen so fake it● Dont overload clicking for both play/pause and show/hiding controls● Dont trigger content with hover
  • Custom Controls Pro Tips● Fingers are fat ○ Average finger is 11mm so make targets at least 40px with 10px padding ○ Use SVGs so icons can be scaled and shared with desktop application ○ Do what you can in css
  • autoplay <video autoplay>
  • autoplayIn Safari on iOS (for all devices, including iPad),where the user may be on a cellular network andbe charged per data unit, preload and autoplay aredisabled. No data is loaded until the user initiates it.
  • autoplayIn Safari on iOS (for all devices, including iPad),where the user may be on a cellular network andbe charged per data unit, preload and autoplay aredisabled. No data is loaded until the user initiates it.
  • What about everyone else?
  • Autoplay Chrome Safari Firefox Opera IE AndroidAttributeScripted Buggy
  • autoplay
  • autoplay <video onclick="this.play()">
  • autoplayfunction someClickEvent(evt) { // In a user initiated thread. myVideoElement.load(); getVideoData(); // Triggers an ajax call.}function onGetVideoDataReturned(data) { // Not in a user initiated thread. setVideoElementSrc(data); myVideoElement.load(); myVideoElement.play();}
  • autoplayfunction someClickEvent(evt) { // In a user initiated thread. myVideoElement.load(); getVideoData(); // Triggers an ajax call.}function onGetVideoDataReturned(data) { // Not in a user initiated thread. setVideoElementSrc(data); myVideoElement.load(); // For Android window.setTimeout(function() { myVideoElement.play(); }, 0);}
  • autoplay setMediaPlaybackRequiresUserAction
  • Embeds
  • Embeds <script>
  • Embeds <script> ○ We need our content to be sandboxed ○ More than just a video tag
  • Embeds <script> ○ We need our content to be sandboxed ○ More than just a video tag <object>
  • Embeds <script> ○ We need our content to be sandboxed ○ More than just a video tag <object> ○ Can load content with the data attribute ○ But no way to interact with it via JavaScript
  • Embeds <iframe>
  • Embeds <iframe> ○ Allows our content to be sandboxed ○ JavaScript API communication
  • Embeds<iframe type="text/html" width="640" height="385" frameborder="0" src="http://www.youtube.com/embed/VIDEO_ID" allowfullscreen></iframe>
  • Daily impressions of EmbedTotalobjectiframe April 2011 April 2012
  • EmbedsPro tip: Plan for the future (if you can)<iframe type="text/html" width="640" height="385" frameborder="0" src="http://www.youtube.com/embed/VIDEO_ID" allowfullscreen></iframe>
  • Embeds Pro Tips:html { /** Hack to fix iPhone resizing. */ overflow: hidden;}
  • Embeds Pro Tips:body { /** Dymanic Resizing **/ background-color: #000; height: 100%; width: 100%; /** Remove highlight when use clicks **/ -webkit-tap-highlight-color: rgba(0, 0, 0, 0);}
  • API
  • API Communication
  • API Communication● Poll the URL fragment? ○ http://youtube.com/embed/video_id#fragment
  • API Communication● Poll the URL fragment? ○ http://youtube.com/embed/video_id#fragment● Messages are one dimensional.● Polling eats up CPU and is not instant.● Both directions of communication use the same fragment.
  • API Communication● Better idea: PostMessage API ○ someWindow.postMessage(message, targetOrigin);
  • API Communication● Better idea: PostMessage API. ○ someWindow.postMessage(message, targetOrigin);● Uses JSON for native encoding and decoding of data.● No polling.● Native event listeners.● Communications are sandboxed per-window.● Calls are asynchronous.● Great for Mobile, but desktop support is missing in IE7
  • Remember all communication is asynchronous
  • Testing
  • Testing: The Old Fashioned Way ● Device lab ○ accurate ○ costly ○ space inefficient ○ boring
  • Testing: With Software ● Hardware Emulators / Simulators ○ available for major systems ○ approximation varies ○ still boring ● Android Emulator ○ bulky, slow, no video codecs ● iOs Simulator ○ both tablet and phone
  • Testing: With Software ● Browser Simulators ○ Poorer approximation ● Opera Mobile Simulator ○ missing video tag support ● Fennec (FF) Simulator ○ poor touch control mapping ○ sends desktop user agent
  • Testing: Automation ● Selenium! ○ use with simulated or real devices ○ not boring
  • Testing: Automation ● Android Webdriver ○ Still no video support ● iOS Webdriver ○ Need to register as ios dev ○ Intermittent issues with playback ● IE ○ No webdriver APIs for mobile
  • Testing: Automation ● Chrome, Opera ○ Driver APIs built-in ○ Remote debugging ● FFMobile ○ No webdriver APIs
  • Testing: Automation Strategy ● Test API methods in mobile context first ○ cheap ○ can approximate video playback by video. currentTime
  • Testing: Automation Strategy ● More sophisticated tests to follow ○ screenreader ● Screen cap processing ○ check for distortion ○ playback accuracy
  • Testing: Automation Strategy ● Screen cap processing
  • Testing: Hardware Assistance
  • The Future Cats in Space by WF&TD
  • Or
  • PointerEventshttp://msdn.microsoft.com/en-us/library/ie/hh673557(v=vs.85).aspx
  • http://imgs.xkcd.com/comics/in_ur_reality.png
  • Track<video> <track src="cats_meow" kind="subtitles"></video>
  • getUserMedia
  • More to learn ● HTML5 Video Spec ○ http://dev.w3.org/html5/spec-author-view/video.html ● Browser Blogs ○ http://dev.opera.com/ ○ http://hacks.mozilla.org/ ○ http://blogs.msdn.com/b/ie/ ○ http://peter.sh/ ● Documentation ○ https://developer.mozilla.org/en-US/ ● Other ○ http://developer.apple. com/library/safari/#documentation/AudioVideo/Conceptual/Using_HT ML5_Audio_Video/Device-SpecificConsiderations/Device- SpecificConsiderations.html
  • Questions? can haz question? By cloudzilla http://www.flickr.com/photos/cloudzilla/378829651/schechter@google.commattward@google.com