N Screens problem with HTML5: W3C Conference Redmond, Nov 15,2011 @iRajLal


Published on

N-Screens Problem is a set of challenges you come across when you develop an App for multiple connected devices. How HTML5 WebApp solution was the most promising solution for the developers. My session was around the 7 Key implementation challenges you will face when developing a Web App solution for multiple devices with different form factors.

The 7 key challenges were, Feature Detection, Layout detection, Using CSS3, Choice of Animation between SVG, Canvas, CSS3 Animation, and WebGL, Audio Video related problems and their work-arounds, and how to use advanced HTML5 API with a brief on Polyfills and Shims.

Here is the online demo of my presentation. Source code is also available at the demo page. Detect your N-device HTML5 features and APIs at


Everything you need to know about developing HTML5 Apps on multiple devices is at W3C conference.
Redmond, Nov 15-16, 2011, amazing conference, first time all the vendors together, instead of talks about individual device/ browsers, all the sessions were focused on the common part, the HTML5 part. If you missed the conference check videos here


Published in: Technology, Art & Photos
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • How to build an app for multiple devices in a most effective manner?
  • This is my plan for next 30 minutesI am going to talk about the N Screens we are discussing here. Then I will talk about the Design Problem for these N-Screens, By design problems I mean the difference in the context in which these screens devices are used. Then I will discuss the development problems you might face when creating apps for these devices. Particulary I will talk about two prominent ways to develop on these devices and talk about the Pros and Cons of them, and then we will see Why WebApp is the positive story every developer is talking about. Then I am going dive into technical Implementation Problems and how to solve them
  • - The Context - How to develop Apps for N-screens - Why the HTML5/ WebApp Solution - What are 8 Key Challenges - Where to look for latest specifications W3 Working Groups
  • We are talking about these 4 types of devices. A Smartphone with an advanced browser A tablet device A PC and a TV There are two things common in all these devicesThey are all Connected to the internetWe can develop Apps for each of them
  • - The Context - How to develop Apps for N-screens - Why the HTML5/ WebApp Solution - What are 8 Key Challenges - Where to look for latest specifications W3 Working Groups
  • A volatile scenarioShort attention spanDivided attention InterruptibleDynamic environment
  • A volatile scenarioShort attention spanDivided attention InterruptibleDynamic environment
  • A volatile scenarioShort attention spanDivided attention InterruptibleDynamic environment
  • A volatile scenarioShort attention spanDivided attention InterruptibleDynamic environment
  • - The Context - How to develop Apps for N-screens - Why the HTML5/ WebApp Solution - What are 8 Key Challenges - Where to look for latest specifications W3 Working Groups
  • W3C: Original specification
  • HTML can be unanimously credited for the success of the World Wide Web. The ease of the simple text based markup language, made the html pages very easy to learn, and so very popular. HTML5 Web Apps are meant for mobile devices but it is the result of five major phases of the web evolution. The “old” Web Web ApplicationsRich Internet ApplicationsHTML5 specification Mobile Web RevolutionThe “Old” WebHTML started as simple web pages development. A basic set of HTML tags were available for rendering tables, images, anchors, etc. in a web page. The HTML page is rendered in the browser sandbox and could not access user’s computer. A simple light weight text files, called Cookies were used to save user’s session information in the form of key value pairs. HTML also allowed including CSS (cascading style sheets) files for separating style from the structure and included a Scripting language, JavaScript for basic programming on the client sideWeb ApplicationsWith the popularity of the web, a more robust distributed application framework started called web applications. These web applications used a client-server architecture, where client side technologies like HTML, CSS and JavaScript were used for programming the front end, and the heavy lifting was done at the server side. HTML was still the “thin” client with the minimal access to the computer. A Rich set of programming languages like Java, PHP, and ASP were used at the server side. These server programs generated HTML pages which are emitted to the user’s browser. The server programming language could access data on the server and can do advance functionality but client side programming was limited to basic scripting like form validations.Rich Internet ApplicationsAs the web evolved the need for richer functionality like 3D, animations, web camera integration and data access grew and, web browser extensions like XPCOM and ActiveX object technologies like Adobe Flash, Java Applet and Microsoft Silverlight became popular. These technologies provided rich islands of functionalities and are also referred as Rich Internet Applications. ActiveX technology was also used for Media players and animated advertisements. Adobe Flash and Microsoft Silverlight objects in the HTML page had bigger storage in the client computer, and could access computers audio and video hardware. Browser extension technology XPCOM allows accessing computer functionality and data through a native dynamic link library.HTML5 Specification While the web was evolving with standardization of CSS and JavaScript in different types of browser, W3C and other companies came together and based on the all the previous usage and requirements of the web, created a specification for the next version of HTML, called HTML5 with a rich set of capabilities. HTML5 had a number of advanced tags for drawing, media player capabilities, content structuring tags and validation in HTML. It also had a local database with bigger Web storage options without the need of an embedded Object. It gave offline Web Application, Geo location and multi threads functionality. It also had Web sockets for peer to peer communications. The CSS3 specification supported by HTML5 included an advanced CSS effects, transitions and animations.Mobile Revolution While HTML5 specification was being created, simultaneously there was a mobile revolution, which started with the advent of advanced smartphones like iPhone, Android, and MeeGo devices. With the need of web on the mobile browser a new paradigm shift took place. There was a demand for accessing device capabilities from the web browser. The device access events and APIs started becoming a standard of its own. Device APIS like Touch and Gesture Events, Capture API for Audio, Video and Webcam captures, Contacts API, Calendar API, Web Notification, Device Status and Battery consumption, Device Orientation, etc. became a part of HTML5 specifications. These advanced APIs together with CSS3 gave birth to a new kind of application called Web Apps. Web AppsA Web App is a web application hosted in a remote server but which look, feel and behave like a native mobile application.
  • - The Context - How to develop Apps for N-screens - Why the HTML5/ WebApp Solution - What are 8 Key Challenges - Where to look for latest specifications W3 Working Groups
  • raw
  • raw
  • raw
  • raw
  • raw
  • - The Context - How to develop Apps for N-screens - Why the HTML5/ WebApp Solution - What are 8 Key Challenges - Where to look for latest specifications W3 Working Groups
  • N Screens problem with HTML5: W3C Conference Redmond, Nov 15,2011 @iRajLal

    1. 1. THE N-SCREENS PROBLEM Building Apps in the world of TV & Mobile Rajesh Lal Senior Engineer
    2. 2. Agenda 7 Key Challenges WebApp Develop Design N-Screens
    3. 3. The N-Screens
    4. 4. Mobile PCTablet TV
    5. 5. 1. Design Problem: N Screens
    6. 6. On the Go MobileScreen size 2-4”Distance 1’Connection 3GNot reliableHighly InterruptibleFinger Input
    7. 7. Screen 7-14” TabletCasual EnvironmentDistance 1’Wi-Fi Loosely connectedInterruptibleInput Touch
    8. 8. Screen size 14-27” Computer Desk Distance 2’ Reliable LAN PC Focused Keyboard Mouse
    9. 9. Screen size 25-65” Couch Distance 10’ LAN Fast TV connection Total Emersion D-Pad Remote
    10. 10. 2. Developing Apps for N Screens Native vs. WebApp
    11. 11. Native: Developing Apps for N-Screens Pros • Each “Screen” with an SDK • Great for THAT device • Robust • Device APIs
    12. 12. Native: Developing for N-Screens Cons Objective C • Learning curve for each SDK Java • Multiple Programing Languages C# C++ • Time and Effort for every device Qt, QML XAML • Maintenance • Versions
    13. 13. 3. The WebApp Solution Web Application 1.0
    14. 14. WebApp: Developing for N-Screens A WebApp is an application created using HTML, CSS and JavaScript. • App in the Web browser (Full screen) • WebApp in a native container (Hybrid)
    15. 15. History“Old" HTML Web RIAs HTML5 WebApp Applications Basic Server Script Media Advanced Touch Table, Img Client-Server Players Canvas Camera Cookies Thin Client Plug-ins Media Contacts Sandbox Validation Extensions Storage Calendar CSS Object Tag Web db Notification JavaScript Pvt. Storage Geo location Battery Audio video Offline Orientation XPCOM Drag Drop Acceleromet Threads CSS3
    16. 16. WebApp: Developing Apps for N-Screens Pros • One “SDK” • Easy to Develop • Powerful HTML5 APIs • CSS3
    17. 17. WebApp: Developing for N-Screens Cons • Work in Progress • Challenging when porting to many Screens
    18. 18. 4. Seven Key Challenges And How to Work Around Them
    19. 19. 1/7 Feature Detection Which Feature Your Screen Support ?
    20. 20. Feature Detection Feature Browser Detection Detection Old method New method
    21. 21. Feature DetectionIn the code
    22. 22. Feature Detection Use libraries • Modernizr • jQuery.support
    23. 23. Feature Detection code <script src="modernizr-1.7.js“/> Modernizr.canvas ? "Canvas" : “No canvas"; <script src="jquery-latest.js“/> jQuery.support.ajax ? "Ajax" : “No Ajax";
    24. 24. Feature DetectionModernizrCanvas SVG Cross Window MsgAudio InlineSVG Web WorkersVideo WebGL Web SQL databaseAnimation Websockets TouchTransformation GeolocationDrag and Drop Local Storage
    25. 25. Feature Detection jQuery.Support Ajax Cross Domain Resource Sharing (CORS) Fixed Position CSS Opacity CSS Float htmlNormalize
    26. 26. 2/7 Layout Detection How to adapt the layout to the device form factor?
    27. 27. Layout Detection Media Orientati Queries Viewport on JavaScript
    28. 28. Layout Detection Media-queries enable style sheets tailored for different ’width’, ‘height’ and ‘color’
    29. 29. Media Queries: Layout Detection <link rel="stylesheet" href="handheld.css" media="only screen and (max-width: 480px), handheld" /> <link rel="stylesheet" href="default.css" media="screen" /> <link rel="stylesheet" href="wider.css" media="only screen and (min-width: 1200px)" />
    30. 30. Media Queries: Layout Detectionhandheld Default screen Wider screen http://www.lancs.ac.uk/
    31. 31. Media Queries: Layout Detection jQuery.mediaqueriesThe script adds basic Media Query-Support (min-width and max-width in px units ) to all browsers <script src="jquery.mediaqueries.js“></script>
    32. 32. LayoutMedia Query for device-aspect-ratio@media screen and (device-aspect-ratio: 16/9)@media screen and (device-aspect-ratio: 32/18)@media screen and (device-aspect-ratio: 1280/720)@media screen and (device-aspect-ratio: 2560/1440)
    33. 33. Orientation Media Query for Orientation*@media screen and (orientation:portrait) { /* Portrait styles */}@media screen and (orientation:landscape) { /* Landscape styles */} *DeviceOrientation Events specification at W3C
    34. 34. ViewportViewable area on the screen<meta name="viewport" content="width=device-width,initial-scale=1, maximum-scale=1">
    35. 35. 3/7 Advanced Styles How to utilize CSS3 for Advanced Graphics ?
    36. 36. Advanced Styles with CSS3 • Background • Border Radius • Drag and Drop • Opacity • Transformation • …
    37. 37. Challenge: Advanced Styles with CSS3 Prefix • -moz- • -ms- • -o- • -webkit-
    38. 38. Workaround: Advanced Styles with CSS3 .style { -moz-border-radius: 1em; -ms-border-radius: 1em; -o-border-radius: 1em; -webkit-border-radius: 1em; border-radius: 1em; }
    39. 39. 4/7 Animation When to Use SVG, Canvas, CSS3 and WebGL Animation?
    40. 40. Animation: Challenge WebGL CSS3 Canvas & Animation SVG & JavaScript JavaScript
    41. 41. Animation: SVG • 2D vector graphics using XML • Object retained in the memory • Full DOM support • SVG elements can be styled • Check Modernizr.svg / Modernizr.inlinesvg Perform better when smaller number of elements and large surface
    42. 42. Animation: SVG <svg id="svgElement" width="800px" height="600px" viewBox="0 0 800 600"> <rect x="0" y="0" width="100%" height="100%" rx="10" ry="10" style="fill: white; stroke: black;" /> <circle id="circle0" cx="40" cy="40" r="40" style="fill: orange; stroke: black; stroke-width: 1;" /> </svg>
    43. 43. Animation: Canvas: Context 2D • Bitmap drawing area • Rectangles, lines, arcs, paths • Stateless • No DOM support, single element • Check Modernizr.canvas Perform better when large number of objects and surface is small
    44. 44. Animation: CSS3 • Styles for Individual elements • Use CSS3 animation if available • Better than JavaScript based animation • Check Modernizr.csstransitions Can perform better with GPU acceleration
    45. 45. Animation: CSS3 var elem = document.getElementById(transelem); elem.addEventListener(click, function loop() { elem.style.left = „100px;}, false); #transelement { … -xxx-transition: opacity 1s ease; transition: opacity 1s ease; }
    46. 46. Animation: WebGL (Canvas Context: webgl) • Canvas based 3D implementation • Use GPU acceleration • Supported only in a few browsers • Check Modernizr.webgl
    47. 47. 5/7 Media How to use Audio-Video?
    48. 48. Audio: Big Challenge Open AL Audio Web RIA Audio Data - Audio - Firefox Webkit HTML5 Flash/ Audio Silverlight Not suitable for Sfx
    49. 49. 6/7 Ajax How to use Ajax and do Cross Domain call?
    50. 50. Ajax: XMLHttpRequest CORS Write a Cross JSONP Proxy Origin Resource Hybrid Not sharing App Standard
    51. 51. Ajax: XMLHttpRequest • Cors Server code self.response.headers.add_header ("Access-Control-Allow-Origin", "*") • Check jQuery.Support.Ajax jQuery.Support.cors
    52. 52. 7/7 HTML5 APIs How to use HTML5 advanced APIs in all Screens? & What are PolyFills and Shims?
    53. 53. HTML5 API: Challenge HTML5 API Polyfills and Shims JavaScript Code
    54. 54. HTML5 API: Polyfills and Shims? Provide support for missing features. • Polyfills: Replicate standard feature API • Shims: Own API with useful features
    55. 55. W3C Working Groups And other resources
    56. 56. Where to look for more information W3C Working Groups http://www.w3.org/Mobile/ http://www.w3.org/TR/css3-mediaqueries/ http://www.w3.org/2011/audio/ http://www.w3.org/testing/browser/ http://www.w3.org/Graphics/SVG/WG/wiki/Main_Page http://www.w3.org/TR/cors/ http://dev.w3.org/geo/api/spec-source-orientation.html Other Resources http://www.modernizr.com/ http://api.jquery.com/jQuery.support/ http://Html5rocks.com http://mediaqueri.es/
    57. 57. The future is already here—it’s just not veryevenly distributed. - William Gibson Thank You @rajeshlalnokia rajesh.lal@nokia.com