HTML5 and Other Modern Browser Game Tech

HTML5 and other Modern Browser Game TechVincent ScheibSoftware Engineer, Chrome GPU teamFebruary 28, 20111

This is an OverviewOverview & MotivationBrowsers, PlatformsTechnology SurveyGraphics, Audio, Networking, Storage…Libraries, Benchmarks2

My Perspective on the Last Decade & Web Tech4GPU AccelerationC++ in BrowserChrome OS

April 1 2010 – Quake II in HTML5Ray Cromwell, Stefan Haustein, Joel Webber – Google- WebGL, canvas- Web Sockets- Local Storage- AudioChrome & Safari5

No Plugins6Not discussing: Flash, Java, Silverlight, Unity

Plugin Use / Installation% of Chrome Users7iOS* Data from users opting in to reporting usage statistics, 2010

Low Friction, Better Security8

Browser Trends10IEFirefoxChrome

Mobile TrendsOperaiPhoneNokiaBlackBerryAndroid11

Mobile TrendsiPhoneAndroidNokiaBlackBerryOperaiPhoneNokiaBlackBerryAndroid12

Mobile Native or Web App?iOS & Android Monetize Native AppsWebApp wrappers: PhoneGap, appMobiPerformanceNative has greater perf, butOn Android, can call through to C++ from JSForms InputHTML5 input forms produce correct input deviceTouch13

Chrome Web Store – Distribution & Monetization14

Chrome Web Store – Chrome Integration15

Chrome Web StoreDiscovery, Distribution and Monetization on the Web16Web Apps120 million usersIn-app payments coming soongoogle.com/checkout/inapp… And Monetization

The Browser is the PlatformChromeOSwebOS17

Canvas 2D – OverviewMostly SpritesFlash Widely supported (mobile & desktop)IE9 brings GPU performanceJavscript Shim Libraries run on old IE, e.g. ExplorerCanvasGPU upgrades Hundreds to Thousands of Draws @30fpsImmediate ModePopular21

Canvas 2D – APIPrimitivesrect, circle arcs, lines, bezier curves, text, imagesDrawingfill and strokeEffectsshadows, gradients, image patterns, line styles, clipping, compositing operationsTransformsrotate, scale, matrix transform (2x2 + translation)22Text

Canvas 2D – Simple Sample<canvas id="e" width="200" height="100"></canvas><script> var context =document.getElementById("e").getContext("2d"); var cat = new Image(); cat.src = "images/cat.png"; cat.onload = function() { context.drawImage(cat, 0, 0); };</script>23

Canvas 2D – Fill and Stroke Sample// Draw eyesctx.fillRect(160, 130, 20, 80);ctx.fillRect(220, 130, 20, 80);// Draw mouthctx.beginPath();ctx.moveTo(100, 230); // Start smilectx.bezierCurveTo(100, 230, 200, 380, 300, 230);ctx.moveTo(219, 298); // Start tonguectx.bezierCurveTo(278, 351, 315, 315, 277, 258);ctx.lineWidth = 20;ctx.stroke();(adapted from IE9 sample)24

Canvas 2D – AvailabilityChrome – Firefox – Safari: GO!Internet Explorer: Soon in IE9Mobile: GO!25- caniuse.com, Feb 2011

SVG – OverviewVector markup languageDeclarative or retainedprocedural APITools (Illustrator, Inkscape)HTML5 Enables SVG Markup Inline with HTMLRetained APIGPU Acceleration Friendly - Retained data structures allow cachingLittle GroundswellComplex API? Retained?28

SVG – Sample<!DOCTYPE html><html><body> <svg id=“mySVG”> <circle id="circle0" cx="100" cy="75" r="50" fill="grey" stroke="black" stroke-width="5“onmousedown="alert('clicked');"gt; <text x="60" y="155">Hello World</text> </svg>...29

SVG – Sample <script>var circle = document.createElementNS( "http://www.w3.org/2000/svg", "circle");circle.setAttribute('cx', 90); circle.setAttribute('cy', 90);circle.setAttribute('r', 30);circle.setAttribute('onmousedown', "alert('no, me!');");document.getElementById("svg0").appendChild(circle); </script></body></html>30

SVG – Linksraphaeljs.comEase of use, charts, etc.burst.bocoup.comRenders SVG into Canvas 2D31

SVG – AvailabilityChrome – Firefox – Safari: GO!Internet Explorer: Soon in IE9Mobile: iOS GO! – Android: No32- caniuse.com, Feb 2011

WebGL – OverviewOpen GL ES 2Textures, Framebuffers, BlendingVertex and Fragment Shaders (GLSL)Khronos group specification 1.0 Q1 2011Matrix libraries come separatelyWhat will Microsoft do?35

WebGL – LibrariesMany Higher Level LibrariesC3DLCopperlichtGLGEO3DProcessing.jsSpiderGLSceneJSthree.jsWebGLUXB PointStream36

WebGL – Sample Spinning Box38

WebGL – Sample Spinning Box<script id="vshader" type="x-shader/x-vertex"> uniform mat4 u_modelViewProjMatrix; uniform mat4 u_normalMatrix; uniform vec3 lightDir; attribute vec3 vNormal; attribute vec4 vColor; attribute vec4 vPosition; varying float v_Dot; varying vec4 v_Color; void main() {gl_Position = u_modelViewProjMatrix * vPosition;v_Color = vColor; vec4 transNormal = u_normalMatrix * vec4(vNormal, 1);v_Dot = max(dot(transNormal.xyz, lightDir), 0.0);39

WebGL – Sample Spinning Box<script id="fshader" type="x-shader/x-fragment"> varying float v_Dot; varying vec4 v_Color; void main() {gl_FragColor = vec4(v_Color.xyz * v_Dot, v_Color.a); }</script>40

WebGL – Sample Spinning Boxgl = canvas.getContext(“webgl”, args);gl.createShader(shaderType);gl.shaderSource(shader, shaderScript.text);gl.compileShader(shader);var colors = new Uint8Array( [ 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, //…gl.box.colorObject = gl.createBuffer();gl.bindBuffer(gl.ARRAY_BUFFER, gl.box.colorObject);gl.bufferData(gl.ARRAY_BUFFER, colors, gl.STATIC_DRAW);//…41

WebGL – Sample Spinning Boxfunction drawPicture(gl) {reshape(gl);gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);// Make a model/view matrix.gl.mvMatrix.makeIdentity();gl.mvMatrix.rotate(20, 1,0,0);gl.mvMatrix.rotate(currentAngle, 0,1,0);//…gl.drawElements( gl.TRIANGLES, gl.box.numIndices, gl.UNSIGNED_BYTE, 0);42

WebGL – DemosNinePointFiveAquariumGoogle Body43

WebGL – AvailabilityChrome – Firefox – Safari: Chrome Shipped. Soon in FF4 & SafariInternet Explorer: NoMobile: Waiting on Firefox 4, other browsers in progress.44- caniuse.com, Feb 2011

aka – your render looprequestAnimationFrame45

requestAnimationFrame – OverviewStatus Quo: Draw, Draw, Draw, Blindly! setInterval(draw, 16)Browser Calls You (FireFox Beta 4, Chrome 10)function draw() { //... Do Drawing Work, then ask to called again:window.requestAnimationFrame(draw);}// kick off first frame:window.requestAnimationFrame(draw);46setInterval(draw, 16)setInterval(draw, 16)setInterval(draw, 16)setInterval(draw, 16)setInterval(draw, 16)setInterval(draw, 16)setInterval(draw, 16)setInterval(draw, 16)

requestAnimationFrame – AvailabilityChrome – Firefox – Safari: Soon in Chrome 10, FF4Use a Shim, future compatible:// shim layer with setTimeout fallback – Paul Irishwindow.requestAnimFrame = (function(){ return window.requestAnimationFrame || window.webkitRequestAnimationFrame || window.mozRequestAnimationFrame || window.oRequestAnimationFrame || window.msRequestAnimationFrame || function(/* function */ callback, /* DOMElement */ element){window.setTimeout(callback, 1000 / 60); };})();47

Cascading Style Sheets – 3D TransformationsCSS 3D48

CSS 3D – OverviewAdd 3D to Any Web ContentHelps make cool HUD50

CSS 3D – Sample<!DOCTYPE html><html><body> <div style="-webkit-perspective: 400;"><iframesrc="http://www.gdconf.com/" width = 1024 height = 768 style="-webkit-transform: rotate3d(1,0,0, 15deg)"></iframe></div></body></html>51

CSS 3D – AvailabilityChrome – Firefox – Safari: Soon in Chrome, FF No, Safari GO!Internet Explorer: NoMobile: iOS GO! – Android: No52- caniuse.com, Feb 2011

Web Fonts – OverviewGame UI via HTMLServe Custom FontsEasy Google Fonts API<link href='http://fonts…?family=Chewy‘rel='stylesheet' type='text/css'>In CSS:h1 { font-family: 'Chewy', arial, serif; }54

Web Fonts – AvailabilityChrome – Firefox – Safari: GO!Internet Explorer: Best in IE9Mobile: Mostly… 55- caniuse.com, Feb 2011

<audio> – OverviewTags in HTML, or created by javascript57

<audio> – Sample var audio = new Audio();audio.addEventListener("canplaythrough", function () { audio.play(); });audio.src = “treasure.ogg”;58

<audio> – IssuesiOS >= 4 – only one sample at a timeNo one codec supported by all browsers. Pick 2 from: MP3, Vorbis, WAVdetect suitability before loading, with e.g. new Audio().canPlayType(“audio/ogg”);59

<audio> – Future1FireFox Audio Data API supports Read/Write samplesvar output = new Audio();output.mozSetup(1, 44100);var samples = new Float32Array(22050);varlen = samples.length; for (vari = 0; i < samples.length ; i++) { samples[i] = Math.sin( i / 20 ); }output.mozWriteAudio(samples);60

<audio> – Future2Web Audio APINode graphLow latencySpatialization, 3dReverbNative implementation performance61

<audio> – LinksSoundManager 2Flash fallbackwww.schillmania.com/projects/soundmanager262

<audio> – AvailabilityChrome – Firefox – Safari: GO!Internet Explorer: Soon in IE9Mobile: (With limits)63- caniuse.com, Feb 2011

<video> – OverviewSource for textures in <canvas> 2D or WebGLvideoElement.play(); videoElement.addEventListener("timeupdate",updateTexture, true); function updateTexture() { gl.bindTexture(gl.TEXTURE_2D, cubeTexture); gl.texImage2D(gl.TEXTURE_2D, 0, videoElement, true); gl.generateMipmap(gl.TEXTURE_2D); }Demo65

<video> – AvailabilityChrome – Firefox – Safari: GO!Internet Explorer: Soon in IE9Mobile: GO!66- caniuse.com, Feb 2011

WebSockets – OverviewLow latency, persistent, full duplex Upgrades from HTTP handshakeUTF8Simplevar socket = new WebSocket(“ws://server.com”);socket.onopen = function(event) { socket.send(“Hello Server”); }socket.onmessage = function(event) { alert(“Server says: “ + event.data);68

WebSockets - FutureFireFox and Opera Behind FlagStandard for protocol upgradeBinaryPeer to PeerUnreliable (vs TCP)69

WebSockets – Linkssocket.ioHigher level libraryTransport on WebSockets, Flash, Ajax, …github.com/gimite/web-socket-jsShim implementation on top of Flash70

Web Sockets – AvailabilityChrome – Firefox – Safari: Yes, Still Stabilizing, Some Behind a FlagInternet Explorer: In HTML5 LabsMobile: iOS Stabilizing, Android?71- caniuse.com, Feb 2011

Javascript on your ServerNode.JS72

Node.JS – OverviewEase of Same Language and Client & ServerSimple, lots of uptakeTools / PluginsWebSockets, manage connections, render server side, just lots...Visual Debugger73

GeoLocation – SampleMobile, and Desktops toonavigator.geolocation.getCurrentPosition( function (location) {doSomething(location.coords.latitude, location.coords.longitude); });75

GeoLocation – AvailabilityChrome – Firefox – Safari: GO!Internet Explorer: Soon in IE9Mobile: GO!76- caniuse.com, Feb 2011

DeviceOrientationDeviceMotionEvent77

DeviceOrientation – OverviewiOS >= 4.2Chrome on Mac78

DeviceOrientation – Samplewindow.addEventListener("devicemotion", function(event) { // event.acceleration, // event.accelerationIncludingGravity, // event.rotationRate, // event.interval }, true);79

DeviceOrientation – AvailabilityChrome – Firefox – Safari: Yes, but not on all devicesInternet Explorer: ?Mobile: Only iOS >= 4.280

Some options…Saving Data on the Client81

Web Storage – OverviewLocal Storage & Session Storage5MB of key value pair stringslocalStorage["levels-unlocked"] = 5 // or .getItem() / .setItem()localStorage.removeItem()localStorage.clear(); // Dump everythingWidely supportedNon TransactionalNo good recourse if over limit83

Web Storage – AvailabilityChrome – Firefox – Safari: GO!Internet Explorer: GO!Mobile: GO!84- caniuse.com, Feb 2011

File API: Directories & Systems – AvailabilityChrome: Yes for Apps and ExtensionsFirefox – Safari: NoInternet Explorer: NoMobile: No85

Application Cache – OverviewCache HTML pages and ResourcesManifest File CACHE MANIFEST index.html stylesheet.css images/logo.png scripts/main.js86

Application Cache – AvailabilityChrome – Firefox – Safari: GO!Internet Explorer: NoMobile: GO!87- caniuse.com, Feb 2011

window.navigator.onLinedocument.body.addEventListener(“online”, callback);Offline88

Multithreading JavascriptWeb Workers89

Web Workers – Samplehtml5rocks.com/tutorials/workers/basics/main.js (main thread):var worker = new Worker('doWork.js');worker.addEventListener('message', function(e) { console.log('Worker said: ', e.data); }, false);worker.postMessage('Hello World'); // Send data to our worker.doWork.js (the worker):self.addEventListener('message', function(e) {self.postMessage(e.data + “? ” + e.data + “!”); }, false);// Output “Worker said: Hello World? Hello World!”90

Web Workers – AvailabilityChrome – Firefox – Safari: GO!Internet Explorer: NoMobile: No91- caniuse.com, Feb 2011

C++ for your web appNative Client92

Native Client – OverviewMachine Code Served to BrowserC/C++, Mono(C#), othersNoPluginsLegacy C++ Codebase in your Web AppPerformanceCross Platform93A Web PageFull of text, lots of interesting stuff, links, blah blah. Don’t read all the links, all this text.JSA Web PageFull of text, lots of interesting stuff, links, blah blah. Don’t read all the links, all this text.C++

Native Client – SecurityStatic analysis & SandboxingRestricted instructionsIsolate codeCustom Toolchain94BrowserNaClv8JSC++BrowserPluginOperating SystemOperating System

Native Client – AvailabilityChrome: Behind flagOpen Source95

GPU AccelerationChrome 9WebGL shipped February 3rdChrome Developer & CanaryVideo, 3D CSS, Canvas 2D, CompositingFirefox 4 BetaWebGL, Video, 2D CSS, Canvas 2D, SVG, CompositingIE 9 Beta“all graphics and text rendering”97

Benchmarks NeededJSGameBench, Facebook“over 125 million people visit Facebook using HTML5 capable browsers just from their mobile phone”Sprite performance, options (WebGL, <canvas>, <img>, <div>, …)1000-4000 @30fps on GPU50-200 mid range desktops20-30 iOS & Android98

LibrariesCanvas 2DImpactJS, AkihabaraWebGLGLGE, C3DL, Copperlicht, SpiderGL, SceneJS, O3D, Processing.js and XB PointStream, WebGLUDocument Object Model (DOM)Rocket Engine, Zynga Germany (was Aves Engine)PhysicsBox2DWeb, Jiglib 3D99

What’s MissingMouse CaptureGamepadsAudio inputWebcam input100

More! At GDCGoogle Sessions Today and TomorrowCloud Services, WebGL, Native Client, YouTube APIsAndroid AndroidAndroidAndroidAndroidGoogle BoothSketchup,Web Store, WebGL Native Client, AppEngine, AndroidOther HTML5 SessionsWebGL News and Technology Updates (Khronos), ThursdayGetting Your Games onto the BlackBerry PlayBook Tablet, ThursdayHTML5: The New UI Library for Games, Friday101

ThanksAlexis Deveria, caniuse.com feature tablesAthena's Pix [flickr], overview imageChris PruettDaniel GalpinDarius KazemiGeoff BlairGregg TavaresMark DeLouraMark Pilgrim (diveintohtml5.org)Matt Hackett102

Q&A103code.google.com/games@GoogleGameDev

HTML5 and Other Modern Browser Game Tech

More Related Content

HTML5 and Other Modern Browser Game Tech

Editor's Notes