How to make Ajax work for you

Slide 1: How to make work for you Simon Willison - http://simonwillison.net/ Web 2.0 Expo Berlin 5th November 2007

Slide 2: In this talk • Where Ajax came from • When you should (and shouldn’t) use it • Ajax fundamentals (including a DOM and JavaScript refresher) • Advanced Ajax techniques • Picking and using a JavaScript library

Slide 3: Please ask questions at any time

Slide 4: February 2005

Slide 5: AJAX v.s. Ajax “Asynchronous JavaScript + XML”

Slide 6: AJAX v.s. Ajax “Any technique that “Asynchronous allows the client to JavaScript + XML” retrieve more data from the server without reloading the whole page”

Slide 7: Why did Ajax happen? • Two reasons: • The browsers that mattered all added support for IE’s XMLHttpRequest object (which IE has had since 1998) • Jesse James Garrett came up with a name that sucked a lot less than “XMLHttpRequest” did

Slide 8: Pre-Ajax... • “Remote Scripting” (not as catchy) • Various unpleasant but ingenious hacks: • Scriptable Java applets • Passing data through changes to a cookie • Passing data through a hidden iframe • (That one’s actually still in use)

Slide 9: Ajax scenarios

Slide 20: Some anti-patterns • Drag and drop shopping carts • Massive page updates e.g. pagination • Unlinkable content (see also bookmarking) • Breaking browser expectations • Forgetting the loading icon!

Slide 21: Fundamentals

Slide 22: The three legged stool (X)HTML CSS JavaScript

Slide 23: The three legged stool The DOM (X)HTML CSS JavaScript

Slide 24: The DOM

Slide 25: The DOM

Slide 26: The DOM <html> <head> <body> <title> <div id=\"head\"> <div id=\"main\"> <h1> <p> <p> <p>

Slide 27: Key DOM methods • document.getElementsByTagName, document.getElementById • el.childNodes, el.parentNode, el.ﬁrstChild, el.nextSibling, el.getAttribute • el.appendChild, el.insertBefore, el.setAttribute, el.removeChild • document.createElement, document.createTextNode http://www.howtocreate.co.uk/tutorials/javascript/domstructure

Slide 28: JavaScript doesn’t suck! • Just remember; your; semicolons; • Avoid accidental globals: always declare your variables with “var” • Take the time to ﬁgure out higher order programming, where you treat functions as objects

Slide 29: CSS class switching • Often if you are dynamically making large- scale changes to the layout of a page... • ... you can do most of the work by deﬁning an alternative CSS class ... • ... and then switching a container element’s className property in your JavaScript

Slide 30: Firebug!

Slide 31: Firebug • Inspect HTML elements to see what CSS rules currently apply to them • Interactive console for experimenting with JavaScript against the loaded page • Full JavaScript debugger, including breakpoints • Proﬁle your code and your site’s assets and Ajax requests • Logging with console.log() and friends

Slide 32: XMLHttpRequest • The object that lets you make HTTP requests from within JavaScript • IE had it ﬁrst, using an ActiveX Object • Almost nothing to do with XML • Asynchronous (so you use callbacks)

Slide 33: XMLHttpRequest • The object that lets you make HTTP requests from within JavaScript • IE had it ﬁrst, using an ActiveX Object • Almost nothing to do with XML • Asynchronous (so you use callbacks)

Slide 34: var xhr = createXHR(); xhr.onreadystatechange = onComplete; xhr.open('GET', '/helloworld.txt', true); xhr.send(null); function onComplete() { if (xhr.readyState == 4) { if (xhr.status == 200) { alert(xhr.responseText); } else { alert('Error code ' + xhr.status); } } }

Slide 37: var xhr = createXHR(); xhr.onreadystatechange = function() { if (xhr.readyState == 4) { if (xhr.status == 200) { alert(xhr.responseText); } else { alert('Error code ' + xhr.status); } } }; xhr.open('GET', '/helloworld.txt', true); xhr.send(null);

Slide 38: function createXHR() { var xhr = null; if (window.XMLHttpRequest) { xhr = new XMLHttpRequest(); } else if (window.ActiveXObject) { try { xhr = new ActiveXObject('Microsoft.XMLHTTP'); } catch (e) {} } return xhr; }

Slide 39: function createXHR() { var xhr = null; if (window.XMLHttpRequest) { xhr = new XMLHttpRequest(); } else if (window.ActiveXObject) { try { xhr = new ActiveXObject('Microsoft.XMLHTTP'); } catch (e) {} } return xhr; }

Slide 40: var xhr = createXHR(); xhr.onreadystatechange = onComplete; xhr.open('POST', '/helloworld.php', true); xhr.setRequestHeader( 'Content-Type', 'application/x-www-form-urlencoded' ); xhr.send('name=Simon&age=26');

Slide 41: xhr.open('POST', '/helloworld.php', true); The third argument speciﬁes that we want to make an asynchronous request (“tell me when you’re done by calling this function”). If you set this to false you’ll get a synchronous request, which will completely hang the browser until the request returns. Don’t do that.

Slide 42: Too much boilerplate? • That’s quite a lot to remember • Best option: don’t remember any of it! • Use a good library instead • Lots more on those later on

Slide 43: doRequest doRequest(\"GET\", \"/blah.txt\", null, function(xhr) { doSomethingWith(xhr.responseText); }); doRequest(\"GET\", \"/getinfo.php\", { \"name\": \"Simon\" }, function(xhr) { doSomethingWith(xhr.responseText); }); doRequest(\"POST\", \"/getinfo.php\", { \"name\": \"Simon\" }, function(xhr) { doSomethingWith(xhr.responseText); });

Slide 44: function doRequest(method, url, args, callback) { var xhr = createXHR(); method = method.toUpperCase(); args = args && buildQueryString(args); xhr.onreadystatechange = function() { if (xhr.readyState == 4 && xhr.status == 200) { callback(xhr); } }; if (method == 'GET' && args) { url += '?' + args; args = null; } xhr.open(method, url, true); if (method == 'POST') { xhr.setRequestHeader( 'Content-Type', 'application/x-www-form-urlencoded' ); } xhr.send(args); }

Slide 45: function buildQueryString(obj) { var bits = []; for (var key in obj) { if (obj.hasOwnProperty(key)) { bits[bits.length] = escape(key) + '=' + escape(obj[key]); } } return bits.join('&'); } var s = buildQueryString({ name: 'Simon', age: 26 }); name=Simon&age=26

Slide 46: Data transport options

Slide 47: Client data submission • Requests can have anything you like in the body: • HTML, XML, JSON... • So if you really want to, you can access SOAP or XML-RPC services from JavaScript • For almost all purposes, regular form arguments sufﬁce • REST APIs are a good ﬁt as well

Slide 48: Server responses • Plain text • HTML fragments • CSV or similar • XML • Executable JavaScript • JSON

Slide 49: Plain text • Just sending back a simple value • xhr.responseText is all you need • Pros: • Absurdly simple • Cons: • Only supports a single value or string

Slide 50: HTML fragments • Return an HTML fragment, and inject it using innerHTML function onSuccess(xhr) { var div = document.getElementById('response'); div.innerHTML = xhr.responseText; } <div id=\"response\"> </div>

Slide 51: Fragment pros and cons • Pros: • Again, really simple • You can leverage existing server-side templates and techniques • Cons: • You have to change your server-side code to change client-side behaviour • You can't reuse your endpoint

Slide 52: Executable JavaScript • We can pass JavaScript code back from the server and execute it directly document.getElementById( 'response').innerHTML = 'Done!'; function onSuccess(o) { eval(o.responseText); }

Slide 53: • Pros: • Incredibly simple client-side code • The server can do abolutely anything, without needing to modify the client-side code • Cons: • Weird factoring of code - dynamically generating code to run on the client makes it hard to keep track of the big picture • How about just sending structured data?

Slide 54: CSV or similar • We want structured data! Simon Willison,26,simon@simonwillison.net • Pros: • Easy to parse: xhr.responseText.split(','); • Cons: • How do we pass values containing commas? • Index based, so brittle and hard to maintain

Slide 55: XML • Surely XML will save us! <person> <name>Simon</name> <age>26</age> </person> function onSuccess(xhr) { var dom = xhr.responseXML; showPerson( dom.getElementsByTagName('name')[0].data, parseInt(dom.getElementsByTagName('age')[0].data, 10) }); }

Slide 56: XML pros and cons • Pros: • We can re-use our DOM knowledge • xhr.responseXML parses it for us • We can call existing Web Services directly • Cons: • Verbose handling code • Is this the simplest thing that could possibly work?

Slide 57: JSON • A standard for representing common data structures, \"discovered\" by Douglas Crockford • Common data structures? • Associative arrays (dictionaries / hashes) • Arrays (lists) • Numbers, strings, booleans • A subset of JavaScript syntax

Slide 58: JSON example {\"name\": \"Simon\", \"age\": 26} function onSuccess(xhr) { var person = eval('(' + xhr.responseText + ')') ... )

Slide 59: More complex JSON {\"people\": [{ \"name\": \"Simon\", \"email\": \"simon@simonwillison.net\" }, { \"name\": \"Natalie\", \"email\": \"nat@natbat.net\" } ]} function onSuccess(xhr) { var people = eval('(' + xhr.responseText + ')'); ... }

Slide 60: To compare... <people> <person> <name>Simon</name> <email>simon@simonwillison.net</email> </person> <person> <name>Natalie</name> <email>nat@natbat.net</email> </person> </people>

Slide 61: To compare... function onSuccess(o) { var dom = o.responseXML; var people = []; var people_els = dom.getElementsByTagName('people'); for (var i = 0, el; el = people_els[i]; i++) { people[people.length] = { 'name': el.getElementsByTagName('name')[0].data, 'email': el.getElementsByTagName('email')[0].data }; } ... }

Slide 62: JSON v.s. XML JSON XML Bytes (excluding whitespace) 95 156 Lines of code to process 1 7+

Slide 63: It ain’t just for JavaScript • Libraries for consuming and generating JSON are available for many languages, including: • Python • Ruby • PHP • Java ... • JSON works really well as a general purpose tool for exchanging data structures

Slide 64: Aside: XSLT • Modern browsers support XSLT transforms - albeit with differing APIs • These can be signiﬁcantly faster than regular DOM manipulation • If you're having performance problems, this may be worth looking in to

Slide 65: Recommendations • For quick and simple apps, HTML fragments offer the path of least resistance • For most purposes, JSON makes the most sense • Use XML if you already have an XML Web Service that you want to consume (or you need an XSLT speed boost)

Slide 66: Let’s see some running code

Slide 67: Advanced Ajax topics

Slide 68: Cross-domain Ajax

Slide 69: The same-origin restriction • XMLHttpRequest is only allowed to communicate with the domain from which the page was loaded • The same is true of JavaScript calls between windows, frames and iframes • This is a critical browser security feature

Slide 70: The intranet attack • http://wiki.private.corp/ contains private data • User is tricked in to visiting evilexample.com • evilexample.com uses XMLHttpRequest (or a hidden iframe child) to load and steal data from http://wiki.private.corp/

Slide 71: • JSON-P offers a (hackish) solution to the cross-domain XMLHttpRequest restriction http://example.com/query?name=Simon callback({\"name\": \"Simon\", \"age\": 26}); function loadJSON(url) { var script = document.createElement('script'); script.type = 'text/javascript'; script.src = url; document.getElementsByTagName('head')[0].appendChild(script); } function callback(person) { ... } loadJSON('http://example.com/query?name=' + name);

Slide 72: • JSON-P offers a (hackish) solution to the cross-domain XMLHttpRequest restriction http://example.com/query?name=Simon callback({\"name\": \"Simon\", \"age\": 26}); function loadJSON(url) { var script = document.createElement('script'); script.type = 'text/javascript'; script.src = url; document.getElementsByTagName('head')[0].appendChild(script); } function callback(person) { ... } loadJSON('http://example.com/query?name=' + name);

Slide 73: Pros and cons • Pros: • Lets you call services hosted on other domains, provided they opt-in • del.icio.us, Flickr, other sites offer this • Cons: • Detecting errors is harder than with XMLHttpRequest • You have to trust the site who’s code you are executing

Slide 74: XSS

Slide 75: • XSS attacks occur when a malicious third party injects untrusted code (HTML, CSS or JavaScript) in to your page • It’s generally a server-side consideration, but you need to take it in to consideration when writing JavaScript as well • Ajax can compound the problem by allowing users to create self-propagating “worms”

Slide 76: CSRF

Slide 77: CSRF for GET <img src=\"http://example.com/admin/delete.php?id=5\"> Trick someone who is signed in to example.com/admin/ in to visiting a page hosting that image and you’ll force them to delete something.

Slide 78: CSRF for POST <form id=\"evil\" action=\"http://example.com/admin/delete.php\" method=\"POST\"> <input type=\"hidden\" name=\"id\" value=\"5\"> </form> <script> document.getElementById('evil').submit() </script>

Slide 79: CSRF protection • For regular forms, add a hidden ﬁeld with a one-time secret token and check for that token on every submission • For Ajax, either do the above or use: xhr.setRequestHeader( \"X-Requested-With\", \"XMLHttpRequest\" );

Slide 80: Unobtrusive JavaScript

Slide 81: Unobtrusive JavaScript • JavaScript isn't always available • Security conscious organisations (and users) sometimes disable it • Some devices may not support it (mobile phones for example) • Assistive technologies (screen readers) may not play well with it • Search engine crawlers won't execute it • Unobtrusive: stuff still works without it!

Slide 82: Progressive enhancement • Start with solid markup • Use CSS to make it look good • Use JavaScript to enhance the usability of the page • The content remains accessible no matter what

Slide 83: Adding events • Unobtrusive JavaScript relies on adding event handlers after the fact • The naive way of doing this: window.onload = function() { for (var i = 0, link; link = document.links[i]; i++) { if (link.className == 'external') { link.onclick = function() { window.open(this.href); return false; } } } };

Slide 84: Problem • If you later assign something else to window.onload (or one of the link.onclicks) you will clobber your behaviour • You should add events \"properly\"... • Different in different browsers • attachEvent v.s. addEventListener • My advice: use a library!

Slide 85: Unobtrusive examples

Slide 86: labels.js • One of the earliest examples of this technique, created by Aaron Boodman (now of Greasemonkey and Google Gears fame)

Slide 89: How it works <label for=\"search\">Search</label> <input type=\"text\" id=\"search\" name=\"q\"> • Once the page has loaded, the JavaScript: • Finds any label elements linked to a text field • Moves their text in to the associated text field • Removes them from the DOM • Sets up the event handlers to remove the descriptive text when the field is focused • Clean, simple, reusable

Slide 90: easytoggle.js • An unobtrusive technique for revealing panels when links are clicked <ul> <li><a href=\"#panel1\" class=\"toggle\">Panel 1</a></li> <li><a href=\"#panel2\" class=\"toggle\">Panel 2</a></li> <li><a href=\"#panel3\" class=\"toggle\">Panel 3</a></li> </ul> <div id=\"panel1\">...</div> <div id=\"panel2\">...</div> <div id=\"panel3\">...</div>

Slide 93: How it works • When the page has loaded... • Find all links with class=\"toggle\" that reference an internal anchor • Collect the elements that are referenced by those anchors • Hide all but the ﬁrst • Set up event handlers to reveal different panels when a link is clicked • Without JavaScript, links still jump to the right point

Slide 94: Django ﬁlter lists • Large multi-select boxes aren't much fun • Painful to scroll through • Easy to lose track of what you have selected • Django's admin interface uses unobtrusive JavaScript to improve the usability here

Slide 97: • Ajax is often used to avoid page refreshes • So... • Write an app that uses full page refreshes • Use unobtrusive JS to \"hijack\" links and form buttons and use Ajax instead • Jeremy Keith coined the term \"Hijax\" to describe this

Slide 98: More progressive Ajax suggestions • Live search / ﬁltering • Adding comments / tags • Smart form validation • Checking if usernames are already taken • You could even make a progressively enhanced chat room

Slide 99: The onload problem • All of these examples use code that runs when the window \"load\" event is ﬁred • Wait until page is loaded, then manipulate the DOM • Problem: If the page takes a while to load (large inline images) there will be a Flash Of Unstyled Content (FOUC) • Also, if the user clicks things before the setup code has ﬁred they won't get the expected behaviour.

Slide 100: document.write(css) • If the effect requires hiding some otherwise visible elements, you can document.write a stylesheet <script type=\"text/javascript\"> document.write('<style type=\"text/css\">'; document.write('.hideme { display: none }'); document.write('</style>'); </script>

Slide 101: body.onclick • If your effect is triggered by the user clicking on something, attach code to the document.body \"click\" event right at the start of your code (no need to wait for the document \"load\" event) YAHOO.util.Event.on(document.body, 'click' function(e) { var clicked_el = YAHOO.util.Event.getTarget(e); // ... }

Slide 102: onDOMReady • A number of attempts have been made to create an onDOMReady event that ﬁres when the DOM has been constructed but before the page has loaded • dean.edwards.name/weblog/2006/06/again/ • Modern libraries all support this in some form • Problem: CSS may not have loaded, so calculated dimensions may be incorrect

Slide 103: • Unobtrusive JavaScript is the Right Way to go about things, but runs in to browser differences even faster than regular JavaScript • Which leads us neatly on to...

Slide 104: Frameworks and Libraries

Slide 105: JavaScript libraries • ajaxpatterns.org lists over 40 general purpose JavaScript libraries • ... and that’s not including the many libraries tied to a speciﬁc server-side language • Why are there so many of them?

Slide 106: “The bad news: JavaScript is broken. The good news: It can be ﬁxed with more JavaScript!” Geek folk saying

Slide 107: • Inconsistent event model (thanks, IE) • Positioning and co-ordinates • Memory management (thanks, IE) • The DOM is a horrible API! • JavaScript-the-language has quite a few warts • But it’s powerful enough to let you ﬁx them • Classes and inheritance can be confusing • Many useful JS utility functions are missing • Drag and drop and Animation are really hard

Slide 108: Narrowing them down... • Prototype (and Scriptaculous) • The Yahoo! User Interface Library - YUI • jQuery • The Dojo Toolkit

Slide 109: Honourable mentions • MochiKit • No updates since early 2006 • The Google Web Toolkit • I don’t do Java • mooTools • Lots of buzz, but I haven’t ﬁgured out why yet

Slide 110: Download Get the latest version—1.5.1 Learn Prototype is a JavaScript Framework that aims to Online documentation and resources. ease development of dynamic web applications. Discuss Featuring a unique, easy-to-use toolkit for class-driven development and the nicest Ajax library around, Prototype Mailing list and IRC is quickly becoming the codebase of choice for web application developers everywhere. Contribute Submit patches and report bugs. Prototype and Scriptaculous Prototype and script.aculo.us: The \"Bungee book\" has landed! Who's using Prototype? Meet the developers Core team member Christophe Porteneuve has been hard at work for the past few months tracking

Slide 111: • Prototype focuses on basic browser compatibility and JavaScript language enhancement • Tries to make JavaScript more like Ruby • Extends most of JavaScript’s built-in objects with new functionality • Scriptaculous adds fancy effects, basic widgets and drag and drop

Slide 112: $$('#bmarks li').each(function(li){ Event.observe(li, 'click', function(e) { this.style.backgroundColor = 'yellow'; }.bindAsEventListener(li)); });

Slide 113: The Yahoo UI Library

Slide 114: • Created at Yahoo!, BSD licensed • Designed for both creating new applications and integration with legacy code • Focused on browser issues; almost no functionality relating to JS language itself • Extensively tested and documented

Slide 115: controls calendar container autocomplete menu slider treeview dragdrop animation dom event connection utilities

Slide 116: YAHOO.util.Event.on(window, 'load', function() { var div = YAHOO.util.Dom.get('messages'); setTimeout(function() { var anim = new YAHOO.util.Anim(div, { height: {to: 0}, opacity: {to: 0} }, 0.4); anim.animate(); anim.onComplete.subscribe(function() { div.parentNode.removeChild(div); }); }, 2000); });

Slide 117: Common YUI idiom $E = YAHOO.util.Event; $D = YAHOO.util.Dom; $E.on(window, 'load', function() { var div = $D.get('messages'); ... });

Slide 118: jQuery

Slide 119: jQuery(function($) { $(\"a.hideme\"). css('color', 'red'). click(function() { $(this).hide(\"slow\"); return false; }); });

Slide 120: • Simple philosophy: ﬁnd some nodes, then do something to them • Minimal impact on your global namespace - it adds two global symbols: jQuery and $, and $ can be easily reverted • API designed around “chaining” - other libraries are now emulating this • Outstanding node selection, based on CSS 3 and custom extensions • Small core library with an intelligent plugin mechanism

Slide 121: Dojo

Slide 122: • The oldest of the current popular libraries, pre-dating even Ajax • Incredible amounts of functionality • Used to suffer from a tough learning curve, although the 0.9 release simpliﬁes things greatly

Slide 123: Dojo components • dojo • Core library, similar to jQuery etc • Smart package managment with dynamic code loading • dijit • Advanced widget system • dojox • Dojo eXperimental - crazy voodoo magic

Slide 124: dijit

Slide 126: <div dojoType=\"dijit.layout.TabContainer\" sizeShare=\"40\"> <div id=\"tab1\" dojoType=\"dijit.layout.ContentPane\" title=\"Form Feel\"> <h2>Various Form Elements:</h2> <form name=\"dijitFormTest\"> <p><input type=\"checkBox\" dojoType=\"dijit.form.CheckBox\" checked=\"checked\"> Standard Dijit CheckBox <br><input type=\"checkBox\" dojoType=\"dijit.form.CheckBox\" disabled=\"disabled\"> Disabled Dijit <br> <input type=\"checkBox\" dojoType=\"dijit.form.CheckBox\" disabled=\"disabled\" checked=\"checked\"> Checked and Disabled Dijit </p> ...

Slide 127: dojox

Slide 128: • Graphics (cross-browser drawing API) • Ofﬂine storage • Cryptography • Templating • Data grids and more • “The future of the browser today”

Slide 130: My library selection criteria • Enables unobtrusive JavaScript • Plays well with other code • Smart use of namespacing • Global variable impact kept to a minimum • Tie breaker: the less code I have to write the better!

Slide 131: • I’m currently using and recommending jQuery for most situations • But... there’s cut-throat competition between the various libraries at the moment • This is one of the reasons I care about interoperability - commit to a single library and you might lose out when one of the others jumps ahead of it

Slide 132: The law of leaky abstractions

Slide 133: http://www.joelonsoftware.com/articles/LeakyAbstractions.html My interpretation: The more you rely on abstractions, the worse off you’ll be when one of them leaks

Slide 134: Thank you!

Slideshare.net (beta)