VNC for web apps Laurent Denoue, Scott Carter, John Adcock, Gene Golovchinsky, Andreas Girgensohn FX Palo Alto Laboratory

What is WebNC? WebNC is Web Network Computing A lightweight framework for realtime webpage sharing

WebNC is Web Network Computing

A lightweight framework for realtime webpage sharing

Why WebNC? Need to share a web browser window co-browsing: husband and wife reserve a car online webcasting: demo-ing latest web ui to designer team tutorial: here is how you can reserve a car on our site link sharing: I found this great web site, look here, click there Need to view anywhere: no download, plugin, any device WebNC viewer is pure HTML/javascript works on iPhone low bandwidth, high quality Need to search and retrieve full-text search of recorded "WebNCasts"

Need to share a web browser window

co-browsing: husband and wife reserve a car online

webcasting: demo-ing latest web ui to designer team

tutorial: here is how you can reserve a car on our site

link sharing: I found this great web site, look here, click there

Need to view anywhere: no download, plugin, any device

WebNC viewer is pure HTML/javascript

works on iPhone

low bandwidth, high quality

Need to search and retrieve

full-text search of recorded "WebNCasts"

System overview

Sender side: firefox extension Grab screenshot of web page: canvas drawWindow Split into tiles (256x256 pixels) Compress each tile to JPEG and PNG and keep the smaller Hash each tile to create a small signature For each tile, if signature has changed: send to server Firefox extension also: gets viewport size, document size, scroll pos., mouse pos. sends this along with changed tiles Firefox extension also: addListeners to FORMS e.g. SELECT, DROP DOWN native C++ component (on Windows) grabs Flash, Applets, etc.

Grab screenshot of web page: canvas drawWindow

Split into tiles (256x256 pixels)

Compress each tile to JPEG and PNG and keep the smaller

Hash each tile to create a small signature

For each tile, if signature has changed: send to server

Firefox extension also:

gets viewport size, document size, scroll pos., mouse pos.

sends this along with changed tiles

Firefox extension also:

addListeners to FORMS e.g. SELECT, DROP DOWN

native C++ component (on Windows) grabs Flash, Applets, etc.

Using the Canvas HTML5 element “canvas” originally introduced by Apple Firefox extension “drawWindow” https://developer.mozilla.org/en/Drawing_Graphics_with_Canvas Draws a snapshot of the contents of a DOM window Only for code running with Chrome privileges Second canvas for image tile Use “drawImage” to copy a tile of the first canvas Use “toDataURI” to get compressed image data data:image/png;base64, iVBO…gg== Compare size of PNG and JPEG MD4 of first 256 bytes to detect changes Upload changed tiles

HTML5 element “canvas” originally introduced by Apple

Firefox extension “drawWindow” https://developer.mozilla.org/en/Drawing_Graphics_with_Canvas

Draws a snapshot of the contents of a DOM window

Only for code running with Chrome privileges

Second canvas for image tile

Use “drawImage” to copy a tile of the first canvas

Use “toDataURI” to get compressed image data data:image/png;base64, iVBO…gg==

Compare size of PNG and JPEG

MD4 of first 256 bytes to detect changes

Upload changed tiles

What Tile Size is Best? Compare different PNG tile sizes for Wikipedia page Larger tiles are much better with respect to total size Smaller tiles can deal more efficiently with local changes 90% overhead for 128x128 vs. 256x256 37% overhead for 256x256 vs. 512x512 Should be worth it if only a few tiles change 2 KB 88 KB 4 512x512 0.5 KB 78 KB 1 1024x1024 8 KB 116 KB 16 256x256 28 KB 208 KB 56 128x128 112 KB 317 KB 224 64x64 HTTP Headers Size #Tiles Tile

Compare different PNG tile sizes for Wikipedia page

Larger tiles are much better with respect to total size

Smaller tiles can deal more efficiently with local changes

90% overhead for 128x128 vs. 256x256

37% overhead for 256x256 vs. 512x512

Should be worth it if only a few tiles change

Server and Viewer For a given sessionid, server maintains in RAM: cursor position document size and viewport size and position each hash and corresponding byte[] of image tile HTML/Javascript viewer : Uses Ajax polling to get meta info and tile images as needed Creates a <div> sized to viewport size Creates <div> sized to document size (scrollable size) Adds <img> tiles at fixed positions inside document <div> Animates position of document <div> behind viewport <div> based on scroll position Animates cursor <img> to cursor position

For a given sessionid, server maintains in RAM:

cursor position

document size and viewport size and position

each hash and corresponding byte[] of image tile

HTML/Javascript viewer :

Uses Ajax polling to get meta info and tile images as needed

Creates a <div> sized to viewport size

Creates <div> sized to document size (scrollable size)

Adds <img> tiles at fixed positions inside document <div>

Animates position of document <div> behind viewport <div> based on scroll position

Animates cursor <img> to cursor position

Some Benchmarks Used WinMacro to automate record a 2.5 min web session Wireshark to measure network utilization Microsoft Remote Desktop: 225 kbps WebNC: 280 kbps UltraVNC: 512 kbps Microsoft SharedView: 729 kbps WebNC bandwidth requirements low enough to be accommodated by typical broadband connections. http://www.webnc.net

Used WinMacro to automate record a 2.5 min web session

Wireshark to measure network utilization

Microsoft Remote Desktop: 225 kbps

WebNC: 280 kbps

UltraVNC: 512 kbps

Microsoft SharedView: 729 kbps

WebNC bandwidth requirements low enough to be accommodated by typical broadband connections.

http://www.webnc.net

Other features Sharing is limited to a single browser tab Simple privacy Document layout structure can be overlayed in viewer enables iPhone double-tap zoom feature Document text can be included in the stream Enables sharing archives to be full-text searchable

Sharing is limited to a single browser tab

Simple privacy

Document layout structure can be overlayed in viewer

enables iPhone double-tap zoom feature

Document text can be included in the stream

Enables sharing archives to be full-text searchable