Implementing Comet using PHP

Implement Comet using PHP Dutch PHP Conference 2011 Jonas Mariën – King Foo www.king-foo.be

Jonas Mariën, King Foo www.king-foo.be Author “Zend Framework Web Services” (php|architect) Online since 93 (yep). PHP for fun since 96, professional since 2001. Master in Biological and Genetic Engineering, University of Leuven.

Comet = data push from server to client

Comet aka Reverse Ajax HTTP server Push Ajax Push HTTP streaming ... abused/umbrella term

Use case Example updates being pushed:
ticker info
news items
chat messages
status updates
e-mail notifications
in-app messaging
monitoring feedback
feedback on processes that take time to complete
...

Push? But, but ... a browser pulls and the server responds, no?
Fake push using polling trickery
Connection is kept open as long as possible
A giant hack
Does the trick for now

The Basics

Short polling
Client tries to get something new from the server and polls at regular and short intervals.
Responsiveness at the cost of performance (on the server side). Wasteful in terms of resources.
Easy, but not really what we are looking for.
Not considered 'comet'.

Short polling function getUpdate() { $.ajax({ type: "POST", url: "server.php", data: "type=news&category=PHP", success : function(msg){ $('#feed').append(msg);/** we do something **/ setTimeout("getUpdate()", 2000); } }); } $(document).ready(function() { getUpdate(); }

Short polling demo

The forever frame
An iframe keeps loading, never stops (or reloads occasionally)
Also called “streaming”
Dirty, e.g IE doesn't like this

The forever frame In the page: <script type="text/javascript">function updateResult(/** do something **/)</script>) <iframe style='width: 0px; height: 0px' src='./serverside.php'></iframe> On the server: <?php set_time_limit(0); header("Cache-Control: no-cache, must-revalidate"); header("Expires: Mon, 24 Jul 1974 05:00:00 GMT"); flush(); ob_flush();//if ob_start (implicitly) called before while(true) { $result = date('Y-m-d H:i:s'); echo '<script type="text/javascript">parent.updateResult(''.$result.'');</script>'; flush(); ob_flush(); $sleeptimer = rand(1, 15); sleep($sleeptimer); }

The forever frame demo

XHR
“XHR streaming”
Output 'busy' && sleep() on the server side untill finishing
We need to get the status before oncomplete/onsuccess
Note: there is also multipart/x-mixed-replace as part of XHR

XHR “ The XMLHttpRequest object can be in several states. The readyState attribute must return the current state, which must be one of the following values” * http://www.w3.org/TR/XMLHttpRequest/#states UNSENT (0) The object has been constructed. OPENED (1) The open() method has been successfully invoked. During this state request headers can be set using setRequestHeader() and the request can be made using the send() method. HEADERS_RECEIVED (2) All redirects (if any) have been followed and all HTTP headers of the final response have been received. Several response members of the object are now available. LOADING (3) The response entity body is being received. DONE (4) The data transfer has been completed or something went wrong during the transfer (e.g. infinite redirects).

XHR
Sine jQuery 1.5, no direct access to the underlying XHR object
No access to loading state
Can use a filter:
http://bugs.jquery.com/ticket/8327
http://jsfiddle.net/d8ckU/1/
Plugin: http://plugins.jquery.com/project/ajax-http-stream
Mixed results using different browsers for direct XHR manipulation

XHR xhr.onreadystatechange = function() { if(http.readyState == 3 && http.status == 200) { //do something document.write(xhr.responseText); } }

Long polling
Longer intervals
Each time something is returned from the server, a new request is started
Using XHR
Server: checks for updates for a few cycles and keeps a counter. If counter limit reached, 'false' or 'error' message goes back. If an update is available before that, exit the loop and return an 'ok' message with payload.

Long Polling function getUpdate(){ $.ajax({ type: "GET", url: "server.php", async: true, cache: false, timeout:50000, success: function(data) { /** do something **/ setTimeout ( 'getUpdate()', 1000 ); }, error: function(XMLHttpRequest, textStatus, errorThrown){ /**show error notice**/ setTimeout ( 'getUpdate()', "20000"); }, }); }; $(document).ready(function(){ getUpdate(); });

Long Polling <?php ... $twitterSearch = new Zend_Service_Twitter_Search('json'); $hashtag = 'dpc11'; if (isset($_GET['hashtag'])) { $hashtag = $_GET['hashtag']; } $result = false; while($result === false) { if (isset($_GET['since_id']) && $_GET['since_id'] > 0) { $since_id = (int) $_GET['since_id'] + 1; $searchResults = $twitterSearch->search($hashtag, array('lang' => 'en', 'rpp' => 30,'since_id' => $since_id)); } else { $searchResults = $twitterSearch->search($hashtag, array('lang' => 'en','rpp' => 30)); } if (isset($searchResults['results']) && count($searchResults['results'])) { $result = json_encode($searchResults['results'][0]); } } echo $result;

Long Polling demo

Thoughts & notes
Piggybacking
Long polling issues:
Requests are kept open for a while
Objects (other then the ones being pulled/pushed) on server side can be changed, need to have these changes fed to the client. Piggybacking to the rescue again?
One or more long polling requests open and then an AJAX call: might give you browser problems with simultaneous requests
Server performance: we need event driven solutions (message queues and/or something like node.js) to fix this
Proxies (chunking stuff) and firewalls (dropping long connections). Some Comet libs (next slides) try to detect intermediate proxies
Comet is not mature. Uses HTTP against most of the basic assumptions and no real standards available. Not a real standard.

Thoughts & Notes Performance is the biggest problem A Comet server should be able to handle as much connections as fast as possible, with a high troughput and low latency. Long polling requests stay idle for a while, so claiming a thread is not ok.

Frameworks/Toolkits

DWR
Direct Web Remoting (http://directwebremoting.org)
Nice, but Java only
Fallback different polling mechanisms
Dynamic Javascript generation
Attempts at PHP implementations, one could use something like json-rpc to mimic this
Could use a Java bridge from PHP perhaps?

Cometd & Bayeux
Dojo Foundation
Cometd: reference implementation (in Java, Perl and Python). Java implementation uses Jetty.
Bayeux: protocol, JSON for request/response
Handshake + Publish/Subscribe
Channels (/segment/channel-name)
/chat/publicroom
/chat/develepersonly
/chat/**
Now has some WS support too (haven't tested that, see http://cometdproject.dojotoolkit.org/node/124 and https://www.dojotoolkit.org/reference-guide/_static/js/dojox/socket/)
http://cometd.org/documentation/installation http://cometd.org/documentation/howtos/primer
Also, keep an eye on: http://opencoweb.org/

Bayeux Handshake [ { "channel": "/meta/handshake", "version": "1.0", "minimumVersion": "1.0beta", "supportedConnectionTypes": ["long-polling", "callback-polling", "iframe"] } ] [ { "channel": "/meta/handshake", "version": "1.0", "minimumVersion": "1.0beta", "supportedConnectionTypes": ["long-polling","callback-polling"], "clientId": "Un1q31d3nt1f13r", "successful": true, "authSuccessful": true, "advice": { "reconnect": "retry" } } ] http://cometdproject.dojotoolkit.org/documentation/bayeux

Bayeux
Connect/disconnect
Subscribe
[ { "channel": "/meta/subscribe", "clientId": "Un1q31d3nt1f13r", "subscription": "/chat/**" } ] [ { "channel": "/meta/subscribe", "clientId": "Un1q31d3nt1f13r", "subscription": "/chat/**", "successful": true, "error": "" } ]

Bayeux dojo.require("dojox.cometd"); dojox.cometd.init("http://example.com/cometd"); dojox.cometd.subscribe("/slideshow/change", function(comet) { $('#slideimg').attr('src', comet.data.src); }); }); .... dojox.cometd.publish("/slideshow/change", { 'src': src }); http://simonwillison.net/2007/Dec/5/comet/

Cometd/Bayeux Demo Download distribution from: http://downloads.dojotoolkit.org/cometd/ $ tar zxvf cometd-1.1.4.tgz $ cd cometd-1.1.4/ $ cd cometd-1.1.4/ $ cd cometd-demo $ mvn jetty:deploy-war http://localhost:8080/

Cometd/Bayeux

Phomet/Bayeux client
Use phomet to talk Bayeux to Cometd
$phomet = new Bayeux(’http://example.com/cometd’); $phomet->publish(’/chat/developersonly’, array( 'user' => 'jonas', 'message' => ‘bayeux is ... well ... euhm’)); http://morglog.alleycatracing.com/bayeux.class.inc.phps

WebSync
Commercial solution based on ASP.NET/IIS, also On-Demand
Comes with an OnDemand SDK for other languages, has some PHP sample code

APE
APE - Ajax Push Engine
Comet but they claim WebSockets support
Fallback
Written in C, has SpiderMonkey (js) bindings allowing for serverside modules in js
GPL
http://www.ape-project.org/
http://www.ape-project.org/wiki/index.php/How_to_write_an_application_with_APE

Kaazing
Kaazing.com
Gateway, handles Comet and WebSockets
Some background:
http://websocket.org/quantum.html
http://tech.kaazing.com/documentation/
Commercial, free dev license for testing

Kaazing

Kaazing demo

node.js

node.js git clone https://github.com/joyent/node.git cd node git tag -l git checkout v0.4.7 ./configure make make install

Socket.io

Socket.io curl http://npmjs.org/install.sh | sh npm install socket.io

Demo time

Want more?
socket.io server side: https://github.com/LearnBoost/Socket.IO-node
socket.io client side: https://github.com/LearnBoost/Socket.IO
http://fzysqr.com/2011/02/28/nodechat-js-using-node-js-backbone-js-socket-io-and-redis-to-make-a-real-time-chat-app/ -> the demo
http://thecoffman.com/2011/02/21/getting-your-feet-wet-with-node.js-and-socket.io/

NPM http://search.npmjs.org

Other solutions
nginx push module (http://pushmodule.slact.net/)
Meteor (http://meteorserver.org)
Freeliberator (http://www.freeliberator.com)
Tornado (http://www.tornadoweb.org)
Hookbox (https://github.com/mcarter/hookbox)
Jetty 6 / Grizzly / Tomcat 6
Rails: Juggernaut plugin
Atmosphere (http://atmosphere.java.net/)
Orbited (https://bitbucket.org/desmaj/orbited)
Noloh (has some push support)
LightStreamer
Pushlets
...

Alternatives
Webhooks
http://www.webhooks.org/
“HTTP callbacks using POST, for notification”
PubSubHubbub
http://code.google.com/p/pubsubhubbub/
“Server-to-server publish/subscribe using webhooks”
ReverseHTTP
http://reversehttp.net/
“Server in the browser”
WebSockets & Server-Sent Events

Message formats
JSON
STOMP
http://en.wikipedia.org/wiki/Streaming_Text_Orientated_Messaging_Protocol
http://framework.zend.com/manual/en/zend.queue.stomp.html
http://pecl.php.net/package/stomp
AMQP
http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol
http://pecl.php.net/package/amqp
http://riteshsblog.blogspot.com/2011/03/rabbitmq-adapter-for-zend-queue-using.html (uses AMQP and AMQP extension)
XMPP

any PHP based solutions?

PHP?
DIY - have a look at existing attempts
Phico (http://www.3site.eu/phico.php)
Simplecomet (http://mandor.net/2008/12/23/12-simplecomet-http-streaming-and-toilet-scrubbing)
Comep (http://sourceforge.net/projects/comet/files/)
Other?
No Bayeux/Cometd replacement in PHP, but doable if you really want to
Alternatives are available

PHP?
Write a daemon
Using nanoweb ?
Using Appserver-in-PHP ? (https://github.com/indeyets/appserver-in-php)
Use Phet (https://github.com/Tim-Smart/phet)
Use existing comet server solutions and add/read data using PHP
Use something like node.js + socket.io and think in terms of upcoming websocket support in all browsers
Try to implement a node.js alternative using http://php.net/libevent (which could be a really interesting solution)

Real-time web Real push
WebSockets
Server-Sent Events (SSE)

SSE Client: <!DOCTYPE html> <html> <head> <meta charset="utf-8" /> </head> <body> <script> var source = new EventSource('./server.php'); source.onmessage = function (event) { document.getElementById('output').innerHTML += event.data + '<br>'; }; </script> <div id="output"></div> </body> </html> Server: <?php header("Content-Type: text/event-streamnn"); echo 'data: ' . date('Y-m-d H:i:s') . "n"; http://googlecodesamples.com/html5/sse/sse.html

WebSockets http://code.google.com/p/phpwebsocket/ Another option: www.waterspout.com

WebSockets

Conclusions Push! Go for a solution that bridges the gap while waiting for websockets and SSE Do keep an eye on the real-time web evolution. It will be here very (very) soon and will impact your job.

Credits
“ Comet and Reverse Ajax: The next Generation Ajax 2.0” by Dave Crane and Phil McCarthy
www.html5rocks.com
www.cometdaily.com
Wikipedia of course
Plus: various links showed in the slides
Arrows: signsanddisplays.wordpress.com/page/2/
Comet image: http://www.igvita.com/2009/10/21/nginx-comet-low-latency-server-push/

Questions? Want us to work for you? [email_address] www.king-foo.be www.king-foo.net/blog @jonasmarien

Implementing Comet using PHP

by King Foo on May 21, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 3

Statistics

Implementing Comet using PHP — Presentation Transcript