This document discusses the architecture for building an interactive radio station website that streams playlists and tracks in real-time to users. It will use Django for the backend, Twisted for the websocket server, SockJS for browser compatibility, and Celery to process tasks asynchronously like fetching album art. The radio station API exports playlists as XML files that will be uploaded to an FTP server and parsed by Django to populate the websocket stream.

1. Blending django, Sockjs,
Twisted, Celery, FTP
and some magic sauce
into a blazing fast
interactive website
Marc Egli

2. Concept

3. The Plan
Web Server
Browser
Radio Station API
Websocket
django
node.js?
twisted?
sockjs?
backbone?
?

4. The Plan
Web Server
Browser
Radio Station API
Websocket
django
twisted
sockjs
backbone
digimedia

5. The Websocket Part
• Twisted Application
• SockJS

6. Websocket Application
Browser
New Tracks API
Initialize
Channels
History

7. Digimedia
• exports the Radio Playlist as an XML File
• does this often
• is able to upload this to an ftp Server

8. <l1>
<Type>M</Type>
<Artist>Bruno Mars</Artist>
<Title>Treasure</Title>
<Label>Warner Music</Label>
<Duration>02:41</Duration>
<ID>P00DC64.dmf</ID>
<RefNr>175122</RefNr>
<timestamp>2013-09-11T23:34</timestamp>
</l1>
<p1>
<Type>J</Type>
<Artist>Jingle RW 111010</Artist>
<Title>ClocKer Shotgun</Title>
<Label></Label>
<Duration>00:07</Duration>
<ID>a01B3AF.dmf</ID>
<RefNr>111535</RefNr>
<timestamp>2013-09-11T23:34</timestamp>
</p1>
<p2>
<Type>M</Type>
<Artist>Mike Candys</Artist>
<Title>Everybody Ft. Evelyn &amp; Tony T</Title>
<Label>Phonag</Label>
<Duration>02:56</Duration>
<ID>a02B3BC.dmf</ID>
<RefNr>177084</RefNr>
<timestamp>2013-09-11T23:34</timestamp>
</p2>

9. FTP?
• watch on filechanges using inotify
• move them aside
• parse them and use Django to create the
corresponding TimelineElements
• push this to the twisted application
• pray to god that this doesn’t break

10. Damn it
• sometimes the file is sent too often
• we needed something which moved the file
aside just after the upload

11. Twisted again
• building an ftp Server using Twisted is easy
• this code is almost as old as Twisted and
therefore should be stable
• move the file aside and tell the django app
where it is

12. What is left for django?
• parse the xml file
• lookup album cover on iTunes
• push TimelineElements to the Twisted
application

13. Django App
XML Playlist
iTunes
lookup
FTP
Django
Celery
Push to API
Websocket App

14. Digimedia
API call
FTP Django Celery
Websocket
Service
Browser
iTunes
playlist upload
API call queue
API call
sockJS

15. Production Setup
• 3 VServers
• 2 HAProxys
• nginx
• 20 gunicorn workers
• redis
• rabbitMQ
• > 2500 rpm on django
• ~ 80 users online