Building systems with Kamaelia

Kamaelia: SnapTogether Software Michael Sparks Senior Research Engineer BBC Research & Innovation Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

BBC R&I ... PAL, Teletext, NICAM, DVB, DAB, Freeview, TV Anytime Majority of BBC Open Source Projects inc Dirac, Kamaelia Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Key Research goal? To make concurrency easy to work with Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

How??? Single Reader/Single Writer Communicating Sequential Things Original context: scalable servers Component Composition Written in Python Ideas are language agnostic Proof of concept in C++ Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

How??? No, I don't believe in “one language to rule them all” *cough*Erlang*cough* Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

(demo) Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

ie this: Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

What I'm aiming to get through P2P Radio What? System Walkthrough Q&A Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

What? Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

What if... ? Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

What has it been used for? Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? Radio & Music Interactive Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? Matt Biddulph Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? Matt Biddulph Open Days “this is what I need” Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? Matt Biddulph Open Days “this is what I need” Record All Radio PVR Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? Matt Biddulph Open Days “this is what I need” Record All Radio PVR Podcasts Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? Matt Biddulph Open Days “this is what I need” Record All Radio PVR Podcasts Proof of concept Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? Matt Biddulph Open Days “this is what I need” Record All Radio PVR Podcasts Proof of concept Prototyping for experimentation for new services Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? Prototyping for experimentation for new services ... without any involvement from R&D Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? Prototyping for experimentation for new services ... without any involvement from R&D due to being open source Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? BBC Macro Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Blog frontend Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/ & Meta Store Programme bbc one transcoder Kamaelia Backend bbc two transcoder DEMUXER bbc three transcoder TUNER bbc four transcoder cbbc transcoder cbeebies transcoder Kamaelia news 24 transcoder parliament transcoder

Who's used it? Rapid Prototyping of Collaborative Community Radio Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Who's used it? DJs in the Channel Controller Community Control Streaming Networ Server k Mixer Playout Web Front End Linked by Music Store videoconf Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Google's Summer of Code Student Work 2006, 2007 Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Bit Torrent 2006 Programmes... WEBSITE Ingest BitTorrent Metadata Split Store Torrent Maker Web Client Torrent Peer Swarm Store Torrent Peer Access Programmes... Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Open GL 2006 Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

2006 Trusted Communications eg secure phone Decrypt ie trusted by the user Audio Decode Encrypt Decrypt BUFFER Encrypt Decrypt AudioPlayer Encrypt Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

2007 Filelike Interface > (Much!) simpler usage outside Kamaelia systems IRC/AIM Components > causes creation of components > allows remote control of systems Visual Component Generation > Sub component model > aim:easier creation of components visually (rather than systems) Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Other Systems Reframing for mobile, remuxing DVBT interactive aps for multicast streams, Whiteboarding, VoIP testing, etc. Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Walkthrough Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Simple Streamer Pipeline( ReadFileAdaptor( filename = 'video.drc', bitrate = 400000 ), SingleServer( ), ).activate() Pipeline( TCPClient( host = \"127.0.0.1\", port = 1601 ), DiracDecoder( ), MessageRateLimit( messages_per_second = 15, buffer = 15 ), VideoOverlay( ), ).run() Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Simple Presentation Tool Graphline( NEXT = Button(caption=\"Next\", msg=\"NEXT\", position=(72,8)), PREV = Button(caption=\"Previous\", msg=\"PREV\", position=(8,8)), FIRST = Button(caption=\"First\", msg=\"FIRST\", position=(256,8)), LAST = Button(caption=\"Last\", msg=\"LAST\", position=(320,8)), CHOOSER = Chooser(items = files), IMAGE = Image(size=(800,600), position=(8,48)), linkages = { (\"NEXT\",\"outbox\") : (\"CHOOSER\",\"inbox\"), (\"PREV\",\"outbox\") : (\"CHOOSER\",\"inbox\"), (\"FIRST\",\"outbox\") : (\"CHOOSER\",\"inbox\"), (\"LAST\",\"outbox\") : (\"CHOOSER\",\"inbox\"), (\"CHOOSER\",\"outbox\") : (\"IMAGE\",\"inbox\"), } ).run() Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Diving Inside? from Axon.ThreadedComponent import \\ threadedcomponent class ConsoleReader(threadedcomponent): def main(self): eol = \"\\n\" while 1: # this blocks so we use a thread line = raw_input(\">>> \") line = line + eol self.send(line, \"outbox\") Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Diving Inside? from Axon.Component import component class ConsoleEchoer(component): def main(self): while 1: while self.dataReady(\"inbox\"): data = self.recv(\"inbox\") _sys.stdout.write(str(data)) _sys.stdout.flush() yield 1 Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Used thus... Pipeline( ConsoleReader(), ConsoleEchoer(), ).run() Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Better Streamer def UserProtocol(): return ReadFileAdaptor( filename = 'video.drc', bitrate = 400000 ), SimpleServer( protocol=UserProtocol, port=1601 ).activate() Pipeline( TCPClient( host = \"127.0.0.1\", port = 1601 ), DiracDecoder( ), MessageRateLimit( messages_per_second = 15, buffer = 15 ), VideoOverlay( ), ).run() Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Minimal Radio P2P Core Backplane(“RADIO”).activate() Pipeline( TCPClient( host = \"192.168.1.1\", port = 1601 ), PublishTo( “RADIO” ), ).activate() Pipeline( SubscribeTo(“RADIO”), UnixProcess(“mplayer - “), ).activate() def UserProtocol(): return SubscribeTo( “RADIO” ), SimpleServer( protocol=UserProtocol, port=1601 ).run() Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Swarming P2P idea Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Swarming P2P setup class SimpleSwarm(Axon.Component.component): clients = [] rr = 0 maxclients = 4 def __init__(self, port): super(SimpleSwarm, self).__init__() self.__class__.port = port Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Swarming P2P server def main(self): while 1: while self.dataReady(\"inbox\"): data = self.recv(\"inbox\") req = data.split(\" \") if req[0] == \"REQCONNECT\": if len(self.clients) < self.maxclients: reqfrom = req[1] self.clients.append(reqfrom) self.send( \"CONNECT \" + str(self.port) , \"outbox\" ) else: reqconn = self.clients[self.__class__.rr] self.__class__.rr = (self.__class__.rr+1) % self.maxclients self.send( \"REQCONNECT \" + reqconn ) if not self.anyReady(): self.pause() yield 1 Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Swarming P2P client @staticmethod def clientRequest(rootip, rootport, myip, myport): servip, servport = rootip, rootport port = 0 while port == 0: l = LikeFile(TCPClient(servip, servport)) l.activate() l.send(\"REQCONNECT %s:%d\" % (myip, myport)) resp_raw = l.recv() resp = resp_raw.split(\" \") if resp[0] == \"CONNECT\": port = int(resp[1]) ip = servip elif resp[0] == \"REQCONNECT\": servip, servport = resp[1].split(\":\") servport = int(servport) return servip, port Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

P2P Radio – Client (was) Pipeline( TCPClient( host = \"192.168.1.1\", port = 1601 ), PublishTo( “RADIO” ), ).activate() Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Swarming Radio – Client Graphline(CONFIGURE= ConnectToSwarm(rootip, rootcontrolport, myip, mycontrolport), CLIENT = Carousel(mkTCPClient), PUBLISHTO = PublishTo(\"RADIO\"), linkages = { (\"CONFIGURE\",\"outbox\"):(\"CLIENT\", \"next\"), (\"CLIENT\", \"outbox\") : (\"PUBLISHTO\", \"inbox\"), } ).activate() Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Swarming Radio – Helpers class ConnectToSwarm(Axon.Component.component): def __init__(self, rootip, rootport, myip, myport): super(ConnectToSwarm, self).__init__() self.rootip = rootip self.rootport = rootport self.myip = myip self.myport = myport def main(self): servip, servport = SimpleSwarm.clientRequest( \\ self.rootip, self.rootport, self.myip, self.myport) yield 1 self.send( ( servip, servport ), \"outbox\") self.pause() yield 1 def mkTCPClient(args): return TCPClient(*args) Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Swarming Radio – Source freq = 682.166670 # Sutton Coldfields feparams = { \"inversion\" : dvb3.frontend.INVERSION_AUTO, \"constellation\" : dvb3.frontend.QAM_16, \"coderate_HP\" : dvb3.frontend.FEC_3_4, \"coderate_LP\" : dvb3.frontend.FEC_3_4, } Pipeline( DVB_Multiplex(freq, [6210], feparams), # RADIO ONE PublishTo(\"RADIO\"), ).activate() Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Kamaelia is already a highly capable platform that has delivered some interesting tools Currently version 0.5.0 Version reflects where we want to be Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

However it also shows that you can make concurrency easier to work with, even in a normal language Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

You can build your own version if you follow our “mini axon” tutorial using your language of choice Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Suggestions: Take it, use it Build cool stuff with it Break it, Fix it Steal the ideas Clone in your own language Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

Thanks for listening :) http://kamaelia.sf.net/ michael.sparks@bbc.co.uk Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/