Timeshift Everything, Miss Nothing - Mashup your PVR with Kamaelia

Timeshift Everything, Miss Nothing Mash up your PVR with Kamaelia Euro OSCON 2006, Brussels Michael Sparks, Senior Research Engineer, BBC Research

Prologue

Time shifting...
Enshrined as a right in UK Law
17A. - (1) The making in domestic premises for private and domestic use of a recording of a broadcast solely for the purpose of enabling it to be viewed or listened to at a more convenient time does not infringe any right conferred by Part 2 in relation to a performance or recording included in the broadcast.

Time shifting...
Enshrined as a right in UK Law
17A. - (1) The making in domestic premises for private and domestic use of a recording of a broadcast solely for the purpose of enabling it to be viewed or listened to at a more convenient time does not infringe any right conferred by Part 2 in relation to a performance or recording included in the broadcast.
TiVO*, Sky+, Goodmans, Grundig, Humax, Sagem, Topfield, Sharp, Sony, Thomson, Technosonic, etc...
* not sold in UK anymore

Kamaelia Macro

Kamaelia Macro
It records and transcodes what is broadcast over DTT for future viewing.

(video)

How do you build this?

Kamaelia Macro: Target DVB Multiplex outbox Channel Transcoder inbox DVB Demuxer BBC_ONE inbox Channel Transcoder inbox Channel Transcoder inbox Channel Transcoder Channel Transcoder Channel Transcoder inbox inbox inbox BBC_TWO BBC_THREE BBC_FOUR CBEEBIES CBBC

Kamaelia Macro: Status DVB Multiplex outbox Channel Transcoder inbox DVB Demuxer BBC_ONE inbox Channel Transcoder inbox Channel Transcoder inbox Channel Transcoder Channel Transcoder Channel Transcoder inbox inbox inbox BBC_TWO BBC_THREE BBC_FOUR CBEEBIES CBBC

Kamaelia Macro: Status Scaling up simply awaiting hardware DVB Multiplex outbox Channel Transcoder inbox DVB Demuxer BBC_ONE inbox Channel Transcoder inbox Channel Transcoder inbox Channel Transcoder Channel Transcoder Channel Transcoder inbox inbox inbox BBC_TWO BBC_THREE BBC_FOUR CBEEBIES CBBC

Top Level Schematic DVB Multiplex outbox Channel Transcoder inbox DVB Demuxer BBC_ONE inbox

Channel Transcoder Schematic Channel Transcoder EITDemux inbox _eit_ inbox EITParsing outbox inbox Carousel transcoder_factory inbox next outbox

Programme Identification Schematic EIT Parsing PSI Packet Reconstructor inbox inbox EIT Packet Parser outbox inbox Now Next service filter Now Next Changes outbox inbox outbox inbox outbox outbox

Programme Transcoding Schematic (a pipeline) Programme Transcoder Pipethrough (“mencoder -o file options...”) inbox control signal inbox control _transcodingcomplete _stop signal

But code isn't pretty web 2.0 esque boxes gradiented , it's never like that...

End of Prologue

Aims of Kamaelia

Aims of this Talk To stave off doom from buggy software on parallel machines

Aims of this Talk ahem

Aims of this Talk (nb, that's something that concerns me, Kamaelia is partly aimed at staving that off, I'm not claiming it does that – but it is one approach we're finding useful)

Aims of this Talk
Understand what Kamaelia is

Aims of this Talk
Take away our research results, and assimilate our code or approach in your systems

Aims of this Talk
To make it clear concurrency can be much simpler if you have the right tools

Aims of this Talk
Can be simpler than single threaded code

Aims of this Talk
To encourage you to think about using Kamaelia, and collaborating with us

Learning More: Kamaelia Open Space
What : kinda like a project's Foo/Bar Camp
Why : To help you get started, share knowledge etc,
When: This Friday 22 nd Sept, 11am-5pm
Where: Here, Brussels, The FoAM Offices
Koolmijnenkaai 30-34, 1080 Brussels
Details: http://tinyurl.com/f8szz
Map: http://tinyurl.com/hsarl

Aims of this Talk
To share an approach we find fun and useful

What ?

What ?
(demo)

What you've just seen.

What you've just seen Component outbox inbox

What you've just seen Component outbox inbox Component outbox inbox

What you've just seen Component outbox inbox Component outbox inbox Component outbox inbox

What you've just seen Component inbox Component Component outbox outbox | inbox outbox | inbox

What you've just seen What's inside the box? Component inbox Component Component outbox outbox | inbox outbox | inbox

Python Generators

Python Generators Fibonacci Demo

Component Basics

Component Basics Rotating Angle Demo “ turning a generator into a component”

Component Basics What use is this? class RotatingAngle(Axon.Component.component): def main(self): while 1: self.send(x, "outbox") x = x + 1 if x > 359: x = 0 yield 1

Component Basics Consider these components: class loopingCounter(Axon.Component.component) class cartesianPingPong(Axon.Component.component) class bouncingfloat(Axon.Component.component) class continuousIdentity(Axon.Component.component) class continuousZero(Axon.Component.component) class continuousOne(Axon.Component.component) (from Kamaelia.Automata.Behaviours...)

Component Basics Bouncing Cats

Component Basics Graphline( CAT = BasicSprite(image=cat), ROTATOR = loopingCounter(rotation_speed), MOVER = cartesianPingPong(position, 800, 600, 40), SCALER = bouncingFloat(scale_speed), IMAGER = continuousIdentity(cat), linkages = { ("ROTATOR","outbox" ) : ("CAT", "rotator"), ("MOVER","outbox" ) : ("CAT", "translation"), ("SCALER","outbox" ) : ("CAT", "scaler"), ("IMAGER","outbox" ) : ("CAT", "imaging"), } ).activate() From Examples/SimpleGraphicalApps/BouncingCatGame/Simplegame.py (shutdown fanout removed for space)

What's it good for?

Some stuff we've done Networked Audio Mixer Matrix Dirac/Vorbis distribution/decode/players Subtitling distribution server (mobiles, PCs) Presentation Tools Collaborative Whiteboards * with audio Simple games Visual programming tools Visual Introspection tools Kamaelia Macro ...

Stuff trainees have done pre-university trainee: Previewing PVR content on a mobile phone Shakespeare script parsing and doing something fun with events & data

Stuff trainees have done pre-university trainee: Previewing PVR content on a mobile phone Shakespeare script parsing and doing something fun with events & data Later adapted simply to have the characters talk the play on an IRC channel

Stuff trainees have done Summer trainee: (2 nd year student) Tools for simple reliable multicast Multicast island joining

Google Summer of Code Students
Ryan Lothian
Thomas Flanitzer
Devendra Laulkar
Anagha Mudigonda

Google Summer of Code
For us SoC was an opportunity to have components implemented that we've wanted for some time
Students implementing them appear to have had fun.
Also they've been guinea pigs to test our ease of use hypothesis
Large code chunks in our 0.5.0 release

For us SoC was an opportunity to have components implemented that we've wanted for some time
Students implementing them appear to have had fun.
Also they've been guinea pigs to test our ease of use hypothesis
Large code chunks in our 0.5.0 release
Hopefully uploaded to sourceforge today :-)

Components for:
Bit Torrent integration, a web server, web clients, IRC clients, icecast clients tone generation
3D systems – widgets, pygame component wrapping, 3D scribbling, simple image viewer
secure communications – including a nascent (choppy) secure phone.
prototype trusted communications tools
eg reporter trusting his comms to the BBC won't be changed without detection

Ease of Use
Now have 6 data points for ease of use
The youngest, most naïve students have produced the most complex and interesting systems
The older ones have generally had more baggage, and unless used to thinking about X talking to Y, to Z, have had more problems.
Even our weakest SoC student has produced interesting data

Stuff other parts of the BBC have done Radio & Music Interactive: Prototyped a system for recording all BBC radio output, transcoding it and making podcasts from the resulting data.

Stuff other parts of the BBC have done Radio & Music Interactive: Prototyped a system for recording all BBC radio output, transcoding it and making podcasts from the resulting data. Bootstrapped the final system rapidly

Stuff other parts of the BBC have done Radio & Music Interactive: Prototyped a system for recording all BBC radio output, transcoding it and making podcasts from the resulting data. Bootstrapped the final system rapidly Created the system without any involvement from us

Stuff other parts of the BBC have done Radio & Music Interactive: Prototyped a system for recording all BBC radio output, transcoding it and making podcasts from the resulting data. This directly led on to the Macro work as a result, including optimisations

Kamaelia Macro: Revisited

What Macro looks like inside
(yes, it really is boxes)

Top Level Schematic DVB Multiplex outbox Channel Transcoder inbox DVB Demuxer BBC_ONE inbox

Top Level Code Graphline( SOURCE=DVB_Multiplex(freq, pids["BBC ONE"]+pids["EIT"], feparams), DEMUX=DVB_Demuxer({ 610: ["BBCONE"], 611: ["BBCONE"], 18: ["BBCONE"], }), BBCONE_HI = ChannelTranscoder(service_ids["BBC ONE"], **params["HI"]), linkages={ ("SOURCE", "outbox"):("DEMUX","inbox"), ("DEMUX", "BBCONE"): ("BBCONE_HI", "inbox"), } ).run()

Channel Transcoder Schematic Channel Transcoder EITDemux inbox _eit_ inbox EITParsing outbox inbox Carousel transcoder_factory inbox next outbox

Channel Transcoder Code def ChannelTranscoder(service_id, mencoder_options, dir_prefix): def transcoder_factory(eit): return ProgrammeTranscoder(eit, mencoder_options, dir_prefix) return Graphline( PROG_CODER = Carousel( transcoder_factory ), EIT_PARSE = EITParsing(service_id), DEMUX = EITDemux(), linkages = { ("self","inbox") : ("DEMUX","inbox"), ("DEMUX","outbox") : ("PROG_CODER","inbox"), ("DEMUX","_eit_") : ("EIT_PARSE", "inbox"), ("EIT_PARSE", "outbox") : ("PROG_CODER", "next"), } )

Programme Identification Schematic EIT Parsing PSI Packet Reconstructor inbox inbox EIT Packet Parser outbox inbox Now Next service filter Now Next Changes outbox inbox outbox inbox outbox outbox

Programme Identification Code def EITParsing(*service_ids): return pipeline( PSIPacketReconstructor(), EITPacketParser(), NowNextServiceFilter(*service_ids), NowNextChanges(), )

Programme Transcoding Schematic (a pipeline) Programme Transcoder Pipethrough (“mencoder -o file options...”) inbox control signal inbox control _transcodingcomplete _stop signal

Programme Transcoding Code class ProgrammeTranscoder(Axon.Component.component): Inboxes = { "inbox" : "TS packets containing audio and video", "control" : "the usual", "_transcodingcomplete" : "signal from the transcoder", } Outboxes = { "outbox" : "", "_stop" : "stop the transcoder", "signal" : "the usual", } def __init__(self, eitdata, mencoder_options, dir_prefix): super(ProgrammeTranscoder,self).__init__() self.eitdata = eitdata self.mencoder_options =mencoder_options self.dir_prefix = dir_prefix def main(self): # first, generate unique filename uid = str(time.time()) encodingfile = "/data/encoding"+self.dir_prefix+"/"+uid+".avi" waitingEIT = "/data/encoding"+self.dir_prefix+"/"+uid+".eit" finishedfile = "/data/finished"+self.dir_prefix+"/"+uid+".avi" finishedEIT = "/data/finished"+self.dir_prefix+"/"+uid+".eit" print uid,"Starting transcoding into: "+encodingfile transcoder = Pipethrough("mencoder -o "+encodingfile+" "+self.mencoder_options) print uid,"Transcoder pipethough =",transcoder.name data_linkage = self.link( (self,"inbox"), (transcoder,"inbox"), passthrough=1 ) ctrl_linkage = self.link( (self,"_stop"), (transcoder,"control")) done_linkage = self.link( (transcoder,"signal"), (self,"_transcodingcomplete") )

Kamaelia Macro
Code is in the distribution is you want to play
(only works under Linux due to Linux having a uniform DVB-T API)

But Why Mashup?
A web mashup often takes data from multiple sources (websites) transforms them and republishes them somehow, allowing recomposition.
Kamaelia Components are essentially the same, except you're not mashupping websites but systems, including potentially your PVR.
It's “just” data.

Kamaelia: Ideas
Things you might want to do (to Macro and beyond)
3D user interface to the PVR
IRC based control (your PVR sitting on
an IRC channel)
Web integration (beyond blogs/rss)
Integrate common sense engine
An open alternative to Second Life
Video Conferencing
Build an open technology for BBC iPlayer?

Kamaelia Futures
We want to replace the need to write code when writing components.
Further optimisations
More concurrency mechanisms
More tutorials
(AKA Specfic stuff we want to do)

Kamaelia Futures
Collaborative Storyboarding/programme production tools
Open source iPlayer
Integrate PGU
Better Tk Components
PyQT, wxWindows integration
(AKA Specfic stuff we want to do)

Thank you! Thank you!

Kamaelia Open Space
This Friday 22 nd Sept, 11am-5pm
FoAM's Offices:
Koolmijnenkaai 30-34, 1080 Brussels
Map: http://tinyurl.com/hsarl
Possible discussions: Get started; integrate your favourite libraries; apply to teaching (gesture recognition); flesh out our system to be more relevant to your problem domain (as we do for the BBC)? Re-implement in C++, Ruby?
http://tinyurl.com/f8szz

http://www.kamaelia.org	285
http://edit.kamaelia.org	10
http://192.168.10.100	1
http://www.fachak.com	1

Timeshift Everything, Miss Nothing - Mashup your PVR with Kamaelia

by kamaelian on Oct 07, 2007

Accessibility

Categories

Tags

Upload Details

Usage Rights

4 Embeds 297

Statistics

Timeshift Everything, Miss Nothing - Mashup your PVR with Kamaelia — Presentation Transcript