Scaling Streaming - Concepts, Research, Goals

Scaling BBC Streaming Concepts, Current Research, Future Goals, Michael Sparks BBC R&D [email_address]

Streaming at the BBC

Peak is around 40-50K concurrent streams

Both live & on-demand

Small audience by BBC standards

Want to enable larger services, cost effectively

Real, because it's cross platform

BBC prefers open standards/platforms

Caveat!

Very mixed talk!

Lots relating to current work

Some blue sky/where next

I'm aiming for high content ratio – means I might go fast

Slides contain: lots of text since mixed audience; illustrative code to show key points – not actually from main codebase

Caveat!

No really, this might be death by powerpoint

Even though powerpoint never came near this presentation, but hey

What is Scalability?

Practical definition:

Predictability and affordability when systems get large

Not just concerned with unit efficiency

However unit efficiency becomes important when you have scarce resources/big numbers

Common Techniques

Server Software:

At operating system level, optimise threading, processes, schedulers

Not portable by definition

In applications build their own concurrency primitives

State machines, event wheels, reactors

Can easily obfuscate the system

Areas Under Research

Alternative concurrency models

Modern language primitives

Alternative composition models

The world is filled with concurrent systems relatively simple to understand:

TVs, Radios

Clocks, Roads, Companies

Unix pipelines

Scaling & Software Cost

Writing out own testbed server

platform

Unencumbered Licensing:

(choice of LGPL, MPL, GPL)

To encourage take up of any experimental protocols, as per RFC

Allows embedding in PVRs and clients

Scaling & Software Cost

Dirac

Was in prototype stage when MPEG 4 costs initially bandied about (initially prohibitively high)

Cost free, high quality codec

Theora was pre-alpha stage at this point

Dirac and Theora complement each other, the more open codecs, the better

Scaling Network Systems

Get the network to distribute your content

We've been working to get multicast deployed by UK ISPs to allow us to serve larger audiences at better quality for lower cost

Having our own server allows integration with effective P2P

Long way off, but that's a goal

Kamaelia

Kamaelia is the name of our in-house testbed media server project

Uses a rather different architecture from other network servers

This talk covers it in as much detail as is practical in an hour

Other aspects to be covered

Protocol scalability

RTP is well designed for VoIP, not aimed primarily at unidirectional streaming

This causes problems

Is Streaming the right thing?

What sort of protocol do we really need?

Social aspects of architecture and enabling participation.

Scaling BBC Streaming

20 million homes, 700,000 hours of video/audio (approx)

Everyone watching something different?

Problems:

Bandwidth, number of servers, physical space, legal issues, costs

Will be here for long term if a success

What problem does streaming solve?

Bandwidth issues

Digital TV:

20000000*5Mbits = 100Tbit/s

Quality fairly predictable

Comfortable DSL:

20000000*370Kbit/s = 7.4 Tbit/s

Quality dependent on codec/content

No single location will work

Number of Servers

Assume servers are capable of flatlining their network card. (Rare)

Assume 1Gbit/s NICs

7.4Tbit/s / 1Gbit/s = 7400 servers

Physical Space Requirements

7400 servers

1U servers, 42U racks – 176 racks ~ 300 sq m

Archive would be about 15,000 Terabytes

Would fill a small building?

How practical?

This is without redundancy – bad, numbers are underestimates

Legal Issues

We don't own all rights outright

Rights need tracking and to be protected

Sample issues:

Geographical distribution agreements, repurposing for internet, background music, background pictures, finding people

Copyright will expire though – what then?

NB. I am not a lawyer, this is not legal advice!

Scaling Internet Costs

Software fees often related to audience size/number of servers

Can we grow service by 3 orders of magnitude?

Software fees increasing 1000 fold would be welcomed by vendors, but cripple ability to make programmes

Here to Stay

BBC TV is 5 decades old (or so)

BBC Radio is 8 decades old

Maintenance is key issue, designing for the long term

Fortran, Lisp, Cobol all 4-5 decades old and still used due to long term systems

Where are the proprietary languages of then?

Hidden Assumptions

Linear scalability at worst case

Moore's law & its variations continue to operate in our favour

Neither are guaranteed. If these fail, we still need to deal with it. Need to plan for outlandish optimisation ideas.

Measuring Cost

Lots of literature about measuring algorithm and system cost

3 key results

Same cost no matter size of problem

Cost is proportional to problem size (or cheaper)

Unit cost increases as problem size increases

Fourth important metric is scarce resouce

Cost/resource is defined as CPU cycles, memory, storage, money, etc

License & Standards “Costs”

F/OSS is fixed software cost for any problem size

Proprietary software is category 2

Custom/bespoke software might be category 3

Open standards are between 1& 2 generally. (MPEG4 is 2, HTTP is 1)

Category 1 is best for consumers

Protocol Cost – Example ICP

Internet Cache Protocol is used by web caches to query others for misses

Each request runs risk of 2s timeout

2 caches - 1 request to the other cache

10 caches - 9 requeststo the other caches

n caches - n-1 requests to the other caches

As you grow service, you add caches

Unit cost per request goes up with service growth

Protocol Cost: Example RTCP

Feedback mechanism in RTP

Scarce resource issue

Multicast session of 20M homes

20M*128bits alone.

If reporting every 5 minutes = 8Mbit/s being sent to everyone

If drop this down to 10% of dialup bandwidth, you get one report per user a week (useless)

So What?

Bandwidth costs will be offset if others redistribute our content

Better for us to figure out how to do this than someone else

Software costs will be offset of costs are closer to that of open source

Working on the this could help with the first point

What are we working on?

2 main projects are working on scaling distribution

Kamaelia is focused on the software aspects, and looking at architectures for long term scaling, not short term win

We set up private multicast peering with ISPs for the Olympics.

We leave the latter for a different time

Server Architectures

Lots of concurrent connections

Scalability often driven by concurrency choice

OS supported concurrency (threads/processes)

Manual concurrency handling (events etc)

Language concurrency primitves

Last two are logically equivalent

OS Concurrency

Used by Apache & Inetd

Scales fairly well - on some operating systems better than others

Code remains relatively clear looking

With high levels of concurrency context switching becomes a problem when dealing cross platform

Ruled out because it restricts OS, and portability

Manual Concurrency

Most commonly used approach for building scalable servers

Unless our approach pans out, use this for your servers!

Often state machine/event based

Problem is this is back to one step above “goto”

Ruled out because it makes joining & maintaining project potentially a lot harder

Language Based Concurrency

Not many languages support this approach

Hence not commonly used

Like manual concurrency risks lock up due to being co-operative multitasking

Allows your code to remain single threaded

Chosen because it simplifies joining & maintaining the project if provide a sensible framework for management

Kamaelia: Architecture

Single threaded, select based concurrency through language support

Concurrency wrapped up in components

Components only communicate via named boxes. Boxes are connected via linkages.

Components do not know who they are communicating with

Same idea as: Unix pipes, CSP, async hardware, hardware systems

Python Generators 1

These are used as the base concurrency mechanism. example: def fib(base1, base2): while True: yield base2 base1,base2 = base2, base1+base2

usage:

>>> fibs=fib(0,1) >>> for i in range(10): print fibs.next(), ... 1 1 2 3 5 8 13 21 34 55

Python Generators 2

More complex usage >>> fibs=[fib(0,i) for i in range(10)] >>> for i in range(10): ... print [fibs[i].next() for x in fibs] ... [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] [1, 1, 2, 3, 5, 8, 13, 21, 34, 55] [2, 2, 4, 6, 10, 16, 26, 42, 68, 110] ... snip ... [9, 9, 18, 27, 45, 72, 117, 189, 306, 495]

Note the inherent concurrency in generators

Python Generators 3

Look like normal functions, but actually return generator objects

yield x rather than return x

Calling gen.next() advances gen to the next yield, which returns value to caller

Essentially allows a “return and continue”

Multiple initialised generators are allowed, and essentially run concurrently

Generators – Issues, Solutions

No means of communication – only globals and initial arguments

Can only yield to immediate caller

Does mean that generators generally stay simple – which can some things

No default scheduler system

Solution: give generators context by wrapping them inside a class

Wrapping Generators

Many ways, this is simplest, we use a slightly more complex approach. Utility functon:

import copy def wrapgenerator(bases=object, **attrs): def decorate(func): class statefulgenerator(bases): __doc__ = func.__doc__ def __init__(self,*args): super(statefulgenerator, self) __init__(*args) self.func=func(self,*args) for k in attrs.keys(): self.__dict__[k] = copy.deepcopy(attrs[k]) self.next=self.__iter__().next def __iter__(self): return iter(self.func) return statefulgenerator return decorate

Wrapping Generators - 2

Suppose we wish to allow the following behaviour:

We communicate only with named queues

We allow components to nest, communicating only with their named queues

We can encapsulate this in a class and then use our utility function to apply it to the generator

Sample Behaviour

Details aren't important, just the fact this is simple to code:

class com(object): def __init__(_, *args): # Default queues _.queues = {"inbox":[],"control":[], "outbox":[], "signal":[]} def send(_,box,obj): _.queues[box].append(obj) def dataReady(_,box): return len(_.queues[box])>0 def recv(_, box): # NB. Exceptions aren't caught X=_.queues[box][0] del _.queues[box][0] return X

Sample Wrapped Generator

@wrapgenerator(com) def forwarder(self): "Simple data forwarding generator" while 1: if self.dataReady("inbox"): self.send("outbox",self.recv("inbox")) elif self.dataReady("control"): if self.recv("control") == "shutdown": break yield 1 self.send("signal","shutdown") yield 0

Essentials of Kamaelia's Components

Generators with named in/out-boxes

Linkages join outboxes to inboxes

A Scheduler

A postman for delivery of messages along linkages

Runs independently of components

A co-ordinating assistant tracker

Kamaelia: Building Communication

Linkages are tied to the substructure of a component, and are formed via a method call. The method call registers the linkage with the local postman. Example:

self.link((socket_handler,"outbox"), (protocol_handler,"inbox")) self.link((protocol_handler,"outbox"), (socket_handler,"inbox"))

Note we link component/box to component/box

Scheduling Generators

Our (simplified) scheduler is trivial: while(self.running): for mprocess in self.threads: if mprocess: try: result = mprocess.next() if self.newProcesses(result): self.activateProcesses(result): except StopIteration: self.shutdownProcess(result)

Postman Logic

The postman is a generator

One per component.

It gets shutdown when it's component is shutdown

Logic is trivial while 1: for link in self.linkages: if link.dataToMove(): link.moveData() yield 1

Co-ordinating Assistant Tracker

Provides book-keeping and service lookup

Services are (component, inbox) pairs

Consider biological systems

Nervous system is point to point

Non-specific comms is useful for certain tasks. Biological systems use hormones for this purpose.

Writing Servers Using Generators

We have components for:

Listening for new connections

Handling active connections

Protocol handling

Checking that network connections for activity

This can all be wrapped up as a component itself.

Code that follows lacks error checking

Note regarding Code Examples

Please don't try to read everything!

I'm going to largely whisk past them

The idea is to show the common structure that results and reuse it encourages

Furthermore it's designed to show that each subsection essentially remains looking single threaded despite an active server being highly concurrent

Listening for connections

@wrapgenerator(component, queues=["DataReady", "_csa_feedback"," protocolHandlerSignal", "signal"]) def TCPServer(self, listenport): self.listenport = listenport self.listener,junk = makeTCPServerPort(listenport, maxlisten=5) service, newlycreated = selectorComponent.getSelectorService(self.tracker) if newlycreated: self.addChildren(newSelector) self.link((self, "signal"),service) self.send(newServer(self, (self,self.listener)), "signal") yield newComponent(*(self.children)) while 1: Handle closed connection messages Handle new connection messages yield 1

Listening for connections 2

Handle closed connection messages

if self.dataReady("_csa_feedback"): data = self.recv("_csa_feedback") if isinstance( data, socketShutdown): theComponent,sock = self.closeSocket(data) self.send( shutdownCSA(self,(theComponent,sock)), "signal") self.removeChild(theComponent)

Listening for Connections 3

Handle new connection messages

if self.dataReady("DataReady"): data = self.recv("DataReady") CSA = self.mkConnectedSocket(self.listener) self.send(_ki.newCSA(self, CSA), "protocolHandlerSignal") self.addChildren(CSA) self.link((CSA, "FactoryFeedback"), (self,"_csa_feedback")) self.send(newCSA(CSA, (CSA,CSA.socket)), "signal") return CSA

Listening for Connections 4

Everything on one slide: (too small to read probably) @wrapgenerator(component, queues=["DataReady", "_csa_feedback","protocolHandlerSignal", "signal"]) def TCPServer(self, listenport): self.listenport = listenport self.listener,junk = makeTCPServerPort(listenport, maxlisten=5) service, newlycreated = selectorComponent.getSelectorService(self.tracker) if newlycreated: self.addChildren(newSelector) self.link((self, "signal"),service) self.send(newServer(self, (self,self.listener)), "signal") yield newComponent(*(self.children)) while 1: if self.dataReady("_csa_feedback"): data = self.recv("_csa_feedback") if isinstance( data, socketShutdown): theComponent,sock = self.closeSocket(data) self.send(shutdownCSA(self, (theComponent,sock)), "signal") self.removeChild(theComponent) if self.dataReady("DataReady"): data = self.recv("DataReady") CSA = self.createConnectedSocket(self.listener) self.send(_ki.newCSA(self, CSA), "protocolHandlerSignal") self.addChildren(CSA) self.link((CSA, "FactoryFeedback"),(self,"_csa_feedback")) self.send(newCSA(CSA, (CSA,CSA.socket)), "signal") return CSA yield 1

Handling active connections

@wrapgenerator(component, queues=["DataReady", "DataSend", "control", "outbox", "FactoryFeedback","signal"]) def ConnectedSocketAdapter(self, listenport): self.socket = listensocket while self.running: if self.dataReady("DataReady"): data = self.recv("DataReady") socketdata = _saferecv(self.socket, 1024) if (socketdata): self.send(socketdata, "outbox") if self.dataReady("DataSend"): data = self.recv("DataSend") _safesend(self.socket, data) if self.dataReady("control"): data = self.recv("control") if isinstance(data, producerFinished): self.send(socketShutdown(self,self.socket), "FactoryFeedback") self.send(socketShutdown(self), "signal")

Protocol handling

Depends on protocol. This implements an echo protocol. Note: we've seen this today!

@wrapgenerator(component) def EchoProtocol(self): "Simple data forwarding generator" while 1: if self.dataReady("inbox"): self.send("outbox",self.recv("inbox")) elif self.dataReady("control"): if self.recv("control") == "shutdown": break yield 1 self.send("signal","shutdown") yield 0

Select Based Connection Tracking

@wrapgenerator(AdaptiveCommsComponent, queues=["inbox", "control","notify","outbox","signal"]) def selectorComponent Initialisation. (not covered, not particularly interesting) while 1: handle notify messages readables, writeables, excepts = select.select( self.readersockets, self.writersockets, [],0) for sock in readables: passBackResult = handle readable socket (sock) if passBackResult: yield passBackResult for sock in writeables: passBackResult = handle writeable socket (sock) if passBackResult: yield passBackResult

Connection Tracking 2

handle notify messages if self.dataReady("notify"): msg = self.recv("notify") (managingComponent, sock) = msg.object if shutdown(message): self.writersockets.remove(sock) else: if message.handlesWriting(): self.writersockets.append(sock) else: self.readersockets.append(sock) self.wireInComponent((managingComponent, sock))

Connection Tracking 3

handle readable socket (feedbackInboxName, signalOutboxName, theComponent) = self.lookupBoxes[sock] self.send(status("data ready"),signalOutboxName)

handle writeable socket (feedbackInboxName, signalOutboxName, theComponent) = self.lookupBoxes[sock] self.send(status("write ready"),signalOutboxName)

Lots of detail omitted, esp error handling

Particularly omitted wiring in & unwiring new components mapped by sockets

Building a Generic Server

The example that follows is that of a generic server.

You provide a protocol component and port

It runs a server on that port passing over all data to the supplied protocol handler

This means to write a server, you just implement the protocol

You don't have to worry about transport at all

Building a Generic Server 2

@wrapgenerator(AdaptiveCommsComponent, queues=["_oobinfo"]) def SimpleServer(self, protocol, port=1601): self.protocolClass = protocol myPLS = _ic.TCPServer(listenport=port) self.link((myPLS,"protocolHandlerSignal"), (self,"_oobinfo")) self.addChildren(myPLS) yield newComponent(myPLS) while 1: result = handle messages received via _oobinfo if result: yield result

handle messages received via _oobinfo if self.dataReady("_oobinfo"): data = self.recv("_oobinfo") if newConnectedSocketAdaptor(data): return self.handleNewCSA(data) if socketShutdownMessage(data): self.handleClosedCSA(data)

handling closed connections

def handleClosedCSA(self,data): CSA = data.caller inboxes, outboxes, pHandler = self.retrieveTrackedResourceInformation(CSA) self.send("shutdown",outboxes[0]) self.removeChild(CSA) self.removeChild(pHandler) self.deleteOutbox(outboxes[0]) self.ceaseTrackingResource(CSA)

handling closed connections def handleNewCSA(self, data): CSA = data.object pHandler = self.protocolClass() pHandlerShutdownOutbox= self.addOutbox( "protocolHandlerShutdownSignal") self.trackResourceInformation(CSA, [], [pHandlerShutdownOutbox], pHandler) self.addChildren(CSA,pHandler) self.link((CSA,"outbox"),(pHandler,"inbox")) self.link((pHandler,"outbox"),(CSA,"DataSend")) self.link((CSA,"signal"),(self,"_oobinfo")) self.link((self,pHandlerShutdownOutbox), (pHandler, "control")) if "signal" in pHandler.Outboxes: self.link((pHandler,"signal"),(CSA, "control")) return newComponent(CSA,pHandler)

Interlude

The code on the preceding slides is the core code for a scalable server which can accept more or less any protocol

This makes it ideal and simple for playing with protocols

That's the idea behind the system – a little more work to make things easy for people

Essentially the entire architecture is plugins

Implementing A TCP Client

The real question is how much reuse can we get in a network client ?

Lets try!

Implementing A TCP Client 2

@wrapgenerator(AdaptiveCommsComponent, queues=["_oobinfo"]) def TCPClient(self,host,port): message = None try: sock = socket.socket(AF_INET, SOCK_STREAM); yield 0.3 sock.setblocking(0); yield 0.6 while not Safe Connect (self,sock,(self.host, self.port)): yield 1 yield newComponent(* Setup CSA (self,sock)) while Wait CSA Close (self): self.pause() ; yield 2 result = sock.shutdown(1) ; yield 3 sock.close() ; yield 4 except socket.error, e: message = e self.send(message, "signal")

Setup CSA

def setupCSA(self, sock): CSA = ConnectedSocketAdapter(sock) self.addChildren(CSA) selectorService , newSelector = getSelectorService(self.tracker)

if newSelector: self.addChildren(newSelector) self.link((self, "_selectorSignal"),selectorService) self.link((CSA, "FactoryFeedback"), (self,"_socketFeedback")) self.link((CSA, "outbox"), (self, "outbox"), passthrough=2) self.link((self, "inbox"), (CSA, "DataSend"), passthrough=1) self.send(newCSA(self, (CSA,sock)), "_selectorSignal") return self.childComponents()

Safe Connect (not so safe – error handling removed(!)...) def safeConnect(self, sock, *sockArgsList): try: sock.connect(*sockArgsList); self.connecting,self.connected=0,1 return True except socket.error, socket.msg: (errno, errmsg) = socket.msg.args if (errno==EALREADY or errno==EINPROGRESS): self.connecting=1 return False raise socket.msg

Wait CSA Close

if self.dataReady("_socketFeedback"):

message = self.recv("_socketFeedback")

if socketShutdownMessage(message):

return False

return True

Notes About the TCP Client

Not really that much simpler than a TCP server really

Corrollary: servers don't have to be hard

Note we reused without any changes:

ConnectedSocketAdaptor

Selector

Small components encourages reuse rather than duplication

Some Key Results

Individual components : Simple.

Complex systems via controlled composition

Replacing select with poll == simple

Replacing TCP with UDP similarly limits knockon changes

Single layer yield encourages small focussed components

Some Key Results

Protocol implementation can be done independent of network code

Provides a toolset for experimenting with protocols, transports

Very similar feel to building unix pipelines

Less Obvious Stuff

Owning the code allows us to ship the server in the client

Lightweight nature makes writing clients this way simple

Merging code from other projects written in python is relatively simple:

Add in a number of yields in strategic points

Add in support for communication/control

More Less Obvious Stuff

Interesting possibility:

Client tuner utility with embedded RTSP/RTP server and Bit Torrent client.

Cursory looks suggest that the component nature makes this a realistic possibility.

High decoupling: parallelism across threads, CPUs, machines practical.

Resulting structure lends itself to optimisation in hardware .

Kamaelia: Ongoing Work

Lots of things to do:

Integration of RTP/RTSP components

Stream oriented UDP

Implement better a/v delivery protocol

Optimisation of bottlenecks (linkages)

Use python features to remove cruft

Investigate Twisted “ integration” with

Lots more!

RTP is not a Streaming Protocol

RTP isn't designed for unidirectional streaming.

Does somethings that break hard in streaming

Misses out key things streaming protocols do

Stream thinning

Stream indexing

Define a mechanism for going over TCP.

RTP For Streaming

Everyone (?) who does open standards streaming and doesn't do HTTP does RTP

Why?

Because it's the only open standard that comes close to the requirements

So let's look at streaming

Audio/Video delivery?

Quickest way of delivery?

Digital Restrictions Management - preventing unauthorised copies?

Audio/Video delivery?

No, HTTP downloading handles that fine

Quickest way of delivery?

No, again, HTTP

Digital Restrictions Management - preventing unauthorised copies?

No – a lot of streaming is over HTTP

Characteristics of streaming?

Capped bitrate connection

Expected to stay at that capped bitrate

Play as you recieve

On demand, play now.

Time oriented seeking

Fine grained, but not conceptual - why not act, chapter, content oriented?

OK, an answer:

Timely delivery of content down pipes normally too small to take it

That's why we have capped bit rates

That's why we use compression.

Timely means people want to watch now, rather than wait for higher quality

Observations

High compression, packet loss == unwatchable video

Buffering common to mask transport errors

Devices have lots of local store

If bitrate of video < capacity of client link, why not switch to download faster and large local cache model? ie Actually switch protocols?

Observations 2

If bitrate >> capacity of link - why deny access - which not allow switch to maximum speed download to local cache model ?

ie allow people to choose quality or timeliness and then act accordingly

What about letting multiple source feed a single stream?

RTP doesn't really fit these models...

Good Transport Protocol Aspects

Content agnostic - you can send anything

Fully defines all aspects of usage

Simple to use

Changing content doesn't require understanding the content structure

P2P is good at this

HTTP is good at this

RTP sucks at this

RTP Problems (for streaming)

Half a transport protocol

Doesn't define how to send anything

Defines protocol profiles for lots of content types

Lots of content types missing containers

Vorbis, Speex, Theora, Dirac, ...

Vorbis has an (expired?) IETF draft

Not anyone's "fault" - protocol issue

Doesn't define framing over TCP

RTP Problems (for streaming)

Time based, not content based

RTCP has issues large multicast groups.

This is due to RTP not really having the concept of unidirectional sessions

Multicast has similar issues generally

Or you don't implement the protocol

Lots of features not needed in streaming

Designed for a/v conferencing

P2P Issues

Unknown quantity really. No, really.

Remember we're after how to serve ~ 7Tbit/s

How many 256Kbit/s DSL uplinks?

Can't do it just from the edge

Can't do it just from the centre

Legitimate Bit Torrent use – eg linux distros has proven moderate sized audience can work

Large numbers grabbing lots of things?

P2P Issues 2

How do ensure producers of art get paid?

We all want the "next Matrix"?

We all want the "next Really Cool Film/Song"

Films are a real problem – not a one man job, nor really suitable for open source style

Current approaches aren't working

Do we really want people breaking laws?

Do we really want to terrify the **AA down roads no-one wants to go?

P2P Issues 3

Copyright infringement isn't really a P2P issue

P2P makes it simpler because P2P is an effective user controlled distribution system

I grew up with Star Wars & Superman as a kid

My kids get LOTR & Spiderman

P2P has the power to ensure that your kids don't get their epics

P2P Issues 4

It's a social responsibility issue.

We can choose to define protocols that will adhere to rights.

Systems that look at creative commons licenses embedded in content are a GREAT start

Maybe new protocols could consider this?

Need for simpler Media Storage

RTP has to understand every media wrapper, and often codec details

Users can ask via RTSP for specific time indexes. This must be satisfied.

Parsing thousands of files at once is VERY expensive

Darwin streaming server hints the content

Patented by apple...

Simpler Media Storage

Rather than leave content unchanged, why not stuff it into a standard format?

Like usenet servers do

Like XML repositories

Like DNS defines

Having a shared approach makes reuse simpler

Note, I'm talking specifically for server use

Simpler Server Storage

One obvious approach:

Define in header “this is maximum bit rate”, “this is time granularity for index points”

Instantly have mapping of bytes and time

Pad chunks for VBR

Content agnostic

Could provide semantic -> time mapping

Very similar to IFF formats (19 years old)

Need for a New Streaming Protocol

Wish list (partial):

Transmission between a/v caches

Transmission mode migration

Downloading should allow swarming behaviour

Similarly for streaming where practical

Stream thinning

Semantic indexing as well as time

Mechanisms for arbitrary content distribution

Need for a New Streaming Protocol

Wish list (partial): (cont)

Provide mechanisms for arbitrary content distribution

Mechanisms similar to those in Scattercast would be useful when combined with P2P for a self organising media distribution network

Too large a topic for this talk

Copyright License

This presentation is part of the Kamaelia Project, and was presented at Firenze World Vision: Free Streaming 2004 session, organised by the Media Innovation Unit of Firenze Technologia