Kamaelia:
             Pragmatic Concurrency
                            A Tutorial


                          Michael Sp...
About me
•   - Been using python for several years
•   - Always been interested in concurrency
    •   - Kamaelia aims to ...
About me

•


    Disclaimer: Like you, I'm doing this on my time,
    not the BBC's. This doesn't represent BBC
    opini...
About Kamaelia

•   - Also born from a desire to make concurrency
      in programs easier to work with
•
    - Because it...
About Kamaelia

•   - Adapts Unix Philosophy to make a program
     concurrent internally, but with the purpose of
     si...
About Kamaelia

•   - Adapts Unix Philosophy to make a program
     concurrent internally, but with the purpose of
     si...
What we're covering

•   - An overview of Kamaelia
         •   & it's view on concurrency
•   - How to build a mini Kamae...
Time Estimates
•   First part:
•   - An overview of Kamaelia (lightning talkstyle : 5-10 mins)
•   - How to build a mini K...
Format
•   I generally welcome questions at any point
•   That said...
    •   'Except in the overview
         •   - the ...
Caveat
•


•   * First time I've given this tutorial !
•   * If we run short of time on any section, we'll
    skip ahead....
Questions?

•   ... before we dive in?
•




    http://www.kamaelia.org/PragmaticConcurrency   sparks.m@gmail.com
Part 1
•



•   First part:
•   - An overview of Kamaelia (lightning talk style : 5-10
    mins)
•   - How to build a mini...
Kamaelia, a 20:20 overview



20:20 Presentation
      A presentation style based on the “Pecha Kucha”
      style 20 slid...
Why?                             Hardware finally going
                                  massively concurrent            ...
Missing Something?
Fundamental Control Structures
... in imperative languages number greater than 3! (despite what you get...
Regarding using
concurrency, what sort
of applications are we
 talking about here?




                                   ...
Think backend                                        Speak 'n Write
                            P2P Whiteboard Programming...
Core Approach:
 Concurrent things with comms points
 Generally send messages
 Keep data private, don't share



          ...
But I must share data?
Use Software Transactional Memory
ie version control for variables.
                           1. C...
Perspectives in APIs! (1/2)
    If you have concurrency it becomes
 natural to think in terms of 1st 2nd and 3rd
  person....
Perspectives in APIs! (2/2)
    If you have concurrency it becomes
 natural to think in terms of 1st 2nd and 3rd
  person....
Actor Systems
          Distinction can be unclear,
          potential source of ambiguity*             private real meth...
Advantages of outboxes
                                               No hardcoding of recipient
                         ...
A Core Concurrency DSL
Pipeline(A,B,C)
Graphline(A=A,B=B, C=C, linkages = {})
Tpipe(cond, C)
Seq(A,B,C), PAR(), ALT()
Back...
Pipeline Example
Pipeline(
    MyGamesEventsComponent(up="p", down="l", left="a", right="s"),
    BasicSprite("cat.png", n...
Graphline Example
Graphline(
  NEXT = Button(...),
  PREVIOUS = Button(...),                      PREVIOUS       NEXT
  FI...
Server Example

                            data
                            from
                            user        ...
Server Example

                                              You therefore
                                              ...
Server Example
from Kamaelia.Chassis.ConnectedServer import ServerCore
from Kamaelia.Util.PureTransformer import PureTrans...
Backplane Example
# Streaming Server for raw DVB of Radio 1
Backplane(“Radio”).activate()

Pipeline(
   DVB_Multiplex(850....
So that's the 5 minute version

                    Short Q&A before we move on?




http://www.kamaelia.org/PragmaticConc...
Part 1
•



•   First part:
•   - An overview of Kamaelia (lightning talk style : 5-10
    mins)
•   - How to build a mini...
Mini Axon

•   Kamaelia is divided into two halves
•   * One part handles all the concurrency
    stuff, providing you a c...
Mini Axon

•   Axon is the part that handles
    concurrency and provides the
    component model, and is the key to
    u...
Mini Axon

•   ... is a collection of exercises where
    you build just such a beast.




    http://www.kamaelia.org/Pra...
Mini Axon
 •



 •
     Generators
        * Python's smallest unit of concurrency
 •
     Microprocesses
        * Genera...
Mini Axon
 •



 •
     Generators
        * Python's smallest unit of concurrency
 •
     Microprocesses
        * Genera...
Generators
•   * Python's smallest unit of concurrency
•   * Single function you call, get a
      generator back
•   * Ca...
Fibonacci Generator
•
    def fib(a,b):
        while 1:
            yield a
            a, b = b, b + a
•




•
    Demo
...
Fibonacci Generator
•
    >>> def fib(a,b):
    ...     while 1:
    ...         yield a
    ...         a, b = b, b + a
 ...
Lots of Generators
•
    Using the same fib generator, make 10 of them:
    >>> GS = [ fib(x,x) for x in range(10) ]
•   A...
Generators as co-routines
•
    def fib(a,b):
        while 1:
            yield 1 # Just to say “keep running me”
       ...
Generators as co-routines
•
    def printer(tag):
        while 1:
            yield 1 # Makes it a generator
            ...
Mini Axon
 •



 •
     Generators
        * Python's smallest unit of concurrency
 •
     Microprocesses
        * Genera...
Microprocesses
•   * Generators with context
•




    http://www.kamaelia.org/PragmaticConcurrency   sparks.m@gmail.com
microprocess exercise
•
    Write a class microprocess with methods:
    •   __init__(self)
           * Takes no argument...
microprocess answer
•



•   class microprocess(object):
        def __init__(self):
            super(microprocess, self)...
microprocess usage
•
    class printer(microprocess):
        def __init__(self, tag):
            super(printer, self).__...
microprocess usage 2
•
    >>> X = printer("Something")
    >>> G = X.main()
    >>> X,G
    (<__main__.printer object at ...
Mini Axon
 •



 •
     Generators
        * Python's smallest unit of concurrency
 •
     Microprocesses
        * Genera...
Scheduler
•


    * Something to run lots of microprocesses
•




    http://www.kamaelia.org/PragmaticConcurrency   spark...
scheduler exercise
•
    Write a class scheduler with 3 methods:
    •   __init__(self)
          * Uses super to call sup...
scheduler exercise
•   Scheduler logic
    •
        main(self)
          * loops 100 times, yields 1 at start of loop
   ...
scheduler usage
•
    Using same printer class..
•


    >>> X = printer("Hello World")
    >>> Y = printer("Game Over") #...
Mini Axon
 •



 •
     Generators
        * Python's smallest unit of concurrency
 •
     Microprocesses
        * Genera...
Component
•


•   * microprocess with a standard interface.
•     boxes as inboxes/outboxes

•




    http://www.kamaelia...
component exercise
•
    Write a class component subclass of
    microprocess with 4 methods:
    •
        __init__(self)...
component exercise 1/4
•
    __init__(self) logic:
      * Add an attribute self.boxes, default value:
           { “inbox...
component exercise 2/4
•
    send(self,value, outboxname) logic:
    Finds the list named outboxname in self.boxes,
    an...
component exercise 3/4
•
    recv(self, inboxname) logic:
    Finds the list named inboxname in self.boxes, and
    pops t...
component exercise 4/4
•


    dataReady(self, inboxname) logic:
    Finds the list named inboxname in self.boxes:
    ret...
component usage
•



    Until we add a means for data to get
    from outboxes to inboxes, using
    components is no mor...
Mini Axon
 •



 •
     Generators
        * Python's smallest unit of concurrency
 •
     Microprocesses
        * Genera...
Postman
•   * Something to do deliveries
•


•   Note:
      This is more conceptual in real Axon,
     but was real in v....
postman exercise
•
    Write a class postman subclass of microprocess
    with 2 methods:
    •
        __init__(self, sou...
postman exercise 1/4
•


    __init__(self, source, sourcebox,
                                 sink, sinkbox) logic:
    ...
postman exercise 2/4
•


    main(self) logic:
•   In a loop:
    •   yield a non -1 value (eg 1)
        Check if source'...
Mini Axon
 •



 •   Using it!




     http://www.kamaelia.org/PragmaticConcurrency   sparks.m@gmail.com
Producer/Consumer 1/3
•



•   class Producer(component):
        def __init__(self, message):
            super(Producer,...
Producer/Consumer 2/3
•



•   class Consumer(component):
        def main(self):
            count = 0
            while ...
Producer/Consumer 3/3
•



•   p = Producer("Hello World")
    c = Consumer()
    postie = postman(p, "outbox", c, "inbox"...
Producer/Consumer Output
•



•   Hello World 2
    Hello World 3
    Hello World 4
•   ...
•
    Hello World 96
    Hello...
Mini Axon
 •



 •   Using it for more useful stuff




     http://www.kamaelia.org/PragmaticConcurrency   sparks.m@gmail...
File Reader
•



•   class FileReader(component):
•        def __init__(self, filename):
             super(FileReader, se...
Multicast sender 1/2
•
    class Multicast_sender(component):
      def __init__(self, laddr, lport, daddr, dport):
      ...
Multicast sender 2/2
•   class Multicast_sender(component):
      ... continued

      def main(self):
        sock = sock...
FileReader -> MulticastSender
•



•   reader = FileReader("Ulysses")
    sender = Multicast_sender("0.0.0.0", 0, "224.168...
Mini Axon --> Axon
 •



 •   The rest is:
       Syntactic Sugar
       Ways of using it
       Optimisations
          (...
Questions?
•


•   ... before we look how to build some
      real components, and how to use them?
•




    http://www.k...
Part 1
•



•   First part:
•   - An overview of Kamaelia (lightning talk style : 5-10
    mins)
•   - How to build a mini...
Building a Video Recorder
•



•   Before the break, we'll build a simple video recorder.
•   Approach:
•     1 Build a we...
Step 1: Build a simple Webcam
•   Pygame 1.9.1 alpha has basic Linux webcam support
    which works nicely, so we're using...
Step 1: Build a simple Webcam
•   The some definitions:
    •   displaysize = (800, 600)
        capturesize = (640, 480)
...
Step 1: Build a simple Webcam
•   Initialise the display, allocate a camera, and activate it
    •   display = pygame.disp...
Step 1: Build a simple Webcam
•   Loop round capturing images and display them
    •   while 1:
          snapshot = camer...
Step 2: Clean up for reuse
•   Given a collection of config options, wrapping this in a
    class makes sense.
•   As befo...
Step 2: Clean up for reuse
•   We then define a class, and put the config options as
    class attributes:
    •   class V...
Step 2: Clean up for reuse
•   Put our initialisation in the __init__. Allow the user to
    override our defaults

    de...
Step 2: Clean up for reuse
•   Wrap up body of loop in a method

    def get_and_flip(self):
      snapshot = self.camera....
Step 2: Clean up for reuse
•   Provide a hook to start it going, wrapping the main loop.

    def main(self):
      while ...
Step 3: Componentise
•


•   In this case, least effort approach is to use a threaded
    component. Would could make it a...
Step 3: Componentise
•


•   Imports change to

    import pygame
    import pygame.camera
    from Axon.ThreadedComponent...
Step 3: Componentise
•


•   Baseclass changes & initialiser changes:

    class VideoCapturePlayer(threadedcomponent ):
 ...
Step 3: Componentise
•


•   Extract display flipping out to a separate method:

    def pygame_display_flip(self):
      ...
Step 3: Componentise
•


•   Change to how we run:

•   We had:
•
         VideoCapturePlayer().main()
•   We now:
•      ...
Step 4: Split into input & output
•


•   Now we can split the component in two:
•     * VideoCaptureSource
•     * Surfac...
Step 4: Split into input & output
•


•   VideoCaptureSource class preamble:
    •     import time
          class VideoCa...
Step 4: Split into input & output
•


•   VideoCaptureSource initialiser
    •     def __init__(self, **argd):
           ...
Step 4: Split into input & output
•


•   Main loop body now just captures images, capturing a
    reference
    •
       ...
Step 4: Split into input & output
•


•   Main loop still runs it, and sends the image out an
    outbox, and then sleeps....
Step 4: Split into input & output
•


•   The surface displayer takes the other code chunks
    •     from Axon.Component ...
Step 4: Split into input & output
•


•   Has it's own initialiser...
•   def __init__(self, **argd):
      super(SurfaceD...
Step 4: Split into input & output
•   Then mainbody waits for surfaces to display, sleeping
    when there isn't any:
•   ...
Step 4: Split into input & output
•


•   We then join these back together in a pipeline:
•
    from Kamaelia.Chassis.Pipe...
Step 5: Recording
•   Working back...
•


•   We want to write to a file... (SimpleWriter)
•   We want to write dirac enco...
Step 5: Recording
•   Our imports therefore need to add:
•


•   from Kamaelia.File.Writing import SimpleFileWriter
•   fr...
Step 5: Recording
•   We then need to join it all together:
•
    Pipeline(
•
       VideoCaptureSource(),
       PureTran...
Step 5: Recording
•


•   This then records dirac encoded video, which we can now
    playback with a simple dirac player!...
Questions?
•   ... before we break?
•




    http://www.kamaelia.org/PragmaticConcurrency   sparks.m@gmail.com
End of Part 1
•


•   Before the second half, download and install Kamaelia
    from the link below, if you haven't alread...
Kamaelia:
             Pragmatic Concurrency
                            A Tutorial


                          Michael Sp...
Part 2
•


•   Second part:
•   - More advanced stuff (demo/etc 20 mins)
•   - Larger systems, embedding and debugging. (2...
Questions?
•   ... before we carry on?
•




    http://www.kamaelia.org/PragmaticConcurrency   sparks.m@gmail.com
Part 2
•


•   Second part:
•   - More advanced stuff (demo/etc 20 mins)
•   - Larger systems and debugging. (20 mins)
•  ...
More advanced Stuff
•   Have seen how to build a basic component and a basic
    system - covers a wide set of problems. N...
Well behaved Shutdown
•   Components should expect to receive these messages on their
    “control” inbox, and behave as f...
Components
•   Inheritable default values – we've already seen this!
•   class VideoCapturePlayer(threadedcomponent):
    ...
Components
•   Inheritable default values – other places:

    class ServerCore(component):
       port=1601
       protoc...
Components
•   Inheritable default values:

    class MailServer(ServerCore):
       port=25
       protocol=SMTPProtocol
...
Components
•   Inheritable default values - nests

    class MyReconfiguredMailServer(MailServer):
       port=25
       p...
Components
•   Inheritable default values – real usage – Kamaelia Grey
    class GreylistServer(MoreComplexServer):
      ...
Pipelines
•   Seen how Pipelines can join components together.
•   However also can do:
     •   Pipeline(
             Ls...
Pipelines
•   Transcode Engine
•   Pipeline(
        DirectoryWatcher(watch = conf["main_incoming_queue"]),
        ImageM...
Pipelines
•   Reusable Dirac Video Source:
•   def DiracWebCam():
       return Pipeline(
          VideoCaptureSource(),
...
Graphlines
•   Graphlines are like pipelines, but any shape. For
    example, a simple presentation tool:
•   Graphline(
 ...
Graphlines
•


•   * Swiss Army Knife of Kamaelia components
•   * Allows almost any topology
•     - BUT an outbox may on...
Graphlines
•   SMS Outbound processor
•   Graphline(
        SMS_SOURCE = Pipeline(
                         DirectoryWatc...
Graphlines
•   SMS Outbound processor (cont)
•       Linkages = {
             ("SMS_SOURCE","outbox") : ("SPLIT","inbox")...
Graphlines
•   Linkages to or from “self” (or “”) link to the Graphline's boxes.
•   Graphline(
        SPLIT = SomeSplitt...
Graphlines
•   Whiteboard using nesting graphlines:
•   Pipeline(
      chunks_to_lines(),
      lines_to_tokenlists(),
  ...
Graphlines
•   Whiteboard using nesting graphlines:
•           Linkages = {
                # incoming messages go to a r...
Backplanes
•


•   * For where you want 1 to many or many to many
•   * Declare a backplane
•   * Components can publish d...
Backplanes
•   Can be useful for updating a central state.
•


•   For example players locations can be posted here:
•   B...
Backplanes
•   Players Post their updates like this:
•   Pipeline(
•       MyGamesEventsComponent(up="p", down="l",
      ...
Backplanes
•   The system can analyse their updates like this:
•   Pipeline(
•       SubscribeTo("PLAYERS"),
•       Playe...
Backplanes & Servers
•   When combined with a server, you instantly get a pub/
    sub capable server, or splitter, or mer...
Backplanes & Servers
•   When combined with a server, you instantly get a pub/
    sub capable server, or splitter, or mer...
PAR & Seq
•   PAR & Seq are wrapper components with concepts
    nabbed from Occam. (hence their names)
•
    * Seq runs e...
Using PAR
•   PAR can be used to simplify some systems:
•   Pipeline(
        PAR(
             Button(caption="Next",    ...
Using PAR
•   ... to run & shutdown multiple subsystems
•   Pipeline(
      timedShutdown(TTL=15),
      PAR(
        Pipe...
Using PAR
•   ... to run & shutdown multiple subsystems
•      Pipeline(
          PAR(Button(caption="Next",     msg="NEX...
Using Seq
•   An example from the mobile reframer:
•   Seq( "Decoding & separating frames...",
         Graphline(
       ...
Using Seq
•   An example of passing state in a protocol handler:
•   def CompositeBulletinBoardProtocol(**argd):
      Con...
Part 2
•


•   Second part:
•   - More advanced stuff (demo/etc 20 mins)
•   - Larger systems and debugging. (20 mins)
•  ...
Larger Systems & Debugging
•



    Large systems which are long running can
    develop bugs which are awkward to debug.
...
Larger Systems & Debugging
•



    Key Tools:
•
    * Instrumentation using backplanes & logging
•
    * Axon Visualiser
...
Larger Systems & Debugging
•




•
    Practical examples.




    http://www.kamaelia.org/PragmaticConcurrency   sparks.m...
Part 2
•



•   Second part:
•   - More advanced stuff (demo/etc 20 mins)
•   - Larger systems and debugging. (20 mins)
• ...
Building a Bulletin Board
•



    We're going to build an “old school” bulletin board
    system.
•   * Someone telnets i...
Building a Bulletin Board
•



    For practicality, we won't implement message posting,
    nor message deletion, nor per...
Building a Bulletin Board
•
    Getting started:
•   * Start with basic server for request/response
       Just echo initi...
Building a Bulletin Board
•
    Intermediate Plan
•   This is the logic of the user protocol
•   Seq(
•
            Authen...
Building a Bulletin Board
•
    Intermediate Plan
•   * Tidy up control handling a touch more & add netPrint method
      ...
Building a Bulletin Board
•
    Final UI Plan
•   * Change usage of waitMsg to yield self.waitMsg() -- BB11.py
•   * Copy ...
Building a Bulletin Board
•
    Final UI Plan
•   * Change usage of waitMsg to yield self.waitMsg() -- BB11.py
•   * Copy ...
Summary
•
    We've covered
•   * the 30,000 view of kamaelia
•   * Building your own core
•
    * Building components, in...
Final Questions?
•


•




    http://www.kamaelia.org/PragmaticConcurrency   sparks.m@gmail.com
Thank you!
                                     ¬




http://www.kamaelia.org/PragmaticConcurrency   sparks.m@gmail.com
Upcoming SlideShare
Loading in …5
×

Kamaelia Europython Tutorial

3,150 views
3,038 views

Published on

This was the Kamaelia Tutorial at Europython. It goes from basics - ie building a mini-kamaelia from scratch, through to a file multicaster, through a video recording application all the walk through to a multiuser bulletin board system.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,150
On SlideShare
0
From Embeds
0
Number of Embeds
1,145
Actions
Shares
0
Downloads
82
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Kamaelia Europython Tutorial

  1. 1. Kamaelia: Pragmatic Concurrency A Tutorial Michael Sparks Europython '09, Birmingham UK http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  2. 2. About me • - Been using python for several years • - Always been interested in concurrency • - Kamaelia aims to embody safe practices. • - Work at BBC R&D's Northern Lab, based in Manchester, moving to Media City:UK in 2011 • - Kamaelia is born out of a variety of R&D projects, and shaped by needs, not aesthetics or purity. • Kamaelia is not an active R&D project. It is used in R&D projects, and hence actively maintained. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  3. 3. About me • Disclaimer: Like you, I'm doing this on my time, not the BBC's. This doesn't represent BBC opinion on anything. • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  4. 4. About Kamaelia • - Also born from a desire to make concurrency in programs easier to work with • - Because it's nice in the shell – I want software concurrency that easy :-) • - Expressiveness is favoured over performance, but not to preclude optimisation • - Means we're cautious about adding syntactic sugar. • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  5. 5. About Kamaelia • - Adapts Unix Philosophy to make a program concurrent internally, but with the purpose of simplifying maintenance. • Unix Philosophy: • Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. • --Doug McIlroy • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  6. 6. About Kamaelia • - Adapts Unix Philosophy to make a program concurrent internally, but with the purpose of simplifying maintenance. • Kamaelia Philosophy: • Write components that do one thing and do it well. Write components to work together. Write components to handle python object streams, because that is a universal interface. • --With apologies to Doug McIlroy • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  7. 7. What we're covering • - An overview of Kamaelia • & it's view on concurrency • - How to build a mini Kamaelia • – to get under the hood. • - Building components & systems • - Examining larger systems, and debugging. • - Building a large(ish) system http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  8. 8. Time Estimates • First part: • - An overview of Kamaelia (lightning talkstyle : 5-10 mins) • - How to build a mini Kamaelia (exercise 40-60 mins) • - Starting building components & systems (remainder) After break: • - More advanced stuff (demo/etc 20 mins) • - Larger systems and debugging. (20 mins) • - Building a large(ish) system (exercise 40 mins) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  9. 9. Format • I generally welcome questions at any point • That said... • 'Except in the overview • - the rest of this morning should answer them!! • Copious notes provided - covers more than today • Materials available from URL below • Is a mixture of “explain then do”. Perhaps hold questions for during “do” :-) ? http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  10. 10. Caveat • • * First time I've given this tutorial ! • * If we run short of time on any section, we'll skip ahead. • • * BUT, have copious notes that cover everything. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  11. 11. Questions? • ... before we dive in? • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  12. 12. Part 1 • • First part: • - An overview of Kamaelia (lightning talk style : 5-10 mins) • - How to build a mini Kamaelia (exercise 40-60 mins) • - Starting building components & systems (remainder) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  13. 13. Kamaelia, a 20:20 overview 20:20 Presentation A presentation style based on the “Pecha Kucha” style 20 slides, 20 seconds each (Similar to a lightning talk) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  14. 14. Why? Hardware finally going massively concurrent ... Opportunity! .... PS3, high end servers, trickling down to desktops, laptops) “many hands make light work” but Viewed as Hard ... do we just have crap tools? “And one language to in Problems the darkness bind them” ... can just we REALLY abandon 50 years of code for Erlang, Haskell and occam? http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  15. 15. Missing Something? Fundamental Control Structures ... in imperative languages number greater than 3! (despite what you get taught...!) Control Structure Traditional Abstraction Biggest Pain Points Sequence Function Global Var Selection Function Global Var Iteration Function Global Var Parallel Thread Shared Data Usually Skipped Lost or duplicate update are most common bugs http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  16. 16. Regarding using concurrency, what sort of applications are we talking about here? Desktop gsoc Media Novice APPS trainee Network 3rd Party http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  17. 17. Think backend Speak 'n Write P2P Whiteboard Programming needed for Simple ER DB Modeller (logo) youtube/flickr Kids Games P2P Radio Torrent type systems 3D Systems Compose Realtime Music UGC Desktop gsoc Paint App Backend P2P Web Server Secure “phone” Transcoder Social Net Vis Media Novice ... Shot Change Detection APPS trainee MiniAxon ScriptReader Mobile DVB MediaPreview Reframing on Mobile Reliable Macro Podcasts Network 3rd Party Multicast Sedna “record everything” XMPP XMLDB Email & AWS pubsub Spam Web (Amazon) SMTP IRC Qt Serving Gtk Greylisting Pop3Proxy ClientSide AIM microblogging Spam Tools http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  18. 18. Core Approach: Concurrent things with comms points Generally send messages Keep data private, don't share outbox inbox outbox inbox signal control signal control ... ... ... ... http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  19. 19. But I must share data? Use Software Transactional Memory ie version control for variables. 1. Check out the collection of values you wish to work on 2. Change them 3. Check them back in 4. If conflict/clash, go back to 1 http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  20. 20. Perspectives in APIs! (1/2) If you have concurrency it becomes natural to think in terms of 1st 2nd and 3rd person. This affects an API's structure, 1st Person - I change my state and can be vital for understanding it! This is one we've found that makes sense 2nd Person – YOU 3rd Person – want to me to do Bla should something do something (you send outbox (I send a message) inbox me a message) control signal ... ... http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  21. 21. Perspectives in APIs! (2/2) If you have concurrency it becomes natural to think in terms of 1st 2nd and 3rd person. This affects an API's structure, private real methods and can be vital for understanding it! This is one we've found that makes sense Messages Messages sent from public to public outboxes inboxes inbox outbox control signal Also, think ... ... Also, think about stdin about stdout http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  22. 22. Actor Systems Distinction can be unclear, potential source of ambiguity* private real methods Messages from public No outbox concept inboxes Possible issues with inbox rate limiting* control Hardcodes recipient ... in the sender *system dependent issue http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  23. 23. Advantages of outboxes No hardcoding of recipient allows: - Late Binding - Dynamic rewiring inbox outbox Concurrency Patterns as control signal Reusable Code ... ... ... a concurrency DSL http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  24. 24. A Core Concurrency DSL Pipeline(A,B,C) Graphline(A=A,B=B, C=C, linkages = {}) Tpipe(cond, C) Seq(A,B,C), PAR(), ALT() Backplane(“name”), PublishTo(“name”), SubscribeTo(“name”) Carousel(...) PureTransformer(...) StatefulTransformer(...) PureServer(...) MessageDemuxer(...) Source(*messages) NullSink Some of these are work in progress – they've been identified as useful, but not implemented as chassis, yet http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  25. 25. Pipeline Example Pipeline( MyGamesEventsComponent(up="p", down="l", left="a", right="s"), BasicSprite("cat.png", name = "cat", border=40), ).activate() MyGames Events Component Basic Sprite http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  26. 26. Graphline Example Graphline( NEXT = Button(...), PREVIOUS = Button(...), PREVIOUS NEXT FIRST = Button(...), (button) (button) LAST = Button(...), CHOOSER = Chooser(...), IMAGE = Image(...), FIRST LAST (button) (button) ... ).run() Chooser Image http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  27. 27. Server Example data from user data Main Socket handler Server Core to user Created at runtime to handle the Protocol Handler Factory connection Remote Protocol handler User http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  28. 28. Server Example You therefore need to provide Main this bit. Server Core Protocol Handler Factory http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  29. 29. Server Example from Kamaelia.Chassis.ConnectedServer import ServerCore from Kamaelia.Util.PureTransformer import PureTransformer def greeter(*argv, **argd): return PureTransformer(lambda x: "hello" +x) class GreeterServer(ServerCore): protocol=greeter port=1601 GreeterServer().run() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  30. 30. Backplane Example # Streaming Server for raw DVB of Radio 1 Backplane(“Radio”).activate() Pipeline( DVB_Multiplex(850.16, [6210], feparams), # RADIO ONE PublishTo("RADIO"), ).activate() def radio(*argv,**argd): return SubscribeTo(“RADIO”) ServerCore(protocol=radio, port=1600).run() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  31. 31. So that's the 5 minute version Short Q&A before we move on? http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  32. 32. Part 1 • • First part: • - An overview of Kamaelia (lightning talk style : 5-10 mins) • - How to build a mini Kamaelia (exercise 40-60 mins) • - Starting building components & systems (remainder) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  33. 33. Mini Axon • Kamaelia is divided into two halves • * One part handles all the concurrency stuff, providing you a component model • * The other is a large collection of components and some apps using them. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  34. 34. Mini Axon • Axon is the part that handles concurrency and provides the component model, and is the key to understanding why & how Kamaelia works. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  35. 35. Mini Axon • ... is a collection of exercises where you build just such a beast. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  36. 36. Mini Axon • • Generators * Python's smallest unit of concurrency • Microprocesses * Generators with context • Scheduler * Something to run lots of microprocesses • Components * A microprocess with boxes (treated as in & outboxes) • Postman * Something to do deliveries http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  37. 37. Mini Axon • • Generators * Python's smallest unit of concurrency • Microprocesses * Generators with context • Scheduler * Something to run lots of microprocesses • Components * A microprocess with boxes (treated as in & outboxes) • Postman * Something to do deliveries http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  38. 38. Generators • * Python's smallest unit of concurrency • * Single function you call, get a generator back • * Can then call it's .next() method to * Get a new value from it * Give it CPU time http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  39. 39. Fibonacci Generator • def fib(a,b): while 1: yield a a, b = b, b + a • • Demo http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  40. 40. Fibonacci Generator • >>> def fib(a,b): ... while 1: ... yield a ... a, b = b, b + a ... >>> g = fib(1,1) >>> g <generator object at 0xb7bf9c6c> >>> [ g.next() for _ in range(10) ] [1, 1, 2, 3, 5, 8, 13, 21, 34, 55] • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  41. 41. Lots of Generators • Using the same fib generator, make 10 of them: >>> GS = [ fib(x,x) for x in range(10) ] • And “run” them: >>> [ G.next() for G in GS ] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> [ G.next() for G in GS ] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> [ G.next() for G in GS ] [0, 2, 4, 6, 8, 10, 12, 14, 16, 18] >>> [ G.next() for G in GS ] [0, 3, 6, 9, 12, 15, 18, 21, 24, 27] >>> [ G.next() for G in GS ] [0, 5, 10, 15, 20, 25, 30, 35, 40, 45] • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  42. 42. Generators as co-routines • def fib(a,b): while 1: yield 1 # Just to say “keep running me” print a a, b = b, b + a • • Demo http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  43. 43. Generators as co-routines • def printer(tag): while 1: yield 1 # Makes it a generator print tag • • Demo http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  44. 44. Mini Axon • • Generators * Python's smallest unit of concurrency • Microprocesses * Generators with context • Scheduler * Something to run lots of microprocesses • Components * A microprocess with boxes (treated as in & outboxes) • Postman * Something to do deliveries http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  45. 45. Microprocesses • * Generators with context • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  46. 46. microprocess exercise • Write a class microprocess with methods: • __init__(self) * Takes no arguments * Uses super to call superclass __init__ method • main(self) * No arguments * Should yield 1 • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  47. 47. microprocess answer • • class microprocess(object): def __init__(self): super(microprocess, self).__init__() def main(self): yield 1 • Generally, we'll skip answers in this presentation, they're all in the web version of this tutorial here: • http://www.kamaelia.org/MiniAxon/ • (and in the notes :-) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  48. 48. microprocess usage • class printer(microprocess): def __init__(self, tag): super(printer, self).__init__() self.tag = tag def main(self): while 1: yield 1 print self.tag • • “Printer” isn't particularly interesting, but allows things like components, but let's see how this can be used. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  49. 49. microprocess usage 2 • >>> X = printer("Something") >>> G = X.main() >>> X,G (<__main__.printer object at 0xb7bfd2ac>, <generator object at 0xb7bfd4cc>) >>> G.next() 1 >>> G.next() Something 1 >>> X.tag = "Something else" >>> G.next() Something else 1 http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  50. 50. Mini Axon • • Generators * Python's smallest unit of concurrency • Microprocesses * Generators with context • Scheduler * Something to run lots of microprocesses • Components * A microprocess with boxes (treated as in & outboxes) • Postman * Something to do deliveries http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  51. 51. Scheduler • * Something to run lots of microprocesses • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  52. 52. scheduler exercise • Write a class scheduler with 3 methods: • __init__(self) * Uses super to call superclass __init__ method * subclasses microprocess * Creates 2 queues – active & newqueue • main(self) * Is a generator * Runs the microprocesses activated • activateMicroprocess(self, someprocess): * Calls somprocess.main() * Adds generator to newqueue • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  53. 53. scheduler exercise • Scheduler logic • main(self) * loops 100 times, yields 1 at start of loop * loop through generators in self.active * call their .next() method * If result is not -1 and no StopIteration, append to newqueue * at end of loop, newqueue becomes active * newqueue reset to empty list • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  54. 54. scheduler usage • Using same printer class.. • >>> X = printer("Hello World") >>> Y = printer("Game Over") # :-) • >>> myscheduler = scheduler() >>> myscheduler.activateMicroprocess(X) >>> myscheduler.activateMicroprocess(Y) • >>> for _ in myscheduler.main(): ... pass • <prints Hello world/Game over repeatedly> http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  55. 55. Mini Axon • • Generators * Python's smallest unit of concurrency • Microprocesses * Generators with context • Scheduler * Something to run lots of microprocesses • Components * A microprocess with boxes (treated as in & outboxes) • Postman * Something to do deliveries http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  56. 56. Component • • * microprocess with a standard interface. • boxes as inboxes/outboxes • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  57. 57. component exercise • Write a class component subclass of microprocess with 4 methods: • __init__(self) send(self, value, outboxname) recv(self, inboxname) dataReady(self, inboxname) • Behaviour coming up! http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  58. 58. component exercise 1/4 • __init__(self) logic: * Add an attribute self.boxes, default value: { “inbox”: [], “outbox”: [] } • * Ensure you call the superclass __init__ method appropriately • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  59. 59. component exercise 2/4 • send(self,value, outboxname) logic: Finds the list named outboxname in self.boxes, and appends value to it. • Before: X.send(“hello”, “outbox”) { “inbox”: [], “outbox”: [] } • After: { “inbox”: [], “outbox”: [“hello”] } • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  60. 60. component exercise 3/4 • recv(self, inboxname) logic: Finds the list named inboxname in self.boxes, and pops the first value • Given: { “inbox”: [“hello”, “world”], “outbox”: [] } • self.recv(“inbox”) returns, “hello” leaving... { “inbox”: [“world”], “outbox”: [] } • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  61. 61. component exercise 4/4 • dataReady(self, inboxname) logic: Finds the list named inboxname in self.boxes: returns the length of the list. • Given: { “inbox”: [“hello”, “world”], “outbox”: [] } • dataReady(“inbox”) --> 2 (allows if self.dataReady(“inbox”) ) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  62. 62. component usage • Until we add a means for data to get from outboxes to inboxes, using components is no more interesting than microprocesses http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  63. 63. Mini Axon • • Generators * Python's smallest unit of concurrency • Microprocesses * Generators with context • Scheduler * Something to run lots of microprocesses • Components * A microprocess with boxes (treated as in & outboxes) • Postman * Something to do deliveries http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  64. 64. Postman • * Something to do deliveries • • Note: This is more conceptual in real Axon, but was real in v. old Axon. ie real Axon is more efficient! http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  65. 65. postman exercise • Write a class postman subclass of microprocess with 2 methods: • __init__(self, source, sourcebox, sink, sinkbox) main(self) • Behaviour coming up! http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  66. 66. postman exercise 1/4 • __init__(self, source, sourcebox, sink, sinkbox) logic: * Copy all the arguments as local attributes • * Ensure you call the superclass __init__ method appropriately! • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  67. 67. postman exercise 2/4 • main(self) logic: • In a loop: • yield a non -1 value (eg 1) Check if source's sourcebox has dataReady • If it has, use recv to collect it from there, and sink's send method to deliver it. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  68. 68. Mini Axon • • Using it! http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  69. 69. Producer/Consumer 1/3 • • class Producer(component): def __init__(self, message): super(Producer, self).__init__() self.message = message • def main(self): while 1: yield 1 self.send(self.message, "outbox") • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  70. 70. Producer/Consumer 2/3 • • class Consumer(component): def main(self): count = 0 while 1: yield 1 count += 1 if self.dataReady("inbox"): data = self.recv("inbox") print data, count • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  71. 71. Producer/Consumer 3/3 • • p = Producer("Hello World") c = Consumer() postie = postman(p, "outbox", c, "inbox") • myscheduler = scheduler() myscheduler.activateMicroprocess(p) myscheduler.activateMicroprocess(c) myscheduler.activateMicroprocess(postie) • for _ in myscheduler.main(): pass • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  72. 72. Producer/Consumer Output • • Hello World 2 Hello World 3 Hello World 4 • ... • Hello World 96 Hello World 97 Hello World 98 • • Not 100, because of yields before start of scheduler loop. (scheduler terminates early) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  73. 73. Mini Axon • • Using it for more useful stuff http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  74. 74. File Reader • • class FileReader(component): • def __init__(self, filename): super(FileReader, self).__init__() self.file = open(filename, "rb",0) • def main(self): yield 1 for line in self.file.xreadlines(): self.send(line, "outbox") yield 1 http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  75. 75. Multicast sender 1/2 • class Multicast_sender(component): def __init__(self, laddr, lport, daddr, dport): super(Multicast_sender, self).__init__() self.laddr = laddr self.lport = lport self.daddr = daddr self.dport = dport • ... continued http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  76. 76. Multicast sender 2/2 • class Multicast_sender(component): ... continued def main(self): sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP) sock.bind((self.laddr,self.lport)) sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 10) while 1: if self.dataReady("inbox"): data = self.recv("inbox") l = sock.sendto(data, (self.daddr,self.dport) ) yield 1 • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  77. 77. FileReader -> MulticastSender • • reader = FileReader("Ulysses") sender = Multicast_sender("0.0.0.0", 0, "224.168.2.9", 1600) postie = Postman(reader, "outbox", sender, "inbox") • • myscheduler = scheduler() myscheduler.activateMicroprocess(reader) myscheduler.activateMicroprocess(sender) myscheduler.activateMicroprocess(postie) • for _ in myscheduler.main(): pass • • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  78. 78. Mini Axon --> Axon • • The rest is: Syntactic Sugar Ways of using it Optimisations (eg direct delivery – no postman) + a couple of other ideas we'll come to (STM, threadedcomponents, services, processcomponents) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  79. 79. Questions? • • ... before we look how to build some real components, and how to use them? • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  80. 80. Part 1 • • First part: • - An overview of Kamaelia (lightning talk style : 5-10 mins) • - How to build a mini Kamaelia (exercise 40-60 mins) • - Starting building components & systems (remainder) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  81. 81. Building a Video Recorder • • Before the break, we'll build a simple video recorder. • Approach: • 1 Build a webcam • 2 Clean up the code for “normal” reuse • 3 Componentise in least effort manner • 4 Separate input/transform/display parts • 5 Hook webcam up to a dirac encoder and filewriter • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  82. 82. Step 1: Build a simple Webcam • Pygame 1.9.1 alpha has basic Linux webcam support which works nicely, so we're using that. • First some initialisation • import pygame import pygame.camera • pygame.init() pygame.camera.init() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  83. 83. Step 1: Build a simple Webcam • The some definitions: • displaysize = (800, 600) capturesize = (640, 480) imagesize = (352, 288) imageorigin = (0,0) device = “/dev/video0” http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  84. 84. Step 1: Build a simple Webcam • Initialise the display, allocate a camera, and activate it • display = pygame.display.set_mode(displaysize) camera = pygame.camera.Camera(device, capturesize) camera.start() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  85. 85. Step 1: Build a simple Webcam • Loop round capturing images and display them • while 1: snapshot = camera.get_image() snapshot = pygame.transform.scale(snapshot, imagesize) display.blit(snapshot, imageorigin) pygame.display.flip() • • Demo! http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  86. 86. Step 2: Clean up for reuse • Given a collection of config options, wrapping this in a class makes sense. • As before these parts are unchanged: • import pygame import pygame.camera • pygame.init() pygame.camera.init() • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  87. 87. Step 2: Clean up for reuse • We then define a class, and put the config options as class attributes: • class VideoCapturePlayer(object): • displaysize = (800, 600) capturesize = (640, 480) imagesize = (352, 288) imageorigin = (0,0) device = “/dev/video0” http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  88. 88. Step 2: Clean up for reuse • Put our initialisation in the __init__. Allow the user to override our defaults def __init__(self, **argd): self.__dict__.update(**argd) super(VideoCapturePlayer,self).__init__() self.display = pygame.display.set_mode(self.displaysize) self.camera = pygame.camera.Camera(self.device, self.capturesize) self.camera.start() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  89. 89. Step 2: Clean up for reuse • Wrap up body of loop in a method def get_and_flip(self): snapshot = self.camera.get_image() snapshot = pygame.transform.scale(snapshot, self.imagesize) self.display.blit(snapshot, self.imageorigin) pygame.display.flip() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  90. 90. Step 2: Clean up for reuse • Provide a hook to start it going, wrapping the main loop. def main(self): while 1: self.get_and_flip() • Then run it! VideoCapturePlayer().main() • • Demo! http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  91. 91. Step 3: Componentise • • In this case, least effort approach is to use a threaded component. Would could make it a generator component later if needed. • Changes: • * imports * class's baseclass / extraction of display_flip into a method. (to simplify later integration with existing components) * how we run it. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  92. 92. Step 3: Componentise • • Imports change to import pygame import pygame.camera from Axon.ThreadedComponent import threadedcomponent • pygame.init() pygame.camera.init() • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  93. 93. Step 3: Componentise • • Baseclass changes & initialiser changes: class VideoCapturePlayer(threadedcomponent ): ... def __init__(self, **argd): # no longer update __dict__ here super(VideoCapturePlayer,self).__init__(**argd ) ... http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  94. 94. Step 3: Componentise • • Extract display flipping out to a separate method: def pygame_display_flip(self): pygame.display.flip() def get_and_flip(self): snapshot = self.camera.get_image() snapshot = pygame.transform.scale(snapshot, self.imagesize) self.display.blit(snapshot, self.imageorigin) self.pygame_display_flip() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  95. 95. Step 3: Componentise • • Change to how we run: • We had: • VideoCapturePlayer().main() • We now: • VideoCapturePlayer().run() • • Demo! http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  96. 96. Step 4: Split into input & output • • Now we can split the component in two: • * VideoCaptureSource • * Surface Displayer • The Video capture source now needs to be self regulating, so it needs to sleep during the loop, relative to a target frame rate. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  97. 97. Step 4: Split into input & output • • VideoCaptureSource class preamble: • import time class VideoCaptureSource(threadedcomponent): capturesize = (352, 288) delay = 0 fps = -1 device = “/dev/video0” • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  98. 98. Step 4: Split into input & output • • VideoCaptureSource initialiser • def __init__(self, **argd): super(VideoCaptureSource, self).__init__(**argd) self.camera = pygame.camera.Camera(self.device, self.capturesize) self.camera.start() if self.fps != -1: self.delay = 1.0/self.fps self.snapshot = None http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  99. 99. Step 4: Split into input & output • • Main loop body now just captures images, capturing a reference • def capture_one(self): • self.snapshot = None self.snapshot = self.camera.get_image() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  100. 100. Step 4: Split into input & output • • Main loop still runs it, and sends the image out an outbox, and then sleeps. • def main(self): • while 1: self.capture_one() self.send(self.snapshot, “outbox”) time.sleep(self.delay) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  101. 101. Step 4: Split into input & output • • The surface displayer takes the other code chunks • from Axon.Component import component • class SurfaceDisplayer(component): displaysize = (800,600) imagesize = (352, 288) imageorigin = (0,0) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  102. 102. Step 4: Split into input & output • • Has it's own initialiser... • def __init__(self, **argd): super(SurfaceDisplayer, self).__init__(**argd) self.display = pygame.display.set_mode(self.displaysize) • Retains the following method unchanged: • def pygame_display_flip(self): pygame.display.slip() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  103. 103. Step 4: Split into input & output • Then mainbody waits for surfaces to display, sleeping when there isn't any: • def main(self): • while 1: while self.dataReady(“inbox”): snapshot = self.recv(“inbox”) snapshot = pygame.transform.scale(snapshot, self.imagesize) self.display.blit(snapshot, self.imageorigin) • while not self.anyReady(): self.pause() yield 1 • yield 1 http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  104. 104. Step 4: Split into input & output • • We then join these back together in a pipeline: • from Kamaelia.Chassis.Pipeline import Pipeline • Pipeline( • VideoCaptureSource(), • SurfaceDisplayer(), • ).run() • Demo! http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  105. 105. Step 5: Recording • Working back... • • We want to write to a file... (SimpleWriter) • We want to write dirac encoded video DiracEncoder • That expects YUV Frames (ToYUV420_planar) • That expects RGB data + metadata PureTransformation of somr RGB data • Which needs a picture source • Which is where we started. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  106. 106. Step 5: Recording • Our imports therefore need to add: • • from Kamaelia.File.Writing import SimpleFileWriter • from Kamaelia.Codec.Dirac import DiracEncoder • from Kamaelia.Video.PixFormatConversion import ToYUV420_planar • from Kamaelia.Util.PureTransformer import PureTransformer • • And we delete everything related to display http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  107. 107. Step 5: Recording • We then need to join it all together: • Pipeline( • VideoCaptureSource(), PureTransformer(lambda F : {"rgb" : pygame.image.tostring(F, "RGB"), "size" : (352, 288), "pixformat" : "RGB_interleaved", }), ToYUV420_planar(), DiracEncoder(preset="CIF", encParams={"num_L1":0}), SimpleFileWriter("X.drc"), • ).run() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  108. 108. Step 5: Recording • • This then records dirac encoded video, which we can now playback with a simple dirac player! • • Demo ! • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  109. 109. Questions? • ... before we break? • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  110. 110. End of Part 1 • • Before the second half, download and install Kamaelia from the link below, if you haven't already. • • http://tinyurl.com/kot49x • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  111. 111. Kamaelia: Pragmatic Concurrency A Tutorial Michael Sparks Europython '09, Birmingham UK http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  112. 112. Part 2 • • Second part: • - More advanced stuff (demo/etc 20 mins) • - Larger systems, embedding and debugging. (20 mins) • - Building a large(ish) system (exercise 40 mins) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  113. 113. Questions? • ... before we carry on? • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  114. 114. Part 2 • • Second part: • - More advanced stuff (demo/etc 20 mins) • - Larger systems and debugging. (20 mins) • - Building a large(ish) system (exercise 40 mins) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  115. 115. More advanced Stuff • Have seen how to build a basic component and a basic system - covers a wide set of problems. Now we broaden the scope. • • If we have time we'll come back to Kamaelia's STM model & concepts of services, and (experimental) multiple process support. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  116. 116. Well behaved Shutdown • Components should expect to receive these messages on their “control” inbox, and behave as follows: • Axon.Ipc.shutdownMicroprocess – if you receive this, you are expected to shutdown immediately, and to pass this message on. • Axon.Ipc.producerFinished – if you receive this, someone sending you data has shutdown. You may want to shutdown depending on your application's logic. You may process all outstanding messages in this case. You may wish to pass this message on, or change it to shutdownMicroprocess if it was unexpected. • Due to different component needs no syntactic sugar exists for this, but common cases are being discussed at present. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  117. 117. Components • Inheritable default values – we've already seen this! • class VideoCapturePlayer(threadedcomponent): displaysize = (800, 600) capturesize = (640, 480) imagesize = (352, 288) imageorigin = (0,0) device = "/dev/video0" • Also usable: • VideoCapturePlayer(device = "/dev/video0") VideoCapturePlayer(device = "/dev/video1") http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  118. 118. Components • Inheritable default values – other places: class ServerCore(component): port=1601 protocol=None socketOptions=None TCPS=TCPServer .... http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  119. 119. Components • Inheritable default values: class MailServer(ServerCore): port=25 protocol=SMTPProtocol • Usage: • MailServer().run() MailServer(port=8025).run() MailServer(socketOptions=...).run() • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  120. 120. Components • Inheritable default values - nests class MyReconfiguredMailServer(MailServer): port=25 protocol=SMTPProtocol socketOptions = socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) class TCPS(TCPServer): CSA=MyDebugWrapper(ConnectedSocketAdapter) • Allows for deep reconfiguration of components, without rewriting. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  121. 121. Components • Inheritable default values – real usage – Kamaelia Grey class GreylistServer(MoreComplexServer): logfile = config["greylist_log"] debuglogfile = config["greylist_debuglog"] socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) port = config["port"] class TCPS(TCPServer): CSA = NoActivityTimeout(ConnectedSocketAdapter, timeout=config["inactivity_timeout"], debug=False) class protocol(GreyListingPolicy): servername = config["servername"] serverid = config["serverid"] smtp_ip = config["smtp_ip"] smtp_port = config["smtp_port"] allowed_senders = config["allowed_senders"] allowed_sender_nets = config["allowed_sender_nets"] ... http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  122. 122. Pipelines • Seen how Pipelines can join components together. • However also can do: • Pipeline( Lsystem(), ConsoleEchoer(tag="n", forwarder=True), Damage(), circular = True, ).run() • To enable a feedback loop. (Demo) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  123. 123. Pipelines • Transcode Engine • Pipeline( DirectoryWatcher(watch = conf["main_incoming_queue"]), ImageMover(destdir = conf["image_queue"]), ).activate() • Pipeline( DirectoryWatcher(watch = conf["image_queue"]), ImageTranscoder(destdir = conf["image_moderation_queue"]), ).activate() • Pipeline( DirectoryWatcher(watch = conf["main_incoming_queue"]), VideoMover(destdir = conf["video_queue"]), ).activate() • Pipeline( DirectoryWatcher(watch = conf["video_queue"]), VideoTranscoder(destdir = conf["video_moderation_queue"]), ).run() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  124. 124. Pipelines • Reusable Dirac Video Source: • def DiracWebCam(): return Pipeline( VideoCaptureSource(), PureTransformer(lambda F : {"rgb" : pygame.image.tostring(F, "RGB"), "size" : (352, 288), "pixformat" : "RGB_interleaved", }), ToYUV420_planar(), DiracEncoder(preset="CIF", encParams={"num_L1":0}), ) • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  125. 125. Graphlines • Graphlines are like pipelines, but any shape. For example, a simple presentation tool: • Graphline( CHOOSER = Chooser(items = files), IMAGE = Image(size=(800,600), position=(8,48)), NEXT = Button(caption="Next", msg="NEXT", position=(72,8)), PREVIOUS = Button(caption="Previous", msg="PREV", position=(8,8)), FIRST = Button(caption="First", msg="FIRST",position=(256,8)), LAST = Button(caption="Last", msg="LAST",position=(320,8)), linkages = { ("NEXT","outbox") : ("CHOOSER","inbox"), ("PREVIOUS","outbox") : ("CHOOSER","inbox"), ("FIRST","outbox") : ("CHOOSER","inbox"), ("LAST","outbox") : ("CHOOSER","inbox"), ("CHOOSER","outbox") : ("IMAGE","inbox"), } ).run() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  126. 126. Graphlines • • * Swiss Army Knife of Kamaelia components • * Allows almost any topology • - BUT an outbox may only link to one inbox • - many outbox may link to one inbox • * Can contain any component – including pipelines and graphlines • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  127. 127. Graphlines • SMS Outbound processor • Graphline( SMS_SOURCE = Pipeline( DirectoryWatcher(watch = "outgoingsms"), FilesToProcessSource(), FileSlurper(tuplemode=True), ), CLEANER = Pipeline( PureTransformer(lambda (x,y): y ), Mover(), ), SENDER = Pipeline( PureTransformer(lambda (x,y): cjson.decode(x) ), SMSSender(), ), SPLIT = TwoWaySplitter(), http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  128. 128. Graphlines • SMS Outbound processor (cont) • Linkages = { ("SMS_SOURCE","outbox") : ("SPLIT","inbox"), ("SPLIT", "outbox") : ("CLEANER", "inbox"), ("SPLIT", "outbox2") : ("SENDER", "inbox"), } ).run() • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  129. 129. Graphlines • Linkages to or from “self” (or “”) link to the Graphline's boxes. • Graphline( SPLIT = SomeSplitter() MERGE = SomeMerger() P1 = SomeTransformer() P2 = SomeTransformer() P3 = SomeTransformer() linkages { (“self”, “inbox”) : (“SPLIT”, “inbox”), (“MERGE”, “outbox”) : (“self”, “outbox”), ... } • Allows graphlines to nest, and higher level abstractions http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  130. 130. Graphlines • Whiteboard using nesting graphlines: • Pipeline( chunks_to_lines(), lines_to_tokenlists(), Graphline( ROUTER = Router( ((lambda T : T[0]=="SOUND"), "audio"), ((lambda T : T[0]!="SOUND"), "whiteboard"), ), WHITEBOARD = FilteringPubsubBackplane(whiteboardBackplane), AUDIO = Pipeline( SimpleDetupler(1), # remove 'SOUND' tag SpeexDecode(3), FilteringPubsubBackplane(audioBackplane, dontRemoveTag=True), RawAudioMixer(), SpeexEncode(3), Entuple(prefix=["SOUND"],postfix=[]), ), http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com •
  131. 131. Graphlines • Whiteboard using nesting graphlines: • Linkages = { # incoming messages go to a router ("", "inbox") : ("ROUTER", "inbox"), • # distribute messages to appropriate destinations ("ROUTER", "audio") : ("AUDIO", "inbox"), ("ROUTER", "whiteboard") : ("WHITEBOARD", "inbox"), • # aggregate all output ("AUDIO", "outbox") : ("", "outbox"), ("WHITEBOARD", "outbox") : ("", "outbox"), • # shutdown routing ("", "control") : ("ROUTER", "control"), ("ROUTER", "signal") : ("AUDIO", "control"), ("AUDIO", "signal") : ("WHITEBOARD", "control"), ("WHITEBOARD", "signal") : ("", "signal") }, ), tokenlists_to_lines(), ) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  132. 132. Backplanes • • * For where you want 1 to many or many to many • * Declare a backplane • * Components can publish data to it • * Components may subscribe to it. • * Subscribers get a copy of all data sent • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  133. 133. Backplanes • Can be useful for updating a central state. • • For example players locations can be posted here: • Backplane("PLAYERS").activate() • • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  134. 134. Backplanes • Players Post their updates like this: • Pipeline( • MyGamesEventsComponent(up="p", down="l", left="a", right="s"), • BasicSprite("cat.png", name = "cat", border=40), • PureTransformer(lambda x: ("Cat ", x)), • PublishTo("PLAYERS"), • ).activate() • • • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  135. 135. Backplanes • The system can analyse their updates like this: • Pipeline( • SubscribeTo("PLAYERS"), • PlayerAnalyser(), • Distancer(), • ConsoleEchoer(), • ).activate() • • • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  136. 136. Backplanes & Servers • When combined with a server, you instantly get a pub/ sub capable server, or splitter, or merger. • Backplane(“SPLIT”).activate() Pipeline( DiracWebCam(), PublishTo(“SPLIT”), ).activate() def VideoProtocol(**argd): return SubscribeTo(“SPLIT”) • ServerCore(protocol=VideoProtocol, port=1700).run() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  137. 137. Backplanes & Servers • When combined with a server, you instantly get a pub/ sub capable server, or splitter, or merger. • Backplane(“CHAT”).activate() def ChatProtocol(**argd): return Pipelines( SubscribeTo(“CHAT”) NullComponent(), PublishTo(“CHAT”) ) • ServerCore(protocol=ChatProtocol, port=1700).run() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  138. 138. PAR & Seq • PAR & Seq are wrapper components with concepts nabbed from Occam. (hence their names) • * Seq runs each component it's given, one after another, wiring up inboxes/outboxes such that each component handles it. Useful in staged protocols (later) • * PAR runs all the components concurrently. Their output is merged. Shutdown messages sent to PAR are forwarded to all subcomponents – making it useful for managing system shutdown. • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  139. 139. Using PAR • PAR can be used to simplify some systems: • Pipeline( PAR( Button(caption="Next", msg="NEXT", position=(72,8)), Button(caption="Previous", msg="PREV", position=(8,8)), Button(caption="First", msg="FIRST" ,position=(256,8)), Button(caption="Last", msg="LAST", position=(320,8)), ), • Chooser(items = files), Image(size=(800,600), position=(8,48)), ).run() • This is the same slideshow as the previous Graphline... http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  140. 140. Using PAR • ... to run & shutdown multiple subsystems • Pipeline( timedShutdown(TTL=15), PAR( Pipeline( ReadFileAdaptor(file, readmode="bitrate", bitrate = 300000*8/5), DiracDecoder(), MessageRateLimit(framerate), VideoOverlay(position=(260,48), size=(200,300)), ), Pipeline( ReadFileAdaptor(file, readmode="bitrate", bitrate = 2280960*8), DiracDecoder(), ToRGB_interleaved(), VideoSurface(size=(200, 300), position=(600,48)), ), http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  141. 141. Using PAR • ... to run & shutdown multiple subsystems • Pipeline( PAR(Button(caption="Next", msg="NEXT", position=(72,8)), Button(caption="Previous", msg="PREV", position=(8,8)), Button(caption="First", msg="FIRST" ,position=(256,8)), Button(caption="Last", msg="LAST", position=(320,8)), ), Chooser(items = files), Image(size=(200,300), position=(8,48), maxpect=(200,300)), ), Pipeline( Textbox(size=(200,300), position=(8,360)), TextDisplayer(size=(200,300), position=(228,360)), ), Ticker(size=(200,300), position=(450,360)), ), ).run() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  142. 142. Using Seq • An example from the mobile reframer: • Seq( "Decoding & separating frames...", Graphline( MAXF = DetermineMaxFrameNumber(edlfile), DO = Carousel( ... ), STOP = TriggeredOneShot(""), ... ), "Processing edits...", Graphline( REFRAMING = ReframeVideo(edlfile... SOUND = PassThroughAudio(edlfile... ENCODING = ReEncode(outFileName... ... ), "Cleaning up...", StopSelector(), ).run() http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  143. 143. Using Seq • An example of passing state in a protocol handler: • def CompositeBulletinBoardProtocol(**argd): ConnectionInfo = {} ConnectionInfo.update(argd) return Seq( Authenticator(State = ConnectionInfo), UserRetriever(State = ConnectionInfo), MessageBoardUI(State = ConnectionInfo), StateSaverLogout(State = ConnectionInfo), ) ServerCore(protocol=CompositeBulletinBoardProtocol, ...) • Worth noting the similarity between this and wsgi. (Except Seq can contain graphlines, pipelines, etc too) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  144. 144. Part 2 • • Second part: • - More advanced stuff (demo/etc 20 mins) • - Larger systems and debugging. (20 mins) • - Building a large(ish) system (exercise 40 mins) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  145. 145. Larger Systems & Debugging • Large systems which are long running can develop bugs which are awkward to debug. • Kamaelia has a collection of tools you can use which we'll walk through / demonstrate next. • • Practical example. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  146. 146. Larger Systems & Debugging • Key Tools: • * Instrumentation using backplanes & logging • * Axon Visualiser • * Embeddable python console • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  147. 147. Larger Systems & Debugging • • Practical examples. http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  148. 148. Part 2 • • Second part: • - More advanced stuff (demo/etc 20 mins) • - Larger systems and debugging. (20 mins) • - Building a large(ish) system (exercise 40 mins) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  149. 149. Building a Bulletin Board • We're going to build an “old school” bulletin board system. • * Someone telnets in & authenticates • * Can get help, exit or read messages • * Reading messages, they can read, reply, exit reading, or get help • * State is restored/saved at session start/end http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  150. 150. Building a Bulletin Board • For practicality, we won't implement message posting, nor message deletion, nor persistent user state. (Though these would be useful exercises) • All on-disk objects encoded as json objects: * users * Messages – stored in “folders” (directories) with filenames == messageid • * message fields: from, to, __body__, date, message, reply-to, subject http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  151. 151. Building a Bulletin Board • Getting started: • * Start with basic server for request/response Just echo initially - BB1.py • * make restarting quicker for debugging - BB2.py * abstract out “getting a line of data” - BB3.py * Then use that abstraction - BB4.py * Simplify “control” box handling - BB5.py * Abstract out reusable bits from domain specific – BB6.py • We'll go through the code for these in the actual code files rather than on these slides • (rather unwieldy as slides) http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  152. 152. Building a Bulletin Board • Intermediate Plan • This is the logic of the user protocol • Seq( • Authenticator( <some shared state> ), • UserRetriever( <some shared state> ), • MessageBoardUI( <some shared state> ), • StateSaverLogout( <some shared state> ), • ) • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  153. 153. Building a Bulletin Board • Intermediate Plan • * Tidy up control handling a touch more & add netPrint method to reusable bit - BB7.py • * Change Authenticator component to support the “passed on state”, change protocol handler factory to create that Seq pipeline. - BB8.py • * Change Authenticator to authenticate against a password file and set “remoteuser” in the shared state - BB9.py • * Write stubs for UserRetriever & StateSaverLogout - BB10.py • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  154. 154. Building a Bulletin Board • Final UI Plan • * Change usage of waitMsg to yield self.waitMsg() -- BB11.py • * Copy Authenticator's patten to create initial menu for MessageBoardUI, including stub for messages menu - BB12.py • * Use same pattern for messages menu, use waitMsg pattern for logic. - BB13.py • * Implement Folders to hold messages. Numbers as filenames (msg 1, 2, 3, 4) Messages as json encoded objects - BB14.py • * Clean up to add a line oriented buffer to handle partial lines. (necessary for cross platform/real world) - BB15.py http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  155. 155. Building a Bulletin Board • Final UI Plan • * Change usage of waitMsg to yield self.waitMsg() -- BB11.py • * Copy Authenticator's patten to create initial menu for MessageBoardUI, including stub for messages menu - BB12.py • * Use same pattern for messages menu, use waitMsg pattern for logic. - BB13.py • * Implement Folders to hold messages. Numbers as filenames (msg 1, 2, 3, 4) Messages as json encoded objects - BB14.py • * Clean up to add a line oriented buffer to handle partial lines. (necessary for cross platform/real world) - BB15.py http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  156. 156. Summary • We've covered • * the 30,000 view of kamaelia • * Building your own core • * Building components, including a video recording application • * Tools for building systems • * Tools for debugging systems • * Built a large(ish) system (an authenticated staged protocol) • * Illustrated the majority of this using real world systems http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  157. 157. Final Questions? • • http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com
  158. 158. Thank you! ¬ http://www.kamaelia.org/PragmaticConcurrency sparks.m@gmail.com

×