Embracing
                                     Concurrency
                     for Fun, Utility & Simpler Code




Michae...
Why?        Hardware finally going
             massively concurrent
   Opportunity!
                                     ...
Missing Something?
 Fundamental Control Structures
 ... in imperative languages number greater than 3! (despite what you g...
Regarding using
 concurrency, what
sort of applications
are we talking about
       here?



                             ...
Think backend                                        Speak 'n Write
                            P2P Whiteboard Programming...
Core Approach:
   Concurrent things with comms points
   Generally send messages
   Keep data private, don't share



    ...
But I must share data?
Use Software Transactional Memory
ie version control for variables.
                         1. Che...
Perspectives in APIs! (1/2)
   If you have concurrency it becomes
natural to think in terms of 1 st 2 nd and 3 rd
 person....
Perspectives in APIs! (2/2)
   If you have concurre ncy it become s
natural to think in terms of 1 st 2 nd and 3 rd
 perso...
Actor Systems
         Distinction can be unclear,
         potential source of ambiguity*      private real methods



 M...
Advantages of outboxes
                                     No hardcoding of recipient
                                   ...
A Core Concurrency DSL
    Pipeline(A,B,C)
    Graphline(A=A,B=B, C=C, linkages = {})
    Tpipe(cond, C)
    Seq(A,B,C), P...
Pipeline Example
    Pipeline(
        MyGamesEventsComponent(up="p", down="l", left="a", right="s"),
        BasicSprite(...
Graphline Example
    Graphline(
      NEXT = Button(...),
      PREVIOUS = Button(...),                PREVIOUS       NEX...
Server Example

                              data
                              from
                              user  ...
Server Example

                                        You
                                        therefore
         Mai...
Server Example
    from Kamaelia.Chassis.ConnectedServer import ServerCore
    from Kamaelia.Util.PureTransformer import P...
Backplane Example
    # Streaming Server for raw DVB of Radio 1
    Backplane(“Radio”).activate()

    Pipeline(
       DV...
Thank you for listening!

             If you have questions, grab me later :-)

             Kamaelia is open source

   ...
Upcoming SlideShare
Loading in …5
×

Kamaelia lightning2010opensource

1,824
-1

Published on

Lightning talk on kamaelia at Europython 2010 in Birmingham leeds

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,824
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Kamaelia lightning2010opensource

  1. 1. Embracing Concurrency for Fun, Utility & Simpler Code Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  2. 2. Why? Hardware finally going massively concurrent Opportunity! ... .... PS3, high end servers, trickling down to desktops, laptops) “many hands make light work” but Viewed as Hard ... do we just have crap tools? “And one language to in Problems the darkness bind them” ... can just we REALLY abandon 50 years of code for Erlang, Haskell and occam? Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  3. 3. Missing Something? Fundamental Control Structures ... in imperative languages number greater than 3! (despite what you get taught...!) Control Structure Traditional Abstraction Biggest Pain Points Sequence Function Global Var Selection Function Global Var Iteration Function Global Var Parallel Thread Shared Data Usually Skipped Lost or duplicate update This slide raises the point that are most common bugs teaching of imperative languages often misses out concurrency as a Michael Sparks fundamental construct, leaving a BBC R&D, http://www.kamaelia.org/Home.html conceptual gap for most developers
  4. 4. Regarding using concurrency, what sort of applications are we talking about here? Desktop gsoc Media Novice APPS trainee Network 3rd Party Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  5. 5. Think backend Speak 'n Write P2P Whiteboard Programming needed for Simple ER DB Modeller (logo) P2P Radio youtube/flickr Kids Games Torrent type systems 3D Systems Compose Realtime Music UGC Desktop gsoc Paint App Backend P2P Web Server Secure “phone” Transcoder Social Net Vis Media Novice ... Shot Change Detection APPS trainee MiniAxon ScriptReader Mobile DVB MediaPreview Reframing on Mobile Reliable Macro Podcasts Network 3rd Party Multicast Sedna “record everything” XMPP XMLDB Email & AWS pubsub Spam Web (Amazon) SMTP IRC Qt Serving Gtk Greylisting Pop3Proxy ClientSide AIM microblogging Spam Tools Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  6. 6. Core Approach: Concurrent things with comms points Generally send messages Keep data private, don't share outbox inbox outbox inbox signal control signal control ... ... ... ... Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  7. 7. But I must share data? Use Software Transactional Memory ie version control for variables. 1. Check out the collection of values you wish to work on 2. Change them 3. Check them back in 4. If conflict/clash, go back to 1 Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  8. 8. Perspectives in APIs! (1/2) If you have concurrency it becomes natural to think in terms of 1 st 2 nd and 3 rd person. This affects an API's structure, and can be vital for unde rstanding it! 1st Person - I change my state This is one we've found that makes sense 2nd Person – YOU 3rd Person – want to me to do Bla should something do something (you send outbox (I send a message) inbox me a message) control signal ... ... Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  9. 9. Perspectives in APIs! (2/2) If you have concurre ncy it become s natural to think in terms of 1 st 2 nd and 3 rd person. This affects an API's structure , and can be vital for understanding it! private real methods This is one we've found that makes sense Messages Messages sent from public to public outboxes inboxes inbox outbox control signal Also, think ... ... Also, think about stdout about stdin Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  10. 10. Actor Systems Distinction can be unclear, potential source of ambiguity* private real methods Messages No outbox concept from public Possible issues with inboxes rate limiting* inbox Hardcodes recipient control in the sender ... *system dependent issue Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  11. 11. Advantages of outboxes No hardcoding of recipient allows: - Late Binding - Dynamic rewiring inbox outbox Concurrency Patterns as control signal Reusable Code ... ... ... a concurrency DSL Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  12. 12. A Core Concurrency DSL Pipeline(A,B,C) Graphline(A=A,B=B, C=C, linkages = {}) Tpipe(cond, C) Seq(A,B,C), PAR(), ALT() Backplane(“name”), PublishTo(“name”), SubscribeTo(“name”) Carousel(...) PureTransformer(...) StatefulTransformer(...) PureServer(...) MessageDemuxer(...) Source(*messages) NullSink Some of these are work in progress – they've been identified as useful, but not implemented as chassis, yet Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  13. 13. Pipeline Example Pipeline( MyGamesEventsComponent(up="p", down="l", left="a", right="s"), BasicSprite("cat.png", name = "cat", border=40), ).activate() MyGames Events Component Basic Sprite Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  14. 14. Graphline Example Graphline( NEXT = Button(...), PREVIOUS = Button(...), PREVIOUS NEXT FIRST = Button(...), (button) (button) LAST = Button(...), CHOOSER = Chooser(...), IMAGE = Image(...), FIRST LAST (button) (button) ... ).run() Chooser Image Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  15. 15. Server Example data from user data Main Socket handler Server Core to user Created at runtime to Protocol Handler Factory handle the Remote Protocol handler connection User Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  16. 16. Server Example You therefore Main need to Server Core provide this bit. Protocol Handler Factory Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  17. 17. Server Example from Kamaelia.Chassis.ConnectedServer import ServerCore from Kamaelia.Util.PureTransformer import PureTransformer def greeter(*argv, **argd): return PureTransformer(lambda x: "hello" +x) class GreeterServer(ServerCore): protocol=greeter port=1601 GreeterServer().run() Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  18. 18. Backplane Example # Streaming Server for raw DVB of Radio 1 Backplane(“Radio”).activate() Pipeline( DVB_Multiplex(850.16, [6210], feparams), # RADIO ONE PublishTo("RADIO"), ).activate() def radio(*argv,**argd): return SubscribeTo(“RADIO”) ServerCore(protocol=radio, port=1600).run() Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  19. 19. Thank you for listening! If you have questions, grab me later :-) Kamaelia is open source Kamaelia Tutorial (from Europython 09) : - http://tinyurl.com/kamaelia Michael Sparks BBC R&D, http://www.kamaelia.org/Home.html
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×