Real world Scala hAkking NLJUG JFall 2011

1,529 views

Published on

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,529
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
26
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • ActorRef is an immutable reference which exclusively gives access to the Actor.Actors run on Dispatchers. A Dispatcher runs a mailbox to process messages.Amount of messages per mailbox run is configurable. (actor.throughput, default 5)
  • One way messaging between two ActorsActors can process messages in parallel
  • One way messaging between two ActorsActors can process messages in parallel
  • !!! Is preferred over !!Futuresalso run onDispatchers.FutureTimeoutException happens onfuture.get. IfonlyonComplete and failure are registered, yougetnoFutureTimeoutException
  • Real world Scala hAkking NLJUG JFall 2011

    1. 1. Real World Scala hAkking Raymond Roestenburg
    2. 2. About me• Java since 1997• Scala since 2010• Mostly backend systems• Lead architect Traffic Management at CSC• Akka committercode: http://github.com/RayRoestenburgblog: http://roestenburg.agilesquad.comtwtr: @RayRoestenburg
    3. 3. Company I work for Traffic Management • Section Control • Weigh in Motion • Traffic Information • Traffic Enforcement
    4. 4. Agenda• @MIGO-BORAS sensordomain• Sensor networks• Typesafe stack• Akka Actors• Apache Camel Integration• Remote Actors• Monitoring• Testing• Deployment
    5. 5. @MIGO-BORAS• Koninklijke Marechaussee• MTV (Mobiel Toezicht Vreemdelingen)• Prevent illegal border crossing• Sensor domain• Fixed locations• Mobile vehicles• Cameras, Radars• Vehicle properties• January 2012
    6. 6. Sensor networks• Usable for many applications in our field • Distributed along highways and provincial roads • Traffic Enforcement • Traffic Information • Other applications • Roadside systems • Range of sensor types – Camera’s, Infrared, Laser, Piezoelectric, Loops, Mobile units, … • Remotely Configure & Control
    7. 7. Why Scala & Akka• Less code, less bugs• Less code, same VM power• Experienced team (Java, C#, C) • 3 experienced Java devs • 1 experienced C# dev with no Java experience• Need to interoperate with existing Java and C (JNA)• Simplify development (C->Java->Scala)• Ubuntu
    8. 8. Scala 2.9.x• Learning curve (Start with simple features)• Traits, Tupels, Stackable Traits• First Class Functions• Scala Collections • map, flatMap, partition, foreach, filter, groupBy..• Option type • map.. orElse, foreach,..• Case classes• REPL, DSLs, Extension methods
    9. 9. Typesafe Stack• Current Stack • Open Source • SBT 0.7.7 • Scala 2.9.0-1 • Akka 1.1.3 • Camel & Camel Components 2.7.0 (FTP,Mina,Jetty,…) • Jboss Netty 3.2.4• When we started early 2010 • Akka 0.7, Scala 2.8.0Beta1
    10. 10. Akka UsageCore Actors, Remote, TestKit, Serialization, STM, Agents, Dataflow, Async HTTP (Mist), FSM, TransactorsModules MicroKernel, Camel, AMQP, Scalaz, Spring, OSGi, DataFlow, Persistence,
    11. 11. Akka Actors ActorRef Mailbox ActorMessage to send Dispatcher Threads
    12. 12. Akka Actors ActorRef Mailbox ActorDispatcher Threads
    13. 13. Akka Actors ActorRef Mailbox ActorDispatcher Threads
    14. 14. Akka Actors ActorRef Mailbox ActorDispatcher Threads
    15. 15. Akka Actors ActorRef Mailbox ActorDispatcher Threads
    16. 16. Akka Actors ActorRef Mailbox ActorDispatcher Threads
    17. 17. One-way messaging Shared Dispatcher
    18. 18. Two-way
    19. 19. Two-way with Futures
    20. 20. Camel Actors• Camel Component integration• Consumers and Producers
    21. 21. Camel Consumers
    22. 22. Camel Producers
    23. 23. Camel Producers• Use CamelContext
    24. 24. DispatchersThreadBasedDispatcherExecutorBasedEventDrivenDispatcherExecutorBasedWorkStealingDispatcherPriorityExecutorBasedEventDrivenDispatcherRollYourOwn
    25. 25. DispatchersCamel Producers Local Processing Camel Consumers Heartbeat, Remote Actors Cleanup, Schedule, Long Running, Redelivery, Remote Error Handling
    26. 26. Messages• Case classes • Simple data containers • toProtobuf and fromProtobuf on case class and companion object for remoting• Every msg has a • correlation Id to the event • Timestamp• Logging, Tracing, Performance
    27. 27. Event Process ChainsLoad Balancer EndPoint (Consumer/Producer)Forwarding Actor
    28. 28. Remote Actors• Client • Actor.remote.actorFor(name, host,port)• Server • Actor.remote.register(name, actorRef)• remoteActorFor: (String, String, Int) => ActorRef = remote.actorFor
    29. 29. Remote Actors Internals • RemoteActorRef • RemoteServer • RemoteClient • Netty Server • Netty Channels • Lifecycle Event • Serialization (Scala, listeners Java, (S)JSON, ProtoBuf • RemoteProtocolRemoteActorRef RemoteProtocol ActorRef Mailbox Actor RemoteClient RemoteServer
    30. 30. Remote Actors (Akka 1.1.3)• JBoss Netty (Channels)• Not as transparent as you would like• Remote Lifecycle Listeners• Closing sockets, reconnect• Own Guaranteed Delivery implementation – Specific requirements – Bases on Idempotent Receiver and Repeating messages after reconnect – Non-Trivial – Heartbeats – Exp Backoff
    31. 31. Remote configuration• #compression-scheme = "zlib“ #leave out• zlib-compression-level = 0• client: reconnection-time-window = 2000000• client: read-timeout (we use 60)• backlog = 4096 # Netty backlog for connections, should suffice, increase if failures happen
    32. 32. Monitoring• Application level monitoring• Listener Actors contain Custom JMX MBeans• Direct passthrough of msg or translation
    33. 33. Testing with TestKit
    34. 34. Testing one-way Actor Output
    35. 35. Microkernel deployment• sbt dist (akka-sbt-plugin)• Creates dist directory• init.d scripts
    36. 36. Questions?

    ×