Downloaded 28 times
Uploaded byRaymond Roestenburg
Real world Scala hAkking NLJUG JFall 2011
The document provides an overview of using Scala and Akka for building distributed sensor networks. Some key points: - The speaker uses Scala and Akka at their company for building distributed systems to manage traffic and sensor networks. - Akka actors are used to build distributed and fault-tolerant systems. Camel is used for integration between actors. Remote actors allow building systems that span multiple machines. - Examples of systems built include a border control sensor network and distributed traffic management systems. Scala and Akka provide benefits like less code, fault tolerance, and ability to interoperate with existing Java systems. - Topics covered include using Akka actors, remote actors, Camel integration,
More Related Content
Similar to Real world Scala hAkking NLJUG JFall 2011
![[PDF Download] Akka in Action 1st Edition Raymond Roestenburg fulll chapter](https://cdn.slidesharecdn.com/ss_thumbnails/10639-240805124441-3617891d-thumbnail.jpg?width=640&height=640&fit=bounds)
Real world Scala hAkking NLJUG JFall 2011
- 1. Real World ScalahAkking Raymond Roestenburg
- 2. About me • Java since 1997 • Scala since 2010 • Mostly backend systems • Lead architect Traffic Management at CSC • Akka committer code: http://github.com/RayRoestenburg blog: http://roestenburg.agilesquad.com twtr: @RayRoestenburg
- 3. Company I workfor Traffic Management • Section Control • Weigh in Motion • Traffic Information • Traffic Enforcement
- 4. Agenda • @MIGO-BORAS sensordomain • Sensor networks • Typesafe stack • Akka Actors • Apache Camel Integration • Remote Actors • Monitoring • Testing • Deployment
- 5. @MIGO-BORAS • Koninklijke Marechaussee • MTV (Mobiel Toezicht Vreemdelingen) • Prevent illegal border crossing • Sensor domain • Fixed locations • Mobile vehicles • Cameras, Radars • Vehicle properties • January 2012
- 6. Sensor networks • Usablefor many applications in our field • Distributed along highways and provincial roads • Traffic Enforcement • Traffic Information • Other applications • Roadside systems • Range of sensor types – Camera’s, Infrared, Laser, Piezoelectric, Loops, Mobile units, … • Remotely Configure & Control
- 7. Why Scala &Akka • Less code, less bugs • Less code, same VM power • Experienced team (Java, C#, C) • 3 experienced Java devs • 1 experienced C# dev with no Java experience • Need to interoperate with existing Java and C (JNA) • Simplify development (C->Java->Scala) • Ubuntu
- 8. Scala 2.9.x • Learning curve (Start with simple features) • Traits, Tupels, Stackable Traits • First Class Functions • Scala Collections • map, flatMap, partition, foreach, filter, groupBy.. • Option type • map.. orElse, foreach,.. • Case classes • REPL, DSLs, Extension methods
- 9. Typesafe Stack • CurrentStack • Open Source • SBT 0.7.7 • Scala 2.9.0-1 • Akka 1.1.3 • Camel & Camel Components 2.7.0 (FTP,Mina,Jetty,…) • Jboss Netty 3.2.4 • When we started early 2010 • Akka 0.7, Scala 2.8.0Beta1
- 10. Akka Usage Core Actors, Remote, TestKit, Serialization, STM, Agents, Dataflow, Async HTTP (Mist), FSM, Transactors Modules MicroKernel, Camel, AMQP, Scalaz, Spring, OSGi, DataFlow, Persistence,
- 11. Akka Actors ActorRef Mailbox Actor Message to send Dispatcher Threads
- 12. Akka Actors ActorRef Mailbox Actor Dispatcher Threads
- 13. Akka Actors ActorRef Mailbox Actor Dispatcher Threads
- 14. Akka Actors ActorRef Mailbox Actor Dispatcher Threads
- 15. Akka Actors ActorRef Mailbox Actor Dispatcher Threads
- 16. Akka Actors ActorRef Mailbox Actor Dispatcher Threads
- 17. One-way messaging Shared Dispatcher
- 18.
- 19.
- 20. Camel Actors • CamelComponent integration • Consumers and Producers
- 21.
- 22.
- 23. Camel Producers • UseCamelContext
- 24.
- 25. Dispatchers Camel Producers Local Processing Camel Consumers Heartbeat, Remote Actors Cleanup, Schedule, Long Running, Redelivery, Remote Error Handling
- 26. Messages • Case classes • Simple data containers • toProtobuf and fromProtobuf on case class and companion object for remoting • Every msg has a • correlation Id to the event • Timestamp • Logging, Tracing, Performance
- 27. Event Process Chains LoadBalancer EndPoint (Consumer/Producer) Forwarding Actor
- 28. Remote Actors • Client • Actor.remote.actorFor(name, host,port) • Server • Actor.remote.register(name, actorRef) • remoteActorFor: (String, String, Int) => ActorRef = remote.actorFor
- 29. Remote Actors Internals • RemoteActorRef • RemoteServer • RemoteClient • Netty Server • Netty Channels • Lifecycle Event • Serialization (Scala, listeners Java, (S)JSON, ProtoBuf • RemoteProtocol RemoteActorRef RemoteProtocol ActorRef Mailbox Actor RemoteClient RemoteServer
- 30. Remote Actors (Akka1.1.3) • JBoss Netty (Channels) • Not as transparent as you would like • Remote Lifecycle Listeners • Closing sockets, reconnect • Own Guaranteed Delivery implementation – Specific requirements – Bases on Idempotent Receiver and Repeating messages after reconnect – Non-Trivial – Heartbeats – Exp Backoff
- 31. Remote configuration • #compression-scheme = "zlib“ #leave out • zlib-compression-level = 0 • client: reconnection-time-window = 2000000 • client: read-timeout (we use 60) • backlog = 4096 # Netty backlog for connections, should suffice, increase if failures happen
- 32. Monitoring • Application levelmonitoring • Listener Actors contain Custom JMX MBeans • Direct passthrough of msg or translation
- 33.
- 34. Testing one-way ActorOutput
- 35. Microkernel deployment • sbtdist (akka-sbt-plugin) • Creates dist directory • init.d scripts
- 36.
Editor's Notes
- #13 ActorRef is an immutable reference which exclusively gives access to the Actor.Actors run on Dispatchers. A Dispatcher runs a mailbox to process messages.Amount of messages per mailbox run is configurable. (actor.throughput, default 5)
- #18 One way messaging between two ActorsActors can process messages in parallel
- #19 One way messaging between two ActorsActors can process messages in parallel
- #20 !!! Is preferred over !!Futuresalso run onDispatchers.FutureTimeoutException happens onfuture.get. IfonlyonComplete and failure are registered, yougetnoFutureTimeoutException