System Integration with Akka and Apache Camel

1. Connecting Akka to the Rest of the World System integration with Akka and Apache Camel by Martin Krasser

2. About me• Freelance software engineer, architect – eHealth application integration (Java, Scala, Groovy) – Bioinformatics, Cheminformatics (C++, Java) – …• Open source committer – Akka – Camel – Scalaz-Camel – …• Java since 1998• Scala since 2009• Twitter: @mrt1nz

3. Overview• Introduction – Camel – Akka• Akka-Camel integration – Consumer actors – Producer actors – Actor components• Comparison – Camel routes – Akka-Camel routes – Scalaz-Camel routes

4. Apache Camel• Integration framework written in Java – Open source, Apache 2 license – http://camel.apache.org• Enterprise Integration Patterns (EIP) – G. Hohpe, B. Woolf• Domain-specific language (DSL) – Internal: Java, Scala – External: XML/Spring• Pluggable protocol and API bindings – HTTP, FTP, JMS, File … and approx. 100 more – Camel “components” http://camel.apache.org/components.html

5. Apache Camel• Usage in Scala applications – Camel Java/Scala DSL • http://camel.apache.org – Akka-Camel integration module • http://akka.io/docs/akka-modules/1.1.2/modules/camel.html – Scalaz-Camel DSL • https://github.com/krasserm/scalaz-camel

6. Apache Camel• Architecture

7. Apache Camel• Routing example D D HTTP jms:queue:docs JMS HTTP Filter Endpoint Endpoint http://example.org/docs Java DSL: from("jms:queue:docs") .filter().xpath("/person[@name=Jon]") .to("http:example.org/docs"); Scala DSL: "jms:queue:docs" when(_.in == "…") to("http:example.org/docs");

8. Apache Camel• Messaging endpoints – Configured via endpoint URI • "jms:queue:docs" • "http://example.org/docs" • … – Endpoint consumer: from(uri) – Endpoint producer: to(uri)• Message processors … – Configured via fluent API • filter().xpath("…") • transform(body().append("…")) • … – Custom Java/Scala code (Processor interface)

9. Akka• Platform for event-driven, scalable and fault- tolerant architectures on the JVM• Concurrency – Actors, agents, dataflow, STM• Scalability Core – Remote actors, cluster membership Services• Fault tolerance – Local and remote actor supervision

10. Akka• Add-on modules – akka-camel • Camel integration (untyped actors) – akka-camel-typed • Camel integration (typed actors) – akka-kernel • Microkernel – akka-scalaz • Scalaz support – akka-spring • Spring integration – …

11. Akka• Actor model – Mathematical model of concurrent computation – Carl Hewitt, 1973• Akka actor – Encapsulates state – Encapsulates behavior – Exchanges messages – Mailbox – Event-driven – Single-threaded• API – Scala – Java

12. Akka• Actor model benefits – Easier to reason about – Raised abstraction level – Easier to avoid • Race conditions • Dead locks • Starvation • Live locks

13. Akka Actors• Receive messages import akka.actor.Actor // actor definition class ExampleActor extends Actor { def receive = { case "test" => … case _ => … } } // create actor reference (ActorRef) val actor = Actor.actorOf[ExampleActor] // start actor actor.start

14. Akka Actors• Send messages – Fire-Forget actor ! "test"

15. Akka Actors• Send messages – Send-And-Receive-Eventually actor !! "test" match { case Some(result) => … // process result case None => … // timeout } uses Future under the hood (with timeout)

16. Akka Actors• Send messages – Send-And-Receive-Future val future = actor !!! "test" future.await val result = future.get returns Future directly

18. Akka-Camel Integration• Exchange messages with Akka actors via – HTTP, FTP, JMS, File … and approx. 100 more – Any Camel component can be used (pluggable)• Integration layer of Akka – Consumer actors: receive message from Camel endpoints – Producer actors: produce messages to Camel endpoints – Message exchange patterns (MEPs): in-out and in-only• Leverages Camel’s asynchronous routing engine – Asynchronous completion of message exchange – No threads blocked waiting for response messages, for example

19. Consumer Actors • Basic example – TCP consumer •import akka.actor._import akka.camel._class TcpConsumer extends Actor with Consumer { def endpointUri = "mina:tcp://localhost:6200?textline=true" def receive = { case Message(body, headers) => { self.reply("received %s" format body) } }} – Camel component: camel-mina – MEP: in-out

20. Consumer Actors• Basic example – TCP consumer activation import akka.actor._ import akka.camel._ // Consumer actors require a running CamelService CamelServiceManager.startCamelService // Activate TCP endpoint (asynchronously) Actor.actorOf[TcpConsumer].start – Asynchronous activation of consumer endpoint

21. Consumer Actors• Basic example – TCP consumer activation import akka.actor._ import akka.camel._ // Consumer actors require a running CamelService val service = CamelServiceManager.startCamelService // Wait for TCP endpoint activation service.awaitEndpointActivation(1) { Actor.actorOf[TcpConsumer].start } – Wait for endpoint activation – Useful for testing purposes

22. Consumer Actors• Basic example – TCP consumer deactivation import akka.actor._ import akka.camel._ val service: CamelService = … val tcpConsumer: ActorRef = … // deactivate TCP endpoint (asynchronously) tcpConsumer.stop // … or wait for TCP endpoint deactivation service.awaitEndpointDeactivation(1) { tcpConsumer.stop }

23. Consumer Actors• Basic example – HTTP consumer class HttpConsumer extends Actor with Consumer { def endpointUri = "jetty:http://localhost:8080/example" def receive = { case msg: Message => { self.reply("received %s" format msg.bodyAs[String]) } } } – Camel component: camel-jetty – MEP: in-out – Jetty continuations used internally – Camel type converter (msg.bodyAs[…])

24. Consumer Actors• Basic example – JMS consumer class JMSConsumer extends Actor with Consumer { def endpointUri = "jms:queue:example" def receive = { case Message(body, headers) => ... } } – Camel component: camel-jms – MEP: in-only

25. Consumer Actors• Basic example – IMAP consumer class MailConsumer extends Actor with Consumer { def endpointUri = "imap://admin@mymailserver.com?password=secret" def receive = { case Message(body, headers) => ... } } – Camel component: camel-mail – MEP: in-only

26. Consumer Actors• Basic example – Scheduled consumer class ScheduledConsumer extends Actor with Consumer { def endpointUri = "quartz://example?cron=0/2+*+*+*+*+?" def receive = { case tick => ... } } – Camel component: camel-quartz – MEP: in-only

27. Consumer Actors• Acknowledgements import akka.camel.Ack ... class FileConsumer extends Actor with Consumer { override def autoack = false // default is true def endpointUri = "file:messages/in?delete=true" def receive = { case Message(body, headers) => { // ... self.reply(Ack) // delete consumed file } } } – Camel component: camel-file – MEP: in-only – Application-level acknowledgement (Ack)

28. Consumer Actors• Failure replies import akka.camel.Failure ... class FileConsumer extends Actor with Consumer { override def autoack = false // default is true def endpointUri = "file:messages/in?delete=true" def receive = { case Message(body, headers) => { // ... self.reply(Failure(reason)) // redeliver file } } } – Application-level negative acknowledgement (Failure) – Redelivery works also with JMS endpoints, for example – Failure replies can also be used with in-out MEP

29. Consumer Actors• Failure replies – Should not be made within receive – Let consumer actors crash (on Exception) – Use a supervisor for failure replies

30. Consumer Actors• Supervised consumerclass SupervisedFileConsumer extends Actor with Consumer { override def autoack = false def endpointUri = "file:messages/in?delete=true" def receive = { case Message(body, headers) => { // if exception thrown: actor is restarted or stopped // else self.reply(Ack) // delete file } } override def preRestart(reason: scala.Throwable) { self.reply_?(Failure(reason)) // redeliver file } override def postStop() { self.reply_?(Failure(…)) // or Ack to delete file }}

31. Consumer Actors• Supervised consumer import akka.actor._ import akka.config.Supervision._ // Create consumer actor reference val consumer = Actor.actorOf[new SupervisedFileConsumer] // Create supervisor for consumer actor val supervisor = Supervisor( SupervisorConfig( OneForOneStrategy( List(classOf[Exception]), 5, 10000), Supervise(consumer, Permanent) :: Nil)) – Restart on any Exception – 5 restart attempts within 10 seconds

32. Consumer Actors• Simplified failure reporting – override def blocking = true – Pros • No need for setting up a supervisor • No need to catch exception within receive • No need to self.reply(Failure(…)) – Cons • Endpoint communicates with actor via !! • Thread blocked waiting for Ack

33. Consumer Actors• Simplified failure reporting class FileConsumer extends Actor with Consumer { override def blocking = true override def autoack = false def endpointUri = "file:messages/in?delete=true" def receive = { case Message(body, headers) => { // if exception thrown: endpoint receives it // else self.reply(Ack) // delete file } } }

34. Typed Consumer Actors• Basic example – HTTP/JMS consumerimport akka.actor._import akka.camel._import org.apache.camel.{Body, Header}trait TypedConsumer { @consume("jetty:http://localhost:8080/example") // in-out def foo(s: String): String @consume("jms:queue:example") // in-only def bar (@Body s: String, @Header("priority") p: Integer) : Unit}class TypedConsumerImpl extends TypedActor with TypedConsumer { def foo(s: String) = "received %s" format s def bar(s: String, p: Integer) = println("received %s (priority = %d)" format(s, p))}

35. Typed Consumer Actors• Annotations – Consumer endpoints • @consume(endpointURI) – Parameter bindings • @Body • @Header •…

36. Typed Consumer Actors• Basic example – HTTP/JMS consumer activation import akka.actor._ import akka.camel._ val service = CamelServiceManager.startCamelService // Activate HTTP and JMS endpoints (asynchronously) TypedActor.newInstance( classOf[TypedConsumer], classOf[TypedConsumerImpl]) // … or wait for HTTP and JMS endpoint activation service.awaitEndpointActivation(2) { TypedActor.newInstance( classOf[TypedConsumer], classOf[TypedConsumerImpl]) }

37. Producer Actors• Basic example – HTTP producer import akka.actor._ import akka.camel._ class HttpProducer extends Actor with Producer { def endpointUri = "jetty:http://localhost:8080/example" } – Camel component: camel-jetty – MEP: in-out – receive inherited from Producer trait – Jetty’s async HTTP client used internally

38. Producer Actors• Basic example – HTTP producer activation import akka.actor._ import akka.camel._ // Producer actors require an initialized CamelContext CamelServiceManager.startCamelService // activate producer actor val producer = Actor.actorOf[HttpProducer].start // POST test message to http://localhost:8080/example producer !! "test" match { case Some(m: Message) => … case Some(f: Failure) => … case None => … } // ! and !!! can also be used

39. Producer Actors• Basic example – JMS producer class JmsProducer extends Actor with Producer { def endpointUri = "jms:queue:example" override def oneway = true } – Camel component: camel-jms – MEP: in-only

40. Producer Actors• oneway = false (default) – Initiates an in-out ME with endpoint – Replies result to initial sender• oneway = true – Initiates an in-only ME with endpoint – No reply to sender• receiveAfterProduce – Change default reply behaviour

41. Producer Actors• Custom replies class JmsProducer extends Actor with Producer { def endpointUri = "jms:queue:example" override def oneway = true override def receiveAfterProduce = { case m: Message => self.reply("enqueue succeeded") case f: Failure => self.reply("enqueue failed") } }

42. Producer Actors• Forward results class HttpProducer extends Actor with Producer { val target: ActorRef = … def endpointUri = "jetty:http://localhost:8080/example" override def receiveAfterProduce = { case msg => target forward msg } } – Chaining of producer actors (pipelines) – Related: Future composition

43. Producer Actors• Future composition val producer1 = Actor.actorOf[HttpProducer1].start val producer2 = Actor.actorOf[HttpProducer2].start // monadic future composition (non-blocking) val future = for { m1: Message <- producer1 !!! Message("test") m2: Message <- producer2 !!! m1 } yield m2 // blocks until result is available – Producer Message val result: pipeline = future.get

44. Actor Components• Are Camel components – Can be used in any Camel route• actor Camel component – Send messages to untyped actors – Provided by akka-camel module• typed-actor Camel component – Send messages to typed actors – Provided by akka-camel-typed module – Extension of Camel’s bean component• Used by akka-camel internally – Routes to consumer actors

45. Actor Components• actor endpoint URI – actor:uuid:[<actor-uuid>][?<params>]• Parameters – autoack: Boolean • System or application-level acknowledgements – blocking: Boolean • Use ! or !! for sending messages to actor• Supported message headers – CamelActorIdentifier • Dynamic routing to actors

46. Actor Components• Example import akka.actor._ // can be any actor (no need for Consumer) val actor = Actor.actorOf[SomeActor].start // ... // Camel route from JMS endpoint to actor from("jms:queue:example") .to("actor:uuid:%s?autoack=false" format actor.uuid) – actor receives messages of type Message

48. Comparison• Criteria – Connectivity – Message processing – Route composition – Other• High-level• Incomplete

49. Camel Routes• Connectivity – Camel components/endpoints• Message processing – Processor interface • Predefined processors (all known EIPs) • Custom processors – Concurrent execution of Processor instances – Mutable messages

50. Camel Routes• Route composition – Camel DSL (Java, Scala, XML)• Other – No built-in mechanism for distributing routes (except via endpoints) – Distribution addressed by ServiceMix and FuseSource Fabric, for example

51. Akka-Camel Routes• Connectivity – Camel components/endpoints managed by • Consumer actors • Producer actors• Message processing – Actor • Predefined (a few in akka.actor.routing) • Custom – Sequential execution of actor instance – Immutable messages

52. Akka-Camel Routes• Route composition – Wiring actors (low-level) – Future composition – … – No integration DSL (yet)• Other – Easy to implementing stateful EIPs (aggregator, resequencer …) – Strong built-in mechanisms for distribution, scalability and fault-tolerance – Basis for a distributed and scalable Enterprise Service Bus (?)

53. Scalaz-Camel Routes• Connectivity – Camel components/endpoints• Message processing – Scala functions • Predefined (some EIPs) • Custom – Concurrent execution – Immutable messages

54. Scalaz-Camel Routes• Route composition – Based on functional programming concepts • Message processors chained via Kleisli composition (>=>) – Continuation-based approach • scala.Responder used as continuation monad – Direct-style DSL• Other – Configurable concurrency strategies – Camel processors can be re-used – Akka integration

55. Scalaz-Camel Routes • Example// custom message processorval validate: MessageProcessor = ...// Kleisli composition of route (+ implicit conversions)val vroute = validate >=> oneway >=> to("jms:queue:valid") >=> { m: Message => m.setBody("order accepted")}// consume messages via HTTP and error handling routesfrom(jetty:http://localhost:8080/orders) attempt { vroute } fallback { case e: ValidationException => { m: Message => ... } >=> failWith(e) case _ => ...} – Details at https://github.com/krasserm/scalaz-camel

56. THANK YOU!