System Integration with Akka and Apache Camel

Connecting Akka to the
Rest of the World
System integration with
Akka and Apache Camel

by
Martin Krasser

About me
• Freelance software engineer, architect
– eHealth application integration (Java, Scala, Groovy)
– Bioinformatics, Cheminformatics (C++, Java)
– …

• Open source committer
– Akka
– Camel
– Scalaz-Camel
– …

• Java since 1998
• Scala since 2009

• Twitter: @mrt1nz

Overview
• Introduction
– Camel
– Akka

• Akka-Camel integration
– Consumer actors
– Producer actors
– Actor components

• Comparison
– Camel routes
– Akka-Camel routes
– Scalaz-Camel routes

Apache Camel
• Integration framework written in Java
– Open source, Apache 2 license
– http://camel.apache.org

• Enterprise Integration Patterns (EIP)
– G. Hohpe, B. Woolf

• Domain-specific language (DSL)
– Internal: Java, Scala
– External: XML/Spring

• Pluggable protocol and API bindings
– HTTP, FTP, JMS, File … and approx. 100 more
– Camel “components” http://camel.apache.org/components.html

Apache Camel
• Usage in Scala applications

– Camel Java/Scala DSL
• http://camel.apache.org

– Akka-Camel integration module
• http://akka.io/docs/akka-modules/1.1.2/modules/camel.html

– Scalaz-Camel DSL
• https://github.com/krasserm/scalaz-camel

Apache Camel
• Routing example
D D

HTTP
jms:queue:docs JMS HTTP
Filter
Endpoint Endpoint
http://example.org/docs

Java DSL:
from("jms:queue:docs")
.filter().xpath("/person[@name='Jon']")
.to("http:example.org/docs");

Scala DSL:
"jms:queue:docs"
when(_.in == "…")
to("http:example.org/docs");

Apache Camel
• Messaging endpoints
– Configured via endpoint URI
• "jms:queue:docs"
• "http://example.org/docs"
• …
– Endpoint consumer: from(uri)
– Endpoint producer: to(uri)

• Message processors …
– Configured via fluent API
• filter().xpath("…")
• transform(body().append("…"))
• …
– Custom Java/Scala code (Processor interface)

Akka
• Platform for event-driven, scalable and fault-
tolerant architectures on the JVM

• Concurrency
– Actors, agents, dataflow, STM

• Scalability Core
– Remote actors, cluster membership Services

• Fault tolerance
– Local and remote actor supervision

Akka
• Add-on modules

– akka-camel
• Camel integration (untyped actors)

– akka-camel-typed
• Camel integration (typed actors)

– akka-kernel
• Microkernel

– akka-scalaz
• Scalaz support

– akka-spring
• Spring integration

– …

Akka
• Actor model
– Mathematical model of concurrent computation
– Carl Hewitt, 1973

• Akka actor
– Encapsulates state
– Encapsulates behavior
– Exchanges messages
– Mailbox
– Event-driven
– Single-threaded

• API
– Scala
– Java

Akka
• Actor model benefits
– Easier to reason about
– Raised abstraction level
– Easier to avoid
• Race conditions
• Dead locks
• Starvation
• Live locks

Akka Actors
• Receive messages
import akka.actor.Actor

// actor definition
class ExampleActor extends Actor {
def receive = {
case "test" => …
case _ => …
}
}

// create actor reference (ActorRef)
val actor = Actor.actorOf[ExampleActor]

// start actor
actor.start

Akka Actors
• Send messages

– Fire-Forget
actor ! "test"

Akka Actors
• Send messages

– Send-And-Receive-Eventually
actor !! "test" match {
case Some(result) => … // process result
case None => … // timeout
}
uses Future under the hood (with timeout)

Akka Actors
• Send messages

– Send-And-Receive-Future
val future = actor !!! "test"
future.await
val result = future.get

returns Future directly

Akka-Camel Integration
• Exchange messages with Akka actors via
– HTTP, FTP, JMS, File … and approx. 100 more
– Any Camel component can be used (pluggable)

• Integration layer of Akka
– Consumer actors: receive message from Camel endpoints
– Producer actors: produce messages to Camel endpoints
– Message exchange patterns (MEPs): in-out and in-only

• Leverages Camel’s asynchronous routing engine
– Asynchronous completion of message exchange
– No threads blocked waiting for response messages, for example

Consumer Actors
• Basic example – TCP consumer
•
import akka.actor._
import akka.camel._

class TcpConsumer extends Actor with Consumer {
def endpointUri = "mina:tcp://localhost:6200?textline=true"
def receive = {
case Message(body, headers) => {
self.reply("received %s" format body)
}
}
}

– Camel component: camel-mina
– MEP: in-out

Consumer Actors
• Basic example – TCP consumer activation
import akka.actor._
import akka.camel._

// Consumer actors require a running CamelService
CamelServiceManager.startCamelService

// Activate TCP endpoint (asynchronously)
Actor.actorOf[TcpConsumer].start

– Asynchronous activation of consumer endpoint

Consumer Actors
• Basic example – TCP consumer activation
import akka.actor._
import akka.camel._

// Consumer actors require a running CamelService
val service = CamelServiceManager.startCamelService

// Wait for TCP endpoint activation
service.awaitEndpointActivation(1) {
Actor.actorOf[TcpConsumer].start
}

– Wait for endpoint activation
– Useful for testing purposes

Consumer Actors
• Basic example – TCP consumer deactivation
import akka.actor._
import akka.camel._

val service: CamelService = …
val tcpConsumer: ActorRef = …

// deactivate TCP endpoint (asynchronously)
tcpConsumer.stop

// … or wait for TCP endpoint deactivation
service.awaitEndpointDeactivation(1) {
tcpConsumer.stop
}

Consumer Actors
• Basic example – HTTP consumer
class HttpConsumer extends Actor with Consumer {
def endpointUri = "jetty:http://localhost:8080/example"
def receive = {
case msg: Message => {
self.reply("received %s" format msg.bodyAs[String])
}
}
}

– Camel component: camel-jetty
– MEP: in-out
– Jetty continuations used internally
– Camel type converter (msg.bodyAs[…])

Consumer Actors
• Basic example – JMS consumer
class JMSConsumer extends Actor with Consumer {
def endpointUri = "jms:queue:example"
def receive = {
case Message(body, headers) => ...
}
}

– Camel component: camel-jms
– MEP: in-only

Consumer Actors
• Basic example – IMAP consumer
class MailConsumer extends Actor with Consumer {
def endpointUri =
"imap://admin@mymailserver.com?password=secret"
def receive = {
case Message(body, headers) => ...
}
}

– Camel component: camel-mail
– MEP: in-only

Consumer Actors
• Basic example – Scheduled consumer
class ScheduledConsumer extends Actor with Consumer {
def endpointUri =
"quartz://example?cron=0/2+*+*+*+*+?"
def receive = {
case tick => ...
}
}

– Camel component: camel-quartz
– MEP: in-only

Consumer Actors
• Acknowledgements
import akka.camel.Ack
...

class FileConsumer extends Actor with Consumer {
override def autoack = false // default is true
def endpointUri = "file:messages/in?delete=true"
def receive = {
// ...
self.reply(Ack) // delete consumed file
}
}
}

– Camel component: camel-file
– MEP: in-only
– Application-level acknowledgement (Ack)

Consumer Actors
• Failure replies
import akka.camel.Failure
...

override def autoack = false // default is true
def receive = {
// ...
self.reply(Failure(reason)) // redeliver file
}
}
}

– Application-level negative acknowledgement (Failure)
– Redelivery works also with JMS endpoints, for example
– Failure replies can also be used with in-out MEP

Consumer Actors
• Failure replies

– Should not be made within receive

– Let consumer actors crash (on Exception)

– Use a supervisor for failure replies

Consumer Actors
• Supervised consumer
class SupervisedFileConsumer extends Actor with Consumer {
override def autoack = false
def receive = {
// if exception thrown: actor is restarted or stopped
// else
self.reply(Ack) // delete file
}
}
override def preRestart(reason: scala.Throwable) {
self.reply_?(Failure(reason)) // redeliver file
}
override def postStop() {
self.reply_?(Failure(…)) // or Ack to delete file
}
}

Consumer Actors
• Supervised consumer
import akka.actor._
import akka.config.Supervision._

// Create consumer actor reference
val consumer = Actor.actorOf[new SupervisedFileConsumer]

// Create supervisor for consumer actor
val supervisor = Supervisor(
SupervisorConfig(
OneForOneStrategy(
List(classOf[Exception]), 5, 10000),
Supervise(consumer, Permanent) :: Nil))

– Restart on any Exception
– 5 restart attempts within 10 seconds

Consumer Actors
• Simplified failure reporting
– override def blocking = true

– Pros
• No need for setting up a supervisor
• No need to catch exception within receive
• No need to self.reply(Failure(…))

– Cons
• Endpoint communicates with actor via !!
• Thread blocked waiting for Ack

Consumer Actors
• Simplified failure reporting
override def blocking = true
override def autoack = false
def receive = {
// if exception thrown: endpoint receives it
// else
self.reply(Ack) // delete file
}
}
}

Typed Consumer Actors
• Basic example – HTTP/JMS consumer
import akka.actor._
import akka.camel._
import org.apache.camel.{Body, Header}

trait TypedConsumer {
@consume("jetty:http://localhost:8080/example") // in-out
def foo(s: String): String

@consume("jms:queue:example") // in-only
def bar (@Body s: String, @Header("priority") p: Integer) : Unit
}

class TypedConsumerImpl extends TypedActor with TypedConsumer {
def foo(s: String) = "received %s" format s
def bar(s: String, p: Integer) =
println("received %s (priority = %d)" format(s, p))
}

• Annotations

– Consumer endpoints
• @consume(endpointURI)

– Parameter bindings
• @Body
• @Header
•…

• Basic example – HTTP/JMS consumer activation
import akka.actor._
import akka.camel._

val service = CamelServiceManager.startCamelService

// Activate HTTP and JMS endpoints (asynchronously)
TypedActor.newInstance(
classOf[TypedConsumer],
classOf[TypedConsumerImpl])

// … or wait for HTTP and JMS endpoint activation
service.awaitEndpointActivation(2) {
TypedActor.newInstance(
classOf[TypedConsumer],
classOf[TypedConsumerImpl])
}

Producer Actors
• Basic example – HTTP producer
import akka.actor._
import akka.camel._

class HttpProducer extends Actor with Producer {
}

– Camel component: camel-jetty
– MEP: in-out
– receive inherited from Producer trait
– Jetty’s async HTTP client used internally

Producer Actors
• Basic example – HTTP producer activation
import akka.actor._
import akka.camel._

// Producer actors require an initialized CamelContext
CamelServiceManager.startCamelService

// activate producer actor
val producer = Actor.actorOf[HttpProducer].start

// POST test message to http://localhost:8080/example
producer !! "test" match {
case Some(m: Message) => …
case Some(f: Failure) => …
case None => …
} // ! and !!! can also be used

Producer Actors
• Basic example – JMS producer
class JmsProducer extends Actor with Producer {
override def oneway = true
}
– Camel component: camel-jms
– MEP: in-only

Producer Actors
• oneway = false (default)
– Initiates an in-out ME with endpoint
– Replies result to initial sender

• oneway = true
– Initiates an in-only ME with endpoint
– No reply to sender

• receiveAfterProduce
– Change default reply behaviour

Producer Actors
• Custom replies
class JmsProducer extends Actor with Producer {
override def oneway = true
override def receiveAfterProduce = {
case m: Message => self.reply("enqueue succeeded")
case f: Failure => self.reply("enqueue failed")
}
}

Producer Actors
• Forward results
class HttpProducer extends Actor with Producer {
val target: ActorRef = …
override def receiveAfterProduce = {
case msg => target forward msg
}
}

– Chaining of producer actors (pipelines)
– Related: Future composition

Producer Actors
• Future composition
val producer1 = Actor.actorOf[HttpProducer1].start
val producer2 = Actor.actorOf[HttpProducer2].start

// monadic future composition (non-blocking)
val future = for {
m1: Message <- producer1 !!! Message("test")
m2: Message <- producer2 !!! m1
} yield m2

// blocks until result is available
– Producer Message
val result: pipeline = future.get

Actor Components
• Are Camel components
– Can be used in any Camel route

• actor Camel component
– Send messages to untyped actors
– Provided by akka-camel module

• typed-actor Camel component
– Send messages to typed actors
– Provided by akka-camel-typed module
– Extension of Camel’s bean component

• Used by akka-camel internally
– Routes to consumer actors

Actor Components
• actor endpoint URI
– actor:uuid:[<actor-uuid>][?<params>]

• Parameters
– autoack: Boolean
• System or application-level acknowledgements
– blocking: Boolean
• Use ! or !! for sending messages to actor

• Supported message headers
– CamelActorIdentifier
• Dynamic routing to actors

Actor Components
• Example
import akka.actor._

// can be any actor (no need for Consumer)
val actor = Actor.actorOf[SomeActor].start

// ...

// Camel route from JMS endpoint to actor
from("jms:queue:example")
.to("actor:uuid:%s?autoack=false" format actor.uuid)

– actor receives messages of type Message

Comparison
• Criteria
– Connectivity
– Message processing
– Route composition
– Other

• High-level
• Incomplete

Camel Routes
• Connectivity
– Camel components/endpoints

• Message processing
– Processor interface
• Predefined processors (all known EIPs)
• Custom processors
– Concurrent execution of Processor instances
– Mutable messages

Camel Routes
• Route composition
– Camel DSL (Java, Scala, XML)

• Other
– No built-in mechanism for distributing routes (except
via endpoints)
– Distribution addressed by ServiceMix and
FuseSource Fabric, for example

Akka-Camel Routes
• Connectivity
– Camel components/endpoints managed by
• Consumer actors
• Producer actors

– Actor
• Predefined (a few in akka.actor.routing)
• Custom
– Sequential execution of actor instance
– Immutable messages

Akka-Camel Routes
– Wiring actors (low-level)
– Future composition
– …
– No integration DSL (yet)

• Other
– Easy to implementing stateful EIPs (aggregator,
resequencer …)
– Strong built-in mechanisms for distribution, scalability
and fault-tolerance
– Basis for a distributed and scalable Enterprise Service
Bus (?)

Scalaz-Camel Routes
• Connectivity
– Camel components/endpoints

– Scala functions
• Predefined (some EIPs)
• Custom
– Concurrent execution
– Immutable messages

Scalaz-Camel Routes
– Based on functional programming concepts
• Message processors chained via Kleisli composition (>=>)
– Continuation-based approach
• scala.Responder used as continuation monad
– Direct-style DSL

• Other
– Configurable concurrency strategies
– Camel processors can be re-used
– Akka integration

Scalaz-Camel Routes
• Example
// custom message processor
val validate: MessageProcessor = ...

// Kleisli composition of route (+ implicit conversions)
val vroute = validate >=> oneway >=> to("jms:queue:valid") >=> {
m: Message => m.setBody("order accepted")
}

// consume messages via HTTP and error handling routes
from(jetty:http://localhost:8080/orders) attempt { vroute } fallback {
case e: ValidationException => { m: Message => ... } >=> failWith(e)
case _ => ...
}

– Details at https://github.com/krasserm/scalaz-camel

System Integration with Akka and Apache Camel

More Related Content

Similar to System Integration with Akka and Apache Camel

Recently uploaded

System Integration with Akka and Apache Camel