Camel Desing Patterns Learned Through Blood, Sweat, and Tears
•
12 likes•5,479 views
Goes over the most commnonly used design patterns with Apache Camel. Based on Camel Design Patterns book
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Camel Desing Patterns Learned Through Blood, Sweat, and Tears
Similar to Camel Desing Patterns Learned Through Blood, Sweat, and Tears (20)
More from Bilgin Ibryam
More from Bilgin Ibryam (12)
Recently uploaded
Recently uploaded (20)
Camel Desing Patterns Learned Through Blood, Sweat, and Tears
- 1. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears June 2016 Bilgin Ibryam @bibryam
- 2. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears2 Bilgin Ibryam ● Senior Middleware Architect at Red Hat UK ● Apache Camel Committer and PMC member ● Apache OFBiz Committer and PMC member ● Author of Camel Design Patterns (new) ● Author of Apache Camel Message Routing ● Twitter: @bibryam ● Email: bibryam@gmail.com ● Blog: http://ofbizian.com ● LinkedIn: http://www.linkedin.com/in/bibryam
- 3. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears3 Apache Camel Project Status It has all necessary ingredients for a successful open source project. ● Community: 52 committers, 903 users ● Support by large vendors (Red Hat) ● Connectors (256), DataFormats (40) ● Enterprise Integration Patterns++ ● Domain Specific Language ● Ecosystem: Karaf, ActiveMQ, CXF, Fabric, Hawtio, Spring and others ● Monolith, SOA, Microservices, Serverless Source https://www.openhub.net/p/camel/ Stats Date: 06/06/2016
- 4. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears4 Application Integration with Camel What do you need to know to create great Camel applications?
- 5. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears5 Happy Path Scenarios How Pipes and Filters Pattern looks like in Camel?
- 6. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears6 VETRO What is a typical processing flow for a Camel route? ● Validate: validation, schematron, MSV, Jing, bean validation components ● Enrich: enrich and pollEnrich EIPs, custom beans ● Transform: Data formats, auto type conversion, templating components ● Route: Message routing EIPs ● Operate: this is the essence of the processing flow
- 7. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears7 Edge Component Let's start with a simple Camel route that consumes files
- 8. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears8 Edge Component How to expose the same business functionality to multiple consumers?
- 9. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears9 Edge Component Encapsulate endpoint-specific details and prevent them from leaking into the business logic of an integration flow.
- 10. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears10 Read vs Write Operations How to evolve Read and Write operations independently?
- 11. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears11 CQRS This decouples read from write operations to allow them to evolve independently.
- 12. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears12 Unhappy Path Scenarios Happy paths are the easy ones. More work is required for designing and implementing the unhappy paths. ● Data Integrity Pattern ● Saga Pattern ● Retry Pattern ● Idempotent Filter Pattern ● Circuit Breaker Pattern ● Error Channels Pattern ● Throttling Pattern
- 13. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears13 Data Integrity How hard can it be to copy files from one location to another? Download Data Integrity Chapter: http://bit.ly/came-design-patterns-sample
- 14. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears14 Data Integrity Transactional systems Local transaction manager Global transaction manager
- 15. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears15 Saga How to avoid distributed transactions and ensure data consistency?
- 16. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears16 Saga Ensures that each step of the business process has a compensating action to undo the work completed in the case of partial failures.
- 17. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears17 Retry To enable applications handle anticipated transient failures by transparently retrying a failed operation with expectation it to be successful. ● Which failures to retry? ● How often to retry? ● Idempotency ● Monitoring ● Timeouts and SLAs
- 18. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears18 Retry Camel RedeliveryPolicy ● The most well known retry mechanism in Camel ● Retries only the failing endpoint ● Fully in-memory ● Thread blocking behavior by default ● Can be asynchronous ● Good for small number of quick retries (in milliseconds)
- 19. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears19 Retry ActiveMQ consumer RedeliveryPolicy ● Retries the message from the beginning of the Camel route ● Not used very often, but enabled by default ● Fully in-memory ● Thread blocking by default ● Good for small number of quick retries (in milliseconds)
- 20. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears20 Retry ActiveMQ Broker Redelivery ● ActiveMQ specific and requires custom logic ● It will consume the message again from a queue ● Persisted at the broker rather than application memory ● Can be clustered and use fail over, load balancing, etc ● Good for long persisted retries (in minutes or hours)
- 21. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears21 Circuit Breaker How to guard a system by cascading failures and slow responses from other systems?
- 22. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears22 Circuit Breaker Improves the stability and the resilience of a system by guarding integration points from cascading failures and slow responses. Closed state Open state
- 23. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears23 Circuit Breaker Improves the stability and the resilience of a system by guarding integration points from cascading failures and slow responses.
- 24. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears24 Circuit Breaker Two Circuit Breaker Implementations in Camel 2.18
- 25. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears25 Bulkhead How to enforce resource partitioning and damage containment in order to preserve partial functionality in the case of a failure?
- 26. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears26 Bulkhead Enforces resource partitioning and damage containment in order to preserve partial functionality in the case of a failure. ● Multi-threaded EIPs: Delayer, Multicast, Recipient List, Splitter, Threads, Throttler, Wire Tap, Polling Consumer, ProducerTemplate, and OnCompletion. ● Async Error Handler ● Circuit Breaker EIP Possible Camel bulkhead points:
- 27. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears27 Scalability Scenarios Vertical scaling (performance tuning) ● Endpoints: messaging client buffers, DB client batching, template caching choices ● Concurrent consumers option: Seda, VM, JMS, RabbitMQ, Disruptor, AWS-SQS ● Data types choice: affects content based router, splitter, filter, aggregator ● Multithreading: parallel processing EIPs, threads DSL construct, Seda component, asynchronous redelivery/retry ● Micro optimizations: log tuning, camel sampler EIP, disable JMX, disable message history, disable original message record ● Startup/Shutdown: Use lazyLoadTypeConverters for a faster application startup, or configure the shutdownStrategy for a faster shutdown ● Tune: JVM options, networking and operating system Camel performance tuning blog post: http://bit.ly/camel-tuning Camel performance tuning blog post: http://bit.ly/camel-tuning
- 28. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears28 Horizontal Scaling Service Instance Pattern for accommodating increasing workloads.
- 29. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears29 Service Instance Areas to consider before horizontally scaling a Camel application. ● Service state: load balancer, circuit breaker, resequencer, sampler, throttler, idempotent consumer and aggregator are stateful EIPs! ● Request dispatcher: Messaging, HTTP, file (what about locking?) ● Message ordering: exclusive consumer, message groups, consumer priority, message priority, virtual topics ● Singleton service requirements: for batch jobs, and concurrent polling ● Other resource contention and coupling considerations
- 30. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears30 What did we cover so far?
- 31. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears31 How patterns are changing? What is happening in the IT industry today? ● Canonical Data Model ● Edge Component ● Reusable Route ● Runtime Reconfiguration ● Singleton Service ● Batch jobs in JVM ● Bounded Context ● Standalone services ● Favor code duplication ● Less configuration, more redeployment ● Container managed singleton ● Container scheduling ● Circuit Breaker, Bulkhead, Health checks..
- 32. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears32 Win a print copy of Camel Design Patterns When was the first commit to Apache Camel project done?
- 33. Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears33 Win a print copy of Camel Design Patterns When was the first commit to Apache Camel project done?
- 34. More Information Learn more about Apache Camel: http://camel.apache.org Check out Camel Design Patterns: https://leanpub.com/camel-design-patterns Develop Apache Camel based integrations using Red Hat JBoss Fuse: red.ht/FuseDev