Process Batch transaction using AzureBlob Integration with Apache Camel explained in the ApacheCon 2021 Integration Track

1. Process Batch transaction using AzureBlob
Integration with Apache Camel
Srikant Mantha
Technical Architect / Infosys Limited

2. About Me
• I am Srikant Mantha, working as a Technology Architect at Infosys Limited, Amsterdam.
• I have 15yrs of experience in IT Industry, Integrating applications using
Apache Camel, Spring Integration and Tibco and WebMethods.
www.linkedin.com/in/srikant-mantha
mantha.srikant@outlook.com

3. Agenda
• Problem Statement
• Solution
• Why Microsoft Azure Blob Storage
• Accessing Azure Blob APIs
• Why use Apache Camel
• How Apache Camel Helps
• Architecture
• Demo
• Performance
• References
• Q&A

4. Problem Statement

5. Problem Statement
• Process customers orders in batches
• Build a robust system
• Process large volumes
• Connect Securely
• Process different file-formats

6. Solution

7. Why Microsoft Azure Blob Storage
• Restricted access to FTP/SFTP
• Optimized for Storing unstructured data
• Resource Types
• Storage Accounts
• Containers
• Blobs
• Secure Access to Blob Storage
• Client libraries supported
• Version 12
Storage Account
ShoppingCartOrders
abc.xml
xyz.xml
123cqd.xml
Corp Orders
da50db74c.gzip
203e15b4.gzip
8d5bbb1.gzip
Payment Orders
3973547e.txt
2f114b8f.txt
1e1307d3.txt

8. Accessing Azure Blob APIs
https://apachecon2021.blob.core.windows.net

9. Why Use Apache Camel
Open Source Integration Framework
• Solves complex integration problems
• Abstract Layer to Connect with Component
Business
Logic

10. Why Use Apache Camel
Automatic suspend/resume of Routes
Graceful Shutdown

11. Why Use Apache Camel
• Integrate with Springboot
+
• Keep the app alive
• Extensive support for Testing  @CamelSpringBootTest

12. How Apache Camel Helps
• Dependencies
• Routing
Customize routes using spring configurations

13. Architecture

14. Architecture

15. DEMO

16. Performance
200
303
493
2838
27458
0 5000 10000 15000 20000 25000 30000
10
100
1000
10000
100000
Processing Time (ms)

17. References

18. References
• AzureBlobConnector  https://github.com/srikant-mvs/AzureBlobConnector
• Camel Azure Storage blob Service  https://camel.apache.org/components/latest/azure-storage-blob-component.html
• Camel Springboot starter  https://camel.apache.org/camel-spring-boot/latest/spring-boot.html
• Testing  https://camel.apache.org/manual/latest/testing.html
• Introduction to Azure Storage Blob  https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction
• Azure Storage Libraries  https://docs.microsoft.com/en-us/java/api/overview/azure/storage?view=azure-java-stable
• Azure Storage API  https://javadoc.io/static/com.azure/azure-storage-blob/12.14.0/index.html

19. Q&A

20. Thank You