Systems Integration in the NoSQL Era         with Apache Camel         Kai Wähner         kwaehner@talend.com         @Kai...
Kai Wähner                                                                                                Main Tasks      ...
What is the problem?                                                                                           Growth     ...
A new era: NoSQL© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Solution: systems integration                                                                 All Roads lead              ...
Wishes for integrators                                                          • Standardized Modeling                   ...
Systems integration in the NoSQL era© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
What is the key message?© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Key messages   NoSQL cannot be avoided, and must be integrated!   NoSQL integration is already possible!   Apache Camel he...
Agenda       1) Introduction to NoSQL       2) Introduction to Apache Camel       3) Integration of a Graph-oriented Datab...
Agenda       1) Introduction to NoSQL       2) Introduction to Apache Camel       3) Integration of a Graph-oriented Datab...
The evolving database landscape© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Complementary Concepts                 NoSQL                                                           Cloud              ...
Agenda       1) Introduction to NoSQL       2) Introduction to Apache Camel       3) Integration of a Graph-oriented Datab...
Alternatives for systems integration                                                                                      ...
Enterprise Integration Patterns (EIP)                                                            Apache Camel             ...
Enterprise Integration Patterns (EIP)© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Enterprise Integration Patterns (EIP)© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Architecture                                                                http://java.dzone.com/articles/apache-camel-in...
Choose your favorite DSL                                                                           XML                    ...
Choose your required components                        TCP   SQL                                                          ...
Deploy it wherever you need Standalone                                                         Application Server         ...
Enterprise-ready                                                                                     • Open Source        ...
Community  Camel rocks!                                                                                         Mailing L...
Live demo                Apache Camel in action...© Talend 2013        “Systems Integration in the NoSQL Era with Apache C...
Agenda       1) Introduction to NoSQL       2) Introduction to Apache Camel       3) Integration of a Graph-oriented Datab...
Graph-oriented database© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Graph-oriented database                                              • Neo Technology                                     ...
Code example: Neo4j Ruby API© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Code example: camel-neo4j component   // Producer    from("jms:createNewNeo4jNode")           .to("neo4j:http://Neo4jServe...
Live demo  Integration of a graph-oriented database in action...© Talend 2013   “Systems Integration in the NoSQL Era with...
Agenda       1) Introduction to NoSQL       2) Introduction to Apache Camel       3) Integration of a Graph-oriented Datab...
Key-Value database© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Key-Value database                                              • Part of Amazon Web Services (AWS)                       ...
Code example: AWS S3 Java SDK                AmazonS3 s3 = new AmazonS3Client(new PropertiesCredentials(                  ...
Code example: camel-aws component   // Producer   from(„jms:toS3Queue")      .setHeader(S3Constants.KEY, simple(“order.txt...
Tooling on top of Camel: Talend ESB           Development                                              Runtime            ...
Tooling on top of Camel: Talend ESB       Development                                                            Runtime  ...
Tooling on top of Camel: Talend ESB                                                                                       ...
Live demo                Integration of a key-value database in action...© Talend 2013         “Systems Integration in the...
Alternative to Vendor APIs                ➜      Generic APIs© Talend 2013   “Systems Integration in the NoSQL Era with Ap...
jClouds (Generic API)                                               Generic API for IaaS© Talend 2013   “Systems Integrati...
jClouds (Generic API)                                                                                         Compute API ...
jClouds (Generic API)                                              Several different                                      ...
jClouds (Generic API) – AWS S3 Blobstore (Java)                                                                           ...
Code example: camel-jclouds component      from("direct:toJcloudsAwsS3")        .setHeader(JcloudsConstants.BLOB_NAME, "jc...
Agenda       1) Introduction to NoSQL       2) Introduction to Apache Camel       3) Integration of a Graph-oriented Datab...
In-memory database© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
In-memory database                                              • In-memory data grid                                     ...
Code example: Hazelcast Java API© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Code example: camel-hazelcast component   // Producer   from("direct:add")       .setHeader(HazelcastConstants.OPERATION, ...
Live demo       Integration of an in-memory database in action...© Talend 2013   “Systems Integration in the NoSQL Era wit...
Agenda       1) Introduction to NoSQL       2) Introduction to Apache Camel       3) Integration of a Graph-oriented Datab...
Document-oriented database© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Document-oriented database                                              • 10gen                                           ...
Code example: MongoDB Java Driver© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Code example: camel-mongodb component        // Producer        from(“jms:FlightDocumentQueue")            .to("mongodb:my...
Live demoIntegration of a document-oriented database in action...© Talend 2013   “Systems Integration in the NoSQL Era wit...
Agenda       1) Introduction to NoSQL       2) Introduction to Apache Camel       3) Integration of a Graph-oriented Datab...
Column-oriented database© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
HBase                                              • Modeled after Googles BigTable                                       ...
Code example: HBase Java API© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Code example: camel-hbase component© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Live demo Integration of a column-oriented database in action...© Talend 2013   “Systems Integration in the NoSQL Era with...
Agenda       1) Introduction to NoSQL       2) Introduction to Apache Camel       3) Integration of a Graph-oriented Datab...
Custom NoSQL components                                                                                         Component ...
Live demo                Custom NoSQL components in action...© Talend 2013     “Systems Integration in the NoSQL Era with ...
Alternative for custom NoSQL components                                                                                   ...
Code example: REST API for Salesforce object store   // Salesforce Query (SOQL) via REST API   from("direct:salesforceViaH...
Live demo                   NoSQL integration via REST in action...© Talend 2013   “Systems Integration in the NoSQL Era w...
SQL is still very alive, of course...                                                                                     ...
Did you get the key message?© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Key messages   NoSQL cannot be avoided, and must be integrated!   NoSQL integration is already possible!   Apache Camel he...
Did you get the key message?© Talend 2013   “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
Thank you for your attention. Questions?                 kwaehner@talend.com                   www.kai-waehner.de         ...
Upcoming SlideShare
Loading in...5
×

Systems Integration in the NoSQL Era with Apache Camel (Neo4j, CouchDB, AWS S3, Riak, MongoDB, HBase, Hazelcast)

3,302

Published on

SQL cannot solve several problems emerging with big data. In the future you will have to integrate NoSQL databases, too. The open source integration framework Apache Camel is already prepared for this challenging task. Several examples are shown for integrating NoSQL databases from CouchDB (Document Store), HBase (Column-oriented), Neo4j (Graph), Amazon Web Services (Key Value Store), and others.

Published in: Technology
1 Comment
7 Likes
Statistics
Notes
No Downloads
Views
Total Views
3,302
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
1
Likes
7
Embeds 0
No embeds

No notes for slide

Transcript of "Systems Integration in the NoSQL Era with Apache Camel (Neo4j, CouchDB, AWS S3, Riak, MongoDB, HBase, Hazelcast)"

  1. 1. Systems Integration in the NoSQL Era with Apache Camel Kai Wähner kwaehner@talend.com @KaiWaehner www.kai-waehner.de 2/28/2013© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  2. 2. Kai Wähner Main Tasks Requirements Engineering Enterprise Architecture Management Business Process Management Architecture and Development of Applications Service-oriented Architecture Integration of Legacy Applications Cloud Computing Big Data Consulting Contact Developing Coaching Email: kwaehner@talend.com Speaking Blog: www.kai-waehner.de/blog Writing Twitter: @KaiWaehner Social Networks: Xing, LinkedIn© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  3. 3. What is the problem? Growth • Applications • Interfaces • Technologies • Products© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  4. 4. A new era: NoSQL© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  5. 5. Solution: systems integration All Roads lead to Rome ...© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  6. 6. Wishes for integrators • Standardized Modeling • Efficient Realization • Automatic Testing© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  7. 7. Systems integration in the NoSQL era© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  8. 8. What is the key message?© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  9. 9. Key messages NoSQL cannot be avoided, and must be integrated! NoSQL integration is already possible! Apache Camel helps a lot!© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  10. 10. Agenda 1) Introduction to NoSQL 2) Introduction to Apache Camel 3) Integration of a Graph-oriented Database 4) Integration of a Key-Value Database 5) Integration of an In-Memory Database 6) Integration of a Document-oriented Database 7) Integration of a Column-oriented Database 8) Custom NoSQL Components Live Demos© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  11. 11. Agenda 1) Introduction to NoSQL 2) Introduction to Apache Camel 3) Integration of a Graph-oriented Database 4) Integration of a Key-Value Database 5) Integration of an In-Memory Database 6) Integration of a Document-oriented Database 7) Integration of a Column-oriented Database 8) Custom NoSQL Components© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  12. 12. The evolving database landscape© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  13. 13. Complementary Concepts NoSQL Cloud Big Data© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  14. 14. Agenda 1) Introduction to NoSQL 2) Introduction to Apache Camel 3) Integration of a Graph-oriented Database 4) Integration of a Key-Value Database 5) Integration of an In-Memory Database 6) Integration of a Document-oriented Database 7) Integration of a Column-oriented Database 8) Custom NoSQL Components© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  15. 15. Alternatives for systems integration Enterprise Service Bus (ESB) Integration Framework API Complexity of Integration Low High Apache Camel vs. Spring Integration vs. Mule http://www.kai-waehner.de/blog/2012/01/10/spoilt-for-choice-which-integration-framework-to-use-spring-integration-mule-esb-or-apache-camel/© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  16. 16. Enterprise Integration Patterns (EIP) Apache Camel Implements the EIPs© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  17. 17. Enterprise Integration Patterns (EIP)© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  18. 18. Enterprise Integration Patterns (EIP)© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  19. 19. Architecture http://java.dzone.com/articles/apache-camel-integration© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  20. 20. Choose your favorite DSL XML (not production-ready yet)© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  21. 21. Choose your required components TCP SQL SMTP JMS Netty Jetty RMI FTP Lucene JDBC EJB Bean-Validation MQ IRC JMX Quartz RSS AMQP Atom Log AWS-S3 HTTP XSLT LDAP File Akka Many many more CXF Custom Components© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  22. 22. Deploy it wherever you need Standalone Application Server Web Container Spring Container OSGi Cloud© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  23. 23. Enterprise-ready • Open Source • Scalability • Error Handling • Transaction • Monitoring • Tooling • Commercial Support© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  24. 24. Community  Camel rocks! Mailing Lists? Forums? Blogs? Articles? Conference talks? ESBs? Professionals? Jobs? Knowledge?© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  25. 25. Live demo Apache Camel in action...© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  26. 26. Agenda 1) Introduction to NoSQL 2) Introduction to Apache Camel 3) Integration of a Graph-oriented Database 4) Integration of a Key-Value Database 5) Integration of an In-Memory Database 6) Integration of a Document-oriented Database 7) Integration of a Column-oriented Database 8) Custom NoSQL Components© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  27. 27. Graph-oriented database© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  28. 28. Graph-oriented database • Neo Technology • Graphs rather than tables • Nodes, edges, and properties to represent and store data • Index-free adjacency • REST API and many SDKs (Java, .NET, Ruby, PHP, Python, etc.) • Embedded, disk-based, fully transactional • Powerful tool for graph-like queries • Example: Facebook friends© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  29. 29. Code example: Neo4j Ruby API© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  30. 30. Code example: camel-neo4j component // Producer from("jms:createNewNeo4jNode") .to("neo4j:http://Neo4jServer:7474/data"); // Consumer from(„neo4j://todo)... Not implemented in current Camel release (2.11)   Use Camel‘s REST components (shown in some minutes...)© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  31. 31. Live demo Integration of a graph-oriented database in action...© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  32. 32. Agenda 1) Introduction to NoSQL 2) Introduction to Apache Camel 3) Integration of a Graph-oriented Database 4) Integration of a Key-Value Database 5) Integration of an In-Memory Database 6) Integration of a Document-oriented Database 7) Integration of a Column-oriented Database 8) Custom NoSQL Components© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  33. 33. Key-Value database© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  34. 34. Key-Value database • Part of Amazon Web Services (AWS) • Online storage web service • Store arbitrary objects (computer files) up to 5 terabytes • REST and SOAP API • SDKs for Java, .NET, PHP, Ruby, etc. • Highly-scalable, reliable, and low-latency • Alternative for Hadoop‘s file system HDFS • Example: DigitalChalk offers creating, delivering and managing training videos© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  35. 35. Code example: AWS S3 Java SDK AmazonS3 s3 = new AmazonS3Client(new PropertiesCredentials( S3Sample.class.getResourceAsStream("AwsCredentials.properties"))); String bucketName = "my-first-s3-bucket-" + UUID.randomUUID(); String key = "MyObjectKey"; try { s3.createBucket(bucketName); s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile())); S3Object object = s3.getObject(new GetObjectRequest(bucketName, key)); ObjectListing objectListing = s3.listObjects(new ListObjectsRequest() .withBucketName(bucketName) .withPrefix("My")); s3.deleteObject(bucketName, key); s3.deleteBucket(bucketName); } catch (AmazonServiceException ase) { // error handling... } catch (AmazonClientException ace) { // error handling... }© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  36. 36. Code example: camel-aws component // Producer from(„jms:toS3Queue") .setHeader(S3Constants.KEY, simple(“order.txt")) .to("aws-s3://myBucket?accessKey=" + a + "&secretKey= " + s) // Consumer from("aws-s3://myBucket?accessKey=“ + a + "&secretKey=" + s) .to("log:S3logging")© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  37. 37. Tooling on top of Camel: Talend ESB Development Runtime Operation Talend Administration Center Service Development Web Services Stack Security Management Mediation & Mediation & Loadbalancing & High Configuration Integration Integration Availability Talend ESB Studio Project Testing Message Broker Business Rules Repository Talend ESB Deployment Performance & Build & Deploy Service Container Repository Availability Documentation & Examples 24x7 Support Training & Certification Indemnification Maintenance Professional Services Certified Partners© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  38. 38. Tooling on top of Camel: Talend ESB Development Runtime Operation Management, Eclipse STP/WTP Apache CXF Secure Token Server Talend Administration Center Configuration & soapUI REST & Web Services Security Monitoring Service Locator & Talend ESB Runtime Route Designer Apache Camel Service Act. Service Activity Monitoring and Service Locator UI Talend ESB Studio Mediation Mediation Monitoring Distributed Registry / Tracking Service Designer Apache ActiveMQ Apache Archiva Repository Integration Message Broker Artifact repository Metadata & Projects Apache Maven Apache Karaf / Cellar Eclipse Equinox vFabric Build & Deploy OSGi / Clustering OSGi Hyperic HQ Documentation & Examples 24x7 Support Training & Certification Indemnification Maintenance Professional Services Certified Partners© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  39. 39. Tooling on top of Camel: Talend ESB Route Builder  Endpoints  EIPs  Processors  Custom components Configuration  Components  Endpoints Code Generation  100% Java  Camel Code  Packaged as OSGi Bundles Execution in the IDE  Debugging  Live statistics  Short dev cycles© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  40. 40. Live demo Integration of a key-value database in action...© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  41. 41. Alternative to Vendor APIs ➜ Generic APIs© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  42. 42. jClouds (Generic API) Generic API for IaaS© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  43. 43. jClouds (Generic API) Compute API Blobstore API© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  44. 44. jClouds (Generic API) Several different Cloud providers supported© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  45. 45. jClouds (Generic API) – AWS S3 Blobstore (Java) Use another provider? Just change this line!© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  46. 46. Code example: camel-jclouds component from("direct:toJcloudsAwsS3") .setHeader(JcloudsConstants.BLOB_NAME, "jclouds-demo-tutorial.txt") .setHeader(JcloudsConstants.CONTAINER_NAME, "kw-s3-data") .to("jclouds:blobstore:aws-s3") from("direct:toJcloudsMicrosoftAzure") .setHeader(JcloudsConstants.BLOB_NAME, "jclouds-demo-tutorial.txt") .setHeader(JcloudsConstants.CONTAINER_NAME, "kw-s3-data") .to("jclouds:blobstore:azureblob")© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  47. 47. Agenda 1) Introduction to NoSQL 2) Introduction to Apache Camel 3) Integration of a Graph-oriented Database 4) Integration of a Key-Value Database 5) Integration of an In-Memory Database 6) Integration of a Document-oriented Database 7) Integration of a Column-oriented Database 8) Custom NoSQL Components© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  48. 48. In-memory database© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  49. 49. In-memory database • In-memory data grid • Clustering and highly scalable data distribution solution for Java platform • Architecture is peer-to-peer • Distributed Java data structures (Queue, Set, List, Map, Lock, Topic) • Java and REST API© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  50. 50. Code example: Hazelcast Java API© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  51. 51. Code example: camel-hazelcast component // Producer from("direct:add") .setHeader(HazelcastConstants.OPERATION, „add“) .to("hazelcast:queue:foo"); // Consumer from("hazelcast:queue:foo") .log(“content of object foo: ${body}");© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  52. 52. Live demo Integration of an in-memory database in action...© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  53. 53. Agenda 1) Introduction to NoSQL 2) Introduction to Apache Camel 3) Integration of a Graph-oriented Database 4) Integration of a Key-Value Database 5) Integration of an In-Memory Database 6) Integration of a Document-oriented Database 7) Integration of a Column-oriented Database 8) Custom NoSQL Components© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  54. 54. Document-oriented database© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  55. 55. Document-oriented database • 10gen • stores structured data as JSON-like documents with dynamic schemas • REST API and several SDKs (Java, .NET, Ruby, PHP, Python, etc.) • Ad hoc queries, indexing, replication, load balancing • Powerful, but also easy to use and flexible • Example: Disney persists state information of online games in a common object repository.© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  56. 56. Code example: MongoDB Java Driver© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  57. 57. Code example: camel-mongodb component // Producer from(“jms:FlightDocumentQueue") .to("mongodb:myDb?database=flights &collection=tickets &operation=insert"); // Consumer from("mongodb:myDb?database=flights &collection=cancellations &tailTrackIncreasingField=departureTime") .to(“jms:CancelledFlightsQueue");© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  58. 58. Live demoIntegration of a document-oriented database in action...© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  59. 59. Agenda 1) Introduction to NoSQL 2) Introduction to Apache Camel 3) Integration of a Graph-oriented Database 4) Integration of a Key-Value Database 5) Integration of an In-Memory Database 6) Integration of a Document-oriented Database 7) Integration of a Column-oriented Database 8) Custom NoSQL Components© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  60. 60. Column-oriented database© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  61. 61. HBase • Modeled after Googles BigTable • Runs on top of HDFS (Hadoop Distributed Filesystem) • Can serve as the input and output for MapReduce jobs run in Hadoop • Stores data tables as sections of columns of data rather than as rows of data • Java API plus REST, Avro or Thrift gateway APIs • Use HBase when you need random, realtime read/write access to your Big Data • Example: Advantages for DWHs, CRMs, and other ad-hoc inquiry systems where aggregates are computed over large numbers of similar data items.© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  62. 62. Code example: HBase Java API© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  63. 63. Code example: camel-hbase component© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  64. 64. Live demo Integration of a column-oriented database in action...© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  65. 65. Agenda 1) Introduction to NoSQL 2) Introduction to Apache Camel 3) Integration of a Graph-oriented Database 4) Integration of a Key-Value Database 5) Integration of an In-Memory Database 6) Integration of a Document-oriented Database 7) Integration of a Column-oriented Database 8) Custom NoSQL Components© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  66. 66. Custom NoSQL components Component Creates Endpoint Consumer Producer© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  67. 67. Live demo Custom NoSQL components in action...© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  68. 68. Alternative for custom NoSQL components • SOAP • REST© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  69. 69. Code example: REST API for Salesforce object store // Salesforce Query (SOQL) via REST API from("direct:salesforceViaHttpLIST") .setHeader("X-PrettyPrint", 1) .setHeader("Authorization", accessToken) .setHeader(Exchange.CONTENT_TYPE, "application/json") .to("https://na14.salesforce.com/services/data/v20.0/query?q=SELECT+name+from +Article__c") // Salesforce CREATE via REST API from("direct:salesforceViaHttpCREATE") .setHeader("X-PrettyPrint", 1) .setHeader("Authorization", accessToken) .setHeader(Exchange.CONTENT_TYPE, "application/json“) .to("https://na14.salesforce.com/services/data/v20.0/sobjects/Article__c")© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  70. 70. Live demo NoSQL integration via REST in action...© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  71. 71. SQL is still very alive, of course... Camel SQL components: • sql:statement • jdbc:dataSourceName • jpa://entityName • mybatis://statementName • hibernate://entityName© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  72. 72. Did you get the key message?© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  73. 73. Key messages NoSQL cannot be avoided, and must be integrated! NoSQL integration is already possible! Apache Camel helps a lot!© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  74. 74. Did you get the key message?© Talend 2013 “Systems Integration in the NoSQL Era with Apache Camel” by Kai Wähner
  75. 75. Thank you for your attention. Questions? kwaehner@talend.com www.kai-waehner.de LinkedIn / Xing @KaiWaehner

×