Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Realtime Business Platform Architecture Review

61 views

Published on

Learn about different real-time platform architectures leveraging technologies like Spark, Akka, Cassandra, and Kafka.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Realtime Business Platform Architecture Review

  1. 1. Real-time platforms using Spark, Akka, Cassandra, Kafka, Alpakka, Kafka Connect, & Kafka Streams. Realtime Business Platform Architecture Review
  2. 2. Business Platform Success We design, build, and manage business platforms by leveraging DataStax, Sitecore, Salesforce, Quickbooks and other cloud software.
  3. 3. Business Use Case for Real-time
  4. 4. Business Use Case for Real-time
  5. 5. Business Use Case for Real-time
  6. 6. Reactive Manifesto
  7. 7. Message Assurance
  8. 8. Machine Message Assurance • Durable • Scalable • Responsive • Available • Auditable ● Sources ○ Queues ○ Rabbit ○ SQS ○ SNS ○ Kinesis ● Sinks ○ S3 ○ Cassandra ○ Redshift ○ RDS
  9. 9. Machine Message Assurance: Get the Message • ? to Kafka • ? to Akka* • ? to Alpakka • ? to Spark • AWS Kinesis • AWS SQS • AWS SNS • RabbitMQ *Does not require Kafka
  10. 10. Machine Message Assurance: Save the Message • Kafka to ? • Akka* to ? • Alpakka* to ? • Spark to ? • S3 • Cassandra • Redshift • Dynamo *Does not require Kafka
  11. 11. Machine Message Assurance: General Strategy ● Standardize message format for durable store. ○ JSON, Avro? ● Standardize heterogenous message processor. ○ Kafka, Spark, Akka, Alpakka? ● Standardize durable storage mechanism. ○ S3, Cassandra?
  12. 12. Real-time Analytics
  13. 13. Real-time + Batch Analytics = Lambda
  14. 14. Real-time Analytics • Deterministic • Available • Machine Learning • Data Enrichment • Business Intelligence Tools • SQL ● Data ○ S3 ○ Cassandra ○ Redshift ○ RDS ● Analysis ○ Spark ■ Python ■ Scala ■ Java ■ R ○ SQL
  15. 15. Real-time Analytics ● Message Processing / Materialization ○ Kafka SQL ○ Kafka Connect ○ Samza ○ Akka ○ Alpakka ● Colocated Data ○ Cassandra + Spark SQL ○ Cassandra + Spark ○ Cassandra + Spark + Solr
  16. 16. Real-time Analytics : SMACK Stack
  17. 17. Kafka
  18. 18. Kafka as Event Store
  19. 19. SQS, Kafka, Spark, Cassandra, Redshift on AWS
  20. 20. Kafka Streams
  21. 21. Kafka Connect + DataStax Connector
  22. 22. Kafka w/ Machine Learning
  23. 23. Spark
  24. 24. Spark + Streaming + Static Data Sources
  25. 25. Spark + Tableau for Business Intelligence
  26. 26. Kafka + Spark
  27. 27. Spark + Kafka Data Processing
  28. 28. Simplified Lambda with Kafka + Spark + Cassandra
  29. 29. Kafka + Spark + Akka
  30. 30. Canonical Kafka Spark Cassandra Stack
  31. 31. Akka + Spark + Kafka + Cassandra
  32. 32. Apache Pulsar
  33. 33. Pulsar Architecture
  34. 34. Pulsar Subscription Modes
  35. 35. Pulsar Geo Replication
  36. 36. Pulsar Tiered Storage
  37. 37. www.anant.us | solutions@anant.us | (855) 262-6826 3 Washington Circle, NW | Suite 301 | Washington, DC 20037 Data & Analytics Cassandra, DataStax, Kafka, Spark Customer Experience Sitecore Information Systems Salesforce, Quickbooks, and more

×