Real Time Analytics with Spring


Published on

Speakers: Luke Taylor and David Turanski
Today's solutions must provide the ability to interpret related events and understand trends that are happening right now. This session will cover some of the out of the box capabilities of Spring XD to tap into big data streams and generate metrics such as simple counters, aggregate counters, moving averages, rates of change, and histograms. Hands-on demos will show you how Spring XD uses Redis and GemFire's Continuous Query and Function Execution to incorporate real-time analytics into event-driven applications.

Published in: Technology, Business

Real Time Analytics with Spring

  1. 1. Real Time Analytics With Spring David Turanski Luke Taylor
  2. 2. Real Time Analytics What's happening right now? Event driven Notify humans or systems in real time Simple counters and metrics Aggregate Counters Gauges, Moving Average Complex Event Processing
  3. 3. Spring XD Overview One stop shop for: Data Ingestion Real­time Analytics Workflow Orchestration Data Export Built on existing Spring Assets Spring Integration, Batch, Data XD='extreme data'
  4. 4. Spring XD Open source Apache license Out of the box executable servers not a bunch of jars Pluggable Module system Configuration via High level DSL, CLI, and REST services
  5. 5. Spring XD DSL Based on Unix pipes and filters syntax ht |fl tp ie ht -pr=00|fl tp -ot88 ie roDr/aa oti=dt/ ' t p  and ' i e  are Module names ht' fl'
  6. 6. Spring XD Streams
  7. 7. Anatomy of a Stream
  8. 8. XD Single Node
  9. 9. Spring XD Taps A Tap is a stream whose source is a point in an existing stream Taps are primarily used for real time analysis Example: titrerh-qeyBee |fl wtesac -ur=ibr ie tptitrerh>fedvlecutra:wtesac il-au-one fedaeette.ahastx
  10. 10. Spring XD Metrics Spring XD provides persistent and in­memory Metrics Repositories Metrics are modules commonly act as sinks for taps XD includes the following Metrics Counter Field Value Counter Aggregate Counter Gauge Rich Gauge See Spring XD Analytics Documentation for details.
  11. 11. What is GemFire? Pivotal GemFire
  12. 12. GemFire Function Execution Efficient way to process large data volumes "where the data lives" Java code deployed to GemFire server nodes The Function runs on local cache node and transports results over the network to the process that executed the function (Execution) Function Execution aggregates results using a ResultCollector
  13. 13. GemFire Continuous Query Good for asynchronous notification of data updates Client subscribes to a data query and binds it to a QueryListener callback GemFire invokes the QueryListener whenever the query's result set changes state
  14. 14. The Demo Perform real time analytics on the Twitter stream Create a Twitter ingest stream with XD Create taps on the Twitter stream to feed XD Metrics Create a tap to pull selected data into GemFire for deeper analysis Show how to build and deploy a custom processor Execute a GemFire Function to perform a complex analysis of hash tags Dashboard web app to view analytics
  15. 15. Demo Components
  16. 16. Questions?
  17. 17. Learn More. Stay Connected Project Sites Spring XD­xd Spring Data GemFire­data­gemfire Related sessions Thu 12:45 Real Time Event Processing and Decision Making ­ Lyndon Adams XD and Pivotal HD Demo ­ Demo lounge © 2013 SpringOne 2GX ­ Do not distribute without permission