• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Utilizing Groovy based closures for flexible real-time analytics
 

Utilizing Groovy based closures for flexible real-time analytics

on

  • 263 views

Speaker: Mark Johnson ...

Speaker: Mark Johnson
Real-time analytics is the process of monitoring and often reacting to web based events or machine-to-machine communication and then quickly deliver the appropriate response to an end-user or machine process. The challenge though is how can we modify the analytics algorithms and reference data without shutting down the data feeds.
In this session we will demonstrate architectural and implementation patterns utilizing Spring Integration Framework, HBase, and of course Groovy to illustrate how it is possible to dynamically change your enterprise’s real-time algorithms without a system restart.

Statistics

Views

Total Views
263
Views on SlideShare
263
Embed Views
0

Actions

Likes
0
Downloads
7
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Utilizing Groovy based closures for flexible real-time analytics Utilizing Groovy based closures for flexible real-time analytics Presentation Transcript

    • Utilizing Groovy based closures for flexible real-time analytics Mark Johnson markfjohnson@gmail.com © Hortonworks Inc. 2013 Page 1
    • Who Am I? •  Director Consulting Hortonworks •  Distributed Compute fan •  Lazy programmer who prefers to reuse rather than re-implement code •  President New England Java Users Group Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 2
    • Real – Time Analytics Discovery Analytics Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 3
    • Real-Time Analytics Real-time analytics is the use of, or the capacity to use, all available enterprise data and resources when they are needed. It consists of dynamic analysis and reporting, based on data entered into a system less than one minute before the actual time of use. Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 4
    • Changing Analytic models Lost Bits too much time to execute Real-Time Analytic Challenges Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 5
    • Architectural Tradeoffs Performance Requirements: •  Required reaction time •  Resource consumption Architecting the Future of Big Data © Hortonworks Inc. 2013 Changeability Requirements: •  24 x 7 processing •  Models change •  JVM class reload issues Page 6
    • Groovy Closures A Groovy Closure is a code block which is usable via a variable instance for later execution. Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 7
    • Groovy Closure: Example Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 8
    • GroovyShell Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 9
    • Groovy closures and Hotspot 140 120 100 80 60 40 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 •  Groovy script improves with hotspot execution Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 10
    • Architecture Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 11
    • Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 12
    • Solution: Dynamic Code deployment •  Embed GroovyShell() into service activator •  Reference an externally defined Groovy script from a service activator base method implementation. •  Groovy script may be located most anywhere: – Property file – Database – External text file Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 13
    • Service Activator Example Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 14
    • Solution: Load Management Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 15
    • Solution: Fast / High Capacity Archival Storage Real – Time Analytics Discovery Analytics •  Archival data necessary to back test real-time analytics •  Sensor Data samples can lead to erroneous conclusions Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 16
    • Demo Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 17