Your SlideShare is downloading. ×
0
Utilizing Groovy based closures for
flexible real-time analytics
Mark Johnson
markfjohnson@gmail.com

© Hortonworks Inc. 2...
Who Am I?
•  Director Consulting Hortonworks
•  Distributed Compute fan
•  Lazy programmer who prefers to reuse rather tha...
Real – Time
Analytics

Discovery
Analytics

Architecting the Future of Big Data
© Hortonworks Inc. 2013

Page 3
Real-Time Analytics
Real-time analytics is the use of, or
the capacity to use, all available
enterprise data and resources...
Changing
Analytic
models
Lost Bits

too much
time to
execute

Real-Time
Analytic
Challenges

Architecting the Future of Bi...
Architectural Tradeoffs

Performance Requirements:
•  Required reaction time
•  Resource consumption

Architecting the Fut...
Groovy Closures
A Groovy Closure is a code block
which is usable via a variable
instance for later execution.

Architectin...
Groovy Closure: Example

Architecting the Future of Big Data
© Hortonworks Inc. 2013

Page 8
GroovyShell

Architecting the Future of Big Data
© Hortonworks Inc. 2013

Page 9
Groovy closures and Hotspot
140
120
100
80
60
40
20
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

•  Groo...
Architecture

Architecting the Future of Big Data
© Hortonworks Inc. 2013

Page 11
Architecting the Future of Big Data
© Hortonworks Inc. 2013

Page 12
Solution: Dynamic Code deployment

•  Embed GroovyShell() into service activator
•  Reference an externally defined Groovy...
Service Activator Example

Architecting the Future of Big Data
© Hortonworks Inc. 2013

Page 14
Solution: Load Management

Architecting the Future of Big Data
© Hortonworks Inc. 2013

Page 15
Solution:
Fast / High Capacity Archival Storage

Real – Time
Analytics

Discovery
Analytics

•  Archival data necessary to...
Demo

Architecting the Future of Big Data
© Hortonworks Inc. 2013

Page 17
Upcoming SlideShare
Loading in...5
×

Utilizing Groovy based closures for flexible real-time analytics

297

Published on

Speaker: Mark Johnson
Real-time analytics is the process of monitoring and often reacting to web based events or machine-to-machine communication and then quickly deliver the appropriate response to an end-user or machine process. The challenge though is how can we modify the analytics algorithms and reference data without shutting down the data feeds.
In this session we will demonstrate architectural and implementation patterns utilizing Spring Integration Framework, HBase, and of course Groovy to illustrate how it is possible to dynamically change your enterprise’s real-time algorithms without a system restart.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
297
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
9
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Utilizing Groovy based closures for flexible real-time analytics"

  1. 1. Utilizing Groovy based closures for flexible real-time analytics Mark Johnson markfjohnson@gmail.com © Hortonworks Inc. 2013 Page 1
  2. 2. Who Am I? •  Director Consulting Hortonworks •  Distributed Compute fan •  Lazy programmer who prefers to reuse rather than re-implement code •  President New England Java Users Group Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 2
  3. 3. Real – Time Analytics Discovery Analytics Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 3
  4. 4. Real-Time Analytics Real-time analytics is the use of, or the capacity to use, all available enterprise data and resources when they are needed. It consists of dynamic analysis and reporting, based on data entered into a system less than one minute before the actual time of use. Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 4
  5. 5. Changing Analytic models Lost Bits too much time to execute Real-Time Analytic Challenges Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 5
  6. 6. Architectural Tradeoffs Performance Requirements: •  Required reaction time •  Resource consumption Architecting the Future of Big Data © Hortonworks Inc. 2013 Changeability Requirements: •  24 x 7 processing •  Models change •  JVM class reload issues Page 6
  7. 7. Groovy Closures A Groovy Closure is a code block which is usable via a variable instance for later execution. Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 7
  8. 8. Groovy Closure: Example Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 8
  9. 9. GroovyShell Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 9
  10. 10. Groovy closures and Hotspot 140 120 100 80 60 40 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 •  Groovy script improves with hotspot execution Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 10
  11. 11. Architecture Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 11
  12. 12. Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 12
  13. 13. Solution: Dynamic Code deployment •  Embed GroovyShell() into service activator •  Reference an externally defined Groovy script from a service activator base method implementation. •  Groovy script may be located most anywhere: – Property file – Database – External text file Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 13
  14. 14. Service Activator Example Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 14
  15. 15. Solution: Load Management Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 15
  16. 16. Solution: Fast / High Capacity Archival Storage Real – Time Analytics Discovery Analytics •  Archival data necessary to back test real-time analytics •  Sensor Data samples can lead to erroneous conclusions Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 16
  17. 17. Demo Architecting the Future of Big Data © Hortonworks Inc. 2013 Page 17
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×