Speaker: Jia-long Wu from Lotame
Big Data Applications Meetup, 08/19/2015
Palo Alto, CA
More info here: http://www.meetup.com/BigDataApps/
Link to talk: https://www.youtube.com/watch?v=GRtQFceQG6k
About the talk:
In this talk, we’ll present the design of our new data stream processing application at Lotame and describe how we achieve significant reduction in cluster resource utilization while allowing faster updates of client audience data and better ad-hoc query support with the new platform.
We will examine the challenges faced in counting uniques in a high volume stream processing environment, and present a novel approach using time windowed HyperLogLog aggregates. We’ll also discuss how CDAP enable us to roll out this new platform quickly and share some valuable lessons and best practices we learned during the development cycle.