Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Reactive Data System in Practice


Published on

how to solve Fast Data Problem and reacting to the World faster

Published in: Technology
  • Be the first to comment

Reactive Data System in Practice

  1. 1. Reactive Data System in Practice Or how to solve Fast Data Problem and reacting to the World faster By Triều Nguyễn at (Reactive Data System Lab) λ(x)
  2. 2. 2008: Java Developer, develop Social Trading Network for a startup (Yopco) 2011: joined FPT Online, software engineer, worked in FPT ID project and VnExpress Mobile (Backend) 2012: backend engineer at Greengar (Scaling server and real- time services) 12/2012 to 05/2015: back to FPT Online, lead engineer at eClick Ad Platform 06/2015 to now: lead engineer at iTVad, a startup project at FPT for OTT and TV Advertising Solutions about me
  3. 3. 1. Just some concepts 2. Ideas in practice 3. What is RFX framework ? 4. Why is RFX ? 5. How to solve problems with RFX ? a. Counting pageview of website b. Counting unique user of website c. Querying data more reactive and adaptive 6. Final ideas Contents of this talk
  4. 4. 1 - Just some concepts
  5. 5. Fast Data vs Big Data
  6. 6. What is Reactive Data System ? is a software design approach that uses the principles of event driven architecture for the design and implementation of data intensive applications. Event Driven Architecture EDA is a framework that promotes production, detection, consumption and reaction to the events. Event based applications will run on RDS framework.
  7. 7. Why is Reactive Data System We want to ask bigger questions faster
  8. 8. 2 - Ideas in practice
  9. 9. Problem (User Story in Practice) ● User does login with FB Connect at Ecommerce website ● User does a click on an item, then system should do the following tasks: 1. Find best items to recommend 2. Find a best item to send email marketing 3. If there are more than 100 users, that click on a item, run a marketing campaign on Facebook automatically.
  10. 10. An example from E-Commerce
  11. 11. 3 - What is RFX framework ?
  12. 12. ● A framework for big fast data problems ● A collection of Open Source Tools from Netty to Spark ● The mission of RFX 1. Build data product quickly with design patterns 2. React to critical events in real-time What is RFX or Reactive Function X ?
  13. 13. RFX is inspired by the Actor Model Theory
  14. 14. Paper:
  15. 15. RFX is designed for startup business I design RFX because I have to solve following problems both at FPT and at ● Marketing Automation ● Product UX personalization ● Real-time Advertising ● System monitoring (frontend and backend) ● Email marketing ● Product Analytics ● Customer Analytics → So I build a full-stack framework (a collection of tools for both backend and frontend)
  16. 16. Philosophy of RFX
  17. 17. Sub modules in RFX framework
  18. 18. 4 - Why is RFX framework ?
  19. 19. Why Rfx ? ● Ideas since 2012 (from Actor model theory) ● R&D and Deployed in Production since 2013 ● Open Source: Apache License, Version 2.0 ● Full Stack: from Frontend to Backend ● Agile for Data Analytics ● Lightweight Lambda Architecture ● Really fast and near-real-time processing ● Tested with 1.000.000 events / second ● Simple development model for data engineer
  20. 20. Roadmap to the future
  21. 21. 5 - How to solve problems with RFX ?
  22. 22. Problems User story in Digital Marketing: 1. Counting pageview of website 2. Counting unique user of website 3. Real-time marketing
  23. 23. Find rush hour in your system
  24. 24. Which device user reach our content
  25. 25. Solution Architecture Tracking (RFX-track) Event Queue (Apache Kafka) Event Processor (RFX-stream) RedisMongoDB Synch Data Jobs (RFX-job) Report (RFX-report) Event Actor (RFX-actor) Ad Server
  26. 26. An example topology
  27. 27. 6 - Final ideas
  28. 28. Summary ● Fast Data is cool ● Fast data: The next step after big data ● RFX is designed to solve common Fast Data problems in Digital Marketing and Advertising
  29. 29.
  30. 30. The end and thank you λ