Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big Data CDR Analyzer - Kanthaka


Published on

This is the presentation at the successful completion of 'Kanthaka'- Big Data CDR (Caller Detail Record) Analyzer, a system to support near real time complex promotion at telecom operators. This includes the details of technology selection, system architecture and final test results on a dual core machine with 3GB RAM and a cluster with two such nodes.

Published in: Technology
  • Be the first to comment

Big Data CDR Analyzer - Kanthaka

  1. 1. © 2012 University of Moratuwa Big Data CDR Analyzer “The Next Generation Mobile Promotions”Project Supervisors- • 080201N – M.K.P.R. JayawardhanaMr. Thilina Anjitha – hSenid • 080254D – P.K.A.M. KumaraDr.Shahani Markus Weerawarana • 080331L – W.D.A.I. Paranawithana • 080357V – T.D.K. Perera
  2. 2. © 2012 University of Moratuwa OVERVIEW Background Current Situation Scope and Assumptions Kanthaka – big data CDR Analyzer System Technology Comparison - Map Reduce - NoSQL Databases Architecture Risks and Possible Remedies References
  3. 3. © 2012 University of MoratuwaBackgroundMobile Promotions
  4. 4. © 2012 University of Moratuwa CURRENT SITUATION• Promotions based only on their network usage• Use only active call switch for triggering promotions• No way of analyzing and processing high volume CDR records• No efficient CDR analyzing method• No access to historical data• Complex rules not supported &@$*#
  5. 5. © 2012 University of Moratuwa TO RESCUE Selecting eligible users for both commercial organizations based and network usage based promotions. Eg- giving 20% discount for pizza lovers within age group 16-40 who have called pizza hut more than 5 times a month High volume CDR analysis. Near real time selection of eligible users for promotions.
  6. 6. © 2012 University of Moratuwa CDR Analyzer system which  can process 30 million records per day  can produce results within 30 seconds  provides a GUI to define dynamic rules  can be used to offer real-time sales promotions for mobile subscribers
  7. 7. © 2012 University of MoratuwaThis location information retrieving from Location Based System(LBS) canbe replaced with any other information retrieving such as subscriber agefrom the Customer Relationship Management system to support attractivepromotions.
  8. 8. © 2012 University of MoratuwaSCOPE AND ASSUMPTIONSSCOPE  30 M  30 M  Multiple Rules  Multiple Rules  Offer Promotion  Select eligibilities for promotion only Real system operation Operation expect by Kanthaka
  9. 9. © 2012 University of MoratuwaASSUMPTIONS CDR records can be only in .CSV format. Event type can be in different types like SMS, Voice call, MMS, USSD, Top-up, GPRS, LBS. CDR can be received as batches to the system asynchronously. Only 6 attributes out of many attributes will be considered during processing.
  10. 10. © 2012 University of MoratuwaTECHNOLOGY COMPARISON
  11. 11. © 2012 University of Moratuwa
  12. 12. © 2012 University of MoratuwaYCSB BENCHMARKS With more big users, active mailing lists, most promising technologies (secondary index, counters) best to try out is Cassandra.
  13. 13. © 2012 University of Moratuwa
  14. 14. © 2012 University of MoratuwaTECHNOLOGY SELECTIONTECHNOLOGIES LEFT BEHIND TECHNOLOGIES SELECTED Complex Event  NoSQL DB - Cassandra Processing engines(CEP)  No persistency Rules Engine  More layers  More latency Hadoop - latency NoSQL DB- Hbase, MongoDB, Hive
  15. 15. © 2012 University of MoratuwaBRIEF ARCHITECTURE OF ‘KANTHAKA’Promotion definition Cassandra Cluster Pre-processing unit
  16. 16. © 2012 University of MoratuwaTEST RESULTS IN SINGLE NODE
  17. 17. © 2012 University of MoratuwaTEST RESULTS IN TWO NODE- CLUSTER
  18. 18. © 2012 University of MoratuwaCLUSTER BETTER IN HIGH LOADS
  19. 19. © 2012 University of MoratuwaRISKS AND POSSIBLE REMEDIES NoSQL databases High performance More memory Use an external cluster with descent memory Concurrency Issues Handling Low speed  Locking database Use shadow copy Handling sudden peaks Should have an auto balancing mechanism ready
  20. 20. © 2012 University of MoratuwaFINAL DELIVERABLES Big Data CDR Analyzer system Research Paper Final Report
  21. 21. © 2012 University of MoratuwaREFERENCES B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears, “Benchmarking cloud serving systems with YCSB,” 2010, pp. 143–154.Visit us at Kanthaka
  22. 22. © 2012 University of Moratuwa Thank you ManojDhanika Amila Pushpalanka