Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mongo db world 2014 billrun

3,424 views

Published on

Published in: Technology, News & Politics
  • Be the first to comment

  • Be the first to like this

Mongo db world 2014 billrun

  1. 1. MongoDB World 2014 by S.D.O.C. Ltd. Billing on Top of MongoDB Ofer Cohen
  2. 2. Who Am I ? ● Open source evangelist ● Former Board member of OpenSourceMatters (Non-profit-org behind Joomla) ● S.D.O.C. Ltd. Co-Founder
  3. 3. What are we doing? ● We increase business success through great open source technologies and services ○ Open & Transparent ○ Don't reinvent the wheel ○ High Quality ○ Lean & Effective
  4. 4. How did we get to billing (history) ● Client: Golan Telecom (Israel)
  5. 5. How did we get to billing (history) ● Client: Golan Telecom ○ New & lean player in the market ○ 0=>~1M subscribers in 4 years ○ Limited resources & love open source
  6. 6. How did we get to billing (history) ● Client: Golan Telecom ○ Start environment from Day 1 ○ Short and aggressive time to market ○ Unlimited plan for ~25$ ○ Customer can do almost everything in the website ■ Obviously requires 24/7 uptime
  7. 7. How did we get to billing (history) ● Start with Anti-Fraud solution ● 2 Different data structure, 2 separated tables ○ Outgoing calls (MOC) ○ incoming calls (MTC) ○ SMS (duration=0 means SMS)
  8. 8. Anti-Fraud in RDBMS How was it look like? Example code... $base_query = "SELECT imsi FROM moc WHERE callEventStartTimeStamp >=" . $start . " UNION SELECT imsi FROM mtc WHERE callEventStartTimeStamp >=" . $start $base_query = "SELECT imsi FROM (" . $base_query . ") AS qry "; if (isset($args['imsi'])) $base_query .= "WHERE imsi = '" . $this->_connection->real_escape_string($args['imsi']) . "'"; $base_query .= "GROUP BY imsi "; $mtc_join_query = "SELECT 'mtc' AS type, imsi, SUM(callEventDuration) AS duration, SUM(CEILING(callEventDuration/60)*60) AS duration_round " . ", SUM(chargeAmount) charge " . ", SUM(IF(SUBSTRING(callingNumber, 1, 3)='972', callEventDuration, IF(CHAR_LENGTH(callingNumber)<=10, callEventDuration, 0))) AS israel_duration " . ", SUM(IF(SUBSTRING(callingNumber, 1, 3)='972', CEILING(callEventDuration/60)*60, IF(CHAR_LENGTH(callingNumber)<=10, CEILING(callEventDuration/60)*60, 0))) AS israel_duration_round " . ", SUM(IF(SUBSTRING(callingNumber, 1, 3)!='972', IF(CHAR_LENGTH(callingNumber)>10, callEventDuration, 0), 0)) AS non_israel_duration " . ", SUM(IF(SUBSTRING(callingNumber, 1, 3)!='972', IF(CHAR_LENGTH(callingNumber)>10, CEILING(callEventDuration/60)*60, 0), 0)) AS non_israel_duration_round " . ", SUM(IF(callEventDuration = 0, 1, 0)) AS sms_count " . "FROM mtc " . "WHERE callEventStartTimeStamp >=" . $start . " " . "GROUP BY type, imsi";
  9. 9. Anti-Fraud in RDBMS How was it look like? Example code... $moc_join_query = "SELECT 'moc' AS type, imsi, SUM(callEventDuration) AS duration, SUM(CEILING(callEventDuration/60)*60) AS duration_round " . ", SUM(chargeAmount) charge " . ", SUM(IF(SUBSTRING(connectedNumber, 1, 3)='972', callEventDuration, IF(CHAR_LENGTH(connectedNumber)<=10, callEventDuration, 0))) AS israel_duration " . ", SUM(IF(SUBSTRING(connectedNumber, 1, 3)='972', CEILING(callEventDuration/60)*60, IF(CHAR_LENGTH(connectedNumber)<=10, CEILING(callEventDuration/60)*60, 0))) AS israel_duration_round " . ", SUM(IF(SUBSTRING(connectedNumber, 1, 3)!='972', IF(CHAR_LENGTH(connectedNumber)>10, callEventDuration, 0), 0)) AS non_israel_duration " . ", SUM(IF(SUBSTRING(connectedNumber, 1, 3)!='972', IF(CHAR_LENGTH(connectedNumber)>10, CEILING(callEventDuration/60)*60, 0), 0)) AS non_israel_duration_round " . ", SUM(IF(callEventDuration = 0, 1, 0)) AS sms_count " . "FROM moc " . "WHERE callEventStartTimeStamp >=" . $start . " " . "GROUP BY type, imsi";
  10. 10. Anti-Fraud in RDBMS How was it look like? Example code... $group_query = "SELECT base.imsi, moc.duration AS moc_duration, moc.charge AS moc_charge, " . "mtc.duration AS mtc_duration, mtc.charge AS mtc_charge, " . "mtc.duration_round AS mtc_duration_round, moc.duration_round AS moc_duration_round, " . "moc.israel_duration AS moc_israel_duration, moc.non_israel_duration AS moc_non_israel_duration, " . "moc.israel_duration_round AS moc_israel_duration_round, moc.non_israel_duration_round AS moc_non_israel_duration_round, " . "mtc.israel_duration AS mtc_israel_duration, mtc.non_israel_duration AS mtc_non_israel_duration, " . "mtc.israel_duration_round AS mtc_israel_duration_round, mtc.non_israel_duration_round AS mtc_non_israel_duration_round, " . "mtc.sms_count AS mtc_sms_count, moc.sms_count AS moc_sms_count " . "FROM " . "( " . $base_query . " ) AS base " . " LEFT JOIN (" . $mtc_join_query . " ) AS mtc ON base.imsi = mtc.imsi " . " LEFT JOIN (" . $moc_join_query . " ) AS moc ON base.imsi = moc.imsi " ; if (isset($args['limit'])) { $limit = (int) $args['limit']; } else { $limit = 100000; }
  11. 11. Anti-Fraud in RDBMS How was it feel…?
  12. 12. After moving to Mongo ● One main collection for 2 types ● Aggregate much more simple
  13. 13. After moving to Mongo $base_match = array( '$match' => array( 'source' => 'nrtrde', 'unified_record_time' => array('$gte' => new MongoDate($charge_time)), ) ); $where = array( '$match' => array( 'record_type' => 'MOC', 'connectedNumber' => array('$regex' => '^972'), 'event_stamp' => array('$exists' => false), 'deposit_stamp' => array('$exists' => false), 'callEventDurationRound' => array('$gt' => 0), // not sms ), ); $group = array( '$group' => array( "_id" => '$imsi', "moc_israel" => array('$sum' => '$callEventDurationRound'), 'lines_stamps' => array('$addToSet' => '$stamp'), ), ); $project = array( '$project' => array( 'imsi' => '$_id', '_id' => 0, 'moc_israel' => 1, 'lines_stamps' => 1, ), );
  14. 14. After moving to Mongo $having = array( '$match' => array( 'moc_israel' => array('$gte' => Billrun_Factory::config()->getConfigValue('nrtrde.thresholds.moc.israel')) ), ); $moc_israel = $lines->aggregate($base_match, $where, $group, $project, $having); //sms out to all numbers $where['$match']['record_type'] = 'MOC'; $where['$match']['callEventDurationRound'] = 0; $group['$group']['sms_out'] = $group['$group']['mtc_all']; unset($group['$group']['mtc_all']); unset($having['$match']['mtc_all']); $group['$group']['sms_out'] = array('$sum' => 1); $having['$match']['sms_out'] = array('$gte' => Billrun_Factory::config()->getConfigValue('nrtrde.thresholds.smsout')); $project['$project']['sms_out'] = 1; unset($project['$project']['mtc_all']); $sms_out = $lines->aggregate($base_match, $where, $group, $project, $having);
  15. 15. What is billing? ● Group of processes of communications service providers ● responsible to collect consumption data ● calculate charging and billing information ● produce bills to customers ● process their payments and manage debt collection Wikipedia
  16. 16. What is billing? ● This is how we see it ● The KISS way
  17. 17. Telecom Today - RDBMS challenges ● Telecom situation world wide today: ○ Unlimited packages, extend 3g usage, with high competition ○ High-volume - 4g, LTE ○ Different Events - different data structure
  18. 18. 5 *main* data-structures from different sources ● NSN - Calls ● SMSC - SMS ● MMSC - MMS ● GGSN - Data ● TAP3 - International usage Billing and MongoDB - Pros
  19. 19. NSN Record > db.lines.findOne({"usaget" : "call", aid:XXXXXXX}) { "_id" : ObjectId("52bafd818f7ac3943a8b96dc"), "stamp" : "87cea5dec484c8f6a19e44e77a2e15b4", "record_type" : "01", "record_number" : "13857000", "record_status" : 0, "exchange_id" : "000006270000", "call_reference" : "700227d9a3", "urt" : ISODate("2013-12-25T15:09:15Z"), "charging_start_time" : "20131225170915", "charging_end_time" : "20131225171517", "call_reference_time" : "20131225170914", "duration" : 362, "calling_number" : "972546918666", "called_number" : "26366667", "call_type" : "3", "chrg_type" : "0", "called_imsi" : "000030000000000", "imsi" : "000089000101076", "in_circuit_group_name" : "", "in_circuit_group" : "", "out_circuit_group_name" : "NBZQZA8", "out_circuit_group" : "0503", "tariff_class" : "000000", "called_number_ton" : "6", "org_dur" : 362, "type" : "nsn", "source" : "binary", "file" : "CF0322.DAT", "log_stamp" : "0a3ffdb7c9ccc175e38be2fcc00f8c28", "process_time" : "2013-12-25 17:35:22", "usaget" : "call", "usagev" : 362, "arate" : DBRef("rates", ObjectId("521e07fcd88db0e73f0001c9")), "aid" : XXXXXXX, "sid" : YYYYY, "plan" : "LARGE", "aprice" : 0, }
  20. 20. SMS Record > db.lines.findOne({"usaget" : "sms", aid:XXXXXXX}) { "_id" : ObjectId("52e52ff1d88db071648b4ad7"), "stamp" : "9328f3aaa114aaba910910053a11b3e8", "record_type" : "1", "calling_number" : "000972546918666", "calling_imsi" : "000089200000000", "calling_msc" : "000972000000000", "billable" : "000000000000000", "called_number" : "000972547655380", "called_imsi" : "425089109386379", "called_msc" : "000972586279101", "message_submition_time" : "140125192517", "time_offest" : "02", "message_delivery_time" : "140125192519", "time_offest1" : "02", "cause_of_terminition" : "100", "call_reference" : "5137864939035049", "message_length" : "050", "concatenated" : "1", "concatenated_from" : "09", "source" : "separator_field_lines", "type" : "smsc", "log_stamp" : "a5950686e364d1400c13dd1857c3340e", "file" : "140125192403_5735golan.cdr", "process_time" : "2014-01-26 17:53:21", "urt" : ISODate("2014-01-25T17:25:17Z"), "usaget" : "sms", "usagev" : 1, "arate" : DBRef("rates", ObjectId("521e07fcd88db0e73f0001db")), "aid" : XXXXXXX, "sid" : YYYYY, "plan" : "LARGE", "aprice" : 0, "usagesb" : 4, "billrun" : "201402" }
  21. 21. Data Record > db.lines.findOne({"usaget" : "data", aid:XXXXXXX}) { "_id" : ObjectId("539076678f7ac34a1d8b9367"), "stamp" : "8a9f891ec85c5294c974a34653356055", "imsi" : "400009209100000", "served_imsi" : "400009209100000", "ggsn_address" : "XX.XX.144.18", "charging_id" : "2814645234", "sgsn_address" : "XX.XX.145.9", "served_pdp_address" : "XX.XX.237.95", "urt" : ISODate("2014-06-05T09:34:47Z"), "record_opening_time" : "20140605123447", "ms_timezone" : "+03:00", "node_id" : "GLTNGPT", "served_msisdn" : "00002546918666", "fbc_uplink_volume" : 61298, "fbc_downlink_volume" : 217304, "rating_group" : 0, "type" : "ggsn", "source" : "binary", "file" : "GLTNGPT_-_0000056580.20140605_-_1251+0300", "log_stamp" : "45a227ced1098bc76a44774eae04eb67", "process_time" : "2014-06-05 16:39:40", "usaget" : "data", "usagev" : 278602, "arate" : DBRef("rates", ObjectId("521e07fcd88db0e73f000200")), "apr" : 0.020264916503503202, "aid" : XXXXXXX, "sid" : YYYYY, "plan" : "LARGE", "aprice" : 0, "usagesb" : 478682116, "billrun" : "201406" }
  22. 22. TAP3 Record (intl roaming) > db.lines.findOne({type:"tap3", aid:9073496}) { "_id" : ObjectId("538d9ac98f7ac3e17d8b4fd6"), "stamp" : "8f6cdc8662307ee2ed951ce640a585b5", "basicCallInformation" : { "GprsChargeableSubscriber" : { "chargeableSubscriber" : { "simChargeableSubscriber" : { "imsi" : "400009209100000" } }, "pdpAddress" : "XX.XX.227.158" }, "GprsDestination" : { "AccessPointNameNI" : "internet.golantelecom.net.il" }, "CallEventStartTimeStamp" : { "localTimeStamp" : "20140529205131", "TimeOffsetCode" : 0 }, "TotalCallEventDuration" : 163 }, "LocationInformation" : { "gprsNetworkLocation" : { "RecEntityCodeList" : { "RecEntityCode" : [ "", "u0000" ] }, "LocationArea" : 00001, "CellId" : 0001 }, "GeographicalLocation" : { "ServingNetwork" : "MMMM" } }, "ImeiOrEsn" : false, "GprsServiceUsed" : { "DataVolumeIncoming" : 195120, "DataVolumeOutgoing" : 48600, "ChargeInformationList" : { "ChargeInformation" : { "ChargedItem" : "X", "ExchangeRateCode" : 0, "ChargeDetailList" : { "ChargeDetail" : { "ChargeType" : "00", "Charge" : 001, "ChargeableUnits" : 100000, "ChargedUnits" : 100000, "ChargeDetailTimeStamp" : { "localTimeStamp" : "20140529205131", "TimeOffset" : 0 } } } } } }, "OperatorSpecInfoList" : { "OperatorSpecInformation" : [ "00000000.0000", "00000000.0000", "00000000.0000" ] }, "record_type" : "e", "urt" : ISODate("2014-05-29T18:51:31Z"), "tzoffset" : "+0200", "imsi" : "400009209100000", "serving_network" : "DEUE2", "sdr" : 0.0001, "exchange_rate" : 1.12078, "type" : "tap3", "file" : "CDBELHBISRGT02253", "log_stamp" : "a0ad109c6e795f6c1feeef9ef649d937", "process_time" : "2014-06-03 12:50:08", "usaget" : "data", "usagev" : 243720, "arate" : DBRef("rates", ObjectId ("521e07fed88db0e73f000219")), "apr" : 0.46640616, "aid" : 9073496, "sid" : 78288, "plan" : "LARGE", "out_plan" : 243720, "aprice" : 0.46640616, "usagesb" : 39139746, "billrun" : "201406" }
  23. 23. Billing and MongoDB - Pros Loose coupling and loose traditional billing components Old w/o MongoDb New with MongoDb
  24. 24. Billing and MongoDB - Pros ● Call can be separated CDRs ● BillRun unify records only on export or presentation layer you are aggregate not in the DB. ● No need for mediation ● Can monitor CDR level and use billing in one system
  25. 25. Billing and MongoDB - Pros Sophisticated rating module ● Rating module depends on CDR structure ● Easy to implement recurring rating
  26. 26. MongoDB advantages with Billing Invoice JSON2PDF process ● Use json as metadata for the PDF ● Easy to export ● Fast processing
  27. 27. MongoDB advantages with Billing Invoice metadata > db.billrun.findOne({billrun_key:"201405", aid:9073496}) { "aid" : NumberLong(9073496), "subs" : [ { "sid" : NumberLong(78288), "subscriber_status" : "open", "current_plan" : DBRef("plans", ObjectId("51bd8dc9eb2f76d2178dd3dd")), "next_plan" : DBRef("plans", ObjectId("51bd8dc9eb2f76d2178dd3dd")), "kosher" : false, "breakdown" : { "in_plan" : { "base" : { "INTERNET_BILL_BY_VOLUME" : { "totals" : { "data" : { "usagev" : NumberLong(1975547725), "cost" : NumberLong(0), "count" : NumberLong(1352) } }, "vat" : 0.18 }, "IL_MOBILE" : { "totals" : { "sms" : { "usagev" : NumberLong(159), "cost" : NumberLong(0), "count" : NumberLong(159) }, "call" : { "usagev" : NumberLong(20788), "cost" : NumberLong(0), "count" : NumberLong(83) } }, "vat" : 0.18 }, "IL_FIX" : { "totals" : { "call" : { "usagev" : NumberLong(1217), "cost" : NumberLong(0), "count" : NumberLong(16) } }, "vat" : 0.18 }, "INTERNAL_VOICE_MAIL_CALL" : { "totals" : { "call" : { "usagev" : NumberLong(60), "cost" : NumberLong(0), "count" : NumberLong(2) } }, "vat" : 0.18 }, "service" : { "cost" : 83.898305085, "vat" : 0.18 } }, "intl" : { "KT_USA_NEW" : { "totals" : { "call" : { "usagev" : NumberLong(149), "cost" : NumberLong(0), "count" : NumberLong(2) } }, "vat" : 0.18 } } },
  28. 28. MongoDB advantages with Billing Invoice metadata "credit" : { "refund_vatable" : { "CRM-REFUND_PROMOTION_1024-BILLRUN_201405" : -33.898305084746 } } }, "lines" : { "data" : { "counters" : { "20140425" : { "usagev" : NumberLong(11991615), "aprice" : NumberLong(0), "plan_flag" : "in" }, …. /* data by date really long, so let’s cut it from this demonstration */ } } }, "totals" : { "vatable" : 50.000000000254005, "before_vat" : 50.000000000254005, "after_vat" : 59.00000000029973 }, "costs" : { "credit" : { "refund" : { "vatable" : -33.898305084746 } }, "flat" : { "vatable" : 83.898305085 } } }, { "sid" : NumberLong(354961), "subscriber_status" : "open", "current_plan" : DBRef("plans", ObjectId("51bd8dc9eb2f76d2178dd3de")), "next_plan" : DBRef("plans", ObjectId("51bd8dc9eb2f76d2178dd3de")), "kosher" : false, "breakdown" : { "in_plan" : { "base" : { "service" : { "cost" : 8.466101695, "vat" : 0.18 } } } }, "totals" : { "vatable" : 8.466101695, "before_vat" : 8.466101695, "after_vat" : 9.9900000001 }, "costs" : { "flat" : { "vatable" : 8.466101695 } } } ], "vat" : 0.18, "billrun_key" : "201405", "totals" : { "before_vat" : 58.466101695254004, "after_vat" : 68.99000000039973, "vatable" : 58.466101695254004 }, "_id" : ObjectId("5382fd3cd88db0c31a8b74cc"), "invoice_id" : NumberLong(14738102), "invoice_file" : "201405_009073496_00014738102.xml" }
  29. 29. Infrastructure BETA First Production Today Data center 1 Data center 2 Data center 1 Data center 2 Data center 1 Data center 2 BillRun application BillRun core PHP Yaf framework Mongodloid, Zend and more libraries MongoDB App stack Web service Cli/Cron services
  30. 30. Pay Attention! What to keep in mind Transactional (tx) processes ● write concern - acknowledged (default in 2.4) ● findAndModify (A.K.A FAM) - document tx ● value=oldValue ● 2 phase commit - app side ● Transaction lock by design
  31. 31. High performance tricks ● With SSD you get x20 more performance ● MongoDB loves RAM ● Follow MongoDB production notes ○ Readahead low as possible ○ THP - transparent hugepages Pay Attention! What to keep in mind
  32. 32. TCO - MongoDB based solution ● 3 Months dev ● 3 Months QA ● Few days of maintenance infra and app ● Easy to scale ● Easy to add features
  33. 33. Thank you, Ofer Cohen S. D.O.C. Ltd. ofer@billrun.net @oc666 BillRun Billing on Top of

×