Cdr stats-vo ip-analytics_solution_mongodb_meetup
Upcoming SlideShare
Loading in...5
×
 

Cdr stats-vo ip-analytics_solution_mongodb_meetup

on

  • 869 views

Call Data Analysis

Call Data Analysis
for Asterisk & FreeSWITCH with MongoDB
Arezqui Belaid @areskib <info>

Statistics

Views

Total Views
869
Views on SlideShare
869
Embed Views
0

Actions

Likes
1
Downloads
26
Comments
1

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Cdr stats-vo ip-analytics_solution_mongodb_meetup Cdr stats-vo ip-analytics_solution_mongodb_meetup Presentation Transcript

    • Call Data Analysisfor Asterisk & FreeSWITCH with MongoDB Arezqui Belaid @areskib <info@star2billing.com>
    • Problems to solve - Millions of Call records - Multiple sources - Multiple data formats - Replication - Fast Analytics - Multi-Tenant - Realtime - Fraud detection
    • Why MongoDB- NoSQL - Schema-Less- Capacity / Sharding- Upserts- Replication : Increase read capacity- Async writes : Millions of entries / acceptable losses- Compared to CouchDB - native drivers
    • What does it look like? Dashboard
    • Hourly / Daily / Monthly reporting
    • Compare call traffic
    • World Map
    • Realtime
    • Under the hood- FreeSWITCH (freeswitch.org)- Asterisk (asterisk.org)- Django (djangoproject.com)- Celery (celeryproject.org)- RabbitMQ (rabbitmq.com)- Socket.IO (socket.io)- MongoDB (mongo.org)- PyMongo (api.mongo.org)- and more...
    • Our Data - Call Detail Record (CDR)1) Call info :2) BSON : CDR = { hangup_cause_q850:20, ... hangup_cause:NORMAL_CLEARING, callflow:{ sip_received_ip:192.168.1.21, caller_profile:{ sip_from_host:127.0.0.1, tts_voice:kal,7, username:1000, accountcode:1000, destination_number:5578193435, sip_user_agent:Blink 0.2.8 (Linux), ani:71737224, answerusec:0, caller_id_name:71737224, caller_id:71737224, ... call_uuid:adee0934-a51b-11e1-a18c- }, 00231470a30c, ... answer_stamp:2012-05-23 15:45:09.856463, }, outbound_caller_id_name:FreeSWITCH, variables:{ billsec:66, mduration:12960, progress_uepoch:0, effective_caller_id_name:Extension 1000, answermsec:0, sip_via_rport:60536, outbound_caller_id_number:0000000000, uduration:12959984, duration:3, sip_local_sdp_str:v=0no=FreeSWITCH end_stamp:2012-05-23 15:45:12.856527, 1327491731n answer_uepoch:1327521953952257, }, billmsec:12960, ... ...3) Insert Mongo : db.cdr.insert(CDR);
    • Pre-Aggregate
    • Pre-Aggregate - Daily CollectionProduce data easier to manipulate : current_y_m_d = datetime.strptime(str(start_uepoch)[:10], "%Y-%m-%d") CDR_DAILY.update({ date_y_m_d: current_y_m_d, destination_number: destination_number, hangup_cause_id: hangup_cause_id, accountcode: accountcode, switch_id: switch.id, },{ $inc: {calls: 1, duration: int(cdr[variables][duration]) } }, upsert=True)Output db.CDR_DAILY.find() :{ "_id" : ..., "date_y_m_d" : ISODate("2012-04-30T00:00:00Z"), "accountcode" : "1000", "calls" : 1, "destination_number": "0045277522", "duration" : 23, "hangup_cause_id" :9, "switch_id" :1 }... - Faster to query pre-aggregate data - Upsert is your friend / update if exists - insert if not
    • Map-Reduce - Emit Step- MapReduce is a batch processing of data- Applying to previous pre-aggregate collection (Faster / Less data) map = mark_safe(u function(){ emit( { a_Year: this.date_y_m_d.getFullYear(), b_Month: this.date_y_m_d.getMonth() + 1, c_Day: this.date_y_m_d.getDate(), f_Switch: this.switch_id }, {calldate__count: 1, duration__sum: this.duration} ) })
    • Map-Reduce - Reduce StepReduce Step is trivial, it simply sums up and counts : reduce = mark_safe(u function(key,vals) { var ret = { calldate__count : 0, duration__sum: 0, duration__avg: 0 }; for (var i=0; i < vals.length; i++){ ret.calldate__count += parseInt(vals[i].calldate__count); ret.duration__sum += parseInt(vals[i].duration__sum); } return ret; } )
    • Map-ReduceQuery : out = aggregate_cdr_daily calls_in_day = daily_data.map_reduce(map, reduce, out, query=query_var)Output db.aggregate_cdr_daily.find() :{ "_id" : { "a_Year" : 2012, "b_Month" : 5, "c_Day" : 13, "f_Switch" :1 }, "value" : { "calldate__count" : 91,"duration__sum" : 5559, "duration__avg" : 0 } }{ "_id" : { "a_Year" : 2012, "b_Month" : 5, "c_Day" : 14, "f_Switch" :1 }, "value" : { "calldate__count" : 284,"duration__sum" : 13318, "duration__avg" : 0 } }...
    • Roadmap- Quality monitoring- Audio recording- Add support for other telecoms switches- Improve - refactor (Beta)- Testing- Listen and Learn
    • WAT else...?- Website : http://www.cdr-stats.org- Code : github.com/star2billing/cdr-stats- FOSS / Licensed MPLv2- Get started : Install script Try it, its easy!!!
    • Questions ? Twitter : @areskibEmail : areski@gmail.comSlides : http://goo.gl/TZLF9