Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Finding Hidden Issues with Machine Learning (callstats.io)

26 views

Published on

WebRTC produces mountains of data that can be used to optimize streams and debug problems—if you know where to look. Chad discusses how callstats.io uses unsupervised learning to discover non-obvious issues inside the vast amount of call quality data the company collects. ML methods reviewed include feature reduction with Principal Component Analysis (PCA), Clustering with Gaussian Mixture Model (GMM), and optimizing cluster sizes with Bayesian Inference Criterion (BIC).

Presented at AllThingsRTC 2019. See the video here: https://www.youtube.com/watch?v=WCC0Vy9aRNE

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Finding Hidden Issues with Machine Learning (callstats.io)

  1. 1. Finding Hidden Call Quality Issues with Machine Learning chad hart 13 June 2019 1 Deliver Better User Experiences
  2. 2. Callstats.io: The WebRTC Monitoring Leader • Founded in 2014 by IETF and W3C authors • Customers across vertical industries • Over 1B collected WebRTC datapoints each month • Integrated into Amazon Connect 2 NXTEC CORP
  3. 3. callstats.io Voice Quality Solutions 3 WebRTC Monitoring • Network status • Service metrics AI-Driven Troubleshooting • Anomaly detection • Root cause analysis • Notifications Active Network Testing • Connectivity verification • Performance metrics
  4. 4. Data Collection and Monitoring Architecture 4
  5. 5. Why do we need ML? 5
  6. 6. Why do we need ML? Dealing with Big Data 6 Photo credit: Flickr User Gavin Bell (CC BY-NC-ND 2.0) https://www.flickr.com/photos/gavinbell/535261899/
  7. 7. Challenge: getStats provides a lot of data Example: • Go to https://webrtc.github.io/samples/src/content/peerconnection/pc1/ • Start the sample • Type: 7 pc1.getStats().then(stats=>stats.forEach(stat=>console.log(stat)))
  8. 8. • [{"id":"RTCCertificate_70:47:DC:96:5C:24:A1:C1:F0:A2:89:39:FC:B1:D9:69:66:EB:4A:F6:12:F0:68:43:1B:F9:D4:ED:D6:E2:D1:FE","timestamp":1560111536209.789,"type":"certificate","fingerprint":"70:47:DC:96:5C:24:A1:C1:F0:A2:89:39:FC:B1:D9:69:66:EB:4A:F6:12:F0:68:43:1B:F9:D4:ED:D6:E2:D1:FE","fingerprintAlgorithm":"sha- 256","base64Certificate":"MIIBFjCBvaADAgECAgkAhuXyX8OJm0EwCgYIKoZIzj0EAwIwETEPMA0GA1UEAwwGV2ViUlRDMB4XDTE5MDYwODIwMTU0MFoXDTE5MDcwOTIwMTU0MFowETEPMA0GA1UEAwwGV2ViUlRDMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEQ6ZyQYSC7xZHSUhq6mxCWa+/VwRdO0U1GuMAUYjYSkAmgzfJ0mW TZbZWJ12V7LlimkOZtdXCv2UPSh3sDOHu/DAKBggqhkjOPQQDAgNIADBFAiBDBGAgqa2dxlewHn5661OgMK6xfanL13l9aLoyuvdCgAIhAOmal8xD0S0ZLRKO/WDOb3 RRLAqxCcn0Jmur/OPdoEDl"},{"id":"RTCCertificate_D1:3F:71:1A:FE:32:78:3D:AA:93:06:F3:E6:AF:22:E9:5A:BC:B7:D2:AD:D6:A4:49:FD:9F:FB:00:49:89:4C:5A","timestamp":1560111536209.789,"type":"certificate","fingerprint":"D1:3F:71:1A:FE:32:78:3D:AA:93:06:F3:E6:AF:22:E9:5A:BC:B7:D2:AD:D6:A4:49:FD:9F:FB:00:49:89:4C:5A","fingerprintAlgorithm":"sha- 256","base64Certificate":"MIIBFzCBvaADAgECAgkA7svx50DzS6swCgYIKoZIzj0EAwIwETEPMA0GA1UEAwwGV2ViUlRDMB4XDTE5MDYwODIwMTU0MFoXDTE5MDcwOTIwMTU0MFowETEPMA0GA1UEAwwGV2ViUlRDMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEg3VCpwhtcNE3rEaBH8GTrWpQTcbnPGWEz0i2XAYE/A8pATEPoXb7mPojyEy8LxkmPSjkwrEodppNzA7X8Yjv2TAKBggqhkjOPQQDAgNJADBGAiEA3MWsKxnwNWssNhZ+mSGhZca63A8tvFIdRyf+CBOzNYwCIQD1JTiEdHntgBQ9/ZcaRVbQK MBU1X6+xYmtnwjJ+x2umw=="},{"id":"RTCCodec_0_Inbound_0","timestamp":1560111536209.789,"type":"codec","payloadType":0,"mimeType":"audio/PCMU","clockRate":8000},{"id":"RTCCodec_0_Inbound_103","timestamp":1560111536209.789,"type":"codec","payloadType":103,"mimeType":"audio/ISAC","clockRate":16000},{"id":"RTCCodec_0_Inbound_104","timestamp":1560111536209.789,"type":"codec","payloadType":104,"mimeType":"audio/ISAC","clockRate":32000},{"id":"RTCCodec_0_Inbound_105","t imestamp":1560111536209.789,"type":"codec","payloadType":105,"mimeType":"audio/CN","clockRate":16000},{"id":"RTCCodec_0_Inbound_106","timestamp":1560111536209.789,"type":"codec","payloadType":106,"mimeType":"audio/CN","clockRate":32000},{"id":"RTCCodec_0_Inbound_110","timestamp":1560111536209.789,"type":"codec","payloadType":110,"mimeType":"audio/telephone- event","clockRate":48000},{"id":"RTCCodec_0_Inbound_111","timestamp":1560111536209.789,"type":"codec","payloadType":111,"mimeType":"audio/opus","clockRate":48000},{"id":"RTCCodec_0_Inbound_112","timestamp":1560111536209.789,"type":"codec","payloadType":112,"mimeType":"audio/telephone-event","clockRate":32000},{"id":"RTCCodec_0_Inbound_113","timestamp":1560111536209.789,"type":"codec","payloadType":113,"mimeType":"audio/telephone- event","clockRate":16000},{"id":"RTCCodec_0_Inbound_126","timestamp":1560111536209.789,"type":"codec","payloadType":126,"mimeType":"audio/telephone- event","clockRate":8000},{"id":"RTCCodec_0_Inbound_13","timestamp":1560111536209.789,"type":"codec","payloadType":13,"mimeType":"audio/CN","clockRate":8000},{"id":"RTCCodec_0_Inbound_8","timestamp":1560111536209.789,"type":"codec","payloadType":8,"mimeType":"audio/PCMA","clockRate":8000},{"id":"RTCCodec_0_Inbound_9","timestamp":1560111536209.789,"type":"codec","payloadType":9,"mimeType":"audio/G722","clockRate":8000},{"id":"RTCCodec_0_Outbound_0","timestamp":1560111 536209.789,"type":"codec","payloadType":0,"mimeType":"audio/PCMU","clockRate":8000},{"id":"RTCCodec_0_Outbound_103","timestamp":1560111536209.789,"type":"codec","payloadType":103,"mimeType":"audio/ISAC","clockRate":16000},{"id":"RTCCodec_0_Outbound_104","timestamp":1560111536209.789,"type":"codec","payloadType":104,"mimeType":"audio/ISAC","clockRate":32000},{"id":"RTCCodec_0_Outbound_105","timestamp":1560111536209.789,"type":"codec","payloadType":105,"mimeType":"a udio/CN","clockRate":16000},{"id":"RTCCodec_0_Outbound_106","timestamp":1560111536209.789,"type":"codec","payloadType":106,"mimeType":"audio/CN","clockRate":32000},{"id":"RTCCodec_0_Outbound_110","timestamp":1560111536209.789,"type":"codec","payloadType":110,"mimeType":"audio/telephone- event","clockRate":48000},{"id":"RTCCodec_0_Outbound_111","timestamp":1560111536209.789,"type":"codec","payloadType":111,"mimeType":"audio/opus","clockRate":48000},{"id":"RTCCodec_0_Outbound_112","timestamp":1560111536209.789,"type":"codec","payloadType":112,"mimeType":"audio/telephone-event","clockRate":32000},{"id":"RTCCodec_0_Outbound_113","timestamp":1560111536209.789,"type":"codec","payloadType":113,"mimeType":"audio/telephone- event","clockRate":16000},{"id":"RTCCodec_0_Outbound_126","timestamp":1560111536209.789,"type":"codec","payloadType":126,"mimeType":"audio/telephone- event","clockRate":8000},{"id":"RTCCodec_0_Outbound_13","timestamp":1560111536209.789,"type":"codec","payloadType":13,"mimeType":"audio/CN","clockRate":8000},{"id":"RTCCodec_0_Outbound_8","timestamp":1560111536209.789,"type":"codec","payloadType":8,"mimeType":"audio/PCMA","clockRate":8000},{"id":"RTCCodec_0_Outbound_9","timestamp":1560111536209.789,"type":"codec","payloadType":9,"mimeType":"audio/G722","clockRate":8000},{"id":"RTCCodec_1_Inbound_100","timestamp":15 60111536209.789,"type":"codec","payloadType":100,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Inbound_101","timestamp":1560111536209.789,"type":"codec","payloadType":101,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_102","timestamp":1560111536209.789,"type":"codec","payloadType":102,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_107","timestamp":1560111536209.789,"type":"codec","payloadType":107,"mimeType":"vi deo/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_108","timestamp":1560111536209.789,"type":"codec","payloadType":108,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_109","timestamp":1560111536209.789,"type":"codec","payloadType":109,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_114","timestamp":1560111536209.789,"type":"codec","payloadType":114,"mimeType":"video/red","clockRate":90000},{"id":"RTCCodec_1_Inbound_115","times tamp":1560111536209.789,"type":"codec","payloadType":115,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_116","timestamp":1560111536209.789,"type":"codec","payloadType":116,"mimeType":"video/ulpfec","clockRate":90000},{"id":"RTCCodec_1_Inbound_119","timestamp":1560111536209.789,"type":"codec","payloadType":119,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_120","timestamp":1560111536209.789,"type":"codec","payloadType":120,"mimeT ype":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_121","timestamp":1560111536209.789,"type":"codec","payloadType":121,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_122","timestamp":1560111536209.789,"type":"codec","payloadType":122,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_123","timestamp":1560111536209.789,"type":"codec","payloadType":123,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_124", "timestamp":1560111536209.789,"type":"codec","payloadType":124,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_125","timestamp":1560111536209.789,"type":"codec","payloadType":125,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_127","timestamp":1560111536209.789,"type":"codec","payloadType":127,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_96","timestamp":1560111536209.789,"type":"codec","payloadType":9 6,"mimeType":"video/VP8","clockRate":90000},{"id":"RTCCodec_1_Inbound_97","timestamp":1560111536209.789,"type":"codec","payloadType":97,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_98","timestamp":1560111536209.789,"type":"codec","payloadType":98,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Inbound_99","timestamp":1560111536209.789,"type":"codec","payloadType":99,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_ 100","timestamp":1560111536209.789,"type":"codec","payloadType":100,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Outbound_101","timestamp":1560111536209.789,"type":"codec","payloadType":101,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_102","timestamp":1560111536209.789,"type":"codec","payloadType":102,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_107","timestamp":1560111536209.789,"type":"codec","payload Type":107,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_108","timestamp":1560111536209.789,"type":"codec","payloadType":108,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_109","timestamp":1560111536209.789,"type":"codec","payloadType":109,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_114","timestamp":1560111536209.789,"type":"codec","payloadType":114,"mimeType":"video/red","clockRate":90000},{"id":"RTC Codec_1_Outbound_115","timestamp":1560111536209.789,"type":"codec","payloadType":115,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_116","timestamp":1560111536209.789,"type":"codec","payloadType":116,"mimeType":"video/ulpfec","clockRate":90000},{"id":"RTCCodec_1_Outbound_119","timestamp":1560111536209.789,"type":"codec","payloadType":119,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_120","timestamp":1560111536209.789,"typ e":"codec","payloadType":120,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_121","timestamp":1560111536209.789,"type":"codec","payloadType":121,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_122","timestamp":1560111536209.789,"type":"codec","payloadType":122,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_123","timestamp":1560111536209.789,"type":"codec","payloadType":123,"mimeType":"video/H264","clockRate" :90000},{"id":"RTCCodec_1_Outbound_124","timestamp":1560111536209.789,"type":"codec","payloadType":124,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_125","timestamp":1560111536209.789,"type":"codec","payloadType":125,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_127","timestamp":1560111536209.789,"type":"codec","payloadType":127,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_96","timestamp":15 60111536209.789,"type":"codec","payloadType":96,"mimeType":"video/VP8","clockRate":90000},{"id":"RTCCodec_1_Outbound_97","timestamp":1560111536209.789,"type":"codec","payloadType":97,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_98","timestamp":1560111536209.789,"type":"codec","payloadType":98,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Outbound_99","timestamp":1560111536209.789,"type":"codec","payloadType":99,"mimeType":"video/r tx","clockRate":90000},{"id":"RTCIceCandidatePair_eDlSn4+Z_o4HO2DpI","timestamp":1560111536209.789,"type":"candidate- pair","transportId":"RTCTransport_0_1","localCandidateId":"RTCIceCandidate_eDlSn4+Z","remoteCandidateId":"RTCIceCandidate_o4HO2DpI","state":"succeeded","priority":9115038255631187000,"nominated":true,"writable":true,"bytesSent":43425072,"bytesReceived":196122,"totalRoundTripTime":0.031,"currentRoundTripTime":0.001,"availableOutgoingBitrate":3539445,"requestsReceived":82,"requestsSent":1,"responsesReceived":82,"responsesSent":82,"consentRequestsSent":81},{"id":"RTCIceCandidate_ eDlSn4+Z","timestamp":1560111536209.789,"type":"local-candidate","transportId":"RTCTransport_0_1","isRemote":false,"networkType":"wifi","ip":"10.10.100.49","port":63479,"protocol":"udp","candidateType":"host","priority":2122260223,"deleted":false},{"id":"RTCIceCandidate_o4HO2DpI","timestamp":1560111536209.789,"type":"remote- candidate","transportId":"RTCTransport_0_1","isRemote":true,"ip":"10.10.100.49","port":52235,"protocol":"udp","candidateType":"host","priority":2122260223,"deleted":false},{"id":"RTCMediaStreamTrack_sender_3","timestamp":1560111536209.789,"type":"track","trackIdentifier":"93c3b5d1-5ad2-4d6f-b4ef- 9026d140cd12","remoteSource":false,"ended":false,"detached":false,"kind":"audio","audioLevel":0,"totalAudioEnergy":0,"totalSamplesDuration":0},{"id":"RTCMediaStreamTrack_sender_4","timestamp":1560111536209.789,"type":"track","trackIdentifier":"a6b75d65-225e-4a17-8566- 5ebfde288572","remoteSource":false,"ended":false,"detached":false,"kind":"video","frameWidth":640,"frameHeight":480,"framesSent":5883,"hugeFramesSent":3},{"id":"RTCMediaStream_OIL9zTwVhYSpGstFDdnOlBGaKGbLz16BJnlb","timestamp":1560111536209.789,"type":"stream","streamIdentifier":"OIL9zTwVhYSpGstFDdnOlBGaKGbLz16BJnlb","trackIds":["RTCMediaStreamTrack_sender_3","RTCMediaStreamTrack_sender_4"]},{"id":"RTCOutboundRTPAudioStream_3291843167","timestamp":156011153 6209.789,"type":"outbound-rtp","ssrc":3291843167,"isRemote":false,"mediaType":"audio","kind":"audio","trackId":"RTCMediaStreamTrack_sender_3","transportId":"RTCTransport_0_1","codecId":"RTCCodec_0_Outbound_111","packetsSent":9748,"retransmittedPacketsSent":0,"bytesSent":856558,"retransmittedBytesSent":0},{"id":"RTCOutboundRTPVideoStream_3763384231","timestamp":1560111536209.789,"type":"outbound- rtp","ssrc":3763384231,"isRemote":false,"mediaType":"video","kind":"video","trackId":"RTCMediaStreamTrack_sender_4","transportId":"RTCTransport_0_1","codecId":"RTCCodec_1_Outbound_96","firCount":0,"pliCount":0,"nackCount":0,"qpSum":31335,"packetsSent":39380,"retransmittedPacketsSent":0,"bytesSent":42009407,"retransmittedBytesSent":0,"framesEncoded":5883,"totalEncodeTime":24.969},{"id":"RTCPeerConnection","timestamp":1560111536209.789,"type":"peer- connection","dataChannelsOpened":0,"dataChannelsClosed":0},{"id":"RTCTransport_0_1","timestamp":1560111536209.789,"type":"transport","bytesSent":43425072,"bytesReceived":196122,"dtlsState":"connected","selectedCandidatePairId":"RTCIceCandidatePair_eDlSn4+Z_o4HO2DpI","localCertificateId":"RTCCertificate_70:47:DC:96:5C:24:A1:C1:F0:A2:89:39:FC:B1:D9:69:66:EB:4A:F6:12:F0:68:43:1B:F9:D4:ED:D6:E2:D1:FE","remoteCertificateId":"RTCCertificate_D1:3F:71:1A:FE:32:78:3D:AA:93:06:F3:E6: AF:22:E9:5A:BC:B7:D2:AD:D6:A4:49:FD:9F:FB:00:49:89:4C:5A"},{"id":"RTCCertificate_70:47:DC:96:5C:24:A1:C1:F0:A2:89:39:FC:B1:D9:69:66:EB:4A:F6:12:F0:68:43:1B:F9:D4:ED:D6:E2:D1:FE","timestamp":1560111849871.613,"type":"certificate","fingerprint":"70:47:DC:96:5C:24:A1:C1:F0:A2:89:39:FC:B1:D9:69:66:EB:4A:F6:12:F0:68:43:1B:F9:D4:ED:D6:E2:D1:FE","fingerprintAlgorithm":"sha- 256","base64Certificate":"MIIBFjCBvaADAgECAgkAhuXyX8OJm0EwCgYIKoZIzj0EAwIwETEPMA0GA1UEAwwGV2ViUlRDMB4XDTE5MDYwODIwMTU0MFoXDTE5MDcwOTIwMTU0MFowETEPMA0GA1UEAwwGV2ViUlRDMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEQ6ZyQYSC7xZHSUhq6mxCWa+/VwRdO0U1GuMAUYjYSkAmgzfJ0mW TZbZWJ12V7LlimkOZtdXCv2UPSh3sDOHu/DAKBggqhkjOPQQDAgNIADBFAiBDBGAgqa2dxlewHn5661OgMK6xfanL13l9aLoyuvdCgAIhAOmal8xD0S0ZLRKO/WDOb3 RRLAqxCcn0Jmur/OPdoEDl"},{"id":"RTCCertificate_D1:3F:71:1A:FE:32:78:3D:AA:93:06:F3:E6:AF:22:E9:5A:BC:B7:D2:AD:D6:A4:49:FD:9F:FB:00:49:89:4C:5A","timestamp":1560111849871.613,"type":"certificate","fingerprint":"D1:3F:71:1A:FE:32:78:3D:AA:93:06:F3:E6:AF:22:E9:5A:BC:B7:D2:AD:D6:A4:49:FD:9F:FB:00:49:89:4C:5A","fingerprintAlgorithm":"sha- 256","base64Certificate":"MIIBFzCBvaADAgECAgkA7svx50DzS6swCgYIKoZIzj0EAwIwETEPMA0GA1UEAwwGV2ViUlRDMB4XDTE5MDYwODIwMTU0MFoXDTE5MDcwOTIwMTU0MFowETEPMA0GA1UEAwwGV2ViUlRDMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEg3VCpwhtcNE3rEaBH8GTrWpQTcbnPGWEz0i2XAYE/A8pATEPoXb7mPojyEy8LxkmPSjkwrEodppNzA7X8Yjv2TAKBggqhkjOPQQDAgNJADBGAiEA3MWsKxnwNWssNhZ+mSGhZca63A8tvFIdRyf+CBOzNYwCIQD1JTiEdHntgBQ9/ZcaRVbQK MBU1X6+xYmtnwjJ+x2umw=="},{"id":"RTCCodec_0_Inbound_0","timestamp":1560111849871.613,"type":"codec","payloadType":0,"mimeType":"audio/PCMU","clockRate":8000},{"id":"RTCCodec_0_Inbound_103","timestamp":1560111849871.613,"type":"codec","payloadType":103,"mimeType":"audio/ISAC","clockRate":16000},{"id":"RTCCodec_0_Inbound_104","timestamp":1560111849871.613,"type":"codec","payloadType":104,"mimeType":"audio/ISAC","clockRate":32000},{"id":"RTCCodec_0_Inbound_105","t imestamp":1560111849871.613,"type":"codec","payloadType":105,"mimeType":"audio/CN","clockRate":16000},{"id":"RTCCodec_0_Inbound_106","timestamp":1560111849871.613,"type":"codec","payloadType":106,"mimeType":"audio/CN","clockRate":32000},{"id":"RTCCodec_0_Inbound_110","timestamp":1560111849871.613,"type":"codec","payloadType":110,"mimeType":"audio/telephone- event","clockRate":48000},{"id":"RTCCodec_0_Inbound_111","timestamp":1560111849871.613,"type":"codec","payloadType":111,"mimeType":"audio/opus","clockRate":48000},{"id":"RTCCodec_0_Inbound_112","timestamp":1560111849871.613,"type":"codec","payloadType":112,"mimeType":"audio/telephone-event","clockRate":32000},{"id":"RTCCodec_0_Inbound_113","timestamp":1560111849871.613,"type":"codec","payloadType":113,"mimeType":"audio/telephone- event","clockRate":16000},{"id":"RTCCodec_0_Inbound_126","timestamp":1560111849871.613,"type":"codec","payloadType":126,"mimeType":"audio/telephone- event","clockRate":8000},{"id":"RTCCodec_0_Inbound_13","timestamp":1560111849871.613,"type":"codec","payloadType":13,"mimeType":"audio/CN","clockRate":8000},{"id":"RTCCodec_0_Inbound_8","timestamp":1560111849871.613,"type":"codec","payloadType":8,"mimeType":"audio/PCMA","clockRate":8000},{"id":"RTCCodec_0_Inbound_9","timestamp":1560111849871.613,"type":"codec","payloadType":9,"mimeType":"audio/G722","clockRate":8000},{"id":"RTCCodec_0_Outbound_0","timestamp":1560111 849871.613,"type":"codec","payloadType":0,"mimeType":"audio/PCMU","clockRate":8000},{"id":"RTCCodec_0_Outbound_103","timestamp":1560111849871.613,"type":"codec","payloadType":103,"mimeType":"audio/ISAC","clockRate":16000},{"id":"RTCCodec_0_Outbound_104","timestamp":1560111849871.613,"type":"codec","payloadType":104,"mimeType":"audio/ISAC","clockRate":32000},{"id":"RTCCodec_0_Outbound_105","timestamp":1560111849871.613,"type":"codec","payloadType":105,"mimeType":"a udio/CN","clockRate":16000},{"id":"RTCCodec_0_Outbound_106","timestamp":1560111849871.613,"type":"codec","payloadType":106,"mimeType":"audio/CN","clockRate":32000},{"id":"RTCCodec_0_Outbound_110","timestamp":1560111849871.613,"type":"codec","payloadType":110,"mimeType":"audio/telephone- event","clockRate":48000},{"id":"RTCCodec_0_Outbound_111","timestamp":1560111849871.613,"type":"codec","payloadType":111,"mimeType":"audio/opus","clockRate":48000},{"id":"RTCCodec_0_Outbound_112","timestamp":1560111849871.613,"type":"codec","payloadType":112,"mimeType":"audio/telephone-event","clockRate":32000},{"id":"RTCCodec_0_Outbound_113","timestamp":1560111849871.613,"type":"codec","payloadType":113,"mimeType":"audio/telephone- event","clockRate":16000},{"id":"RTCCodec_0_Outbound_126","timestamp":1560111849871.613,"type":"codec","payloadType":126,"mimeType":"audio/telephone- event","clockRate":8000},{"id":"RTCCodec_0_Outbound_13","timestamp":1560111849871.613,"type":"codec","payloadType":13,"mimeType":"audio/CN","clockRate":8000},{"id":"RTCCodec_0_Outbound_8","timestamp":1560111849871.613,"type":"codec","payloadType":8,"mimeType":"audio/PCMA","clockRate":8000},{"id":"RTCCodec_0_Outbound_9","timestamp":1560111849871.613,"type":"codec","payloadType":9,"mimeType":"audio/G722","clockRate":8000},{"id":"RTCCodec_1_Inbound_100","timestamp":15 60111849871.613,"type":"codec","payloadType":100,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Inbound_101","timestamp":1560111849871.613,"type":"codec","payloadType":101,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_102","timestamp":1560111849871.613,"type":"codec","payloadType":102,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_107","timestamp":1560111849871.613,"type":"codec","payloadType":107,"mimeType":"vi deo/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_108","timestamp":1560111849871.613,"type":"codec","payloadType":108,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_109","timestamp":1560111849871.613,"type":"codec","payloadType":109,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_114","timestamp":1560111849871.613,"type":"codec","payloadType":114,"mimeType":"video/red","clockRate":90000},{"id":"RTCCodec_1_Inbound_115","times tamp":1560111849871.613,"type":"codec","payloadType":115,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_116","timestamp":1560111849871.613,"type":"codec","payloadType":116,"mimeType":"video/ulpfec","clockRate":90000},{"id":"RTCCodec_1_Inbound_119","timestamp":1560111849871.613,"type":"codec","payloadType":119,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_120","timestamp":1560111849871.613,"type":"codec","payloadType":120,"mimeT ype":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_121","timestamp":1560111849871.613,"type":"codec","payloadType":121,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_122","timestamp":1560111849871.613,"type":"codec","payloadType":122,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_123","timestamp":1560111849871.613,"type":"codec","payloadType":123,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_124", "timestamp":1560111849871.613,"type":"codec","payloadType":124,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_125","timestamp":1560111849871.613,"type":"codec","payloadType":125,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_127","timestamp":1560111849871.613,"type":"codec","payloadType":127,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_96","timestamp":1560111849871.613,"type":"codec","payloadType":9 6,"mimeType":"video/VP8","clockRate":90000},{"id":"RTCCodec_1_Inbound_97","timestamp":1560111849871.613,"type":"codec","payloadType":97,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_98","timestamp":1560111849871.613,"type":"codec","payloadType":98,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Inbound_99","timestamp":1560111849871.613,"type":"codec","payloadType":99,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_ 100","timestamp":1560111849871.613,"type":"codec","payloadType":100,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Outbound_101","timestamp":1560111849871.613,"type":"codec","payloadType":101,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_102","timestamp":1560111849871.613,"type":"codec","payloadType":102,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_107","timestamp":1560111849871.613,"type":"codec","payload Type":107,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_108","timestamp":1560111849871.613,"type":"codec","payloadType":108,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_109","timestamp":1560111849871.613,"type":"codec","payloadType":109,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_114","timestamp":1560111849871.613,"type":"codec","payloadType":114,"mimeType":"video/red","clockRate":90000},{"id":"RTC Codec_1_Outbound_115","timestamp":1560111849871.613,"type":"codec","payloadType":115,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_116","timestamp":1560111849871.613,"type":"codec","payloadType":116,"mimeType":"video/ulpfec","clockRate":90000},{"id":"RTCCodec_1_Outbound_119","timestamp":1560111849871.613,"type":"codec","payloadType":119,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_120","timestamp":1560111849871.613,"typ e":"codec","payloadType":120,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_121","timestamp":1560111849871.613,"type":"codec","payloadType":121,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_122","timestamp":1560111849871.613,"type":"codec","payloadType":122,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_123","timestamp":1560111849871.613,"type":"codec","payloadType":123,"mimeType":"video/H264","clockRate" :90000},{"id":"RTCCodec_1_Outbound_124","timestamp":1560111849871.613,"type":"codec","payloadType":124,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_125","timestamp":1560111849871.613,"type":"codec","payloadType":125,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_127","timestamp":1560111849871.613,"type":"codec","payloadType":127,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_96","timestamp":15 60111849871.613,"type":"codec","payloadType":96,"mimeType":"video/VP8","clockRate":90000},{"id":"RTCCodec_1_Outbound_97","timestamp":1560111849871.613,"type":"codec","payloadType":97,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_98","timestamp":1560111849871.613,"type":"codec","payloadType":98,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Outbound_99","timestamp":1560111849871.613,"type":"codec","payloadType":99,"mimeType":"video/r tx","clockRate":90000},{"id":"RTCIceCandidatePair_eDlSn4+Z_o4HO2DpI","timestamp":1560111849871.613,"type":"candidate- pair","transportId":"RTCTransport_0_1","localCandidateId":"RTCIceCandidate_eDlSn4+Z","remoteCandidateId":"RTCIceCandidate_o4HO2DpI","state":"succeeded","priority":9115038255631187000,"nominated":true,"writable":true,"bytesSent":114210539,"bytesReceived":509086,"totalRoundTripTime":0.085,"currentRoundTripTime":0.001,"availableOutgoingBitrate":3580817,"requestsReceived":207,"requestsSent":1,"responsesReceived":207,"responsesSent":207,"consentRequestsSent":206},{"id":"RTCIceCandi date_eDlSn4+Z","timestamp":1560111849871.613,"type":"local-candidate","transportId":"RTCTransport_0_1","isRemote":false,"networkType":"wifi","ip":"10.10.100.49","port":63479,"protocol":"udp","candidateType":"host","priority":2122260223,"deleted":false},{"id":"RTCIceCandidate_o4HO2DpI","timestamp":1560111849871.613,"type":"remote- candidate","transportId":"RTCTransport_0_1","isRemote":true,"ip":"10.10.100.49","port":52235,"protocol":"udp","candidateType":"host","priority":2122260223,"deleted":false},{"id":"RTCMediaStreamTrack_sender_3","timestamp":1560111849871.613,"type":"track","trackIdentifier":"93c3b5d1-5ad2-4d6f-b4ef- 9026d140cd12","remoteSource":false,"ended":false,"detached":false,"kind":"audio","audioLevel":0,"totalAudioEnergy":0,"totalSamplesDuration":0},{"id":"RTCMediaStreamTrack_sender_4","timestamp":1560111849871.613,"type":"track","trackIdentifier":"a6b75d65-225e-4a17-8566- 5ebfde288572","remoteSource":false,"ended":false,"detached":false,"kind":"video","frameWidth":640,"frameHeight":480,"framesSent":15281,"hugeFramesSent":3},{"id":"RTCMediaStream_OIL9zTwVhYSpGstFDdnOlBGaKGbLz16BJnlb","timestamp":1560111849871.613,"type":"stream","streamIdentifier":"OIL9zTwVhYSpGstFDdnOlBGaKGbLz16BJnlb","trackIds":["RTCMediaStreamTrack_sender_3","RTCMediaStreamTrack_sender_4"]},{"id":"RTCOutboundRTPAudioStream_3291843167","timestamp":15601118 49871.613,"type":"outbound-rtp","ssrc":3291843167,"isRemote":false,"mediaType":"audio","kind":"audio","trackId":"RTCMediaStreamTrack_sender_3","transportId":"RTCTransport_0_1","codecId":"RTCCodec_0_Outbound_111","packetsSent":25431,"retransmittedPacketsSent":0,"bytesSent":2185963,"retransmittedBytesSent":0},{"id":"RTCOutboundRTPVideoStream_3763384231","timestamp":1560111849871.613,"type":"outbound- rtp","ssrc":3763384231,"isRemote":false,"mediaType":"video","kind":"video","trackId":"RTCMediaStreamTrack_sender_4","transportId":"RTCTransport_0_1","codecId":"RTCCodec_1_Outbound_96","firCount":0,"pliCount":0,"nackCount":0,"qpSum":85518,"packetsSent":103425,"retransmittedPacketsSent":0,"bytesSent":110558779,"retransmittedBytesSent":0,"framesEncoded":15281,"totalEncodeTime":65.099},{"id":"RTCPeerConnection","timestamp":1560111849871.613,"type":"peer- connection","dataChannelsOpened":0,"dataChannelsClosed":0},{"id":"RTCTransport_0_1","timestamp":1560111849871.613,"type":"transport","bytesSent":114210539,"bytesReceived":509086,"dtlsState":"connected","selectedCandidatePairId":"RTCIceCandidatePair_eDlSn4+Z_o4HO2DpI","localCertificateId":"RTCCertificate_70:47:DC:96:5C:24:A1:C1:F0:A2:89:39:FC:B1:D9:69:66:EB:4A:F6:12:F0:68:43:1B:F9:D4:ED:D6:E2:D1:FE","remoteCertificateId":"RTCCertificate_D1:3F:71:1A:FE:32:78:3D:AA:93:06:F3:E 6:AF:22:E9:5A:BC:B7:D2:AD:D6:A4:49:FD:9F:FB:00:49:89:4C:5A"},{"id":"RTCCertificate_70:47:DC:96:5C:24:A1:C1:F0:A2:89:39:FC:B1:D9:69:66:EB:4A:F6:12:F0:68:43:1B:F9:D4:ED:D6:E2:D1:FE","timestamp":1560111929491.604,"type":"certificate","fingerprint":"70:47:DC:96:5C:24:A1:C1:F0:A2:89:39:FC:B1:D9:69:66:EB:4A:F6:12:F0:68:43:1B:F9:D4:ED:D6:E2:D1:FE","fingerprintAlgorithm":"sha- 256","base64Certificate":"MIIBFjCBvaADAgECAgkAhuXyX8OJm0EwCgYIKoZIzj0EAwIwETEPMA0GA1UEAwwGV2ViUlRDMB4XDTE5MDYwODIwMTU0MFoXDTE5MDcwOTIwMTU0MFowETEPMA0GA1UEAwwGV2ViUlRDMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEQ6ZyQYSC7xZHSUhq6mxCWa+/VwRdO0U1GuMAUYjYSkAmgzfJ0mW TZbZWJ12V7LlimkOZtdXCv2UPSh3sDOHu/DAKBggqhkjOPQQDAgNIADBFAiBDBGAgqa2dxlewHn5661OgMK6xfanL13l9aLoyuvdCgAIhAOmal8xD0S0ZLRKO/WDOb3 RRLAqxCcn0Jmur/OPdoEDl"},{"id":"RTCCertificate_D1:3F:71:1A:FE:32:78:3D:AA:93:06:F3:E6:AF:22:E9:5A:BC:B7:D2:AD:D6:A4:49:FD:9F:FB:00:49:89:4C:5A","timestamp":1560111929491.604,"type":"certificate","fingerprint":"D1:3F:71:1A:FE:32:78:3D:AA:93:06:F3:E6:AF:22:E9:5A:BC:B7:D2:AD:D6:A4:49:FD:9F:FB:00:49:89:4C:5A","fingerprintAlgorithm":"sha- 256","base64Certificate":"MIIBFzCBvaADAgECAgkA7svx50DzS6swCgYIKoZIzj0EAwIwETEPMA0GA1UEAwwGV2ViUlRDMB4XDTE5MDYwODIwMTU0MFoXDTE5MDcwOTIwMTU0MFowETEPMA0GA1UEAwwGV2ViUlRDMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEg3VCpwhtcNE3rEaBH8GTrWpQTcbnPGWEz0i2XAYE/A8pATEPoXb7mPojyEy8LxkmPSjkwrEodppNzA7X8Yjv2TAKBggqhkjOPQQDAgNJADBGAiEA3MWsKxnwNWssNhZ+mSGhZca63A8tvFIdRyf+CBOzNYwCIQD1JTiEdHntgBQ9/ZcaRVbQK MBU1X6+xYmtnwjJ+x2umw=="},{"id":"RTCCodec_0_Inbound_0","timestamp":1560111929491.604,"type":"codec","payloadType":0,"mimeType":"audio/PCMU","clockRate":8000},{"id":"RTCCodec_0_Inbound_103","timestamp":1560111929491.604,"type":"codec","payloadType":103,"mimeType":"audio/ISAC","clockRate":16000},{"id":"RTCCodec_0_Inbound_104","timestamp":1560111929491.604,"type":"codec","payloadType":104,"mimeType":"audio/ISAC","clockRate":32000},{"id":"RTCCodec_0_Inbound_105","t imestamp":1560111929491.604,"type":"codec","payloadType":105,"mimeType":"audio/CN","clockRate":16000},{"id":"RTCCodec_0_Inbound_106","timestamp":1560111929491.604,"type":"codec","payloadType":106,"mimeType":"audio/CN","clockRate":32000},{"id":"RTCCodec_0_Inbound_110","timestamp":1560111929491.604,"type":"codec","payloadType":110,"mimeType":"audio/telephone- event","clockRate":48000},{"id":"RTCCodec_0_Inbound_111","timestamp":1560111929491.604,"type":"codec","payloadType":111,"mimeType":"audio/opus","clockRate":48000},{"id":"RTCCodec_0_Inbound_112","timestamp":1560111929491.604,"type":"codec","payloadType":112,"mimeType":"audio/telephone-event","clockRate":32000},{"id":"RTCCodec_0_Inbound_113","timestamp":1560111929491.604,"type":"codec","payloadType":113,"mimeType":"audio/telephone- event","clockRate":16000},{"id":"RTCCodec_0_Inbound_126","timestamp":1560111929491.604,"type":"codec","payloadType":126,"mimeType":"audio/telephone- event","clockRate":8000},{"id":"RTCCodec_0_Inbound_13","timestamp":1560111929491.604,"type":"codec","payloadType":13,"mimeType":"audio/CN","clockRate":8000},{"id":"RTCCodec_0_Inbound_8","timestamp":1560111929491.604,"type":"codec","payloadType":8,"mimeType":"audio/PCMA","clockRate":8000},{"id":"RTCCodec_0_Inbound_9","timestamp":1560111929491.604,"type":"codec","payloadType":9,"mimeType":"audio/G722","clockRate":8000},{"id":"RTCCodec_0_Outbound_0","timestamp":1560111 929491.604,"type":"codec","payloadType":0,"mimeType":"audio/PCMU","clockRate":8000},{"id":"RTCCodec_0_Outbound_103","timestamp":1560111929491.604,"type":"codec","payloadType":103,"mimeType":"audio/ISAC","clockRate":16000},{"id":"RTCCodec_0_Outbound_104","timestamp":1560111929491.604,"type":"codec","payloadType":104,"mimeType":"audio/ISAC","clockRate":32000},{"id":"RTCCodec_0_Outbound_105","timestamp":1560111929491.604,"type":"codec","payloadType":105,"mimeType":"a udio/CN","clockRate":16000},{"id":"RTCCodec_0_Outbound_106","timestamp":1560111929491.604,"type":"codec","payloadType":106,"mimeType":"audio/CN","clockRate":32000},{"id":"RTCCodec_0_Outbound_110","timestamp":1560111929491.604,"type":"codec","payloadType":110,"mimeType":"audio/telephone- event","clockRate":48000},{"id":"RTCCodec_0_Outbound_111","timestamp":1560111929491.604,"type":"codec","payloadType":111,"mimeType":"audio/opus","clockRate":48000},{"id":"RTCCodec_0_Outbound_112","timestamp":1560111929491.604,"type":"codec","payloadType":112,"mimeType":"audio/telephone-event","clockRate":32000},{"id":"RTCCodec_0_Outbound_113","timestamp":1560111929491.604,"type":"codec","payloadType":113,"mimeType":"audio/telephone- event","clockRate":16000},{"id":"RTCCodec_0_Outbound_126","timestamp":1560111929491.604,"type":"codec","payloadType":126,"mimeType":"audio/telephone- event","clockRate":8000},{"id":"RTCCodec_0_Outbound_13","timestamp":1560111929491.604,"type":"codec","payloadType":13,"mimeType":"audio/CN","clockRate":8000},{"id":"RTCCodec_0_Outbound_8","timestamp":1560111929491.604,"type":"codec","payloadType":8,"mimeType":"audio/PCMA","clockRate":8000},{"id":"RTCCodec_0_Outbound_9","timestamp":1560111929491.604,"type":"codec","payloadType":9,"mimeType":"audio/G722","clockRate":8000},{"id":"RTCCodec_1_Inbound_100","timestamp":15 60111929491.604,"type":"codec","payloadType":100,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Inbound_101","timestamp":1560111929491.604,"type":"codec","payloadType":101,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_102","timestamp":1560111929491.604,"type":"codec","payloadType":102,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_107","timestamp":1560111929491.604,"type":"codec","payloadType":107,"mimeType":"vi deo/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_108","timestamp":1560111929491.604,"type":"codec","payloadType":108,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_109","timestamp":1560111929491.604,"type":"codec","payloadType":109,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_114","timestamp":1560111929491.604,"type":"codec","payloadType":114,"mimeType":"video/red","clockRate":90000},{"id":"RTCCodec_1_Inbound_115","times tamp":1560111929491.604,"type":"codec","payloadType":115,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_116","timestamp":1560111929491.604,"type":"codec","payloadType":116,"mimeType":"video/ulpfec","clockRate":90000},{"id":"RTCCodec_1_Inbound_119","timestamp":1560111929491.604,"type":"codec","payloadType":119,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_120","timestamp":1560111929491.604,"type":"codec","payloadType":120,"mimeT ype":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_121","timestamp":1560111929491.604,"type":"codec","payloadType":121,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_122","timestamp":1560111929491.604,"type":"codec","payloadType":122,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_123","timestamp":1560111929491.604,"type":"codec","payloadType":123,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_124", "timestamp":1560111929491.604,"type":"codec","payloadType":124,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_125","timestamp":1560111929491.604,"type":"codec","payloadType":125,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_127","timestamp":1560111929491.604,"type":"codec","payloadType":127,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Inbound_96","timestamp":1560111929491.604,"type":"codec","payloadType":9 6,"mimeType":"video/VP8","clockRate":90000},{"id":"RTCCodec_1_Inbound_97","timestamp":1560111929491.604,"type":"codec","payloadType":97,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Inbound_98","timestamp":1560111929491.604,"type":"codec","payloadType":98,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Inbound_99","timestamp":1560111929491.604,"type":"codec","payloadType":99,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_ 100","timestamp":1560111929491.604,"type":"codec","payloadType":100,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Outbound_101","timestamp":1560111929491.604,"type":"codec","payloadType":101,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_102","timestamp":1560111929491.604,"type":"codec","payloadType":102,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_107","timestamp":1560111929491.604,"type":"codec","payload Type":107,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_108","timestamp":1560111929491.604,"type":"codec","payloadType":108,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_109","timestamp":1560111929491.604,"type":"codec","payloadType":109,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_114","timestamp":1560111929491.604,"type":"codec","payloadType":114,"mimeType":"video/red","clockRate":90000},{"id":"RTC Codec_1_Outbound_115","timestamp":1560111929491.604,"type":"codec","payloadType":115,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_116","timestamp":1560111929491.604,"type":"codec","payloadType":116,"mimeType":"video/ulpfec","clockRate":90000},{"id":"RTCCodec_1_Outbound_119","timestamp":1560111929491.604,"type":"codec","payloadType":119,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_120","timestamp":1560111929491.604,"typ e":"codec","payloadType":120,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_121","timestamp":1560111929491.604,"type":"codec","payloadType":121,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_122","timestamp":1560111929491.604,"type":"codec","payloadType":122,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_123","timestamp":1560111929491.604,"type":"codec","payloadType":123,"mimeType":"video/H264","clockRate" :90000},{"id":"RTCCodec_1_Outbound_124","timestamp":1560111929491.604,"type":"codec","payloadType":124,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_125","timestamp":1560111929491.604,"type":"codec","payloadType":125,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_127","timestamp":1560111929491.604,"type":"codec","payloadType":127,"mimeType":"video/H264","clockRate":90000},{"id":"RTCCodec_1_Outbound_96","timestamp":15 60111929491.604,"type":"codec","payloadType":96,"mimeType":"video/VP8","clockRate":90000},{"id":"RTCCodec_1_Outbound_97","timestamp":1560111929491.604,"type":"codec","payloadType":97,"mimeType":"video/rtx","clockRate":90000},{"id":"RTCCodec_1_Outbound_98","timestamp":1560111929491.604,"type":"codec","payloadType":98,"mimeType":"video/VP9","clockRate":90000},{"id":"RTCCodec_1_Outbound_99","timestamp":1560111929491.604,"type":"codec","payloadType":99,"mimeType":"video/r tx","clockRate":90000},{"id":"RTCIceCandidatePair_eDlSn4+Z_o4HO2DpI","timestamp":1560111929491.604,"type":"candidate- pair","transportId":"RTCTransport_0_1","localCandidateId":"RTCIceCandidate_eDlSn4+Z","remoteCandidateId":"RTCIceCandidate_o4HO2DpI","state":"succeeded","priority":9115038255631187000,"nominated":true,"writable":true,"bytesSent":132172362,"bytesReceived":588712,"totalRoundTripTime":0.097,"currentRoundTripTime":0,"availableOutgoingBitrate":3580817,"requestsReceived":239,"requestsSent":1,"responsesReceived":239,"responsesSent":239,"consentRequestsSent":238},{"id":"RTCIceCandidate _eDlSn4+Z","timestamp":1560111929491.604,"type":"local-candidate","transportId":"RTCTransport_0_1","isRemote":false,"networkType":"wifi","ip":"10.10.100.49","port":63479,"protocol":"udp","candidateType":"host","priority":2122260223,"deleted":false},{"id":"RTCIceCandidate_o4HO2DpI","timestamp":1560111929491.604,"type":"remote- candidate","transportId":"RTCTransport_0_1","isRemote":true,"ip":"10.10.100.49","port":52235,"protocol":"udp","candidateType":"host","priority":2122260223,"deleted":false},{"id":"RTCMediaStreamTrack_sender_3","timestamp":1560111929491.604,"type":"track","trackIdentifier":"93c3b5d1-5ad2-4d6f-b4ef- 9026d140cd12","remoteSource":false,"ended":false,"detached":false,"kind":"audio","audioLevel":0,"totalAudioEnergy":0,"totalSamplesDuration":0},{"id":"RTCMediaStreamTrack_sender_4","timestamp":1560111929491.604,"type":"track","trackIdentifier":"a6b75d65-225e-4a17-8566- 5ebfde288572","remoteSource":false,"ended":false,"detached":false,"kind":"video","frameWidth":640,"frameHeight":480,"framesSent":17672,"hugeFramesSent":3},{"id":"RTCMediaStream_OIL9zTwVhYSpGstFDdnOlBGaKGbLz16BJnlb","timestamp":1560111929491.604,"type":"stream","streamIdentifier":"OIL9zTwVhYSpGstFDdnOlBGaKGbLz16BJnlb","trackIds":["RTCMediaStreamTrack_sender_3","RTCMediaStreamTrack_sender_4"]},{"id":"RTCOutboundRTPAudioStream_3291843167","timestamp":15601119 29491.604,"type":"outbound-rtp","ssrc":3291843167,"isRemote":false,"mediaType":"audio","kind":"audio","trackId":"RTCMediaStreamTrack_sender_3","transportId":"RTCTransport_0_1","codecId":"RTCCodec_0_Outbound_111","packetsSent":29412,"retransmittedPacketsSent":0,"bytesSent":2502933,"retransmittedBytesSent":0},{"id":"RTCOutboundRTPVideoStream_3763384231","timestamp":1560111929491.604,"type":"outbound- rtp","ssrc":3763384231,"isRemote":false,"mediaType":"video","kind":"video","trackId":"RTCMediaStreamTrack_sender_4","transportId":"RTCTransport_0_1","codecId":"RTCCodec_1_Outbound_96","firCount":0,"pliCount":0,"nackCount":0,"qpSum":98593,"packetsSent":119697,"retransmittedPacketsSent":0,"bytesSent":127973312,"retransmittedBytesSent":0,"framesEncoded":17672,"totalEncodeTime":75.133},{"id":"RTCPeerConnection","timestamp":1560111929491.604,"type":"peer- connection","dataChannelsOpened":0,"dataChannelsClosed":0},{"id":"RTCTransport_0_1","timestamp":1560111929491.604,"type":"transport","bytesSent":132172362,"bytesReceived":588712,"dtlsState":"connected","selectedCandidatePairId":"RTCIceCandidatePair_eDlSn4+Z_o4HO2DpI","localCertificateId":"RTCCertificate_70:47:DC:96:5C:24:A1:C1:F0:A2:89:39:FC:B1:D9:69:66:EB:4A:F6:12:F0:68:43:1B:F9:D4:ED:D6:E2:D1:FE","remoteCertificateId":"RTCCertificate_D1:3F:71:1A:FE:32:78:3D:AA:93:06:F3:E 6:AF:22:E9:5A:BC:B7:D2:AD:D6:A4:49:FD:9F:FB:00:49:89:4C:5A"}]
  9. 9. Several data dimensions Transport Endpoint Platform Infrastructure ISP name / AS number Device type Topology (P2P / SFU / MCU) User network: End-user location , Network Type Operating system Server locations Media engine: Audio and video frame rate and size variations Browser / RTC stack Server network Packetization: Round Trip Time, Jitter, Packet loss CPU type App Version
  10. 10. Challenge: dealing with lots of data appVer buildVer buildName osName osVer totalMeanAudioRtt totalMeanVideoRtt countVideoRtt countAudioRtt totalInboundAudioPacketsLost totalInboundVideoPacketsLost totalOutboundAudioPacketsLost totalOutboundVideoPacketsLost totalInboundAudioPackets totalInboundVideoPackets totalOutboundAudioPackets totalOutboundVideoPackets totalMeanInboundVideoThroughput countInboundVideoThroughput totalMeanInboundAudioThroughput countInboundAudioThroughput totalMeanOutboundAudioThroughput countOutboundAudioThroughput totalMeanOutboundVideoThroughput Many Metrics
  11. 11. Challenge: dealing with lots of data appVer buildVer buildName osName osVer totalMeanAudioRtt totalMeanVideoRtt countVideoRtt countAudioRtt totalInboundAudioPacketsLost totalInboundVideoPacketsLost totalOutboundAudioPacketsLost totalOutboundVideoPacketsLost totalInboundAudioPackets totalInboundVideoPackets totalOutboundAudioPackets totalOutboundVideoPackets totalMeanInboundVideoThroughput countInboundVideoThroughput totalMeanInboundAudioThroughput countInboundAudioThroughput totalMeanOutboundAudioThroughput countOutboundAudioThroughput totalMeanOutboundVideoThroughput Many Metrics Measures× • Average • Mean • Percentile • Skew • Kurtosis
  12. 12. Challenge: dealing with lots of data appVer buildVer buildName osName osVer totalMeanAudioRtt totalMeanVideoRtt countVideoRtt countAudioRtt totalInboundAudioPacketsLost totalInboundVideoPacketsLost totalOutboundAudioPacketsLost totalOutboundVideoPacketsLost totalInboundAudioPackets totalInboundVideoPackets totalOutboundAudioPackets totalOutboundVideoPackets totalMeanInboundVideoThroughput countInboundVideoThroughput totalMeanInboundAudioThroughput countInboundAudioThroughput totalMeanOutboundAudioThroughput countOutboundAudioThroughput totalMeanOutboundVideoThroughput Many Metrics Measures× × n samples • Average • Mean • Percentile • Skew • Kurtosis • Sample rate • Number of participants • Call duration
  13. 13. Challenge: dealing with lots of data appVer buildVer buildName osName osVer totalMeanAudioRtt totalMeanVideoRtt countVideoRtt countAudioRtt totalInboundAudioPacketsLost totalInboundVideoPacketsLost totalOutboundAudioPacketsLost totalOutboundVideoPacketsLost totalInboundAudioPackets totalInboundVideoPackets totalOutboundAudioPackets totalOutboundVideoPackets totalMeanInboundVideoThroughput countInboundVideoThroughput totalMeanInboundAudioThroughput countInboundAudioThroughput totalMeanOutboundAudioThroughput countOutboundAudioThroughput totalMeanOutboundVideoThroughput Many Metrics Measures× × n samples • Average • Mean • Percentile • Skew • Kurtosis • Sample rate • Number of participants • Call duration × m customers • Customer segments
  14. 14. Why do we need ML? Reducing costs 1 Image source: http://xingwu.me/2014/12/10/My-AWS-Account-Got-Compromised/
  15. 15. Challenge: figuring out what matters Customer Value Operating Complexity
  16. 16. Why do we need ML? Saving users from analytics overload 1
  17. 17. Typical troubleshooting approach 17 Generate hypothesis Look for anomalies Filter Evaluate sub- segment
  18. 18. Traditional troubleshooting challenges 18 Generate hypothesis Look for anomalies Filter Evaluate sub- segment Experience required to generate good hypotheses Not all anomalies are obvious Many filters to define Many sub- segments to review
  19. 19. Ideal solution: the system should tell you where to look 19
  20. 20. Finding Hidden Call Quality Issues with Machine Learning 20 Deliver Better User Experiences
  21. 21. Machine Learning approaches 21 Image source: https://towardsdatascience.com/deep-learning-for-image-classification-why-its-challenging-where-we-ve-been-and-what- s-next-93b56948fcef
  22. 22. Machine Learning approaches 22 Image source: https://towardsdatascience.com/deep-learning-for-image-classification-why-its-challenging-where-we-ve-been-and-what- s-next-93b56948fcef
  23. 23. Methodology 23 Collect data Check the data Clean the data Feature reduction Clustering Labeling
  24. 24. Collect data 24
  25. 25. Check your data 25
  26. 26. Clean your data • Missing values • Duplicates • Outliers • Balancing • Normalization 26
  27. 27. Dimension reduction 27 Feature elimination Feature selection Feature extraction Reducing unnecessary complexity
  28. 28. Feature selection Correlation matrix to determine significance 28 meanAudioRtt meanVideoRtt Local Latitude Local Longitude Server Latitude Server Longitude meanAudioRtt 100% 42% -24% 59% 0% 0% meanVideoRtt 42% 100% -15% 29% 0% 0% Local Latitude -24% -15% 100% -23% 0% 0% Local Longitude 59% 29% -23% 100% 0% 0% Server Latitude 0% 0% 0% 0% 100% -100% Server Longitude 0% 0% 0% 0% -100% 100%
  29. 29. Round Trip Time (RTT) 29 Round-trip time is the time it takes for a packet to travel through an IP network, from a sending endpoint to a receiving endpoint and back, not including the time to process the packet at its destination. Many factors affect RTT, like propagation delay, processing delay, queuing delay, and transmission delay. Sender ReceiverNetwork 𝑡 𝑠𝑡𝑎𝑟𝑡 𝑡 𝑎𝑐𝑘 𝑅𝑇𝑇 = 𝑡 𝑎𝑐𝑘 − 𝑡 𝑠𝑡𝑎𝑟𝑡 − 𝑡 𝑠𝑙𝑟 𝑡 𝑠𝑙𝑟
  30. 30. Feature reduction Principal Component Analysis 30 http://setosa.io/ev/principal-component-analysis/ • Find feature-combinations that have the most variation • Creates a new set of dimensions to maximize these variations • Remove the feature-combinations that don’t show much variation
  31. 31. Clustering with Gaussian Mixture Model (GMM) 31
  32. 32. Optimizing the model 32 Overfit model Fit model Minimize the number of variables needed in your model Minimize the number of clusters Maintain predictiveness on new datasets
  33. 33. Determining the optimal cluster size with Bayesian Inference Criterion (BIC) 33Number of Clusters BICScore Number of PCA dimensions
  34. 34. Methodology: Iterative in practice 34 Collect data Check the data Clean the data Dimension reduction Clustering Labeling
  35. 35. Example of a customer handling Millions of calls a day 3
  36. 36. RTT distribution for this service is 36 Range Proportion of the data (%) Low 0 < RTT < 150 68% Medium 150 < RTT < 250 19% Large 250 < RTT < 700 12% Very large RTT > 700 1%
  37. 37. RTT distribution for this service is 37 Area for exploration Range Proportion of the data (%) Low 0 < RTT < 150 68% Medium 150 < RTT < 250 19% Large 250 < RTT < 700 12% Very large RTT > 700 1%
  38. 38. Cluster results 38
  39. 39. Cluster analysis results 39
  40. 40. Cluster analysis results 40
  41. 41. Cluster 0: 16% of high RTTs • Network conditions are good • Local users in a specific country • Distance can be the main cause for large RTTs
  42. 42. Cluster 5: 20% of high RTTs • Bad network conditions • Large jitter, Large fraction loss • Users all over the world • Roughly large distance • Network congestion and distance can be the main cause of large RTTs 42
  43. 43. Cluster 3: 50% of high RTTs • Good Network condition • Local users in Asia • Main cause of large RTT can be large distance
  44. 44. Analysis summary 44 32% of calls have high RTT's 20% due to congestion 2 geographic problem areas Check server infrastructure & network Add/move servers
  45. 45. Real solution: the system tells you where to look 45

×