Let’s play with WebRTC statistics and the getStats() API. WebRTC Stats expert Varun Singh will review the many statistics WebRTC provides, what they mean, and how to use them to keep your service running smoothly.

1. WebRTC Statistics and
Analytics
Varun Singh
CEO, @callstatsio
twitter: @vr000m
São Paulo, 22-11-2016

2. Multimedia Systems
• connected to a network
• capture and transmit media
• receive and render media
• video codecs: e.g., H.264, VP8, VP9, …
• audio codecs: e.g., G.711, Opus, …
2

3. Real-time Transport Protocol (RTP)
3
Sender Receiver
• Dejittering, sync, playout
• Monitoring + reporting
RTP media stream
(encoded media, FEC, repair)

4. RTP Control Protocol (RTCP)
4
Sender Receiver
RTCP Sender Reports (SRs)
RTCP Receiver Reports (RRs)
• Dejittering, sync, playout
• Monitoring + reporting
RTP media stream
(encoded media, FEC, repair)

5. RTCP Sender Report (SR)
5
Sender Receiver
RTP media stream
(encoded media, FEC, repair)
RTCP Sender Reports (SRs)
• Timing, synchronisation
• Sending rate, packet count
RTCP Receiver Reports (RRs)
• Dejittering, sync, playout
• Monitoring + reporting

6. RTCP Receiver Report
6
Sender Receiver
RTP media stream
(encoded media, FEC, repair)
RTCP Sender Reports (SRs)
• Timing, synchronisation
• Sending rate, packet count
RTCP Receiver Reports (RRs)
• Rough statistics
• Congestion cues
• Dejittering, sync, playout
• Monitoring + reporting

7. Adaptation
7
Sender Receiver
RTP media stream
(encoded media, FEC, repair)
RTCP Sender Reports (SRs)
• Timing, synchronisation
• Sending rate, packet count
RTCP Receiver Reports (RRs)
• Rough statistics
• Congestion cues
• Dejittering, sync, playout
• Monitoring + reportingShort-term adaptation
• Error-resilience (NACK, PLI)
• Congestion control
• Adaptive source coding

8. http://geek-and-poke.com/
8

9. It is not all about throughput*
… it is about latency†! *)capaci
bandwid
speed
audio only call
9
†)delay,
one-way delay
time (sec)

10. Measuring User Experience
10
collect metrics and user actions
CallExperience

11. Measuring User Experience
11
measure setup time
CallSetup
CallExperience

12. Measuring User Experience
12
Ask the right question!
CallSetup
CallExperience
CallFeedback

13. Measuring User Experience
13
CallSetup
CallExperience
CallFeedback
CallInit(Context)
PageLoad
represents overall quality

14. What to Measure?
• Network metrics
• bits per second, RTT, jitter, packet losses, …
• Multimedia pipeline metrics
• playout delay, frames metrics, …
• Quality Models from metrics
• Annoyances
• resolution/frame rate changes, interface changes, …
• failure to setup (NATs, incompatibility)
• dropped calls (insufficient capacity, high latency, high losses)
14

15. getStats() API
15
Track
pc.getStats(mst, …)
IceTransportDtlsTransport
PeerConnection
Tracks
Datachannels
RtpReceiver
SctpTransport
can invoke the API as often (>150ms)
most metrics are cumulative metrics
pc.getStats(null, …)
partial interface RTCPeerConnection {
Promise<RTCStatsReport> getStats(optional MediaStreamTrack? selector = null);
};

16. getStats() API
Example:
var selector = pc.getRemoteStreams()[0].getAudioTracks()[0];
pc.getStats(selector).then(function (report) {
processStats(report);
})
.catch(function (error) {
log(error.toString()); //if something went wrong
});
function processStats(report) {
for (var i in report) {
var now = report[i];
if (now.type == "outbound-rtp") {
//do something
}
}
16
Output:
{
timestamp:1479753828
ssrc:3938144725
packetsSent:15656
bytesSent:1562150
roundTripTime:31
…
}

17. Example
17
Internet
PeerConnection
PeerConnection
Alice
Bob
media
direction

18. Sender-side pipeline
18
Internet
IceTransportDtlsTransport
PeerConnection
Tracks RtpReceiver
DtlsTransport IceTransport
PeerConnection
Tracks RtpSender
media
direction

19. Receiver-side pipeline
19
IceTransportDtlsTransport
PeerConnection
Tracks RtpReceiver
DtlsTransport IceTransport
PeerConnection
Tracks RtpSender
Internet
media
direction

20. Frame height
20
DtlsTransport IceTransport
PeerConnection
Tracks RtpSender

21. RTP Stats
21
DtlsTransport IceTransport
PeerConnection
Tracks RtpSender

22. ICE Stats
22
DtlsTransport IceTransport
PeerConnection
Tracks RtpSender

23. Simplified E-model (audio)
23
ITU-TREC-G.114
https://en.wikipedia.org/wiki/Mean_opinion_score

24. Example (1/3)
var selector = pc.getRemoteStreams()[0].getAudioTracks()[0];
var rttMeasures = [];
setTimeout(function () {
pc.getStats(selector, function (report) {
// do something
}, logError);
}, 1000);
24

25. Example (2/3)
var selector = pc.getRemoteStreams()[0].getAudioTracks()[0];
var rttMeasures = [];
setTimeout(function () {
pc.getStats(selector, function (report) {
for (var i in report) {
var now = report[i];
if (now.type == "outbound-rtp") {
//do something
}
}
}, logError);
}, 1000);
25

26. Example (3/3)
var selector = pc.getRemoteStreams()[0].getAudioTracks()[0];
var rttMeasures = [];
setTimeout(function () {
pc.getStats(selector, function (report) {
for (var i in report) {
var now = report[i];
if (now.type == "outbound-rtp") {
rttMeasures.append(now.roundTripTime);
var avgRoundTripTime = average(rttMeasures);
var emodel = simplemodel(avgRoundTripTime);
console.log ("e-model: "+str(emodel));
//can add jitter, losses, framesDiscarded, etc
}
}
}, logError);
}, 1000);
26

27. What more can you do?
27

28. ICE States
28
https://w3c.github.io/webrtc-pc/

29. Disruption: loss of connectivity when
network interfaces change, low available
capacity,or high delay
The light grey vertical lines show disruption, highlighted by
the red bounding boxes.29
Network Disruptions

30. Disruptions and user behaviour
User Behaviour: The user tries to
correct for the disruption by
turning on and off video
30

31. 31

32. Lessons learnt
• Deploy TURN servers with TCP support
• Detect crashes, disruptions and re-setup connections
• media pipelines sometimes crash
• screen sharing plugin crash
• network connectivity re-establishment
32

33. Summary
• RTP and RTCP
• getStats() API
• Simplified E-model
33