Update: Handling Multiple Media Stream Tracks

WebRTC is progressing rapidly; the main trend seems to be transitioning from peer-to-peer to conferences with centralized servers. This is occurring mainly for conferences with more than 5 participants. In a full-mesh video, every participant sends media to all other participants, i.e., if there are M participants, an endpoint sends M-1 media streams and receives M-1 media streams (shown in Fig 1) 1. In this case, each peerconnection carries media stream tracks corresponding to the audio, camera, screen, and data between the pair of participants.

Figure 1. Mesh topology

Alternatively, when using a conferencing server a participant sends its media stream to the conference server and receives media streams from the conference server and not the individual participants. Hence, an endpoint sends 1 media stream and receives M-1 media streams (shown in Fig 2). Mapping this to WebRTC 1.0 terminology, an M participant conference will have M links 2. However, in this case, a single peer connection carries multiple incoming media streams corresponding to each participant.

Figure 2. Star topology

In our API, when setting up a peer connection, the developer specifies the localUserID and remoteUserID for the peerconnection (See addNewFabric()). The remoteUserID in a mesh topology is the remote participant’s userID (see Figure 3a). However, when interacting with the conference server, the developer is most likely going to use the name or identifier of the conference bridge as the remoteUserID, as it cannot use a particular remote user ID as they may be several. A typical conference bridge (for example, Jitsi Videobridge) transmits multiple media stream tracks within a peer connection (see figure 3b).

Figure 3. Connections viewed by a single participant.

Furthermore, correlating the received Synchronization Source Identifier (SSRC) with the SSRC generated by every participant for each media stream track may not work if the conferencing server rewrites the incoming SSRC. Therefore, making it difficult for the callstats.js to identify which incoming media stream belongs to which participant even though we might know the SSRCs generated by each endpoint.

Today, we are announcing a new API that solves the issue and will be available in callstats.js version 3.3.0 associateMstWithUserID() maps each media stream track received by an endpoint to a remote userID (see figure 4). The example below shows how the API can be integrated:

Figure 4. Associating userID with MST.

At callstats.io, our vision to provide high quality of media experience to every interaction extends beyond serving endpoints and to middleboxes. There are several types of middleboxes in the wild: relay servers (for NAT/firewall traversals), conferencing server, and transcoders (H.264 to VP8, AMR to Opus). Getting metrics and statistics from the middlebox is important for scaling large deployments. Unlike endpoints which either send or receive a particular stream, middleboxes send and receive each stream, unless a local policy dictates not sending a particular subset of streams (e.g., send only active speakers). To facilitate middleboxes to report statistics on the media they received and they sent, we use the following structure: