What is Objective Quality?

Objective Quality (OQ) is a callstats.io proprietary metric that helps our customers evaluate the quality of their conferences. It evaluates the quality of a conference by giving each stream a score. OQ is a popular high-order metric displayed on our dashboard.

We evaluate the quality of the conference by estimating the user annoyance and, consequently, the user behavior, which is the central element to the OQ score. For example, in video communication, if the audio and video are desynced for more than 200 ms, it is noticeable to the participants, which is an unpleasant experience. Despite this, our data indicates that users typically stay and do not drop out of the call because they are willing to cope with the constant delay between audio and video. While the quality of desynced audio and video is not excellent, it might still be good enough for the user to continue the call. Dissimilarly, if, the audio and video delay constantly fluctuates, the quality should be lower. In many cases, the audio going in and out of sync annoys the user to the point of dropping out and rejoining, also known as churn.

Typically, real-time communications have a strict timeline for playback and rendering for the communication to be interactive. ITU-T recommends the maximum threshold for end-to-end (capture to render) latency to be 150ms. Similarly, 300ms end-to-end latency is imposed on video. Most video applications consider changing frame size at short intervals (less than every few seconds) to affect the quality of experience significantly. 

Objective quality takes all the above into account and penalizes streams that have the following features (non-exhaustive list):  

1. Audio or video out of sync.
2. Variation in audio or video sync at short intervals.
3. Variation in frame size at short intervals.
4. Variation in delay.
5. Throughput, jitter, audio output, and more.

Objective quality returns a value between 0 and 4dB, where below 1 is considered poor quality and above 2.0 is considered excellent quality. 

As always, participants are still able to rate their experience and provide feedback, which can be compared to the objective quality score provided by callstats.io.