Update: Optimize - AI for WebRTC

Since we began working on callstats.io, we considered machine learning techniques as a natural progression to our work on congestion control. Congestion control does bandwidth estimation and uses the input from the remote endpoint and depending on the ISP, assistance from the network in the form of Explicit Congestion Control (ECN) to send media across the Internet. Consequently, when a call starts or when network interfaces change, for example, switching from WiFi to Mobile or Ethernet or vice-versa, the congestion control either uses the default settings provided by the platform or the application developer.

Over the past 4 years, callstats.io has collected vast amounts of data from the end-users of our ever-growing customer base. In 2017, we put together our first AI-team (Lennart, Balazs, Navid, Marcin, Mateusz, and Jeremi) to develop AI-algorithms that provide best quality – always and everywhere. Optimize assists our customers deliver optimal quality media, diagnose media quality issues, and as a consequence helps our customers build better real-time communication products.

Optimal Configurations

The aim for every real-time communication product is to deliver the best media quality. That is optimal for the set of end-users in the call, based on their individual locations, the diverse device capabilities, and/or prevalent network conditions between them. At callstats.io, we call this combination, context. Callstats.io’s Optimize is an intelligent system that ingests the data from all the end-users and provides the optimal configuration for those set of end-users. It is a multi-variable trade-off and the decision made in real-time. The configurations are provided automatically in the background without any explicit action from an end-user.

Figure 1: Optimize provides the optimal settings for the best possible media quality in diverse contexts.

Figure 1 shows that the end-users will get better quality despite diverse contexts. Situation on the left, the device is highly capable and the network is highly variable, results in varying media quality (shown by the purple area). Optimize tries to provide settings that maximizes media quality (marked by the deeper shades of purple). Alternatively, devices that have fewer capabilities and poorly performing networks, usually results in bad media quality (inaudible or jittery audio, freeszing or out of sync video). Optimize in this case attempts to provide media settings that result in consistent media quality, not neccessarily highest possible quality.

Anomaly Detection

Monitoring products are used in various ways: proactively when actively looking for issues, reactively when notified by alerts or by the customer, or passively as a dashboard at a glance. Learn more about these situations from our blog posts:

• Who is responsible for WebRTC monitoring
• Benefits of using callstats.io for WebRTC monitoring

In all these situations the customer is looking for emergent problems. At any large-scale services, the teams are continuously developing new features to enhance engagement, increasing reliability by decreasing the amount of annoyances, or improving scalability by lowering the cost of delivering the service. At the same time they are trying to monitor the service to make sure the service is performant. On callstats.io, they do this with our Objective Quality metric.

However, to keep up with the ever changing release of new browsers, diverse end-user Internet Service Providers and end-user devices, it is hard to come up with a hypothesis for why the media quality may not be perfect globally. Callstats.io’s Optimize finds the needles in the haystack of data. Our algorithms assess in real-time all factors that might affect the engagement, reliability and scalability of the service, and highlights issues that might have gone unnoticed by the customer. At any given time, the teams are aware of some issues that affect media quality but not all. Optimize surfaces all the emergent issues, the customer can provide input on which of these need to be resolved urgently, ergo the algorithm learns from customer input on which areas to focus on.

To summarize, low media quality occurs due to a variety of reasons, most of which can be fixed automatically. Optimize gives real-time communications providers immediate access to call quality insights that would otherwise take months or even years to accumulate.