Related to video playback Quality of Service (QoS), there are a number of Key Performance Indicators (KPIs) that are important to establish an acceptable playback experience versus what should cause concern for any content creator/distributor. A lot of work goes into adjusting streaming profiles to ensure optimized perceptual quality, but there are factors outside of the playback profile that cause a poor experience. Things like drop in average bitrate, failed or slow video startup, general playback failures and re-buffering are all indicators of a negative video experience. The primary goal of a Streaming Operations team is to ensure the highest possible video playback experience, using the traditional television experience as the baseline. Datazoom has the capability to quickly capture datapoints used in calculating these KPIs.
|Average bitrate||Calculation of the average playback bitrate reported for a given time period or for the last X minutes/seconds in a real-time scenario.|
|Buffering||Calculation where playback stalled for a given time period or for the last X minutes/seconds in a real-time scenario. Represented as a count, duration, or percentage of viewing experience.|
|Playback failures||General metric with any number of causes, sometimes aligned with a player or platform specific error code.|
|Startup delays/failures||Calculation between request for video and when the first frame of video is played or never played.|
|Concurrent User Counts||Total number of viewers at a given time. Useful as reference to understand performance over load (number of people actively consuming content)|
|Views/Sessions per User||Useful to understand if this number increases or decreases by platform.|
|Usage by Device type/Platform||Good indicator of which device/platform is popular. Also a good way to track app updates/OS updates. Did usage go up or down post release compared to the average?|
|Content Completion Percentage||Are people watching content to completion? Useful in determining other factors that might impact playback completion beyond people just abandoning content.|
Customers will usually have alerts configured to fire when these metrics cross a certain threshold. If the analytics platform allows, those alerts can be configured specifically for things like a particular live event, platform, device, app, version, etc. Once an alert fires, drilling down into additional data like CDN, ISP, region is important to help isolate and understand impact. Further, getting detail about the specific user session is sometimes valuable in determining other environmental variables that could have contributed to the issue. In addition to content playback, it's important to know if the stream was in an ad break as that will expose another set of variables that will impact QoE. Capturing those datapoints related to ad playback QoE is equally important.