Here, we’ll review KPIs you can build in most analytics systems based on Datazoom’s Data Dictionary. We’ll provide you with some generalized formulas, with Data Dictionary terms bolded.
You can then translate these expressions into the specific querying language of the tool you’re using, or another format your system of choice may require. Keep in mind that these formulas are sample starting points. They are by no means the “end all and be all” of formulas. Part of the beauty of the Data Dictionary is its role as a springboard for customizing metrics in a fashion which best suits your organization.
However, please note that similar to how players expose data points by different names, each player exposes different data points entirely. Thus, some of the metrics below may not be supported across players. To check if the players your team relies on are supported, check out our documentation here.
Index
- General Metric Formulas
- Play Attempts
- Play Starts
- Active Plays
- Video Start Failures
- Exits Before Video Start
- Concurrent Plays
- Video Bitrate
- Video Startup Time
- Completion Rates
- Total Time Watched
- Total Engagement Time
- Ad Watched Time
- Content Watch Time
- Stall Ratio
- Buffer Ratio
- Percent Complete
- Playback Stalls
- Playback Stall Rate
- Playback Stall Ratio
- Filters & Dimensions
General Metric Formulas
Play Attempts
(count)
Play Attempts metric is the summation of total user attempts to initiate video playback. This QoE metric provides a good way to obtain an understanding of an audience’s interaction with a specific video asset title.
Required Data Points
Play Request
Calculation
=(count of Play_Requests) over a time duration
Play Starts
(count)
Sometimes referred to as simply “Plays,” this QoE metric is the total count of Playback Start events. As such, this metric indicates the number of playback experiences which successfully initiated.
Required Data Points
Playback Start
Calculation
=(count of Playback_Start) over a time duration
Active Plays
(count)
Number of video views that are active during a given time period.
Required Data Points
Heartbeat, videoType, viewId
Calculation
=Count the number of unique viewId values for Heartbeat events received during a specified time period.
Notes
For analyzing just content, use videoType=Content.
Video Start Failures
(percentage)
This QoE metric is an important gauge of service performance. Reflected as a percentage, Video Start Failures indicate the total play requests which fail to reach first frame. In other words, this metric compares Play Requests with Playback Starts.
Required Data Points
Play Request, Playback Start
Calculation
=(((sum of Play_Requests) – (sum of Playback Starts)) / (sum of Play_Requests)))*100
Notes
Under ideal circumstances, a value close to 0% is desired.
Exits Before Video Start
(count)
Exit Before Video Start metric computes the percentage of users who exit a video playback experience before the first frame is visible. EBVS is a useful metric for gauging the percentage of viewers discontinuing their playback experience before the first frame commences, thus indicating start times are lasting longer than a user’s interest in remaining in their experience.
Required Data Points
Play Request, Playback Start
Calculation
=(count Play_Request – count Playback Start)
Concurrent Plays
(count)
This metric will show the number of simultaneous video playback sessions currently in progress. It can be used as a representation of counting anonymous unique users.
Required Data Points
session_id
Calculation
=(unique count of session_id)
Video Bitrate
(average)
This metric reflects the mean video bitrate persisting over the course of a playback experience. It is useful for understanding the average data being transferred per second of playback.
Required Data Points
renditionVideoBitrate
Calculation
=((sum of renditionVideoBitrate)/(count of events with renditionVideoBitrate))/100
Notes
This QoE metric is useful for understanding the average connectivity level of the end-user during their experience.
Average is the basic metric for bitrate but the more useful but possibly harder to calculate is tp95 and/or tp99 metrics. Consult your analytics platform to see if they can support these calculations.
Video Startup Time
(average)
This metric reflects the mean time which has elapsed between the user initiating playback by pressing the play button and the commencement of playback, excluding any pre-roll ad.
Required Data Points
Playback Start, timeSinceRequested
Calculation
=(sum of timeSinceRequested for Playback Start event)/count of Playback Start events
Notes
This is an important QoE metric which, under ideal circumstances, should be kept as low as possible. A metric complementary to Average Time to First Frame is Exits Before Video Start (EBVS).
Completion Rates
(percentage)
This metric computes the percentage of users who reached each completion milestone. This is useful for tracking user engagement with individual pieces of content or the entire platform
Required Data Points
Milestone, Playback Start, Playback Complete, milestonePercent, videoType
Calculation
100% Milestone =(count Playback Start - count Playback_Complete)/count Playback Start)*100
other Milestones are available:
n Milestone = count Playback Starts - count Milestones where milestonePercent = n)
Total Time Watched
(average)
This metric reflects the total playback time viewed by users. This KPI is useful for obtaining an understanding of the total time users spent viewing content.
Required Data Points
Heartbeat, playbackDurationContent, timeSinceLastHeartbeat, videoType, viewId
Calculation
=sum (timeSinceLastHeartbeat from event.type=Heartbeat)/1000/60
OR
=sum(max(playbackDurationContent for each unique viewId)/1000/60
Notes
Total Time Watched is a useful example of the versatility of the Data Dictionary as a foundation for metrics. Our team has identified two different generalized formulas which will yield a value for Total Time Watched.
Our team has also identified several items you might exclude from your formula, such as time spent in Pause, Ad playtime, or Rebuffering time.
Total Engagement Time
(average)
Engagement time shows the length of time that a user has spent on a page with the Datazoom SDK watching video or executing custom events.
Required Data Points
engagementDuration, session_id
Calculation
=average of max(engagementDuration) for a session_id
Ad Watched Time
(average)
This metric creates an average duration of time spent watching ads during a series of video views.
Required Data Points
playbackDurationAds
Calculation
=average of MAX(playbackDurationAds) for each viewId
Content Watch Time
(average)
This metric creates an average duration of time spent watching content during a series of video views.
Required Data Points
playbackDurationContent
Calculation
=average of MAX(playbackDurationContent) for each viewId
Notes
In order to increase the accuracy of this metric you may want to exclude on-going Session Views by eliminating any video views that have generated a Heartbeat event within the most current heartbeat interval.
Stall Ratio
(average)
Alternatively called the “Buffer Ratio or Connection Induced Rebuffering,” this metric, reflected as a percentage, compares the amount of time a viewer spends waiting for video to resume after an interruption against time spent watching a playing video. Stall Ratio is useful for understanding the fraction of a user’s playback experience spent loading the video again once playback commenced.
Required Data Points
Stall End, playbackDuration, timeSinceStallStart, viewId
Calculation
=sum(timeSinceStallStart from event.type=Stall End)/sum(max(playbackDuration for each unique viewId))
Another way to view Stall is to create a time-series metric that charts the average time viewers are in a stalled state:
=time series chart with a span=10sec of avg(timeSinceStallStart from event.type=Stall End)
Buffer Ratio
(average)
This metric, reflected as a percentage, compares the amount of time a viewer spends waiting for video to start up or resume after an interruption against total time spent watching a playing video. Buffer Ratio is useful for understanding the fraction of a user’s playback experience spent loading the video. Includes all buffering needed for video start-up, resuming after seek or stalls.
Required Data Points
Stall End, playbackDuration, timeSinceBufferStart, viewId
Calculation
=sum(timeSinceBufferStart from event.type=Buffer End)/sum(max(playbackDuration for each unique viewId))
Another way to view Buffer is to create a time-series metric that charts the average time viewers are in a stalled state:
=time series chart with a span=10sec of avg(timeSinceBufferStart from event.type=Buffer End)
Average Frame Rate
(average)
In some Adaptive Bitrate Streaming setups, using Frame Rate may be a good representation of video quality presented to users.
Required Data Points
frameRate
Calculation
== AVERAGE(frameRate of events that contain frameRate)
Notes
In most cases, this is an outdated metric that has less relevancy in locations where moderate to good internet connections are the standard.
Percent Complete
(average)
Percent complete represents how much of the total video has been viewed by a user.
Required Data Points
playbackDurationContent, duration, viewId
Calculation
=average of (playbackDuration/duration for each viewId, max 100)
Notes
This metric will account for users who skip through the timeline by seeking by using the playbackDurationContent data point instead of just relying on Milestone events.
Playback Stalls
(average)
Average number of times viewers' playback was interrupted by a video stall from rebuffering.
Required Data Points
playbackStallCount, playbackStallCountAds, playbackStallCountContent
Calculation
=Average (playbackStallCount / count of viewIds)
Notes
You can focus in on stalls that occur just during ads or content by using playbackStallCountAds and playbackStallCountContent respectively.
Playback Stall Rate
(average)
The average number of interruptions per minute of viewing time.
Required Data Points
playbackDurationContent, playbackStallCountContent, playbackStallDurationContent
Calculation
=playbackStallCountContent / ((playbackStallDurationContent + playbackDurationContent )/ 1000 / 60) * 100
Playback Stall Ratio
(percentage)
The percentage of time a user spent waiting for a video interruption to resume compared to the total time spent viewing.
Required Data Points
playbackDurationContent, playbackStallDurationContent
Calculation
=playbackStallDurationContent / (playbackStallDurationContent + playbackDurationContent) * 100
Filters & Dimensions
Nearly all of these metrics can be focused into narrower cohorts by filtering data based on key metadata attributes in the Datazoom Data Dictionary. By slicing up metrics you can better identify specific issues that may play a factor in quality of experience issues.
Network
ISP & ASN
An AS is an individual network on the internet. Each network, or AS, has an autonomous system number (ASN). ... An ISP can own multiple AS's. Use this filter to identify if QoE issues are isolated to a specific network or ISP.
Connection Type
The type of network connection currently being used to access the content. WiFi, Mobile, LAN. This dimension is valuable to compare performance quality across different types of internet connections.
Geo Location
Location information is an important dimension for isolating performance data in various geographies to understand usage as well as quality of experience differences.
Available Location Dimensions
City
Country
Country Code
Region
Region Code
Zip Code
Device, Video & Player
When troubleshooting quality issues it may be valuable to isolate aspects of the device, content or application that may be a factor.
Available Device, Video & Player Dimensions
Browser & Browser Version
OS & OS Version
Device ID
Device Type
Asset ID
Content Type
Player Name & Version
Comments
0 comments
Article is closed for comments.