Here, we’ll review KPIs you can build in most analytics systems based on Datazoom’s Data Dictionary. We’ll provide you with some generalized formulas, with Data Dictionary terms bolded.
You can then translate these expressions into the specific querying language of the tool you’re using, or another format your system of choice may require. Keep in mind that these formulas are sample starting points. They are by no means the “end all and be all” of formulas. Part of the beauty of the Data Dictionary is its role as a springboard for customizing metrics in a fashion which best suits your organization.
However, please note that similar to how players expose data points by different names, each player exposes different data points entirely. Thus, some of the metrics below may not be supported across players. To check if the players your team relies on are supported, check out our documentation here.
Index
- General Metric Formulas
- Play Attempts
- Play Starts
- Active Plays
- Video Start Failures
- Exits Before Video Start
- Concurrent Plays
- Video Bitrate
- Video Startup Time
- Completion Rates
- Total Time Watched
- Total Engagement Time
- Ad Watched Time
- Content Watch Time
- Stall Ratio
- Buffer Ratio
- Percent Complete
- Playback Stalls
- Playback Stall Rate
- Playback Stall Ratio
- Filters & Dimensions
General Metric Formulas
Play Attempts
(count)
Play Attempts metric is the summation of total user attempts to initiate video playback. This QoE metric provides a good way to obtain an understanding of an audience’s interaction with a specific video asset title.
Required Data Points
Media Request
Calculation
=(count of media_request) over a time duration
Play Starts
(count)
Sometimes referred to as simply “Plays,” this QoE metric is the total count of Playback Start events. As such, this metric indicates the number of playback experiences which successfully initiated.
Required Data Points
Playback Start
Calculation
=(count of playback_start) over a time duration
Active Plays
(count)
Number of video views that are active during a given time period.
Required Data Points
media_type, content_session_id
Calculation
=Count the number of unique content_session_id values for all events received during a specified time period.
Notes
For analyzing just content, use media_type=Content.
Video Start Failures
(percentage)
This QoE metric is an important gauge of service performance. Reflected as a percentage, Video Start Failures indicate the total play requests which fail to reach first frame. In other words, this metric compares Media Requests with Playback Starts.
Required Data Points
Media Request, Playback Start
Calculation
=(((sum of media_request) – (sum of playback start)) / (sum of media_request)))*100
Notes
Under ideal circumstances, a value close to 0% is desired. For analyzing just content, use media_type=Content.
Exits Before Video Start
(count)
Exit Before Video Start metric computes the percentage of users who exit a video playback experience before the first frame is visible. EBVS is a useful metric for gauging the percentage of viewers discontinuing their playback experience before the first frame commences, thus indicating start times are lasting longer than a user’s interest in remaining in their experience.
Required Data Points
Media Request, Playback Start
Calculation
=(count media_request – count playback_start)
Concurrent Plays
(count)
This metric will show the number of simultaneous video playback sessions currently in progress. It can be used as a representation of counting anonymous unique users.
Required Data Points
content_session_id
Calculation
=(unique count of content_session_id)
Video Bitrate
(average)
This metric reflects the mean video bitrate persisting over the course of a playback experience. It is useful for understanding the average data being transferred per second of playback.
Required Data Points
rendition_video_bitrate_kbps
Calculation
=((sum of rendition_video_bitrate_kbps)/(count of events with rendition_video_bitrate_kbps))/100
Notes
This QoE metric is useful for understanding the average connectivity level of the end-user during their experience.
Average is the basic metric for bitrate but the more useful but possibly harder to calculate is p95 and/or p99 metrics. Consult your analytics platform to see if they can support these calculations.
Video Startup Time
(average)
This metric reflects the mean time which has elapsed between the user initiating playback by pressing the play button and the commencement of playback, excluding any pre-roll ad.
Required Data Points
Playback Start, startup_duration_content_ms
Calculation
=average of startup_duration_content_ms for playback_start event
Notes
This is an important QoE metric which, under ideal circumstances, should be kept as low as possible. A metric complementary to Average Time to First Frame is Exits Before Video Start (EBVS).
Completion Rates
(percentage)
This metric computes the percentage of users who reached each completion milestone. This is useful for tracking user engagement with individual pieces of content or the entire platform
Required Data Points
Milestone, Playback Start, Playback Complete, milestone_percent, media_type
Calculation
100% Milestone =(count playback_start - count playback_complete)/count playback_start )*100
other Milestones are available:
n Milestone = count playback_start - count milestone where milestone_percent = n)
Total Time Watched
(average)
This metric reflects the total playback time viewed by users. This KPI is useful for obtaining an understanding of the total time users spent viewing content.
Required Data Points
Heartbeat, playback_duration_content_ms, time_since_last_heartbeat_ms, media_type, content_session_id
Calculation
=sum (time_since_last_heartbeat_ms from event.type=heartbeat)/1000/60
OR
=sum(max(playback_duration_content_ms for each unique content_session_id)/1000/60
Notes
Total Time Watched is a useful example of the versatility of the Data Dictionary as a foundation for metrics. Our team has identified two different generalized formulas which will yield a value for Total Time Watched.
Our team has also identified several items you might exclude from your formula, such as time spent in Pause, Ad playtime, or Rebuffering time.
Total Engagement Time
(average)
Engagement time shows the length of time that a user has spent on a page with the Datazoom SDK watching video or executing custom events.
Required Data Points
engagement_duration_ms, app_session_id
Calculation
=average of max(engagement_duration_ms) for a app_session_id
Ad Watched Time
(average)
This metric creates an average duration of time spent watching ads during a series of video views.
Required Data Points
playback_duration_ads_ms
Calculation
=average of MAX(playback_duration_ads_ms) for each content_session_id
Content Watch Time
(average)
This metric creates an average duration of time spent watching content during a series of video views.
Required Data Points
playback_duration_content_ms
Calculation
=average of MAX(playback_duration_content_ms) for each content_session_id
Notes
In order to increase the accuracy of this metric you may want to exclude on-going Session Views by eliminating any video views that have generated a Heartbeat event within the most current heartbeat interval.
Stall Ratio
(average)
Alternatively called the “Buffer Ratio or Connection Induced Rebuffering,” this metric, reflected as a percentage, compares the amount of time a viewer spends waiting for video to resume after an interruption against time spent watching a playing video. Stall Ratio is useful for understanding the fraction of a user’s playback experience spent loading the video again once playback commenced.
Required Data Points
Stall End, playback_duration_content_ms, time_since_last_stall_start_content_ms, content_session_id
Calculation
=sum(time_since_last_stall_start_content_ms from event.type=stall_end)/sum(max(playback_duration_content_ms for each unique content_session_id))
Another way to view Stall is to create a time-series metric that charts the average time viewers are in a stalled state:
=time series chart with a span=10sec of avg(time_since_last_stall_start_content_ms from event.type=stall_end)
Buffer Ratio
(average)
This metric, reflected as a percentage, compares the amount of time a viewer spends waiting for video to start up or resume after an interruption against total time spent watching a playing video. Buffer Ratio is useful for understanding the fraction of a user’s playback experience spent loading the video. Includes all buffering needed for video start-up, resuming after seek or stalls.
Required Data Points
Stall End, playback_duration_content_ms, time_since_last_buffer_start_content_ms, content_session_id
Calculation
=sum(time_since_last_buffer_start_content_ms from event.type=buffer_end)/sum(max(playback_duration_content_ms for each unique content_session_id))
Another way to view Buffer is to create a time-series metric that charts the average time viewers are in a stalled state:
=time series chart with a span=10sec of avg(time_since_last_buffer_start_content_ms from event.type=buffer_end)
Average Frame Rate
(average)
In some Adaptive Bitrate Streaming setups, using Frame Rate may be a good representation of video quality presented to users.
Required Data Points
frame_rate
Calculation
== AVERAGE(frame_rate of events that contain frame_rate)
Notes
In most cases, this is an outdated metric that has less relevancy in locations where moderate to good internet connections are the standard.
Percent Complete
(average)
Percent complete represents how much of the total video has been viewed by a user.
Required Data Points
playback_duration_content_ms, duration_sec, session_view_id
Calculation
=average of (playback_duration_content_ms/duration_sec for each content_session_id, max 100)
Notes
This metric will account for users who skip through the timeline by seeking by using the playback_duration_content_ms data point instead of just relying on Milestone events.
Playback Stalls
(average)
Average number of times viewers' playback was interrupted by a video stall from rebuffering.
Required Data Points
stall_count_content, session_view_id
Calculation
=Average (stall_count_content / count of content_session_id)
Notes
You can focus in on stalls that occur just during ads or content by using stall_count_ads and stall_count_content respectively.
Playback Stall Rate
(average)
The average number of interruptions per minute of viewing time.
Required Data Points
playback_duration_content_ms, stall_count_content, stall_duration_content_ms
Calculation
=stall_count_content / ((stall_duration_content_ms + playback_duration_content_ms )/ 1000 / 60) * 100
Playback Stall Ratio
(percentage)
The percentage of time a user spent waiting for a video interruption to resume compared to the total time spent viewing.
Required Data Points
playback_duration_content_ms, stall_duration_content_ms
Calculation
=stall_duration_content_ms / (stall_duration_content_ms + playback_duration_content_ms) * 100
Filters & Dimensions
Nearly all of these metrics can be focused into narrower cohorts by filtering data based on key metadata attributes in the Datazoom Data Dictionary. By slicing up metrics you can better identify specific issues that may play a factor in quality of experience issues.
Network
ISP & ASN
An AS is an individual network on the internet. Each network, or AS, has an autonomous system number (ASN). ... An ISP can own multiple AS's. Use this filter to identify if QoE issues are isolated to a specific network or ISP.
Connection Type
The type of network connection currently being used to access the content. WiFi, Mobile, LAN. This dimension is valuable to compare performance quality across different types of internet connections.
Geo Location
Location information is an important dimension for isolating performance data in various geographies to understand usage as well as quality of experience differences.
Available Location Dimensions
City
Country
Country Code
Region
Region Code
Zip Code
Device, Video & Player
When troubleshooting quality issues it may be valuable to isolate aspects of the device, content or application that may be a factor.
Available Device, Video & Player Dimensions
Browser & Browser Version
OS & OS Version
Device ID
Device Type
Asset ID
Content Type
Player Name & Version
Comments
0 comments
Article is closed for comments.