CONTENTS. BARC: Introduction Relative Error: Foreword Sampling and Data Collection Relative Error... 4

Transcription

1 RELATIVE ERROR

2 CONTENTS BARC: Introduction... 3 Relative Error: Foreword... 3 Sampling and Data Collection... 4 Relative Error... 4 Factors affecting Relative Error... 5 Why is relative error important?... 6 Relative Error for 22K and 33K HH Panel RE levels for genres across TC Total TV RE levels across State Group x TC Summary Appendix... 14

3 BARC: INTRODUCTION brings together the three key Broadcast Audience Research Council of India (BARC) a not for profit body with an equal representation from Indian Broadcasting Foundation (IBF), Indian Society of Advertisers (ISA) and Advertising Agencies Association of India (AAAI). Guided by the recommendations of the TRAI (Telecom Regulatory Authority of India) and MIB (Ministry of Information and Broadcasting) notifications of January 2014, BARC India Stakeholders in television audience measurement - broadcasters, advertisers, and advertising and media agencies, via their apex bodies. The stakeholders are responsible for the design, commission, supervise and own an accurate, reliable and timely television audience measurement system for India. Relative Error: Foreword There is a tendency to look at granular target groups to compute ratings. Ideally, this approach of targeting a focused audience may seem attractive but has its own caveats. The term Relative Error is simply the permissible deviation that would be encountered in any statistical sample. As R.E is primarily driven by the sample size of any Panel and the confidence level used, this can fluctuate sporadically. The R.E for a Hindi GEC would be lower than an English News channel. The audience base for a Hindi GEC would always be higher leading to meaningful insights even at granular cuts. On the contrary for an English News Channel, the same would not hold true. Looking at a wider target group over a longer time-period would yield better insights. For a channel with a low audience base, the point estimate observed for an extremely granular cut would not give the true performance of a channel. Nonetheless, any topical event shows a discernible movement in the audience which can be utilized to derive meaningful insights. For e.g. during events like Exit Polls where the News channels are sampled to a greater extent, it is possible to have a finer demographic cut with a lower R.E as compared to a regular day. Similarly, channels belonging to genres like sports with live telecast or movies with blockbuster airings would show higher consumption among audiences. At BARC India, we recommend looking at estimates that fall under an acceptable limit of R.E the estimate should not be a one-off event derived from a granular cut of demographic, which cannot be reproduced.

4 "Slicing and dicing data can be counter-productive when looking at audience data based on small sample sizes. It can give no meaningful insights neither for the content manager nor for a media planner. This is particularly true for niche channels that don t attract high viewing levels; the data have a higher margin of error. Looking at larger target groups and longer time periods makes the data more robust, and that s the way to go for mature markets - Justin Sampson, CEO, BARB UK SAMPLING AND DATA COLLECTION It is near impossible to collect TV viewing data at census level, the sample based approach taken by BARC is such that it represents the television owning population in India. The sample is rigorously selected to minimize sampling and non-sampling errors with a buffer sample over the design to reduce the impact of errors further. The collection of viewership data is an intensive process, wherein second by second information is polled from panel homes to BARC India servers via Baro-Meters placed in panel homes. The reliability and accuracy in ratings derived from TV panel carries significant importance as it is used across the ecosystem. RELATIVE ERROR The estimates published by BARC are based on a household Panel sample. A representative household sample result will be projected to the entire universe. Though BARC publishes ratings as point estimates, in reality they are interval/range estimates. The range depends on the Relative Error associated with each estimate. Relative error is invoked to prove how reliable or significant an estimate is, bearing in mind this point estimate may be have divergent views variable data is treated as reality from that point estimate. A performance of any channel is not judged based on a single point in time but as a trend. Users tend to forget that there is a high relative error riding behind these estimates. It is therefore necessary to understand the relative error in play across different data points; which is ever present albeit smaller variations are observed when we do not exceed an extreme threshold.

5 As with all TV audience measurement based on panels, the issue of size when analyzing data is an important one of which to be aware. When looking at niche channels or narrow demographics, users should recognize that data volatility and statistical error will be greater than for mass channels or broad demographics. However, such data volatility and statistical error can also occur with mass channels that may experience low viewership at certain times. It is advisable to use longer time periods when running such analyses. - Nick Burfitt, MD APAC, Kantar Media FACTORS AFFECTING RELATIVE ERROR BARC caters to over 550+ channels, where each channel can be categorized in to a genre and language. This mixture coupled with the viewing preferences of individuals which is dynamic and heterogeneous. Channels within genres strive to appeal to their target audience, some carry multiple audio feeds to reach a larger audience base. BARC is transparent in terms of publishing ratings as per the viewing panel, discrepancies or aberrations observed are incidences of actual viewing this is after the panel has been selected by a stringent process to represent the TV universe. These incidences are human behaviour which we know is idiosyncratic at times. Unexpected and unpredictable patterns emerging from a single point in time can be attributed to the very tenet of human behaviour which would never follow the same trend. Let us take a program which has an estimated average daily reach of 15%. And let us assume that Relative error associated with that program is 10% at a 90% confidence interval. This means that there is a 90% chance that the actual estimate lies between 15% ± (15%*10%) i.e. 13.5% to 16.5%.... Z = 1.64 for a 90% confidence level P Q 2 Z ቀ ቀ Relative Error = n P 1... Q = (1 - P) % of the sample NOT viewed that event P = % of the sample viewed that event (Reach%) N = Total Sample TWO FACTORS THAT AFFECT RELATIVE ERROR ARE 1. Sample Size of the TG (n) 2. Reach (P) We can see from the above Relative Error formula that: The Relative Error and Sample Size are inversely proportional to each other. i.e. If the sample size goes down then RE levels will go up and vice versa.

6 The Relative Error and Reach (%) are inversely proportional to each other. i.e. if the P value goes down then RE levels will go up and vice versa. Consider two programs of same TG with Reach 10% for Program-1 and Reach 1% for Program-2. Sample size will be same for both the programs as we are looking at the same TG for both the programs. However, the Relative errors associated with the Program-1 will be lower than Program-2 as the reach (%) for Program-1 is higher than Program-2. Now, consider two programs with same Reach (%) in two different TG s. Let us assume TG-1 sample is higher than TG-2. The Relative error associated with the Program-1 will be lower than Program-2 as the sample for TG- 1 is higher than TG-2. WHY IS RELATIVE ERROR IMPORTANT? Consider a Hindi GEC with reach of 10% at an all India level - average across 28 days has a relative error of 1%. However, the more granular cut on demographics leads to an increase in relative error. As observed below the R.E. for UP/UTK for the channel is at 4%, proceeding with a more minute cut as seen below i.e. 30 min time-band with reach at 3% average across 28 days a R.E. of 88% is observed. On the same lines, R.E for a 30-minute time-band with an all India level reach of 2% - average across 28 days has a relative error of 3%.

7 The Relative Errors associated with Rat % of a Hindi GEC All India - RE - 1 % UP/UTK - RE - 4 % UP/UTK L - RE - 7 % UP/UTK L NCCS A - RE - 10 % UP/UTK L NCCS A Male - RE - 15 % UP/UTK L NCCS A Male RE - 38 % UP/UTK L NCCS A Male 31-40: 30 Min Time - band - RE - 88 %

8 The Relative Errors associated with Rat % of a Hindi GEC program All India - RE - 3 % UP/UTK - RE - 9 % UP/UTK L - RE - 15 % UP/UTK LNCCS A - RE - 23 % UP/UTK L NCCS A Male - RE - 35 % UP/UTK L NCCS A Male RE - 88 % To exemplify the variance in R.E - another example is an English News Channel with reach of 0.6% at an all India level - average across 28 days has a relative error of 6%. However, the more granular cut on demographics leads to an increase in relative error. As observed below the R.E. for UP/UTK for the channel is at 18%, proceeding with a more minute cut as seen below i.e. 30 min time-band with reach at 0.1% average across 28 days a R.E. of 501% is observed. On the same lines, R.E for a 30-minute time-band with an all India level reach of 0.03% - average across 28 days has a relative error of 25%.

9 The Relative Errors associated with Rat % of an English News Channel All India - RE - 6 % UP/UTK - RE - 18 % UP/UTK L - RE - 25 % UP/UTK LNCCS A - RE - 31 % UP/UTK L NCCS A Male - RE - 43 % UP/UTK L NCCS A Male RE % UP/UTK L NCCS A Male 31-40: 30 Min Time - band - RE %

10 The Relative Errors associated with Rat % of an English News Program All India - RE - 25 % UP/UTK - RE - 78 % UP/UTK L - RE % UP/UTK LNCCS A - RE % UP/UTK L NCCS A Male - RE % UP/UTK L NCCS A Male RE % In Singapore we have clear guidelines in place for channels which do not have a significant viewership base in the panel and this is indicated in reports where low samples are present. Broadcasters and agencies need to be very careful when they deep dive into data for niche channels for planning purposes - Lee Risk, Commercial Director APAC, GfK

11 RELATIVE ERROR FOR 22K & 33K HH PANEL ( Total TV All India) RE - 22 K panel and 33K Panel 0.8 % 0.7 % 0.7 % 0.6 % 0.6 % 0.5 % 0.4 % 0.3 % 0.4 % 0.3 % 0.5 % 0.4 % 0.2 % 0.1 % 0.0 % All India U+R All India U All India R 22 K HH Panel 33 K HH Panel *22K Weeks=Avg. Wk19'17-Wk22'17 & 33K Weeks= Avg. Wk02'18-Wk05'18; TG- BARC 2+ Individuals Relative error from 22K HH panel to 33K HH panel for total TV has decreased by 24% at All India, 26% at All India and 19% at All India Rural. RE LEVELS FOR GENRES ACROSS TC Genres U+R Metro 10-75L Below 75L U Below 10L U Rural All India 0.3% 0.6% 0.8% 0.4% 0.5% 0.4% 0.6% Movies 0.5% 1.1% 1.3% 0.7% 0.8% 0.6% 1.0% News 0.7% 1.3% 1.9% 0.9% 1.1% 0.8% 1.3% English Movies 2.7% 4.5% 7.2% 3.8% 4.4% 2.9% 6.6% English News 3.8% 2.3% 9.5% 5.3% 6.3% 4.0% 11.1% GEC 0.40% 0.80% 1.00% 0.50% 0.60% 0.40% 0.70% Music 0.80% 1.80% 2.20% 1.10% 1.30% 0.90% 1.50% Kids 1.00% 1.90% 2.50% 1.40% 1.70% 1.10% 2.50% Sports 1.30% 2.70% 3.30% 1.80% 2.20% 1.50% 2.60% Infotainment 1.80% 3.70% 4.40% 2.40% 2.90% 2.00% 4.10% EEC 5.40% 11.40% 13.30% 7.00% 8.20% 5.90% 12.70% *BARC 2+ Individuals - Wk02'18-Wk05'18 Low RE High RE The R.E associated with the genres give an indicator to the variance expected. Genres can show a dramatic shift in their R. E`s in case of topical events being telecast.

12 TOTAL TV RE LEVELS ACROSS STATE GROUP X TC Region U+R Metro 10-75L Below 75L U Below 10L U Rural All India 0.3% 0.6% 0.8% 0.4% 0.5% 0.4% 0.6% PB/HR/CH/HP/JK 1.5% % 2.5% UP/UTK 1.1% - 2.6% - 1.7% - 1.9% Rajasthan 1.6% % 2.6% DELHI 1.3% 1.3% North East / Sikkim 2.1% % 3.4% Orissa 1.8% % 2.9% West Bengal 1.1% 1.7% - 1.7% % GUJ/DNH 1.2% - 1.7% - 2.2% - 2.9% MH/GOA 0.8% 1.4% 2.1% - 1.3% - 1.6% Andhra Pradesh 0.9% 1.7% - 1.3% % Karnataka 1.0% 1.7% % - 1.7% Kerala 1.6% - 2.6% - 3.2% - 2.9% TN/Pondicherry 0.9% 1.8% - 1.3% % Bihar & Jharkhand 1.5% % 2.5% MP & Chhattisgarh 1.2% - 2.1% - 1.8% - 2.2% *BARC 2+ Individuals - Wk02'18-Wk05'18; Non-reporting High RE and Noncuts have been excluded. Low RE sampled The Relative Error at All India for Total TV ratings is 0.3%. The Relative Error at State Group level for Total TV varies between 0.8% to 2.1% The Relative Errors for Total TV at each of the six metros varies between 1.3% to 1.8% One should not treat Niche channels or genres in the same manner as mass channels/genres. Considering the relative error is increasing with smaller audience rates but is decreasing when cumulated by named days over larger periods, it is advisable to calculate audience ratings for Niche channels/genres cumulated by named days on a 4 week or more data set. - Gwilherm NICOLAS, M.D. Metricline Business Unit, Médiamétrie

13 SUMMARY The decrease in RE observed is due to BARC panel expansion driven by the increase in sample size. There are smaller genres showing a higher variance in RE due to lower incidence of viewing. The increase in RE is substantial if we dissect and look at finer cuts in demographics each cut increasing the magnitude significantly. A demographic cut with a high RE does not indicate the true performance of any channel, a longitudinal trend would be a better indicator. The audience sampling a channel has a bearing on its RE. BARC has striven to increase its panel to have a more robust base. This expansion would in turn lead to a more accurate and true portrayal of a channel. Also, the niche channels would have a higher probability of being sampled. More Reading Materials: afer_and_zhang)/06%3a_sampling_distributions/6.3%3a_the_sample_proportio n

14 APPENDIX VI. 1 GEC GENRE ACROSS INDIAN LANGUAGE: RE LEVELS ACROSS SG x TC Region U+R Metro 10-75L Below 75L Below 10L Rural PB/HR/CH/HP/JK 1.9% % 3.1% UP/UTK 1.4% - 3.2% - 2.0% - 2.3% Rajasthan 2.0% % 3.2% DELHI 1.7% 1.7% North East / Sikkim 3.0% % 4.9% Orissa 2.3% % 3.6% West Bengal 1.3% 2.1% - 2.1% % GUJ/DNH 1.6% - 2.2% - 2.9% - 3.7% MH/GOA 0.9% 1.7% 2.6% - 1.6% - 2.0% Andhra Pradesh 1.0% 2.0% - 1.5% % Karnataka 1.1% 2.0% % - 1.9% Kerala 1.9% - 2.9% - 3.6% - 3.2% Tamilnadu/Pondicherry 1.0% 2.0% - 1.4% % Bihar & Jharkhand 1.9% % 3.0% MP & Chhattisgarh 1.4% - 2.6% - 2.3% - 2.6% All India 0.4% 0.8% 1.0% 0.5% 0.6% 0.4% 0.7% VI. 2 MOVIES GENRE ACROSS INDIAN LANGUAGES: RE LEVELS ACROSS SG X TC Region U+R Metro 10-75L PB/HR/CH/HP/JK UP/UTK Rajasthan DELHI Below 75L Below 10L Rural 2.3% % 3.7% 1.7% - 3.9% - 2.5% - 2.7% 2.5% % 4.1% 2.1% 2.1% North East / Sikkim 4.1% % 6.7% Orissa West Bengal GUJ/DNH MH/GOA Andhra Pradesh Karnataka Kerala Tamilnadu/Pondicherry 3.1% % 5.0% 1.8% 2.3% - 2.9% % 1.9% - 2.7% - 3.4% - 4.3% 1.2% 2.4% 3.4% - 2.0% - 2.4% 1.6% 2.9% - 2.4% % 1.9% 3.6% % - 3.2% 2.9% - 4.3% - 5.7% - 5.2% 1.7% 3.3% - 2.6% %

15 Bihar & Jharkhand 2.3% % 3.6% MP & Chhattisgarh 1.7% - 3.2% - 2.8% - 3.0% All India 0.5% 1.1% 1.3% 0.7% 0.8% 0.6% 1.0% VI. 3 NEWS GENRE ACROSS INDIAN LANGUAGES: RE LEVELS ACROSS SG X TC Region U+R Metro 10-75L Below 75L Below 10L Rural PB/HR/CH/HP/JK 3.0% % 4.7% UP/UTK 2.3% - 5.4% - 3.3% - 3.9% Rajasthan 3.6% % 5.5% DELHI 2.7% 2.7% North East / Sikkim 4.2% % 7.3% Orissa 3.8% % 6.8% West Bengal 2.6% 4.0% - 4.3% % GUJ/DNH 3.0% - 4.4% - 5.5% - 6.5% MH/GOA 1.9% 3.2% 5.3% - 3.3% - 4.2% Andhra Pradesh 1.8% 3.2% - 2.6% % Karnataka 2.0% 3.1% % - 3.5% Kerala 3.5% - 5.5% - 6.6% - 6.2% Tamilnadu/Pondicherry 1.8% 3.4% - 2.7% % Bihar & Jharkhand 3.3% % 5.8% MP & Chhattisgarh 2.8% - 5.3% - 4.4% - 5.0% All India 0.7% 1.3% 1.9% 0.9% 1.1% 0.8% 1.3% VI. 4 MUSIC GENRE: RE LEVELS ACROSS STATE GROUP X TC Region U+R Metro 10-75L Below 75L Below 10L Rural PB/HR/CH/HP/JK 3.5% % 6.0% UP/UTK 2.7% - 6.7% - 4.1% - 4.3%

16 Rajasthan 4.2% % 6.8% DELHI 3.7% 3.7% North East / Sikkim 7.4% % 12.7% Orissa 4.7% % 7.4% West Bengal 3.4% 5.4% - 5.4% % GUJ/DNH 3.5% - 5.1% - 6.1% - 7.5% MH/GOA 2.0% 4.1% 5.4% - 3.2% - 4.2% Andhra Pradesh 2.4% 4.8% - 3.5% % Karnataka 2.4% 4.1% % - 4.1% Kerala 4.5% - 6.9% - 8.8% - 8.0% Tamilnadu/Pondicherry 2.1% 3.6% - 3.2% % Bihar & Jharkhand 4.1% % 6.7% MP & Chhattisgarh 3.0% - 5.6% - 4.7% - 5.2% All India 0.8% 1.8% 2.2% 1.1% 1.3% 0.9% 1.5% VI. 5 KIDS GENRE: RE LEVELS ACROSS STATE GROUP X TC Region U+R Metro 10-75L Below 75L Below 10L Rural PB/HR/CH/HP/JK 4.5% % 9.5% UP/UTK 4.5% - 7.8% - 5.8% % Rajasthan 6.3% % 22.7% DELHI 3.5% 3.5% North East / Sikkim 5.3% % 8.4% Orissa 6.0% % 12.9% West Bengal 3.4% 5.3% - 5.2% % GUJ/DNH 3.7% - 5.1% - 6.7% - 9.7% MH/GOA 2.5% 3.9% 6.7% - 4.5% - 7.1% Andhra Pradesh 3.6% 6.3% - 5.3% % Karnataka 3.5% 5.7% % - 6.5% Kerala 6.5% % % % Tamilnadu/Pondicherry 3.2% 5.5% - 5.1% % Bihar & Jharkhand 5.7% % 14.6% MP & Chhattisgarh 4.1% - 5.8% - 6.0% % All India 1.0% 1.9% 2.5% 1.4% 1.7% 1.1% 2.5%

17 VI. 6 SPORTS GENRE: RE LEVELS ACROSS STATE GROUP X TC Region U+R Metro 10-75L Below 75L Below 10L Rural PB/HR/CH/HP/JK 6.0% % 10.7% UP/UTK 4.6% - 9.7% - 7.1% - 7.6% Rajasthan 6.5% % 10.8% DELHI 5.6% 5.6% North East / Sikkim 7.4% % 12.1% Orissa 7.9% % 14.2% West Bengal 4.7% 7.1% - 7.2% % GUJ/DNH 5.0% - 7.3% - 9.0% % MH/GOA 3.2% 5.4% 9.5% - 5.7% - 6.6% Andhra Pradesh 4.7% 6.6% - 7.9% % Karnataka 4.3% 7.4% % - 8.9% Kerala 7.3% % % % Tamilnadu/Pondicherry 4.2% 8.8% - 6.1% % Bihar & Jharkhand 6.6% % 12.1% MP & Chhattisgarh 4.7% - 8.3% - 7.6% - 8.5% All India 1.3% 2.7% 3.3% 1.8% 2.2% 1.5% 2.6% VI. 7 INFOTAINMENT GENRE: RE LEVELS ACROSS STATE GROUP X TC Region U+R Metro 10-75L Below 75L Below 10L Rural PB/HR/CH/HP/JK 8.2% % 15.1% UP/UTK 7.5% % % % Rajasthan 9.9% % 19.3% DELHI 7.0% 7.0% North East / Sikkim 9.3% % 15.7% Orissa 11.2% % 22.9% West Bengal 6.3% 9.4% % % GUJ/DNH 6.7% - 9.8% % % MH/GOA 4.3% 7.1% 10.8% - 7.6% %

18 Andhra Pradesh 6.8% 13.5% - 9.6% % Karnataka 7.4% 11.7% % % Kerala 13.3% % % % TN/Pondicherry 5.0% 10.0% - 6.8% % Bihar & Jharkhand 11.4% % 24.6% MP & Chhattisgarh 6.7% % % % All India 1.8% 3.7% 4.4% 2.4% 2.9% 2.0% 4.1% VI. 8 EEC GENRE: RE LEVELS ACROSS STATE GROUP X TC Region U+R Metro 10-75L Below 75L Below 10L Rural PB/HR/CH/HP/JK 24% % 47% UP/UTK 20% - 36% - 26% - 73% Rajasthan 34% % 189% DELHI 23% 23% North East / Sikkim 20% % 33% Orissa 38% % 83% West Bengal 21% 39% - 29% % GUJ/DNH 20% - 32% - 30% - 46% MH/GOA 13% 22% 37% - 22% - 34% Andhra Pradesh 19% 36% - 27% % Karnataka 17% 39% % - 32% Kerala 24% - 37% - 46% - 43% Tamilnadu/Pondicherry 17% 25% - 29% % Bihar & Jharkhand 37% % 119% MP & Chhattisgarh 22% - 31% - 34% - 129% All India 5.4% 11.4% 13.3% 7.0% 8.2% 5.9% 12.7% VI. 9 ENGLISH MOVIES GENRE: RE LEVELS ACROSS STATE GROUP X TC Region U+R Metro 10-75L Below 75L Below 10L Rural

19 PB/HR/CH/HP/JK 18% % 36% UP/UTK 15% - 23% - 21% - 63% Rajasthan 23% % 92% DELHI 14% 14% North East / Sikkim 11% % 19% Orissa 19% % 42% West Bengal 9% 13% - 16% % GUJ/DNH 13% - 20% - 22% - 30% MH/GOA 7% 10% 23% - 14% - 22% Andhra Pradesh 8% 13% - 11% % Karnataka 7% 11% % - 17% Kerala 11% - 16% - 22% - 21% Tamilnadu/Pondicherry 6% 8% - 9% % Bihar & Jharkhand 22% % 76% MP & Chhattisgarh 14% - 20% - 21% - 66% All India 2.7% 4.5% 7.2% 3.8% 4.4% 2.9% 6.6% VI. 10 ENGLISH NEWS GENRE: RE LEVELS ACROSS STATE GROUP X TC Region U+R Metro 10-75L Below 75L Below 10L Rural PB/HR/CH/HP/JK 17% % 35% UP/UTK 16% - 22% - 24% - 113% Rajasthan 22% % 104% DELHI 14% 14% North East / Sikkim 24% % 41% Orissa 36% % 113% West Bengal 10% 16% - 13% % GUJ/DNH 23% - 32% - 40% - 61% MH/GOA 11% 15% 32% - 26% - 36% Andhra Pradesh 22% 35% - 32% % Karnataka 9% 12% % - 23% Kerala 22% - 25% - 63% - 55% Tamilnadu/Pondicherry 11% 15% - 17% % Bihar & Jharkhand 20% % 57% MP & Chhattisgarh 17% - 22% - 26% - 135% All India 3.8% 2.3% 9.5% 5.3% 6.3% 4.0% 11.1%

20