Monday, December 15, 2008

Ratings: What is meaningful variation?

Soap Opera Network has started an interesting new feature labelled "Daytime's 40 Most Popular Shows" The first week ratings analysis, for the week of 12/01/2008, were posted today.

Writer and site manager Errol says: "The mindset behind the new chart was to compare episodes on a week to week basis, instead of just a weekly average. This is how Cable ratings are compared, not as averages. For instance, Bill O'Reilly's Wednesday episode can rank #1 in the Nielsen's, but it wouldn't if it were averaged for the whole week."

And, indeed, his accompanying ratings analysis makes a point of showing how show rankings can vary across the week. For example:

While NBC's "Days of our Lives" loses 40% of its budget in the upcoming year and a loss of its two stars (Deidre Hall and Drake Hogestyn), three out of its five weekly episodes (Friday, Tuesday, and Monday) outranked every episode of ABC's "All My Children" and "One Life to Live." The highest episode was its Friday broadcast with 2.9 million viewers. For the week, "DAYS" averaged 2.7 million viewers.
So, I'm interested by Errol's basic claim that people are interested in the dailies. I know he is right. SON's weekly ratings thread is among its most active. Every week, posters try to link day-to-day variations in the ratings to stories that were happening, characters that appeared, or particular promotions that were run.

I'm not so sure this sheds much light, for the most part. For example, I took the data Errol charted and plotted in here (click for full-size versions):

Is this meaningful fluctuation, or random variation? For example, there is a general trend for mid-week to be slumpy, and for many shows to start the week stronger than they finish. When you look at it in the millions of viewers (top chart), shows can lose 10% or more of their audience in a given week.

What I am struggling with is whether this is meaningful. I don't really think so. Take Errol's analysis about Days of Our Lives above, for example. Although the ranking may vary over the course of week, I question whether these ranking fluctuations are statistically meaningful. To me, it basically says the shows are neck-in-neck. Do advertisers care about these fluctuations? I suspect they should not.

As I said to Errol, I think a longer-time frame is the appropriate unit of analysis:

Even weeklies, I would argue, are not often meaningful barometers of a show's performance. The most useful number, I would argue, is the SLOPE (unit change over time). Perhaps even more useful is the STANDARDIZED SLOPE (which puts all shows in a common metric before examining their trajectory). That implies the gathering of these ratings over some period of time.
The sole exception, in my opinion, is when the dailies are being used to check on the effectiveness of a particular promotion. I, for example, was glued to the ratings after CBS' heavy promotion of Y&R: Sudden Impact (and, to be fair, a year earlier, Y&R: Out of the Ashes). Sudden Impact got on-network promotion, off-network promotion (E!), and banner ads in lots of places. In addition, the soap press was breathlessly promoting the "big changes" that would come in August. Sure enough, at least in the short term, the dailies responded...Y&R got a bump.

Interestingly, even here I question the value of the dailies. Because the Y&R Sudden Impact bump wore off...and it returned to fluctuating about its regular and lower average.

I have a vague sense that over-focus on the dailies and weeklies may actually be destructive, because it promotes stunt plotting and stunt promotion. (In primetime, the weekly ratings make more sense...because shows have fewer episodes). The barometer of a show's health, I think, is where it is trending.

In that sense, I am actually encouraged by what Barbara Bloom is quoted as saying this week: “It's not just about stopping the erosion. It’s still possible to raise the ratings.”

I am encouraged by the fact that, to me, this suggests Bloom is more focused on trend than snapshot performance. On the other hand, I see no evidence in any quarter that ratings are improvable over the short- or long-haul. I'd love to know what she is thinking, and how she plans to get there, and whether it can work.

No comments: