Letter from the Council
The Council for Research Excellence (CRE) hosted two events since our last newsletter. Our September 20 discussion of best practices in ROI, hosted by CBS's Dave Poltrack, featured a discussion of the CRE ROI Committee's (which Dave chairs) study of "The State of Marketing Mix Modeling." Our October 8 "Local Measurement Mini-Summit," hosted by me, in my capacity as chair of the CRE's Sample Quality Committee, and Raycom's Billy McDowell, in his role as chair of the Local Measurement Committee, offered a discussion of the CRE's recent studies of diary relative error and response bias in audience samples.
Both events were distinguished by lively dialogue and debate, and by an unvarnished presentation of the facts as learned by the CRE studies.
And I think both events strongly demonstrated an important aspect of the CRE's mission: to "reflect the values of transparency, inclusiveness, diversity, imagination, collegiality, practicality and client leadership." Our research, and our committee chairs, didn't pull any punches. We objectively reported, without fear or favor, a number of findings which we believe researchers, media executives and marketers alike need to know to make the best-informed decisions, even though those findings may not have cast industry practices and methodologies in the best of lights. Due credit goes to my CRE colleagues and to Nielsen for undertaking these efforts and advancing understanding by confronting our industry's challenges head-on. You'll read more about these events, and our various recent study findings, in this edition.
We still have much planned for the balance of the year, including greater exploration of the role of Big Data in media measurement. If you or your colleagues have not already done so, please sign up to learn more about the CRE's many activities via any of our regular webinars. Join our LinkedIn group and follow us on Twitter. Also, if you would like to receive future editions of this newsletter, and are not already a subscriber, we invite you to sign up. You can do all these from the CRE homepage.
On a personal note, I will take this opportunity to acknowledge Susan Whiting, who will be retiring from her position as vice chair at Nielsen at the end of this year. I want to thank Susan for her efforts in forming the Council for Research Excellence and her continued support since its inception.
I’d like also to thank once again all the hard-working members of the CRE and its committees, as well as those readers who have participated in our recent events for contributing to the dialogue.
Ceril Shagrin
Executive Vice President, Audience Measurement Innovation and Analytics
Univision Communications, Inc.
Chair, Council for Research Excellence
The CRE's ROI Committee, chaired by David Poltrack, Chief Research Officer at CBS, hosted a presentation and discussion on September 20 to address the latest trends in, and outlook for, marketing mix modeling.
Poltrack was joined by Sequent Partners researchers Jim Spaeth and Alice Sylvester in presenting findings of the CRE whitepaper on The Current State of Marketing Mix Models, reflecting some 50 in-person interviews including leading modeling firms, prominent academics, agencies, and modeling clients at major advertisers recruited with the support of ANA.
Joining the September 20 discussion were guest speakers Gayle Fuguitt, CEO of the Advertising Research Foundation (ARF), Bob Liodice, President & CEO of the Association of National Advertisers (ANA) and Mike Donahue, EVP, Strategic Partnerships, of the American Association of Advertising Agencies (4As). Among other areas, the discussion encompassed current practices and elements of the marketing mix, and the potential role of Big Data
The CRE's objectives in what is an ongoing effort include the development of an in-depth understanding of principal ROI methods used; defining best practices; and recommending priority areas for industry improvement.
Visitors to the CRE site can find the announcement of the whitepaper findings and the ARF AM 8.0 presentation of the findings, as well as a YouTube video, posted to the ROI Committee page, of the full September 20 event.
The CRE's Sample Quality Committee, chaired by Ceril Shagrin, Executive Vice President, Audience Measurement Innovation and Analytics, Univision Communications, Inc., and its Local Measurement Committee, chaired by Billy McDowell, Vice President of Research at Raycom Media, Inc., hosted a "Local Measurement Mini-Summit" on October 8 to present and discuss findings of the respective committees' recent studies. The event was characterized by candid commentary and lively give-and-take, earning praise from a number of guest panelists and attendees for materially advancing the dialogue about an ever-important area of media measurement.
The Local Measurement Committee completed an analysis, with assistance from BIA/Kelsey, of two sets of data provided by Nielsen -- on 10-year trends of relative errors in diary markets, and posted advertising schedules. The Committee evaluated changes in effective sample size and ratings trends over time, and sought to address questions of interest to both buyers and sellers of these data.
In the much larger two-year study of response bias, the Sample Quality Committee, working with the Local Measurement, Return Path Measurement and Universe Estimates Committees, conducted a three-market study encompassing various audience-measurement methods to better understand response bias in address-based audience samples, with the objective of improving diary-based TV-audience measurement and, in turn, ratings quality.
The three television markets were Dallas-Fort Worth, a Nielsen local people-meter (LPM) market; Albuquerque-Santa Fe, a set-meter market; and Paducah, Kentucky-Cape Girardeau, Missouri-Harrisburg, Illinois -- a diary market. The markets were selected due to varying measurement methodology and market characteristics.
Data collected from the study, conducted with the assistance of RTI International, were compared with Nielsen LPM, set-metered and diary market data in an effort to determine, among other details, the impact on response bias of weighting, geographic and demographic variables, and ownership of a traditional TV set as well as a land-line phone or cellphone only.
The studies uncovered persistent issues in widening diary relative error and in non-response, the latter despite Nielsen's more effective use of address-based sampling. The response bias study also provided valuable insights regarding consumer-electronics ownership and usage.
McDowell and Nielsen Chief Methodologist Michael Link also presented a case study on a test conducted at the University of Alabama by Nielsen and Raycom of Nielsen's new "Watcha Watchin'?" app.
Guest speaker Paul Donato, Executive Vice President and Chief Research Officer of Nielsen, addressed the findings of the two studies and some of the issues and concerns raised, indicating Nielsen would take a fresh look at weighting measures and at the increased role of non-traditional viewing further evidenced by the response-bias study.
Donato also noted possible technology solutions: hybrid measurement using set-top box and diary data, mobile measurement, the portable people meter (PPM) device that is now part of the Nielsen Audio unit, the online diary, and the forthcoming Code Reader. "A lot of the alternatives" for local measurement described in the study-findings presentation "we have absolutely dead-center on our radar screens," he said.
"There's a lot of innovation in our future methodology but we need to evolve the way we use the diary data," McDowell observed. "Each market is so different; if you can use the diary as an economical way to 'rank' the market, that would be a win for everybody." He added that qualitative data, enabling marketers to "target based on consumer habits" should be increasingly considered.
"The CRE came to be because of issues with local measurement," Shagrin noted, adding that the two recent CRE studies in effect have brought the CRE "full circle now."
"We as an industry get very enamored with all the new things out there," Shagrin continued. "(But) if you don't have a good foundation, everything crumbles. It's incredibly important that we continue to look at the foundation, and look at the quality of the data that we're using."
The full Local Measurement Mini-Summit presentation can be found here, and a video of the full event is posted to the Sample Quality Committee page of the CRE site.
The CRE's Digital Committee, chaired by Bryon Schafer, Director, Research & Sales Strategy at Hulu, is nearing launch of its previously announced ethnographic study to understand the impacts of internet technology and video platform proliferation on video viewing practices from a cultural perspective.
The study will examine how people talk and think about – and value and, ultimately, use – these technologies in relation to their current television/video viewing practices and attitudes; it will also examine relationships with other technologies such as the DVR.
As an off-shoot of the digital ethnography, the Digital Committee is evaluating an "acceleration" study to examine potential future consumer behavior, to help in identifying technology that may impact the current media landscape
Updates on the project will be provided in future editions of this newsletter.
The CRE welcomes its newest members:
- Cheryl Brink, Vice President, Digital Research & Analytics, Scripps Networks Interactive
- Laura Cowan, Research Director, LIN Media
- Tanya Giles, Executive Vice President, Strategic Insights and Research, Viacom Entertainment Group
- Jed Meyer, U.S. Research Director, Annalect, a unit of Omnicom Media Group
- Bryon Schafer, Director of Research and Sales Strategy, Hulu LLC
- Stacey Schulman, Senior Vice President and Chief Research Officer, Television Bureau of Advertising (TVB)
The CRE encourages audience research executives to apply for
membership – to share views and ideas with fellow research leaders
and to become part of our effort to advance the quality and practice
of audience research. We also invite researchers interested in
our work who simply would like to join one of our Committees. In
either case, please drop us a note at info@researchexcellence.com.
The CRE is pushing ahead with previously announced plans to open two new areas of focus: comparability of cross-platform impressions and pod effects. CRE Media Consumption & Engagement Committee Chair Laura Cowan, research director of LIN Media, is leading the cross-platform impressions effort, while Hadassa Gerber, director of research and systems at the Syndicated Network Television Association (SNTA) is leading an effort to explore the impact of varying commercial pod lengths in different venues.
Additionally, the CRE is planning a December symposium on Non-Probability Sampling. Nielsen clients will receive more information shortly.
Any readers interested in participating with the CRE in the discussion of these topics are invited to email us at info@researchexcellence.com.
Nearly 500 research professionals and others throughout the media and advertising industries are regularly sharing news and viewpoints on the Council for Research Excellence LinkedIn Group.
Join us on Twitter as well – @CREtalk – where we provide news, views and updates on the CRE and all things audience research.
We Want to Hear From You
The CRE seeks to expand the discussion to any and all topics of interest to you. We invite you to submit your suggestions, requests, questions and comments to info@researchexcellence.com
Similarly, please e-mail us if you'd like to get involved with a CRE research committee.
Note to Readers
We hope you enjoyed this edition of the CRE Newsletter. If you would like to receive future editions, and are not already a subscriber, we invite you to sign up at the CRE homepage. If you think a colleague may be interested in receiving future editions of this newsletter, we encourage you to invite them to sign up as well.
Thank you for your interest!
About the Council for Research Excellence
The Council for Research Excellence (CRE) is an independent research group created (in 2005) and funded by Nielsen. CRE is dedicated to advancing the knowledge and practice of audience measurement methodology and comprises senior-level industry researchers representing advertisers, agencies, broadcast networks, cable, syndicators, local stations, and industry associations.
CRE members represent advertising agencies, media-buying firms, media companies, advertisers and industry organizations including ABC, AMC Networks, CBS, Comcast, Cox, Discovery, Disney, ESPN, GroupM, Horizon Media, Hulu LLC, Kimberly-Clark, LIN Media, Magna Global, the Media Rating Council, Media Storm, the National Association of Broadcasters, NBC Universal, News Corporation, Nielsen, Omnicom, Palisades Media Group, Raycom Media, Scripps Networks Interactive, Starcom MediaVest, the Syndicated Network Television Association, TargetCast tcm, the Television Bureau of Advertising, Time Warner, Tribune Co., Twitter, Univision and Viacom.
For more information about the Council for Research Excellence, please visit: http://www.researchexcellence.com/