A global agenda: the world gets serious about communication measurement

Communication World, Jan-Feb, 2005 by Richard Gaunt

IABC members played an important role in a recent global survey on communication measurement conducted by Benchpoint, with the help of Donald K. Wright, professor of communication at the University of South Alabama, USA.

PR measurement is a hot topic. Most conference programmes now have a section on measurement, where the latest high-tech, real-time media evaluation tools are rolled out and people debate the best methods.

The Measurement Summit, held annually in New Hampshire, is an international meeting devoted entirely to measurement. To put the ongoing debate into some kind of global context for the 2004 meeting, Benchpoint Ltd. decided to try and find out who is measuring what, which tools are working and which are not.

More than 1,000 communicators in 25 countries responded to an online survey organised by Benchpoint. IABC responded enthusiastically to the call, as did the LI.S. IPR and many national associations in the U.K., Germany, Italy, Poland, Eastern Europe, South Africa and Australia. The influential Arthur W. Page Society and the International Public Relations Association (IPRA) also supported the exercise.

EXECUTIVES PUSH FOR MEASUREMENT

The good news is that nearly 70 percent of the world's professional communicators working in PR say they measure the effectiveness of what they do, and 61 percent go so far as to say that measurement is an integral part of the PR process. There were no significant differences among regions or job functions. Everyone agreed.

A cynical response might be, "Well, they would say that, wouldn't they?" But as we dug deeper into the results, an interesting picture emerged.

The movement toward measurement is being led from the top. Board directors and CEOs are slightly more likely to say, "measurement is part of the PR process" and "we will be doing more measurement in the future" than other employees. In fact, 27 percent of all respondents say they will be doing more measurement in the future, and a massive 77 percent of those who say they do not currently measure, plan to do so in the future.

That still leaves about 22 percent who do not plan to measure anything, and we'll examine their reasons in just a bit.

So what are the measurers measuring?

To get a clearer picture, the survey was split between external communication and internal communication.

EXTERNAL COMMUNICATION

Professional communicators currently measure outputs rather than outcomes, and the majority rate the effectiveness of the tools they use as "uncertain" or only "somewhat effective," which suggests that there is a lot of scope, both for tool refinement and the education of measurers.

* The survey asked, "What criteria do you use to judge if you have been successful?" Respondents answered that achieving the desired outcome, hitting target media and having an effect on awareness and image is more important than being on time/budget, although the scores were close.

* There was a much greater divergence when it came to the actual tools used for measurement. Media evaluation, internal reviews and benchmarking are the most-used tools, although one could argue that these are measuring outputs rather than outcomes. Focus groups and opinion surveys, which actually measure outcomes, got lower scores than AVEs (advertising value equivalents).

* Participants were asked which tools they regarded as most effective. Most tools scored around 44 percent for "uncertain" or "somewhat effective," which suggests that as far as the profession is concerned, the jury is out.

* Opinion surveying is viewed as the most effective tool, with higher scores for "somewhat" and "very" effective.

* Dashboards, league table rankings and AVEs rank as the least effective tools.

* Board level and senior practitioners are more likely to prefer dashboards and AVEs than their junior colleagues.

* Consultants and chief communication officers tended to rate the effectiveness of measurement tools slightly higher than their colleagues, including consultancy board directors.

INTERNAL COMMUNICATION

Internal communicators said that they measure more outcomes than outputs, presumably because it is easier to deploy the tools, but 23 percent still rely on instinct to judge their success.

* Employee feedback is the most-used criterion for success, followed by "desired outcome."

* Nearly 23 percent of respondents use "instinct" to judge whether they have been successful.

* Only 18 percent regard meeting the budget as a criterion for success.

* Employee surveys are the most used and effective tools. Focus groups, benchmarking and dashboards score significantly lower than other tools for internal communication.

* All of the internal measurement tools have higher, "very effective," scores than the external tools.

* North American communicators consistently rate the tools' effectiveness higher than their European counterparts.

* Thirty-one percent of internal communicators correlate the results of their measurements with HR data, and 24 percent correlate with marketing.

WHY DON'T PEOPLE MEASURE?