Rethinking the IQ test: 2004 J. D. Power initial quality survey consumers understand brands, not sales-weighting

Automotive Industries, July, 2004 by Ray Windecker

J.D. Power conducts a technically sound, quality/satisfaction research program that apparently generates interpretations that are often misunderstood by consumers and some of the general media. One misunderstanding was brought to mind at an automotive-related luncheon when a long-time acquaintance, a detail-minded individual, asked how it was possible that Power rated the Japanese brands as having the best overall quality in the U.S. market when more of the Japanese brands were below Power's own average than were above?

Being unfairly tagged by my friend as a "numbers guy," he wanted an explanation. Not having a Power report in my back pocket, nor having seen the actual report, my comments, based on 60 years of mucking around in the numbers business, was that sales-weighting was probably involved and that if a fleet bought 100,000 vehicles in the exact proportion that they had been sold by the 37 brands listed by Power, the Japanese brands would

But, if 37 individuals bought one of each brand, the domestics would be best. These explanations were, despite my best and more detailed efforts, met by indifference, glazed eyes and near hostility by my table mates.

My friend, echoed by others, insisted that vehicles were bought by brand, not by sales weight. The question was raised, "what would the numbers look like based on the individual vehicle, not the sales weight?" The answer to that question is shown within the accompanying table. One version is statistics, one is real world as seen by the consumer.

However, it was apparent that with the exception of one person who actually looked at and had some interest in the Power data, the others remembered a headline or a few words from some article dealing with the Power report. The belief, nurtured over many years, that Japanese vehicles were all clear winners, while the actuality is, as a later checking of the Power listings verified, is that six Japanese brands were below and only five above Power's own sales-weighted averages.

That particular oddity of more Japanese brands below average than above apparently escaped most media people, as there is no mention of it in the articles that passed across my desk However, top-10 comparisons and discussions are common, often noting that only three domestics (Cadillac, Buick and Mercury) appear in the top ten, but bottom-10 comparisons are rare or perhaps nonexistent. Therefore, a table showing that four Japanese brands (Suzuki, Nissan, Mazda and Scion) inhabit the bottomd0 is included for those who do not read articles and only glance at the tables.

The Power organization cheerfully provided a multi-page, multi-graph press package that was chock flail of overviews and top-three segments lists, but was totally devoid of specific model data, and when asked, just as cheerfully advised that detailed model findings are for subscribing (very rich) clients. They will, however, provide a few bits and pieces, but not a complete listing of all models of all brands.

The model detail would be helpful when deciphering the importance of being HIGHEST RANKED within the 17 segments listed in the Power press release. The unanswered questions is, "what is the sampling error and is there any real difference between the highlighted highest and the small-print second, or third-place, for that matter?"

If one is inquisitive enough, digging around in the data and tying together bits and pieces from the press release can provide some clues. An example is that the plant that builds the two Lincoln car lines has a rating 27 points above average, yet the average total Lincoln brand is three points below average.

The conclusion would be that the Lincoln cars are really excellent and the SUVs are somewhere below average. Also, the plant that builds the Porsche cars is 47 points above average but the brand is 40 points below, forcing the conclusion that the Porsche Cayenne is far worse than the Hummer, the media poster-boy for ineptitude. The helpful Power public affairs people confirmed those concepts, but the consumer, and many in the media, would be clueless.

Power's total brand numbers are no doubt solid, but they are clouded by a considerable amount of consumer misunderstanding, a lack of real detail and shallow media reporting. Think of the fun a numbers guy could have if every bit of data by model and type of complaint were available. Hey, then this article could have been less shallow.

SHALLOW RANKINGS (Vehicles sold in U.S.)

             Sales Weighted       Brand
             Average             Average

Japanese       111               Domestic     123
Korean         117               Japanese     126
European       122               European     128
Domestic       123               Korean       128

Top Ten        Bottom Ten

Japanese       4                 Japanese     4
Domestic       3                 European     3
European       2                 Domestic     2
Korean         1                 Korean       1

Note: All brand data from J.D. Power 2004 IOS. The brand average is
not sales weighted.