A Critical Look: What Is a Test Report Worth?

The answer to this question can range from “useless” to “extremely helpful.” To evaluate a given test, some basic knowledge is necessary. This practical tip is intended to share that knowledge with a wider audience of interested readers. Those who take this into account will be better able to evaluate the claims made in such reports and will also understand the weight to be given to advertising claims.

Whenever a test report or so-called “user review” appears, it’s worth checking the author’s name before reading it. There are several possibilities here. The first is also the simplest: if no author is listed, it’s highly likely to be disguised self-promotion. Such articles aim to give the impression of a test but are actually geared solely toward the author’s own interests. Naturally, they contain only positive statements about the product being described. This raises a reasonable suspicion of self-interest.

In the second scenario, an author is named, but one gets the impression that the review was commissioned. Such reports often contain phrases taken directly from the manufacturer’s brochures or advertisements.

Another possibility is what are known as “courtesy reviews.” For example, a well-known, skilled competition pilot—supported by the manufacturer through free deliveries of helicopters and parts—sings the praises of Brand X. Such reviews or advertisements are, by their very nature, subjective. In this context, the average consumer must ask himself whether he actually needs a helicopter with the same capabilities as those required by a national or other champion pilot to demonstrate his skills. Think about your car. What good is a McLaren or a Formula 1 Lotus to you if you just want to go for a leisurely drive with your family? So anyone who uses arguments drawn from the realm of champions can, by nature, only appeal to those same champions. Therefore, be cautious about applying such glory to us ordinary consumers.

If the limitations described so far do not apply, or

if you already know from past experience that the author of a review also mentions a product’s negative aspects, you still cannot be absolutely certain that the review accurately reflects the actual conditions. To get as close to the truth as possible here, comparisons would need to be made with similar products from other manufacturers. The prices of replacement parts and the service life of individual components should also be compared. To do this requires a significant investment of time as well as a long period of use. If you wait for that, the test is no longer up to date; if you write it beforehand, it is intentionally incomplete.

So what are we to make of all this information, some of which is contradictory? I believe that the author’s name, together with the claims made in the advertisement, provides the picture that offers the greatest possible connection to reality. So if, for example, an advertisement highlights a “super-luxury” solution for a mechanical part, but a test report by a trusted, critical author makes no mention of it, that’s already suspicious. After all, the tester should also notice in practice what is the best, the greatest, in a class of its own, or simply the very best—and then point it out with appropriate emphasis. If they do not, they are behaving much like an employer who, in a reference letter for a departing employee, fails to mention that the employee’s performance did not meet expectations, but instead writes: “He always made an effort to meet the requirements placed on him.” You then have to be able to read between the lines to realize that the actual level of performance isn’t mentioned at all.

With these considerations in mind, you can critically evaluate test reports as well as advertisements from manufacturers or retailers; and if you then also apply the old saying, “Believe in church, know in business (and in daily life),” to the promises in advertisements, then usually only a small remnant remains that conveys genuine information. But this remnant is at least closer to reality than the original, heavily embellished version.