Pixel 2 XL Highlights a Crisis of Reviewer Confidence


Just how long does it take to review a phone?

That is a question that needs to be asked by consumers and tech readers all over the world, following yet another case of a hyped device seemingly coming up short. Yes, the jury is still out on the Pixel 2 XL’s display issues. But the stark fact is that they seemed to have been totally missed by a number of reviewers who seemed more obsessed with getting their reviews online as soon as possible after the device was released. Yes, a few have revised their reviews (some have withdrawn them), but the big issue is that every time something like this happens, the gadget review process and its practitioners lose perhaps their most important possession:


This reviewer volte-face (or red face) is not a one-off issue but one that seems to occur time and again in the tech media, ourselves included. Remember the ecstatic reviews of the Samsung Galaxy Note 7 last year before it literally exploded into action? Or let us just go back a few months and see the stirringly positive reviews people bestowed on the OnePlus 5, after which the company itself (to its immense credit) issued a bevy of updates to fix a number of issues and bugs, that had been surprisingly missed by these same reviewers. And this is not a new phenomenon – as far back as in the case of the Lumia 920, we had reviewers hailing its camera as one of the best only for the company to issue an update that claimed to fix camera performance – the same performance that many reviewers had been hailing as being the best ever.


Indeed, far too often, especially in the case of high-profile devices, we see reviewers (once again, we include ourselves in this list) increasingly seeming to sacrifice sustained experience on the altar of speed. The issue, in this age of super competitive tech portals, seems to be to get one’s review out there on the Web as soon as possible, preferably before one’s competitors do. Which is all very well if this did not in any way compromise with the whole review process. But then, they did not say “haste makes waste” because they were simply feeling lazy. Reviewing products often takes time for the simple reason that most products change their behavior over time – they might slow down, crash in certain circumstances, take more time to start up and shut down, and so on.

The brutal truth is: while you CAN review a phone or notebooks in three days, your review is likely to be a shockingly limited one. There are several things to be experienced in a device and some become evident only over a period of time – this was most visible in the case of Nokia’s Symbian devices when some of my editors used to warn me against submitting a review until I had got at least two hundred messages in my inbox, as these often slowed down the device. The same was in the case of photo galleries in many Samsung phones – they would sometimes end up slowing down the device. And then, of course, there were Sony’s Vaio notebooks whose battery life sometimes used to start dipping after a few weeks.

Now, all these issues do make a difference to the consumer and the reader. Most of us might switch to the next one that comes in for review, but a consumer is likely to stick with a device for months if not years and any information that they get on its long-term performance is valuable. More valuable, I would say, than the speed at which the review is put up. A review that is written quickly reflects more on its author’s ability to conjure text than their ability to actually evaluate a device. This might offend some colleagues, but companies take weeks to test devices – how on earth can someone claim to review them comprehensively in a matter of days?

(adsbygoogle = window.adsbygoogle || []).push({});

So what is the answer, some might ask? One cannot take weeks to review a product when readers are seeking information about it from the very day of its release. True enough. Some companies are efficient enough to deliver review units a few weeks in advance of their market launches, but these are the exceptions rather than the rule. On the flip side, however, does the fact the readers need information fast justify giving them reviews that are based on very limited testing and could be inaccurate? Should you even – EVER – consider misleading your reader, that very person who comes to your site, depending on your expertise or knowledge? A “hands-on review” or a rushed review based on a few days of usage (often with a prototype unit and/or beta software) is an insult to this faith and trust.

The stark fact is that reviewers today (and again I stress, this includes us) need to figure out what is more important – speed to get online and get picked up by search engines, or information to enhance credibility and perhaps long-term readership. There is perhaps a golden mean here – a period in which you can reasonably evaluate a product while consumers are still thinking about it. Of course, this is different for different people and sites.

And while doing this is important, even more so is the need to figure out who is our target – the search engines or the reader needing credible information? I do not think I am qualified to give an answer, but I would like to end by paraphrasing one of India’s best-known journalists, Arun Shourie in a lecture to us when we were studying journalism:

People pay to read what you have written.
Never forget that.
People pay to read it.
Do not lie to them.
Do not try to mislead them.
That person who is buying your newspaper (or accessing your website) is trusting you.
He or she is giving you
Their hard-earned money.
Do not make them regret it.
Repay their faith.
In words.
Honest words.Arun Shourie

Speed is an asset, I know, but credibility?
Credibility is surely not a liability, is it?


Please enter your comment!
Please enter your name here