Why Consumer Reports' MacBook Pro battery tests were wrong | Cult of Mac

Why Consumer Reports’ MacBook Pro battery tests were wrong


The new MacBook Pro has great battery life.
Photo: Ste Smith/Cult of Mac

The new MacBook Pro is set to receive a “recommended” rating from Consumer Reports after working with Apple to discover why tests showed the laptops suffered from unreliable battery performance.

The culprits? A hidden Safari setting and “an obscure and intermittent” bug.

It turns out that the testers at Consumer Reports got horrible battery life because they were using a Safari setting that turns off browser cache. The setting is used by web site developers and isn’t normally used by consumers.

Now that a fix has been implemented, Consumer Reports is redoing its tests.

“We have now downloaded the software fix and are rerunning our battery tests with the fix in place on the same computers previously tested,” Consumer Reports wrote in a blog post. “If the battery life results are consistently high, the ratings score for MacBook Pros would rise, and those laptops will then receive Consumer Reports’ Recommended rating given their performance in all our other evaluations.”

Fixing the tests

Consumer Reports originally rated battery life as “highly inconsistent” on the MacBook Pro after test machines got between 3.75 hours and 19.5 hours of battery life on the same machine. It was the first time the well-regarded reviews organization failed to recommend an Apple laptop.

Immediately after Consumer Reports published its test, Apple SVP of Marketing Phil Schiller tweeted that the company would work with the organization to understand the strange battery test results. After looking into the matter, it was discovered that Consumer Reports use of the hidden Safari setting triggered a bug that reloads icons.

The following statement was released by Apple concerning the new findings:

“We appreciate the opportunity to work with Consumer Reports over the holidays to understand their battery test results. We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab. After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life. We have also fixed the bug uncovered in this test. This is the best pro notebook we’ve ever made, we respect Consumer Reports and we’re glad they decided to revisit their findings on the MacBook Pro.”

Cult of Mac's Newsletters

Daily round-ups or a weekly refresher, straight from Cult of Mac to your inbox.

  • Weekender

    The week's best Apple news, reviews and how-tos from Cult of Mac, every Saturday morning. Our readers say: "Thank you guys for always posting cool stuff" -- Vaughn Nevins. "Very informative" -- Kenly Xavier.