Meizu and OnePlus caught cheating on benchmark tests


Meizu and OnePlus caught cheating on benchmark tests

The OnePlus 3, OnePlus 3T, and Meizu Pro 6 are rigged to cheat in benchmarks! Revealed by XDA-Developers, with the assistance from Primate Labs (makers of the popular benchmark suite Geekbench), they discovered that these three phones are programmed to artificially increase their CPU performance when running certain benchmarks.

The software on the smartphones specifically finds for a number of broadly-used benchmarks and commands the CPU to boot into overdrive whenever they are detected.

In the case of the OnePlus 3 and OnePlus 3T, the hard-coded list of benchmarks (e.g: Geekbench, AnTuTu, Androbench, Quadrant, Vellamo, and GFXBench) are the evidence of the cheating which can directly be found in the devices’ firmware.

Technically, the CPU is rigged to idle at 1.29 GHz for the big cores and 0.98 GHz for the small cores, even when there’s no load on the processor. For non-benchmark apps, both the small cores and big cores idle at 0.31 GHz.

Meizu Pro 6 rigged differently, where the big, high-performance cores of the CPU are programmed to activate whenever benchmarks are detected. These big cores don’t kick in when running regular apps. Meizu programmed the CPU to simulate normal functioning when benchmarks are detected.

OnePlus admitted the existence of the benchmark rigging behaviour when confronted by XDA. To its credit, OnePlus acknowledged its mistake and agreed to fix it quickly. XDA-Developers claimed the rigging only appeared when the Oxygen OS (global) and Hydrogen OS (China) development teams were merged, as it was not present in the firmware when the OnePlus 3 launched.

Nonetheless, there is no justification for this deceptive behaviour that ultimately hurts consumers.

It’s neither a new practice nor the first time phone makers are caught red-handed when it comes to benchmarks. In 2013, AnandTech discovered that the Galaxy S4 gamed benchmarks and also surveyed that multiple devices from Asus, HTC, LG, and Samsung were rigged to fool benchmark applications. Some companies consented to change their practices, while others, including Samsung, declined to acknowledge any violation.

In the XDA report: they also examined other smartphones from multiple manufacturers and found some that failed the hidden benchmark test that exposed the OnePlus 3 and Meizu Pro 6.

Pending more analysis, the report did not reveal which devices are suspect, but did reveal the manufacturers that were not caught cheating, namely HTC, Xiaomi, Huawei, Honor, Google, and Sony. Some big names are left out!