SPEC invalidates thousands of benchmark results — multiple Intel Xeon CPUs affected, are they not as fast as they seem?
Has Intel been cheating its benchmark results?
When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.
The Standard Performance Evaluation Corporation (SPEC), which tasks itself with maintaining standardized benchmarks for computing components, has invalidated more than 2,600 results forIntelXeon chips.
The non-profit consortium revealed that it would no longer publish SPEC CPU 2017 results for Intel PCs running a specific version of the Intel compiler.
The decision stems from concerns over what the consortium describes as a targeted optimization for a particular workload. In other words, SPEC is accusing Intel of tweaking its products to obtain better benchmark results.
2,600 Intel benchmarks invalidated
SPEC CPU 2017 is a widely used benchmark for high-end servers, data centers and workstations, testing performance across a total of 43 benchmarks. The controversy relates to the Intel oneAPI DPC++/C++ Compiler, which has been allegedly optimized for the 523.xalancbmk_r / 623.xalancbmk_s benchmarks.
A disclaimer is now attached to more than 2,600 results: “The compiler used for this result was performing a compilation that specifically improves the performance of the 523.xalancbmk_r / 623.xalancbmk_s benchmarks using a priori knowledge,” thus invalidating the results.
Although benchmark figures are generally non-reflective of real-world conditions, testing is designed to be as life-like as possible, so optimizations designed to cater to specific benchmarks can result in skewed (and unrealistic) results.
According to aPhoronixreport, 2022.0 to 2023.0 models are affected, leaving models either side of this date range unaffected by the disclaimer notices. The article continues: “The SPEC CPU specific optimization could result in a 9% overall SPECint speed uplift and around 4% for the SPECint rate.”
Are you a pro? Subscribe to our newsletter
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
Intel told us in a statement: “We respect the SPEC committee decision and are focused on delivering real world workload performance for our customers.”
TechRadar Prohas reached out to SPEC for any further information, but we did not receive an immediate response.
More from TechRadar Pro
With several years’ experience freelancing in tech and automotive circles, Craig’s specific interests lie in technology that is designed to better our lives, including AI and ML, productivity aids, and smart fitness. He is also passionate about cars and the decarbonisation of personal transportation. As an avid bargain-hunter, you can be sure that any deal Craig finds is top value!
US government agency warns workers of possible Chinese cellphone hacks
AI and robotics really could be holding back some workers from their dream career
‘That was never the plan’: Arcane creators dismiss claims that the hit Netflix show was going to run for five seasons
