News
 

Bookmark and Share

(13) 

Advanced Micro Devices on Tuesday officially said that it would not endorse SYSmark 2012 benchmark and will also quit BAPCo (Business Applications Performance Corp.) that developers to PC performance measurement software. AMD and, according to some reports, Nvidia Corp. and Via Technologies have disagreements with BAPCo over the benchmark results.

SYSmark 2012 is an application-based benchmark that reflects usage patterns of business users in the areas of office productivity, data/financial analysis, system management, media creation, 3D modeling and web development. Applications used in SYSmark 2012 were selected based on market research and include Microsoft Office, Adobe Creative Suite, Adobe Acrobat, WinZip, Autodesk AutoCAD and 3ds Max, and others. The benchmark is used by many enterprises, PC makers, government organizations and other institutions to determine the performance in business applications and make the right choice of PCs or components.

The main concern of AMD, and presumably other semiconductor companies, is that BAPCo SYSmark 2012 does not utilize graphics processing units (GPUs) for general purpose computing tasks (GPGPU) and solely relies on performance of central processing units (CPUs). According to AMD, such approach is misleading as many applications nowadays take huge advantage of GPGPU technologies, including Adobe Flash 10.2 (SYSmark 2012 uses 10.1), Microsoft Office 11 (SM2012 uses Office 10), Microsoft Internet Explorer 9 (SM2012 uses IE8), Microsoft Movie Maker and many others. Besides, AMD accuses BAPCo of implementing unrepresentative workloads into the benchmark in order to favour AMD's competitor Intel.

"The SYSmark benchmark is not only comprised of unrepresentative workloads (workloads that ignore the importance of heterogeneous computing and, frankly, favor our competitor’s designs), but it actually generates misleading results that can lead to very poor purchasing decisions, causing governments worldwide to historically overspend somewhere in the area of approximately $8 billion," said Nigel Dessau, chief marketing officer of AMD.

BAPCo does not agree with AMD and claims that the sole purpose of the company's demarche is to devaluate SYSmark 2012 in order to prevent the customers from using it to evaluate performance of AMD-powered systems.

"BAPCo is disappointed that a former member of the consortium has chosen once more to violate the confidentiality agreement they signed, in an attempt to dissuade customers from using SYSmark to assess the performance of their systems. BAPCo believes the performance measured in each of the six scenarios in SYSmark 2012, which is based on the research of its membership, fairly reflects the performance that users will see when fully utilizing the included applications," a statement by BAPCo reads.

The company also states that AMD voted in support of over 80% of the SYSmark 2012 development milestones, and were supported by BAPCo in 100% of the SYSmark 2012 proposals they put forward to the consortium.

Scandals about alleged bias of benchmarks happen regularly and mostly due to the fact that certain systems or components cannot get competitive results in those measurement programs. This is by far not the first scandal between BAPCo and AMD over the last ten years. Previously AMD simply accused BAPCo of using Intel-optimized software within the test suite.

At present AMD's microprocessors cannot compete against Intel's chips head to head across a wide variety of applications. BAPCo's SYSmark 2007 shows results that are pretty similar with other benchmarks: Intel is well ahead of AMD. The problem is that SYSMark 2012 will be used for several years to come and software in 2013 will be different from programs 2010 as developers learn how to take advantage of GPU. Naturally, nowadays the majority of programs still rely on CPU, but several years down the road many more applications will use heterogeneous computing model involving both CPU and GPU (including GPUs integrated into Ivy Bridge and Haswell microprocessors).

What AMD and other apologists of heterogeneous model  want is a benchmark that would show the glory of their graphics processing units here and now.

Tags: AMD, BAPCo, SYSmark, Nvidia, Via Technologies, Business

Discussion

Comments currently: 13
Discussion started: 06/22/11 06:53:28 AM
Latest comment: 06/24/11 05:37:49 AM
Expand all threads | Collapse all threads

[1-8]

1. 
Anyone know how Office 11 uses the GPU?
1 0 [Posted by: Daniel Faken  | Date: 06/22/11 06:53:28 AM]
Reply
- collapse thread

 
As I know, the only Office 11 application which uses GPU is PowerPoint. It has use GPU for slides changing effects.
1 1 [Posted by: Tester128  | Date: 06/22/11 01:44:29 PM]
Reply

2. 
couldn't they just agree on a new update at the end of 2012? till then GPGPU is'nt that main stream. but, a year from now things may very well change. unless amd and BAPCo are hiding something (payoffs from intel or a simple attempt to discredit SYSmark ) they are being childish.
2 0 [Posted by: naorai  | Date: 06/22/11 11:22:42 AM]
Reply
- collapse thread

 
the disagreement between sysmark en AMD (and others, including via and nvidia btw) goes back years.
AMD has tried for years to make sysmark benchmarks non-biased but have been completely unsuccessfully.

they are not being childish, the sysmark benchmarks are completely biased. and sysmark's response to evidence showing how biased they are, has been to encrypt their process so its no longer possible to show that just a few tests (favouring Intel) make up about 90% of the total score.
1 2 [Posted by: Countess  | Date: 06/22/11 06:59:22 PM]
Reply

3. 
Fact is that AMD have better GPU unit than Intel.So imagine sysmark that test only CPU part of Fusion (APU).Its well known that many application support AMD APP which work with GPU help and its faster than Intel with help of GPU.
2 4 [Posted by: Blackcode  | Date: 06/22/11 02:32:30 PM]
Reply

4. 
sounds to me like open cl isnt a sysmark strategy could be that open gl and cuda programming arent included in it either.

They should fund a non biased bench program like call it open mark or source mark :D
2 0 [Posted by: verndewd  | Date: 06/22/11 03:04:59 PM]
Reply

5. 
Might want to credit Semiaccurate for this:

http://semiaccurate.com/2...-bapco-over-sysmark-2012/
1 1 [Posted by: RtFusion  | Date: 06/22/11 05:11:26 PM]
Reply

6. 
I'd have to agree with BAPCO/SYSmark 2012 on this one.Most businesses don't upgrade to the newest thing.The company I work at still uses the corporate version of Office 2003.It's a shame but most companies don't always upgrade to the newest software technology because it will cost them a lot of money.
1 0 [Posted by: goury  | Date: 06/22/11 06:53:30 PM]
Reply
- collapse thread

 
show the post
1 4 [Posted by: Countess  | Date: 06/23/11 11:43:53 AM]
Reply
 
I'd have to say I stand with my point as their main reason for quitting is the lack of GPU-accelerated programs in the mix of the generic tests. It would be one thing if AMD flat out said that it was because of something else (in which they have been pretty vocal about it in the past, yet they didn't quit then...)

IMHO, they were just trying to get leverage for their technology, and rightfully so because the performance gain is really good. But as I've stated on my prior post, most companies won't be upgrading "en masse" to these newer version of software that AMD wants to include... so I see why BAPCo didn't include them and instead, used the older programs that most companies are actually using (and would probably still be using for years to come), so why put these newer programs if the companies they are trying to target their product towards won't be taking advantage of it?

I do realize that BAPCo, in a nutshell, is pretty Intel-biased, but for what its worth, their is some skewed validity to their research. Most companies do use software that is optimized for Intel (it's because of that whole compiler fiasco that AMD already won, lawsuit-wise. And companies can even rewrite or revise their code at Intel's expense), would it be fair to add foreign software that most companies might not even have just for the sake of "fair play" for the other competitors? Wouldn't that be more misleading?

Well, since AMD left BAPCo, they can form a new group with Nvidia, VIA and others who don't agree with SYSmark's way of testing and create their own. They can take out some of their marketing funds and invest it towards that since it would be used for marketing anyways. That way they can fully showcase their much touted technology.
0 0 [Posted by: goury  | Date: 06/24/11 05:09:22 AM]
Reply

7. 
show the post
0 3 [Posted by: Blackcode  | Date: 06/23/11 04:30:46 AM]
Reply
- collapse thread

 
so far the only response from bepco to people showing them their benchmarks are biased is to encrypted them so there is no longer any means of determining how the end score was derived at.

so we can only assume their currrent benchmarks are atleast as biased at the ones from the past.
their grant from intel must be substantial it seems.
2 4 [Posted by: Countess  | Date: 06/23/11 11:38:49 AM]
Reply

8. 
Even NVidia and VIA left Bapco.

The fact that AMD voted in support of over 80% of the SYSmark 2012 development milestones doesn't mean anything, if the scoring is imbalanced or misleading.

Is there anyone who really think that VIA left Bapco because SYSMark doesn't show "the glory of their graphics processing units"?
0 0 [Posted by: PeSi  | Date: 06/24/11 05:37:49 AM]
Reply

[1-8]

Add your Comment




Related news

Latest News

Thursday, November 6, 2014

6:48 am | LG’s Unique Ultra-Wide Curved 34” Display Finally Hits the Market. LG 34UC97 Available in the U.S. and the U.K.

Wednesday, October 8, 2014

8:52 pm | Lisa Su Appointed as New CEO of Advanced Micro Devices. Rory Read Steps Down, Lisa Su Becomes New CEO of AMD

Thursday, August 28, 2014

12:22 pm | AMD Has No Plans to Reconsider Recommended Prices of Radeon R9 Graphics Cards. AMD Will Not Lower Recommended Prices of Radeon R9 Graphics Solutions

Wednesday, August 27, 2014

9:09 pm | Samsung Begins to Produce 2.13GHz 64GB DDR4 Memory Modules. Samsung Uses TSV DRAMs for 64GB DDR4 RDIMMs

Tuesday, August 26, 2014

6:41 pm | AMD Quietly Reveals Third Iteration of GCN Architecture with Tonga GPU. AMD Unleashes Radeon R9 285 Graphics Cards, Tonga GPU, GCN 1.2 Architecture