by Doors4ever , Ilya Gavrichenkov
10/18/2007 | 08:06 AM
The problem of choice that each of us inevitably faces one day is very closely connected with another problem: the comparison. The decision is especially tough when it comes to computers, because there are processors with different architectures, different classes of graphics cards, memory of different types, mainboards and storage solutions with different interfaces in the today’s market. Unfortunately, the technical specifications can not always give you a definite answer. And even if you already have an idea of what your future system is going to be like, there still remain a lot of more specific but not any less important questions to answer. What is more important: faster processor or faster graphics cards? What is faster: large memory capacity with slower memory chips or small memory capacity with faster memory chips? Will one high-capacity hard disk drive be enough or it would be better to have a few smaller-capacity HDDs in a RAID array?
There is only one way-out here: to judge by the benchmark results. But the situation here is also not that simple at all, because the results may differ dramatically depending on the benchmarks you pick. We definitely need a complex system test, but this is where the problem emerges: there are no reliable complex benchmarking suites yet. We had to turn to a widely spread Futuremark PCMark05 suite, but all of you are probably aware of its drawbacks. However, Futuremark Company didn’t waste their time all these years and today they are proud to announce their new complex system performance benchmark that even received a proprietary name instead of a numeric version index. It is called PCMark Vantage .
Our today’s article will be devoted to the peculiarities of the new benchmarking suite, installation and usage techniques.
If you have broadband internet connection you can download the 700MB PCMark Vantage installation file. However, before you start, make sure that your system meets the minimum system requirements, which look as follows:
The installation process is very simple. All you need to do is to launch the installation file and accept the terms of the PCMark Vantage and Microsoft DirectX license agreement. By default you will be prompted to use Express-install, which doesn’t require any action on your part. By selecting “Custom Install” you can change the location and some secondary options such as creation of a desktop icon, display of the readme file, etc.
Upon first program launch after successful installation of PCMark Vantage you will need to register your copy by entering the license number in the corresponding field.
Non-commercial and home users get the opportunity to use a trial version of PCMark Vantage without registration, but in this case you will need to obtain a trial key. This is also fairly easy: just get online, enter your email address and receive your registration key along with the two code words that need to be typed into the corresponding fields for security purposes (to prevent and automatic registrations).
The PCMark Vantage Basic Edition Trial version has a few significant limitations: a trial code can only be sent once to each email address, the tests can be performed only one time , however, obtained results can be stored online.
PCMark Vantage Basic Edition is a more functional and convenient version. There are a few uncertainties about its pricing: one the site – as you could have seen on the screenshot above – the price is set at $4.95; in the user’s manual – at $6.95. By paying $5-$7 USD you can launch PCMark Test Suite unlimited number of times, but will only have limited on-line and off-line functionality: you will only be able to view the obtained results online.
PCMark Vantage Advanced Edition version allows using not only the basic set of tests, but also all other tests and will offer you full access to test settings and complete on-line functionality. It sells for $19.95. All of the above mentioned PCMark versions are available only online and are intended strictly for non-commercial and home use.
By purchasing PCMark Vantage Professional Edition for $495 you acquire a CD disk with the complete benchmarking suite, technical support, opportunity to view your test results off-line, use of command prompt, export of test results in Microsoft Excel and the right for commercial use.
Now that we have dealt with all the formalities, we cannot wait to get down to actual testing. However, in order to get accurate and repeatable benchmark results, you should follow the requirements and recommendations below:
The main program window with a broad promo banner at the bottom looks as follows:
I have to admit that the new benchmarking suite became much more commercial: you see ad banners in the program windows, in the actual tests and even in the score reports.
PCMark Vantage Advanced and Professional allow you to change the settings from defaults to something else.
These versions also allow you to select all tests you wish to run. Each test is provided with a brief description for your convenience. The full testing cycle takes about one and a half hours.
Now let’s take a closer look at the available testing suites.
The basic testing suite that is available in all PCMark Vantage versions is called PCMark Suite. Only this testing suite will provide a PCMark Score. This suite includes the following individual tests:
All tests included into PCMark Suite are split into several groups:
Besides the basic PCMark Suite, PCMark Vantage Advanced and Professional also allow to perform some more in-depth system check with the help of the following special benchmarks:
Memories Suite includes the following tests:
TV and Movies Suite includes the following tests:
Gaming Suite includes the following tests:
Music Suite includes the following tests:
Communications Suite includes the following tests:
Productivity Suite includes the following tests:
HDD Suite includes the following tests:
Since the basic PCMark Suite test is available in all versions of PCMark Vantage, it takes about 30 minutes to complete and only the results obtained in this testing suite form the PCMark Score, it will evidently be the most popular of all. Let’s take a closer look at the benchmarks it contains.
PCMark Memories 1 test contains two simultaneously running threads:
PCMark Memories 2 test transcodes the video from archive format into portable device format.
PCMark TV and Movies 1 test includes two simultaneous threads:
PCMark TV and Movies 2 test also contain two simultaneous threads:
Note: breaks in sound in PCMark TV and Movies 2 test indicate the system is under an extreme load.
PCMark Gaming 1 test contains two simultaneous threads:
PCMark Gaming 2 test works with two simultaneous threads:
PCMark Music 1 test deals with three tasks:
PCMark Music 2 test transcodes audio from WAV to WMA lossless format.
PCMark Communications 1 testing suite works with three simultaneous threads:
PCMark Communications 2 is also a three-thread test:
PCMark Productivity 1 test edits text in WordPad application. Numerous hard-to-pronounce Finnish names should make this task harder on the system, so no other applications are running at the same time.
PCMark Productivity 2 test involved four simultaneous threads at a time:
Now that the testing session is complete, you will be able to view the PCMark score only if you have PCMark Vantage Professional Edition:
More details on each of the tests are also available:
However, the owners of PCMark Vantage Basic or Advanced Edition shouldn’t envy the Professional users, because the score reports online are much more illustrative and informative anyway. This way you can not only view but also analyze the scores. First, you can clearly see on the charts where your system stands among all other registered systems:
Unfortunately, at the time we were working on this article, this service wasn’t fully activated yet, so there were no systems to compare against ours at this time.
The next page shows the general score and the specific score in each test.
Configuration page will refresh your memory regarding the system configuration and will reveal the configuration of your rivals.
Once again, the comparison is not very informative just yet, because there are simply no rivals in the system, but this is obviously a temporary issue, which will be resolved once PCMark Vantage is officially launched.
We have already mentioned before that this Futuremark product has become considerably more commercial. Studying the obtained results may prove very useful not only from the theoretical but also from the practical standpoint. Suppose, some system very similar to yours is far ahead of you in graphics tests. In this case you can check out the rival’s system configuration and order a graphics card like that immediately using the provided “Get It” links.
Our first impression from this test would be incomplete if we hadn’t checked it out in practice, i.e. for contemporary platforms comparison. Of course, the new Futuremark benchmark will very soon occupy and important position among our testing tools that is why very soon we will be able to offer you a pretty significant database of results. And in the meanwhile we decided to perform some preliminary tests on three different platforms based on a dual-core and quad-core Intel processors and a dual-core CPU from AMD.
Our testbeds were built using the following equipment:
The main difference between the test platforms was the CPU. Of course, it was not the only difference, the systems were based on different mainboards and used different memory. However, our experience shows that these are secondary factors that do not have that big of an influence on the system performance in real applications after all. Graphics cards and hard disk drives were the same in all three systems.
So, first of all let’s take a look at the main score provided by PCMark Suite:
The obtained results are quite adequate. In most cases, this situation can be observed between these systems on Intel and AMD processors in other modern applications. In other words, this benchmark provides pretty adequate estimate of the average system performance. At the same time, as you can see from the results, PCMark vantage is pretty nicely optimized for quad-core platforms which may make it fit for comparative analysis of upcoming multi-core platforms performance. Now let’s see what the other scores obtained from the rest of the tests look like.
I have to say that the overall picture is pretty much the same as in PCMark Suite. However, the developers didn’t actually promise anything else. All these test suites are specifically developed not to deny the results demonstrated in PCMark Suite, but to find out if any of the computer subsystems affect its overall performance in any way. That is why it makes sense to use any of the additional benchmarking suites besides the PCMark Suite only if you are comparing similar platforms, which don’t differ from one another dramatically.
Note that despite everything we have just said, the results of TV and Movies Suite stand out a bit. Here the AMD Athlon 64 X2 6000+ processor demonstrates higher performance than normally. The optimistic AMD’s score is most likely based on Futuremark developers’ belief that this processor copes well with video content processing tasks. However real application tests that we performed in our lab do not confirm this tendency. Therefore, we can question the results in this case.
Luckily, this is the only situation when the practical scores provided by PCMark vantage seem unjustified. The previous PCMark05 test was much more doubtful from this prospective, but nevertheless, it managed to become very popular. We hope that this only issue will not spoil the overall impression from the new PCMark vantage test.
The new complex performance benchmark – Futuremark PCMark Vantage - will become a very useful tool for a lot of users out there. Of course, it will be of interest to professionals, such as chipset, processor and graphics card developers and system integrators. Reviewers on numerous hardware review web-sites will also hardly do without it. However, PCMark Vantage will also be very useful for ordinary users thanks to the new Microsoft Windows Vista OS support and to the real applications used as basis for its tests.
As for the credibility and repeatability of the results, correctness of the comparisons between CPUs with different architectures, graphics cards from different manufacturers and systems in general, the answer can only come with practice. However, Futuremark’s extensive experience with benchmark development makes us very optimistic.