Unlike 3DMark2000 and 3DMark2001 the new benchmark tests the performance of nearly all major computer systems (CPU, memory, HDD, graphics), and not only the graphics subsystem. Introducing a new benchmark, MadOnion stressed its simple and user friendly interface at the same time pointing out the professional algorithms. Here is a quote:
This easy-to-use benchmark makes professional strength benchmarking software available even to novice users.
Before we pass over to our test system and testing methodology, I would like to share my vision of a perfect benchmark. I believe that each good benchmark should meet the following requirements:
- It should be of reasonable size;
- It should be free for the end-user;
- It should work properly under any popular OS;
- It should not be optimized for the products of some particular company;
- It should generate "repeating" results, i.e. the difference in the results obtained during multiple tests shouldn't be too big;
- The test algorithm and the results obtained should be easy to explain.
As for professional benchmarks, I think they should have the following features:
- Follow the scenario;
- Save results in a simple format ready for further processing and analysis;
- Cost reasonable money :)
So, what can we say about PCMark2002 before we start testing our hard disk drives with it?
The test of normal size (about 8.5MB), is nearly free and works under Windows98, ME, 2000 and XP. It is optimized for all modern processors and the results it generates are intuitively clear (at least those dealing with hard disk drives).
As for the second part of requirements listed above, PCMark2002 can't boast anything but low price. There is no way you can set the system to reboot automatically after the test is completed. Besides, the results are saved in a format, convenient only for viewing but not for further processing.
The PCMark2002 benchmark set for storage subsystem includes 5 tests measuring the time required to:
- Write a file with caching;
- Write a file without caching;
- Read a file with caching;
- Read a file without caching;
- Copy a file.
As we see, all these tests are very easy to understand :)
Testbed and Methods
We tested the newcomer in the following test system:
- Intel Pentium III (Coppermine) 600MHz CPU;
- ASUS CUBX-E mainboard, bios 1008;
- 2 x 128MB PC133 SDRAM by Hyundai;
- Matrox Millennium 4MB graphics card;
- Windows 2000 Pro/SP2.
We experimented with 5 hard disk drives: WD800BB, WD1000JB, IBM 120GXP (80GB), Maxtor D740X-6L (80GB) and Seagate Barracuda ATA IV (80GB).
All hard disk drives were tested with Promise Ultra100 controller integrated into the mainboard. We used driver version 1.60 build 33 for this controller. All drives were tested in UDMA100 protocol.
To run PCMark2002 we used FAT32 and NTFS file systems to format each of the tested HDDs as one logical drive of the maximum size with the default cluster. All the tests were run 5 times and the system was rebooted after each test. The HDDs didn't rest for cooling down between the tests.
The tables below contain the results obtained for each hard disk drive separately. Each table shows the results of all 5 test sessions, the average performance and the maximum results deviation from the average. I would like to draw your attention specifically to the last parameter, as it allows us to evaluate how stable the test is and how big the measuring error can be.
As you can notice, PCMark2002 test is pretty self-willed, so that the results of two runs-through may differ quite significantly. Of course, we know that "lost here, gained there" rule works, however, the measuring error even in the final result of PCMark2002 can sometimes exceed 6%! Now that we sometimes have to resort to photo-finish to find out who the winner is, a measuring error like that is absolutely unacceptable.
Well, let's try to compare our hard disk drives (we will take the average and the average and the maximum result for each HDD):
Hm… We see a very paradoxical picture! The indisputable leader boasting huge advantage over the competitors is IBM 120GXP, outperforming the second fastest racer, WD800BB, by the good 100 points.
The slowest HDD according to PCMark2002, appeared Maxtor D740-6L, and the third and fourth positions were shared between Seagate Barracuda ATA IV and WD1000JB. The remarkable thing about the WD1000JB solution is that its large 8MB cache buffer didn't have any influence on the HDD performance in PCMark2002, so that WD1000JB appeared about 40-50 points behind WD800BB.
In NTFS the situation got a little bit different. WD1000JB seemed to have suddenly woken up and jumped straight to the first position of our rating. We positioned the drives according to the average result, so don't be surprised to wee WD1000JB with the absolute victory of 989 points only on the second spot. It is stability in the first place that matters to me so I decided to give the laurels to IBM Deskstar 120GXP this time. The Maxtor drive again turned out an outsider here.
Our first close contact with PCMark2002 test showed us that:
- The test provides very unstable results (low level of repetition);
- The test is very easy to use (the only drawback is that you cannot save or download the user customized testing set);
- The results shown by PCMark2002 sometimes contradict with the existing opinion about the HDD performance.
Judging by everything mentioned above, I decided not to include this benchmark into my testing methodology for hard disk drives. However, I will continue collecting the stats for all HDDs ever tested in our lab. And then maybe we will see a patch to this test :)
P.S.: And of course, I look forward to seeing what other reviewers will say about PCMark2002 :)