"It’s a little strange that every site points amd as the best (faster), but this isn’t the case here.
"

No, since the time-demo shows the A64's KILLING the Intel CPUs.... just like everyother site using the Time-Demo to test the preformance of CPUs in Doom 3. The myth that the Time-Demo in D3 resembles real-life gameplay is now broken... by X-Bit Labs here...

I agree with #10 (I think).

You've got all of the extremes in these benchmarks, while failing to test the actual *real* performance. Timedemo certainly doesn't work, for the reasons you indicated. No AI, sound, or input calculations means the CPU is doing less.

Your second timedemo, however, has some odd results. If indeed the id Demo1 is not correct, because it is not taxing the CPU enough, how then could your demo (which is supposedly taxing the CPU more) end up with universally higher FPS rates!?

The last FRAPS test goes to the other extreme. Getting 20+ monsters in a room with God mode on and watching them duke it out represents gameplay in what way? [H] seems to have the better FRAPS scores.

Another issue is that you included all the high-end CPUs without any of the low-end CPUs. I just upgraded from an Athlon XP-M running at 2.4 GHz to an Athlon 64 Newcastle running at 2.42 GHz (yes, both are overclocked). Sticking with my 9800 Pro card at HQ 1024x768, it is difficult to tell the difference between the systems. Dropping to Medium quality, the A64 feels a little faster, but not substantially so. In Timedemo, incidentally, both systems scored almost the same: ~42 FPS on the second run of Demo1. (The A64 was slightly faster on the initial run, incidentally.) I would have liked to see *real world* benchmarks with the lower Athlon XP and P4 chips included. (i.e. play the ACTUAL game like MOST people play it, and report FRAPS' low and average FPS).

Going along with that last part, the RAM claims you make are not at all correct. Running a Timedemo, the second and later runs tend to level out with 512 MB of RAM on Demo1. Playing the ACTUAL game, however, there is a definite difference between 512 MB and 1 GB. You get momentary lag on both as graphics are loaded, but you get it a LOT more often on the 512 MB systems. Doom 3 in regular play seems to grow my swap file usage (I have 1 GB of RAM) to ~1 GB while running, and having tried it on a 512 MB system, it's a very perceptible difference. After a few minutes running around a level with 1 GB of RAM, the pauses pretty much disappear, while they remain relatively constant on a 512 MB system.

Synthetic (i.e. not simply playing the game) benchmarks are almost always inaccurate for a variety of reasons. This article illustrates that quite well, at least to me.

I would agree with #10. Timedemo1 gives 50% higher framerates for the FX53 than AthlonXP3200+, but with your demo this advantage is only 30% or so. That says to me that in your demo the CPU is less important for the scores an other factors limit the performance.

#17,
It would be wise to read carefully before posting. Xbit article clearly states right in the beginning that their custom test was not gpu limited, because they have tested for that and that was the result.

Very well, then explain to me how all the processors except the Athlon XP scored over a range of 28.5% in the default demo, yet only 10.2% in their timedemo. They never stated any numbers for their performance difference with graphics disabled, and they never mentioned performing that test on the default timedemo.

Also, having read your other posts, I am having a hard time understanding what you think is going on here. You appear to concede there is a possibility "this demo is not more cpu depending." referring (I assume) to the xbit timedemo, so do you think then it is identically cpu dependant to the timedemo? Or are you scolding me based on evidence you don't believe? Or, possibly, due you think it's limited by something other than the cpu or gpu?

A few more things I think I should clear up:

Their fraps test has shown to be more cpu limited than their timedemo (though still not by as much as demo1, judging by the spread of 20.6% for the fraps test in avg fps excluding the AXP's). How did intel claw back the considerable advantage held by amd in the default demo and even manage to come out slightly ahead? I don't know, but I have some ideas:
-The sound subsystems on the intel motherboards could be more efficient with cpu usage. xbit made no mention of any discreet sound cards.
-The player will be getting hit much more often than normal in this test. Perhaps the effects used when hit run better on intel processors?
-There will be physics and AI (much more than normal, with all the monsters), which may run better on intel processors, especialy if they are run on a separate thread from the rendering engine which would be taken advantage of by hyperthreading (anybody know about this?).
-I have never used fraps, but if it uses any cpu time than intel chips would gain an advantage due to its being run in a separate thread from doom3.

It is unfortunate that there is no good way to measure real-world performance in doom3, the ideal solution would be to have the timedemo mode perform all the same calculations (and play the same sounds) as the normal mode, but just not apply the results of the aI and physics calculations to the environment to avoid a random roll or slightly different fpu calculation from altering the course of the demo.

Finally, regarding the xbit timedemo, xbit is essentially saying that when the cpu is stressed in doom3, all high end cpu's perform nearly identically, whereas when the cpu is not stressed, great differences in performance are observed. This makes absolutely no sense and is the reason why their conclusions are so suspect.

In response to your suggestion of using fraps to measure the timedemo speeds, I do not know what the results would be, but it would not prove anything either way. Even if the general assumption that low fps=cpu dependant holds true, there could be a split second of very low fps (=cpu dependency) in an otherwise gpu limited timedemo. This theoretical demo would still be more adept at measuring graphics card performance.

Imagine a demo that runs, on a baseline system, at:

100 fps for 55 seconds (completely gpu limited)
1 fps for 5 seconds (completely cpu limited)

there are 100*55+1*5=5505 frames, completed in 60 seconds

the average fps =5505/60=91.75 fps

doubling the gpu speed would yield: 200 fps for 27.5 seconds and still 1fps for 5 seconds. avg fps is 5505/32.5=169.38 fps
which is quite differnt from baseline.

doubling the cpu speed yields: 5505/(55+2.5)=95.74 fps
which is hardly different from baseline.

Clearly, this first example is a primarily gpu dependant test.

Now if a second demo has a scene that is not quite as demanding as the difficult scene (say 2 fps baseline), but went on for longer (say 55 seconds), the results would be (I will spare you the math, but you can check it yourself, the other 5 seconds are at 100fps).

baseline: 10.17 fps
2xgpu: 10.61 fps
2xcpu: 18.77 fps

which is clearly a cpu limited test, even though it has higher min fps (2) than the first example (1). These examples may not be very realistic, but they serve to provide a counterpoint to the assertion that low min fps=cpu limited, as it isn't always (when looking at the entire test). As a side note, this is the reason average fps is still a more popular measure than min fps, because average fps contains information of the entire test and is therefore more accurate than min fps which only contains information on a single point. Perhaps a hybrid measure, something like would be more useful?

The board system seems to have removed my suggestion. The last line of the above post should read:
Perhaps a hybrid measure, something like "fps exceeded for 90% of the test" would be more useful?

I do not agree with neil.
He assumes, that if the framerate drops to 1% of the maximal value it means cpu-dependence. Here's example of situation where this isn't true:
http://www.techreport.com/etc/2004q3/deltachrome-s4pro/ind ex.x?pg=9
The SplinterCell graph shows some drops in framerate because of the card.

I have found an antoher example of clear GPU-dependance. Please look at SS2 over here
http://www.techreport.com/reviews/2004q2/radeon-x800/index. x?pg=11

In my opinion CPU dependence means
the change in CPU frequency % ~ the change in framerates %

The GPU dependence means
the change in GPU frequency, class [vertex, pixel throughput] % ~ the change in framerates %

What do I see in Xbits two demos [the third one contains only integer values, this makes error margins a litlle bit to high]
1.AthlonXP 2.0Ghz vs 2.2Ghz 10% freq -> 7% fps 10/7~1.4
Pentium 4 3.0Ghz vs 3.4GHz 13% freq -> 9% fps 13/9~1.4
AthlonFX 2.4Ghz vs 2.2Ghz 9% freq -> 6% fps 9/6~1.5

2.AthlonXP 2.0Ghz vs 2.2Ghz 10% freq -> 5% fps 10/5~2
Pentium 4 3.0Ghz vs 3.4GHz 13% freq -> 4% fps! 13/4~3.3
AthlonFX 2.4Ghz vs 2.2Ghz 9% freq -> 1% fps! 9/1~9
The last test shows some CPU-dependence only for AXP.

The most interesting pair is
1. P4XE i875P vs i925X -> 1.5%
P4XE 3.2GHz vs 3.4Ghz [i875p] 6% freq -> 4% fps
2. P4XE i875P vs i925X -> 2.2% fps
P4XE 3.2GHz vs 3.4Ghz [i875p] 6% freq -> 1.1% fps
We see the impact of chipset is greater for the 2nd test than impact of CPU!
For other processors I see the signs of other dependencies...

An another concern is the error margins of fraps. As far as I know it is aboot 1-2% and varies from system to system, so this makes whole testing a little bit stochastic.

Since the athlon xp performs so poorly relative to the other cpu's, the question becomes: how should athlon xp users tweak Doom 3 in order to stabilize the framerate? This would probably be accomplished by customizing the config file by turning down graphics effects that are handled by the cpu as opposed to the video card.