X-bit labs

Started by: boner | Date 06/21/07

Comments: 26 | Last Comment:  11/12/07

[1-15]

1. I'm not sure why you're not specifying the software used - maybe I'm overlooking them in your Testbed but it's obviously Diskeeper of some flavor or another. Regardless the software is a variable that is unaccounted for without testing any competing defragmenters out there. Granted, Diskeeper technology is what's embedded into Windows - and I love its full-fledged products - but it's still a variable here.

I agree. I just had to go back and reread the first page to find out what they were using.
"And now we want to compare HDDs using the defragmentation tool integrated into Windows XP."
Also, is it just me or is there not a "Testbed" page?

There are a lot of defragmentation utilities out there, but the objective of this article wasn't to compare all of them side by side. A detailed discussion of different defragmentation tools will require more time and a totally different approach.

2. Windows' built-in disk defragmenter really doesn't work well, especially for heavily fragmented drives -- my main storage drive for a few years, an old Maxtor DiamondMax 740, was so badly fragmented that it not only put your testbed to shame but couldn't even be disentangled by Windows' defragmenter.

Actually one thing I would like to see, especially from a more or less independent source like X-Bit Labs, is a shootout of defragmenter programs. I've tried Windows' built-in, Diskeeper v7 through 10, Raxco Perfect Disk, and O&O Defrag. In my experience, with nearly-full and extremely fragmented drives, O&O performed best out of that group -- but an independent test would be really nice.

As we said in the Conclusion, we will continue discussing the NCQ efficiency during defragmentation with one of the commenrical tools. We will also consider the possibility of comparing the existing defragmentation utilities and their efficiency.

3. I was shocked by the difference NCQ made. Thanks for the great objective test!

4. NCQ gives better performance only when many processes uses the same disk simultaneously. Disk defragmenters have one process for disk, so enabling NCQ doesn't make difference.

NCQ boosts performance when reading and writing successive clusters and not random ones, when the commands can be queued if necessary. Defragmentation is performed in this particular way that is why we could theoretically expect a performance improvement. Some of our fellow testers claim they've seen improvement up to tens of percents.

Aleksey, I think your understanding of NCQ may be a little confused. Rather than re-explaining it myself I suggest you check out the following: http://en.wikipedia.org/wiki/Native_Command_Queuing

With this in mind it should be obvious why NCQ doesn't boost performance when defragmenting a single partition--all operations are performed sequentially rather than interleaved, because the defragmentation functions have to maintain additional transaction journals in order for the data's safety to be guaranteed (the NTFS driver always journals metadata but only journals the data itself under special circumstances).

Anyway, it might be possible to see some benefit if you defragment multiple partitions on the same drive simultaneously because then it's possible for queue depth to become greater than 1. Of course under ordinary circumstances it would be pretty stupid to do such a thing but it might be interesting for benchmarking purposes--you will need Diskkeeper or some other such product as the built-in defragmenter doesn't support multiple simultaneous operations. Sysinternals contig could be used to make a FC-test script I guess.

Incidentally you don't say how many times you repeated each measurement, so I assume they were all done once? I'd rather see three sets of repeats and times measured in seconds, not minutes:seconds, please.

Otherwise, good review--worthwhile results in respect of the different drives though I find the NCQ results not exactly surprising. :)

Exactly. For NCQ to have any effect, the defragmenter would need to issue simulatenous disk requests first.

5. Good article. It was quite a surprise to see my OS HDD at the bottom of the list. Not that it needs much defragmenting but still.

I would like to see Raxco PerfectDisk tests myself as that has been used in pretty much all the companies I have worked for.

Not really sure if there is any valid reason to make a "defragmenter speed shootouts" as the defragmenting end results are different. Other aspects of the different programs might on the other hand be interesting.

6. You should have added a SSD to the comparison !

Sorry, we don't have any SSD in our lab now, but we will definitely check them out ASAP

7. Slightly off-topic comment:

In my experience, if the first pass of defragmentation doesn't give perfect results (as shown on page 2), another pass will deliver a perfect result, and this second pass will be much faster.

Not only does 2 passes with the default defragger improve results, but also the windows prefetching mechanism seems flawed, perhaps geared to ancient setups when drives were different and much smaller?
Anyway if a HD gets full near capacity it helps to remove layout.ini from %system%/windows/prefetch, it makes the drive boot much faster and it prevents the infamous missing trayicons bug I discovered.

I meant to say remove layout.ini before a defrag, it is supposedly used by defraggers to make the most used programs be oriented on HD's in such a way that they can be accessed quicker, but for some reason it seems not to work as expected and do the opposite in many cases.
Note that layout.ini is constantly updated/newly created so if you remove it it will re-appear after no longer than a few days, sometimes after minutes of normal windows use.
Perhaps some site should experiment with that?

8. In my experience both defrag and virus detection software scans of the HD give a pretty good indication of real world HD performance.

I've never been happy with the data reliability on S-ATA drives so I stick with traditional 15K SCSI drives and get excellent performance and reliability. Yeah you pay a little more, but to me it's worth it to NOT have to reconstruct the data from a POS HD that takes a dive.

The new serial SCSI's, (SAS), data reliability remains to be determined. The reason serial SCSI drives are being hyped is the reduced production cost - same as S-ATA drives.

9. Why didn't you test all the disks that support low noise mode?

10. Defrag ? I never do that !

Oh ! but wait ! That's only for windoze users !!!

Ah ah ah ah ah ah ah aha ha hahahah ahahahahahha !!!

You never defragment the filesystem because your filesystem can only be defragmented by deleting everything and restoring from a backup. No wonder you don't understand much about it or why it is necessary for some usage scenarios and not others.

For the record, NTFS also implements all the anti-fragmentation features that extfs (for example) does, but for single-user applications performance characteristics are much more likely to be dominated by sequential access than random access, whereas in a multi-user mode it's the other way around. Since Linux until recently was rarely used by only single users, sequential access wasn't important enough to bother implementing a defragmenter.

Maybe since you evidently do use Linux in a single-user mode, maybe it's time you got your head out of the sand and started thinking about such things rather than posting worthless uninformed flames like the above.

11. Disk defragmentation is a terrible test of NCQ. The program is trying to make sure that what it does, is done safely. For this, it waits for each IO to complete before issuing another. Therefore no Queue buildup occurs on the drives.

NCQ will only have an effect if the number of IOs issues independently exceeds the rate at which the drive can satisfy them in the order they are issued.

I expect Xbitlabs' clever guys to know simple stuff like this. I like to refer people to this site because of its general good quality of articles compared to some other sites that I won't name but are very popular despite all the misinformation they spread as fact.

Of course we understand the nature, purposes and effects of Native Command Queuing. Initially some other resources published their measurements, where NCQ enabled drives leave the competition in dust, so we decided to repeat the experiment in most matching conditions.

Now we prepare the second part of the article, with more hard disk drives and a commercial defragger (PerfectDisk 8.0) in test, so don't miss!

12. not bad big surprise in the NCQ on tests but i think you should have atleast compared the top 3 defragmentation programs as we all know that the proggy's that windows comes with are usualy cut rate versions of the real thing o&o , executive software and perfect disk all have very fast algorithyms now that leave the dusty old version of defrag in windows for dead

13. After googling, I found some information here on Diskeeper blog about NCQ and fragmentaion.

http://www.diskeeperblog.com/archives/2007/08/sata_ncq_and_di_1.html

Maybe this will throw some light on the subjec of NCQ and fragmentation.

14. Useful review ---- I would like to see these sort of tests

15. gooooood........

[1-15]