by Alexander Yuriev , Nikita Nikolaichev
06/11/2004 | 09:21 AM
Some time ago we covered the SATA-RAID Escalade 8500-8 controller from 3ware in our review. It showed the best results among the multi-channel SATA-RAID controllers we had tested, but the missing support of the PCI 64/66MHz bus hamstringed it cruelly. When testing the 8500-8 controller, we only used four hard disk drives and the bus bandwidth didn’t practically affect the linear read speed of the arrays. However, the low bandwidth of the PCI 64/33MHz bus will surely be noticeable with an array composed of more devices.
The problem of insufficient bandwidth is solved in the next product from 3ware: the new SerialATA RAID 8506-8 Escalade controller differs from the 8500-8 model exactly by the support of the PCI 64/66MHz bus.
The Escalade 8506 product series, like the previous line, consists of three models, supporting 4, 8 and 12 HDDs respectively. In our today’s review we want to compare the new controller with the results of the 3ware 8500-8, so we took an eight-channel controller but tested it in the four-channel mode, like in the previous testing session.
The 3ware 8506-8 Escalade controller allows building RAID arrays of levels 0, 1, 5, 10 and JBOD. It supports up to eight SerialATA devices, connecting to the computer across the 64-bit PCI bus at 66MHz. The controller has 2.25MB of cache memory onboard.
The basic parameters of the controller are listed below:
Specifications | |
Chip | 3ware 200-0017-00 |
Memory | 2.25MB SRAM (3.3V) |
PCI | PCI 2.2 64bit 66MHz bus |
SerialATA | 8 channels SerialATA 150 |
RAID | RAID 0, 1, 10 and JBOD |
The testbed was configured as follows:
We used the following benchmarking software:
We created one partition for the total capacity of the array in WinBench 99. We ran WinBench tests seven times each and then analysed the best result.
We used File Server and Web Server patterns for tests with Intel IOMeter.
These patterns help to benchmark the disk subsystem performance under a workload typical for file and web servers.
The Workstation pattern is created by Sergey Romanov aka GReY basing on the statistical data about the disk subsystem workload as given in the SR Testbed 3 description. The statistical data are gathered for the NTFS5 file system in three operational modes: Office, Hi-End and Boot-up.
This pattern shows how well the controller performs in a typical Windows environment.
Lastly, we checked out the controller’s ability to process sequential read/write requests of variable size and its performance in the Database pattern, which loads the disk subsystem with SQL-like requests.
Our controller had the firmware version 7.6.3 and we used the driver from the same pack (version 7.6.3). The 3DM Disk Management utility helped us control and synchronize the arrays. We installed the controller into a PCI-X/133MHz slot.
WD360GD (Raptor) hard disk drives were installed into the rails of the SC5200 system case and fastened at the bottom.
The BIOS of the 3ware 8506-8 Escalade controller doesn’t have an option of changing the lazy write status for the hard disk drives. We assume, however, that the states of the caches (of the controller and of the hard disk drives in the array) all change when we switch between WriteBack/WriteThrough modes. That’s why we use WriteBack/WriteThrough terms (WB/WT) – which stand for the settings in the controller BIOS – to describe the test modes. So, all caches (of the drives and our controller) are enabled in the WriteBack mode and disabled in the WriteThrough mode.
This pattern is sending a mixed stream of requests to read and write 8KB random-address data blocks. By changing the ratio of reads to writes, we can find out how good the controller’s driver is at sorting the requests out.
You see the results for the WriteBack mode in the following table:
The following diagrams show the dependence of the data-transfer rate on the reads/writes ratio for different request queue depths. For easier reading, I drew two diagrams for two groups of arrays:
All arrays show similar speeds under linear workload, at the beginning of the graph (Random Read). However, the RAID1 and RAID10 are faster than the others because 3ware controllers use the exclusive TwinStor technology. It is alternating requests between the drives of the mirror couple, depending on which drive will respond faster according to the current position of its read/write heads (the controller determines the position by keeping track of the request history).
The performance of the single drive is going up as there appear more write requests in the queue due to the lazy write algorithms of the drive.
The speed of the RAID0 grows according to the number of disks in the array, but this proportion is only felt when the percentage of writes is high. The RAID1 and RAID10 arrays also speed up, but more slowly. The RAID5 arrays drew a down-sloping graph: write requests impede them greatly. However, when there’s a big share of writes, the controller even increases its pace somewhat – we saw this curious thing in our review of the 3ware 8500-8 Escalade.
As the workload increases, we see the arrays show different speeds. The “mirror” arrays perform much faster in the Random Read mode, as no write requests and many random-address read requests allow the TwinStor technology to show its best. Ideally, you get a double gain in the read speed because reading is performed from two drives at a time, accounting for what head is closer to the desired address. As you see, the RAID1 and RAID10 arrays are really twice faster than the single drive and the two-disk RAID0, respectively.
TwinStor is less efficient when there is a good deal of write requests in the queue (that is, there are few read requests). TwinStor only works with read requests, as you know. Thus, it’s no surprise that “mirror” arrays slow down in such operational modes. On the other hand, when there’re many writes, the effect of the WriteBack caching mode is felt stronger, so the mirror arrays even speed up a little. Overall, under the 16 requests workload, the RAID1 is always faster than the single drive, while the RAID10 is in its turn faster than the two-disk RAID0 across all the operational modes.
The speed of the single drive continuously increases as there are ever more writes in the queue. The graphs of the RAID0 arrays have a shape, similar to the graph of the single drive, in proportion to the number of drives in the array. The RAID0 arrays are nearly always faster than other arrays of the same number of drives (except the modes with a high reads percentage where TwinStor strongly influences the performance of mirror arrays). The RAID0 arrays of two and four drives have slumps in their graphs on 10% writes – as you know from our previous reviews (for example, of the Adaptec Serial ATA RAID 2410SA controller), the cache of the hard disk drive is to blame for that.
Read requests are processed with the same speed for the RAID5 arrays and for the RAID0 arrays, but the RAIDs 5 slow down as the percentage of writes goes up.
The graphs of the two- and four-disk RAID0 arrays have the same shape as the graph of the single drive, indicating that the StorSwitch technology shows excellent results at sorting requests out and sending them to the appropriate hard disk. On the other hand, the three-disk RAID0 behaves somewhat out of the ordinary way when loaded with a lot of reads.
Interestingly, the four-disk RAID0 and RAID10 show very similar speeds in the Random Read mode, and the RAID5 is even a little faster than both of them, although we might have expected to see a small advantage of the RAID10. Probably, the mirror array doesn’t get any advantage from alternating requests between the drives under this type of load.
Now let’s see the numbers we have with four-disk arrays in the WriteThrough mode:
To compare the speeds of the RAID arrays in different caching modes, we fill the table with ratios of the controller’s speed in the WB mode to its speed in the WT mode. A higher number indicates higher efficiency of WB caching in this mode. If the number is below 1 (marked with red), WB caching is harmful. If the number is above 1 (marked with blue), WB caching brings a performance gain. If you see “1.0”, then WB and WT caching modes are equally useful.
As you see, you can only reduce the speed of the RAID array by enabling WT caching in the controller’s BIOS. We see no speed growth even in the Random Read mode (when there are no write requests at all), while in the Random Write mode the speed degenerates by a factor of six in some cases. Only the RAID5 array speeds up with WT caching at requests queue = 256, but such a long queue is highly improbable in real operation.
The results we got with different caching modes are better viewed as graphs. We drew three of them, for three queue depths (1, 16 and 256 requests).
The RAID0 is losing in speed as we switch the caching mode from WB to WT and increase the share of writes (the maximum loss amounts to 600%). As the number of requests in the queue grows, the gap between WriteBack and WriteThrough modes becomes smaller, but the advantages of WB caching are perfectly seen everywhere, save for the Random Read mode where there’re no write requests and, accordingly, there’s nothing to optimize!
WT caching negatively affects the array’s performance under linear workload. The speed loss amounts to 125%! However, the difference between the two caching modes becomes smaller as the request queue is getting longer. At request queue = 256, WT caching is even preferable, bringing about a 10% performance gain.
Switching from WB to WT caching, we see the same effect as with the RAID0 array, but the value of the loss is smaller (244%).
So WT caching very negatively affects performance of RAID0 and RAID10 arrays. You’ve got choice to make: between the high speed of WriteBack and the high reliability of WriteThrough. For the RAID5, the speed depends on the type of caching only under small workloads. This dependency diminishes suddenly under heavier loads, so you should enable WriteThrough for your RAID5.
The array receives a stream of read/write requests with a request queue depth of 4. Every minute the size of the data block changes, so we can see the dependence of the linear read/write speed on the size of the data block. The results for the WriteBack mode are listed in the following table:
For better reading, we divide the arrays into two groups in the diagrams:
The advantages of the RAID arrays that consist of many HDDs become apparent when the data block is big enough, that is, when the request is so big that the controller can split it up into portions for the drives to process those portions simultaneously. Mirror arrays don’t act very well here. When the data block size is over 32K, the read speed suddenly degenerates! Probably, the TwinStor technology brings about this effect… Well, we should confess that this technology is not always good.
As you remember, the main discrepancy between 3ware 8506-8 Escalade and 8500-8 Escalade controllers is the support of PCI 64/66MHz, so the results of the four-disk RAID0 and RAID5 are most interesting. Really, comparing the results with those of the 3ware 8500-8 Escalade, we notice that the maximum read speed of the RAID0 array is now 203.5MB/s against those 180.2MB/s that we squeezed out of the Escalade 8500.
The read speed of the RAID5 array is 141.6MB/s with the Escalade 8506 – higher than the 115.2MB/s speed with the Escalade 8500.
Anyway, even the higher performance of the controller chip is not enough for the four-disk RAID0 to be as fast as the quadrupled speed of the single drive.
Now let’s see how the controller behaves in the WriteThrough mode and compare it to the results we’ve just seen:
The difference in the results is negligible when there are no write requests. The status of lazy write doesn’t practically affect the speed of sequential reading.
Let’s turn to sequential writing now. The WriteBack mode comes first:
The speed of the RAID1 array nearly matches that of the single drive, while the speed of the RAID10 is a little lower than of the two-disk RAID0.
The two- and three-disk RAID0 arrays show good scalability according to the quantity of drives in the array. For the RAID5 arrays, when the data block size is small, the writing speed is higher than that of the single drive or of the “mirror” as R5 Fusion shows itself at work (it “sticks” write requests together in the controller cache). But when the requests are big, there’s simply not enough space in the controller cache (which is four times smaller than the cache of a single HDD). However, if we compare the speeds of the four-disk RAID0 and RAID5 on the 3ware 8506-8 Escalade controller and on the 3ware 8500-8 Escalade controller, we’ll notice that the maximum read speed of the RAID0 grew from 164.9MB/s to 195.7MB/s and the maximum read speed of the RAID5 array grew from 64.3MB/s to 71.4MB/s. This speed growth, especially for the RAID5 array, whose read speed greatly depends on the speed of the XOR processor, makes me think that the Escalade 8506 differs from the Escalade 8500 not only in the support of a faster bus, but also in a higher clock rate of the main chip of the controller (StorSwitch).
Now let’s see how the caching mode affects the performance of four-disk arrays:
By disabling lazy write we greatly reduce the performance of every array!
These patterns simulate the operation of the disk subsystem of a typical file- or web-server. First goes the File Server pattern with caching enabled:
The File Server pattern has only 20% of write requests and all arrays find this mode favorable. The RAID1 is faster than the two-disk RAID0, while the RAID10 is close to the four-disk RAID0 in performance. It means that TwinStor is working excellently at this reads/writes ratio.
The RAID0 arrays show nice scalability in speed: the more drives are included into the array, the faster the array is.
Let’s calculate the performance rating for each array by averaging the controller speed under each workload:
The results are predictable, although I’d single out the high number of the RAID10. Let’s now compare four-disk arrays in WriteBack and WriteThrough modes:
Even 20% of writes is enough for the speed to suffer from turning lazy write off. The ratings follow:
Depending on the array type, the performance in the WriteThrough mode goes down by 10-30%. Well, that’s not a very big pay for security…
Let’s watch the controller getting through the Web Server pattern with caching enabled:
This pattern has no write requests, so the RAID5 arrays are very fast. Mirror arrays are also doing very well. However, the requests queue being 256, the RAID10 is behind the four-disk RAID10 (we saw something like that in the Database pattern), while the RAID1 has a mysterious slump in the graph at requests queue = 64. The graphs of the RAID0 arrays haven’t changed their shapes since the File Server pattern.
These are the ratings of the arrays, which we calculated by the same formula as in the File Server pattern:
There are no writes to be performed, so RAID1, RAID10 and RAID5 arrays all show very high speeds. Each of them outperformed the RAID0 of the same number of drives, while the RAID10 is the best of all here.
Let’s compare four-disk arrays with different caching types.
There is nothing to optimize as there are absolutely no write requests! Thus, the status of the cache of the controller and the drives has no effect on the array performance.
The Workstation pattern imitates intensive work of a user in various applications in the NTFS5 file system. The table for the WriteBack mode:
The RAID0 arrays show good scalability in speed in relation to the number of the drives per array. The RAID1 is faster than the single drive and even than the two-disk RAID1 (under small workloads), while the RAID10 outperforms the RAID0 of three disks. The RAID5 arrays are poor, but that’s reasonable since this pattern abounds in write requests, which hit severely on the performance of the RAID5.
We calculate the performance rating for the Workstation pattern by the following formula:
Performance Rating = Total I/O (queue=1)/1 + Total I/O (queue=2)/2 + Total I/O (queue=4)/4 + Total I/O (queue=8)/8 + Total I/O (queue=16)/16 + Total I/O (queue=32)/32.
The PRs for all the arrays are listed below:
The RAID5 arrays fall behind even the single drive in performance. The RAID0 arrays lined up in “size order” (according to the number of disks in them), while the RAID10 was just a little faster than the RAID0 of three disks. The RAID1 has a higher PR than the RAID0 of two disks. That’s because we assume that short queues are more likely to occur in a workstation, and short queues have higher weights in the total result.
Let’s see how lazy write affects the operation of the four-disk arrays in this pattern:
There are write requests in the pattern and the caching mode affects the performance greatly. Switching to the WriteThrough mode, the arrays perform slower by 28-38% (depending on the array type).
WinBench is going to be our last test today. This benchmarking set helps to estimate the disk subsystem performance in desktop applications. Test results for the NTFS file system are listed first, in the WriteBack mode:
The following table compares the arrays in two integral subtests: Business Disk Winmark and High-End Disk Winmark:
The RAID0 arrays show a dependence of the speed on the number of drives in the array. WinBench 99 abounds in write operations, so the RAID5 arrays have poor results due to their slow writing. And that’s also the reason for the RAID1 and RAID10 arrays to be not very fast here. We turn off lazy writing and compare the results:
All arrays are much slower in the WriteThrough mode than in the WriteBack one.
FAT32 is our next file system. The results for the WriteBack mode are listed below:
The arrays have the following speeds in Business Disk Winmark and High-End Disk Winmark:
The arrays ranked up just like they did in NTFS, but the speeds grew significantly.
The WriteBack mode influences the speed of the arrays in FAT32 as greatly as in NTFS. The rankings don’t change, but the speeds do change noticeably.
The linear read speeds are the same for both file systems, so we’ll show you just one diagram:
The linear read speed depends linearly on the number of the drives, so the RAID0 arrays of two and three devices show a double or triple speed of the single drive. The RAID0 array of four disks doesn’t unfortunately keep this tendency on. That’s the same as we saw in our review of the 3ware 8500-8 Escalade.
The linear read speed doesn’t depend on lazy writing, as we have expected.
The linear read graphs for every array are found by the following links:
The 3ware 8506-8 Escalade controller left a very good impression, just like its predecessor, the 3ware 8500-8 Escalade. This controller showed good results in all tests, especially those that imitate work in real applications. There was certain instability in speed in synthetic tests, but I think such things are only of interest to specialists. I would like to emphasize the fact that the controller is now much faster at processing sequential read and write requests.
This controller showed most stable and predictable results among all SATA RAID controllers we have tested so far. We will be carrying out our tests to compare it to competitor products, though. So, stay tuned!
At the time of writing this, only the version 7.7.0 driver was available for download from the manufacturer’s website. This set of firmware, drivers and utilities works in Windows 2003/XP/2000, SuSE Linux 8x, Red Hat Linux 8x and 9x, and FreeBSD 4.8 Beta. Regrettably, other driver versions (for example, the 7.6.3 release we used in our tests) that support other operation systems were unavailable at the moment.