SSD for PCI-Express: OCZ IBIS vs. OCZ RevoDrive

Today we are going to talk about SSDs with PCI –Express interface. This type of devices is free from a common bottleneck – insufficiently fast SATA interface, so this is where we should be looking for a performance leader.

by Aleksey Meyev
02/16/2011 | 11:23 AM

The considerable reduction in flash memory prices has helped solid state drives reach the mass market and disrupted the long-established trends and traditions of top-performance storage systems. There used to be but one way to achieve high disk subsystem performance which involved building RAID arrays out of 15,000RPM hard disk drives. The more HDDs you had, the higher the resulting performance of the RAID. Thus, a high-performance disk subsystem would always be large and hot.

 

Then came multichannel solid state drives. Flash memory endowed them with a fantastic read response time while the multichannel design ensured high sequential speeds. They even boasted high write speed thanks to the introduction of a spare pool of memory blocks always ready for writing and to their efficient algorithms of wiping unused cells that worked together with the TRIM command. Most importantly for server applications, you could have all this in a single and rather compact device with very low power consumption.

SSDs soon encountered performance limitations. The conventional SATA interface proved to be too narrow for them, and the number of channels in the multichannel controllers could not be increased infinitely. The solutions were found easily, though. SSDs acquired PCI Express interface which, in its 2.0 version, can transfer up to 0.5 gigabytes per second in both directions along a single lane. The number of memory channels and storage capacity were increased by introducing a RAID controller directly into the SSD design. In other words, such an SSD is actually a composite device in which there are actually multiple SSDs, each with a dedicated multichannel flash controller accessing its own memory chips, hiding behind a RAID controller.

In our previous article we already took a look at one such product and compared it with conventional SSDs. Now we are going to discuss this kind of SSDs in more detail.

Testing Participants

OCZ RevoDrive: 50, 80, 120 and 240 GB

 

 

 
OCZ RevoDrive 120, 240 GB

The OCZ RevoDrive looks like a RAID controller, especially if you are old enough to remember how RAID controllers had looked before they transformed into today’s highly integrated products with just a couple of electronic chips. It is an expansion card with PCI Express x4 interface that carries two SF1200 flash controllers, a RAID controller Silicon Image SiI3124, a PCI-X à PCI Express converter, and some memory chips. The exact number of the flash memory chips depends on the storage capacity: 32 in the two top-end models, 24 in the 80GB model, and 16 chips in the 50GB RevoDrive. Clearly, the number of memory channels is reduced proportionally in the junior models, which should have an effect on their performance. We will see that in today's tests.

The RevoDrive series does not support TRIM because of the RAID controller but claims to deliver fantastic performance: a read speed up to 540 MBps and a write speed of 400 or 480 MBps (sustained and peek write speed, respectively). Interestingly, you can access the BIOS of the integrated RAID controller and even change its operation mode from RAID0 to RAID1 or JBOD. We did this with one of our samples out of curiosity, effectively cutting the number of memory channels by half. This provides a unique opportunity to see how important the number of memory channels is for an SSD, the total amount of flash memory being the same.

OCZ RevoDrive x2: 100 GB

 

If the maximum capacity (480 GB) or performance of the RevoDrive series does not satisfy you, OCZ offers a more advanced alternative called RevoDrive x2. It is the same card with an addition of a second PCB that carries two more SF1200 flash controllers connected to the free channels of the SiI3124 and more memory chips. Thus, the internal RAID0 consists of as many as four disks now. Our sample is a junior RevoDrive x2, though. It carries only 100 gigabytes of memory, so each SF1200 controller has a reduced number of memory channels. It is a kind of a double version of the 50GB RevoDrive, so we have another chance to see what benefits we can achieve from having more memory access channels. The specified speeds of our sample are much higher than those of the ordinary RevoDrive: up to 740 MBps at reading and up to 550 and 690 MBps at writing (sustained and peak speed, respectively). The top-end RevoDrive x2 models are even faster: up to 740 MBps at reading and 720 MBps at writing.

OCZ IBIS: 100, 160, 240 GB

 

The RevoDrive family will be competing with the OCZ IBIS series which is dramatically different visually. These are 3.5-inch devices that connect to an included controller (a PCI Express x4 card) via a special High-Speed Data Link interface developed by OCZ. With a peak bandwidth of 1 GBps, the HSDL interface borrows the physical infrastructure from SAS, a popular interface of server-oriented HDDs. It has the same connectors and cables. Thus, OCZ has effectively redesigned the solid state drive by moving the flash memory controllers and chips into a separate device that is connected to a PCIe card. Why? The PCIe card with 4 HSDL interfaces, already available for ordering, makes the benefits clear. This design allows using up to four full-featured SSDs with only one PCIe slot. You can connect up to eight SSDs using only two of those PCIe cards to a typical server. You wouldn’t be able to do that with the RevoDrives and ordinary mainboards because you would just run short of PCI Express slots!

The innards of the 3.5-inch box are almost the same as the RevoDrive x2: a RAID controller SiI3124 with four SF1200 flash controllers and some flash memory chips.

Now let’s move on to our tests and be ready to see very high numbers.

Testbed and Methods

The following testing utilities were used:

Testbed configuration:

SSDs are tested with generic OS drivers. We format them as one NTFS partition with the default cluster size (for FC-Test we create two 32GB partitions). We have transitioned to new testing methodology, by the way.

Performance in Intel IOMeter

Sequential Read & Write Patterns

IOMeter is sending a stream of read and write requests with a request queue depth of 4. The size of the requested data block is changed each minute, so that we could see the dependence of an SSD’s sequential read/write speed on the size of the data block. This test is indicative of the maximum speed the tested SSD can achieve.

The numeric data can be viewed in tables by clicking the links below. We will be discussing graphs and diagrams.

Although somewhat lower than specified, the sequential speeds are indeed very, very high. The RevoDrive x2 and the IBIS series, excepting the 160GB model, hit the 600MBps mark. Yes, each of these small quad-controller SSDs would outperform a RAID0 array built out of four modern 15,000RPM hard disks. This is very, very fast. The RevoDrives with only two flash controllers are slower, delivering a top read speed of 400 MBps. And our custom-made single-controller version is no faster than 200 MBps. So, the performance of these SSDs scales up depending to the number of memory access channels since they are not limited by the interface bandwidth or by the RAID controller. It must be noted that the difference in speed can only be felt on 32KB or larger data blocks. The top speeds are achieved rather late into the test. In other words, your applications must be able to work with large data blocks for these SSDs to show their best.

One remark is necessary about what you cannot see in the diagram. This very high performance of the disk subsystem calls for a high-performance CPU. Our testbed is equipped with a rather old but fast Pentium 4 620 processor, and its load varied from 27% (large data blocks) to 84% (small data blocks) throughout this test. Yes, even though the CPU is not heavily involved into disk operations, it takes a lot of CPU resources when there are over 30 thousand data blocks being read from the disk each second!

We’ve got very interesting results in the sequential write test. The top-capacity IBIS takes first place as it has four flash controllers with a full set of chips on each. Second place is shared by the IBIS 160 GB and the RevoDrive 240 GB. The former has four SF1200 controllers but each of these controllers only uses six out of eight memory access channels. The RevoDrive 240 GB has only two SF1200 controllers but these work with full memory sets, i.e. use all of their memory access channels. As a result, these differently designed SSDs have almost identical graphs.

With half the memory chips, the 100GB IBIS draws its graphs at about half the level of its full-capacity cousin, indicating that a direct correlation exists between the number of memory access channels and write speed. Interestingly, the RevoDrive x2 is somewhat slower than the similarly designed IBIS 100 GB and even inferior to the RevoDrive 120 GB which has only two flash controllers but a full set of memory chips. The junior models of the RevoDrive series are rather a depressing sight, yet our custom-made sample with only one memory access channel is even worse.

Disk Response Time

For 10 minutes IOMeter is sending a stream of requests to read and write 512-byte data blocks with a request queue of 1. The total of requests processed by each SSD is much larger than its cache, so we get a sustained response time that doesn’t depend on the SSD’s buffer size.

Well, once again we have to admit in our response time tests that today’s SSDs are just perfect in this respect. Modern flash memory is highly responsive at reading whereas advanced controllers with effective algorithms, TRIM instruction and a pool of spare memory blocks have helped improve the response time at writing. The IBIS 240 GB even has a better response time at writing than at reading! Well, the numbers are really so small that we shouldn’t compare them directly considering the inaccuracies of our statistical measurement method.

Anyway, we want to note one thing that stands out in the diagram: the RevoDrive products with the reduced number of chips, including the RevoDrive x2, are somewhat slower than their full-capacity counterparts. The JBOD-based model is downright poor, by the way. We suspect the SiI3124 controller has something to do with its high read response time compared to the same SSD in its normal operation mode.

Random Read & Write Patterns

Now we will see how the performance of the SSDs in random read and write modes depends on the size of the requested data block.

We cannot expect very different results at random-address reading. SSDs usually all behave in a similar manner here, and they do so today, but we can note a couple of exceptions. The JBOD sample and the 50GB RevoDrive have the worst results. The reduction of memory channels by half (by disabling one of the two SF1200 chips in the former and by halving the amount of the latter’s memory chips) has a very negative effect of their performance. It looks like an SSD must have 2x8 or 2x6 memory access channels in order to show maximum performance. Adding even more channels is not that rewarding.

You can see a characteristic performance peak on 4KB data blocks, which is the size of a data page in SSDs, and data are actually written into the memory cells in such pages. We’ve got one obvious loser here: the cut-down sample with JBOD. If you are looking for maximum performance, the 240GB RevoDrive should please you. It even outperforms the same-capacity IBIS model with its four flash controllers.

Database Patterns

In the Database pattern the SSD is processing a stream of requests to read and write 8KB random-address data blocks. The ratio of read to write requests is changing from 0% to 100% with a step of 10% throughout the test while the request queue depth varies from 1 to 256.

You can click this link to view the tabled results for IOMeter: Database pattern.

We will build diagrams for request queue depths of 1, 16 and 256.

The overall trends are clear enough, even though not so easy to distinguish in the tangle of graphs. The full-capacity IBIS products are in the lead, joined by the RevoDrive 240 GB which proves that its dual-controller design is a viable solution.

This test seems to depend not on the number of memory access channels but on the total amount of memory and, consequently, on the pool of spare blocks ready for writing. On the other hand, it wouldn’t be correct to claim that the number of memory channels is absolutely unimportant as is indicated by our JBOD sample which takes last place here.

As the request queue grows longer, the IBIS models become unchallenged. On the other side of the spectrum we see three losers: the 50GB and 80GB models which have two flash controllers and a reduced number of memory chips connected to each, and the RevoDrive which we switched into JBOD mode.

Winding up this part of our tests, we will build diagrams showing the performance of each SSD at five different request queue depths.

Comparing the performance of the different-capacity IBIS series products, we can see that the models with reduced number of flash memory chips have a performance hit at high percentages of writes. This is especially clear with the 100GB model in which each SF1200 controller has only four, not eight, channels to access flash memory. But when it comes to reading, these SSDs produce similar results.

The RevoDrive series isn’t as good as the OCZ IBIS although did well against conventional SSDs in our earlier tests. It is only the senior RevoDrive models with a full set of flash memory chips per controller that can compete with the IBIS series. The junior ones are slower, both at writing and reading. The RevoDrive x2 is somewhere in between. It might be able to compete with its quad-controller cousins if it were not for its reduced capacity.

It is the RevoDrive we switched into JBOD mode that proves to be the weakest link. It is slower than its cousins everywhere, proving the fact that modern SSDs benefit greatly from doubling the number of memory access channels by means of an internal RAID controller.

Web-Server, File-Server Patterns

SSDs are tested under loads typical of servers here. The names of the patterns are self-explanatory. The results are presented as performance ratings which are calculated as the average speed of the tested SSD at every load.

These tests suggest that when the load doesn’t include many write requests, SSDs with four flash controllers are preferable. But when the load is far from pure reading, it is the total storage capacity of an SSD that's important unless you have some queer product like our custom-made JBOD-based sample. Why? Because when there are a lot of writes to be done, a larger pool of spare memory blocks provides a higher overall performance.

As a side remark, it is really amazing that you can go to any computer shop nowadays and buy, for quite a reasonable sum of money, a disk capable of processing as many as 20 thousand requests per second!

Multithreaded Read & Write Patterns

The multithreaded tests simulate a situation when there are one to four clients accessing the disk at the same time – the clients’ address zones do not overlap. We will discuss diagrams for a request queue of 1 as the most illustrative ones. When the queue is 2 or more requests long, the speed doesn’t depend much on the number of simultaneously running applications. You can also click the following links for the full results:

The multithreaded load produces interesting results. With no read request queue, we see that the real speeds of the SSDs (we will also see them shortly in FC-Test below) are very far from their maximum ones. Why? Because they are just not loaded heavily enough to show their very best. In our sequential read test the SSDs were nearly as fast as specified at a queue depth of 4 requests, but here, with no queue at all (and that’s the scenario we have when a single user is accessing a single file), we see them deliver only one third or one fourth of their maximum. The funny consequence is that first place goes to the outsider of this test, our custom-made JBOD sample.

But when there are two data threads, the SSDs can read them in parallel and double the speed. The only model that doesn’t speed up is the same JBOD-based sample.

When reading four data threads, the IBIS series go ahead. Even the IBIS model with the fewest number of memory chips proves to be faster than the highest-capacity model with two flash controllers.

We’ve got a different picture at multithreaded writing. There are no performance hits when the number of data threads increases, yet there is no performance growth, either. So, the SSDs do not seem to find this load too easy. It is the capacity rather than the number of memory channels that becomes the crucial factor here. This is indicated by the quad-controller 100GB products being outperformed by the 240GB RevoDrive.

Performance in FC-Test

For this test two 32GB partitions are created on the drive and formatted in NTFS. A file-set is then created, read from the drive, copied within the same partition and copied into another partition. The time taken to perform these operations is measured and the speed of the drive is calculated. The Windows and Programs file-sets consist of a large number of small files whereas the other three patterns (ISO, MP3, and Install) include a few large files each.

You should be aware that the copying test not only indicates the speed of copying within the same disk but is also indicative of the latter’s behavior under complex load. In fact, the tested SSD is processing two data threads then, one for reading and another for writing.

This test produces too much data, so we will only discuss the results obtained in the Install, ISO and Programs file-sets. You can use the following link to view the full results for FC-Test.

We want to note how efficient the OS’s data caching mechanisms are. It is because of them that the SSDs are faster in the Install pattern than in the ISO one. If we had more system memory (some of our readers ask us to add it to our testbed), the whole file-set might fit into Windows 7's file cache, resulting in unrealistic speeds.

Here, we can see that the quad-controller models have an advantage over the products with the same capacity but a different number of controllers. The RevoDrive x2 is expectedly as fast as the same-capacity IBIS. Our JBOD-based sample is the only disappointment. It proves the importance of multichannel access to flash memory and this point is also confirmed by the junior RevoDrive models which lack some of the memory chips and do not use all of the memory channels of their SF1200 controllers.

We’ve got rather surprising results at reading. It is good that the quad-controller products are as fast as 400 MBps. This is below their specifications, yet not as low as we might expect after their results in the multithreaded tests (with a single data thread). We guess it is due to Windows 7 algorithms that managed to perform the reading of large files in multiple threads. But why did the IBIS 160 GB perform so poorly with the Install file-set? We guess the test caught this SSD reordering its memory blocks. And the most inexplicable thing is that the 50GB RevoDrive is ahead of its 80GB cousin.

We don’t see anything new in the copying test. The SSDs are limited by their write speeds which are less affected by caching. The load itself is all right for them because, as we already know, these SSDs are fond of multithreading.

Performance in PCMark Vantage

Compared with the previous versions, the Vantage version of PCMark is more up-to-date and advanced in its selection of subtests as well as Windows Vista orientation. Each subtest runs ten times and the results of the ten runs are averaged.

Here is a brief description of each subtest:

Basing on these subtests, the drive’s overall performance rating is calculated.

PCMark suggests that quad-controller SSDs of small storage capacities are the ideal option for home computers: the 100GB RevoDrive x2 and IBIS share top place with very close results. They are followed not by the rest of the quad-controller products but by the RevoDrive series whereas the IBIS could only outperform the outsider of this test session, the RevoDrive in JBOD mode. We really can't explain these standings but the trends are too clear to be disregarded as a mere coincidence.

Defragmentation

Next goes our homemade test of defragmentation speed. We created a very defragmented file system on a 32GB partition of a disk by loading it with music, video, games and applications. Then we saved a per-sector copy of it. Now we copy that partition to the disk we want to test. We run a script that evokes the integrated defragmenter of Windows 7 and marks the time of the beginning and end of the defragmentation process. For more information about this test, you can refer to this article.

We must remind you that defragmentation is useless and even harmful for solid state drives due to their operation principles. However, we use this test as it allows to benchmark their performance at a rather peculiar load that involves both reading and writing of small data blocks.

This test often produces unpredictable results but this time around it agrees with what we've seen earlier. The quad-controller models are good, especially if each of their SF1200 controllers has all of its eight channels filled with memory chips. The only surprise is the large difference between the RevoDrive x2 and the IBIS 100 GB.

Performance in WinRAR

Now we are going to show you one more interesting test in which we use WinRAR version 3.91 to compress and then uncompress a 1.13GB folder with 8118 files in 671 subfolders. The files are documents and images in various formats. These operations are done on the tested drive. This test depends heavily on CPU performance, but the storage device affects its speed, too.

The 18 seconds of difference between the best and worst runners amount to less than 3%. Although we test each SSD under the same conditions, it’s hard to say whether some variations in speed are due to their performance or to some random factors. So, in this test we are inclined to view all the SSDs as delivering the same performance.

The difference is large when the SSDs are used for unpacking the archive. We’ve got a group of leaders including large-capacity SSDs and a pair of losers with too few memory access channels.

Conclusion

First of all, we want to note the fact that the existence of these three product series is indeed justifiable. The OCZ IBIS series includes top-end products designed for users who need to get a large number of fantastically fast disks into a server rack. You may only want to buy a single such SSD if you are going to add a second one to it in the near future or if your system case cannot accommodate an OCZ RevoDrive x2, which provides the same performance but doesn’t occupy a 3.5-inch bay. And if you don’t care about having a record-breaking performance, but want high storage capacity instead, you may want to prefer an ordinary RevoDrive as you will surely get a very high speed even with this dual-controller SSD. But do not switch its integrated RAID controller into JBOD mode as we did in our experiment!

We would like to award OCZ IBIS with our Ultimate Innovation title:

As for the question what is better for an SSD, the total storage capacity or the number of memory access channels, the answer is rather trivial: both. However, the speed of writing depends more on the storage capacity whereas the number of channels has a larger effect on the SSD’s read speed, especially when there is a request queue or multiple threads of data to process.