by Ilya Gavrichenkov
07/31/2012 | 12:02 PM
The three basic components of each solid state drive that fully describe its consumer properties are its controller, flash memory and firmware. Two of them are hardware and the last one is software, which means that the manufacturer has some control over its product even after the latter's release. The firmware can be easily updated by the end-user with special tools offered by SSD makers. New firmware can change an SSD’s behavior completely as it contains the algorithms that dictate that behavior. Thanks to that, some bugs or imperfections can be corrected without withdrawing the product from the market. Firmware updates with optimized internal algorithms can also bring about some performance benefits.
Well, it seems that the manufacturers have come to abuse this opportunity by releasing non-optimized products and polishing their firmware off afterwards. It is quite normal for a new SSD to get several firmware updates within its first months on the market which correct its critical (or not very critical) bugs and introduce some changes, often substantial, into its behavior. This situation transforms early buyers, who presumably want a new and stable product with known specifications, into a kind of beta testers. Fortunately for the manufacturers, this hasn’t been an irritating factor for users so far, especially as SSDs are mostly used as system disks rather than for storing important data. Still, we guess that ideally SSDs should have the same stable firmware throughout their entire life cycle.
Many companies have been noticed to polish their SSD firmware off after a product release. It would be even easier to name those who don’t do that. But today we are going to talk about OCZ with its flagship Vertex 4 series which is based on OCZ’s very own Indilinx Everest 2 controller. Not so long ago we tested this SSD with version 1.4 firmware which had changed its speed characteristics considerably, but recently the firmware has been updated to version 1.5. And again it brings about substantial changes in performance, making the Vertex 4 quite an unknown product. This alone is a good reason to test the series once again, yet we don’t want to repeat our previous review completely. So, this time around we choose another pair of Vertex 4 models and want to test the most popular capacities of 128 and 256 gigabytes.
We already wrote a review of the Vertex 4 series and the Indilinx Everest 2 controller, so you can refer to it for details on them.
We can just remind you that the Vertex 4 series includes top-end SATA III SSDs from OCZ. They are based on the Indilinx Everest 2 controller and equipped with conventional 25nm synchronous flash memory with ONFI interface. So, it is the controller that makes the Vertex 4 fast, especially in synthetic benchmarks and at write operations. The Everest 2 incorporates two ARM cores and, by some evidence, is closely related to Marvell’s controllers.
For this test session we are going to use firmware 1.5 which became available just a few weeks ago. The firmware leads to a substantial change in SSD specs, a second such change over the short market life of the Vertex 4 series. And like the last time, OCZ has managed to increase the speed of sequential operations. The following table tracks the evolution of the Vertex 4 specs since the announcement of the series:
Besides the increased speed, firmware 1.5 is expected to correct the issue with the SMART Remaining Life attribute, improve compatibility with various RAID controllers and perfect the garbage collection technique.
We’ll first check out our Vertex 4 SSDs with new firmware in the synthetic benchmark AS SSD. It will help us see if there is really any performance growth as promised by the manufacturer.
OCZ Vertex 4 128 GB
OCZ Vertex 4 256 GB
Indeed, firmware 1.5 makes the SSDs faster at sequential operations and, fortunately, without any side effects. They are as fast as before processing 4KB data blocks, for example.
So, here is a comparison of the current specs of the OCZ Vertex 4 models:
Take note that the evolution of specs doesn’t involve the junior 64GB model. Remaining as slow as before, it looks somewhat out of place among its faster cousins. So, you should keep it in mind that the junior Vertex 4 is inferior to the higher-capacity models in speed.
On the other hand, the models with capacities of 128 gigabytes and higher have improved as much as to be formally superior to the best-in-class competitors such as Intel SSD 520, Plextor M3 Pro or Corsair Performance Pro. That seems like a good reason for us to re-test them.
For our today’s SSD test session we put together a system on an Intel H67 based mainboard. This chipset provides support for two SATA 6 Gbit/s ports, which we use to connect the tested SSDs.
For our today’s test session OCZ Technology provided us with the two most popular Vertex 4 models – SSDs with 128 GB and 256 GB capacity. And now these weren’t the engineering samples, but the actual mass production units. We are going to compare them against several products with the same storage capacity based on controllers other than Indilinx Everest 2. SandForce products with 120 GB and 240 GB capacities are represented by typical SSDs with 25 nm synchronous ONFI memory (Corsair Force Series GT – analogue to OCZ Vertex 3) and typical SSDs with 25 nm asynchronous memory (Corsair Force Series 3 – analogue to OCZ Agility 3). Marvell 88S9174 controller will be represented by two Crucial m4 SSDs with 128 GB and 256 GB storage capacity built with 25 nm memory. Moreover, we also included the unique SandForce based Intel 520 240 GB SSD featuring exclusive firmware and common 25 nm synchronous memory, and Corsair Performance Pro SSD with 256 GB storage capacity built with Marvell 88S9174 controller and 34 nm Toggle Mode flash memory.
Overall our testbed was configured as follows:
We use CrystalDiskMark 3.0.1 benchmark to test the random- and sequential read and write speed. This benchmark is convenient to work with as it can measure the speed of an SSD with both incompressible random and fully compressible recurring data. This feature is important for testing SSDs based on SF-2281/2282 controller, which tries to compress the data before writing it into memory. So, there are two numbers in the diagrams below that reflect the maximum and minimum SSD speed. The real-life performance of an SSD is going to be in-between those two numbers depending on how effective the controller data compression is.
Note that the performance tests in this section refer to SSDs in their “Fresh Out-of-Box” state (FOB). No degradation could have taken place yet.
In our previous test of Vertex 4 SSDs with firmware 1.4 we saw them deliver a somewhat lower speed of sequential reading compared to top-end SSDs with SandForce and Marvell controllers. Things have changed now. Firmware 1.5 makes the Vertex 4 series as fast at such operations as their competitors.
However, during our tests we could see that the sequential read speed was not always at the stable level above 500 MB/s. It could drop by 50-100 MB/s, following some internal logic of the controller and making the Vertex 4 slower than expected. Unfortunately, we didn’t see any pattern in such slowdowns, so you just have to be aware that the Vertex 4 may turn out to be somewhat worse than shown in the diagrams in certain scenarios.
Anyway, the numbers suggest that the Vertex 4 series claims to be the fastest available right now, especially at write operations and when the request queue is long. This refers not only to the 256GB but also to the 128GB model, by the way. The latter is a worthy representative of the series, being only inferior to its 256GB cousin in terms of the sequential write speed. SandForce-based SSDs have a stronger correlation between performance and capacity than Everest 2 based ones.
Unfortunately, SSDs are not always as fast as in their “fresh” state. In most cases their performance goes down after some time and in real life we deal with completely different write speeds than what we see on the diagrams in the previous chapter of our review. The reason for this phenomenon is the following: as the SSD runs out of free pages in the flash memory, its controller has to clear memory page blocks before saving data into them, which causes substantial delays. Although, modern SSD controllers can alleviate the performance drop by erasing unused flash memory pages ahead of time, when idle. They use two techniques for that: idle-time garbage collection and TRIM.
Of course, users are more interested in the consistent performance of their SSDs over a long period of time rather than the peak speed they are going to see only during the initial short-term usage period, while the drive is still “fresh”. The SSD makers, however, declare the speed characteristics of “fresh” SSDs for marketing reasons. That’s why we decided to test the performance hit that occurs when a “fresh” SSD becomes a “steady” one.
To get a complete picture of SSD performance degradation we ran special tests based on the SNIA SSSI TWG PTS (Solid State Storage Performance Test Specification) methodology. The main idea of this approach is to measure write speed consecutively in four different cases. First we measure the “fresh” SSD speed. Then we measure the speed after the SSD has been fully filled with data twice. The third test occurs after a 30-minute break during which the controller can partially restore performance by running the idle-time garbage collection. And finally, we measure the speed after issuing a TRIM command.
We ran the tests in synthetic IOMeter 1.1.0 RC1 benchmark, where we measured random write speed when working with 4 KB data blocks aligned to flash memory pages at 32 requests queue depth. The test data were pseudo-random.
We had no complaints about the TRIM command implementation in the Vertex 4 series before and firmware 1.5 doesn’t change anything in this respect. The Vertex 4 SSDs can both use TRIM to bring their performance back to the original out-of-box state. The new firmware complements this with background garbage collection. Now, when idle, Vertex 4 series SSDs can partially restore their performance in OSes that lack TRIM support. However, only SSDs with Marvell 88S9174 can really be brought back to their original performance by means of that technique.
Since the characteristics of most SSDs do change once they transition from fresh out-of-the-box state into steady state, we measure their performance once again using CrystalDiskMark 3.0.1 benchmark. The diagrams below show the obtained results. We use random data writing and measure only performance during writes, because read speed remains constant.
The adequate TRIM implementation makes the Vertex 4 series even more attractive compared to SandForce-based SSDs which slow down after having been used for a while. They are going to enjoy a large advantage in steady-state performance when it comes to writing with a long request queue and to sequential writing.
Well, it remains to be seen if the Vertex 4 series can convert their high results in the synthetic benchmark into high performance in real-life applications. So, let’s run some tests that simulate this kind of load.
The popular PCMark 7 contains an individual disk subsystem benchmark. It is not a synthetic test, but is based on real-life applications. This benchmark reproduces typical disk usage scenarios and measures how fast they are completed in popular applications. Moreover, the disk access commands are not executed as a steady uninterrupted flow, but in a more realistic manner – with certain pauses caused by the need to process the data. The benchmark generates an overall disk subsystem performance rating as well as speed readings in MB/s in individual usage scenarios. Note that the absolute speed in these scenarios is not too high because of the above mentioned pauses between individual input/output operations. In other words, PCMark 7 shows you the speed of the disk subsystem from the application’s point of view. Numbers like that show us not only the pure performance of an SSD, but mostly how big of a performance gain a certain SSD can guarantee in real life.
We ran PCMark 7 on “steady” SSDs, which is what they are going to be in actual computer systems most of the time. Their performance in this case is affected not only by their controller or flash memory speed but also by the efficiency of their internal algorithms that fight performance degradation.
The PCMark 7 score is a good guide for users who want to get a comprehensible evaluation of an SSD’s performance without delving into technicalities. And PCMark 7 corrects our first impression about the Vertex 4 series. Despite their excellent results in the synthetic benchmarks above, these SSDs are far from brilliant here. They are inferior to the fast SandForce-based SSDs as well as to the top-end products with the Marvell 88S9174 controller and take a medium position in the diagram.
So, firmware 1.5 isn’t of much help for the Vertex 4 series at real-life loads. The SandForce and Marvell 88S9174 controllers have also received firmware updates, so the overall situation has remained the same as in our previous review.
Now let’s check out the individual tests to get a more detailed picture of what our SSDs are capable of under various types of operational load:
The Vertex 4 series can’t boast the impressive results them showed in the synthetic benchmarks in any of the PCMark subtests. Unfortunately, they are not so good in real-life usage scenarios, according to PCMark. Let's see what the other benchmarks have to say.
Intel NASPT is another disk sub-system test that uses real-life usage scenarios. Like PCMark 7, Intel NASPT reproduces predefined disk activity traces and measures how fast they are executed. However, the default traces are designed for network attached storage devices rather than for SSDs. Therefore during our test session we replace them with the specially developed SSD Benchmarking Suite which offers more relevant usage scenarios such as compressing and decompressing files, compiling large projects, copying files and folders, loading 3D game levels, installing software, batch-processing photos, searching a digital library for data, mass-launching applications, and transcoding video.
Like PCMark 7, this benchmark gives us a true-to-life illustration of disk subsystem performance. Here the SSDs are again tested in their “steady” state.
Firmware 1.5 improves the performance of the Vertex 4 series in Intel NASPT. They used to fall behind the fastest SSDs with LSI (SandForce) and Marvell controllers but now they can leave quite a lot of their opponents behind. In fact, the Vertex 4 series is only outperformed by the premium solutions: Corsair Performance Pro and Intel SSD 520. So, the improved specs do reflect increased performance of Vertex 4 SSDs.
The detailed INASPT results help us see what usage scenarios are the most suitable for our today’s testing participants. Take note that the data-transfer rate is higher than the SATA III interface bandwidth in some subtests. That’s because INASPT is a high-level test that uses standard Windows functions to access the disk subsystem. As a result, the OS caching mechanisms also affect the results.
Since OCZ focuses on improving the speed of sequential operations, it is in such scenarios that the Vertex 4 SSDs show their best. They can deliver impressive performance at copying or writing large files. The downside is that they are far from brilliant at other loads.
We use AS SSD version 1.6.4237.30508 test to benchmark the speed of copying files within a single partition the size of the whole SSD. The SSDs are tested in their steady state.
The Vertex 4 series could be expected to win this test. As we noted earlier, they are very good at writing, which explains their high speed of copying.
A serious problem with earlier firmware versions for the Vertex 4 series was the performance hit these SSDs suffered when filled with data by more than 50%. Their write speed would plummet down, which was especially conspicuous with the 128GB model. The higher-capacity models were affected as well, though.
This problem wasn’t observed with other SSDs, so many customers preferred not to buy OCZ products. After all, an SSD is supposed to get filled with data, isn't it? Unfortunately, OCZ’s statements on this issue were rather vague. The company claimed that the Vertex 4 firmware could work in two modes: Performance and Data Storage. In the first mode the SSD delivers its maximum performance. In the second mode, there are other priorities, and the performance hit observed when the SSD is filled up by more than 50% is due to switching from the first to the second mode.
End users were concerned that the Vertex 4 series was going to be not as fast in real-life applications as in benchmarks which are usually run on empty SSDs. However, Vertex 4 SSDs turn to be more sophisticated than they seem and can get fast even when filled up with data.
Firmware 1.5 is meant to improve the switching between the two operation modes. The enhanced garbage collection algorithms are in fact a revised performance mode which is now enabled for more scenarios.
So, we’ve carried out some special tests to check out this performance hit issue with firmware 1.5. We've taken the 128GB model which used to suffer the most from that problem.
Firmware 1.5 seems to change nothing at first. Benchmarking the sequential speed of the SSD on its entire capacity, we eventually see the old problem. The speed plummets down when half the capacity is filled with data.
The performance hit is really awful. The SSD can only deliver one fifth of its original sequential write speed.
We can see that especially clear with the AS SSD benchmark. Here's the performance of an empty Vertex 4 128GB:
And here’s what we have after we wrote a single 64GB file on it (this equals 53.7% of the SSD’s full capacity):
We guess nobody is going to buy an SSD with such a low speed. The sequential write speed is only one fifth the speed of the empty SSD. The speed of writing 4KB data blocks at a long request queue is only one third of the original one, too. The sequential read speed is reduced by half. So, that's bad, but not as bad as it seems. After about half an hour of being idle, the SSD can recuperate as is confirmed by the AS SSD benchmark:
The SSD is still filled with data by more than 50%, but its speed is almost as high when it was empty. Thus, Vertex 4 SSDs with firmware 1.5 indeed work in performance mode most of the time, preventing any reduction in speed. The SSD is going to slow down again after the next half of the remaining capacity is filled up, but it will recuperate after a while in the same manner.
So, even though the performance hit is still an issue due to the SSD’s internal algorithms, it only remains such an issue for a limited period of time. In ordinary usage scenarios users of Vertex 4 SSDs (with the new firmware) can hardly notice that problem.
Moreover, the new firmware eliminates the performance hit altogether with Vertex 4 SSDs larger than the 128GB model. For example, here is the sequential write graph of the Vertex 4 256GB:
Instead of one performance hit at 50% we can see two bends in the graph. There’s a small reduction in write speed when one fourth of the disk capacity is filled with data, and then there’s a more substantial performance hit at 75%. The drive’s behavior is overall the same, though. It slows down but for a short while. After half an hour of being idle, the SSD returns to its original performance.
Firmware developers do not disclose any details about their work, so we don’t know for sure what provokes this behavior of Vertex 4 SSDs. However, we can suppose that the Everest 2 controller separates all flash memory within an SSD into fast and slow. It writes data into the fast part first, switching to the slow one only after the fast part is all used up. When idle, the controller automatically moves data from the fast into the slow part of the SSD. This theory may be true if the Everest 2 has different latencies programming the first and second bits in each MLC NAND cell.
However, this technology would have a downside. It would increase the load on the flash memory by constantly moving data in it. Unfortunately, the SMART information reported by the Vertex 4 is far from exhaustive, so we can’t objectively measure its write amplification value. Our apprehensions about quick memory wear were only confirmed by the fact that the health attribute of our 128GB disk decreased from 100 to 98%.
Considering the small amount of data written during our tests, we suspect the Vertex 4 to have a shorter service life than other SSDs. We’ve only met with one more instance of such a quick deterioration of a disk’s health during our tests. It was the OCZ Petrol series which is known to have a short service life. On the other hand, the number of reallocated sectors of flash memory remained at zero. Moreover, the health of the Vertex 256GB disk remained at 100% throughout our tests.
We were very pleased with the OCZ Vertex 4 series, based on the Everest 2 controller, in our previous review. Being rather affordable, these SSDs almost matched the performance of top-end SSDs with other controllers and seemed to lack just a little to claim absolute superiority in the consumer segment. Fortunately, OCZ keeps on improving this product by updating its firmware.
Firmware 1.5 is the next step in the optimization process. It improves the specs and eliminates the weakness of the Vertex 4 series, which was their low speed of sequential reading. Now the Vertex 4 specs look impressive and this series can beat any competitor in synthetic benchmarks.
However, the performance benefits are not that obvious at complex loads typical of real-life usage scenarios. The Vertex 4 series still remains a midrange offer, being inferior in performance not only to such recognized leaders as Corsair Performance Pro and Intel SSD 520 but also to popular SandForce-based SSDs with synchronous memory. In other words, the new firmware doesn’t lift the Vertex 4 up to the absolute leading position just yet.
What it does, though, is that it resolves the problem that used to bother Vertex 4 customers much more than benchmarking results. The performance of OCZ’s new SSDs used to decline after more than half the capacity of the disk was filled with data. This effect occurred because of the internal algorithms of the Everest 2 controller, but the new firmware helps mitigate it. The garbage collection technique implemented in the Vertex 4 can quickly restore the disk's performance to its original level, making this problem unnoticeable in real-life usage scenarios.
So now, four months after its release, the Vertex 4 series seems to have got rid of serious downsides. There are no unresolved problems with its hardware and firmware and its performance weaknesses have been corrected. Now, these SSDs seem to have become a very attractive option, especially as OCZ offers them at highly appealing prices and with a 5-year warranty.