by FastSite
07/04/2001 | 12:00 AM
Adobe Photoshop can be considered the most popular graphics application today. Besides, we suppose it to be the most self-sufficient of all the professional software, in other words, many Photoshop users work only with this application and do not need to engage any other programs. This allows us to assume that checking the performance of different hardware/software configurations in Photoshop will be of interest to many people. This is one of the reasons why many frequently used Photoshop operations are included into all complex benchmark packages, such as Winstone, Winbench, Sysmark. And most testers provide the results obtained in these particular benchmarks. Is it good or bad? Of course, using complex benchmarks is pretty convenient for the guys running all the tests, because they don't need to interfere with the testing process. All the testers need to do in this case is to assemble the system and to click the "Start" button. Again, it is very convenient and absolutely correct to compare different hardware pieces with one another considering the results obtained by running the same benchmarks and to make up some databases with the test results for different hardware components, which can be very helpful in the future. However, although the mentioned benchmark sets feature some indisputable advantages, they are also not free from some unpleasant drawbacks. By the way, the drawbacks, we are going to touch upon in this article are just the reverse side of the mentioned advantages.<%BANNER[article]%>
First of all, all the testing packages do not include only Photoshop benchmarks and hence aren't optimized for Photoshop. As a result, the set of Photoshop functions and filters selected is far from being complete and in many cases it is just not optimal at all. Moreover, the developers of testing packages like that usually do their best to make sure that their offspring will be able to run smoothly in absolutely different hardware/software configurations. And you do understand that the difference between these configurations can be incredibly significant. Since all the testing packages are arranged so that they could work fine on any hardware platform, even on a not very up-to-date one, the benchmarks included into these packages are somehow averaged and reduced to a common "hardware" denominator. For instance, basing on the memory size, say, 64MB. This way the initial file, which will be processed by Photoshop part of the testing package, gets smaller. And since the file is smaller, less time is needed to carry out all the operations. As a result, when the tests are run on the today's fast systems, the influence of measuring errors appears unacceptably great. For example, if you need to increase the image size from 1MB up to 25MB it will take your system built on 1GHz+ CPU only 1 (!) second. However, for the operation to be carried out correctly much more memory is required: 512MB. And as we have just said above, all the complex benchmarks are optimized for a much smaller memory size. Therefore, the similar operation is most likely to be undertaken for a 100KB file to be modified into 1MB one instead of 1MB file into 25MB. On the today's platforms this operation will hardly take more than one tenth of a second. Add the measuring error to this value and you will understand why Pentium III 1GHz will be able to easily beat a much more powerful Athlon 1.33GHz according to results obtained in the Photoshop part of the benchmarks package. And this "phenomenon" will be explained not by some secret optimization, but by a simple mistake caused by the measuring error and incomparably short time required for the considered operation. By the way, since we came to speak about measuring errors, we have to stress that Photoshop is a very unstable benchmark, i.e. it can show very different results for one and the same hardware configuration depending on the operations carried out before testing in Photoshop. In other words, if it doesn't require much effort to achieve some correct results in 3ds max, for instance, then testing in Photoshop is quite far from being a trivial task. You need to restart the system every time after the test is completed and run the same tests for 2-3 times in order to get some more or less exact average results. Otherwise, you will get at least up to 20% performance difference for testing with larger files. As for smaller files, testing modern platforms with them doesn't make any sense at all, because the absolute error will be several times greater than the time spent even on the most complicated operations.
Well, we tried to explain why you shouldn't take the results obtained in the Photoshop part of any test packages seriously. However, besides test packages, there are also the so-called scripts, i.e. successions of actions for Photoshop operations. These scripts can boast a great advantage over the test packages: they are optimized for Photoshop, since they are intended only for testing in this application. You can easily find scripts with the initial image of any size you like. Some progressive scripts can also repeat different operations several times. However, all the scripts run without a single system restart, because you can't add a system restart command into a Photoshop script. And without restarting the system, its operation memory very soon appears occupied with some "bits" remaining from the previous actions, which definitely tells on the results obtained. You can check this out very easily. Create a file of the maximum size, which can be operated on your system without requesting the HDD and apply to it the Gaussian Blur filter (with any settings), Undo, Gaussian Blur (with the same settings), again Undo, etc. 5 times, for instance. You will notice that the second cycle takes less time than the first one, and the third on - less than the second one. It means that Photoshop saves partially in the cache the results of the user's actions. That is why it doesn't make any sense to measure the same parameters without rebooting the system beforehand, which is never done in Photoshop scripts. But this is not the end. If this kind of caching has a positive effect on the time required for the same type of actions, the effect on the time required for a different action will be just the opposite. First of all, caching occupies some memory and it may turn out insufficient for the new operation to be carried out, and secondly, sorting out and updating the already cached data also requires some time. In case of smaller files, it won't be that noticeable. However, if you keep working with some medium sized files for a while, your system may get into a real stupor, when a considerable part of the memory used suddenly turns out virtual, i.e. much slower than the operation memory. It means that RAM is more than enough for a single operation over a medium size file, but as it comes to a series of operations, the memory gets stuffed with too much cached data and shifts to a swap-file. That is why many professional users working with large files now and then run special programs cleaning the memory. Therefore, you see that testing with scripts doesn't have any practical value at all. You can also download any script of the kind, run it several times and compare the results obtained: the difference will be great, believe us. Again we would like to stress that the problem of getting correct measurements is especially acute for modern systems testing.
So, in our case we decided not to use any scripts and benchmarks packages. We measured the time required for each operation to be completed with the help of a special function, then rebooted the system and ran the same test two more times (also with system rebooting). The obtained results were averaged and then taken for the database.
Before passing over to testing some hardware platforms, we have to find out which operation system suits best of all for Photoshop. It is very important, because as you know, choosing the proper OS may sometimes provide a performance increase almost as significant as in case of shifting to a more powerful hardware system. We considered Windows 98, Windows NT 4.0 SP6 and Windows 2000 SP1. The results for all three are given in the table below. Just for your reference, we would like to point out that in this section we won't describe the benchmarks in detail. We will provide only the names of Photoshop filters and operations. The detailed description of all these tests will come below in the Testing Hardware Configurations.
| Win98 | Win2000 | WinNT | |
|---|---|---|---|
| Gaussian Blur | 7 | 5.8 | 6.7 |
| Smart Blur | 4.2 | 4.1 | 4.1 |
| Diffuse Glow | 13.2 | 10.2 | 10.3 |
| Glass | 11.8 | 10.3 | 10.4 |
| Crystallize | 20.9 | 19.8 | 20 |
| Lens Flare | 4.5 | 3.6 | 3.9 |
| Lighting Effects | 5.9 | 5.4 | 5.6 |
| Sharpen Edges | 3.2 | 3 | 3.5 |
| Unsharp Mask | 2.6 | 1.9 | 2 |
| Chrome | 13.9 | 12.8 | 12.9 |
| Bas Relief | 12.7 | 9.3 | 11.8 |
| Water Paper | 29.6 | 22.4 | 22.6 |
| Rotate Canvas (5o) | 5.4 | 5 | 5 |
| Convert to CMYK | 12.1 | 11.8 | 11.6 |
| Convert to Lab Color | 7.1 | 6.8 | 6.9 |
As you can see, Windows NT and Windows 2000 show almost the same performance results, however, Windows 2000 still is a little bit faster. As for Windows 98, it proved dramatically slow. Since we tested the configurations with 512MB of memory and used large files for the tests, its memory distribution algorithm probably turned out less efficient than that of other operation systems. Therefore, we selected Windows 2000. However, it is not all. We also have to find the most suitable Photoshop version. Today we have to choose between versions 5.5 and 6.0. We tested both versions following the same methodology and found out that their results appear absolutely identical. So, we decided not to post the end table with all the benchmarks here. We made up our mind to test the hardware configurations in Photoshop version 6.0.
The fastness of the graphics card doesn't tell on your work in Photoshop that is why our end-task sounded as follows: which is the best mainboard + CPU + memory combination for Photoshop needs? To answer this question we took Dune.tif file from Photoshop 6.0 distributive.
The initial size of this pic is 600x600 pixels and it is saved in RGB format.
Since the time spent on each benchmark working on a 1MB file is two short (about 1sec), we increased the file size up to 3000x3000 pixels and hence up to 17MB. The time was measured with the help of a special Photoshop "timer" function. Each operation was run three times and the average result was taken. After any operation was completed, we restarted the system. In the ongoing chapters you will see the description of benchmarks together with the analysis of the results obtained, which will give you a clear idea of what platforms are most suitable for this or that type of Photoshop operations. The filters were applied with those settings, which you can see on the screenshots, if no special comment is made.
We assembled two platforms on Intel processors, namely on Pentium III 1GHz and Pentium 4 1.7GHz and two platforms on AMD Athlon 1.33GHz (one with SDR SDRAM and the other one with DDR SDRAM). Here is the detailed description of their configuration:
Platform 1:
Platform 2:
Platform 3:
Platform 4:
Besides these platforms, we also added two more ones, which could allow us to compare the pure processor performance. We took Pentium 4 and Athlon working at 1.4GHz both. These two platforms were configured as follows:
Platform 5:
Platform 6:
All the platforms features IBM DTLA HDD with 15GB storage capacity and 7,200rpm spindle rotation speed. The graphics card used was Matrox Millennium G450.
GIF is one of the most popular formats for graphics files. The initial 17MB TIFF-file was converted to GIF with the following settings:
Pentium 4 1.4GHz showed the same performance as Pentium III 1GHz, which can't be regarded as a very promising beginning for the new CPU. Besides, we should also point out that in systems with AMD Athlon processor the increase in core clock frequency seems to be absolutely inefficient, as it doesn't stimulate any performance growth. However in systems with Intel Pentium 4 processor, the core frequency increase results into a significant performance improvement, which can be seen with the naked eye.
The filters from the Blur section serve to smoothen very contrasting image segments and are really helpful for curing grainy images. Gaussian Blur filter allows setting the radius of action for this filter, within which all the pixels will be averaged. Gaussian Blur is one of the most frequently used Photoshop filters. In order to load the processor, mainboard and memory quite heavily, we set a pretty large radius. It is equal to 75 pixels, which is not very common for usual tasks.
In this test Pentium 4 1.4GHz appeared very close to Athlon 1.4GHz. Also we have to draw your attention to the fact that Pentium III 1GHz turned out just a little bit slower than Athlon 1.33GHz with SDR memory.
This is one more kind of "smoothing" filters. It allows changing far more parameters than Gaussian Blur:
As you can see, in this test Pentium 4 1.4GHz has shown something to be really proud of: it managed to surpass Athlon 1.4GHz by about 20%! Well, this is a very nice achievement for Pentium 4 CPU. However, this is only the third test. Let's see what comes next.
Here, unlike the previous benchmark, Pentium 4 fell far behind Athlon. Working at the same core clock frequency, Pentium4 appeared 15-20% slower. One interesting observation: the results shown by all the three Athlon based systems differ just a tiny bit from one another.
This filter is just ideal to model some glass wall between the image and the viewer. The settings are really diverse, which offers a user a rich choice of different effects.
Here the memory bus bandwidth doesn't have any effect on anything: Athlon 1.33GHz with SDR as well as with DDR memory performs just the same. Just as in the previous benchmark, Pentium 4 loses the game: it appears 10% slower.
Well, the name of this filter speaks for itself. As for the available settings, there is only one: cell size.
In this benchmark the performance on the contrary, depends a lot on the memory bus bandwidth. However, the situation is again not very favorable for Pentium 4: it falls 15% behind AMD processor.
Here are some settings provided for this filter:
This filter makes active use of the FPU and loads the memory bus quite heavily. As a result, the advantage of AMD Athlon CPU in floating point operation was compensated by wider memory bus of the Pentium 4 based system.
This filter allows creating a great lot of various lighting effects. It has a lot of adjustable settings, which determine this diversity.
Although this filter carries out all the calculations via FPU, the results show clearly that the calculation algorithm is optimized for Pentium III processor, which outperforms all the other CPUs except Pentium 4 with almost twice as high working frequency. By the way, this is the only filter so evidently optimized for Pentium III. Of course, you shouldn't draw any conclusions about different platforms in Photoshop basing only on the results of this benchmark, however, you should certainly bear in mind that different filters load different parts of the system with different intensity.
This filter is used basically to smoothen the borders and edges when the colors change very radically from one to another. The filter doesn't have any settings at all.
In this benchmark Athlon is just unreachable for Intel processors. We would also like to note that the gap between Pentium 4 1.4GHz and Pentium III 1GHz is really small.
This filter is aimed at sharpening the image and increasing its contrast. In other words, it does just the opposite thing to what Gaussian Blur is.
Here Pentium 4 totally beats Athlon working at the same core frequency. Frankly speaking, it is an impressive victory.
Judging by the name you understand that this filter chromes the entire image or a part of it. The settings available within this filter can't be called numerous:
Just look at the performance increase following the growth of the Pentium 4 clock frequency! As for Athlon case, the increase in core frequency as well as in memory bus bandwidth seems to have no real effect on the result. Note that Pentium 4 and Athlon, both working at 1.4GHz, perform equally fast.
This filter applies relief to the selected image.
The situation repeats what we have just discussed in the previous benchmark. Increase in Pentium 4 core clock frequency results into a significant performance growth, while for Athlon it doesn't work at all. But unlike the previous case, here AMD Athlon managed to outperform Pentium 4 working at the same frequency.
In this benchmark Pentium 4 is a little bit slower than AMD Athlon.
Extrude filter allows getting really funny effects. Something like this:
In this benchmark the settings looked as follows:
The leaders swapped places: now Pentium 4 is a bit ahead of Athlon.
Find Edges filter allows making the borders between different colors a bit sharper and more evident.
It doesn't have any settings, which could be changed manually.
Here Pentium 4 is again a dozen percents slower than its rival.
We carried out two rotations: by 1o and by 99o clockwise.
Well, in this benchmark Pentium 4 proved simply disgraceful! Pentium 4 1.7GHz was almost destroyed by Pentium III 1GHz. The latter managed to perform almost 25% faster! As for Athlon, it is almost twice as fast as Pentium 4 working at the same core frequency.
As you remember, the initial image was in RGB format. We converted it to Grayscale, CMYK and Lab Color.
So, what do we see? Again Pentium 4 is disgracefully behind Pentium III based platform. AMD Athlon is as usual beyond any competition.
As we can see from the benchmarks results, AMD based platforms are undoubtedly preferable for work in Photoshop application. In some filters Pentium 4 managed to beat Athlon, however, in most cases the situation is still not in its favor. Moreover, in most frequently used filters, such as "Convert to Other Color Systems" and "Rotate Canvas", Athlon based systems are simply unreachable. Also we found out that the memory bus bandwidth is not a serious bottleneck for Photoshop since the results shown by the Athlon 1.33GHz based system with SDR and DDR memory hardly ever differed by something over 3-5%. It seems to be one of the reasons why Pentium 4 turns out so slow here. As you remember, in another professional graphics application, 3ds max, Pentium 4 based platform with its huge memory bus bandwidth was far ahead of all the other testing participants during viewports rendering (see our article called Choosing Optimal Platform Configuration for 3D Studio MAX). This example once again proves that you should never draw any conclusions judging only by the results obtained in one single application, especially as far as the overall platform performance goes.