Case Study mp3

Case Study mp3 Soundtest / Home

A number of tests and sound examples have already been published. But many readers (or better listeners) will still have asked themselves: "Is that all true?", "Aren’t they listening right?" or "Yeah, well that won’t work for music, will it?" Although the statements with regard to mp3 have ranged from "junk" to "CD quality" and have caused thorough uncertainty among the music community, it must be made clear up front that the mp3 compression algorithm is by far not the weakest link in the chain. Rather, the influence of the surroundings, for example, or the speakers is much greater. Many analog devices such as reel-to-reel or cassette players alter the transmission to a far greater degree. DAT recorders, CD players or cable generally have significantly better characteristics.

The Criteria

Tests for cables or various CD players should not be fully relegated to the level of reading tea leaves, but there are certain common elements to mp3. The differences are minimal. We have therefore not occupied ourselves with the question as to whether mp3 is good enough for the average user, but rather if the compression algorithm is sufficient for high demands. This case is relevant if one wishes to shrink their CD collection to a few mp3 CDROM´s or if there simply is a high demand on fidelity. The question as to whether a blind test, that is without a direct comparison to the originals, can be used to detect mp3 files is only of marginal interest. After all, a bad feeling remains if one knows that distortion exists, even if it was only detected in a direct comparison.
The implementation of a systematic test can be extremely complicated due to the large number of parameters used for classification: the listeners, the music, the recording quality, the test equipment and many more. To place a limit on the hardware costs, a hifi system costing ca. $4000 was used along with high-quality Sennheiser headphones. A number of technical measurements were made along with the basic ear test. The music sources were digitally-recorded CDs from various music genres. A voice recording was also used in order to have an actual original sample in the test as well.

Elaborate preparations were necessary for the actual test. This made for a lot of time for speculating. Many listeners don’t find mp3 recordings significantly worse. Which sampling rate should be used for testing? Too many variations would make the tests extremely complex, so 128kBit/s was chosen. Most files on the Internet are of this form, as it is a good compromise between data compression (ca. 11-fold) and quality. This would allow, for example, all 9 Beethoven symphonies to be recorded on a single CD! A very good, comprehendible contribution on which algorithms exactly make this high compression rate possible without compromising the music is given in [1,2].

Figure 1 : Sensitivity of the human ear versus frequency.

Which music should be used? The best choice would be something one knows well and which is available in good quality. It is impractical to listen to a complete work for comparison reasons. A faster and more conclusive method is to switch directly back and forth. The sampling times may not be too short however, since it is the development, the contrast in a piece that is of interest. For the soundtest, the CD originals were stored as WAV files and stored along with the mp3s on a CDROM. This was all done using the freeware programs Cdex Version 1.10 and the Dema mp3 software player.
Quantitative statements can hardly be made from merely listening to a piece. Therefore, a number of technical signals were also generated as WAV files (with CoolEdit96) and then compressed. The difference between these results directly in the distortion. A 1kHz sinus burst with different sampling rates and a two-tone signal were used. The first tests primarily the dynamic qualities of the mp3 compression and the second the intermodulation distortion. The intermodulation ratio for the original is better than 90dBc.

Very meticulous tests are conceivable. For example, blocking signals could be superimposed on normal music which could then be mp3 compressed, and then the blockers subtracted. This would present real problems for mp3, because large blocking signals would cover details in the original. Mp3 would then fail to include these details in compression and they would be missing in the end, which would most likely be detectable since the subtraction would eliminate the covering effect. Only linear and time invariant systems are completely immune to such effects, and mp3 is not such a system. Neither is an amplifier, a DA converter or even a loudspeaker.

There were also certain expectations for the indirect comparison of loudspeaker vs. Headphones. A set of headphones cannot deliver the same natural sound quality as loudspeakers can, but there are $250 headphones which have better parameters such as pulse response, noise detection, etc. than even $5000 speakers. Headphones also make it possible to implement a volume level above and beyond room level without causing problems with the neighbors.

In some ways, the mp3 test is much simpler than loudspeaker tests. The hardware is always the same. The original and mp3 version are subjected to the same imperfect equipment such as CD burner, CD player, speaker, amplifier, etc. There is no advantage for either variation. At the most, the differences might become blurred. Only very bad equipment might actually cause negative effects (e.g. frequency peaking) to recur.

The Music
1st Test
The Soundtest
Detective Needed
Conclusion

Literature:

[1] http://www.trail-n-error.de/mp3theorie1.htm
[2] http://hellgate.hs-bremen.de/mp3/artikel.htm
[3] http://www-stud.fht-esslingen.de/~alkoit00/mp3enc/
[4] http://www.raum.com/mpeg/reviews_quality.html
[5] C. Meyer, ct´ Heft 6/00, S. 92ff
[6] ct´ Heft 22/00
[7] http://www.mp3-tech.org

To Top - To Homepage - German Version