A number of tests and sound examples have already been published. But many readers (or better listeners) will still have asked themselves: "Is that all true?", "Arent they listening right?" or "Yeah, well that wont work for music, will it?" Although the statements with regard to mp3 have ranged from "junk" to "CD quality" and have caused thorough uncertainty among the music community, it must be made clear up front that the mp3 compression algorithm is by far not the weakest link in the chain. Rather, the influence of the surroundings, for example, or the speakers is much greater. Many analog devices such as reel-to-reel or cassette players alter the transmission to a far greater degree. DAT recorders, CD players or cable generally have significantly better characteristics.
The Criteria
Tests for cables or various CD players should not be fully
relegated to the level of reading tea leaves, but there are
certain common elements to mp3. The differences are minimal. We
have therefore not occupied ourselves with the question as to
whether mp3 is good enough for the average user, but rather if
the compression algorithm is sufficient for high demands. This
case is relevant if one wishes to shrink their CD collection to a
few mp3 CDROM´s or if there simply is a high demand on fidelity.
The question as to whether a blind test, that is without a direct
comparison to the originals, can be used to detect mp3 files is
only of marginal interest. After all, a bad feeling remains if
one knows that distortion exists, even if it was only detected in
a direct comparison.
The implementation of a systematic test can be extremely
complicated due to the large number of parameters used for
classification: the listeners, the music, the recording quality,
the test equipment and many more. To place a limit on the
hardware costs, a hifi system costing ca. $4000 was used along
with high-quality Sennheiser headphones. A number of technical
measurements were made along with the basic ear test. The music
sources were digitally-recorded CDs from various music genres. A
voice recording was also used in order to have an actual original
sample in the test as well.
Elaborate preparations were necessary for the actual test. This made for a lot of time for speculating. Many listeners dont find mp3 recordings significantly worse. Which sampling rate should be used for testing? Too many variations would make the tests extremely complex, so 128kBit/s was chosen. Most files on the Internet are of this form, as it is a good compromise between data compression (ca. 11-fold) and quality. This would allow, for example, all 9 Beethoven symphonies to be recorded on a single CD! A very good, comprehendible contribution on which algorithms exactly make this high compression rate possible without compromising the music is given in [1,2].
Figure 1 : Sensitivity of the human ear versus frequency.
Which music should be used? The best choice would be something
one knows well and which is available in good quality. It is
impractical to listen to a complete work for comparison reasons.
A faster and more conclusive method is to switch directly back
and forth. The sampling times may not be too short however, since
it is the development, the contrast in a piece that is of
interest. For the soundtest, the CD originals were stored as WAV
files and stored along with the mp3s on a CDROM. This was all
done using the freeware programs Cdex Version 1.10 and the Dema
mp3 software player.
Quantitative statements can hardly be made from merely listening
to a piece. Therefore, a number of technical signals were also
generated as WAV files (with CoolEdit96) and then compressed. The
difference between these results directly in the distortion. A
1kHz sinus burst with different sampling rates and a two-tone
signal were used. The first tests primarily the dynamic qualities
of the mp3 compression and the second the intermodulation
distortion. The intermodulation ratio for the original is better
than 90dBc.
Very meticulous tests are conceivable. For example, blocking signals could be superimposed on normal music which could then be mp3 compressed, and then the blockers subtracted. This would present real problems for mp3, because large blocking signals would cover details in the original. Mp3 would then fail to include these details in compression and they would be missing in the end, which would most likely be detectable since the subtraction would eliminate the covering effect. Only linear and time invariant systems are completely immune to such effects, and mp3 is not such a system. Neither is an amplifier, a DA converter or even a loudspeaker.
There were also certain expectations for the indirect comparison of loudspeaker vs. Headphones. A set of headphones cannot deliver the same natural sound quality as loudspeakers can, but there are $250 headphones which have better parameters such as pulse response, noise detection, etc. than even $5000 speakers. Headphones also make it possible to implement a volume level above and beyond room level without causing problems with the neighbors.
In some ways, the mp3 test is much simpler than loudspeaker tests. The hardware is always the same. The original and mp3 version are subjected to the same imperfect equipment such as CD burner, CD player, speaker, amplifier, etc. There is no advantage for either variation. At the most, the differences might become blurred. Only very bad equipment might actually cause negative effects (e.g. frequency peaking) to recur.
The Music
1st Test
The Soundtest
Detective Needed
Conclusion
Literature:
[1] http://www.trail-n-error.de/mp3theorie1.htm
[2] http://hellgate.hs-bremen.de/mp3/artikel.htm
[3] http://www-stud.fht-esslingen.de/~alkoit00/mp3enc/
[4] http://www.raum.com/mpeg/reviews_quality.html
[5] C. Meyer, ct´ Heft 6/00, S. 92ff
[6] ct´ Heft 22/00
[7] http://www.mp3-tech.org
To Top - To Homepage - German Version |
![]() |
Copyright © 2000 Stephan Weber. All rights reserved.
Stand: März 03, 2001.