I am wondering about swiching my raw files to DNG, mainly because of space constraints.

Right now, my Minolta (5D) raw files are about 9 MB each one. Of course, it's quickly becoming (storage) space consuming, but I'm still reluctant to only keep jpeg versions. When you decide to edit a file, it seems to me that having the full 12 bits to start with is adding a lot more freedom, and seems to take care of the uniform areas that tend to appear if you use curves too much on jpeg files.

So I decided to have a serious look at DNG. I downloaded the Adobe DNG converter, and tryed it on a few files. The first feelings about the converter were positive:
*DNG converter is a single executable, no installation needed
*the converter is quite fast, and seems to be using all the available cpu cores
*it provided a great size reduction over the original raw files (about 30% smaller files)

Then I tryed to open the DNG files with Paint Shop Pro X, and things started to go wrong: even if PSPX is supposed to support DNG, it failed to open my newly created DNG files. I then tryed to browse the files using XnView, and it was displaying the files with some obviously broken parts. Quite disappointing. I think that it's likely that DNG support will improve in the future, a some camera manufacturers (especially Pentax) are starting to use it as a native file format in their DSLR, but it seems that right now there are still a few problems.

I still decided to have a look at the compressibility of my raw files:

original mrw file: 8.77MB
DNG converted: 5.13MB

A good size reduction, but:

zip: 5.79MB
bz2: 4.76MB
7zip: 5.13MB

So if I have some big storage space issues, I could still decide to use 7zip on my older raw files.

Conclusion: not yet switching to DNG

Some samples of Intel "quad cores" processors are now available. As with the introduction of dual cores, we'll see a lot of hardware websites doing some benchmarks. Several of them will also use LAME encoding as one of the tests.

In the current version (3.97) LAME is not multithreaded at all, so for a single encoding it's obvious that adding cores will not change anything, no need to run a benchmark to know it, and no need to display bar graphs of encoding speed results (as they are quite pointless).


Message to benchmarkers:

If you want to use mp3 encoding for a multicores/multiprocessors benchmarks, then you have the following choices:

*use a natively multithreaded encoder (example: iTunes)

*use a multithreaded frontend and encode several files. As an example, EAC allows you to choose the number of threads. If you use more than 1, of course you will benefit from the added cores.

For the sake of curiosity, I tested the evolution of encoding speed of Lame using different versions. To simplify things, encoding is done at default settings (128kbps/cbr). Computer features MMX/3dnow/SSE/SSE2, audio file is about 1 minute long.

Here are the results:

3.20: 3.51s
3.50: 6.49s
3.70: 3.78s
3.80: 3.70s
3.90: 3.74s
3.93.1: 4.58s
3.96.1: 5.53s
3.97: 5.31s

By looking at this, it's clear that overall Lame has became slower over time. We tryed to keep speed reasonable, but as we increased quality our speed optimisations were not enough to compensate for the extra computations.

We had a notable speed decrease when releasing 3.94. In 3.94, we switched our default psychoacoustic model from GPsycho to NSPsytune. NSPsytune was already used (starting with 3.90) when using the preset/alt-preset settings. To have a comparison value, here is the "preset cbr 128" speed of versions prior to 3.94:

3.90 --alt-preset cbr 128: 9.62s
3.93.1 --preset cbr 128: 6.55s

So between 3.90 and 3.97, NSPsytune's encoding time decreased from 9.62s to 5.31s. Not bad at all, but still quite slower than early Lame releases.

ADPCM is a very basic audio coding scheme, and is quite fast to encode and decode. The drawback is that compression is quite limited (fixed to 4 bits per sample), and so is the quality.

As it's based on encoding a difference from the predicted sample, and that prediction is based on previous samples, I already thought about a modified ADPCM encoder that would work on a window of a few samples, in order to select the optimal encoding for this whole window, instead of only considering the current sample.

I've since found a blog post by M.Neidermayer about this, and his explanations are way better than mine.

Now, let's take this further: what about a noise shaped ADPCM encoding? Instead of a simple SNR optimisation we could shape the ADPCM noise in the frequency domain. We could base it on SMR (signal to mask ratio) values, but even just using the ATH (absolute threshold of hearing) would probably be a big improvement.
To do this, we could consider a bigger window of samples. 128 samples would be comfortable regarding frequency resolution, but down to 32 samples would probably be useable. Over this window, we would do a time to frequency transform, and do an election of the best ADPCM encoded vector based on the distance from the ideal frequency based distortion.

The problem is now the computationnal cost of this. Using a coarse 32 samples window, we would need to compute 4^32 quantizations and transforms, which would be prohibitive. (please note how I managed to change ADPCM from a very fast coding scheme to something that would take years to compute)

The open question is now: how to reduce the complexity of this?