I believe the encoder switches to lossless mode automatically if the encoded file is bigger than the original, but I don't think it takes header sharing into account now.
Shared headers are taken into account when deciding whether to keep the OGG sample, but I think the problem you're seeing is that there is not yet a header available to be shared. For example, when the first sample is processed, there are no headers from any other samples yet.
To get around that, I have now added a 2nd-pass to the sample processing... www.un4seen.com/stuff/mo3.zip
I'd also like the song/pattern/instrument header data compressed using deflate
instead of your LZ77 implementation. There are nice optimizing encoders
for that, saving 15% in the headers on average (PAQ8o9 -9 saves 65%, but you know).
"deflate" is also "LZ77" based (as is most general purpose compression I would guess), and in my tests it gave worse (larger) results than the current MO3 method. I think there are better alternatives, like LZMA; I did actually implement LZMA in a test version a while ago, but I didn't get round to optimizing the decoder size. Decoder size is one of the main considerations, to avoid bloating BASS/XMPlay. To illustrate the sort of size I'm talking about, the current decoder takes 159 bytes. Any new compression method would have to be fairly close to that and/or provide substantially better results
Quick question Ian: That zip doesn't include the command-line app. Did you discontinue that?
No, that hasn't been discontinued; it just hasn't been updated yet
I'll do that after things have been finalized, eg. the "Single sample rate" option.