Topic: Sample joining

[rrrola]

I've had the idea of modules with Vorbis-compressed samples too and was ready to implement it, but remembered to check the web first .

Ogg Vorbis files have big headers, but the largest part with codebooks (the setup header) doesn't change when the same encoding options are used. I've checked this in libvorbis 1.2.0 and aoTuV beta 5.

Ian, could you make the headers shareable in some future version of MO3? Big Ogg Vorbis files are about 60% the size of MP3s with corresponding quality and this would kill the overhead for small files.

Another option would be to join waveforms with the same settings (and shift start/end/loop points). That would be a lot easier to implement, too — the sample header would only need a 'shared waveform' flag. It would also work with other compression modes.

[Ian @ un4seen]

Yep, that's not a bad idea. I'll stick it on the "look into it" list

[Ian @ un4seen]

In order for it to make it into the BASS 2.4 release, I decided to look into this straightaway; and the results are quite nice

Here's an updated MO3 encoder to try...

   www.un4seen.com/stuff/mo3.zip

And here's an XMPlay update to play the results...

   www.un4seen.com/stuff/xmplay.exe

Please report any problems/etc...

[rrrola]

It works better than I expected, saves about 30%. The header sizes are nicely distributed among the joined samples .

I believe the encoder switches to lossless mode automatically if the encoded file is bigger than the original, but I don't think it takes header sharing into account now. The header might be in the file anyway while the rest might be smaller when compressed... a chicken-and-egg problem. Maybe add a 'force compression' switch (or make it default)?

I'd also like the song/pattern/instrument header data compressed using deflate instead of your LZ77 implementation. There are nice optimizing encoders for that, saving 15% in the headers on average (PAQ8o9 -9 saves 65%, but you know).

Shutting up now.

[Zarggg]

Quick question Ian: That zip doesn't include the command-line app. Did you discontinue that?

[Ian @ un4seen]

I believe the encoder switches to lossless mode automatically if the encoded file is bigger than the original, but I don't think it takes header sharing into account now.

Shared headers are taken into account when deciding whether to keep the OGG sample, but I think the problem you're seeing is that there is not yet a header available to be shared. For example, when the first sample is processed, there are no headers from any other samples yet.

To get around that, I have now added a 2nd-pass to the sample processing...

   www.un4seen.com/stuff/mo3.zip

I'd also like the song/pattern/instrument header data compressed using deflate instead of your LZ77 implementation. There are nice optimizing encoders for that, saving 15% in the headers on average (PAQ8o9 -9 saves 65%, but you know).

"deflate" is also "LZ77" based (as is most general purpose compression I would guess), and in my tests it gave worse (larger) results than the current MO3 method. I think there are better alternatives, like LZMA; I did actually implement LZMA in a test version a while ago, but I didn't get round to optimizing the decoder size. Decoder size is one of the main considerations, to avoid bloating BASS/XMPlay. To illustrate the sort of size I'm talking about, the current decoder takes 159 bytes. Any new compression method would have to be fairly close to that and/or provide substantially better results

Quick question Ian: That zip doesn't include the command-line app. Did you discontinue that?

No, that hasn't been discontinued; it just hasn't been updated yet

I'll do that after things have been finalized, eg. the "Single sample rate" option.