next »

[Ian @ un4seen]

You don't answer my question about 2 phases compression I propose for mo3enc > 2.2

I didn't realize you were asking a question. You basically described what the MO3 encoder already does

Are patterns played by BASS "in place" in RAM after MO3 decoding ? Or another transformation is required ?
Amiga protracker packers (see Exotica replay pages) were designed to by replayable "in place". I not 100% for MO3 decoded structure.

Sure, MO3 can be played "in place", but I don't know about "designed" to be so - that to me suggests consideration for things like alignment and padding for run-time variables, which is not the case in MO3.

[Laurent]

hi Ian,

that to me suggests consideration for things like alignment and padding for run-time variables, which is not the case in MO3.

no, but you can have a format very very compact, but hard to "replay". forget this.

another thing : I think there is another kind of repetition not yet exploited for compression by mo3enc : sequence of row repetion in a voice. Ex: imagine a given voice is like that (the number is one row data) : 1 2 3 0 1 2 3 0 1 2 3 0, why not coding this repetition ? for example : 1 2 3 0 (already done today) then "the 4 last row is repeated 3 times" (encoded 83 03, "-4 , 3 times").
And this kind of repetition is -very- common in module, removing it can save a lot of room, no ?

I'm still working at http://lclevy.free.fr/mo3, if you could tell me if there is errors...

Laurent

[Ian @ un4seen]

another thing : I think there is another kind of repetition not yet exploited for compression by mo3enc : sequence of row repetion in a voice. Ex: imagine a given voice is like that (the number is one row data) : 1 2 3 0 1 2 3 0 1 2 3 0, why not coding this repetition ? for example : 1 2 3 0 (already done today) then "the 4 last row is repeated 3 times" (encoded 83 03, "-4 , 3 times").
And this kind of repetition is -very- common in module, removing it can save a lot of room, no ?

The compression layer takes care of block repetition. Adding extra opcodes for it may well actually increase the size of the end result.

I'm still working at http://lclevy.free.fr/mo3, if you could tell me if there is errors...

Seems to be coming along nicely. One thing though, it's not only MPEG 2 layer 3 that's supported - MPEG 1 and MPEG 2.5 are also supported

[Laurent]

The compression layer takes care of block repetition. Adding extra opcodes for it may well actually increase the size of the end result.

OK so I should have missed it...

Seems to be coming along nicely. One thing though, it's not only MPEG 2 layer 3 that's supported - MPEG 1 and MPEG 2.5 are also supported

OK, I'm going to fix that.
The 'DeltaSlope' unpacker for 16 bits is working now. Do you have a better name for this kind of compression ?

I could not find a module which one of the 16bits sample is compressed with the 'delta' compression : I can not test yet my unpacker.

Laurent

[Laurent]

Well, the full code to unpack (5 function) is ready and released on the web page.

More code to come in the following week.

Laurent

[Ian @ un4seen]

The 'DeltaSlope' unpacker for 16 bits is working now. Do you have a better name for this kind of compression ?

I haven't named it, but it basically predicts the samples and encodes the error/difference, so maybe "prediction error" encoding? I believe most lossless audio compressors use the same principle, with varying prediction and encoding schemes.

Btw, I think you could change/remove the "an empty voice is encoded len=7, 10 f0 f0 f0 f0 30 00" pattern data example. That less than optimal output is an encoder bug (rather than part of the specs), and it'll be just "00" with the 2.2 encoder anyway

[Laurent]

I haven't named it, but it basically predicts the samples and encodes the error/difference, so maybe "prediction error" encoding? I believe most lossless audio compressors use the same principle, with varying prediction and encoding schemes.

OK for "ErrorPrediction". And if you look in the 'unofficial spec' I have already talked about that.

Btw, I think you could change/remove the "an empty voice is encoded len=7, 10 f0 f0 f0 f0 30 00" pattern data example. That less than optimal output is an encoder bug (rather than part of the specs), and it'll be just "00" with the 2.2 encoder anyway

OK, I going to change that. So you have read the doc until the 2.3.2 at least, hum ?
You talked about 'block repetition detection done by the encoder'. You're talking about the whole structure encoder, right ? I have seen no specific -pattern- (voice in fact) block repetition encoding... unless for blocks with only 1 row.

Another questions:
-how do you choose the best 'compression' between Delta/ErrorPrediction/NotCompressed ? by testing the 3 ones ?
-why not choosing FLAC or shorten ? except for short samples of course. ?
-or do you want to keep your compact decoder to use it in embedded devices with low memory ?

Other opinion :
- on a marketing point of view, for musicians, "MO3" seems not such a good name, since mp3 is associated with loss, and loss of quality. I would instead someting like choose 'Tracker Module Compression', to highlight the fact that the know-how is the compression of music data, rather than using MP3, which is only -one- of the compression method.

Laurent
 

[raina]

About the marketing, I think tracker musicians pretty much know MO3 already. MIDI musicians also need to learn to track before worrying about compression. It's the computer music newbies who have no idea and who repeatedly ask how to convert MP3s to MO3. They think MO3 is a more efficient stream compression method because the filesizes are so small, some just want to make the conversion to substitute a game's original music with their favourites. "Tracker Module Compression" is informative, but not a good name on its own IMHO. Combination of the two like this: "MO3 - Tracker Module Compression" (with or without the dash) could be used to refer the technology but MO3 would still be short for that. Also, "MO3 Tracker Module Compression" would be better than "MO3, mod format" in the MO3 page.

[Laurent]

I see marketing the way to explain 'simply' things to -real world- people (not only technical ones), including newbies 

Laurent

[raina]

In my view, MO3 offers nothing to newbies no matter what it was called.

The way I would explain MO3 to layman would be: "Yes, it's a tiny music format that sounds good. No, you cannot make your MP3s smaller by converting them to MO3." To further questions: "You can only make MO3s out of module formats such as MOD/S3M/XM/IT." To even further questions: "Modules are music files you write yourself using a tracker program or download from the Internet. You won't find any contemporary hits though. Although commercial music and tracker music meet sometimes, they still come from different worlds."

[Ian @ un4seen]

You talked about 'block repetition detection done by the encoder'. You're talking about the whole structure encoder, right ?

Yep, that is what I meant by "compression layer". It will efficiently compress any repeated blocks - repeated blocks is what general purpose compressors do best.

-how do you choose the best 'compression' between Delta/ErrorPrediction/NotCompressed ? by testing the 3 ones ?

The encoder uses 2 passes... the 1st pass just checks what the size of each method would be, and then the 2nd pass does the actual encoding using the smallest method.

-why not choosing FLAC or shorten ? except for short samples of course. ?

I don't really want to bloat the decoding side of things, especially as the lossless compression is primarily a backup plan for when MP3/OGG compression doesn't work well, eg. short samples. I think there's probably more to be squeezed out of the current methods, though. I'll look into it (for 2.3).

- on a marketing point of view, for musicians, "MO3" seems not such a good name, since mp3 is associated with loss, and loss of quality.

Personally, I like "MO3". It doesn't necessarily have to stand for "MOD with MP3", it could mean "MOD with 3 sample compression forms" (MP3/OGG/lossless). Although it does actually stand for "MOD with MP3"

[Laurent]

I don't really want to bloat the decoding side of things, especially as the lossless compression is primarily a backup plan for when MP3/OGG compression doesn't work well, eg. short samples.

So why not offer to use FAAC ? mpeg4 audio ?

I think there's probably more to be squeezed out of the current methods, though. I'll look into it (for 2.3).

And it could have for you (but not for the unofficial spec) an interesting -temporary- side effect.

Laurent

[Ian @ un4seen]

I don't really want to bloat the decoding side of things, especially as the lossless compression is primarily a backup plan for when MP3/OGG compression doesn't work well, eg. short samples.

So why not offer to use FAAC ? mpeg4 audio ?

I don't see how that would help lossless compression, or avoid bloatage.

I think there's probably more to be squeezed out of the current methods, though. I'll look into it (for 2.3).

And it could have for you (but not for the unofficial spec) an interesting -temporary- side effect.

I'm not sure what you mean. What side effect?

[Laurent]

I'll have to adapt my C code to new unpack methods...

Laurent

[Laurent]

$ ./unmo3 -s 4 -v 2 1 6500-ast_ogg.mo3
unmo3 v0.5 (opensource version)

Songname = "A Step to the Stars"
Nb channels = 61
Song length = 24
Song restart = 0
Nb pattern = 24
Nb unique voice = 661
Nb instr = 99
Nb sample = 99
Ticks/row = 12
Initial tempo = 125
Initial format is IT (0x00020f69)

Displaying channel #1 of pattern #2 (length 64): unique voice #33
[3, 2 3 1 3c f 15] 0*15 [3, 2 3 1 46 f 15] 0*15 [3, 2 3 1 4b f 15] 0*15 [3, 2 3 1 3e f 15] 0*15

saving sample004.dat (offset 0x9e2e, length 4009/2109 bytes, compression LosslessDeltaPred8, resolution 8bits)...

laurent@becane-21d7cd1a /cygdrive/e/mo3
$ ./unmo3 -s 81 -v 1 1 6500-ast_ogg.mo3
unmo3 v0.5 (opensource version)

Songname = "A Step to the Stars"
Nb channels = 61
Song length = 24
Song restart = 0
Nb pattern = 24
Nb unique voice = 661
Nb instr = 99
Nb sample = 99
Ticks/row = 12
Initial tempo = 125
Initial format is IT (0x00020f69)

Displaying channel #1 of pattern #1 (length 64): unique voice #9
[3, 2 0 1 30 b 0] [1, 22 f1] [6*1, 22 0] [1, 22 f2] [3*1, 22 0] [1, 22 f3] [3*1, 22 0] [3, 2 0 1 30 b ff] [1, 22 f1] [6*
1, 22 0] [1, 22 f2] [3*1, 22 0] [1, 22 f3] [3*1, 22 0] [3, 2 0 1 32 b 0] [1, 22 f1] [6*1, 22 0] [1, 22 f2] [3*1, 22 0] [
1, 22 f3] [3*1, 22 0] 0*11 [1, b 42] [3, 2 3 1 46 f 18] [4, 2 3 1 3c f 17 6 5] [2, f 16 6 3] [2, f 15 6 1]

saving sample081.ogg (offset 0x563b2, length 33600/6874 bytes, compression ogg, resolution 8bits)...

next step is explicit pattern displaying and why not .mod output...

Laurent

[Laurent]

Hi Ian,

I see some space in mo3 structure for which I don't understand the usage. Are they used by BASS to replay the module ? Like the space (16+16+64)*4 bytes just before the song sequence (list of replayed patterns)...

Laurent

[Ian @ un4seen]

It's the IT MIDI macro stuff. First come the SF0-SFF settings, which are 1 byte each, and equate to "F0F0<value-1>z" in IT. Then come the Z80-ZFF settings, which are 2 bytes each, and equate to "F0F0<value1-1><value2>".

Before that, are the default channel volume and pan tables, which are 64 bytes each (64 channels).

[Laurent]

Thank you

Could you describe the instruments structure and especialy the 3*104 bytes (section 2.3.3 is poor) ? it is mostly vol and panning values right ? Could you give me some details ?

Laurent

[Ian @ un4seen]

They're the volume/pan/pitch envelopes, with basically the same structure as IT envelopes, except that the node points are 16-bit pos/val pairs.

[Laurent]

Hi there,

The source code of unmo3 is released at http://lclevy.free.fr/mo3.

"The piece of code has been written as a compagnion (validation code) of the document "the unofficial MO3 specification".

It is targeted to developpers or technical people, not for end users. It can be used by IT/XM/S3M modules specialists (tracker editor developper or modules players) to write a MO3 import loader, or more generally to handle MO3 modules in any way.

The MO3 format has been created by Ian Luck (http://www.un4seen.com). If you are looking for a good encoder and decoder (but without the source code) and a good module player, Ian's web site is the right place to go.

The features of unmo3 (opensource version) are:

    * uncompress the MO3 header and samples with lossless compressions.
    * able to save to a file uncompressed header and samples
    * able to extract mp3 and ogg compressed samples
    * can display a channel of a given pattern into 2 forms
          o as encoded inside MO3 file
          o as it usually appears in a tracker editor (for .mod only)

"

Laurent