Author Topic: MO3 header flags  (Read 23862 times)

saga

  • Posts: 2778
MO3 header flags
« on: 6 Nov '15 - 00:26 »
Since the unofficial MO3 documentation is incomplete, I tried deciphering some of the header flags myself. However, some things are not quite clear yet. Here are my findings:

0x01: Linear Slides (IT, XM)
0x02 | 0x08: Is an S3M file
0x04: S3M fast slides
0x08: Is an MTM file, if 0x02 is not set
0x10: S3M Amiga Limits
0x20: unused?
0x40: unused?
0x80: Is a MOD file
0x100: Is an IT file
0x200: Instrument Mode
0x400: IT compatible Gxx
0x800: IT old FX
0x1000: unused?
0x2000: unused?
0x4000: unknown
0x8000: unused?
0x10000 | 0x200000: Extended filter range (why two flags)?
0x10000 | 0x100000: Has plugins (why two flags)?
0x20000: always set?

Also, the header seems to contain some kind of sample volume thing that is only used in IT and XM files, but I don't quite understand how this maps to IT's "sample volume".

Regarding sample flags - I guess values 0x02, 0x04, 0x08, 0x40, 0x80, 0x800 are unused?
Also, I noticed that the "compressed length" of a sample is either FFFFFFFF or 0 (depending on the codec) if it's a duplicate sample. But how do I know which sample it is duplicated from?
« Last Edit: 8 Nov '15 - 16:17 by saga »

Ian @ un4seen

  • Administrator
  • Posts: 26178
Re: MO3 header flags
« Reply #1 on: 6 Nov '15 - 17:03 »
That flag list looks pretty much correct. One little correction is that the 0x08 flag will be set for any non-XM file, so an MTM file would have that set and none of 0x02 (S3M) / 0x80 (MOD) / 0x100 (IT). The 0x10000 flag indicates MPT-compatibility mode, which is why it's also set when 0x100000/0x200000 are (that may change in a future MO3 encoder release with ".FSM" chunk support). 0x20000 is indeed always set.

Regarding the sample volume thing, that corresponds to the "Mix volume" IT header value. The MO3 value is calculated in a pretty strange way, and the inverse looks like this:

Code: [Select]
if (mo3.mixvol<0)
it.mixvol=mo3.mixvol+52;
else
it.mixvol=exp(mo3.mixvol*3.1/20.0)+51;

It's not used with XM files. I don't remember if it was in the past, but you can ignore it if you see it.

A sample's compressed length value will be negative when it's a duplicate sample, with the value being a relative sample index: -1 = a duplicate of the previous sample, -2 = 2 samples back, etc.

saga

  • Posts: 2778
Re: MO3 header flags
« Reply #2 on: 6 Nov '15 - 22:57 »
Thanks for the clarification. A few more points came up... :)

1. The unofficial MO3 specs mention that the header flag 0x4000 is "currently used" but not what it is supposed to be. At least in my tests with the latest MO3 and MO3 2.2, I couldn't generate a file that uses this flag. Is it ever set?

2. I have found that in some files the flags 0x20 and 0x40 are set (e.g. Skaven's ramagardens_remake.mo3), but I cannot quite deduce their meaning from that.

3. Also, regarding Zxx macros... As far as I understand there is no flag for these as there is in the IT header, right? When the macros are all 0, the default IT macro config (SF0 = Cutoff, Z80-Z8F = resonance) is assumed, is that correct? This effectively means that an all-empty MIDI macro config cannot be generated with MO3 (which in return means that Zxx effects would need to be stripped by MO3, but it does currently not do that). This might seem like a theoretical construct at first, but Zxx effects could be used for syncing if used like that - and in fact, in IT files made with older versions of Impulse Tracker these commands need to be either removed or the macro config needs to be emptied anyway (I didn't test if MO3 does either).

4. I noticed that in some older MO3 files, the IT and S3M frequency is not stored in Hz. The formula seems to be freq = 15787.0 * pow(2.0, finetune / 1536.0) - is this correct, and how can I determine when to use this formula? It seems to be independent from the linear slides flag, the MO3 version or any of the unknown header flags.

5. Interestingly, it seems like the "global volume" field in the sample headers doubles for the sample finetune in the XM format for some reason. I think I caught most fields that are supposed to be interpreted only in some formats, but can you maybe provide a list of fields that are only valid in some formats (like the global volume field) or need to interpreted differently (like the instrument sample map or sample frequency)? Maybe you could have a quick look over my implementation and check if you see any missing format checks?

6. And are there any other structural changes in the format apart from this and the other additions in version 5? I know flags have been added in various versions, but are there any other things that need to be interpreted differently based on the format version apart from that?

7. Regarding shared OGG headers... To reconstruct those, I suppose MO3 uses the first two Ogg frames (and adjusts the stream serial number)? Or is there more behind it?

8. How does the encoding delay in the sample header work? On first sight it seems to be in bytes rather than samples, and additionally it seems to include the LAME padding, which makes it a bit difficult to use with MP3 decoders that already remove the padding?

9. The encoder delay seems to be set for OGG samples as well, although it seems to be unnecessary?

10. From reading the manual (;)) I deduced that there must be a header flag for VBlank mods - 0x80000, right? With this and the two unknown 0x20 and 0x40 flags, the "holes" in the flag set are slowly filling up. However, the 0x1000-0x4000 range and 0x40000 are still missing and I have not seen them in the wild so far - But I guess you didn't leave arbitrary holes in the flag set and surely these values have meaning, too?
« Last Edit: 9 Nov '15 - 23:51 by saga »

Ian @ un4seen

  • Administrator
  • Posts: 26178
Re: MO3 header flags
« Reply #3 on: 10 Nov '15 - 15:37 »
1. The unofficial MO3 specs mention that the header flag 0x4000 is "currently used" but not what it is supposed to be. At least in my tests with the latest MO3 and MO3 2.2, I couldn't generate a file that uses this flag. Is it ever set?

No, 0x4000 isn't currently used.

2. I have found that in some files the flags 0x20 and 0x40 are set (e.g. Skaven's ramagardens_remake.mo3), but I cannot quite deduce their meaning from that.

They aren't currently used either. Looking back at an ancient version of the code, it looks like they may have been used to indicate some things that can be inferred from the original file format anyway. If you see them, you can ignore them, which is what BASS/XMPlay/UNMO3 will do.

3. Also, regarding Zxx macros... As far as I understand there is no flag for these as there is in the IT header, right? When the macros are all 0, the default IT macro config (SF0 = Cutoff, Z80-Z8F = resonance) is assumed, is that correct? This effectively means that an all-empty MIDI macro config cannot be generated with MO3 (which in return means that Zxx effects would need to be stripped by MO3, but it does currently not do that). This might seem like a theoretical construct at first, but Zxx effects could be used for syncing if used like that - and in fact, in IT files made with older versions of Impulse Tracker these commands need to be either removed or the macro config needs to be emptied anyway (I didn't test if MO3 does either).

MO3 shouldn't assume the default IT macro config, but BASS/XMPlay do currently. That would apply to the original file as well as the MO3 file. Are you hearing something different, ie. apparently no macros applied when playing the original file with BASS/XMPlay?

4. I noticed that in some older MO3 files, the IT and S3M frequency is not stored in Hz. The formula seems to be freq = 15787.0 * pow(2.0, finetune / 1536.0) - is this correct, and how can I determine when to use this formula? It seems to be independent from the linear slides flag, the MO3 version or any of the unknown header flags.

That's very close. The calculation is freq = 8363.0 * pow(2.0, (finetune+1408) / 1536.0), and that applies when version<5 in the header.

5. Interestingly, it seems like the "global volume" field in the sample headers doubles for the sample finetune in the XM format for some reason. I think I caught most fields that are supposed to be interpreted only in some formats, but can you maybe provide a list of fields that are only valid in some formats (like the global volume field) or need to interpreted differently (like the instrument sample map or sample frequency)? Maybe you could have a quick look over my implementation and check if you see any missing format checks?

As XM samples don't have a "global volume" setting, that space is used to store the instrument number (1=1st) instead...

Code: [Select]
union {
uint8  globalVol; // 0...64
uint8  xmInstrument;
};

6. And are there any other structural changes in the format apart from this and the other additions in version 5? I know flags have been added in various versions, but are there any other things that need to be interpreted differently based on the format version apart from that?

Not that I recall. Let me know if you encounter something :)

7. Regarding shared OGG headers... To reconstruct those, I suppose MO3 uses the first two Ogg frames (and adjusts the stream serial number)? Or is there more behind it?

That's pretty much it. The size of the OGG headers is also stored, in the same space as the MP3 encoder delay...

Code: [Select]
union {
uint16 encoderDelay;
uint16 oggHeadSize;
};

8. How does the encoding delay in the sample header work? On first sight it seems to be in bytes rather than samples, and additionally it seems to include the LAME padding, which makes it a bit difficult to use with MP3 decoders that already remove the padding?

It is indeed in bytes. If your MP3 decoder already removes some samples, then you can reduce the delay value by that amount.

10. From reading the manual (;)) I deduced that there must be a header flag for VBlank mods - 0x80000, right? With this and the two unknown 0x20 and 0x40 flags, the "holes" in the flag set are slowly filling up. However, the 0x1000-0x4000 range and 0x40000 are still missing and I have not seen them in the wild so far - But I guess you didn't leave arbitrary holes in the flag set and surely these values have meaning, too?

Yep, 0x80000 = VBlank MOD. BASS/XMPlay use the same range of flags internally, so some of the holes will be for things that only apply by them, eg. enabling ProTracker MOD mode.

saga

  • Posts: 2778
Re: MO3 header flags
« Reply #4 on: 10 Nov '15 - 15:56 »
MO3 shouldn't assume the default IT macro config, but BASS/XMPlay do currently. That would apply to the original file as well as the MO3 file. Are you hearing something different, ie. apparently no macros applied when playing the original file with BASS/XMPlay?
Ok, let me clarify. I think the attached examples (macros.7z) should illustrate it nicely.
it_default_macros_embedded: The default macro configuration (set filter / cutoff) is explicitely embedded in the IT and MO3 file. A filtered sample should be heard.
it_default_macros_not_embedded: The default macro configuration is not explicitely embedded, so all macros are 0 in the MO3 file. A filtered sample should be heard.
it_empty_macros_embedded: A completely empty macro configuration is embedded in the IT file. In the MO3 file, it looks exactly like the previous test case, i.e. all macros are 0. The macros should not do anything in this case, but it's impossible to deduce that from the file. Maybe in an upcoming format revision, a new header flag to explicitely designate the presence of macros would be required to cover this case? This would also help with S3M files that contain Zxx effects that should not be interpreted (e.g. because they were made with ScreamTracker or an older version of Impulse Tracker).

That's very close. The calculation is freq = 8363.0 * pow(2.0, (finetune+1408) / 1536.0), and that applies when version<5 in the header.
Even with the adjusted formula this can't be the whole truth. ;) I did indeed check the header version initially but there are some files where the sample frequency is stored in Hz despite not being version 5 files, hence my question. Please see the attached file, which is version 3 but apparently stores the sample frequency in Hz.

As XM samples don't have a "global volume" setting, that space is used to store the instrument number (1=1st) instead...
So I suppose this is just stored for convenience then? (OpenMPT does not require this information at play time and can reconstruct which sample belongs to which instrument while saving the file).
I initially assumbed it was doubling the XM finetune because I created a testcase where sample 1 had finetune 1, sample 2 had finetune 2, ... ooops. ;D

That's pretty much it. The size of the OGG headers is also stored, in the same space as the MP3 encoder delay...
Aaaah, that makes sense, and the whole header reconstruction a lot easier. :)

It is indeed in bytes. If your MP3 decoder already removes some samples, then you can reduce the delay value by that amount.
I see, this is going to be fun...

Yep, 0x80000 = VBlank MOD. BASS/XMPlay use the same range of flags internally, so some of the holes will be for things that only apply by them, eg. enabling ProTracker MOD mode.
Alright... Supposing that a future version of MO3 could determine ProTracker MOD mode on its own, which flag would that be? Just so that I can already add it in OpenMPT before the fact... ;D
« Last Edit: 10 Nov '15 - 16:41 by saga »

Ian @ un4seen

  • Administrator
  • Posts: 26178
Re: MO3 header flags
« Reply #5 on: 10 Nov '15 - 18:00 »
Ok, let me clarify. I think the attached examples (macros.7z) should illustrate it nicely.
it_default_macros_embedded: The default macro configuration (set filter / cutoff) is explicitely embedded in the IT and MO3 file. A filtered sample should be heard.
it_default_macros_not_embedded: The default macro configuration is not explicitely embedded, so all macros are 0 in the MO3 file. A filtered sample should be heard.
it_empty_macros_embedded: A completely empty macro configuration is embedded in the IT file. In the MO3 file, it looks exactly like the previous test case, i.e. all macros are 0. The macros should not do anything in this case, but it's impossible to deduce that from the file. Maybe in an upcoming format revision, a new header flag to explicitely designate the presence of macros would be required to cover this case? This would also help with S3M files that contain Zxx effects that should not be interpreted (e.g. because they were made with ScreamTracker or an older version of Impulse Tracker).

I'll look into that.

Even with the adjusted formula this can't be the whole truth. ;) I did indeed check the header version initially but there are some files where the sample frequency is stored in Hz despite not being version 5 files, hence my question.

Oops. I omitted that it also only applies to IT files with linear slides enabled, ie. when 0x01 and 0x100 are set.

As XM samples don't have a "global volume" setting, that space is used to store the instrument number (1=1st) instead...
So I suppose this is just stored for convenience then? (OpenMPT does not require this information at play time and can reconstruct which sample belongs to which instrument while saving the file).

Indeed it doesn't affect playback; it's only used by UNMO3. I don't remember for certain, but I think it would have been added just in case there are samples that aren't assigned to any keys (so wouldn't be present in the key/sample mappings).

It is indeed in bytes. If your MP3 decoder already removes some samples, then you can reduce the delay value by that amount.
I see, this is going to be fun...

I should also mention that the delay value is always based on the data being 16-bit, so you can divide by 2 (or 4 if stereo) to convert it to samples.

Yep, 0x80000 = VBlank MOD. BASS/XMPlay use the same range of flags internally, so some of the holes will be for things that only apply by them, eg. enabling ProTracker MOD mode.
Alright... Supposing that a future version of MO3 could determine ProTracker MOD mode on its own, which flag would that be? Just so that I can already add it in OpenMPT before the fact... ;D

As it's only internal, the flag could be repurposed in future, so best not prevent that unnecessarily now I think :)

saga

  • Posts: 2778
Re: MO3 header flags
« Reply #6 on: 11 Nov '15 - 16:18 »
Ok, let me clarify. I think the attached examples (macros.7z) should illustrate it nicely.
it_default_macros_embedded: The default macro configuration (set filter / cutoff) is explicitely embedded in the IT and MO3 file. A filtered sample should be heard.
it_default_macros_not_embedded: The default macro configuration is not explicitely embedded, so all macros are 0 in the MO3 file. A filtered sample should be heard.
it_empty_macros_embedded: A completely empty macro configuration is embedded in the IT file. In the MO3 file, it looks exactly like the previous test case, i.e. all macros are 0. The macros should not do anything in this case, but it's impossible to deduce that from the file. Maybe in an upcoming format revision, a new header flag to explicitely designate the presence of macros would be required to cover this case? This would also help with S3M files that contain Zxx effects that should not be interpreted (e.g. because they were made with ScreamTracker or an older version of Impulse Tracker).

I'll look into that.
Actually, if there's going to be any format change, maybe apart from that header flag you should also reconsider how macros are stored. As I have pointed out before, it's possible to use other variables than just the "z" variable in macros, so you can for example control the filter based on the current volume of the playing sample rather than on the Zxx effect parameter. Such macros cannot be saved using the current format, and any kind of "external" MIDI messages (that could be sent to plugins), i.e. messages not starting with F0F0 are also not usable. Maybe in the next format revision, all macros should just be copied verbatim as 0-terminated strings?

And if you're looking for more things to include in a possible new format revision: OpenMPT's artist name field would be great to have! :)

Oops. I omitted that it also only applies to IT files with linear slides enabled, ie. when 0x01 and 0x100 are set.
Ah, thanks, now all files seem to play as intended. A weird choice to have less precise sampling rates in linear slides mode but I guess there are historic reasons for that. :)

Indeed it doesn't affect playback; it's only used by UNMO3. I don't remember for certain, but I think it would have been added just in case there are samples that aren't assigned to any keys (so wouldn't be present in the key/sample mappings).
Ah, that does make sense.

I should also mention that the delay value is always based on the data being 16-bit, so you can divide by 2 (or 4 if stereo) to convert it to samples.
Could you maybe have a look at this module? Something seems to be dodgy about the encoder delay there, no matter if the samples are 8-bit (e.g. sample 2) or 16-bit (sample 25):
I disabled gapless support in mpg123 prior to decoding, and then I basically do this...
Code: [Select]
memmove(sampleData, sampleData + encoderDelay, decodedSampleSize - encoderDelay);Where decodedSampleSize returns the size of the decoded 16-bit sample in bytes, and sampleData is a char* pointing to the sample start. This seems to remove too much from the sample, so some silence is heard at the end of the samples and they no longer loop properly. If I enable gapless support, this is gone but the sample is also shorter than it should be.
The samples are imported correctly if the encoder delay is ignored when LAME gapless headers are present. In this example the encoder delay is 3362 bytes, but in fact there are only around 1100 samples (so ~2200 bytes) of silence at the sample start. But gathering from your previous posts, this is not how it should be done?


Oh, and one more thing... It seems that the sample finetune value is different in MTM. The unit seems to be the same as XM finetune, but with 256 as the "center" value rather than 128?


By the way, I did some fuzzing on my own MO3 implementation and generated a few test cases which also crash the official UNMO3. I uploaded them as mo3-fuzzing.7z to your FTP (some of the files in the package won't crash UNMO3, but quite a few will do). The source for those crashes can most likely be found in the unpack routines - judging from the unofficial unmo3 code that was originally translated from the disassembled code, there is no validation of how far the input data is being read, and when copying strings, the source pointer of the string is not validated to be inside decoded block.
« Last Edit: 11 Nov '15 - 19:38 by saga »

Ian @ un4seen

  • Administrator
  • Posts: 26178
Re: MO3 header flags
« Reply #7 on: 12 Nov '15 - 16:53 »
Actually, if there's going to be any format change, maybe apart from that header flag you should also reconsider how macros are stored. As I have pointed out before, it's possible to use other variables than just the "z" variable in macros, so you can for example control the filter based on the current volume of the playing sample rather than on the Zxx effect parameter. Such macros cannot be saved using the current format, and any kind of "external" MIDI messages (that could be sent to plugins), i.e. messages not starting with F0F0 are also not usable. Maybe in the next format revision, all macros should just be copied verbatim as 0-terminated strings?

And if you're looking for more things to include in a possible new format revision: OpenMPT's artist name field would be great to have! :)

Yep, perhaps it would be best to simply copy the entire macro block as is (compression will reduce it to almost nothing), even if BASS/XMPlay doesn't use all of it. An artist name field could be added at the same time.

Could you maybe have a look at this module? Something seems to be dodgy about the encoder delay there, no matter if the samples are 8-bit (e.g. sample 2) or 16-bit (sample 25):
I disabled gapless support in mpg123 prior to decoding, and then I basically do this...
Code: [Select]
memmove(sampleData, sampleData + encoderDelay, decodedSampleSize - encoderDelay);Where decodedSampleSize returns the size of the decoded 16-bit sample in bytes, and sampleData is a char* pointing to the sample start. This seems to remove too much from the sample, so some silence is heard at the end of the samples and they no longer loop properly. If I enable gapless support, this is gone but the sample is also shorter than it should be.
The samples are imported correctly if the encoder delay is ignored when LAME gapless headers are present. In this example the encoder delay is 3362 bytes, but in fact there are only around 1100 samples (so ~2200 bytes) of silence at the sample start. But gathering from your previous posts, this is not how it should be done?

That MO3 file appears to have been created with a very old MO3 encoder version, as the samples still contain LAME headers (more recent MO3 encoder versions remove those headers). I think it is those headers that are confusing matters. If the MP3 decoder supports LAME headers, then it's probably best to just let it handle the delay in this case and ignore the delay value in the MO3 file.

Oh, and one more thing... It seems that the sample finetune value is different in MTM. The unit seems to be the same as XM finetune, but with 256 as the "center" value rather than 128?

You can convert the finetune back to MTM form by shifting right 4 bits: mtm.finetune=mo3.finetune>>4. For XM subtract 128: xm.finetune=mo3.finetune-128

By the way, I did some fuzzing on my own MO3 implementation and generated a few test cases which also crash the official UNMO3. I uploaded them as mo3-fuzzing.7z to your FTP (some of the files in the package won't crash UNMO3, but quite a few will do). The source for those crashes can most likely be found in the unpack routines - judging from the unofficial unmo3 code that was originally translated from the disassembled code, there is no validation of how far the input data is being read, and when copying strings, the source pointer of the string is not validated to be inside decoded block.

Yep, the MO3 structure decompressor doesn't include much error detection, so corrupt MO3 files may well result in a crash. I'll look into improving that.

saga

  • Posts: 2778
Re: MO3 header flags
« Reply #8 on: 12 Nov '15 - 20:51 »
That MO3 file appears to have been created with a very old MO3 encoder version, as the samples still contain LAME headers (more recent MO3 encoder versions remove those headers). I think it is those headers that are confusing matters. If the MP3 decoder supports LAME headers, then it's probably best to just let it handle the delay in this case and ignore the delay value in the MO3 file.
Hm, I'm using mpg123 as a cross-platform decoder, and as far as I can see there's no API to tell whether the MP3 data has any gapless info in it. So I guess I'd have to scan the first MP3 frame myself to see if contains a LAME header.
However, I can see that version 5 MO3s still have a LAME header, but after the "LAME3.xx" version string, all following bytes are 0xAA. I guess this is what you mean by "removing the headers"? So the encoding delay should only be considered if no LAME header is found, or if it's all 0xAA bytes?
Or maybe it would make more sense to read the first MPEG frame, check if it's a valid LAME header, and if that is the case, subtract the first frame length from the encoder delay value? I guess if mpg123 doesn't give me any indication about the LAME headers, it's about the same amount of work either way...
I suppose in the "mp3-free" Linux BASS version, you rather decode the samples by disabling MPG123_GAPLESS and then apply the LAME header yourself in BASS if required?

EDIT: I seem to have found a hack that seems to work with mpg123 (first scan the file; then rewind and call mpg123_framebyframe_next before decoding). It seems to work with MP3 samples that have no LAME header, those with a LAME header and those with a LAME header full of 0xAA, but hacks are hacks so I'd much rather have a solution where I can actually understand how it works.

You can convert the finetune back to MTM form by shifting right 4 bits: mtm.finetune=mo3.finetune>>4. For XM subtract 128: xm.finetune=mo3.finetune-128
Yeah, if mo3.finetune is interpreted as an int8 that makes sense. But I observed that the lowest bit of the second finetune byte is also set (so it's 00 01 00 00 rather than 00 00 00 00 for centered finetune), and I was not sure if that means that either the center would be 256, or if this 01 byte is supposed to be a flag, or something completely different? I've attached an example to demonstrate what I mean (nevermind that it's completely out of tune due to the finetune experiments... :)).
« Last Edit: 13 Nov '15 - 00:10 by saga »

Ian @ un4seen

  • Administrator
  • Posts: 26178
Re: MO3 header flags
« Reply #9 on: 13 Nov '15 - 15:29 »
Hm, I'm using mpg123 as a cross-platform decoder, and as far as I can see there's no API to tell whether the MP3 data has any gapless info in it. So I guess I'd have to scan the first MP3 frame myself to see if contains a LAME header.
However, I can see that version 5 MO3s still have a LAME header, but after the "LAME3.xx" version string, all following bytes are 0xAA. I guess this is what you mean by "removing the headers"? So the encoding delay should only be considered if no LAME header is found, or if it's all 0xAA bytes?

Are you sure it's a LAME header, ie. do you see "Xing" or "Info"? It might just be some frame padding (LAME will advertise itself in there). Details on the LAME header can be found here:

   http://gabriel.mp3-tech.org/mp3infotag.html

Or maybe it would make more sense to read the first MPEG frame, check if it's a valid LAME header, and if that is the case, subtract the first frame length from the encoder delay value? I guess if mpg123 doesn't give me any indication about the LAME headers, it's about the same amount of work either way...
I suppose in the "mp3-free" Linux BASS version, you rather decode the samples by disabling MPG123_GAPLESS and then apply the LAME header yourself in BASS if required?

Yep, BASS still removes the gap itself then, but trying your test file with it just now, I do see the same problem that you described. The issue seems to be that mpg123 is skipping the LAME header frame, while the old MO3 encoder knew nothing about LAME headers and assumed that all frames would produce output, so the delay ends up being shorter than expected and too much data gets removed. As you say, you could compensate for that by subtracting the LAME header frame's length (576 samples) from the delay. To be safe, you should also skip the frame in the data you give to mpg123, ie. start with the 2nd frame. I'll make that change for the next BASS release; it does actually already skip LAME headers for normal MP3 file decoding, but not for MO3 sample decoding.

saga

  • Posts: 2778
Re: MO3 header flags
« Reply #10 on: 15 Nov '15 - 19:28 »
I just found out that mpg123 added a MPG123_IGNORE_INFOFRAME flag in API version 31 (march 2011), so using that might also be a viable option for MO3 decoding.

Can you please have a look at the MTM file from my previous post as well to clarify the finetune issue there?

Ian @ un4seen

  • Administrator
  • Posts: 26178
Re: MO3 header flags
« Reply #11 on: 16 Nov '15 - 15:19 »
I just found out that mpg123 added a MPG123_IGNORE_INFOFRAME flag in API version 31 (march 2011), so using that might also be a viable option for MO3 decoding.

Nice find. That seems to do the trick, so you can simply pass the entire MP3 sample to mpg123 and then skip the delay amount specified in the MO3 file.

Can you please have a look at the MTM file from my previous post as well to clarify the finetune issue there?

I see now that I didn't give the exact formula that is used, which is this: mtm.finetune=(mo3.finetune&0xff)>>4, ie. only the low 8 bits are used. Let me know if it's still not working right.

saga

  • Posts: 2778
Re: MO3 header flags
« Reply #12 on: 16 Nov '15 - 16:25 »
Thanks, it's indeed working with the files I've tried, I was just wondering if the high byte had any specific meaning that was escaping me.
As far as I can see my MO3 decoder is pretty much complete now. :)

Ian @ un4seen

  • Administrator
  • Posts: 26178
Re: MO3 header flags
« Reply #13 on: 16 Nov '15 - 16:46 »
Very nice :)

saga

  • Posts: 2778
Re: MO3 header flags
« Reply #14 on: 17 Nov '15 - 17:52 »
By the way, since you mentioned possible future support for the ".FSM" chunk, is there any work done on this yet? I'm just asking since I'm currently revising OpenMPT's compatibility playback settings to be a bit more fine-grained (one flag per playback quirk), so it's possible that the .FSM chunk will look a bit different in the near future. I think for compatibility reasons I will keep setting the lowest bit there if compatible playback flags are set, while the more fine-grained compatibility flags will follow in the next few bytes. I guess in the end it might be a good idea to just copy all OpenMPT chunks into the MO3 file so that it's just up to some player to support them?
« Last Edit: 17 Nov '15 - 17:59 by saga »

Ian @ un4seen

  • Administrator
  • Posts: 26178
Re: MO3 header flags
« Reply #15 on: 18 Nov '15 - 15:38 »
The MO3 encoder uses the same loaders as BASS/XMPlay, so a current build would include the ".FSM" support. Only the 1st bit is used, so you could indeed use the other bits for other purposes without affecting BASS/XMPlay/MO3.

Regarding including OpenMPT chunks in MO3 files, I'm thinking of making it possible to include basically anything in optional chunks after the normal MO3 structure. That would be back-compatible with older BASS/XMPlay/UNMO3, as they would just ignore the extra chunks. That system could be used to store the entire IT MIDI config block, for example. I'll send you something to try.