Topic: Proposal to support rmid files with embedded SF2 soundfonts in Bassmidi

[Falcosoft]

Perhaps I didn't explain it well enough. The solution I'm proposing is that the SF2 is modified to assume DBNK=0 when embedding it in the RMID file, eg. add 1 to the banks in the Creative demo case. That avoids the need for a DBNK chunk because it's always 0. Then your player (and any others that use SFMAN32) can do what I wrote in my last post when using SFMAN32, while other software can use the soundfont as it is (they don't need to avoid bank 0).

Maybe I really misunderstand you so please send me an example so I can test it.

Because the soundfont already contains bank information, I think the DBNK chunk adds unnecessary complexity, ie. it's an extra thing that all software will need to process. Assuming bank 1 will also add complexity, ie. if a MIDI file uses bank 0 then you will need to modify it to use bank 1 by inserting bank select events - simpler to patch the soundfont (if necessary) instead.
I think it's best to keep things as simple as possible for general usage, and have any software with special requirements (eg. using SFMAN32) adapt themselves to it, rather than the other way round.

But the soundfont itself does not contain Bank offset information that is needed to satisfy all cases. I do not think it really means complexity for any modern SF2 engines (definitely not for Bassmidi).

@Edit:
And let me repeat: 'DBNK' is an optional extension. I still have not heard a real argument about how this could hurt the standard or add unnecessary complexity when it is fully optional.
No one is forced to make rmi files with 'DBNK' chunks. But If someone would like to (like me) just allow him to do it. And 3 from the 4 SF2 capable engines/players I know of that should be able to recognize 'DBNK' chunks already support it. And I do not think that FluidSynth would have a hard time adding support for it either.
So I really do not see what would be the advantage of fully dropping it.

But what if you want the embedded soundfont to provide all sounds, not just some? I think tailoring the format to variations limits its potential usage.

I do not think so. The 2 scenarios are not contradictory. Both can be used according the current specification without any problems. The embedded soundfont can provide all sounds. The possibility to provide only variation banks does not prevent this any way.
I have alraedy presented cases for both scenario in my first post:
1. GRABBAG_EmbeddedSF2.rmi is an example for a fully contained rmi.
2. Rock_test.rmi is an example for only embedding a variation bank (and the rmi relies on a full GM bank at bank 0).

[Ian @ un4seen]

But the soundfont itself does not contain Bank offset information that is needed to satisfy all cases.

The soundfont doesn't need a separate bank offset because that can be included in its standard bank information. For example, if you want a bank offset of 1 then just add 1 to the soundfont's banks. "DBNK" chunk not needed. Here's some code for adding a bank offset to a soundfont:

Code: [Select]

struct sfPresetHeader
{
CHAR achPresetName[20];
WORD wPreset;
WORD wBank;
WORD wPresetBagNdx;
DWORD dwLibrary;
DWORD dwGenre;
DWORD dwMorphology;
};

struct RIFFChunk {
char id[4];
DWORD size;
BYTE data[];
};

// return the "phdr" chunk from an SF2 file in memory
RIFFChunk *GetPHDR(BYTE *file)
{
RIFFChunk *chunk = (RIFFChunk*)file;
DWORD end = 8 + chunk->size, pos = 12;
while (pos + 8 < end) {
chunk = (RIFFChunk*)(file + pos);
DWORD chunk1end = pos + 8 + chunk->size;
if (chunk1end > end) break;
if (!memcmp(chunk->id, "LIST", 4) && !memcmp(chunk->data, "pdta", 4)) { // found "pdta" list
pos += 12;
while (pos + 8 < chunk1end) {
chunk = (RIFFChunk*)(file + pos);
DWORD chunk2end = pos + 8 + chunk->size;
if (chunk2end > chunk1end) break;
if (!memcmp(chunk->id, "phdr", 4)) return chunk; // found "phdr"
pos = chunk2end + (chunk2end & 1);
}
}
pos = chunk1end + (chunk1end & 1);
}
return 0;
}

...

RIFFChunk *phdr = GetPHDR(sf2ptr);
if (phdr) {
sfPresetHeader *presets = (sfPresetHeader*)phdr->data;
int num_presets = phdr->size / sizeof(sfPresetHeader);
// add bank offset to all presets
for (int a = 0; a < num_presets; a++)
presets[a].wBank += bankoffset;
}

And here's how you would find and deduct a bank offset from a soundfont for use with SFMAN32:

Code: [Select]

RIFFChunk *phdr = GetPHDR(sf2ptr);
if (phdr) {
sfPresetHeader *presets = (sfPresetHeader*)phdr->data;
int num_presets = phdr->size / sizeof(sfPresetHeader);
// find bank offset (lowest bank)
bankoffset = 0xffff;
for (int a = 0; a < num_presets; a++)
if (bankoffset > presets[a].wBank) bankoffset = presets[a].wBank;
// deduct it
if (bankoffset) {
for (int a = 0; a < num_presets; a++)
presets[a].wBank -= bankoffset;
}
// load modified soundfont to bank "bankoffset" with SFMAN32 here
}

Let me know if anything is unclear or you have questions.

And let me repeat: 'DBNK' is an optional extension. I still have not heard a real argument about how this could hurt the standard or add unnecessary complexity when it is fully optional.
No one is forced to make rmi files with 'DBNK' chunks. But If someone would like to (like me) just allow him to do it. And 3 from the 4 SF2 capable engines/players I know of that should be able to recognize 'DBNK' chunks already support it. And I do not think that FluidSynth would have a hard time adding support for it either.
So I really do not see what would be the advantage of fully dropping it.

It'd be optional for end-users but not for software. I just think it's unnecessary given that the soundfont already contains bank information. As I understand it, the "DBNK" chunk is a recent addition and not being used yet apart from your test files? If so, I think it would be a good idea to remove it now before it's too entrenched

[Falcosoft]

Hi,
I'm sorry Ian, but I'm not convinced. It's way over my head why you think that patching the SF2 is easier and more elegant than simply handling a marker that tells where the SF2 should be placed in Midi space. We simply disagree here. It's not the first time and I think it's not a problem. I think I never pushed you too far when we had a disagreement. It was not because I did not think at that time that my arguments were valid but because Bassmidi is your child and the final decision is yours. I hope this can work the other way around.
We had a conversation with Spessasus and we agreed that the 'DBNK' chunk can stay as an optional feature.
The choice is yours if you want to support it or not. I think it's a perfectly valid position that Bassmidi will not support it since you do not like it and you think it's unnecessary. I can accept it.
Bye.

[spessasus]

Hi Ian,

About the DLS version: I've managed to implement the support for older DLS RMID files.

The only files of this kind I have can be found here: https://archive.org/details/RIFF-MIDI-DLS

And one thing: some of these files use a bank offset of 1! And indeed expect a full GM soundfont at bank 0.
Others use bank 0! It's really messy. So Falco was right in suggesting the DBNK solution, it really cleans up the mess in the SF2 version we created.

I hope this clears things up,
Spessasus

PS: Is the support in BASSMIDI fully implemented now?

[Ian @ un4seen]

Yep, I think so. Here's the latest build:

   www.un4seen.com/stuff/bassmidi.zip

Let me know if there's any problem.

[Falcosoft]

...
And one thing: some of these files use a bank offset of 1! And indeed expect a full GM soundfont at bank 0.
Others use bank 0! It's really messy. So Falco was right in suggesting the DBNK solution, it really cleans up the mess in the SF2 version we created.
...

Hi,
Nice finding!
In the linked package actually 11 of the 15 files use destination Bank 1 and 13 of 15 files rely on a default GM soundfont at Bank 0 (even ones that do not use Bank offset of 1).
So despite what Ian suspected the majority of RMI files with embedded DLS were not self-contained in practice at all and many of them did not use Bank 0 as deafult.
What is more it seems the 'default' in practice was just the opposite (and the same as the default in our RMI+SF2 specification and in case of Creative's demo MID+SF2 files).
So it seems the demand for variaton based and offset using soundfonts was real even back then but the MMA made RMI+DLS specification was not explicit how to handle such cases.
Our RMI+SF2 specification is much more clear in this sense.     

PS:
Most of the files even with a bank offset of 1 can be played back properly by Winamp since MS Soft Synth can use Capital Tone Fallback so even if the Midi file refers to bank 1 and the soundfont is loaded to bank 0 by Winamp the missing bank 1 falls back to default bank 0. But one of the Midi files (HTONIGHT.rmi) uses instruments at bank 2 (in the DLS file these presets are at bank 1) and this way the CTF feature does not work since it skips the proper bank and falls back to bank 0 that results in wrong instruments. So this file strictly requires a bank offset of 1 for proper playback.
But to the rescue here is our new format so this file can be played back the way it is meant to be played
(The Bassmidi version linked by Ian above also works with this file).
Listen to channel 4. You should hear 'house tonight 22kHz' instrument with a human voice (from about 23 sec.):


 

[spessasus]

Well, winamp can't really play the external RMIs properly. Take AWEBLOWN.rmi and compare that result to my synth. Winamp only uses presets provided by the inner DLS file, which are kick&snare, bass, sax and slap bass. While the MID itself uses regular drums for hihats, piano, string and sax too.

Also, what's funny that the house, techno and rock demos (which were creative demos) expect bank 0, not 1. Strange, given that the hw synths can't get the bank 0 overriden according to Falco.

[Falcosoft]

Well, winamp can't really play the external RMIs properly. Take AWEBLOWN.rmi and compare that result to my synth. Winamp only uses presets provided by the inner DLS file, which are kick&snare, bass, sax and slap bass. While the MID itself uses regular drums for hihats, piano, string and sax too.

Also, what's funny that the house, techno and rock demos (which were creative demos) expect bank 0, not 1. Strange, given that the hw synths can't get the bank 0 overriden according to Falco.

Hi,
Just some corrections:
1. This is not true. Winamp (actually MS Soft synth) can play default instruments if they are not provided by the embedded DLS. There is an explicit setting for this problem on the Midi plugin's UI of Winamp.
I have attached an mp3 about how AWEBLOWN sounds with Winamp's Midi Plugin + Directmusic MS Soft synth. There are no missing instruments. Hihats, piano, string and sax are provided by the default MS Soft synth DLS file (gm.dls).

2. The house, techno and rock demos in the package are not from Creative but from Analog Devices (for their own DLS based soft synth/SoundMAX driver). Creative never made demo files based on DLS soundfonts. Analog Devices also provided an XG/GM compatible DLS file for the missing base instruments together with the demos (called DLSbyXG.DLS).
Here is a link for the original package:
http://www.vgmpf.com/Wiki/images/c/c9/Analog_Devices_Sound_Demo_%28W32%29.zip

But e.g. SURPRISE.rmi, TEKKNIKO.rmi, ONLYCHOI.rmi are originally SF2 demo files form Creative and these Midis use Bank 1 even in the DLS version.
Also from the names we can conclude that AWExxx.rmi files most likely were made for AWE32/64 that never handled DLS sondfonts so the originals most likely were SBK/SF2 files (these Midis also use Bank 1).
So it seem there is an overlap and the same files were converted from SBK/SF2 to DLS and vice versa.



   

[spessasus]

Hi again,

1. Well, for me either on wine or Windows 10, only the embedded presets play in AWEBLOWN. I've selected both "fix missing drum kits" and "fix missing instruments" and neither use the GM.DLS file. Glad it works for you though, maybe my install is broken or something?

2. I assumed that the rock_test.zip you uploaded on vogons was one of the Creative demos you talked about earlier there. Now I've checked the topic again and you never mentioned it. So that's a mistake on my end, I apologize.

Still, these 3 RMIs and things like PMDawn use bank 0, so the software needs to detect the bank offset since they lack DBNK.

PS: Will DLS RMIs be supported in FSMP6?

[Falcosoft]

Hi again,

1. Well, for me either on wine or Windows 10, only the embedded presets play in AWEBLOWN. I've selected both "fix missing drum kits" and "fix missing instruments" and neither use the GM.DLS file. Glad it works for you though, maybe my install is broken or something?

2. I assumed that the rock_test.zip you uploaded on vogons was one of the Creative demos you talked about earlier there. Now I've checked the topic again and you never mentioned it. So that's a mistake on my end, I apologize.

Still, these 3 RMIs and things like PMDawn use bank 0, so the software needs to detect the bank offset since they lack DBNK.

PS: Will DLS RMIs be supported in FSMP6?

1. I have tested with latest v5.92 of Winamp (Midi plugin version v3.57) and older v5.61 (Midi Plugin version 3.2) and the default instruments work with both.
I have pressed 'Reset' on the Midi Plugin's dialog and only changed the output to 'DirectMusic/Microsoft Synthesizer' and still both work without problems so you should try it. Make sure that on the 'Misc' tab playback method is 'Streamed' (it is the default anyway).
Do you hear the default gm.dls when you play MID/RMI files without embedded soundfont? 

2. Yes, there are definitely problems regarding destination Bank handling in RMI files with embedded DLS soundfont. I have just mentioned that there are no further 'strange' things since the files with Creative origins all use bank offset of 1 and the ones that do not use bank 1 are not from Creative originally.

3. FSMP already supports DLS handling in RMI files when using SB HW output (and SB Live! or higher is detected) since SB HW synths with newer drivers support loading DLS soundfonts. No special bank detection logic is present in FSMP.
DLS support in Bassmidi is waiting for Ian

[Ian @ un4seen]

It would be nice to have some DLS support in BASSMIDI. It won't be in the next release (which should be fairly soon), but perhaps in a future release.

[spessasus]

It would be nice to have some DLS support in BASSMIDI. It won't be in the next release (which should be fairly soon), but perhaps in a future release.

Hi Ian,
This code might be of use to you:
https://github.com/spessasus/SpessaSynth/tree/master/src/spessasynth_lib/soundfont/dls

It converts the DLS into an SF2 file while maintaining correct parameters and even converting articulators into SF2 modulators.
This means that you could internally convert it into SF2 in memory and then load it normally, without the need for a new synthesizer.
I hope this helps!

[Ian @ un4seen]

Thanks. Yep, that's the sort of thing I had in mind, translating the DLS stuff to already supported SF2 equivalents.