Author Topic: SIDex plugin  (Read 2979 times)

Keltic Danor

  • XMPlay Support
  • Posts: 829
SIDex plugin
« on: 19 Jul '21 - 09:51 »
Plugin Progress

So just a little update and there is good news and bad news.

The good news is that I asked the libsidplayfp guys the question and while I didn't get the answer I was after they gave me a soultion nonetheless, so I am using MingW (via NetBeans) instead which appears to work just fine so far.
With that worked out after a week+ of solidly smacking my face against the wall, I knocked some bricks loose and have an almost functional new plugin with all the tag loading, sub song handling, some basic settings and so forth.

The bad news is the only thing left is, of course, the most important bit, actually getting the beeps and boops to XMPlay. Math is not my strong point, nor is audio or conversion thereof which at least to my brain makes this bit of a problem. I understand the concept I think but the execution is another story.


Current Problem

So! With all that in mind I'll slap the details of the problem here and maybe one of you clever chaps can tell me "Why Keltic you silly sausage, you just divide by the square root of pie!" and we can try this puppy out.

This is the XMPlay Process function that is used to get the audio samples from the plug in as follows.
Code: [Select]

// get some sample data, always floating-point
// count=number of floats to write (not bytes or samples)
// return number of floats written. if it's less than requested, playback is ended...
// so wait for more if there is more to come (use CheckCancel function to check if user wants to cancel)

DWORD WINAPI SIDex_Process(float *buffer, DWORD count) {}


To get the goods we need to use the following function from libsidplayfp.
Code: [Select]

// Run the emulation and produce samples to play if a buffer is given.
// Parameter:   buffer pointer to the buffer to fill with samples.
// Parameter:   count the size of the buffer measured in 16 bit samples or 0 if no output is needed (e.g. Hardsid)
// Returns:   the number of produced samples. If less than requested and #isPlaying() is true an error occurred, use #error() to get a detailed message.

uint_least32_t Player::play(short *buffer, uint_least32_t count) {}



Thoughts So Far

So from start to finish with this whole thing I've had a very poor track record of thinking something is 10,000 more complicated than it actually is, or just plain over complicating things when I don't need to. *cough*Sorry for the silly emails Ian  :-X*cough*

Despite that I was thinking it would work something like the following.
Code: [Select]

DWORD WINAPI SIDex_Process(float *buffer, DWORD count){
     // I figure count can just be used in the play command since it is the same thing right?
     // I'll pop the result into a temporary buffer since XMPlay wants floats
     // sidDone is just to pass back how many we actually got
     short * sidBuffer;
     int sidDone=0;

     // Then we'll do the command to get some samples
     sidDone = sidEngine.m_engine->play(sidBuffer, count);

     // Next we'll do some fancy conversion of the sidBuffer from short to float and populate the buffer provided by XMPlay
     // I either have no idea how to do this, or my conversion was fine and the above is not actually correct.
     // I tested with memcpy(&buffer,&sidBuffer,std::min(sizeof(buffer),sizeof(sidBuffer))); just to get something through and it plays nonsense
     // Despite playing nonsense it does play which suggests the sidDone number is correct and it returned the count as requested
     // So my theory is if I can convert he short properly it will presumably work?

     // Let XMPlay know how many we got if it needs to stop
     return sidDone;
}


For reference though I have tried to figure it out by looking at foobars foo_sid, winamps in_sidplay2, hippoplayer sid plugin, the sid support in another xmplay plugin (zxtune) and the sidplayfp app. Over the development I have also looked at probably every single plug in for XMPlay that has source available, so quite the adventure. All of which has been very useful in the general building of the plug in but not for this bit in particularly.

Any thoughts are welcome.  ;D

Dhry

  • Posts: 121
Re: SIDex plugin
« Reply #1 on: 19 Jul '21 - 17:37 »
Outstanding work!! Anxiously awaiting anything you produce - happy to beta test.

DRS

drfiemost

  • Guest
Re: SIDex plugin
« Reply #2 on: 19 Jul '21 - 20:18 »
Float samples are usually in the range [-1,1] while short are in [-32768,32767] so you need to loop on each sample, cast it to float and dividing it by 32768.
Something along the lines of:

Code: [Select]
for (i=0;i<sidDone;i++)
    buffer[i] = (float)(sidBuffer[i])/32768.f

Keltic Danor

  • XMPlay Support
  • Posts: 829
Re: SIDex plugin
« Reply #3 on: 20 Jul '21 - 09:56 »
Float samples are usually in the range [-1,1] while short are in [-32768,32767] so you need to loop on each sample, cast it to float and dividing it by 32768.
Something along the lines of:

Code: [Select]
for (i=0;i<sidDone;i++)
    buffer[i] = (float)(sidBuffer[i])/32768.f

Oh hello again! ;D

No luck with that one unfortunately, it just crashes out. I've since gone through the whole thing checking the engine is working proper, that config is being applied okay and that isplaying returns true and as far as I can tell everything is working properly on that side of things. I also checked all the error() functions and all returned NA or N/A so I guess there are no sneaky errors on the sidplay side.

Edit 1:
That said whenever I refer to
Code: [Select]
sidBuffer[i] at all things crash and writing sidBuffer to a file without an index it is always 0, so maybe that's where my problem is.

Edit 2:
I figure since it was giving me 0 maybe it was the library somehow, I tried a different approach and built libsidplayfp via command line in MSYS2 (just ./configure && make so I didn't mess with it), I then made a brand spanking new project and included that libsidplayfp in the project instead of the one I built in netbeans to see if maybe I had made a mistake when building it myself. But after all that it produced the same result unfortunately. So must be something else! Back to the drawing board.


« Last Edit: 21 Jul '21 - 10:21 by Keltic Danor »

drfiemost

  • Guest
Re: SIDex plugin
« Reply #4 on: 20 Jul '21 - 19:55 »
Oh, yes! You should obviously allocate sdiBuffer before using it. Player::play just fills it with samples and won't do it for you.

Keltic Danor

  • XMPlay Support
  • Posts: 829
Re: SIDex plugin
« Reply #5 on: 21 Jul '21 - 09:21 »
Oh, yes! You should obviously allocate sdiBuffer before using it. Player::play just fills it with samples and won't do it for you.

It works! :o Thank you for all the help! ;D

I've attached a version for testing Dhry, I don't think it has any DLL dependency issues that I can see and also be aware of the following:
- Config settings may not save properly, it should work just... you know.
- Seeking is disabled - I don't think I'll do that bit + I don't know how to.
- Separating SID files in the play list and playing them individually doesn't work, not sure why but will look into it another time.
- Only the ReSIDfp core is available at this time, I may or may not add ReSID later, not sure it is necessary.
- Only full paths work for the songlength.md5 database for the moment.
« Last Edit: 22 Jul '21 - 10:55 by Keltic Danor »

Dhry

  • Posts: 121
Re: SIDex plugin
« Reply #6 on: 21 Jul '21 - 16:13 »
On it! Just woke up - will report back later.
THANK YOU!!!!!!!!!!

PS: Seeking's a must, for me anyway - there are quite a few SIDs where I skip them to parts that I like, but that's low pri as it's early days. I'm just glad for development to be active on SOMETHING that plays my favourite music format.

DRS

Dhry

  • Posts: 121
Re: SIDex plugin
« Reply #7 on: 21 Jul '21 - 16:24 »
Oops. Looks like there are missing dependencies - the plugin doesn't load or show in xmplay's list at all.

Dependency Walker shows the following, along with a list (attaching) of missing dependencies.

Error: At least one required implicit or forwarded dependency was not found.
Error: Modules with different CPU types were found.
Warning: At least one delay-load dependency module was not found.


All files in the attached list are marked: Error opening file. The system cannot find the file specified (2).

saga

  • Posts: 2550
Re: SIDex plugin
« Reply #8 on: 21 Jul '21 - 16:48 »
I hope you'll put this plugin on GitHub / GitLab / etc. so that others can finish the parts you mentioned you don't want to implement, and so that this whole situation is not going to happen a second time. :)

Dhry

  • Posts: 121
Re: SIDex plugin
« Reply #9 on: 21 Jul '21 - 16:58 »
I hope you'll put this plugin on GitHub / GitLab / etc. so that others can finish the parts you mentioned you don't want to implement, and so that this whole situation is not going to happen a second time. :)

Totally agreed!

BTW I tried installing both the Visual Studio 2015, 2017 and 2019 files (vc_redist.x86.exe and vc_redist.x64.exe) runtimes from https://support.microsoft.com/en-us/topic/the-latest-supported-visual-c-downloads-2647da03-1eea-4433-9aff-95f26a218cc0 and rebooted, and still the plugin does not load when starting xmplay. Googling the error I see something about statically linking libraries during compile time?

DRS

Sebby75

  • Posts: 23
Re: SIDex plugin
« Reply #10 on: 21 Jul '21 - 17:33 »

It works! :o Thank you for all the help! ;D

I've attached a version for testing Dhry, I don't think it has any DLL dependency issues that I can see and also be aware of the following:


Bloody marvelous.. thanks for this... but at the moment it doesn't seem to show up in plugin list... seem to require libstdc++-6.dll and libgcc_s_dw2-1.dll to run..
I think you need to link these during compilation??



Regards
Sebby75

drfiemost

  • Guest
Re: SIDex plugin
« Reply #11 on: 21 Jul '21 - 18:05 »
Guess the "-static-libgcc -static-libstdc++" flags are needed for linking

Keltic Danor

  • XMPlay Support
  • Posts: 829
Re: SIDex plugin
« Reply #12 on: 22 Jul '21 - 07:01 »
Hmmm... well poo. :(

Okay, added static linking for the MingW libraries which seems to make it work in my tests on another machine. It makes the .dll massive though which I'm not happy about... but it's okay *deep breath* just need to make sure it works before worrying about that I hope Scratch that, Dr. Fiemost to the rescue again, it makes it bigger but not too bad, you can UPX it to 400KB~ if you are hurting for space for portability reasons. I swear I've spent about a day and a half writing the plugin and 2 weeks fighting with IDE's and that nonsense.  ::)

Anyway here we are for now, too big to attach on the forum.

Downloads
xmp-sidex0.4.zip
xmp-sidex0.4source.zip

I don't know much about Github to be honest but I've added it here for the moment: https://github.com/KelticDanor/xmp-SIDex

Updates
v0.4
- Seeking sort of enabled, it is probably a bit janky
v0.3
- Separating SID files in the play list and playing them individually works now
- Config saving seems to work okay now

Notes
- Only the ReSIDfp core is available at this time, I may or may not add ReSID later, not sure it is necessary.
- Only full paths work for the songlength.md5 database for the moment.
- Unicode characters arn't handled so Mr Chris Hülsbeck is causing me mischief, still works just names are cut off.
« Last Edit: 22 Jul '21 - 11:22 by Keltic Danor »

Dr. Fiemost

  • Posts: 18
Re: SIDex plugin
« Reply #13 on: 22 Jul '21 - 08:56 »
Maybe it's just debugging symbols, you can try strip the DLLs or using the -s linking flag.

Keltic Danor

  • XMPlay Support
  • Posts: 829
Re: SIDex plugin
« Reply #14 on: 22 Jul '21 - 09:23 »
Maybe it's just debugging symbols, you can try strip the DLLs or using the -s linking flag.
Oh... you're good at this stuff. ;D

That took it from 11MB+ to 1.4MB, I'm happier with that, thanks again!

Dhry

  • Posts: 121
Re: SIDex plugin
« Reply #15 on: 22 Jul '21 - 16:21 »
It works!!! Great job!

Seeking also works here as well, as do subtunes. And you're right, unicode-style characters are cut off but honestly that's super low pri for me since I don't look at that particular section of xmplay (it's always been totally irrelevant for me) - I prefer the separate library/playlist window and display in there looks perfectly fine to me.

Just woke up and am madly playing with this like it's christmas morning. I'll keep messing with it throughout the day and let you know what I find. Thank you again!

DRS

Dhry

  • Posts: 121
Re: SIDex plugin
« Reply #16 on: 22 Jul '21 - 16:27 »
Oops. First defect: plays songs too fast.

Try:
\HVSC\MUSICIANS\H\Hubbard_Rob\Lightforce.sid
\HVSC\MUSICIANS\H\Hubbard_Rob\Master_of_Magic.sid

Both tracks play at something like double speed versus the original speed.

Just figured out what it was. I had changed the frequency from 44100 to 48000. This appears to increase playback speed where it should not.

Cheers
DRS

Dhry

  • Posts: 121
Re: SIDex plugin
« Reply #17 on: 22 Jul '21 - 18:25 »
Question: What SID filter settings are you defaulting to for playback?
Have a listen to Subtune 3 for Rob's Life (http://deepsid.chordian.net/?file=/MUSICIANS/H/Hubbard_Rob/Robs_Life.sid&subtune=3) - in particular the "snare drum" sound.

In your plugin's playback it sounds a bit muted. Sounds brighter in the Deepsid playback.
I know that the SID filter affects this particular sound significantly as I've messed around with it using the jSIDPlay device output and Acid64.

Just wondering if there are any modes that you can select in the libsidplay library for filter so that the user can choose 1) to use it or not, and/or 2) to either choose from preset default strengths (as with Acid64), or else (as with Fraggie's plugin) have a slider controlling the strength as a percentage for either 6581 or 8580. The implementation right now doesn't sound horrible, but that snare should sound a bit brighter IMO.

I know it's super early days right now so just curious.

Cheers
DRS

Sebby75

  • Posts: 23
Re: SIDex plugin
« Reply #18 on: 22 Jul '21 - 18:41 »
Now we're talking... Great Progress !!

Couple of things...

- displays 6581 PAL in player window regardless of what sid type is detected...   seem to detect and play 8580 tunes correctly but still displays 6581.. fiddling with sid type seem to crash it if something is playing..
- default duration... i have tried to put in 0 to get infinity but it is just not having it.. plays nothing and resets to 120seconds upon restart (edit... play duration doesnt seem to stick at all... goes back to 120seconds upon restart)
- play speed... If you start XMplay with this plugin set up as mono and then switch it to stereo... it plays half the speed..  If you start XMplay with this plugin set up as stereo and then switch it to mono... it plays double speed..


something to add possibly in the future (if possible) is adding SIDId (playroutine identity scanner) that was built into Fraggies plugin..
https://github.com/cadaver/sidid

Thanks again !!! Now i'll carry on testing :)
« Last Edit: 23 Jul '21 - 08:25 by Sebby75 »

Keltic Danor

  • XMPlay Support
  • Posts: 829
Re: SIDex plugin
« Reply #19 on: 23 Jul '21 - 11:00 »
Hey hey!

Ian gave it a once over and polished things up a little bit so seeking among other things should work a bit better (or properly in some cases). So thanks for that Chief! ;)

Feedback
Just figured out what it was. I had changed the frequency from 44100 to 48000. This appears to increase playback speed where it should not.
The problem with that one is changing the frequency in the plugin but XMPlay doesn't know about it hence fast/slow or some other oddities. One of the adjustments above also got rid of the frequency/channel settings in the plugin itself and they come from XMPlay instead so you won't get frequency issues like that anymore since XMPlay will know what is going on.

Question: What SID filter settings are you defaulting to for playback?
None, whatever libsidplayfp is doing by default is/was the level used. My old man ears can't tell much ;D so you will have to let me know on this one.

- displays 6581 PAL in player window regardless of what sid type is detected...   say it detects 8580 and plays correctly but still displays 6581.. fiddling with sid type seem to crash it if something is playing..
- default duration... i have tried to put in 0 to get infinity but it is just not having it.. plays nothing and resets to 120seconds upon restart
Yes I wasn't sure how to get those details but I think I have worked it out.
I forgot 0 duration was a thing, if you set it to 0 now it will play forever as expect but it will disable seek and won't show duration (unless songlengths.md5 provides one).
« Last Edit: 24 Jul '21 - 05:57 by Keltic Danor »

Sebby75

  • Posts: 23
Re: SIDex plugin
« Reply #20 on: 23 Jul '21 - 12:49 »
Yay !! Testing again...

And after 2 minutes I have managed to stumble upon tune (attached) that is crashing XMPlayer with version 0.5 of the plugin but plays fine with version 0.4.
Player window closes when I attempt to play it... Error message pops up when I try to add it to playlist.

Also had it play up with songlengths moment ago... it worked and then suddenly stopped to work and insisted on play time inputted in "default duration" instead.
Deleting path to songlengths (leaving box blank) and attempting to play anything results in XMPlay crashing.. After restart XMPlay is still crashing until new path to songlengths is provided..

Sorry !! Back to testing

Kindest regards
Sebby75

Keltic Danor

  • XMPlay Support
  • Posts: 829
Re: SIDex plugin
« Reply #21 on: 23 Jul '21 - 14:03 »
Thanks for the feedback!

Don't forget the DOCUMENTS path is now like this, no Songlengths.md5 in the path:

C:\_Resources\C64\C64Music\DOCUMENTS\
../C64Music/DOCUMENTS/

Sorry about the problems, I guess I changed a lot of stuff around today. :P
« Last Edit: 23 Jul '21 - 17:11 by Keltic Danor »

Sebby75

  • Posts: 23
Re: SIDex plugin
« Reply #22 on: 23 Jul '21 - 16:51 »
Damn you're quick...
 
I was writing comments to 0.5a as it was still trying to show STIL info in samples tab and crashing when switching between messages and samples tab (showing sometimes different garbage in samples tab after every switch)

...and you have gone through 2 further builds since...

Love it !!! back to testing :)

Keltic Danor

  • XMPlay Support
  • Posts: 829
Re: SIDex plugin
« Reply #23 on: 23 Jul '21 - 17:07 »
Actually about that, some more changes, figured out the 8580 filter issue....

-New Update listed below-
« Last Edit: 24 Jul '21 - 01:53 by Keltic Danor »

Dhry

  • Posts: 121
Re: SIDex plugin
« Reply #24 on: 23 Jul '21 - 18:18 »
Echoing Sebby's thoughts. Woke up this morning, had a crapload of work to get through and have only just checked in here to find multiple new versions to look at. Man, you have done an IMMENSE amount of work in the last couple of days and I cannot begin to express my appreciation for it. Have download 0.6b and am listening right now. I love it so far - filter mode and percentage works great, although I'd like number fields or sliders rather than 5-increment dropdown. Everything else works and sounds good, seeking is great and just from a five min messaround it seems to have everything I want. Will keep playing throughout the day.

Thanks again!
DRS