Author Topic: Channels appear to be stuck (they loop the last half-second over and over)  (Read 780 times)

srylain

  • Posts: 22
https://clips.twitch.tv/SpeedySmallFishCharlieBitMe

This is a clip of someone playing my game, and at about 8 seconds in you can hear the audio as if it got stuck and is replaying the last half a second or so continuously. Because he's streaming (possibly local recording as well) I'd assume he's using up quite a bit of his CPU which might be causing this? His audio files are on an SSD and are being streamed using Bass, so I doubt it's caused by HDD usage being way too high. My game also uses the position of the audio to determine how to stay synced over time to correct for drifting, which is why it rewinds when it happens so obviously this also causes adverse gameplay effects when it happens along with the audio not properly playing.

The audio in question that he was playing has 8 tracks, all of them are OGG and I believe they're all VBR if that helps any. Would it be better to maybe load the audio in all at once to try and prevent streaming issues (assuming this is a streaming issue)?

Ian @ un4seen

  • Administrator
  • Posts: 21188
It sounds like BASS's update thread may be stuck/deadlocked, so the playback buffers aren't getting updated. When that happens, it is usually a deadlock with the main thread. For example, a callback function does some UI stuff that needs to wait for the main thread, while at the same time the main thread is waiting for the other thread to release a lock. That doesn't appear to be the case in your video, as the entire game isn't frozen? Perhaps there's a deadlock with a different thread then. The first thing to check is whether you are using any callback functions (eg. SYNCPROC/DSPPROC/STREAMPROC), and if so, what they are doing, ie. do any of them do anything that means they may have to wait for another thread?

srylain

  • Posts: 22
I'm not using any callback functions, all I'm doing with the audio is loading the channels (the audio is streamed) and then setting the tempo effect (to be able to speed up or slow down the audio) on them and then linking them together and playing it. AFAIK most everybody says the game will run on around ten year old hardware just fine, but obviously people running that kind of hardware usually aren't streaming or local recording like in the video so maybe it is caused by the broadcasting software?

Unity mostly does everything on a single thread save for some of the rendering stuff, and the only problem I've had with the audio playing before was a post I made a couple weeks ago (about how playing a sound effect would cause the audio to stutter) which I fixed by preloading the sound effects instead of streaming them. All of the game logic itself is all on a single thread as well (which would be the same thread Unity uses).

So far this is the only time I've seen this happen out of the ten thousand or so people that have played the game, so it seems to be extremely rare. If I can get his PC specs and whether he was local recording, would that help in any way to try and determine something?

EDIT: I forgot I didn't post the longer version of the clip. Basically what ended up happening is that the audio did start playing normally again, but it started from the point that it started stuttering instead of where it would normally be (which caused the gameplay to rewind to the audio position). Is there a way to instead have it pickup where it should be at instead of the earlier position?
« Last Edit: 8 Nov '17 - 05:44 by srylain »

Ian @ un4seen

  • Administrator
  • Posts: 21188
Ah. If the audio did eventually resume from the same position, then it could perhaps have been an I/O delay (a very long one by the looks of it). I'm not sure what could cause such a long delay on a local drive; perhaps a damaged disk? Do you know if the user is able to reproduce the problem when playing the same file(s) again?

If you are playing multiple files, you could enable multiple update threads via the BASS_CONFIG_UPDATETHREADS option. If one stream gets delayed for any reason then, the others can continue playing normally. I'm not sure if that would actually sound better in the video's scenario, but it would also allow BASS to make better use of multiple CPU cores.

srylain

  • Posts: 22
AFAIK he has the game and the audio all stored on an internal SSD, and he was broadcasting and local recording at the same time (which is what he usually does) and the recording was going to an external HDD. I'll have to ask him to see if he can replay it to see if it happens again.

As for enabling multiple update threads, would there be any noticeable performance impact from doing that? I believe most broadcasting software (I think OBS is used mostly for people broadcasting my game) try and use up as much CPU they can get access to so hopefully any performance loss isn't too much that it would cause problems.

Ian @ un4seen

  • Administrator
  • Posts: 21188
Using multiple update threads allows multiple streams to have their playback buffers updated simultaneously, so it can only really help the audio performance (reduce the chances of playback buffers running out of data).

The BASS update thread(s) run at time critical priority, so they shouldn't usually be starved of CPU for the length of time shown in the video. An I/O delay seems more likely, but I guess it could also be something else, eg. a temporary deadlock, but if you're not using any callback functions, that seems unlikely. If the user can produce the problem again, it would be good to get a dump file of it happening (can use Task Manager's "Create Dump File" option for that).

srylain

  • Posts: 22
I just asked him to create that dump file the next time it happens, it happened again earlier today (with different files though) and it sounds like he wasn't local recording at the same time this time so that's less HDD usage and it still happened. So I'll send that when it happens.

srylain

  • Posts: 22
Hey, sorry about bringing up an old issue but there's been a few more people that have had problems with the audio getting stuck. Here's a video with some debug stats: https://youtu.be/O0Lbsh0V-Wk

There's some debug text on screen, which is the audio's playback position, the note position (it's kept separate so it updates more smoothly), and the difference between the two (there's logic that syncs the note position to the audio position over time to fix gradual floating point issues) but you can clearly see the audio position stops moving after a point and that's what causes the gameplay to rewind.

If it were possible, the easiest thing to do would be for BASS to have a way to keep going even if it can't play anything. What I mean is say that the audio gets stuck 10 seconds in and it's stuck for 20 seconds. Instead of resuming from 10 seconds in it would resume from 30 seconds in. Is there anyway to get BASS to do that? If not there's logic I can make that will get BASS to seek forward after it's done being stuck to get back to what it should be at.

Also, last time you asked for a DMP file from the game Ian and I think I have one. It's over 700MB, not sure if that's usual or is a problem. https://drive.google.com/file/d/1XUZNwOyx-clewiKBDSaMHVoc4ASpb8kP/view

Ian @ un4seen

  • Administrator
  • Posts: 21188
Looking at the dump file, there appear to be 8 update threads, ie. BASS_CONFIG_UPDATETHREADS is set to 8? All of the them are free (none mid-processing), so the problem doesn't appear to be caused by a deadlock or long I/O delay. There are only 2 streams existing (a decoder and a BASS_FX stream), so apparently no stream leaks. Was the dump file definitely generated while the problem was happening?

Could you try upgrading to the latest BASS.DLL version and see if the problem still happens then?

srylain

  • Posts: 22
I had initially set the thread count to 2 and the problem still happened for that person, so I had tried 8 to see if anything was different. I'd have to ask him if he generated the dump while it was happening, but it's definitely possible that he did it after.

I'll get to updating and report back.

srylain

  • Posts: 22
So after updating to 2.4.13.8 it seems like he hasn't had any problems with the audio getting stuck, but there may be a problem with BASS reporting the wrong playback position.

I'm checking the playback position every second and then comparing it to the elapsed game time, and then smoothing the game time towards the playback position over the next second if the difference between the two is great enough. This is what causes the gameplay to slow down, speed up or rewind if the difference is too great and since BASS seems to be reporting the wrong position it ends up trying to smooth the times together even though it doesn't need to.

I need to get a more debug friendly build out to him to help see when it's actually happening which I can then hopefully get another dump file. But is there anything that would normally cause BASS to report the wrong playback position?

Ian @ un4seen

  • Administrator
  • Posts: 21188
What time source are you comparing the position with? How big is the difference, and does it get there gradually or suddenly? If it's sudden, then perhaps playback was stalled? If you aren't already doing so, you can set a BASS_SYNC_STALL sync (via BASS_ChannelSetSync) to get notified when playback stalls (due to lack of buffered data).

srylain

  • Posts: 22
I keep track of an elapsed time value (so basically the time between frames) so it keeps the game updating smoothly, and that's what I compare the playback position with. The difference is normally really small, and if it gets too big (3ms or more) it then gets synced over the next second. So normally when the syncing happens, it's a small enough difference that it's not visible and it usually take a good amount of time to happen (upwards of 20 minutes or so).

For this person the desync happens gradually, but before updating BASS it would happen suddenly and that's when it would start looping repeatedly. As far as I can tell the audio never stops so I don't think it gets stalled, but I'll see what happens when I add that sync.