23 Oct '14 - 12:01 *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
 
   Home   Help Search Login Register  
Pages: 1 2 [All]
  Reply  |  Print  
Author Topic: Play from memory?  (Read 5903 times)
garson
Posts: 110


« on: 6 Jan '12 - 20:58 »
Reply with quoteQuote

Can XMPlay play files from memory?
There is ReadAhead option in .ini file, maybe it is for playing from memory?

Logged
Dotpitch
Posts: 2717


« Reply #1 on: 7 Jan '12 - 11:10 »
Reply with quoteQuote

See Illustrated Manual - 6.02 Secret INI settings.
Quote from: XMPlay Illustrated Manual
ReadAhead [3.6.0.14 and later] For mp3 files, XMPlay scans the entire file and then maintains the file in a read-ahead buffer (unless NoScanMP3=1 is set). XMPlay does not do this for other types of files, which may cause unwanted disc activity (repeated small reads). ReadAhead=[max file size in MB] will enable the specified read-ahead buffer for non-mp3 files. [Forum Link]
Logged
garson
Posts: 110


« Reply #2 on: 7 Jan '12 - 22:37 »
Reply with quoteQuote

Thanks for reply. I thought that is what I was looking for but this confuses me:
Quote
I'm not sure it's a good idea for XMPlay to cache files. Windows already uses free memory to cache files, so it's likely to be a waste of memory if XMPlay does that too, ie. 2 copies of the file in memory.

It sounds like what you really want is for the reading to be done as quickly as possible, ie. have it read ahead. That should already happen with MP3 files, as XMPlay scans the entire file to get its length (unless "NoScanMP3=1" is set), but here's an update with an option to do similar with other formats too...

If I play file that is 290MB big, and ReadAhead is set to 300MB, it means that whole file should be copied in memory and played from memory.
I've tried that but Used memory isn't 290MB bigger.
« Last Edit: 7 Jan '12 - 22:44 by garson » Logged
Ian @ un4seen
Administrator
Posts: 17037


« Reply #3 on: 10 Jan '12 - 15:36 »
Reply with quoteQuote

The "ReadAhead" memory belongs to Windows rather than XMPlay, so it doesn't show up in XMPlay's memory usage. As in the quoted post, it would be a bit wasteful to have Windows and XMPlay both allocating memory for the file Smiley
Logged
garson
Posts: 110


« Reply #4 on: 10 Jan '12 - 22:21 »
Reply with quoteQuote

I've just checked Cached memory in Task Manager and it did increase for 350MB when playing 350MB file. Smiley
Logged
Zzyzx
Posts: 20


« Reply #5 on: 28 Jan '12 - 23:28 »
Reply with quoteQuote

Maybe someone can help... I'm having a hard time telling if the read ahead is actually enabled and working properly.

The Options and Stuff dialog says XMPlay version 3.0.6.50 at the bottom (just downloaded today).  Running under Windows 7 Ultimate 64 bit.

I added ReadAhead=1024 to xmplay.ini right under the [XMPlay] header.  Restarted XMPlay.  Note I set the read ahead to a very large size (a Gig) to accommodate some pretty large files in my library (longer duration and/or 24 or 32 bit, 96 KHz, etc.).

I'm playing a 8:02 duration 24 bit 44.1 KHz uncompressed PCM wave file, size 124.7 MB.  It's coming off a NAS box, via a Windows shared drive.

I look at Task Manager:  The xmplay.exe process shows 12,076 K memory usage.  Memory usage is 5.48 GB of 16 GB total physical memory, cached is 8476 MB.

In the playlist, XMPlay moves to the next file, 20:04 duration, 311.2 MB in size.

I look again at Task Manager:  No change at all to the process or overall memory usage numbers.  Shouldn't I see increases when moving to the bigger file?  Or would XMPlay allocate the entire 1024 MB read ahead cache when it starts up?

I tried another test:  I was playing another similar 20 minute wave file, and let it get a couple of minutes into it.  I would assume by now it should have had a chance to read ahead for most, if not all of the file.

I started streaming some video from the NAS drive, and also copied a couple of large files around for a minute.  Enough that prior to read ahead it would have overloaded the NAS and/or network and interrupted the music playback.

I would have expected the read ahead would allow playback to continue uninterrupted from the read ahead buffer.  But it was interrupted just as before I had enabled the read ahead.

So by the numbers in Task Manager and the observed performance I'm not getting that it's working.  What do you think?  Am I missing something, or not understanding how this is supposed to work, or what?  Is there some other way to tell if this is working as designed?

Thanks in advance,
Paul
« Last Edit: 28 Jan '12 - 23:30 by Zzyzx » Logged
garson
Posts: 110


« Reply #6 on: 29 Jan '12 - 21:25 »
Reply with quoteQuote


I look at Task Manager:  The xmplay.exe process shows 12,076 K memory usage.  Memory usage is 5.48 GB of 16 GB total physical memory, cached is 8476 MB.

In the playlist, XMPlay moves to the next file, 20:04 duration, 311.2 MB in size.

I look again at Task Manager:  No change at all to the process or overall memory usage numbers.  Shouldn't I see increases when moving to the bigger file?  Or would XMPlay allocate the entire 1024 MB read ahead cache when it starts up?

XMPlay doesn't allocate that memory, it's cached by Windows.
Try to monitor cached memory before and after adding and playing some file.
Logged
Zzyzx
Posts: 20


« Reply #7 on: 30 Jan '12 - 04:20 »
Reply with quoteQuote

XMPlay doesn't allocate that memory, it's cached by Windows.
Try to monitor cached memory before and after adding and playing some file.

Where should I see the memory usage show up?  As I mentioned, in Task Manager on the Performance tab, the memory chart doesn't change nor do the number in the Physical Memory pane (including the Cached value).  If I pull up the Resource Manager and watch the Memory tab, I also see no changes including in the Physical Memory section where there is a Cached value.  Where exactly are you seeing the effects?  Under what conditions -- empty playlist loading a single file, or with multiple files, or Huh  What would make for the best test -- perhaps close down XMPlay, load it up with an empty playlist and then load one file?  I thought I had done that, but I'll try again and see...

Thanks,
Paul

Logged
Zzyzx
Posts: 20


« Reply #8 on: 30 Jan '12 - 04:36 »
Reply with quoteQuote

I thought I had done that, but I'll try again and see...

Nope, I shut every program down.  Opened XMPlay with an empty playlist.  The In Use memory went up about 6 MB.  I loaded a 25+ minute 24 bit 44.1 KHz wave file that's 401 MB in size.  No movement of In Use or Cached memory number.  I must be looking in the wrong place, or something's not working as expected?

Logged
Ian @ un4seen
Administrator
Posts: 17037


« Reply #9 on: 30 Jan '12 - 14:50 »
Reply with quoteQuote

The "ReadAhead" option doesn't currently apply when playing from networked drives (only local drives), so it won't be having any effect in your case. Regarding the interruptions... what buffer length do you have set in the "Output" options, and does increasing that help at all?
Logged
garson
Posts: 110


« Reply #10 on: 30 Jan '12 - 18:55 »
Reply with quoteQuote

Ian,
is there a plan to apply this when playing from network drive?
Logged
Ian @ un4seen
Administrator
Posts: 17037


« Reply #11 on: 31 Jan '12 - 15:51 »
Reply with quoteQuote

Not really, but if there is good reason to do so, it could be extended to apply then too. The "ReadAhead" option was originally added to allow hard drives to sleep (rather than working throughout playback), so network files were excluded.
Logged
Zzyzx
Posts: 20


« Reply #12 on: 4 Feb '12 - 02:32 »
Reply with quoteQuote

The "ReadAhead" option doesn't currently apply when playing from networked drives (only local drives), so it won't be having any effect in your case. Regarding the interruptions... what buffer length do you have set in the "Output" options, and does increasing that help at all?

3.000s  - can't get it any higher from the options dialog.  Is there a way to make it higher by directly editing the .ini file?

Thanks,
Paul
« Last Edit: 4 Feb '12 - 15:44 by Zzyzx » Logged
Zzyzx
Posts: 20


« Reply #13 on: 4 Feb '12 - 02:38 »
Reply with quoteQuote

Not really, but if there is good reason to do so, it could be extended to apply then too. The "ReadAhead" option was originally added to allow hard drives to sleep (rather than working throughout playback), so network files were excluded.

Well, from my perspective, there is a good reason to do so.   Smiley

Specifically, a network storage device can host a lot of different content (and a whole lot of content, mine is an 8 TB box -- can't get that much on a local drive and wouldn't want to).  If it gets thumped with requests to initiate video streams, that can starve the playing audio stream.  Also, the uVerse router I'm forced to use by AT&T is poor at handling bursts of network traffic, which can also interrupt an audio stream.

I'm just curious, why did you decide a read ahead buffer would only be applicable to a local drive?  Is there something really different or difficult about implementing it for a network drive?

Thanks,
Paul
« Last Edit: 4 Feb '12 - 15:43 by Zzyzx » Logged
Jimmy Neutron
Posts: 436


« Reply #14 on: 4 Feb '12 - 20:01 »
Reply with quoteQuote

While I had nothing to do with the process, I think the present results came from fixing a specific problem someone had.  They complained "my hard drive...." and so the fix was devised to address "my hard drive".

Of course, I could be wrong on this....
Logged
Zzyzx
Posts: 20


« Reply #15 on: 4 Feb '12 - 22:50 »
Reply with quoteQuote

The "ReadAhead" option doesn't currently apply when playing from networked drives (only local drives), so it won't be having any effect in your case. Regarding the interruptions... what buffer length do you have set in the "Output" options, and does increasing that help at all?
3.000s  - can't get it any higher from the options dialog.  Is there a way to make it higher by directly editing the .ini file?

I notice when I change from the minimum 01.s to the maximum 3.0s, 4 bytes in the DeviceMode entry in xmplay.ini change from 0000 to 540B.  The curious part of me wants to try changing those 4 bytes to FFFF to try for an output buffer setting larger than 3.0s, but without a bit map of that whole DeviceMode string, I have no idea if that would work or mess with some other settings bits causing something bad to happen...  Huh

Also, I'm wondering that with an output buffer setting of 3.0s, it seems kinda weird that I'd get interrupted playback.  I mean, wouldn't the flow of audio data from the network storage to XMPlay have to be interrupted for more than 3 seconds before I hear an issue?  Maybe that's really happening, but 3 seconds seems a bit extreme for the NAS not to deliver some additional audio data to keep playback going.

BTW, I've monitored performance while the interrupted playback happens, and it doesn't seem to be CPU related -- the thread is never pegged for CPU usage.

Thanks,
Paul
Logged
Dotpitch
Posts: 2717


« Reply #16 on: 5 Feb '12 - 10:21 »
Reply with quoteQuote

Also, I'm wondering that with an output buffer setting of 3.0s, it seems kinda weird that I'd get interrupted playback.  I mean, wouldn't the flow of audio data from the network storage to XMPlay have to be interrupted for more than 3 seconds before I hear an issue?  Maybe that's really happening, but 3 seconds seems a bit extreme for the NAS not to deliver some additional audio data to keep playback going.
BTW, I've monitored performance while the interrupted playback happens, and it doesn't seem to be CPU related -- the thread is never pegged for CPU usage.
Could you try what happens if you run XMPlay and the video streams on different machines?

Ian, is there a way to view what files Windows currently has cached?
Logged
Zzyzx
Posts: 20


« Reply #17 on: 5 Feb '12 - 21:46 »
Reply with quoteQuote


Could you try what happens if you run XMPlay and the video streams on different machines?


I normally play audio on my office machine, while the video streams are running on other machines in other rooms.  I'm pretty sure it's the NAS and/or router that gets overloaded for a few seconds and interrupts my audio.  My local machine seems pretty quiet as far as CPU and its local network adapter activity.  3 seconds seems a long time, but I guess it's possible especially given the burst of file activity when a new video stream starts up.  I'd like to see what happens if my local machine can cache more than 3.0 seconds of audio, but apparently I've maxxed out the output buffer setting on XMPlay, and the file caching read ahead doesn't apply to network drives.
Logged
Ian @ un4seen
Administrator
Posts: 17037


« Reply #18 on: 6 Feb '12 - 17:38 »
Reply with quoteQuote

Specifically, a network storage device can host a lot of different content (and a whole lot of content, mine is an 8 TB box -- can't get that much on a local drive and wouldn't want to).  If it gets thumped with requests to initiate video streams, that can starve the playing audio stream.  Also, the uVerse router I'm forced to use by AT&T is poor at handling bursts of network traffic, which can also interrupt an audio stream.

Wouldn't the reverse be true too? If XMPlay tried to read the audio file as quickly as possible, it'd interrupt video streams? Smiley

I'm just curious, why did you decide a read ahead buffer would only be applicable to a local drive?

As I mentioned, it was to allow local drives to sleep (eg. to save laptop battery) rather than having to work throughout playback.

I notice when I change from the minimum 01.s to the maximum 3.0s, 4 bytes in the DeviceMode entry in xmplay.ini change from 0000 to 540B.  The curious part of me wants to try changing those 4 bytes to FFFF to try for an output buffer setting larger than 3.0s, but without a bit map of that whole DeviceMode string, I have no idea if that would work or mess with some other settings bits causing something bad to happen...  Huh

FFFF is a bit excessive, but you can indeed try increasing that number to increase the buffer length. Note it's in little-endian order, ie. the number is actually 0B54, or 2900 decimal (add 100 for the buffer length in milliseconds).

Also, I'm wondering that with an output buffer setting of 3.0s, it seems kinda weird that I'd get interrupted playback.  I mean, wouldn't the flow of audio data from the network storage to XMPlay have to be interrupted for more than 3 seconds before I hear an issue?  Maybe that's really happening, but 3 seconds seems a bit extreme for the NAS not to deliver some additional audio data to keep playback going.

BTW, I've monitored performance while the interrupted playback happens, and it doesn't seem to be CPU related -- the thread is never pegged for CPU usage.

Yep, it doesn't sound like a CPU issue, but rather a network issue. A 3 second buffer should indeed mean that an interruption shorter than that won't affect playback, so long as it can quickly recover afterwards, eg. there could be a delay and then a burst. In your case, perhaps the data flow is just constantly very slow, ie. not enough to sustain playback, meaning there will eventually be an interruption regardless of buffer length (as long as the slow flow continues).

Ian, is there a way to view what files Windows currently has cached?

I don't think so.
Logged
Zzyzx
Posts: 20


« Reply #19 on: 7 Feb '12 - 04:52 »
Reply with quoteQuote

Wouldn't the reverse be true too? If XMPlay tried to read the audio file as quickly as possible, it'd interrupt video streams? Smiley

LOL, yeah probably, but the kids deal with a glitch in Harry Potter better than I deal with a glitch in Pink Floyd.   Wink

FFFF is a bit excessive, but you can indeed try increasing that number to increase the buffer length. Note it's in little-endian order, ie. the number is actually 0B54, or 2900 decimal (add 100 for the buffer length in milliseconds).

Thanks for the details.  I'll play around with bigger numbers, although I understand what you're saying next...

Yep, it doesn't sound like a CPU issue, but rather a network issue. A 3 second buffer should indeed mean that an interruption shorter than that won't affect playback, so long as it can quickly recover afterwards, eg. there could be a delay and then a burst. In your case, perhaps the data flow is just constantly very slow, ie. not enough to sustain playback, meaning there will eventually be an interruption regardless of buffer length (as long as the slow flow continues).

Yes, it's possible it just can't keep up overall, especially since a lot of my audio is 24 bit.  Although I'm skeptical, since 24 bit 44.1 KHz is still only 2,116 kbps.  But I want to at least try a larger buffer to see what happens.  If it still gets interrupted with, say, a 10 second buffer, I'm probably out of luck.  Stay tuned...

Many Thanks,
Paul
« Last Edit: 7 Feb '12 - 04:55 by Zzyzx » Logged
Zzyzx
Posts: 20


« Reply #20 on: 27 Feb '12 - 02:11 »
Reply with quoteQuote

Yes, it's possible it just can't keep up overall, especially since a lot of my audio is 24 bit.  Although I'm skeptical, since 24 bit 44.1 KHz is still only 2,116 kbps.  But I want to at least try a larger buffer to see what happens.  If it still gets interrupted with, say, a 10 second buffer, I'm probably out of luck.  Stay tuned...

Just for a little closure...

I set the output buffer to 10 seconds by editing the DeviceMode setting in xmplay.ini and changing the "540B" (former 3 second maximum) value to "AC26".  The Output settings dialog shows the buffer as 10.000s.

After a lot of use and some specific load testing (nothing formal, just starting several video streams while playing audio, starting several large file copies while playing audio -- things that have resulted in interrupted playback before), I have not experienced any playback interruptions.

So, I guess my conclusion is that the audio stream was getting interrupted by at least 3 seconds.  I did not test out values other than 10 seconds to see what the threshold was for uninterrupted playback, but 10 seconds seems to be at least enough.  It also seems that there is enough network throughput normally for the buffer to refill itself, although I don't think there is any way to monitor the buffer status in real time (is there?).

I've seen some other threads elsewhere about the crappy performance of the u-Verse routers AT&T provides, especially the wireless network, so I suspect the root cause of this problem (and some other network slowness I see) is the router.

Since this has fixed my problem, I guess the ability to preload network based audio files is moot for me.

Thanks to everyone for comments and help!
Paul
Logged
Ian @ un4seen
Administrator
Posts: 17037


« Reply #21 on: 28 Feb '12 - 17:41 »
Reply with quoteQuote

Good to hear that you found a solution. Perhaps XMPlay's buffer slider range should be increased for scenarios like this. The problem is doing that makes setting smaller sizes more fiddly, but perhaps the slider could be made to follow a non-linear curve to overcome that. I'll look into it.
Logged
Ian @ un4seen
Administrator
Posts: 17037


« Reply #22 on: 2 Mar '12 - 15:08 »
Reply with quoteQuote

Here's an update that increases the buffer slider range to 5 seconds...

   www.un4seen.com/stuff/xmplay.exe
Logged
robertcollier4
Posts: 51


« Reply #23 on: 30 Dec '12 - 08:05 »
Reply with quoteQuote

I tried setting ReadAhead=200 option and it doesn't seem as if it is working. I am playing a 1 hour MP3 file and there are constant reads to the hard drive still showing up in Sysinternals Process Monitor. I understand that the Windows Filesystem Cache will do some buffering on its own of all files requested of it (unless instructed not to with FILE_FLAG_NO_BUFFERING) (http://www.codeproject.com/Articles/51678/Improve-responsiveness-in-Windows-with-the-FILE_FL) -- however, it would be nice if XMPlay would maintain and create/allocate its own buffer and keep the song in memory because the Windows Filesystem Cache can easily get invalidated from other computer usage. Also the Windows Filesystem Cache will not buffer the entire file and is not entirely controllable if you want to play something entirely from memory and then have a laptop completely spin down the hard-drive.
« Last Edit: 30 Dec '12 - 08:10 by robertcollier4 » Logged
Ian @ un4seen
Administrator
Posts: 17037


« Reply #24 on: 31 Dec '12 - 16:55 »
Reply with quoteQuote

If XMPlay buffered the entire file itself, there's still no guarantee that the buffer wouldn't get swapped out of memory if the system is low on memory, so I'm not sure doing that would be an improvement. It would most likely just mean that there are 2 copies of the file in memory: one held by XMPlay and one held by Windows' file cache. Besides, if you want the drive to sleep, it's unlikely that you will be doing anything (eg. loading files) that might result in the audio file being removed from the file cache Smiley

Regarding Process Monitor, the file reads there don't necessarily equate to disk activity, ie. they may be satisfied by the file cache. The Disk Monitor tool can be used to check disk activity. With the "ReadAhead" option enabled, you should see a burst of reads at the start and then no more (caused by XMPlay) during playback.
Logged
robertcollier4
Posts: 51


« Reply #25 on: 31 Dec '12 - 20:04 »
Reply with quoteQuote

If XMPlay buffered the entire file itself, there's still no guarantee that the buffer wouldn't get swapped out of memory if the system is low on memory, so I'm not sure doing that would be an improvement. It would most likely just mean that there are 2 copies of the file in memory: one held by XMPlay and one held by Windows' file cache.
Okay thanks, I understand that most of the time we can just rely on Windows file cache to do it.

However can I ask exactly what "ReadAhead" does? Or is the flag not used? The documentation seems to be misleading by saying that XMPlay is maintaining the buffer when it says:
Quote
For mp3 files, XMPlay scans the entire file and then maintains the file in a read-ahead buffer (unless NoScanMP3=1 is set). XMPlay does not do this for other types of files, which may cause unwanted disc activity (repeated small reads). ReadAhead=[max file size in MB] will enable the specified read-ahead buffer for non-mp3 files.

This snippet quoted from the documentation seems to imply that "XMPlay maintains the read-ahead buffer" when it should say something more like "XMPlay requests sectors ahead in the file in an attempt to get Windows to load the data into its own Windows file cache".
« Last Edit: 31 Dec '12 - 20:06 by robertcollier4 » Logged
garson
Posts: 110


« Reply #26 on: 1 Jan '13 - 16:45 »
Reply with quoteQuote

Maybe this can help for some future development.
Quote
cPlay uses Address Windowing Extensions (AWE) to allocate RAM to music files during playback. To enable this, the Lock Pages in Memory privilege has to be set as described here. It is advisable not to make this change until after the system is known to be stable.

cPlay diagnostics show whether AWE has successfully allocated the RAM; Process Explorer or Task Manager will show only a reduction in Available RAM. If Windows does not allocate enough RAM to hold the music to be played, cPlay reverts to standard memory management. (Adding more RAM will cure this. AWE should allow up to four GB to be added though this has not been tested.)
Logged
Pages: 1 2 [All]
  Reply  |  Print  
 
Jump to:  

Powered by SMF 1.1.20 | SMF © 2013, Simple Machines