It's been a while but I just wanted to say I did resolve this issue after countless hours of trial, error, debugging out values etc. I discovered that the problem wasn't solved, it would still drift, especially after I moved everything up from being 44100Hz to 48000Hz.
I debugged out the DSP_StreamCopy's buffer, as I had done with the other issue where I had an increasing delay on the output of audio, and found that, at what seems to be random points, the buffer would increase. This lead to the audio being delayed. I would start a recording with RecordStart and process it using my own RECORDPROC (without the Record Proc the recording would get delayed - possibly due to this problem, but I'm not changing it back). I then do a DSP_StreamCopy of the RecordStart and set the target mixer to be a No Sound mixer at 48000Hz. This is the mixer that would display the delayed recording while the recording itself was bang on. I know this from the VU's I had attached to the record and the mixer.
I implemented my own buffer marshal and it is now keeping the buffer down. For this testing period I'm still debugging out when it adjusts the sample rate of the mixer receiving the stream copy and I can see it just randomly increased at 00:06:03 in the morning, then at 07:18:09 and at 09:43:49 at time of writing this.
I have no idea why this would happen, but it does happen and it's a royal pain in the backside, so I'm hoping my findings here will help others get to the bottom of similiar issues after they've changed just about every flag they can and tried every option including not having a stream copy and just doing the record without a proc and plugging it into the mixer direct - which has the same drifting problem. Using a stream copy appeared to be the only way I could monitor and adjust the buffer. Also, No, the mixer that gets delayed doesn't report any stalls.
I'm just keeping my fingers cross that there are no other gotchas.