Topic: Issues sending opus without ogg over socket

[mkllc]

Hi, I just saw a recent version of bassopus allows to use opus without the ogg container. I tried a simple experiment: record from line/mic input, encode it with opus, on the ENCODEPROC send it thru a UDP socket and then receive it and push it thru BASS_OPUS_StreamCreate on the receiving socket. In theory everything should be working but I'm getting stuttering sound, so looks like something gets recorded and arrives on the receiving side but not in good shape, maybe small fragments or without the full frame.

Maybe it's something related with flags, or I did not understood some parts on the documentation. The goal was to try to do it with less latency as possible allowing for some package loss.

What I did on the sender:

Code: [Select]

BASS_RecordInit
rec = BASS_RecordStart with MAKELONG(BASS_SAMPLE_FLOAT, 10)
opus = BASS_Encode_OPUS_Start  with framesize 10 and no-ogg
on the ENCODEPROC send buffer and size over UDP
On the receiver:

Code: [Select]

BASS_Init
play = BASS_OPUS_StreamCreate (with BASS_STREAMPROC_OPUS_LOSS flag and STREAMPROC_PUSH)
While receiving UPD data, push it using BASS_OPUS_StreamPutData(play, buffer, length)
I tried in the same computer and over two computers on the same network, the stuttering it's the same. What I'm missing? Also will BASS_STREAMPROC_OPUS_LOSS needs to be used on the BASS_Encode_OPUS_Start too?

I tried to debug a bit and on RECORDPROC I get 960 bytes on each call, and on the ENCODEPROC this gets reduced to 320 bytes. The first packet is 19 bytes, if I understood properly the documentation this information is part of the BASS_OPUS_HEAD struct, I tried on the receiver to wait to get that first packet, copy it to the BASS_OPUS_HEAD struct and then initialize the BASS_OPUS_StreamCreate, but looks like it doesn't contain proper information like version, channels, sample rate. So maybe I misunderstood the documentation. Also I have the return true on RECORDPROC I think no more logic is required there.

Looks like everything should work and is very close to do it, but I need a bit of help or some pointers on how to debug or what settings to do. Thanks!

PS: I attached the audio that was playing on the tx side and the audio recorded on rx side (recorded outside the program, is what it's heard on each side)

[Ian @ un4seen]

I tried to debug a bit and on RECORDPROC I get 960 bytes on each call, and on the ENCODEPROC this gets reduced to 320 bytes. The first packet is 19 bytes, if I understood properly the documentation this information is part of the BASS_OPUS_HEAD struct, I tried on the receiver to wait to get that first packet, copy it to the BASS_OPUS_HEAD struct and then initialize the BASS_OPUS_StreamCreate, but looks like it doesn't contain proper information like version, channels, sample rate. So maybe I misunderstood the documentation. Also I have the return true on RECORDPROC I think no more logic is required there.

Looks like everything should work and is very close to do it, but I need a bit of help or some pointers on how to debug or what settings to do. Thanks!

You should indeed use the first packet contents in the BASS_OPUS_StreamCreate call, but you need to skip the "OpusHead" signature, ie. the first 8 bytes. So something like this:

Code: [Select]

if (!memcmp(packetdata, "OpusHead", 8)) { // check we've got an Opus header
BASS_OPUS_HEAD *header = (BASS_OPUS_HEAD*)(packetdata + 8); // skip the signature
stream = BASS_OPUS_StreamCreate(header, 0, STREAMPROC_PUSH, 0);
...

Let me know if it's still not working properly.

[mkllc]

Hi! I just did it, sorry I forgot to check the RFC7845 to see the header format. I was expecting directly the BASS_OPUS_HEAD. Now it's solved and I get the proper values on the BASS_OPUS_HEAD from the first packet prior to do the BASS_OPUS_StreamCreate. But the problem with the sound is the same, the same stuttering that appears in the audio_rx.mp3 from previous post.

Shoudl I'll use MAKELONG(flags, 10) on BASS_RecordStart to get lower latencies? I set framesize 10 on the Opus Encoder, maybe those values should match? Also I have a BASS_CONFIG_REC_BUFFER of 200ms. I'll play with the buffers/latencies because the issue should be around there, what do you think? I'm missing something?

Edit: looks like setting BASS_CONFIG_REC_BUFFER to 2000 ms improve the situation, still there are something it's missing in the sound, I uploaded a file called buffer_2000ms_rx_tx_samples.zip (and _example2.zip) to the incoming folder with the sample on how it sounds now on the reception side. I'm using: bitrate 256 hard-cbr comp 9 framesize 40 expect-loss 30 set-ctl-int 4012=1 no-ogg, on the Opus encoder. And the two computers are on the same LAN, so no internet traffic.

Another related question, how the packet recovering works? From the documentation looks like I have to signal it from BASS_OPUS_StreamPutData adding BASS_STREAMPROC_OPUS_LOSS instead of the length. I was expecting this to be managed by Opus codec itself. At this time I'm sending raw data on the socket, so I have no way to know if any packet was missing, so to do something like this I'll have to add some header to the data with a id or serial and check if any packet was missing and trigger the BASS_STREAMPROC_OPUS_LOSS but don´t know if it's the way it should work.

[Ian @ un4seen]

I'm using: bitrate 256 hard-cbr comp 9 framesize 40 expect-loss 30 set-ctl-int 4012=1 no-ogg, on the Opus encoder.

Trying those settings here, it looks like "expect-loss 30 set-ctl-int 4012=1" doesn't fit well with "hard-cbr". Please try removing "hard-cbr" (possibly replacing with "cvbr") and see what it sounds like then.

Another related question, how the packet recovering works? From the documentation looks like I have to signal it from BASS_OPUS_StreamPutData adding BASS_STREAMPROC_OPUS_LOSS instead of the length. I was expecting this to be managed by Opus codec itself. At this time I'm sending raw data on the socket, so I have no way to know if any packet was missing, so to do something like this I'll have to add some header to the data with a id or serial and check if any packet was missing and trigger the BASS_STREAMPROC_OPUS_LOSS but don´t know if it's the way it should work.

You could detect lost (or out-of-order) packets by having a packet counter at the start of each UDP packet. If the current packet isn't 1 higher than the last, then fill the gap with BASS_STREAMPROC_OPUS_LOSS (one for each lost packet).

[mkllc]

Trying those settings here, it looks like "expect-loss 30 set-ctl-int 4012=1" doesn't fit well with "hard-cbr". Please try removing "hard-cbr" (possibly replacing with "cvbr") and see what it sounds like then.

Wow, thanks a lot Ian, removing the hard-cbr do the trick. I didn't specified any, just leave it without the parameter. Is totally counter-intuitive, at least for me as usually fixed links are encoded using CBR to avoid time glitching or similar. But happy that works now under 1 seconds of delay from recording from mic to playing thru speakers in LAN. I'll investigate the way to simulate and signal packet loss, as you suggested adding some header field to indicate current packet number. Again, thanks for your time.