Author Topic: Bass encode opus encapsulation for websocket broadcast  (Read 160 times)

grahamg

  • Posts: 4
I am attempting to broadcast an opus encoded channel via websockets.

Using BASS_Encode_OPUS_Start I am successfully able to send the encoded bytes via a websocket, though the websocket client is unable to decode the opus data. This is likely due to the opus packets sent lacking any transport container defining the packet length ( From opus spec:
"to use the internal framing described in Section 3, the decoder must know the total length of the Opus packet, in bytes.") ,which according to the encoder specs usually exists in the transport layer -

"... it assumes
   that a lower layer (such as UDP or RTP [RFC3550] or Ogg [RFC3533] or
   Matroska [MATROSKA-WEBSITE]) will communicate the length, in bytes,
   of the packet, and it uses this information to reduce the framing
   overhead in the packet itself.  A decoder implementation MUST support
   the framing described in this section.  An alternative, self-
   delimiting variant of the framing is described in Appendix B."

There is an optional internal framing encoding included in Appendix B - though it still requires a transport layer to set the framing criteria.

So - has anyone done this previously? Any suggestions on how I can get the opus encoded data into the necessary containers using the un4seen libraries prior to sending over websockets?

I am aware of the built in server,  and have used this before for broadcasting via icecast - however the switch to websockets is for a low-latency solution. 

Ian @ un4seen

  • Administrator
  • Posts: 21333
The Opus data delivered by BASSenc_OPUS is in an Ogg container. Perhaps the client needs to be told that it's Ogg?

grahamg

  • Posts: 4
Is that each block of encoded data (packet?) delivered by the Bassenc_OPUS callback or just the first block? I ask this because for a test I tried writing all bytes from the callback to disk and then playing the raw file in VLC. If I i do not write the absolute first data block encoded to disk, then VLC does not play the file.

So instead of a plain opus decoder in my websocket client, I need an ogg-opus decoder?

Ian @ un4seen

  • Administrator
  • Posts: 21333
All of the data is in an Ogg container (your callback function will receive Ogg pages). The first packets contain headers, so should not be skipped.

grahamg

  • Posts: 4
Still battling with this... I am receiving the ogg opus packets in my client OK, but I gather the reason that Web Audio API or any other library cannot decode it for playback is that I am transmitting arbitrary packets of encapsulated opus data when a websocket client connects,  without the header packets which were generated one time when the opus encoder started.  The end goal is to be able to feed opus data to a WebRTC  audio stream which will likely also require these header packets. Options:

a) Capture the header packets received in the BASSenc_Opus callback process, and transmit them first to any connected client. Are you able to tell me how many bytes are contained in the header, or how to isolate the header bytes to persist?

b) Choose another payload encoder for streaming, so that a client can pick up the audio stream at any point in time without header files.

Any ideas appreciated.

Ian @ un4seen

  • Administrator
  • Posts: 21333
a) Capture the header packets received in the BASSenc_Opus callback process, and transmit them first to any connected client. Are you able to tell me how many bytes are contained in the header, or how to isolate the header bytes to persist?

The Ogg format details can be found here:

   www.xiph.org/ogg/doc/framing.html

The Opus headers end when there's an Ogg page with a granule position that isn't 0 or -1. A simple way to find that is to look for "OggS" (the start of an Ogg page) and then check the 64-bit value (granule position) 6 bytes after that, something like this:

Code: [Select]
for (int a = 0; a < datalen-14; a++) {
if (data[a] == 'O' && data[a+1] == 'g' && data[a+2] == 'g' && data[a+3] == 'S')) {
if (*(QWORD*)(data+a+6) && *(QWORD*)(data+a+6) != (QWORD)-1) {
headerlen = a;
headers = memdup(data, headerlen);
break;
}
}
}