Author Topic: Bass encode opus encapsulation for websocket broadcast  (Read 689 times)

grahamg

  • Posts: 4
I am attempting to broadcast an opus encoded channel via websockets.

Using BASS_Encode_OPUS_Start I am successfully able to send the encoded bytes via a websocket, though the websocket client is unable to decode the opus data. This is likely due to the opus packets sent lacking any transport container defining the packet length ( From opus spec:
"to use the internal framing described in Section 3, the decoder must know the total length of the Opus packet, in bytes.") ,which according to the encoder specs usually exists in the transport layer -

"... it assumes
   that a lower layer (such as UDP or RTP [RFC3550] or Ogg [RFC3533] or
   Matroska [MATROSKA-WEBSITE]) will communicate the length, in bytes,
   of the packet, and it uses this information to reduce the framing
   overhead in the packet itself.  A decoder implementation MUST support
   the framing described in this section.  An alternative, self-
   delimiting variant of the framing is described in Appendix B."

There is an optional internal framing encoding included in Appendix B - though it still requires a transport layer to set the framing criteria.

So - has anyone done this previously? Any suggestions on how I can get the opus encoded data into the necessary containers using the un4seen libraries prior to sending over websockets?

I am aware of the built in server,  and have used this before for broadcasting via icecast - however the switch to websockets is for a low-latency solution. 

Ian @ un4seen

  • Administrator
  • Posts: 21539
The Opus data delivered by BASSenc_OPUS is in an Ogg container. Perhaps the client needs to be told that it's Ogg?

grahamg

  • Posts: 4
Is that each block of encoded data (packet?) delivered by the Bassenc_OPUS callback or just the first block? I ask this because for a test I tried writing all bytes from the callback to disk and then playing the raw file in VLC. If I i do not write the absolute first data block encoded to disk, then VLC does not play the file.

So instead of a plain opus decoder in my websocket client, I need an ogg-opus decoder?

Ian @ un4seen

  • Administrator
  • Posts: 21539
All of the data is in an Ogg container (your callback function will receive Ogg pages). The first packets contain headers, so should not be skipped.

grahamg

  • Posts: 4
Still battling with this... I am receiving the ogg opus packets in my client OK, but I gather the reason that Web Audio API or any other library cannot decode it for playback is that I am transmitting arbitrary packets of encapsulated opus data when a websocket client connects,  without the header packets which were generated one time when the opus encoder started.  The end goal is to be able to feed opus data to a WebRTC  audio stream which will likely also require these header packets. Options:

a) Capture the header packets received in the BASSenc_Opus callback process, and transmit them first to any connected client. Are you able to tell me how many bytes are contained in the header, or how to isolate the header bytes to persist?

b) Choose another payload encoder for streaming, so that a client can pick up the audio stream at any point in time without header files.

Any ideas appreciated.

Ian @ un4seen

  • Administrator
  • Posts: 21539
a) Capture the header packets received in the BASSenc_Opus callback process, and transmit them first to any connected client. Are you able to tell me how many bytes are contained in the header, or how to isolate the header bytes to persist?

The Ogg format details can be found here:

   www.xiph.org/ogg/doc/framing.html

The Opus headers end when there's an Ogg page with a granule position that isn't 0 or -1. A simple way to find that is to look for "OggS" (the start of an Ogg page) and then check the 64-bit value (granule position) 6 bytes after that, something like this:

Code: [Select]
for (int a = 0; a < datalen-14; a++) {
if (data[a] == 'O' && data[a+1] == 'g' && data[a+2] == 'g' && data[a+3] == 'S')) {
if (*(QWORD*)(data+a+6) && *(QWORD*)(data+a+6) != (QWORD)-1) {
headerlen = a;
headers = memdup(data, headerlen);
break;
}
}
}

IVGSoft

  • Posts: 4
I got almost same situation.
I have UDP server that capture audio stream and sends it to subscribers.
OPUS headers captured successfully, it's always 841 byte long.
Here is a procedure :
Code: [Select]
procedure TRoomFrame.BuildVoiceHeader(const Data: pointer; const Size: integer);
var
  buf : pointer;
  ogg : PAnsiChar;
  i : integer;
  granule : Int64;
begin
  GetMem(buf, fVoiceHeaderSize + Size);
  if Assigned(fVoiceHeader) then
    System.Move(fVoiceHeader^, buf^, fVoiceHeaderSize);
  System.Move(Data^, pointer(NativeUInt(buf) + fVoiceHeaderSize)^, Size);
  ogg := PAnsiChar(buf);
  for I := 0 to Size + fVoiceHeaderSize - 15 do
    if (ogg[I] = 'O') and (ogg[I+1] = 'g') and (ogg[I+2] = 'g') and (ogg[I+3] = 'S') then
    begin
      System.Move(pointer(NativeUInt(buf) + 6 + I)^, granule, 8);
      if (granule <> 0) and (granule <> -1) then
      begin
        fVoiceHeaderSize := I;
        FreeMem(fVoiceHeader);
        GetMem(fVoiceHeader, fVoiceHeaderSize);
        System.Move(buf^, fVoiceHeader^, fVoiceHeaderSize);
        fVoiceHeaderReady := true;
      end;
    end;
  if not fVoiceHeaderReady then
  begin
    FreeMem(fVoiceHeader);
    fVoiceHeaderSize := fVoiceHeaderSize + Size;
    GetMem(fVoiceHeader, fVoiceHeaderSize);
    System.Move(buf^, fVoiceHeader^, fVoiceHeaderSize);
  end;
  FreeMem(buf);
end;

Encoder initialized with default settings
Code: [Select]
    fEncoder := BASS_Encode_OPUS_Start(fRecordingHandle, nil,
                             BASS_ENCODE_AUTOFREE {$IFDEF UNICODE} or BASS_UNICODE {$ENDIF},
                             PushEncodedData, Self);

procedure PushEncodedData(handle : HENCODE; channel : DWORD; buffer : Pointer; length : DWORD; user : Pointer); stdcall;
var
  frame : TRoomFrame;
begin
  frame := TRoomFrame(user);
  if not frame.fVoiceHeaderReady then
    frame.BuildVoiceHeader(buffer, length)
  else
    if Assigned(frame.fVoiceSender) then
      frame.fVoiceSender.PushData(Buffer, length);
end;

Decoder initialized in this way :
Code: [Select]
  BASS_RecordInit(-1);
  BASS_Init(-1, 44100, 0, AppHandle, nil);
  BASS_PluginLoad('bassopus.dll', BASS_UNICODE);
 ..........
  chan := BASS_OPUS_StreamCreateFileUser(STREAMFILE_BUFFERPUSH, BASS_STREAM_AUTOFREE, VoiceProcs, Self);

VoiceProcs is :
Code: [Select]
function VoiceStreamLen(user: Pointer): QWORD; {$IFDEF MSWINDOWS}stdcall{$ELSE}cdecl{$ENDIF};
begin
  Result := 0;
end;

function VoiceStreamRead(buffer: Pointer; length: DWORD; user: Pointer): DWORD; {$IFDEF MSWINDOWS}stdcall{$ELSE}cdecl{$ENDIF};
var
  player : TVoicePlayer;
begin
  //!!!!!!!!!!!!!! JUST NEED TO FILL buffer WITH HEADER DATA
  player := TVoicePlayer(user);
  Result := Min(length, player.OPUSHeaderSize);
  Move(player.OPUSHeader^, buffer^, Result);
end;

procedure VoiceStreamClose(user: Pointer); {$IFDEF MSWINDOWS}stdcall{$ELSE}cdecl{$ENDIF};
begin
end;

BASS_OPUS_StreamCreateFileUser function returns 0, error is BASS_ERROR_FILEFORM
When i put breakpoint into VoiceStreamRead I see it is called few times with different LENGTH parameter (512, 3584, 2743, 1902, 1061, 220)

What i do wrong?

Ian @ un4seen

  • Administrator
  • Posts: 21539
Your VoiceStreamRead function is currently returning data from the start of the buffered headers in every call. It needs to keep track of the current file position (ie. advance that by the returned amount each time), so that it doesn't return the same data every time. Let me know if it still isn't working after adding that.

IVGSoft

  • Posts: 4
Thank you, Ian.
I thought this function needed only to retrieve opus headers...
How many data it need to start "push" stream?
If I understood correctly, I need to buffer some data to feed an opus stream and then just use bass_streamputdata?
But how I will know when I must switch?

IVGSoft

  • Posts: 4
Finally initialization of OPUS stream done! :)
But got another problem.
After init i feed that stream in this way :
Code: [Select]
            e := BASS_StreamPutData(fChannels[UID], fQueue[0].Data, fQueue[0].Size);
            if e <= 0 then
              e := BASS_ErrorGetCode;

But got an error. :( BASS_ERROR_NOTAVAIL
What could be wrong now?

IVGSoft

  • Posts: 4
Re: Bass encode opus encapsulation for websocket broadcast
« Reply #10 on: 19 Dec '18 - 13:59 »
Sorry, that was my fault. I should use BASS_StreamPutFileData instead.
Thank you for your attention, Ian!