How to implement simulcast?

Help
1

i need to implement simulcast on react native webrtc, and set two size of video, i don’t found info for implement it

2

I’m not entirely sure if we’ve actually implemented that feature yet.
We’re open to contributions though.

3

Can you open to contributions?

4

I can see you’ve opened an issue on GitHub :+1:
For future reference we’re open to contributions over here.

5

@8BallBomBom But i don’t receive info hahaha, what should we do?

6

I’m not entirely sure of all the specifics but theres an example here.

You also need to use this pre-release version of the RN WebRTC module as the new track and transceivers handling is required.

7

Hi @8BallBomBom

The related change you suggested does not update SDP, not sure but I think sendEncodings part is not implemented in the module for sending the simulcast.

I get something like:
a=fmtp:122 apt=123
a=rtpmap:125 ulpfec/90000
a=ssrc-group:FID 2891755832 1645302191
a=ssrc:2891755832 cname:SahDPJVJbchTmi20
a=ssrc:2891755832 msid:a3b8d1d7-7ca0-4b46-9513-2c8199b9e020 87d2a85b-903b-4ab5-a660-dc7d8cd70341
a=ssrc:1645302191 cname:SahDPJVJbchTmi20
a=ssrc:1645302191 msid:a3b8d1d7-7ca0-4b46-9513-2c8199b9e020 87d2a85b-903b-4ab5-a660-dc7d8cd70341

But when you give the transreceiver options like that in chrome you see:

a=rtcp-fb:41 goog-remb
a=rtcp-fb:41 transport-cc
a=rtcp-fb:41 ccm fir
a=rtcp-fb:41 nack
a=rtcp-fb:41 nack pli
a=rtpmap:42 rtx/90000
a=fmtp:42 apt=41
a=rtpmap:114 red/90000
a=rtpmap:115 rtx/90000
a=fmtp:115 apt=114
a=rtpmap:116 ulpfec/90000
a=rid:h send
a=rid:m send
a=rid:l send
a=simulcast:send h;m;l

8

Thanks for the heads up, will look into it a bit more.

Edit:
You are correct, sendEncodings doesn’t seem to be implemented as a transceiver init parameter.
So as it stands support for Simulcast may not be fully there yet.
But i haven’t tested to confirm.

That being said, there is an implementation of calling setParameters on the RTPSender.
@golgetahir Can you test to confirm if that works?

10

@8BallBomBom Hi again,

Normally on web, that parameter is read-only and when you set the encodings with setParameters Uncaught (in promise) DOMException: Read-only field modified in setParameters().

In the react-native-webrtc, it actually set the parameter. However, it has no effect on SDP neither have more than one remote outbound video stream.

However, as an answer, manually changing the SDP seems to be working as I see more than one outbound RTP video streams on stats like the logs:

LOG {“active”: true, “bytesSent”: 1088396, “codecId”: “RTCCodec_0_Outbound_96”, “encoderImplementation”: “OMX.Exynos.VP8.Encoder”, “firCount”: 0, “frameHeight”: 720, “frameWidth”: 1280, “framesEncoded”: 218, “framesPerSecond”: 24, “framesSent”: 218, “headerBytesSent”: 22676, “hugeFramesSent”: 211, “id”: “RTCOutboundRTPVideoStream_136728387”, “keyFramesEncoded”: 2, “kind”: “video”, “mediaSourceId”: “RTCVideoSource_1”, “mediaType”: “video”, “mid”: “0”, “nackCount”: 0, “packetsSent”: 1039, “pliCount”: 14, “qpSum”: 4019, “qualityLimitationDurations”: {“bandwidth”: 9.288, “cpu”: 0, “none”: 0.21, “other”: 0}, “qualityLimitationReason”: “bandwidth”, “qualityLimitationResolutionChanges”: 1, “remoteId”: “RTCRemoteInboundRtpVideoStream_136728387”, “retransmittedBytesSent”: 0, “retransmittedPacketsSent”: 0, “rid”: “h”, “ssrc”: 136728387, “timestamp”: 1664809667089.581, “totalEncodeTime”: 5.756, “totalEncodedBytesTarget”: 0, “totalPacketSendDelay”: 358.002, “trackId”: “DEPRECATED_RTCMediaStreamTrack_sender_1”, “transportId”: “RTCTransport_0_1”, “type”: “outbound-rtp”}
LOG {“active”: true, “bytesSent”: 0, “codecId”: “RTCCodec_0_Outbound_96”, “encoderImplementation”: “OMX.Exynos.VP8.Encoder”, “firCount”: 0, “framesEncoded”: 0, “framesSent”: 0, “headerBytesSent”: 0, “hugeFramesSent”: 0, “id”: “RTCOutboundRTPVideoStream_1651138696”, “keyFramesEncoded”: 0, “kind”: “video”, “mediaSourceId”: “RTCVideoSource_1”, “mediaType”: “video”, “mid”: “0”, “nackCount”: 0, “packetsSent”: 0, “pliCount”: 0, “qualityLimitationDurations”: {“bandwidth”: 9.288, “cpu”: 0, “none”: 0.21, “other”: 0}, “qualityLimitationReason”: “bandwidth”, “qualityLimitationResolutionChanges”: 1, “retransmittedBytesSent”: 0, “retransmittedPacketsSent”: 0, “rid”: “m”, “ssrc”: 1651138696, “timestamp”: 1664809667089.581, “totalEncodeTime”: 0, “totalEncodedBytesTarget”: 0, “totalPacketSendDelay”: 0, “trackId”: “DEPRECATED_RTCMediaStreamTrack_sender_1”, “transportId”: “RTCTransport_0_1”, “type”: “outbound-rtp”}
LOG {“active”: true, “bytesSent”: 0, “codecId”: “RTCCodec_0_Outbound_96”, “encoderImplementation”: “OMX.Exynos.VP8.Encoder”, “firCount”: 0, “framesEncoded”: 0, “framesSent”: 0, “headerBytesSent”: 244, “hugeFramesSent”: 0, “id”: “RTCOutboundRTPVideoStream_808685676”, “keyFramesEncoded”: 0, “kind”: “video”, “mediaSourceId”: “RTCVideoSource_1”, “mediaType”: “video”, “mid”: “0”, “nackCount”: 0, “packetsSent”: 1, “pliCount”: 0, “qualityLimitationDurations”: {“bandwidth”: 9.288, “cpu”: 0, “none”: 0.21, “other”: 0}, “qualityLimitationReason”: “bandwidth”, “qualityLimitationResolutionChanges”: 1, “retransmittedBytesSent”: 0, “retransmittedPacketsSent”: 0, “rid”: “l”, “ssrc”: 808685676, “timestamp”: 1664809667089.581, “totalEncodeTime”: 0, “totalEncodedBytesTarget”: 0, “totalPacketSendDelay”: 0, “trackId”: “DEPRECATED_RTCMediaStreamTrack_sender_1”, “transportId”: “RTCTransport_0_1”, “type”: “outbound-rtp”}

Can you confirm manually munging SDP to what I shared in my first comment is working ?

  let newSDP = ""
  let lines = offer.sdp.split('\n')
  for(let i = 0; i < lines.length; i++){
    if(lines[i].includes('a=ssrc-group:FID')){
      break;
    }
    newSDP += lines[i] + "\n";
  }
  newSDP += "a=rid:h send \n"+
            "a=rid:m send \n"+
            "a=rid:l send \n"+
            "a=simulcast:send h;m;l \n";
  offer.sdp = newSDP;
  console.log(offer)
1 Like
11

No worries about the mobile development.
Bit unusual, wasn’t aware that setParameters had any read only parameters.
I’ll have a lookie, see if i can get things working and get back to you.

1 Like
12

I can confirm the SDP Munging method does work but isn’t ideal.
Currently going through testing an implementation of sendEncodings.
Will report back soon.

1 Like
13

If you need anything please feel free to ping me @8BallBomBom I can even help on development if I can but I’m not really experienced on mobile. :laughing:

14

Contributions are always appreciated :slight_smile:

I’ve managed to get sendEncodings working on Android so far.
Will be testing and looking to also get iOS nudged out :+1:
Once things have been pushed i’ll link the pull request and then hopefully it will be in the next release.

Edit:
Tracked over here currently.

15

Thanks to Saghul for testing and fixing things up, should be included with the next release soon :+1:

2 Likes
16

Can you share example code for public simulcast? @8BallBomBom

17

I’ll be looking to provide example projects soon but i can confirm that the example posted here will output the correct SDP once a new module release has gone out.

18

Thanks a lot @8BallBomBom I’ve seen the commit is merged and I’ve tried on Android, seems like it is working. The logs I gathered here:

LOG {“active”: true, “bytesSent”: 0, “codecId”: “RTCCodec_0_Outbound_96”, “encoderImplementation”: “OMX.Exynos.VP8.Encoder”, “firCount”: 0, “framesEncoded”: 0, “framesSent”: 0, “headerBytesSent”: 0, “hugeFramesSent”: 0, “id”: “RTCOutboundRTPVideoStream_4131707387”, “keyFramesEncoded”: 0, “kind”: “video”, “mediaSourceId”: “RTCVideoSource_1”, “mediaType”: “video”, “mid”: “0”, “nackCount”: 0, “packetsSent”: 0, “pliCount”: 0, “qualityLimitationDurations”: {“bandwidth”: 0.512, “cpu”: 0, “none”: 9.1, “other”: 0}, “qualityLimitationReason”: “none”, “qualityLimitationResolutionChanges”: 1, “retransmittedBytesSent”: 0, “retransmittedPacketsSent”: 0, “rid”: “m”, “ssrc”: 4131707387, “timestamp”: 1666255083683.586, “totalEncodeTime”: 0, “totalEncodedBytesTarget”: 0, “totalPacketSendDelay”: 0, “trackId”: “DEPRECATED_RTCMediaStreamTrack_sender_1”, “transportId”: “RTCTransport_0_1”, “type”: “outbound-rtp”}
LOG {“active”: true, “bytesSent”: 0, “codecId”: “RTCCodec_0_Outbound_96”, “encoderImplementation”: “OMX.Exynos.VP8.Encoder”, “firCount”: 0, “framesEncoded”: 0, “framesSent”: 0, “headerBytesSent”: 244, “hugeFramesSent”: 0, “id”: “RTCOutboundRTPVideoStream_468313530”, “keyFramesEncoded”: 0, “kind”: “video”, “mediaSourceId”: “RTCVideoSource_1”, “mediaType”: “video”, “mid”: “0”, “nackCount”: 0, “packetsSent”: 1, “pliCount”: 0, “qualityLimitationDurations”: {“bandwidth”: 0.512, “cpu”: 0, “none”: 9.1, “other”: 0}, “qualityLimitationReason”: “none”, “qualityLimitationResolutionChanges”: 1, “retransmittedBytesSent”: 0, “retransmittedPacketsSent”: 0, “rid”: “l”, “ssrc”: 468313530, “timestamp”: 1666255083683.586, “totalEncodeTime”: 0, “totalEncodedBytesTarget”: 0, “totalPacketSendDelay”: 0, “trackId”: “DEPRECATED_RTCMediaStreamTrack_sender_1”, “transportId”: “RTCTransport_0_1”, “type”: “outbound-rtp”}
LOG {“active”: true, “bytesSent”: 1438526, “codecId”: “RTCCodec_0_Outbound_96”, “encoderImplementation”: “OMX.Exynos.VP8.Encoder”, “firCount”: 0, “frameHeight”: 720, “frameWidth”: 1280, “framesEncoded”: 209, “framesPerSecond”: 24, “framesSent”: 209, “headerBytesSent”: 45972, “hugeFramesSent”: 12, “id”: “RTCOutboundRTPVideoStream_47212017”, “keyFramesEncoded”: 1, “kind”: “video”, “mediaSourceId”: “RTCVideoSource_1”, “mediaType”: “video”, “mid”: “0”, “nackCount”: 0, “packetsSent”: 1554, “pliCount”: 0, “qpSum”: 7209, “qualityLimitationDurations”: {“bandwidth”: 0.512, “cpu”: 0, “none”: 9.1, “other”: 0}, “qualityLimitationReason”: “none”, “qualityLimitationResolutionChanges”: 1, “remoteId”: “RTCRemoteInboundRtpVideoStream_47212017”, “retransmittedBytesSent”: 0, “retransmittedPacketsSent”: 0, “rid”: “h”, “ssrc”: 47212017, “timestamp”: 1666255083683.586, “totalEncodeTime”: 2.749, “totalEncodedBytesTarget”: 0, “totalPacketSendDelay”: 25.346, “trackId”: “DEPRECATED_RTCMediaStreamTrack_sender_1”, “transportId”: “RTCTransport_0_1”, “type”: “outbound-rtp”}

2 Likes
19

Thanks for testing, fantastic news :smiley:

1 Like
21

how are you about example project?

22

Sadly nothing yet.
But there are still plans to provide a few example projects along with a signalling backend server.
Best bet is to keep an eye over here.

1 Like