Here is the scene.
Client A make a video call to ClientB. at the start parse, Both ClientA and ClientB render the video data from the font-camera capture. and then just fine.
Then the Client B received the video calling request and join the meeting, then try to build the peerconnection to Janus Server, and also the ClientA do so.
But Client B work fine, and is correctly render the local video data from font-camera, and also the remote video data from ClientA.
But sometimes, Client A render fail while the remote data have received… then I see the log, EglRenderer report that the frame drop like below.
2022-05-24 16:12:07.118 20211-20285/? I/-org.webrtc.Logging-: 0-EglRenderer: videoRenderDuration: 4006 ms. Frames received: 0. Dropped: 0. Rendered: 0. Render fps: .0. Average render time: NA. Average swapBuffer time: NA.
2022-05-24 16:12:07.352 20211-20329/? I/-org.webrtc.Logging-: 0-EglRenderer: videoRenderDuration: 4008 ms. Frames received: 0. Dropped: 0. Rendered: 0. Render fps: .0. Average render time: NA. Average swapBuffer time: NA.
2022-05-24 16:12:07.353 20211-20329/? I/-org.webrtc.Logging-: 0-EglRenderer: videoRenderDuration: 4008 ms. Frames received: 0. Dropped: 0. Rendered: 0. Render fps: .0. Average render time: NA. Average swapBuffer time: NA.
2022-05-24 16:12:09.972 20211-20327/? I/-org.webrtc.Logging-: 0-EglRenderer: videoRenderDuration: 4007 ms. Frames received: 0. Dropped: 0. Rendered: 0. Render fps: .0. Average render time: NA. Average swapBuffer time: NA.
2022-05-24 16:12:10.999 20211-20324/? I/-org.webrtc.Logging-: 0-EglRenderer: videoRenderDuration: 4006 ms. Frames received: 0. Dropped: 0. Rendered: 0. Render fps: .0. Average render time: NA. Average swapBuffer time: NA.
2022-05-24 16:12:11.123 20211-20322/? I/-org.webrtc.Logging-: 0-EglRenderer: videoRenderDuration: 4006 ms. Frames received: 0. Dropped: 0. Rendered: 0. Render fps: .0. Average render time: NA. Average swapBuffer time: NA.
2022-05-24 16:12:11.124 20211-20322/? I/-org.webrtc.Logging-: 0-EglRenderer: videoRenderDuration: 4006 ms. Frames received: 0. Dropped: 0. Rendered: 0. Render fps: .0. Average render time: NA. Average swapBuffer time: NA.
2022-05-24 16:12:11.362 20211-20290/? I/-org.webrtc.Logging-: 0-EglRenderer: videoRenderDuration: 4007 ms. Frames received: 0. Dropped: 0. Rendered: 0. Render fps: .0.
And then i realized that is probably there’s some problems happen in the network data transmission…then i get the RTCStatsType from differnt peerconnection. and i found a weird log in ClientA.
As I known from PeerConnection, its suppose to be theres only one inboundVideoStream and inboundAudioStream, and only one outboundVideoStream and outboundVideoStream…But i found theres two different inboundVideoStream and inboudAudioStream. And why? May this can be the problem to lead to this issue?
-----inbound.
2022-05-24 16:36:42.911 20211-20326/? D/-ConnectionStatusHelper-: 0-
userName:69b8cd9238004cf9bb33d36c4f2f
OriginData：ConnectionStatus(key='RTCInboundRTPAudioStream_3474935701', id='RTCInboundRTPAudioStream_3474935701', type='inbound-rtp', timeStamp=1653381402904495, info=AudioInboundConnectionStatus(packetsReceived=4897, bytesReceived=320554, packetsLost=0, lastPacketReceivedTimestamp=242214.814, jitter=0.002, fractionLost=0.0, packetsReceivedDiff=0, bytesReceivedDiff=0, packetsLostDiff=0), BaseConnectionStatus(ssrc=3474935701, isRemote=false, mediaType='audio', kind='audio', transportId='RTCTransport_audio_1', trackId='', codecId='RTCCodec_audio_Inbound_111', updateTime=1653381402910), isValid=true)
2022-05-24 16:36:42.912 20211-20291/? D/-ConnectionStatusHelper-: 0-
userName:69b8cd9238004cf9bb33d36c4f2f
OriginData：ConnectionStatus(key='RTCInboundRTPAudioStream_4186376904', id='RTCInboundRTPAudioStream_4186376904', type='inbound-rtp', timeStamp=1653381402904495, info=AudioInboundConnectionStatus(packetsReceived=0, bytesReceived=0, packetsLost=0, lastPacketReceivedTimestamp=0.0, jitter=0.0, fractionLost=0.0, packetsReceivedDiff=0, bytesReceivedDiff=0, packetsLostDiff=0), BaseConnectionStatus(ssrc=4186376904, isRemote=false, mediaType='audio', kind='audio', transportId='RTCTransport_audio_1', trackId='RTCMediaStreamTrack_receiver_17', codecId='', updateTime=1653381402910), isValid=true)
2022-05-24 16:36:42.913 20211-20322/? D/-ConnectionStatusHelper-: 0-
userName:69b8cd9238004cf9bb33d36c4f2f
OriginData：ConnectionStatus(key='RTCInboundRTPVideoStream_1827282291', id='RTCInboundRTPVideoStream_1827282291', type='inbound-rtp', timeStamp=1653381402904495, info=VideoInboundConnectionStatus(firCount=0, pliCount=3, nackCount=24, sliCount=0, packetsReceived=19182, bytesReceived=21350593, jitter=0.0, packetsLost=4, fractionLost=0.0, framesDecoded=2763, packetsReceivedDiff=0, bytesReceivedDiff=0, packetsLostDiff=0), BaseConnectionStatus(ssrc=1827282291, isRemote=false, mediaType='video', kind='video', transportId='RTCTransport_audio_1', trackId='', codecId='RTCCodec_video_Inbound_96', updateTime=1653381402910), isValid=true)
2022-05-24 16:36:42.915 20211-20326/? D/-ConnectionStatusHelper-: 0-
userName:69b8cd9238004cf9bb33d36c4f2f
OriginData：ConnectionStatus(key='RTCInboundRTPVideoStream_391733382', id='RTCInboundRTPVideoStream_391733382', type='inbound-rtp', timeStamp=1653381402904495, info=VideoInboundConnectionStatus(firCount=0, pliCount=0, nackCount=0, sliCount=0, packetsReceived=0, bytesReceived=0, jitter=0.0, packetsLost=0, fractionLost=0.0, framesDecoded=0, packetsReceivedDiff=0, bytesReceivedDiff=0, packetsLostDiff=0), BaseConnectionStatus(ssrc=391733382, isRemote=false, mediaType='video', kind='video', transportId='RTCTransport_audio_1', trackId='RTCMediaStreamTrack_receiver_18', codecId='', updateTime=1653381402910), isValid=true)
-----outbound
2022-05-24 16:36:42.915 20211-20283/? D/-ConnectionStatusHelper-: 0-
userName:69b8cd9238004cf9bb33d36c4f2f
OriginData：ConnectionStatus(key='RTCOutboundRTPAudioStream_3823193211', id='RTCOutboundRTPAudioStream_3823193211', type='outbound-rtp', timeStamp=1653381402904495, info=AudioOutboundConnectionStatus(packetsSent=0, retransmittedPacketsSent=0, bytesSent=0, retransmittedBytesSent=0, packetsSentDiff=0, bytesSentDiffer=0), BaseConnectionStatus(ssrc=3823193211, isRemote=false, mediaType='audio', kind='audio', transportId='RTCTransport_audio_1', trackId='RTCMediaStreamTrack_sender_35', codecId='RTCCodec_audio_Outbound_111', updateTime=1653381402910), isValid=true)
2022-05-24 16:36:42.916 20211-20326/? D/-ConnectionStatusHelper-: 0-
userName:69b8cd9238004cf9bb33d36c4f2f
OriginData：ConnectionStatus(key='RTCOutboundRTPVideoStream_584071059', id='RTCOutboundRTPVideoStream_584071059', type='outbound-rtp', timeStamp=1653381402904495, info=VideoOutboundConnectionStatus(firCount=0, pliCount=0, nackCount=0, sliCount=0, qpSum=0, packetsSent=0, retransmittedPacketsSent=0, bytesSent=0, retransmittedBytesSent=0, framesEncoded=0, totalEncodeTime=0.0, packetsSentDiff=0, bytesSentDiffer=0), BaseConnectionStatus(ssrc=584071059, isRemote=false, mediaType='video', kind='video', transportId='RTCTransport_audio_1', trackId='RTCMediaStreamTrack_sender_36', codecId='RTCCodec_video_Outbound_96', updateTime=1653381402910), isValid=true)
And I also capture the ClientB RTP log. then I just see it is normally. One PeerConnection have two inbound(video and audio) and outbound(video and audio)…the log like below
2022-05-24 16:53:03.367 4727-4802/onecloud.cn.xiaohui.rc D/-ConnectionStatusHelper-: 0-
userName:a2c6f25ad1a84
OriginData：ConnectionStatus(key='RTCInboundRTPAudioStream_694635647', id='RTCInboundRTPAudioStream_694635647', type='inbound-rtp', timeStamp=1653382383359261, info=AudioInboundConnectionStatus(packetsReceived=1054, bytesReceived=66294, packetsLost=2, lastPacketReceivedTimestamp=48083.358, jitter=0.002, fractionLost=0.0, packetsReceivedDiff=0, bytesReceivedDiff=0, packetsLostDiff=0), BaseConnectionStatus(ssrc=694635647, isRemote=false, mediaType='audio', kind='audio', transportId='RTCTransport_audio_1', trackId='RTCMediaStreamTrack_receiver_1', codecId='RTCCodec_audio_Inbound_111', updateTime=1653382383366), isValid=true)
2022-05-24 16:53:03.368 4727-4846/onecloud.cn.xiaohui.rc D/-ConnectionStatusHelper-: 0-
userName:a2c6f25ad1a84
OriginData：ConnectionStatus(key='RTCInboundRTPVideoStream_4075081030', id='RTCInboundRTPVideoStream_4075081030', type='inbound-rtp', timeStamp=1653382383359261, info=VideoInboundConnectionStatus(firCount=0, pliCount=2, nackCount=1, sliCount=0, packetsReceived=4027, bytesReceived=4450787, jitter=0.0, packetsLost=0, fractionLost=0.0, framesDecoded=628, packetsReceivedDiff=0, bytesReceivedDiff=0, packetsLostDiff=0), BaseConnectionStatus(ssrc=4075081030, isRemote=false, mediaType='video', kind='video', transportId='RTCTransport_audio_1', trackId='RTCMediaStreamTrack_receiver_2', codecId='RTCCodec_video_Inbound_96', updateTime=1653382383366), isValid=true)
2022-05-24 16:53:03.369 4727-4807/onecloud.cn.xiaohui.rc D/-ConnectionStatusHelper-: 0-
userName:a2c6f25ad1a84
OriginData：ConnectionStatus(key='RTCOutboundRTPAudioStream_3510873794', id='RTCOutboundRTPAudioStream_3510873794', type='outbound-rtp', timeStamp=1653382383359261, info=AudioOutboundConnectionStatus(packetsSent=0, retransmittedPacketsSent=0, bytesSent=0, retransmittedBytesSent=0, packetsSentDiff=0, bytesSentDiffer=0), BaseConnectionStatus(ssrc=3510873794, isRemote=false, mediaType='audio', kind='audio', transportId='RTCTransport_audio_1', trackId='RTCMediaStreamTrack_sender_1', codecId='RTCCodec_audio_Outbound_111', updateTime=1653382383366), isValid=true)
2022-05-24 16:53:03.369 4727-4805/onecloud.cn.xiaohui.rc D/-ConnectionStatusHelper-: 0-
userName:a2c6f25ad1a84
OriginData：ConnectionStatus(key='RTCOutboundRTPVideoStream_2281618726', id='RTCOutboundRTPVideoStream_2281618726', type='outbound-rtp', timeStamp=1653382383359261, info=VideoOutboundConnectionStatus(firCount=0, pliCount=0, nackCount=0, sliCount=0, qpSum=0, packetsSent=0, retransmittedPacketsSent=0, bytesSent=0, retransmittedBytesSent=0, framesEncoded=0, totalEncodeTime=0.0, packetsSentDiff=0, bytesSentDiffer=0), BaseConnectionStatus(ssrc=2281618726, isRemote=false, mediaType='video', kind='video', transportId='RTCTransport_audio_1', trackId='RTCMediaStreamTrack_sender_2', codecId='RTCCodec_video_Outbound_96', updateTime=1653382383366), isValid=true)
And I have capture the correctly connection to ClientB, I found the log just show theres only two inbound(video and audio) and two outbound(video and audio) in ClientB PeerConnection… That’s work fine.
So why the PeerConnection to ClientB have four inbound (two audio and two video…)??