Hello everyone!
I’m building a video-call app in react native with react-native-webrtc package and I’m facing some problems when I run the app. Here is the situation (all the code used is below). To test my code, I run it on two devices (one android smartphone and one android tablet). Let’s name those “Client 1” and “Client 2”, Client 1 been the one who join a room and wait for Client 2. When Client 2 joins the room, he starts sending ice-candidate and offer to Client 1 who responds with answer. After that they share their stream and see each other’s camera. For connecting the clients, I use a NodeJS server deployed on Heroku on which clients connect through socket io. There are no problems with the connection to the server.
That’s what I want to do. However, it doesn’t succeed all the times. When the two devices are connected to the same wifi, each device can play the role of Client 1 and Client 2. But problems come when they are not on the same wifi or if one device is connected through mobile network. Here are the situation. I tried to be as exhaustive as possible to make you see where the problems come.
When I write “It works”, it means that both clients are connected through WebRTC and can see their local stream and remote stream. When I write “it fails”, the clients are not connected and can not see the remote stream.
-
If tablet (on wifi) = Client 1 and smartphone (on mobile network 4G) = Client 2 ==> it works
-
If smartphone (on mobile network 4G) = Client 1 and tablet (on wifi) = Client 2 ==> if fails
-
If tablet (on wifi 1) = Client 1 (or Client 2) and smartphone (on wifi 1) = Client 2 (or Client 1) ==> it works
-
If tablet (on wifi 1) = Client 1 and smartphone (on wifi 2) = Client 2 ==> it works
-
If tablet (on wifi 1) = Client 2 and smartphone (on wifi 2) = Client 1 ==> it fails
I already searched on Google and on this forum. I see that some people had the same problem and resolved it by adding a TURN server but I did it (after signing up on numb) and it’s not working for me:
const peer = new RTCPeerConnection({
iceServers: [
{urls: 'stun:stun.l.google.com:19302'},
{
url: 'turn:numb.viagenie.ca',
credential: '************',
username: '***********@***',
},
],
});
This is really weird because even if it fails, I manage to receive the remoteStream and compute the remoteStream.toURL() so I don’t understand why I can’t display the stream.
Here is the react native code:
import React, {useRef, useState, useEffect} from 'react';
import {Text, View, SafeAreaView, Button, StyleSheet, ScrollView} from 'react-native';
import FormButton from '../components/FormButton';
import {RTCPeerConnection, RTCView, mediaDevices, RTCIceCandidate, RTCSessionDescription} from 'react-native-webrtc';
import io from "socket.io-client";
export default function VideoCallScreen({navigation}) {
const [localStreamCached, setLocalStreamCached] = React.useState();
const [remoteStream, setRemoteStream] = React.useState();
const roomID = "*****************";
let localStream;
const peerRef = useRef();
const socketRef = useRef();
// TMP const pour les afficher dans le DOM et avoir plus facile à comprendre comment tous les messages sont transmis
const [copySocket, setCopySocket] = useState("");
const [whichClient, setWhichClient] = useState("");
const [hasAnyoneInRoomForClient1, setHasAnyoneInRoomForClient1] = useState(false);
const [hasAnyoneInRoomForClient2, setHasAnyoneInRoomForClient2] = useState(false);
const [otherUser, setOtherUser] = useState("");
const [userJoined, setUserJoined] = useState("");
// Peer connection
const [receivedOfferFrom, setReceivedOfferFrom] = useState("");
const [receivedAnswerFrom, setReceivedAnswerFrom] = useState("");
const [receivedIceCandidate, setReceivedIceCandidate] = useState("");
useEffect(() => {
// ---------------------
// Starting local stream
// ---------------------
startLocalStream();
// -----------------
// SOCKET CONNECTION
// -----------------
socketRef.current = io("https://*********.herokuapp.com", {transports: ['websocket']});
socketRef.current.on("connect", () => {
setCopySocket(socketRef.current.id);
console.log("CONNECTION : ", socketRef.current.id);
console.log(socketRef.current.id, " emit join room to server");
socketRef.current.emit("join room", roomID);
socketRef.current.on("other user", userID => {
setWhichClient("Client 2");
console.log(" Client 2: (", socketRef.current.id, ") : other user in the room = ", userID);
setHasAnyoneInRoomForClient2(true);
setOtherUser(userID);
// -------------------------------------------------
// Peer configuration: ice candidate & offer sending
// -------------------------------------------------
peerRef.current = peerConfiguration(userID, "Client 2");
})
socketRef.current.on("user joined", userID => {
setWhichClient("Client 1");
console.log("Client 1 : (", socketRef.current.id, ") : user joined the room = ", userID);
setHasAnyoneInRoomForClient1(true);
setUserJoined(userID);
})
// Réception de l'ice candidate du socket avec lequel on veut se connecter
socketRef.current.on("ice-candidate", incoming => {
handleReceivedIceCandidate(incoming, "Client 1");
});
// Réception de l'offre envoyée par l'autre pair
socketRef.current.on("offer", incoming => {
console.log("Client 1 : received offer from ", incoming.caller);
handleReceiveOffer(incoming, "Client 1");
});
// Réception de la réponse que l'autre pair nous a envoyée après avoir reçu l'offre envoyée par le socket
socketRef.current.on("answer", incoming => {
console.log("Client 2 : received answer from ", incoming.caller);
handleAnswer(incoming, "Client 2");
});
});
socketRef.current.on('connect_error', (err) => {
console.log("ERROR connection: ", err);
})
socketRef.current.on("disconnect", () => {
console.log("Disconnected Socket");
})
}, []);
const handleReceivedIceCandidate = (incomingIceCandidate, clientNb) => {
console.log(clientNb, " : received ice-candidate");
setReceivedIceCandidate(incomingIceCandidate.candidate);
const candidate = new RTCIceCandidate(incomingIceCandidate);
try {
peerRef.current.addIceCandidate(candidate);
console.log(clientNb, " : add ice candidate");
} catch(e) {
console.error(e);
}
}
const handleReceiveOffer = (incoming, clientNb) => {
setReceivedOfferFrom(incoming.caller);
peerRef.current = peerConfiguration(null, clientNb);
const desc = new RTCSessionDescription(incoming.sdp);
peerRef.current.setRemoteDescription(desc)
.then(() => {
return peerRef.current.createAnswer();
})
.catch(err => console.error("Erreur 1: ", err))
.then(answer => {
return peerRef.current.setLocalDescription(answer);
})
.catch(err => console.error("Erreur 2: ", err))
.then(() => {
const payload = {
target: incoming.caller,
caller: socketRef.current.id,
sdp: peerRef.current.localDescription
}
socketRef.current.emit("answer", payload);
console.log(clientNb, " : emit answer to ", incoming.caller);
})
.catch(err => console.error("Erreur 3: ", err))
}
const handleAnswer = (incomingAnswer, clientNb) => {
setReceivedAnswerFrom(incomingAnswer.caller);
const desc = new RTCSessionDescription(incomingAnswer.sdp);
peerRef.current.setRemoteDescription(desc)
.catch(e => console.error("Erreur handle answer : ", e));
console.log(clientNb, " : set remote description");
}
const startLocalStream = async () => {
// isFront will determine if the initial camera should face user or environment
const isFront = true;
const devices = await mediaDevices.enumerateDevices();
const facing = isFront ? 'front' : 'environment';
const videoSourceId = devices.find(device => device.kind === 'videoinput' && device.facing === facing);
const facingMode = isFront ? 'user' : 'environment';
const constraints = {
audio: true,
video: {
mandatory: {
minWidth: 500, // Provide your own width, height and frame rate here
minHeight: 300,
minFrameRate: 30,
},
facingMode,
optional: videoSourceId ? [{sourceId: videoSourceId}] : [],
},
};
const newStream = await mediaDevices.getUserMedia(constraints);
localStream = newStream;
setLocalStreamCached(newStream);
};
const peerConfiguration = (userID, clientNb) => {
console.log(clientNb, " creates its RTCPeerConnection");
const peer = new RTCPeerConnection({
iceServers: [
{urls: 'stun:stun.l.google.com:19302'},
{
url: 'turn:numb.viagenie.ca',
credential: '************',
username: '***********@***',
},
],
});
// Ajout du stream local dans le RTCPeerConnection
peer.addStream(localStream);
console.log(clientNb, " : add local stream");
// Ajout du stream remote
peer.onaddstream = function (e){
if (e.stream && remoteStream !== e.stream){
console.log(clientNb, " : add remote stream whose url = ", e.stream.toURL());
setRemoteStream(e.stream);
}
};
peer.onnegotiationneeded = () => {
// Création et envoi de l'offre
console.log("current peerRef = ", peerRef.current);
peer.createOffer()
.then(offer => {
return peer.setLocalDescription(offer)
.then(() => {
const payload = {
target: userID,
caller: socketRef.current.id,
sdp: peerRef.current.localDescription
};
console.log(clientNb, " : emit offer to ", userID);
socketRef.current.emit("offer", payload);
})
.catch(error => console.error("ERROR set local description", error));
})
.catch(error => console.error("ERROR create offer: ", error));
};
// Envoi de l'ice candidate
peer.onicecandidate = function (e) {
if (userID !== null){
if (e.candidate) {
const payload = {
target: userID,
candidate: e.candidate,
}
console.log(clientNb, " : emit ice-candidate to ", userID);
socketRef.current.emit("ice-candidate", payload);
} else {
console.log(clientNb, " : all ice candidates sent");
}
}
}
peer.onicecandidateerror = error => {
console.error("ERROR ice candidate: ", error);
}
peer.oniceconnectionstatechange = (val) => {
console.log("oniceconnectionstatechange : ", val);
}
peer.onconnectionstatechange = () => {
console.log(peer.connectionState);
if(peer.connectionState === "connected") {
console.log("PEERS CONNECTED");
}
}
return peer;
}
const disconnectSocket = () => {
socketRef.current.disconnect();
navigation.navigate("Contact");
}
return(
<>
<Text>Test socket</Text>
<FormButton buttonTitle="Disconnect" onPress={() => disconnectSocket()}/>
{copySocket !== "" && (
<>
<Text>ID socket : {copySocket}</Text>
<Text>{whichClient}</Text>
</>
)}
{/*Client 1*/}
{whichClient === "Client 1" && (
<>
{hasAnyoneInRoomForClient1 == false && (
<Text>No one is in the room for now, Client 1.</Text>
)}
{hasAnyoneInRoomForClient1 == true && (
<Text>User joined: {userJoined}</Text>
)}
</>
)}
{/*Client 2*/}
{whichClient === "Client 2" && (
<>
{hasAnyoneInRoomForClient2 == false && (
<Text>No one is in the room for now, Client 2.</Text>
)}
{hasAnyoneInRoomForClient2 == true && (
<Text>Other user : {otherUser}</Text>
)}
</>
)}
{receivedOfferFrom !== "" && (
<Text>Offer received from : {receivedOfferFrom}</Text>
)}
{receivedIceCandidate !== "" && (
<Text>Received candidate : {receivedIceCandidate}</Text>
)}
<View style={styles.rtcview}>
<RTCView style={styles.rtc} streamURL={localStreamCached && localStreamCached.toURL()}/>
{remoteStream && (
<>
<Text>{remoteStream.toURL()}</Text>
<RTCView style={styles.rtc} streamURL = {remoteStream.toURL()}/>
</>
)}
</View>
</>
)
}
Here are the logs for the point 4 above.
Client 1 (tablet on wifi 1) logs:
Client 2 (smartphone on wifi 2) logs:
And here are the logs for the point 5 above.
Client 1 (smartphone on wifi 2) logs:
Client 2 (tablet on wifi 1) logs:
The error Failed to set local offer sdp: Called in wrong state: kHaveRemoteOffer comes everytime I run the app, whether it works or not. It comes from the onnegotiationneeded event but I don’t know why. I guess it’s not the problem I’m looking but if you know how to resolve it, go ahead!
Thansk for you help!