React Native Webrtc Video Calling

General040642add70aa006ef56b5af5b5acae3
1

Hi there,
I am new to react native and trying to create an application with react native webrtc and socket io with node js.
I have the server code ready which accepts the user and keeps the record of their socket and one user who creates the offer emits to the other and the answer is created and this works fine but i am not getting the video of the other user.
These are the logs of my code please check and help.

rn-webrtc:pc:DEBUG 0 ctor +0ms
LOG rn-webrtc:pc:DEBUG 0 addTrack +58ms
LOG rn-webrtc:pc:DEBUG 0 addTrack +173ms
LOG rn-webrtc:pc:DEBUG 0 createOffer +160ms
LOG rn-webrtc:pc:DEBUG 0 createOffer OK +378ms
LOG rn-webrtc:pc:DEBUG 0 setLocalDescription +38ms
LOG rn-webrtc:pc:DEBUG 0 setLocalDescription OK +391ms
LOG [object Object]offer
LOG rn-webrtc:pc:DEBUG 0 ctor +0ms
LOG rn-webrtc:pc:DEBUG 0 setRemoteDescription +37ms
LOG [object Object]offer
LOG rn-webrtc:pc:DEBUG 1 ctor +11ms
LOG rn-webrtc:pc:DEBUG 1 addTrack +25ms
LOG rn-webrtc:pc:DEBUG 1 addTrack +86ms
LOG rn-webrtc:pc:DEBUG 1 setRemoteDescription +15ms
LOG rn-webrtc:pc:DEBUG 0 ontrack +15ms
LOG rn-webrtc:pc:DEBUG 0 ontrack +5ms
LOG rn-webrtc:pc:DEBUG 0 setRemoteDescription OK +32ms
LOG rn-webrtc:pc:DEBUG 0 createAnswer +3ms
LOG rn-webrtc:pc:DEBUG 1 ontrack +8ms
LOG rn-webrtc:pc:DEBUG 1 ontrack +3ms
LOG rn-webrtc:pc:DEBUG 1 setRemoteDescription OK +13ms
LOG rn-webrtc:pc:DEBUG 1 createAnswer +1ms
LOG rn-webrtc:pc:DEBUG 0 setLocalDescription +21ms
LOG rn-webrtc:pc:DEBUG 1 setLocalDescription +15ms
LOG rn-webrtc:pc:DEBUG 0 setLocalDescription OK +223ms
LOG rn-webrtc:pc:DEBUG 1 setLocalDescription OK +56ms
LOG iceCandidate Event[object Object]
LOG iceCandidate Event[object Object]
LOG iceCandidate Event[object Object]
LOG iceCandidate Event[object Object]
LOG iceCandidate Eventnull
LOG iceCandidate Event[object Object]
LOG iceCandidate Eventnull

another important point is i am just triggering the event once but it is happening many times.

this is my
App.js

/* eslint-disable prettier/prettier */
/* eslint-disable react-hooks/exhaustive-deps */
// App.js
import React, { useState, useEffect } from 'react';
import { SafeAreaView, StyleSheet, View, Button, Alert } from 'react-native';
import io from 'socket.io-client';
import { RTCView, mediaDevices, RTCIceCandidate, RTCPeerConnection, RTCSessionDescription } from 'react-native-webrtc'; // Import RTCIceCandidate and RTCPeerConnection
const App = () => {
  const [localStream, setLocalStream] = useState(null);
  const [peerStream, setPeerStream] = useState(null);
  const [socket, setSocket] = useState(null);
  const [peerConnection, setPeerConnection] = useState(null);
  const [isCalling, setIsCalling] = useState(false);

  useEffect(() => {
    const socket = io('http://192.168.100.137:3000');
    setSocket(socket);

    // Cleanup when the component unmounts
    return () => {
      socket.disconnect();
      socket.removeAllListeners();
    };
  }, []);

  const startLocalStream = async () => {
    const isFrontCamera = true;
    const devices = await mediaDevices.enumerateDevices();

    const facing = isFrontCamera ? 'front' : 'environment';
    const videoSourceId = devices.find(
      (device) => device.kind === 'videoinput' && device.facing === facing
    );
    const facingMode = isFrontCamera ? 'user' : 'environment';
    const constraints = {
      audio: true,
      video: {
        mandatory: {
          minWidth: 500,
          minHeight: 300,
          minFrameRate: 30,
        },
        facingMode,
        optional: videoSourceId ? [{ sourceId: videoSourceId }] : [],
      },
    };
    const newStream = await mediaDevices.getUserMedia(constraints);
    var sid = Math.floor(Math.random() * 100);
    // Once you have the local stream, initiate the connection to the socket server
    socket.emit('readyToStream', { streamId: sid });

    // Set the local stream in the state
    setLocalStream(newStream);
  };

  useEffect(() => {
    // Listen for 'offer' events from the socket server
    if (socket) {
      socket.on('offer', async (offer) => {
        if (offer && offer.sdp) {
          console.log(offer + 'offer');
          // Create a new peer connection
          const pc = new RTCPeerConnection();
          setPeerConnection(pc);

          // Add the local stream to the peer connection
          if (localStream) {
            localStream.getTracks().forEach((track) => pc.addTrack(track, localStream));
          }

          // Set the remote description received from the server
          const remoteDesc = new RTCSessionDescription(offer); // Convert the offer to RTCSessionDescription
          await pc.setRemoteDescription(remoteDesc);

          // Create an answer to the offer
          const answer = await pc.createAnswer();
          await pc.setLocalDescription(answer);

          // Send the answer back to the server
          socket.emit('answer', answer);
          
          pc.onicecandidate = (event) => {
            console.log('iceCandidate Event'+ event.candidate);
            if (event.candidate) {
              // Send the ICE candidate to the server
              socket.emit('ice-candidate', {  candidate: event.candidate });
            }
          };
        } else {
          console.error('Invalid offer received:', offer);
        }
      });

      socket.on('answer', async (answer) => {
        Alert.alert('answer received from other user');
        if (answer && answer.sdp) {
          // Set the remote description received from the server
          if (peerConnection) {
            const remoteDesc = new RTCSessionDescription(answer); // Convert the answer to RTCSessionDescription
            await peerConnection.setRemoteDescription(remoteDesc);
            Alert.alert('answer received from other user');
          }
        } else {
          console.error('Invalid answer received:', answer);
        }
      });


      // Listen for 'ice-candidate' events from the socket server
      socket.on('ice-candidate', (candidate) => {
        // Add the received ICE candidate to the peer connection
        if (peerConnection) {
          peerConnection.addIceCandidate(candidate);
        }
      });
    }
  }, [socket, localStream]);

  // Function to initiate the call
  const initiateCall = async () => {
    if (socket && localStream) {
      setIsCalling(true);

      // Signal the server that the client wants to call
      // socket.emit('callUser');

      try {
        // Create a new peer connection
        const pc = new RTCPeerConnection();
        setPeerConnection(pc);

        // Add the local stream to the peer connection
        if (localStream) {
          localStream.getTracks().forEach((track) => pc.addTrack(track, localStream));
        }

        // Create an offer to initiate the call
        const offer = await pc.createOffer();
        await pc.setLocalDescription(offer);
        // console.log(JSON.stringify(offer) + 'i created this offer');
        // Send the offer to the server
        socket.emit('callUser', offer);
      } catch (error) {
        console.error('Error creating offer:', error);
      }

    };
  }

  return (
    <SafeAreaView style={styles.container}>
      <Button title="Start Local Stream" onPress={startLocalStream} />
      <Button title="Start Call" onPress={initiateCall} disabled={isCalling} />
      <View style={styles.rtcview}>
        {localStream && <RTCView style={styles.rtc} streamURL={localStream.toURL()} />}
        {peerStream && <RTCView style={styles.rtc} streamURL={peerStream.toURL()} />}
      </View>
    </SafeAreaView>
  );
};

const styles = StyleSheet.create({
  container: {
    backgroundColor: '#333',
    justifyContent: 'space-between',
    alignItems: 'center',
    height: '100%',
  },
  rtcview: {
    justifyContent: 'center',
    alignItems: 'center',
    height: '100%',
    width: '100%',
    backgroundColor: 'black',
  },
  rtc: {
    width: '100%',
    height: '50%',
  },
});

export default App;

and this is my server code

// server.js

const http = require('http');
const express = require('express');
const socketIO = require('socket.io');

const app = express();
const server = http.createServer(app);
const io = socketIO(server);

// Keep track of connected users
const users = []

io.on('connection', (socket) => {
  console.log("a client connected " + socket.id)
  socket.on('readyToStream', ({ streamId }) => {
    users.push(socket.id);
    console.log(users)
  });

  socket.on('callUser', (offer) => {
    console.log(offer + 'offer reached on server')
    const calleeId = users.find(value => value != socket.id);
    console.log('calleeId '+calleeId +' callerId '+socket.id);
    if (calleeId) {
      // Get the caller's socket id
      const callerId = socket.id;

      // Send the 'offer' event to the callee
      io.to(calleeId).emit('offer', offer);
    }
    console.log(users)
  });

  socket.on('answer', (answer ) => {
    const callerId = users.find(value => value !== socket.id);
    if (callerId) {
      // Send the 'answer' event to the caller
      io.to(callerId).emit('answer', answer);
    }
  });

  socket.on('disconnect', () => {
    // Remove the user from the list of connected users on disconnect
      users.pop(socket.id);
    
    console.log('a client disconnected ' + socket.id)
  });

  // Handle ICE candidates
  socket.on('ice-candidate', ( candidate ) => {
    console.log('ice candidate '+JSON.stringify(candidate));
    const remoteSocketId = users.find(value => value != socket.id);
    if (remoteSocketId) {
      // Send the ICE candidate to the remote peer
      io.to(remoteSocketId).emit('ice-candidate', candidate);
    }
  });
});

const PORT = 3000;
server.listen(PORT, () => console.log(`Server is running on port ${PORT}`));

right now i am doing it for just two users I will handle the logics later on
waiting for some nice help.

Thank you in Advance.

Regards
MrrJatt

2

Based on the logs you’ve shown the connection sequence doesn’t appear to be as it should be.
Most likely caused by having a lot of the logic handled within a component render function.
When the component decides to re-render it will essentially run all of that code again.

But just for reference the sequance should be in this order.

  • The caller creates an offer and sets it as their local description.
  • The caller sends the offer to the callee and they set their remote description as the offer.
  • The callee creates an answer and sets it as their local description.
  • The callee sends the answer over to the caller and that gets set as their remote description.

You should now have a connection between both clients.

What i didn’t include is that as soon as you start creating an offer you will have ice candidates to send and process on each side as they are delivered.

If you check over here i wrote a guide some while back which explains the processes.
Also have a look at this as it is important.