Reconnect in background, screen off, and play audio

iOS
1

Similar issue to

  • #223
  • #444

I’m building a walkie-talkie style app with a persistent WebRTC connection. By enabling the relevant background modes and playing some empty audio, I can achieve long term stability if the network is undisturbed.
When the network changes it is crucial that the app reconnects timely and resumes playing audio. Think of driving around with this app in your pocket.

I can successfully reconnect in the background, but I cannot start audio. I get a specific error message which gives me ideas about how to resolve this.

[aurioc] AURemoteIO.cpp:1936:Run: AURemoteIO@0x10603d840: IOThread exiting with error 0x10004006
[aurioc] AURemoteIO.cpp:1598:Start: AUIOClient_StartIO failed (561145187)

No failure handlers are received, the connection appears healthy, and audio is not heard. This is logged only in the Xcode console and otherwise the app thinks it connected and started playing OK.

I can’t tell from the code where this happens, but I assume we’re creating a new AudioSession for each connection.

Some options I’m considering, and would help contribute code for:

  • Is it possible to re-use the audio session as a singleton session for all connections? Expose this as an option, so we’re not violating the precise start in background. We can play audio great in the foregound, and later go background. We must be able to resume playing audio.

  • Having the internals of the WebRTC library do audio handling is unfamiliar to me. [1] We’re using SIP.JS which expects implementors to create their own MediaStream with the PeerConnection tracks and render the audio themselves. This gives us extreme control over mixing and adjusting levels. Would it be possible to expose the media streams in that way? How would we render this without an Audio() node available in Hermes?

  • 1 https ://github com/react-native-webrtc/react-native-webrtc/blob/master/Documentation/BasicUsage.md#controlling-remote-audio-tracks

  • Emit the specific AURemoteIO error to the userland code, so we know something didn’t work starting audio.

  • Push Notification. I can’t test if a push notification wakes up the app to count as being in the foreground right now. (Lacking Apple Developer status, coming soon).

  • Anything else? Has this been widely asked for? I can’t find many other instances of this specific error.

2

#1 could be addressed here: sdk/objc/native/src/audio/audio_device_ios.mm - src - Git at Google

By checking audio_is_initialized_

So we don’t re-create a new unit, as the comment in Update suggests:

https://webrtc.googlesource.com/src/+/3d2210876e31d0bb5c7de88b27fd02ceb1f4e03e/sdk/objc/native/src/audio/audio_device_ios.mm#753