More Related Content
Viewers also liked
Recently uploaded
Recently uploaded (20)
Developing appear.in for iOS and Android
- 1. developing appear.in for iOS and Android Svein Willassen, ph.d. VP, head of appear.in, Telenor Digital AS
- 5. iOS WebRTC quirks - initially: did not work! - SDP mangling - dependency management - missing support for Perfect Forward Secrecy
- 7. Android quirks - Feed UI code is only suitable for AppRTC - WebRTC team only tests code on Nexus device? - Crash prone, especially related to deallocation
- 8. Current learnings - Lack of documentation - Do your own QA - Things will break
- 9. Verdict possible, not for the faint-hearted
- 10. developing appear.in for iOS and Android Svein Willassen, ph.d. VP, head of appear.in, Telenor Digital AS
Editor's Notes
- People were being sent links in FB chat, sms, email etc. and reading this on their iOS device WebRTC not supported in native web browser – nor in other browsers. (iOS Chrome, Opera is just a skin on native browser) Need to develop native app to be able to communicate with video on iOS Additionally: wanted to enable people to contact others. iOS native notifications is a powerful way to do so.
- - Started in January 2014 Initial implementation: just took the AppRTC demo compiled for iOS, tore it apart and adapted it to work with the server implementation of appear.in The AppRTC demo application contains Objective-C wrapper classes that wraps the core WebRTC library calls. The easiest way to build a new application is just to start with these wrapper classes. Used an externally provided socket.io library for iOS to realize server communication. Later moved to an CocoaPod for WebRTC provided by pristine.io. This removes the need for constantly recompliling the WebRTC library yourself.
- Initially, we experienced that WebRTC communication from iOS worked in only one out of four cases The AppRTC demo does some SDP mangling, especially with regards to audio codec, preferring ISAC over OPUS. Possible due to resource constrains, but not really well explained anywhere. We have chosen to inherit these. The need to build AppRTC demo to get the Objective-C wrapper classes can result in dependency problems. I.e. AppRTC uses SocketRocket third party library. We also use this library in appear.in, with the result of linking problems due to the same library being included twice. Starting from Firefox 38, that browser will require Perfect Forward Secrecy. This is not supported in the CocoaPod built by pristine.io because it is linked towards NSS and not OpenSSL. This will break our connectivity between iOS and Firefox. To fix it, we have to go back to compiling ourselves. Pristine has said they won’t change the linking – probably because connectivity with browsers is not a priority for them.
- Why? Also did a mobile web version, but wanted access to contact list and send notifications. Also: get an icon on the home screen! Started from AppRTC demo Quicker than iOS, mostly because of experiences with developing iOS app. Uses a JNI interface towards the native WebRTC library. Also contains a Java-side wrapper library that realizes things such as screen rendering of video streams. Currently use an automated build of the WebRTC library offered by the guys at pristine.io
- The feed UI is really only suitable for the AppRTC demo. It does not work well for the case with multiple video feeds for example. Issue, only working in Nexus: Nexus has support for hardware decoding. WebRTC wrote code that would only work on Nexus devices. Working on HTC One, asked to reproduce in AppRTC code - many days of work. We can reproduce it ; they said we cant reproduce it, can you do some more figuring out. Eventually someone else found the issue. Then they asked can you figure out in which commit this started. The other guy said it does work on a Nexus, but not the other device. The issue had been there in a couple of weeks. Did not make it into the pristine.io build. Now, the state of building is much better. Can now include the latest library based on commit. Quicker to spot bugs now. Can now use more recent versions of the library from pristine.io Kept crashing - crashes a lot and it’s hard to know why. It has to do with the way objects are cleaned. RTCPeerConnection.dispose() in Java might segfault, because it might already have been cleaned up in C-land. Solution: not call dispose(), only call close(). We don’t know the right thing to do, because there is little documentation. 9 out of 10 dispose() will crash the app. - don’t do it. Don’t do it, then WebRTC developers marks it as fixed.
- There is current little documentation for the libraries on Android and iOS. You basically have to look into the code and try to understand what it does. Unsure if WebRTC does its own QA for the code it produces. Users must be prepared to do QA, not only for their own code, but also for the WebRTC code. Currently high maintenance: things will break with new versions of the library, new versions of browsers etc.
- - But rapidly improving