Non-Obvious Challenges and Opportunities for VoIP and Video on the iPhone Jan Linden, Global IP Solutions

Jan Linden, Global IP Solutions

Introduction Global IP Solutions (GIPS) Founded 1999 Recognized leader in world class voice and video processing technology for IP networks GIPS s/w is deployed in over 800 million end-points Enables developers to offer the highest quality regardless of network conditions and device Jan Linden Vice President Engineering – GIPS R&D in speech and video Processing for more than 15 years

Global IP Solutions (GIPS)

Founded 1999

Recognized leader in world class voice and video processing technology for IP networks

GIPS s/w is deployed in over 800 million end-points

Enables developers to offer the highest quality regardless of network conditions and device

Jan Linden

Vice President Engineering – GIPS

R&D in speech and video Processing for more than 15 years

Mobile Market Trends Fixed Mobile Convergence Unified communications Migration from legacy cellular to next-gen networks Allows for faster data connection Growth of Smartphone adoption iPhone big part of growth, especially in consumer space Increased accessibility and affordability Attractive data pricing models Handset prices declining No ideal mobile VoIP platform exists yet Development of 3 rd Party Applications VoIP over mobile in high demand

Fixed Mobile Convergence

Unified communications

Migration from legacy cellular to next-gen networks

Allows for faster data connection

Growth of Smartphone adoption

iPhone big part of growth, especially in consumer space

Increased accessibility and affordability

Attractive data pricing models

Handset prices declining

No ideal mobile VoIP platform exists yet

Development of 3 rd Party Applications

VoIP over mobile in high demand

Mobile VoIP Design Considerations Coping with network degradation Power Consumption – Battery life Hardware issues (Processor, OS, acoustics, etc.) Echo cancellation Additional voice processing Environment – Background Noise, speaker phone mode Speech codec VoIP Design Challenges Network Codec Hardware Echo Power Voice Environment

The iPhone as a VoIP Platform iPhone one of the most VoIP friendly phone on market… Open APIs for easy development Sufficient CPU resources Support of HD Voice (headset only) Intuitive UI makes for simple navigation between applications Demand for 3 rd party apps is higher than other phones … but strengths present unique challenges Running too many applications can drain resources Usage scenarios can negatively impact voice quality

iPhone one of the most VoIP friendly phone on market…

Open APIs for easy development

Sufficient CPU resources

Support of HD Voice (headset only)

Intuitive UI makes for simple navigation between applications

Demand for 3 rd party apps is higher than other phones

… but strengths present unique challenges

Running too many applications can drain resources

Usage scenarios can negatively impact voice quality

iPhone Specific Challenges Only VoIP over WiFi currently possible 3G network not available for Apple compliance Applications cannot run in the background Push notifications announced but not delivered Rumors about background application support in next release Regular Mac audio APIs not available on iPhone Very iPhone specific solution had to be designed Application must monitor change of audio route (e.g. headset in/out) and handle this Wideband audio not available via built-in microphone

Only VoIP over WiFi currently possible

3G network not available for Apple compliance

Applications cannot run in the background

Push notifications announced but not delivered

Rumors about background application support in next release

Regular Mac audio APIs not available on iPhone

Very iPhone specific solution had to be designed

Application must monitor change of audio route (e.g. headset in/out) and handle this

Wideband audio not available via built-in microphone

What’s Needed? Robustness Robust speech codec and smooth packet loss concealment Adapt to network busts and excessive mobile jitter Efficiency Optimized for ARM processors on iPhones prolongs battery life Audio quality Codecs designed for IP networks HD voice Aggressive Voice Quality Enhancement Detect and cancel background noise Suppress acoustic echo

Robustness

Robust speech codec and smooth packet loss concealment

Adapt to network busts and excessive mobile jitter

Efficiency

Optimized for ARM processors on iPhones prolongs battery life

Audio quality

Codecs designed for IP networks

HD voice

Aggressive Voice Quality Enhancement

Detect and cancel background noise

Suppress acoustic echo

Summary High quality VoIP on iPhone and iPod touch available today Nimbuzz, truphone,… Remaining challenges Applications cannot run in the background VoIP over 3G not allowed Two-way video not yet possible No application access to camera No front-facing camera And no, Apple doesn’t have a patent on video conferencing for touch screen phones

High quality VoIP on iPhone and iPod touch available today

Nimbuzz, truphone,…

Remaining challenges

Applications cannot run in the background

VoIP over 3G not allowed

Two-way video not yet possible

No application access to camera

No front-facing camera

And no, Apple doesn’t have a patent on video conferencing for touch screen phones