Vernacular SMS and MVASA tool for mass communication in near futureExploring Opportunities and Challenges in India

Mobile: Socio-economic impact in IndiaImpact of Increase in Mobile Penetration on Growth in GD1st order of impact of increase in mobile penetration on growth in GDP (impact of voice primarily by allowing people to connect / talk)VAS and off-deck usage will bring in 2nd order of impact on GDP growth

VAS: Taking delivery thru mobile further Universal Reach / Inclusive GrowthEnterprise / SME / Rural / Poor urban areasLeapfrogging the PCLow cost access medium for entertainment, informationUtility of Content and ServicesCommerce, Rural applications, Local ContentDriving InnovationLocalization, Voice based VAS

The future of MVASA INR 121,000 crore industry over the next 5 yearsVAS as % of total telecom revenue in India remains lower as compared to other markets today; However it is expected to contribute INR 35,400 crores in 2013

Off deck mobile VASKey Growth drivers for Off deck Mobile VASEvolving ecosystem, continued push by D2C players and emergence of alternate billing channels will be the key growth drivers for off-deck mobile VAS in India

The importance of SMSA2P SMS remains the most effective and timely way to reach out to people (for business, social organizations & govt.) & is therefore critical from a social and economic perspective

SMS: Evolution of an effective mass communication mediaOften overlooked, this A2P SMS is a very powerful mass market medium. With 400 mn expected subs and 5 A2P SMS/day this corresponds to 2 Billion pieces of information delivered per day

Reach of Indian Languages10 Million Indians study Hindi every year. Similar numbers for others.Top 10 Indian newspapers are regionalTop TV channels are vernacular. Star TV converted 100% to HindiIn urban India, only 37% of the literate population knows English; the number drops to 17% in rural areas10 out of top 30 most-widely used languages worldwide are Indian 8 out of top 50 best-selling newspapers in the world are published in an Indian languageCompared to only 3 English language papers from India in the list Despite these impressive statistics, not a single Indian language makes to the list to most-widely used languages in the mobile/Internet world

SMS Usage in IndiaSMS continues to be most popular non-voice service but usage levels are still low compared to other countries

Explaining the GapOne of the contributing factors towards high SMS usage in Philippines and China is pricing In Philippines, SMS started as a free service! Voice calls are cheaper in India but SMS pricing levels are relatively higher Another important factor is the use of standards-based solutions for local language SMSIn Philippines, local languages use Roman script, so 7-bit default GSM alphabet can be used The inherent efficiency of Chinese language mitigates the inefficiency of 2- byte UCS-2 encoding Both countries have lower English-literacy levels compared to IndiaLocal language SMS is used heavily

The ChallengesEstablish a standard in the eco-systemTelecom OperatorsDevice ManufacturersApplication Providers

The ChallengesTo evolve a core system of Indic MVAS/SMS to be implemented uniformly across vendors and service providers Font rendering to make the fonts standard across all handsets Key pad design challenge How do legacy handsets which donot have this Indic feature be included in them Regulatory Aspects

The ChallengesTo evolve a core system of Indic MVAS/SMS to be implemented uniformly across vendors and service providers Font rendering to make the fonts standard across all handsets Key pad design challenge How do legacy handsets which donot have this Indic feature be included in them Regulatory Aspects

Existing Coding schemes for messagingGSM Alphabet – Supports languages that use Latin character set UCS-2 – Supports all major languages of the world User-defined – Any arbitrary character set can be defined

Encoding Indian Languages in UCS-2Users who want to send an SMS in an Indian language are faced with two disadvantages from the encoding point of viewComplexity of Indic scripts results in relatively high number of characters per word on the averageWith UCS-2 encoding, only 70 characters are allowed per messageThe net result is that only a few words can go in a single SMS

Current ScenarioSMS in several Indic languages is currently supported by operatorsBased on proprietary picture-messaging based solutions since UCS-2 is inefficient and/or not supportedPicture-enabled handsets needed to display SMS

Indic SMS ChallengesIndic SMS challengesStandards-based solution at the same cost as English SMSInter-operability across operators and devices

RequirementsBasic requirementStandards-based solution with the same level of encoding efficiency as the 7-bit GSM alphabet Additional requirementsAll 22 official languages must be supportedSupport for bi-lingual messaging (Indic + English) Optional requirementEnable easy transliteration between Indian languages

Support for National LanguagesTwo methods have been specified for including National Language characters in a short messageSingle shift mechanism‘Escape’ character is used to signal that the next character is encoded using a National Language Shift TableLocking shift mechanismAll the characters are encoded using a National Language Locking Shift Table In both cases, the language being used is indicated by an identifier added in the message payload

Work till dateIndian language tables developedBased on the 3GPP templates for 7-bit locking and single shift tablesUsing the character sets for Indic languages defined by Unicode ConsortiumTables for 10 languages/scripts have been definedBengali, Gujarati, Hindi, Kannada, Malayalam, Oriya, Punjabi, Tamil, Telugu, and UrduAs the same script is sometimes used by more than one language, the 10 tables can support all the 22 official languages of India

Indic Locking Shift TablesLocking Shift Table of an Indic language consists of Control characters pre-defined in the template (4) The lower case English alphabet (26) The numbers 0-9 (10) A small subset of special characters commonly used in SMS (8) The commonly used characters of that language (80, at most) Non-Indic characters appear in the same positions as in the GSM tableThis ensures that even if a handset does not have the locking table, at least the English part of a bi-lingual SMS can be decoded Similar sounding characters of different languages are put in the same positions (not always possible because of differences in character sets)Enables easy transliteration

Indic Single Shift TablesSingle Shift Table of an Indic language consists of Control characters pre-defined in the template (4)The upper case English alphabet (26)Remaining special characters from default GSM alphabet (31)Characters of the Indic language which do not appear in its locking shift table (67, at most)

Key Benefits in New Encoding SchemeAn SMS written in any of the 22 official Indian languages can have up to 154 charactersNearly as many as in an English SMS!English words can be included in the message without extra overheadTransliteration from one language to another is very simpleCompatibility with Unicode character sets for Indian languagesPossibility to augment the character sets using free spaces available in the single shift tables

The ChallengesTo evolve a core system of Indic MVAS/SMS to be implemented uniformly across vendors and service providers Font rendering to make the fonts standard across all handsets Key pad design challenge How do legacy handsets which donot have this Indic feature be included in them Regulatory Aspects

English is “NOT” an Indic Script

Understanding Unicode for “Indian languages”

Language over script

Language over scriptUnicode script code pages are almost exhaustive for a “script”Support for a language is significantly simpler3GPP Indic Language Tables also adopt the simplicity“LANGUAGE” is different from a “SCRIPT”

The ChallengesTo evolve a core system of Indic MVAS/SMS to be implemented uniformly across vendors and service providers Font rendering to make the fonts standard across all handsets Key pad design challenge How do legacy handsets which donot have this Indic feature be included in them Regulatory Aspects

The text entry Challenge: The English Bias

Indianization of the key pad

Incomplete solutions!

Closing the LoopIntroducingStatistical Techniques to Language SMSFaster than ENGLISH keypadBest predictive efficiencyLeast effort in key movementsLeast number of key taps ( > 3 taps per character is unacceptable)Most IntuitiveWins 97% of the times

The ChallengesTo evolve a core system of Indic MVAS/SMS to be implemented uniformly across vendors and service providers Making SMS services cost effective. Presently India is high on the SMS versus Call costings. Penetration will be a factor of pricing. Font rendering to make the fonts standard across all handsets Key pad design challenge How do legacy handsets which donot have this Indic feature be included in them Regulatory Aspects

Issue : Backward Compatibility with Legacy DevicesA handset that supports the new 7bit Indic encoding sends an Indic language SMS to a legacy device

Solution CConversion from 7bit Indic to UCS2 and visa versaA Silent Conversion application is requiredA Mechanism by which received messages are converted to UCS2Multiple concatenated messaged can be handled.

Solution P: Conversion before deliveryRecipient phone does not support UCS2Received Message can be converted to picture messageSolution1: Before delivering to the RecipientSolution2: SIM can send it for Conversion

Solution P: Conversion after delivery

The ChallengesTo evolve a core system of Indic MVAS/SMS to be implemented uniformly across vendors and service providers Making SMS services cost effective. Presently India is high on the SMS versus Call costings. Penetration will be a factor of pricing. Font rendering to make the fonts standard across all handsets Key pad design challenge How do legacy handsets which donot have this Indic feature be included in them Regulatory Aspects

Regulatory AspectsThe standard only defines a method for representing Indian language characters in short messages using 7-bit codes Handset vendors may or may not choose to implement it A vendor may decide to support only a subset of the Indian languages for which 7-bit tables have been defined Furthermore, the 3GPP specification (TS 23.038) states that: Encoding of a message using the national locking shift mechanism is not intended to be implemented until a formal request is issued by the relevant national regulatory body Regulatory intervention is required to mandate the support for 7-bit Encoding and other necessary features required for Indic SMS

ThanksCreditsDr. NadeemAkhtar (CeWIT)Biju R Balagopal (ComViva Technologies)Sharad Sharma (NASSCOM)SwaranLata (TDIL)VikasPhogat (SagemOrga)VivekananadaPani (Reverie Technologies)