CEWiT
INDIA
Indic SMS Support in 3GPP Standard
Dr. Nadeem Akhtar
Centre of Excellence in Wireless Technology
Workshop on Indian Language SMS
September 23, 2009

CEWiT
INDIA
CEWiT – An Introduction
 Government of India (DIT) initiative - started in 2004
 A Public-Private Partnership - Academia/Industry/Government
 Vision:
 Enable India to become a true leader in wireless technology and make it an engine of
its economic growth
 Mission:
 To develop indigenous next generation wireless technologies and products
 Actively participate in global wireless research and standardization
 Strong emphasis on contributing to 4G standards and developing IPR
 The Broadband Wireless Consortium of India (BWCI) was launched in 2007 to
provide a national forum for broadband wireless
 CEWiT is co-ordinating BWCI activities
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
SMS Usage in India
 SMS continues to be most popular non-voice service but usage levels
are still low compared to other countries
SMS Usage (2008)
800
Messages sent per user per month
700
600
500
400
300
200
100
0
India China Philippines
Sources:
1. Philippines & China: Mobile Messaging Futures 2009-2013, Portio Research
2. India: The Indian Telecom Services Performance Indicators October– December 2008, TRAI
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Explaining the Gap
 One of the contributing factors towards high SMS usage in Philippines
and China is pricing
 In Philippines, SMS started as a free service!
 Voice calls are cheaper in India but SMS pricing levels are relatively higher
 Another important factor is the use of standards-based solutions for
local language SMS
 Both countries have lower English-literacy levels compared to India which
means that local language SMS is used heavily
 In Philippines, local languages use Roman script, so 7-bit default GSM
alphabet can be used
 The inherent efficiency of Chinese language mitigates the inefficiency of 2-
byte UCS-2 encoding
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Reach of Indian Languages
 In urban India, only 37% of the literate population knows English; the
number drops to 17% in rural areas
 10 out of top 30 most-widely used languages worldwide are Indian
 8 out of top 50 best-selling newspapers in the world are published in an
Indian language
 Compared to only 3 English language papers from India in the list
 Despite these impressive statistics, not a single Indian language makes
to the list to most-widely used languages in the mobile/Internet world
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Indic SMS: Key Elements
Text Entry Encoding Decoding Display
Transmission
Sender Receiver
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
SMS in Indian Languages
 Current Scenario
 SMS in several Indic languages is currently supported by operators
 Based on proprietary picture-messaging based solutions since UCS-2 is inefficient
and/or not supported
 Picture-enabled handsets needed to display SMS
 Handsets with Indian language keypads are available
 Variation in keypad layouts across vendors/devices
 Lack of memory-efficient display solutions
 CEWiT has worked extensively on the encoding issue over the last 18
months
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Encoding Schemes
 GSM Alphabet – Supports languages that use Latin character set
 UCS-2 – Supports all major languages of the world
 User-defined – Any arbitrary character set can be defined
GSM Alphabet (7 bits) 160 characters
User-defined (8 bits) 140 characters
UCS-2 (16 bits) 70 characters
Message Size
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Encoding Indian Languages in UCS-2
 Users who want to send an SMS in an Indian language are faced with
two disadvantages from the encoding point of view
 Complexity of Indic scripts results in relatively high number of characters
per word on the average
 With UCS-2 encoding, only 70 characters are allowed per message
 The net result is that only a a few words can go in a single SMS
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Encoding Requirements
 Basic requirement
 Standards-based solution with the same level of encoding efficiency
as the 7-bit GSM alphabet
 Additional requirements
 All 22 official languages must be supported
 Support for bi-lingual messaging (Indic + English)
 Optional requirement
 Enable easy transliteration between Indian languages
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
3GPP Standards
 3GPP (3rd Generation Partnership Project) is the premier body when it
comes to global mobile telephony standards
 Maintains the GSM standards
 Defined the 3G-WCDMA standard (now evolved to HSPA)
 Currently finalising the specification of Long Term Evolution (LTE) standard;
work has already started on LTE-Advanced
 Technical specification of various services such as SMS, CBS, USSD etc is
also done by 3GPP
 3GPP amended the SMS standards in 2008 to accommodate a request
from Turkey to support the full Turkish alphabet
 Introduced 7-bit ‘National Language’ tables
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Support for National Languages
 Two methods have been specified for including National Language
characters in a short message
 Single shift mechanism
 ‘Escape’ character is used to signal that the next character is encoded using a
National Language Shift Table
 Locking shift mechanism
 All the characters are encoded using a National Language Locking Shift Table
 In both cases, the language being used is indicated by an identifier added in
the message payload
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
CEWiT Proposal
 Indian language tables developed
 Based on the 3GPP templates for 7-bit locking and single shift tables
 Using the character sets for Indic languages defined by Unicode Consortium
 Tables for 10 languages/scripts have been defined
 Bengali, Gujarati, Hindi, Kannada, Malayalam, Oriya, Punjabi, Tamil, Telugu,
and Urdu
 As the same script is sometimes used by more than one language, the 10
tables can support all the 22 official languages of India
 In addition to Indic characters, each table also includes the English alphabet
and the numbers 0-9 plus special characters (punctuation marks etc)
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Key Benefits of New Encoding Scheme
 An SMS written in any of the 22 official Indian languages can have up
to 154 characters
 Nearly as many as in an English SMS!
 English words can be included in the message without extra overhead
 Transliteration from one language to another is very simple
 Compatibility with Unicode character sets for Indian languages
 Possibility to augment the character sets using free spaces available in
the single shift tables
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Standardisation Process
 CEWiT submitted a Change Request (CR) to 3GPP Core Networks and
Terminals Working Group 1 (CT1) in Feb 2009
 Due to concerns regarding impact on legacy devices, no decision was taken
to allow further discussion
 The matter was referred to the Systems and Architecture Working
Group 1 (SA1) and was discussed during its meeting in May 2009
 SA1 concurred with our view that the proposal does not violate guidelines
regarding National Language tables
 The CR was discussed again at a CT1 meeting in June 09 and the
proposal was accepted
 The final approval was given by the CT plenary last week
 Indian language tables will be included in the next revision of TS 23.038
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Example
 Scenario: A handset that supports the new encoding scheme sends an
Indic language SMS to a legacy device
Handset that supports
new encoding
 Outcome: The output on legacy handset is mostly gibberish except for
English words in the original message
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Support for Legacy Devices
 Existing handsets will not be able to decode messages encoded using
the new 7-bit tables
 Solution(s) for ensuring backward compatibility are needed
 Message encoded in 7-bit at the sending side can either be converted to
UCS-2 encoding or transformed into a picture
 Terminal-based solution
 User application
 SIM application
 Network-based solution
 Conversion in network based either on prior knowledge of terminal
capabilities or in response to a trigger from the receiving device/user
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Keypad Layout
 Having an efficient encoding scheme for is not enough
 The user interface is also of critical importance
 Unlike the English keypad which has been standardised by ITU/ETSI, no similar
effort has been done for Indian keypad layouts
 Each handset vendor has designed its own layouts
 Current Indian language keypads show variation in
 Number of characters on each key
 Choice of characters placed on different keys
 Position of characters on a specific key
 Height and width of characters displayed on the keypad
 Some form of harmonisation of keypad layouts is needed
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Regulatory Aspects
 The standard only defines a method for representing Indian language
characters in short messages using 7-bit codes
 Handset vendors may or may not choose to implement it
 A vendor may decide to support only a subset of the Indian languages for
which 7-bit tables have been defined
 Regulatory intervention is required to mandate the support for 7-bit
Encoding and other necessary features required for Indic SMS
24 September, 2009 Mobile VAS Summit 2009

CEWiT
INDIA
Next Steps
 Build a national consensus on how to leverage this opportunity and
maximise its impact for the benefit of all stakeholders
 Mechanisms to handle legacy devices
 Harmonisation of keypad layouts and other user interface features
 Case for regulatory intervention
 Roadmap for implementation and roll-out
 Special Interest Group is being formed to address these issues
 Active participation of all the stakeholders is crucial
 If you are interested in this SIG, drop me a line at nadeem@cewit.org.in
24 September, 2009 Mobile VAS Summit 2009