The document discusses multi-modal search capabilities in on-device portals (ODPs) that are preloaded mobile applications. It describes Medio Systems' universal search ODP that allows searching by text, voice, image, and barcode across different mobile platforms. Challenges in developing ODPs include operator integration, network efficiency, and bugs in the Java ME implementation. A live demonstration of the universal search ODP on a Nokia feature phone is also presented.

1. Just Say “Britney Spears”:
Multi-Modal Search in On Device Portals
Gail Rahn Frederick
Medio Systems, Inc.
January 2009
M3DD 2009

2. Agenda
• Introduction
• On Device Portals (ODPs)
• Multimodal ODP
• Medio Universal Search ODP
• Solving Development Challenges
• Live Demonstration
– Medio Universal Search ODP on J2ME Featurephone
Copyright 2009 Medio Systems, Inc. 2

3. Introduction
Gail Rahn Frederick
gail@medio.com or 206-262-3719
Mobile Software Architect at Medio Systems, Inc.
– Mobile Search, Advertising and Analytics
• Universal Handset Support
– J2ME, BREW, Windows Mobile, Android, iPhone, RIM, etc…
– WAP, Mobile AJAX, HTML4 on Mobile Devices
– 1600+ handset models
– Live on 9 mobile operators in 6 countries
• t-zones J2ME ODP for T-Mobile USA
– 22% Market Share in Mobile Search
• Microsoft has 6% share
– Site Search Provider for Leading Mobile Publishers
Copyright 2009 Medio Systems, Inc.

4. On Device Portals
• Handset-resident mobile application
– Preloaded or free download
• Enhances discovery and consumption
of endorsed mobile content and services
• Search, browse and purchase
• Provides a compelling user experience
– Overcomes / circumvents browser flaws
– Targets mass-market featurephones
• Personalized experience
• Sometimes monetized with advertising
Copyright 2009 Medio Systems, Inc.

5. Multimodal ODP
• Many Ways to Use an ODP
– Search, Browse, Discover, Purchase
• Many Search Modes in ODP
– Text, Voice, Bar Code, Image
• Navigate an ODP using
familiar actions on a mobile device
– saying a phrase
– entering text
– snapping a photo
Copyright 2009 Medio Systems, Inc.

6. Medio Universal Search ODP
• Consistent UI/UX across All Mobile
Platforms
– Native App, WAP and Mobile Web
• Multi-modal, search-driven J2ME ODP
– Drives repeat customer usage
– Increases usage of browsable and consumable
mobile content
– Consistent UI/UX despite J2ME fragmentation
– Search by text, voice, image and barcode
– Also available on other mobile platforms and mobile
web
• Integrated Advertising
– Leaderboards
– Interstitials
– Sponsored Links
Copyright 2009 Medio Systems, Inc.

7. Solving Challenges with Voice Search
• Sound Capture
J2ME Platforms require JSR-135 with sound record.
Requires AMR or other low-bitrate codec.
• Environment
Mobile environments are typically noisy: utterances come from cars,
bars, airports... rarely from quiet home or office environments.
• Complex Grammars
Must be efficient at handling unstructured/natural utterances
– Past: Walled-Garden ODP grammars includes song artists and titles, wallpaper titles,
game titles and studios, etc.
– Present and Future: Web-Enabled ODPs support millions of music tracks, mobile
site names, URLs, free-text queries – requires flexible voice inputs
– Consider “grammar-less” voice to text translation using phonemes.
Copyright 2009 Medio Systems, Inc.

8. Solving ODP Development Challenges
• Operator Integration
– Handoffs to operator services to preview and purchase content
• HTTP 302 redirects, browser launches, J2ME app launches
• Network Efficiency
– Compact binary protocol (See M3DD 2008 Poster by Medio)
– Piggyback status updates with search requests
– Asynchronous network communication
– Dynamic UI updates to refresh links, pages, chrome and
advertisements
• J2ME Implementation Bugs
– Share MIDlet with failed test case with operator and OEM
• Prove KVM bug with simple, commented code
• Three-way communication (ODP vendor, operator, OEM)
Copyright 2009 Medio Systems, Inc.

9. Live Demonstration
Demo Devices Sample queries
• Nokia 6263 • Britney Spears Ringtones
– T-Mobile USA 3G • Barack Obama Inauguration
– Series 40 5th Edition • Nordstrom in Santa Clara,
– MIDP 2.1, CLDC 1.1 California
– JSR 135 for Voice Recording • Facebook
– Released in Dec 2007 • Kate Winslet and Leonardo
– Price: FREE DiCaprio
• Santa Clara Weather
• Blackberry 8120 (Pearl)
– See me for demo
Copyright 2009 Medio Systems, Inc.

10. Just Say “Britney Spears”:
Multi-Modal Search in On Device Portals
Gail Rahn Frederick
Medio Systems, Inc.
January 2009
M3DD 2009