Your SlideShare is downloading. ×
Android Programming                         Lecture 13                       Vladimir Kulyukin                Department o...
Outline   ●       AIDL and Bound Services   ●       AIDL Complex Types   ●       Speech Synthesis   ●       Overcoming TTS...
AIDL and Bound Serviceswww.youtube.com/vkedco            www.vkedco.blogspot.com
Steps of Developing a Bound Service   ●     Define your interface to clients in an AIDL file   ●     An AIDL file has .aid...
Steps of Developing a Bound Service   ●       Android Eclipse plug-in calls the AIDL compiler to       generate a Java int...
AIDLService Projectwww.youtube.com/vkedco                www.vkedco.blogspot.com
The Problem      Write a bound (remote) service that other      applications can call to compute the sums of the      firs...
AIDLService Project                              Demowww.youtube.com/vkedco                www.vkedco.blogspot.com
Defining Remote Interface in .aidl File   // save this file, ISumService.aidl, in /src directory   package org.vkedco.andr...
Extending ISumService.Stub   public class AIDLSumService extends Service {       public class SumServiceImpl extends ISumS...
AIDLSumService.onBind()   public class AIDLSumService extends Service {           public class SumServiceImpl extends ISum...
Declaring Service in AndroidManifest.xml   <application      android:icon="@drawable/ic_launcher"      android:label="@str...
AIDLService Project                            Source Codewww.youtube.com/vkedco                www.vkedco.blogspot.com
Consuming a Bound Service   ●     Create a consumer application project whose     package is different from the package of...
AIDLServiceConsumer Project                       Source Codewww.youtube.com/vkedco          www.vkedco.blogspot.com
Passing Complex Types to Remote Serviceswww.youtube.com/vkedco       www.vkedco.blogspot.com
AIDL Type Support   ●       AIDL supports String and CharSequence   ●       AIDL supports other AIDL interfaces (they must...
What is android.os.Parcelable?   ●       Parcelable specifies how Android should serialize and       deserialize objects d...
The Problem      Write a bound (remote) service that other      applications can call to compute the range sums      of th...
Implementing SumRange   import android.os.Parcel;   import android.os.Parcelable;   public class SumRange implements Parce...
Implementing SumRange   public class SumRange implements Parcelable {       public SumRange() { mFrom = 0; mUpto = 0; }   ...
Declaring SumRange in SumRange.aidl   package org.vkedco.android.aidlrangesumservice;   parcelable SumRange;www.youtube.co...
Defining ISumRangeService in ISumRangeService.aidl                package org.vkedco.android.aidlrangesumservice;         ...
Implementing AIDLSumRangeService   public class AIDLSumRangeRervice extends Service {       public class SumRangeServiceIm...
Speech Synthesiswww.youtube.com/vkedco              www.vkedco.blogspot.com
TTS: Text To Speech   ●       The Problem: Take a sequence of characters and       generate a waveform   ●       Words are...
TTS Engine Anatomy   ●       Text Analysis – parse text (after transliterating it if       necessary) and identify words a...
TTS Approaches   ●       Full Automation – machine does everything   ●       Mixed Initiative – human records a set of kno...
PICO   ●     Pico is an Android multi-lingual speech     synthesis language   ●     Pico can be used as a black box: text ...
TTS Sharing   ●     TTS engine is shared among all activities on     the device   ●     It is never a certainty that the t...
Starting TTS   ●     It is best practice to check if TTS is available     on the device   ●     This is done via Intent to...
Checking TTS Availability   ●       In Activity.onCreate():       Intent ttsi = new Intent();       ttsi.setAction(TextToS...
Creating TTS Instance   ●       In Activity.onActivityResult():       switch ( result_code ) {        case TextToSpeech.En...
Handling Missing TTS Data ●     In Activity.onActivityResult():     switch ( result_code ) {      case TextToSpeech.Engine...
Handling TTS Unavailability ●     In Activity.onActivityResult():     switch ( result_code ) {      case TextToSpeech.Engi...
What To Do When TTS Is Ready ●     Override Activity.onInit():     @Override     public void onInit(int status) {         ...
Overriding onPause() and onDestroy() ●   When your Activity is paused (e.g., it loses focus),   have TTS stop synthesizing...
Overriding onPause() and onDestroy()   public void onPause() {         super.onPause();       if ( mTTS != null ) mTTS.sto...
Overriding onPause() and onDestroy()   public void onDestroy() {         super.onPause();       if ( mTTS != null ) mTTS.s...
SimpleTTSDemo Project                            Demowww.youtube.com/vkedco            www.vkedco.blogspot.com
SimpleTTSDemo Project                            Sourcewww.youtube.com/vkedco            www.vkedco.blogspot.com
Overcoming TTS Limitations ●     Every TTS engine mispronounces some words ●     There are two ways of overcoming this lim...
Overcoming TTS Limitations Through Phonetic                    Spelling          Teaching Android to Pronounce “Don Quixot...
TTSDonQuixote Project                         Demowww.youtube.com/vkedco          www.vkedco.blogspot.com
Teaching Android to Speak Sanskrit and Russian     Overcoming TTS Limitations through Human Recordingwww.youtube.com/vkedc...
What Is This?www.youtube.com/vkedco   www.vkedco.blogspot.com
Bhagavatgita: Verse 1www.youtube.com/vkedco     www.vkedco.blogspot.com
What Is This?                Что свершили, - скажи Санджая, -                сыновья мои и Пандавы,                ради би...
Bhagavatgita, V. 1 (Russian Translation)                Что свершили, - скажи Санджая, -                сыновья мои и Панд...
Bhagavatgita, V. 01 (Latin Transliteration)                Chto svershili, - skazhi Sandzhaya, -                synovya mo...
Bhagavatgita: Verse 1 (English Translation)           Oh, Sanjaya, tell me what happened at           Kurukshetra, the fie...
BhagavatGita Projectwww.youtube.com/vkedco            www.vkedco.blogspot.com
The Problem   Have Android read the first verse of Bhagavatgita in   Sanskrit and Russian.www.youtube.com/vkedco          ...
Logical Steps of One Solution  ●      Write a Devanagari transliterator that takes Sanskrit      texts and produces their ...
Real Steps  ●    Skip transliterator implementation  ●    Record .wav files  ●    Associate .wav files with specific strin...
BhagavatGitaTTS Project                          Demowww.youtube.com/vkedco          www.vkedco.blogspot.com
BhagavatGitaTTS Project                      Source Codewww.youtube.com/vkedco          www.vkedco.blogspot.com
Picking up Where You Stopped ●     TTS has a useful callback     OnUtteranceCompletedListener.onUtteranceCompleted() ●   T...
References & Reading Suggestions   ●       http://developer.android.com/reference/android/speech/tts/TextToSpeech.html   ●...
Feedback      Bugs, errors to vladimir dot kulyukin at gmail      dot comwww.youtube.com/vkedco             www.vkedco.blo...
Android Programming: Lecture 13
Android Programming: Lecture 13
Upcoming SlideShare
Loading in...5
×

Android Programming: Lecture 13

2,660

Published on

1) AIDL and Bound Services
2) AIDL Primitive and Complex Types
3) Speech Synthesis
4) Overcoming TTS Limitations through Phonetic Spelling and Human Recoding
5) Bhagavatgita on Android in Sanskrit and Russian
6) Class home page is at http://vkedco.blogspot.com/2011/08/android-programming-home.html

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,660
On Slideshare
0
From Embeds
0
Number of Embeds
41
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Android Programming: Lecture 13"

  1. 1. Android Programming Lecture 13 Vladimir Kulyukin Department of Computer Science Utah State Universitywww.youtube.com/vkedco www.vkedco.blogspot.com
  2. 2. Outline ● AIDL and Bound Services ● AIDL Complex Types ● Speech Synthesis ● Overcoming TTS Limitations Through Phonetic Spelling ● Overcoming TTS Limitations Through Human Recordingwww.youtube.com/vkedco www.vkedco.blogspot.com
  3. 3. AIDL and Bound Serviceswww.youtube.com/vkedco www.vkedco.blogspot.com
  4. 4. Steps of Developing a Bound Service ● Define your interface to clients in an AIDL file ● An AIDL file has .aidl extension that uses Java syntax ● The package should be the same as the projects package ● The .aidl file should be saved in the /src directory of your Eclipse projectwww.youtube.com/vkedco www.vkedco.blogspot.com
  5. 5. Steps of Developing a Bound Service ● Android Eclipse plug-in calls the AIDL compiler to generate a Java interface file from the .aidl file ● Implement the onBind() method to return the interface object generated by the compiler ● Implement the method(s) specified in the .aidl file ● Declare the service in AndroidManifest.xmlwww.youtube.com/vkedco www.vkedco.blogspot.com
  6. 6. AIDLService Projectwww.youtube.com/vkedco www.vkedco.blogspot.com
  7. 7. The Problem Write a bound (remote) service that other applications can call to compute the sums of the first n Catalan or Fibonacci numberswww.youtube.com/vkedco www.vkedco.blogspot.com
  8. 8. AIDLService Project Demowww.youtube.com/vkedco www.vkedco.blogspot.com
  9. 9. Defining Remote Interface in .aidl File // save this file, ISumService.aidl, in /src directory package org.vkedco.android.aidlsumservice; interface ISumService { long catalanSum(long n); long fibonacciSum(long n); }www.youtube.com/vkedco www.vkedco.blogspot.com
  10. 10. Compiling AIDL File ● When you build a project, the AIDL compiler will generate ISumService.java and place it into /gen directory where R.java is placed ● ISumService.java will have the class ISumService.Stub that needs to be extend to implement the methods of the ISumService interfacewww.youtube.com/vkedco www.vkedco.blogspot.com
  11. 11. Extending ISumService.Stub public class AIDLSumService extends Service { public class SumServiceImpl extends ISumService.Stub { public long catalanSum(long n) throws RemoteException { // implementation } public long fibonacciSum(long n) throws RemoteException { // implementation } } }www.youtube.com/vkedco www.vkedco.blogspot.com
  12. 12. AIDLSumService.onBind() public class AIDLSumService extends Service { public class SumServiceImpl extends ISumService.Stub { // } @Override public IBinder onBind(Intent arg0) { return new SumServiceImpl(); } }www.youtube.com/vkedco www.vkedco.blogspot.com
  13. 13. Declaring Service in AndroidManifest.xml <application android:icon="@drawable/ic_launcher" android:label="@string/app_name" > <activity ...> </activity> <service android:name="AIDLSumService"> <intent-filter> <action android:name="org.vkedco.android.aidlsumservice.ISumService" /> </intent-filter> </service> </application>www.youtube.com/vkedco www.vkedco.blogspot.com
  14. 14. AIDLService Project Source Codewww.youtube.com/vkedco www.vkedco.blogspot.com
  15. 15. Consuming a Bound Service ● Create a consumer application project whose package is different from the package of the .aidl file ● Copy the .aidl from from the service project into the the new package in the /src directory ● Build the project to generate the Java interface file in the /gen directorywww.youtube.com/vkedco www.vkedco.blogspot.com
  16. 16. AIDLServiceConsumer Project Source Codewww.youtube.com/vkedco www.vkedco.blogspot.com
  17. 17. Passing Complex Types to Remote Serviceswww.youtube.com/vkedco www.vkedco.blogspot.com
  18. 18. AIDL Type Support ● AIDL supports String and CharSequence ● AIDL supports other AIDL interfaces (they must be imported) ● AIDL supports complex types that implement android.os.Parcelable ● AIDL suports java.util.List and java.util.Map so long as these containers contain String, CharSequence or android.os.Parcelable ● Complex types require directional indicator: – in (value set by client) – out (value set by service) – inout (value set by both client and service)www.youtube.com/vkedco www.vkedco.blogspot.com
  19. 19. What is android.os.Parcelable? ● Parcelable specifies how Android should serialize and deserialize objects during marshalling and unmarshalling ● There are four steps of implementing Parcelable interface: – Make your class implement Parcelable – Implement writeToParcel() that writes the current state of the object to a Parcel – Add a static CREATOR field – Create an aidl file that declares the classwww.youtube.com/vkedco www.vkedco.blogspot.com
  20. 20. The Problem Write a bound (remote) service that other applications can call to compute the range sums of the Catalan or Fibonacci sequences. A range sum is the sum of the sequence elements from element start upto element end. Implement a range sum as a complex AIDL type.www.youtube.com/vkedco www.vkedco.blogspot.com
  21. 21. Implementing SumRange import android.os.Parcel; import android.os.Parcelable; public class SumRange implements Parcelable { private long mFrom; private long mUpto; private String mSumType; public static final Parcelable.Creator<SumRange> CREATOR = new Parcelable.Creator<SumRange>() { @Override public SumRange createFromParcel(Parcel source) {return new SumRange(source); } @Override public SumRange[] newArray(int size) { return new SumRange[size]; } }; // other class stuff }www.youtube.com/vkedco www.vkedco.blogspot.com
  22. 22. Implementing SumRange public class SumRange implements Parcelable { public SumRange() { mFrom = 0; mUpto = 0; } private SumRange(Parcel source) { readSumRangeFromSource(source); } @Override public int describeContents() { return 0; } @Override public void writeToParcel(Parcel p, int flags) { p.writeLong(mFrom); p.writeLong(mUpto); p.writeString(mSumType); } public void readSumRangeFromSource(Parcel source) { this.mFrom = source.readLong(); this.mUpto = source.readLong(); this.mSumType = source.readString(); } }www.youtube.com/vkedco www.vkedco.blogspot.com
  23. 23. Declaring SumRange in SumRange.aidl package org.vkedco.android.aidlrangesumservice; parcelable SumRange;www.youtube.com/vkedco www.vkedco.blogspot.com
  24. 24. Defining ISumRangeService in ISumRangeService.aidl package org.vkedco.android.aidlrangesumservice; import org.vkedco.android.aidlrangesumservice.SumRange; interface ISumRangeService { long catalanSum(long n); long fibonacciSum(long n); long catalanRangeSum(in SumRange sr); long fibonacciRangeSum(in SumRange sr); }www.youtube.com/vkedco www.vkedco.blogspot.com
  25. 25. Implementing AIDLSumRangeService public class AIDLSumRangeRervice extends Service { public class SumRangeServiceImpl extends ISumRangeService.Stub { @Override public long catalanRangeSum(SumRange sr) { return AIDLSumRangeRervice.this.catRangeSum(sr); } @Override public long fibonacciRangeSum(SumRange sr) { return AIDLSumRangeRervice.this.fiboRangeSum(sr); } }www.youtube.com/vkedco www.vkedco.blogspot.com
  26. 26. Speech Synthesiswww.youtube.com/vkedco www.vkedco.blogspot.com
  27. 27. TTS: Text To Speech ● The Problem: Take a sequence of characters and generate a waveform ● Words are pronounced as a sequence of individual units called phones ● Phonetic alphabets describe how phones are pronounced ● Phonological rules specify how phones combine into speechwww.youtube.com/vkedco www.vkedco.blogspot.com
  28. 28. TTS Engine Anatomy ● Text Analysis – parse text (after transliterating it if necessary) and identify words and utterances ● Linguistic Analysis – identify phrases and assign prosodies (accents, emphasis, duration, pauses, etc) ● Waveform Generation - generate a waveform from a fully specified linguistic descriptionwww.youtube.com/vkedco www.vkedco.blogspot.com
  29. 29. TTS Approaches ● Full Automation – machine does everything ● Mixed Initiative – human records a set of known texts; machine learning is used to extract the rules ● Human-Based Recording – human records words/sentences/texts; machine plays them as neededwww.youtube.com/vkedco www.vkedco.blogspot.com
  30. 30. PICO ● Pico is an Android multi-lingual speech synthesis language ● Pico can be used as a black box: text in, speech out ● Pico is the only TTS engine on the devicewww.youtube.com/vkedco www.vkedco.blogspot.com
  31. 31. TTS Sharing ● TTS engine is shared among all activities on the device ● It is never a certainty that the text is spoken and, if it is, when it is spoken ● The only way to find out what is going on is to use the TTS callback methodswww.youtube.com/vkedco www.vkedco.blogspot.com
  32. 32. Starting TTS ● It is best practice to check if TTS is available on the device ● This is done via Intent to check TTS data ● If the check is successful, a instance of TTS can be created ● Activity that uses TTS implements OnInitListener interfacewww.youtube.com/vkedco www.vkedco.blogspot.com
  33. 33. Checking TTS Availability ● In Activity.onCreate(): Intent ttsi = new Intent(); ttsi.setAction(TextToSpeech.Engine.ACTION_CHECK_TTS_DATA); startActivityForResult(ttsi, CHECK_TTS_REQ);www.youtube.com/vkedco www.vkedco.blogspot.com
  34. 34. Creating TTS Instance ● In Activity.onActivityResult(): switch ( result_code ) { case TextToSpeech.Engine.CHECK_VOICE_DATA_PASS: mTTS = new TextToSpeech(this, this); break; }www.youtube.com/vkedco www.vkedco.blogspot.com
  35. 35. Handling Missing TTS Data ● In Activity.onActivityResult(): switch ( result_code ) { case TextToSpeech.Engine.CHECK_VOICE_BAD_DATA: case TextToSpeech.Engine.CHECK_VOICE_DATA_MISSING_DATA: case TextToSpeech.Engine.CHECK_VOICE_DATA_MISSING_VOLUME: Intent insti = new Intent(); insti.setAction(TextToSpeech.Engine.ACTION_INSTALL_TTS_DATA); startActivity(insti); }www.youtube.com/vkedco www.vkedco.blogspot.com
  36. 36. Handling TTS Unavailability ● In Activity.onActivityResult(): switch ( result_code ) { case TextToSpeech.Engine.CHECK_VOICE_DATA_FAIL: // Let the user know that TTS is not available – Toast, Log Message, Notification, etc. }www.youtube.com/vkedco www.vkedco.blogspot.com
  37. 37. What To Do When TTS Is Ready ● Override Activity.onInit(): @Override public void onInit(int status) { if ( status == TextToSpeech.SUCCESS ) { // do something when you know that TTS is available } }www.youtube.com/vkedco www.vkedco.blogspot.com
  38. 38. Overriding onPause() and onDestroy() ● When your Activity is paused (e.g., it loses focus), have TTS stop synthesizing ● When your Activity is destroyed, shut TTS down to notify Android that the resources can be released and given to other activities or applicationswww.youtube.com/vkedco www.vkedco.blogspot.com
  39. 39. Overriding onPause() and onDestroy() public void onPause() { super.onPause(); if ( mTTS != null ) mTTS.stop(); }www.youtube.com/vkedco www.vkedco.blogspot.com
  40. 40. Overriding onPause() and onDestroy() public void onDestroy() { super.onPause(); if ( mTTS != null ) mTTS.shutdown(); }www.youtube.com/vkedco www.vkedco.blogspot.com
  41. 41. SimpleTTSDemo Project Demowww.youtube.com/vkedco www.vkedco.blogspot.com
  42. 42. SimpleTTSDemo Project Sourcewww.youtube.com/vkedco www.vkedco.blogspot.com
  43. 43. Overcoming TTS Limitations ● Every TTS engine mispronounces some words ● There are two ways of overcoming this limitation: – Phonetic spelling: spell mispronounced words the way they sound, generate waveforms, and associate words with waveforms – Human recording: have a human record mispronounced words and use the fileswww.youtube.com/vkedco www.vkedco.blogspot.com
  44. 44. Overcoming TTS Limitations Through Phonetic Spelling Teaching Android to Pronounce “Don Quixote”www.youtube.com/vkedco www.vkedco.blogspot.com
  45. 45. TTSDonQuixote Project Demowww.youtube.com/vkedco www.vkedco.blogspot.com
  46. 46. Teaching Android to Speak Sanskrit and Russian Overcoming TTS Limitations through Human Recordingwww.youtube.com/vkedco www.vkedco.blogspot.com
  47. 47. What Is This?www.youtube.com/vkedco www.vkedco.blogspot.com
  48. 48. Bhagavatgita: Verse 1www.youtube.com/vkedco www.vkedco.blogspot.com
  49. 49. Bhagavatgita: Verse 1 (Latin Transliteration) dharmakshetre kurukshetre samaveta yuyutsavah mamakah pandavashcaiva kim akurvata samjayawww.youtube.com/vkedco www.vkedco.blogspot.com
  50. 50. What Is This? Что свершили, - скажи Санджая, - сыновья мои и Пандавы, ради битвы сойдясь на поле Kурукшетры, на поле дхармы? Перевод В.С. Семенцовwww.youtube.com/vkedco www.vkedco.blogspot.com
  51. 51. Bhagavatgita, V. 1 (Russian Translation) Что свершили, - скажи Санджая, - сыновья мои и Пандавы, ради битвы сойдясь на поле Kурукшетры, на поле дхармы? Перевод В.С. Семенцовwww.youtube.com/vkedco www.vkedco.blogspot.com
  52. 52. Bhagavatgita, V. 01 (Latin Transliteration) Chto svershili, - skazhi Sandzhaya, - synovya moi i Pandavy, radi bitvy soydyas na pole Kurukshetry, nа pоlе dharmy? Translated by V.S. Sementsovwww.youtube.com/vkedco www.vkedco.blogspot.com
  53. 53. Bhagavatgita: Verse 1 (English Translation) Oh, Sanjaya, tell me what happened at Kurukshetra, the field of dharma, where my family and the Pandavas gathered to fight? Translated by Eknath Easwaranwww.youtube.com/vkedco www.vkedco.blogspot.com
  54. 54. BhagavatGita Projectwww.youtube.com/vkedco www.vkedco.blogspot.com
  55. 55. The Problem Have Android read the first verse of Bhagavatgita in Sanskrit and Russian.www.youtube.com/vkedco www.vkedco.blogspot.com
  56. 56. Logical Steps of One Solution ● Write a Devanagari transliterator that takes Sanskrit texts and produces their Latin transliterations ● Write a Cyrillic transliterator that takes Russian texts and produces their Latin transliterations ● Have human readers record Sanskrit and Russian words ● Associate strings with specific recordingswww.youtube.com/vkedco www.vkedco.blogspot.com
  57. 57. Real Steps ● Skip transliterator implementation ● Record .wav files ● Associate .wav files with specific strings ● Implement an IntentService that plays those files when strings are sent to itwww.youtube.com/vkedco www.vkedco.blogspot.com
  58. 58. BhagavatGitaTTS Project Demowww.youtube.com/vkedco www.vkedco.blogspot.com
  59. 59. BhagavatGitaTTS Project Source Codewww.youtube.com/vkedco www.vkedco.blogspot.com
  60. 60. Picking up Where You Stopped ● TTS has a useful callback OnUtteranceCompletedListener.onUtteranceCompleted() ● This method is invoked when TTS has completed speaking a piece of text ● This callback can be used to save the last utterance number and pick up where TTS has stopped when Activity is active againwww.youtube.com/vkedco www.vkedco.blogspot.com
  61. 61. References & Reading Suggestions ● http://developer.android.com/reference/android/speech/tts/TextToSpeech.html ● http://developer.android.com/resources/articles/tts.html ● http://en.wikipedia.org/wiki/Catalan_number ● http://developer.android.com/guide/developing/tools/aidl.html ● Hashimi, Komatineni, MacLean, “Pro Android 2”www.youtube.com/vkedco www.vkedco.blogspot.com
  62. 62. Feedback Bugs, errors to vladimir dot kulyukin at gmail dot comwww.youtube.com/vkedco www.vkedco.blogspot.com

×