• Like
  • Save
Android Programming: Lecture 13
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Android Programming: Lecture 13

  • 2,523 views
Published

1) AIDL and Bound Services …

1) AIDL and Bound Services
2) AIDL Primitive and Complex Types
3) Speech Synthesis
4) Overcoming TTS Limitations through Phonetic Spelling and Human Recoding
5) Bhagavatgita on Android in Sanskrit and Russian
6) Class home page is at http://vkedco.blogspot.com/2011/08/android-programming-home.html

Published in Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,523
On SlideShare
0
From Embeds
0
Number of Embeds
41

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Android Programming Lecture 13 Vladimir Kulyukin Department of Computer Science Utah State Universitywww.youtube.com/vkedco www.vkedco.blogspot.com
  • 2. Outline ● AIDL and Bound Services ● AIDL Complex Types ● Speech Synthesis ● Overcoming TTS Limitations Through Phonetic Spelling ● Overcoming TTS Limitations Through Human Recordingwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 3. AIDL and Bound Serviceswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 4. Steps of Developing a Bound Service ● Define your interface to clients in an AIDL file ● An AIDL file has .aidl extension that uses Java syntax ● The package should be the same as the projects package ● The .aidl file should be saved in the /src directory of your Eclipse projectwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 5. Steps of Developing a Bound Service ● Android Eclipse plug-in calls the AIDL compiler to generate a Java interface file from the .aidl file ● Implement the onBind() method to return the interface object generated by the compiler ● Implement the method(s) specified in the .aidl file ● Declare the service in AndroidManifest.xmlwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 6. AIDLService Projectwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 7. The Problem Write a bound (remote) service that other applications can call to compute the sums of the first n Catalan or Fibonacci numberswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 8. AIDLService Project Demowww.youtube.com/vkedco www.vkedco.blogspot.com
  • 9. Defining Remote Interface in .aidl File // save this file, ISumService.aidl, in /src directory package org.vkedco.android.aidlsumservice; interface ISumService { long catalanSum(long n); long fibonacciSum(long n); }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 10. Compiling AIDL File ● When you build a project, the AIDL compiler will generate ISumService.java and place it into /gen directory where R.java is placed ● ISumService.java will have the class ISumService.Stub that needs to be extend to implement the methods of the ISumService interfacewww.youtube.com/vkedco www.vkedco.blogspot.com
  • 11. Extending ISumService.Stub public class AIDLSumService extends Service { public class SumServiceImpl extends ISumService.Stub { public long catalanSum(long n) throws RemoteException { // implementation } public long fibonacciSum(long n) throws RemoteException { // implementation } } }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 12. AIDLSumService.onBind() public class AIDLSumService extends Service { public class SumServiceImpl extends ISumService.Stub { // } @Override public IBinder onBind(Intent arg0) { return new SumServiceImpl(); } }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 13. Declaring Service in AndroidManifest.xml <application android:icon="@drawable/ic_launcher" android:label="@string/app_name" > <activity ...> </activity> <service android:name="AIDLSumService"> <intent-filter> <action android:name="org.vkedco.android.aidlsumservice.ISumService" /> </intent-filter> </service> </application>www.youtube.com/vkedco www.vkedco.blogspot.com
  • 14. AIDLService Project Source Codewww.youtube.com/vkedco www.vkedco.blogspot.com
  • 15. Consuming a Bound Service ● Create a consumer application project whose package is different from the package of the .aidl file ● Copy the .aidl from from the service project into the the new package in the /src directory ● Build the project to generate the Java interface file in the /gen directorywww.youtube.com/vkedco www.vkedco.blogspot.com
  • 16. AIDLServiceConsumer Project Source Codewww.youtube.com/vkedco www.vkedco.blogspot.com
  • 17. Passing Complex Types to Remote Serviceswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 18. AIDL Type Support ● AIDL supports String and CharSequence ● AIDL supports other AIDL interfaces (they must be imported) ● AIDL supports complex types that implement android.os.Parcelable ● AIDL suports java.util.List and java.util.Map so long as these containers contain String, CharSequence or android.os.Parcelable ● Complex types require directional indicator: – in (value set by client) – out (value set by service) – inout (value set by both client and service)www.youtube.com/vkedco www.vkedco.blogspot.com
  • 19. What is android.os.Parcelable? ● Parcelable specifies how Android should serialize and deserialize objects during marshalling and unmarshalling ● There are four steps of implementing Parcelable interface: – Make your class implement Parcelable – Implement writeToParcel() that writes the current state of the object to a Parcel – Add a static CREATOR field – Create an aidl file that declares the classwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 20. The Problem Write a bound (remote) service that other applications can call to compute the range sums of the Catalan or Fibonacci sequences. A range sum is the sum of the sequence elements from element start upto element end. Implement a range sum as a complex AIDL type.www.youtube.com/vkedco www.vkedco.blogspot.com
  • 21. Implementing SumRange import android.os.Parcel; import android.os.Parcelable; public class SumRange implements Parcelable { private long mFrom; private long mUpto; private String mSumType; public static final Parcelable.Creator<SumRange> CREATOR = new Parcelable.Creator<SumRange>() { @Override public SumRange createFromParcel(Parcel source) {return new SumRange(source); } @Override public SumRange[] newArray(int size) { return new SumRange[size]; } }; // other class stuff }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 22. Implementing SumRange public class SumRange implements Parcelable { public SumRange() { mFrom = 0; mUpto = 0; } private SumRange(Parcel source) { readSumRangeFromSource(source); } @Override public int describeContents() { return 0; } @Override public void writeToParcel(Parcel p, int flags) { p.writeLong(mFrom); p.writeLong(mUpto); p.writeString(mSumType); } public void readSumRangeFromSource(Parcel source) { this.mFrom = source.readLong(); this.mUpto = source.readLong(); this.mSumType = source.readString(); } }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 23. Declaring SumRange in SumRange.aidl package org.vkedco.android.aidlrangesumservice; parcelable SumRange;www.youtube.com/vkedco www.vkedco.blogspot.com
  • 24. Defining ISumRangeService in ISumRangeService.aidl package org.vkedco.android.aidlrangesumservice; import org.vkedco.android.aidlrangesumservice.SumRange; interface ISumRangeService { long catalanSum(long n); long fibonacciSum(long n); long catalanRangeSum(in SumRange sr); long fibonacciRangeSum(in SumRange sr); }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 25. Implementing AIDLSumRangeService public class AIDLSumRangeRervice extends Service { public class SumRangeServiceImpl extends ISumRangeService.Stub { @Override public long catalanRangeSum(SumRange sr) { return AIDLSumRangeRervice.this.catRangeSum(sr); } @Override public long fibonacciRangeSum(SumRange sr) { return AIDLSumRangeRervice.this.fiboRangeSum(sr); } }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 26. Speech Synthesiswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 27. TTS: Text To Speech ● The Problem: Take a sequence of characters and generate a waveform ● Words are pronounced as a sequence of individual units called phones ● Phonetic alphabets describe how phones are pronounced ● Phonological rules specify how phones combine into speechwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 28. TTS Engine Anatomy ● Text Analysis – parse text (after transliterating it if necessary) and identify words and utterances ● Linguistic Analysis – identify phrases and assign prosodies (accents, emphasis, duration, pauses, etc) ● Waveform Generation - generate a waveform from a fully specified linguistic descriptionwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 29. TTS Approaches ● Full Automation – machine does everything ● Mixed Initiative – human records a set of known texts; machine learning is used to extract the rules ● Human-Based Recording – human records words/sentences/texts; machine plays them as neededwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 30. PICO ● Pico is an Android multi-lingual speech synthesis language ● Pico can be used as a black box: text in, speech out ● Pico is the only TTS engine on the devicewww.youtube.com/vkedco www.vkedco.blogspot.com
  • 31. TTS Sharing ● TTS engine is shared among all activities on the device ● It is never a certainty that the text is spoken and, if it is, when it is spoken ● The only way to find out what is going on is to use the TTS callback methodswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 32. Starting TTS ● It is best practice to check if TTS is available on the device ● This is done via Intent to check TTS data ● If the check is successful, a instance of TTS can be created ● Activity that uses TTS implements OnInitListener interfacewww.youtube.com/vkedco www.vkedco.blogspot.com
  • 33. Checking TTS Availability ● In Activity.onCreate(): Intent ttsi = new Intent(); ttsi.setAction(TextToSpeech.Engine.ACTION_CHECK_TTS_DATA); startActivityForResult(ttsi, CHECK_TTS_REQ);www.youtube.com/vkedco www.vkedco.blogspot.com
  • 34. Creating TTS Instance ● In Activity.onActivityResult(): switch ( result_code ) { case TextToSpeech.Engine.CHECK_VOICE_DATA_PASS: mTTS = new TextToSpeech(this, this); break; }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 35. Handling Missing TTS Data ● In Activity.onActivityResult(): switch ( result_code ) { case TextToSpeech.Engine.CHECK_VOICE_BAD_DATA: case TextToSpeech.Engine.CHECK_VOICE_DATA_MISSING_DATA: case TextToSpeech.Engine.CHECK_VOICE_DATA_MISSING_VOLUME: Intent insti = new Intent(); insti.setAction(TextToSpeech.Engine.ACTION_INSTALL_TTS_DATA); startActivity(insti); }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 36. Handling TTS Unavailability ● In Activity.onActivityResult(): switch ( result_code ) { case TextToSpeech.Engine.CHECK_VOICE_DATA_FAIL: // Let the user know that TTS is not available – Toast, Log Message, Notification, etc. }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 37. What To Do When TTS Is Ready ● Override Activity.onInit(): @Override public void onInit(int status) { if ( status == TextToSpeech.SUCCESS ) { // do something when you know that TTS is available } }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 38. Overriding onPause() and onDestroy() ● When your Activity is paused (e.g., it loses focus), have TTS stop synthesizing ● When your Activity is destroyed, shut TTS down to notify Android that the resources can be released and given to other activities or applicationswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 39. Overriding onPause() and onDestroy() public void onPause() { super.onPause(); if ( mTTS != null ) mTTS.stop(); }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 40. Overriding onPause() and onDestroy() public void onDestroy() { super.onPause(); if ( mTTS != null ) mTTS.shutdown(); }www.youtube.com/vkedco www.vkedco.blogspot.com
  • 41. SimpleTTSDemo Project Demowww.youtube.com/vkedco www.vkedco.blogspot.com
  • 42. SimpleTTSDemo Project Sourcewww.youtube.com/vkedco www.vkedco.blogspot.com
  • 43. Overcoming TTS Limitations ● Every TTS engine mispronounces some words ● There are two ways of overcoming this limitation: – Phonetic spelling: spell mispronounced words the way they sound, generate waveforms, and associate words with waveforms – Human recording: have a human record mispronounced words and use the fileswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 44. Overcoming TTS Limitations Through Phonetic Spelling Teaching Android to Pronounce “Don Quixote”www.youtube.com/vkedco www.vkedco.blogspot.com
  • 45. TTSDonQuixote Project Demowww.youtube.com/vkedco www.vkedco.blogspot.com
  • 46. Teaching Android to Speak Sanskrit and Russian Overcoming TTS Limitations through Human Recordingwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 47. What Is This?www.youtube.com/vkedco www.vkedco.blogspot.com
  • 48. Bhagavatgita: Verse 1www.youtube.com/vkedco www.vkedco.blogspot.com
  • 49. Bhagavatgita: Verse 1 (Latin Transliteration) dharmakshetre kurukshetre samaveta yuyutsavah mamakah pandavashcaiva kim akurvata samjayawww.youtube.com/vkedco www.vkedco.blogspot.com
  • 50. What Is This? Что свершили, - скажи Санджая, - сыновья мои и Пандавы, ради битвы сойдясь на поле Kурукшетры, на поле дхармы? Перевод В.С. Семенцовwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 51. Bhagavatgita, V. 1 (Russian Translation) Что свершили, - скажи Санджая, - сыновья мои и Пандавы, ради битвы сойдясь на поле Kурукшетры, на поле дхармы? Перевод В.С. Семенцовwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 52. Bhagavatgita, V. 01 (Latin Transliteration) Chto svershili, - skazhi Sandzhaya, - synovya moi i Pandavy, radi bitvy soydyas na pole Kurukshetry, nа pоlе dharmy? Translated by V.S. Sementsovwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 53. Bhagavatgita: Verse 1 (English Translation) Oh, Sanjaya, tell me what happened at Kurukshetra, the field of dharma, where my family and the Pandavas gathered to fight? Translated by Eknath Easwaranwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 54. BhagavatGita Projectwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 55. The Problem Have Android read the first verse of Bhagavatgita in Sanskrit and Russian.www.youtube.com/vkedco www.vkedco.blogspot.com
  • 56. Logical Steps of One Solution ● Write a Devanagari transliterator that takes Sanskrit texts and produces their Latin transliterations ● Write a Cyrillic transliterator that takes Russian texts and produces their Latin transliterations ● Have human readers record Sanskrit and Russian words ● Associate strings with specific recordingswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 57. Real Steps ● Skip transliterator implementation ● Record .wav files ● Associate .wav files with specific strings ● Implement an IntentService that plays those files when strings are sent to itwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 58. BhagavatGitaTTS Project Demowww.youtube.com/vkedco www.vkedco.blogspot.com
  • 59. BhagavatGitaTTS Project Source Codewww.youtube.com/vkedco www.vkedco.blogspot.com
  • 60. Picking up Where You Stopped ● TTS has a useful callback OnUtteranceCompletedListener.onUtteranceCompleted() ● This method is invoked when TTS has completed speaking a piece of text ● This callback can be used to save the last utterance number and pick up where TTS has stopped when Activity is active againwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 61. References & Reading Suggestions ● http://developer.android.com/reference/android/speech/tts/TextToSpeech.html ● http://developer.android.com/resources/articles/tts.html ● http://en.wikipedia.org/wiki/Catalan_number ● http://developer.android.com/guide/developing/tools/aidl.html ● Hashimi, Komatineni, MacLean, “Pro Android 2”www.youtube.com/vkedco www.vkedco.blogspot.com
  • 62. Feedback Bugs, errors to vladimir dot kulyukin at gmail dot comwww.youtube.com/vkedco www.vkedco.blogspot.com