Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Programming Media on WP

Nguyen Tuan [ Microsoft Certified Trainer
Agenda

- The Windows Phone Camera
- Taking still images
‘ Manipulatingthe video stream

- The Windows Phone Microphone

-...
Using the Camera
The Camera

- There are a number of ways the camera can be used by an application

- The application can launch the Camera...
Capturing a photo

cameraTask =  new CameraCaptureTasl<(); 

cameraTask. Comp1eted +=  new Ex/ entHand1er‘<PhotoResu1t>
(c...
Capture complete

void cameraCapture_Completed(object sender‘, 
Photokesult e)

{

if (e. TaskResult = = TaskResult.0K)

{...
In application capture

- It is also possible to capture an image from within your application
- An application can also d...
The Photocamera class

using MicrosoFt. Devices; 
Photocamera camera; 

camera =  new PhotoCamera(); 
// Set the VideoBrus...
Displaying the Viewfinder

<Rectang1e widt "329" Height: "249" Horizontalnignment: "Left" >
<Rectang1e. Fi11>
<VideoErush ...
Initiating the Capture

private void PhotoButton_Click(object sender,  RoutedEventArgs e)

{

camera .  CaptureImage(); 

...
Saving the image

using Microsoft. Xna. Framework, Media; 
void camera_CaptureImageAvailable(object sender, 
ContentReadyE...
Displaying the image

using System. windows. Med1a. Imag1ng; 
void camera_CaptureImageAva1lable(object sender, 
ContentRea...
Saving to the Local Folder

using System.  I0;
u sing System.  I0. Isolatedstorage; 

using (IsolatedStor‘ageFile isstore ...
Creating a Lens

- A Lens is a custom camera application
which can be accessed from within the
camera application

- An ap...
Creating a Lens application

<Extensions>
<Extension ExtensionName= "Camer‘a_Captur‘e_App"
ConsumerID=  "(5E94E775»356E-4A...
Adding the Lens Icons

   

- Three Icons are required,  one for each Windows Phone screen size

- WVGA 173 x 173 AssetsLe...
Lens Startup

- You can create a URI Mapper to direct the application to the page that

implements the viewfinder for the ...
Image Processing in a Lens

- The PhotoCamera class is used to provide access to the video stream

- This can be used by a...
Funky Image Processing

internal int FunkyColor(int color)

{
int a
int r

color >> 24;

(color & exeefflaeee) >> 16;

int ...
Starting the Camera

camera :  new Microsoft. Devices. PhotoCamera(); 

/ / Create the destination for the processed image...
Starting the Camera

void camera_Initialized(object sender,  Cameraoper‘aticnCompletedEventAr‘g5 e)
{

pumpARGEFrames :  t...
Getting the Image Data

int[] ARGBPX =  new int[649 * 480]; 

captureEvent. Hait0ne( ); 

pauseFramesEvent. Hait0ne();  //...
Drawing the Image Data

private writeab1eBitmap wb; 

pauseF ramesEvent .  Reset ( ); 
Deployment .  Cu rrent .  Dispatche...
Demo

PhotoEditor
Creating an Auto—Up| oader for Photos

<Extensions>
<Extension E><tensionName: "Photos_Auto_Upload"
Consumerl *"{5B94B775-...
Using the Microphone
The Microphone

- The Windows Phone microphone can capture 16 bit audio
- Microphone input is managed as part ofthe XNA fr...
Sensors

sue/ ma

ti! 

Sgeeth ms

vow

Mmm mrmdenzm

2a
Sensors Available

- There are a number of different sensors: 

- Accelerometer

- Compass

- Gyroscope

- Inclinometer

-...
The Windows Runtime Sensors Library

using windows . Devices . Sensors; 

- The Windows Phone Runtime sensors AP|5 are com...
Determining Sensor Availability

/ / Determine whether we have a gym on the phone
_gyr*nmeter- =  Gyrometor‘. GetDeFau1t()...
Starting and Stopping a Sensor

/ / Establish tne report interval (units are milliseconds)
uint repartlnterval =  me; 

if...
Using the Sensor Readingchanged event

_gyrumeter. RepartInterva1 =  132;
_gyr*umeter*Jleadingchanged +:  (gyr*umeter*_Rea...
Getting Readings By Polling a Sensor

/ / Alternative to Readingchanged event,  can GetCurrentReading() to poll the sens r...
Demo

Sensors
Video Content
Video on the Phone

<MediaE1ement Name:  "MediaP1aybacl<"
Source:  "myvideo. wmv" AutoPlay: "True"/ >

- An application ca...
Streaming Video on the Phone

 

<MediaE1ement Name 'MediaPlaybacl<"
Sourc http : //mschanne 19 . vo .  msecnd . net/ o9/m...
Controlling Playback

private void pauseEutton_Clicl<(object sender,  RuutedEventArgs e)

{
}

MediaPlaybacl< . Pau se( );...
Smooth Streaming

- Windows Phone also supports Smooth Streaming

- This is an adaptive streaming that manages the quality...
Demo
MediaP| a/ back
Speech on Windows Phone 8
Speech Support

- Windows Phone 7.x had voice support built into the operating system

' Programs and phone features could...
Speech Synthesis
Enabling Speech Synthesis

- If an application wishes to use speech
output the
ID_CAP_SPEECH_RECOGNITION

capability must ...
Simple Speech

async void CheeseLiker()

{

Speechsynthesizer synth =  new SpeechSynthesizer‘(), ‘

await synth. SpeakText...
Selecting a language

/ / Query for a voice that speaks French, 

var Frenchvoices :  from voice in Insta11edVoices. All
w...
Demo
Speechsemples
Speech Synthesis Markup Language

<? xml version '1.e" encoding= "ISO-8859-1"P)
<speak versio 1.9"
xmlns= httg: [[www. w3....
Controlling Applications using
Voice Commands
Using Voice command

- The Voice Command feature of Windows Phone 7 allowed users to start
applications

- In Windows Phon...
The Fortune Teller Program

- The Fortune Teller program will
tell your future

- You can ask it questions and it
will dis...
The Voice Command. ..

<CommandPrefix> Fortune Teller </ CommandPre-Fix)
<Example> will I find money </ Example)
<Command ...
The Voice Command. ..(cnt)

<CommandPrefix> Fortune Teller </ CommandPrefix>
<Example> will I find money </ Example) 1
<Co...
The Voice Command. ..(cnt)

<CommandPrefix> Fortune Teller </ CommandP
<Examp1e> will I find money </ Example)
<Command Na...
The Voice Command. ..(cnt)

<CommandPrefix> Fortune Teller </ CommandPrefi
<Example> will I find money
<Command Name= "sho...
The Voice Command. ..(cnt)

<CommandPrefix> Fortune Teller </ CommandPre-Fix)
<Example> will I find money </ Example)
<Com...
The Voice Command. ..(cnt)

<CommandPrefix> Fortune Teller </ CommandPr*e-Fix)
<Example> will I find money </ Example)
<Co...
The Voice Command. ..(cnt)

<CommandPrefix> Fortune Teller </ CommandPre-Fix)
<Example> will I find money </ Example)
<Com...
The Voice Command. ..(cnt)

<CommandPrefix> Fortune Teller </ CommandPr*e-Fix)
<Examp1e> will I find money </ Example)
<Co...
The Voice Command. ..(cnt)

<CommandPrefix> Fortune Teller </ CommandPre-Fix)
<Example> will I find money </ Example)
<Com...
The Voice Command. ..(cnt)

<CommandPrefix> Fortune Teller </ CommandPre-Fix)
<Example> will I find money </ Example)
<Com...
Installing a Voice Command

async void setupVoiceCommands()
{
await / oicecommandservice. Insta1lCommandSetsFmmFileAsync(
...
Launching Your App With a Voice Command

- If the user now presses and holds the Windows button,  and says: 
Fortune Telle...
Handling Voice Commands

if (e. NavigationMode : = System. windows. Navigation, NavigationMode. New) {
if (Navigationconte...
Identifying phrases

<PhraseList Labe1:"w‘utur*eMoney">

<Item> money </ Item)

<Item> riches </ Item)

<Item> gold </ Ite...
Demo

V0iceRec0der
Review

- There are two types of scheduled notifications,  Alarm and Reminder
- An Alarm allows you to specify a sound file...
Upcoming SlideShare
Loading in …5
×
Upcoming SlideShare
Daniel's Family Tree
Next
Download to read offline and view in fullscreen.

0

Share

Download to read offline

06.Programming Media on Windows Phone

Download to read offline

Windows Phone 8 progamming

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

06.Programming Media on Windows Phone

  1. 1. Programming Media on WP Nguyen Tuan [ Microsoft Certified Trainer
  2. 2. Agenda - The Windows Phone Camera - Taking still images ‘ Manipulatingthe video stream - The Windows Phone Microphone - The Windows Phone Sensors ‘ The Motion sensor - Video Content - Text to Speech and Speech to Text in windows Phone 8
  3. 3. Using the Camera
  4. 4. The Camera - There are a number of ways the camera can be used by an application - The application can launch the CameraCaptureTask chooser to allow the user to take a photograph - The application can use the Photocamera class to capture photos or stream video data from the camera - can use this ror product recognition or augmented reality - Use the PhotoCaptureDevice class for advanced photo capture and AudioVideoCaptureDevice for advanced video capture - A real-time video processing application can be registered as a "Lens” - it can be selected by the userfrom a menu or available lens types - The Lens application will provides a viewfinder display and performs video processing on the signal from the camera
  5. 5. Capturing a photo cameraTask = new CameraCaptureTasl<(); cameraTask. Comp1eted += new Ex/ entHand1er‘<PhotoResu1t> (cameraTask_Comp1eted); cameraTask . Show( ); - This task launches the camera so that the user can take a picture - The camer'aTask_Completed event fires when the picture is taken - The emulator will return an image containing a small block
  6. 6. Capture complete void cameraCapture_Completed(object sender‘, Photokesult e) { if (e. TaskResult = = TaskResult.0K) { photoImage. Source = new BitmapImage( new Ur‘i(e.0riginalFileName)); } } - When capture has been made, completed method is executed - This version just displays the image on the screen - You can access the photo stream data to save it
  7. 7. In application capture - It is also possible to capture an image from within your application - An application can also display a viewfinder - The application can also access the video data directly and use this for augmented reality or to create a customised viewfinder ‘ eg. Barcode scanners - The PhotoCamer‘a class provides camera control and access to the camera video screen
  8. 8. The Photocamera class using MicrosoFt. Devices; Photocamera camera; camera = new PhotoCamera(); // Set the VideoBrush source to the camera viewfinderBrush. SetSource(camera); camera. CaptureImageAvai1ab1e += new EventHand1er<ContentReadyEventArgs> (camera_CaptureImageAvai1ab1e); - This creates a camera and binds a handler to the captured event - The view-Finder'Br'ush source is set to the camera
  9. 9. Displaying the Viewfinder <Rectang1e widt "329" Height: "249" Horizontalnignment: "Left" > <Rectang1e. Fi11> <VideoErush x2Name: "viewFinder~Er~ush" / > </ Rectang1e. Fi11> </ Rectangle) - This is the rectangle in the xaml for the camera viewfinder page - This will display the viewfinder on the screen - The source for the viewfinder brush is set to the camera
  10. 10. Initiating the Capture private void PhotoButton_Click(object sender, RoutedEventArgs e) { camera . CaptureImage(); ) - This is the event handler for the photo button - It asks the camera to take a picture - The current camera settings are used for this - You can override these (for example turn the flash on or off) by setting properties on the camera instance
  11. 11. Saving the image using Microsoft. Xna. Framework, Media; void camera_CaptureImageAvailable(object sender, ContentReadyEventArgs e) { Deployment. Current. Dispatcher. BeginInvoke(de1egate() { string fileName = DateTime. Now. Ticks. ToString() + "Jug"; MediaLibrary library = new MediaLibrary(); library. SavePictureToCameraRoll(fileName e. ImageStream); ))s ) - This saves the image in the camera roll
  12. 12. Displaying the image using System. windows. Med1a. Imag1ng; void camera_CaptureImageAva1lable(object sender, ContentReadyEventAr‘gs e) { Deploymentcurrent. Dispatcher. Beg1nInvoke(delegate() { Bitmaplmage b = new BitmapImage(); b. Cr*eate0pt1ons = Bitmapcreateoptions. None; b. SetSource(e. Imagest ream); P1ctureImage. Source = b; D; ) - This saves the image into a Bitmaplmage which is displayed on the screen in an <Image>
  13. 13. Saving to the Local Folder using System. I0; u sing System. I0. Isolatedstorage; using (IsolatedStor‘ageFile isstore = Isolatedstoragefile. GetuserstoreFompplication()) { using (Iso1atedStorageFilestream targetstream = isstore.0penFile(fileName, FileMode. Create, Fi1eA: :ess. lrlrite)) { writeableflitmap bitmap = new wr‘iteableBitmap(b), ' bitmap. Save: Ipeg(targetStream, bitmap. Pixellrlidth, bitmap. Pixe1Height, 9, 100); ) } - This saves the image in the local folder
  14. 14. Creating a Lens - A Lens is a custom camera application which can be accessed from within the camera application - An application is flagged as a Lens application by setting a flag in the manifest and providing icons that can be used to browse for the Lens when the camera is in use - I've created a FunkyCamera lens application which I have registered in this way
  15. 15. Creating a Lens application <Extensions> <Extension ExtensionName= "Camer‘a_Captur‘e_App" ConsumerID= "(5E94E775»356E-4AA9-AAF8~6491FFEA5631}" TaskID= "_defau1t" / > </ Extensions> - This text must be added to the WMAppManifest. xm| file for the application, just after the <Tokens> section - There is no GUI for this alteration, you have to edit the XML directly
  16. 16. Adding the Lens Icons - Three Icons are required, one for each Windows Phone screen size - WVGA 173 x 173 AssetsLens. Screen—WVGA. png - 720p 259 x 259 AssetsLens. Screen—720 p. png ' WXGA 277 x 277 AssetsLens. Screen—WXGA. png - They are all placed in the Assets folder of the application - Use a transparent background to match the Windows Phone color scheme
  17. 17. Lens Startup - You can create a URI Mapper to direct the application to the page that implements the viewfinder for the Lens ‘ This can use the string "Viewfinder| aunch” in the destination uri that is activated when the user selects the lens application - Alternatively, if the program only contains a single page this page is displayed - If the user "backs out” of the lens application they will be returned to the camera
  18. 18. Image Processing in a Lens - The PhotoCamera class is used to provide access to the video stream - This can be used by applications to perform image processing on the live data - Augmented reality ‘ Scanning applications - We are going to use it to make the funky camera display a funky image
  19. 19. Funky Image Processing internal int FunkyColor(int color) { int a int r color >> 24; (color & exeefflaeee) >> 16; int g (color & exeeeeffee) >> 8; int b (color & exeeeeeeff); r +: redoffset; g +: greenoffset; b +: blueoffset, ‘ return ((a & 9xFF) << 24) | ((r & 9xFF) << 16) | ((g & 9xFF) << 8) | (b & 9xFF); - This separates out the primaries and adds an offset to each - It is called for each pixel in the image
  20. 20. Starting the Camera camera : new Microsoft. Devices. PhotoCamera(); / / Create the destination for the processed image wb : new NriteableEitmap(649, 489); this. ProcessedImage. Source : wb; / / Start the image pump when the camera is ready camera. Initialized +: new EventHandler‘<CameraoperationcompletedEventArgs> (came ra_Initialized); - This creates the camera and the bitmap that will contain the image processed output - It binds to the event fired when the camera is ready
  21. 21. Starting the Camera void camera_Initialized(object sender, Cameraoper‘aticnCompletedEventAr‘g5 e) { pumpARGEFrames : true; ARGBFramesThread : new System. Thr‘eading. Thr‘ead(PumpARGBFr‘ames); ARGBFrame sThread . Start( )3 - When the camera is ready we start the thread that will pump frames into our image processor - This will run alongside our application
  22. 22. Getting the Image Data int[] ARGBPX = new int[649 * 480]; captureEvent. Hait0ne( ); pauseFramesEvent. Hait0ne(); // Thread sync with camera / /Copies the current viewfinder frame into a buffer camera . GetPreviewBuFferArgb32(ARGBPX); / /Conversion to ‘Funky colours for (int i = 0; i < ARGBPx. Length, ' i++) { ) ARGBPX [i] = FunkyCo1or(ARGBPx[i]) ; - This code grabs the preview buffer from the camera and processes it
  23. 23. Drawing the Image Data private writeab1eBitmap wb; pauseF ramesEvent . Reset ( ); Deployment . Cu rrent . Dispatcher . Beginlnvoke (de1egate() { / /Copy to writeableaitmap ARGBPx. CopyTn(wb. Pixels, 0); wb. Inval1date(), ' pauseF ramesEvent . Set( ); D; - This code writes the processed pixels back to a writeable bitmap that is displayed on the screen
  24. 24. Demo PhotoEditor
  25. 25. Creating an Auto—Up| oader for Photos <Extensions> <Extension E><tensionName: "Photos_Auto_Upload" Consumerl *"{5B94B775-3565-4AA9-AAF8-6491FFEA5632}" TaskID: _de'Fault" / > </ Extensions) - vou can also create an application with an associated ”resaurce intensive” background task that can auto-upload picturesthat the user may take to a cloud storage service - This is a resource intensive background process and may run when the phone is on charge - The user can select your app vrom the Camera settings - The application must set the extension shown above and display a settings page wherethe user can set authentication and upload options
  26. 26. Using the Microphone
  27. 27. The Microphone - The Windows Phone microphone can capture 16 bit audio - Microphone input is managed as part ofthe XNA framework - It is possible to record sound, process the audio and either store or replay it - There is a complete example of how to do this on MSDN
  28. 28. Sensors sue/ ma ti! Sgeeth ms vow Mmm mrmdenzm 2a
  29. 29. Sensors Available - There are a number of different sensors: - Accelerometer - Compass - Gyroscope - Inclinometer - Orientation - All the sensors are used in the same way: - They will fire an event when they have a reading - Two APIs for managed code developers: - Microsoft. Devices. Sensors (Windows Phone 05 7.1 API set) - Windows. Devices. Sensors (Windows Phone Runtime)
  30. 30. The Windows Runtime Sensors Library using windows . Devices . Sensors; - The Windows Phone Runtime sensors AP|5 are compatible with sensor usage in WinRT on Windows 8 - Accelerometer — returns G-force values with respect to the x, y, and z axes - All phones have an accelerometer - Inclinometer - returns pitch, roll, and yaw values that correspond to rotation angles around the x, y, and z axes, respectively - The inclinometer readings are derived rrorn multiple sensors - Gyrometer - retums angularvelocity values with respect to the x, y, and z axes - Compass - retums a heading with respect to True North and, possibly, Magnetic North - Orientationsensor - returns a rotation matrix and a Quaternion that can be used to adjust the user's perspective in a game application - Combines the data from the accelerometer, compass, and gyrometer — known as ”Sensor Fusion"
  31. 31. Determining Sensor Availability / / Determine whether we have a gym on the phone _gyr*nmeter- = Gyrometor‘. GetDeFau1t(); if (_gyr*ameter* 1: null) ( / / Establish the renurt interval (units are milliseconds) zvrmnetenkenartlnterval = 199: _gyr*ameter. ReadingChanged += (gyr*ameter‘_ReadingChanged; l else I l MessageBo><. Shaw("No gymneter found"); - All the sensor classes have a GetDefau1t() method - This method only returns values for hardware that has been integrated into the computer by the manufacturer - Returns null if the sensor is not available on that device - All phones will have an accelerometer
  32. 32. Starting and Stopping a Sensor / / Establish tne report interval (units are milliseconds) uint repartlnterval = me; if (_gyrmneter. MinimuniRepar-tlnterval > repartlnterval) l } repartlnterval = /gyrnmeter‘. MinimumRepar‘tInter*va1; _gyrmneter. Repar-tlnterval = repartlnterval; - Application must set the report interval to a non-zero value prior to registering an event handler or calling GetCurrentReading to activate it - When finished with the sensor, set it to zero - Check the MinimumReport| nterva| property - Setting a value below the minimum supported interval will either trigger an exception or have undefined results. - Sensor driver will determine the actual report interval
  33. 33. Using the Sensor Readingchanged event _gyrumeter. RepartInterva1 = 132; _gyr*umeter*Jleadingchanged +: (gyr*umeter*_Readingchanged; private / aid _gyr*z: meter*_ReadingChanged(6}/ romatcr sender, GyremoterRead1ngChangedEvcntArgs args) ( Dispatcher. aegin1nvuke(() : > ( Gymuivweterllcadlng reading = ar*gs. Reading; x_Reading. Text = String. Fur*mat("(B, S:B. BB)"_, reading. Angu1arve1ucityx); v_Reading. Text = Stung. Fur*mat("(B, S*B. BB}"_, reading. Angu1arve1ucityv); z_Reading. Text = String. Fur*mat("(B, S:B. BB)"_, reading. Angu1arve1acityz); D; - Registerthe ReadingChanged event handler to obtain sensor readings - Must set the Reportlnterval property first
  34. 34. Getting Readings By Polling a Sensor / / Alternative to Readingchanged event, can GetCurrentReading() to poll the sens r GyrometerReading reading : _gyrometer. GetCurrentReading()3 if (reading ! : null) { X_Reading, Text : String. Format("{a,5:9.99)", reading. AngularVelocityX); Y_Reading, Text : String. Format("{9,5:9.99}", reading. AngularVelocityY); Z_Reading. Text : String. Format("{9,5:9.99)", reading. AngularVelocityZ); } - An application can poll the sensor for the current reading as an alternative to registering a Readingchanged event handler 0 The preferred alternative for an application that updates its user interface at a specific frame rate - Must still establish a desired Reportlnterval before polling in order to activate the sensor
  35. 35. Demo Sensors
  36. 36. Video Content
  37. 37. Video on the Phone <MediaE1ement Name: "MediaP1aybacl<" Source: "myvideo. wmv" AutoPlay: "True"/ > - An application can contain a single MediaE| ement that can play video - The sample above plays a resource file that is part of the project containing the application - You can find a list of supported codecs here: http 2 / /msdn . mic rosoft . com/ en— us/ library/1313462687 . aspx
  38. 38. Streaming Video on the Phone <MediaE1ement Name 'MediaPlaybacl<" Sourc http : //mschanne 19 . vo . msecnd . net/ o9/mix/99/wmv/ l<eya1,wmv" AutoP1ay: "True"/ > - If you want to stream from the internet, just replace the source file with a url - This implementation has the location hard coded into the XAML - You can also do this under program control
  39. 39. Controlling Playback private void pauseEutton_Clicl<(object sender, RuutedEventArgs e) { } MediaPlaybacl< . Pau se( ); - The MediaE| ement exposes methods that can be used to control the media playback - An application can also determine the properties ofthe media stream - To determine if it can be paused for example
  40. 40. Smooth Streaming - Windows Phone also supports Smooth Streaming - This is an adaptive streaming that manages the quality of the video signal in response to the abilities of the network connection - Playback quality is managed in real time to handle changes in network performance during viewing - It uses a server side plugin and client code on the viewing device - You can download the viewing software here: http: //sm-F. codeplex. com
  41. 41. Demo MediaP| a/ back
  42. 42. Speech on Windows Phone 8
  43. 43. Speech Support - Windows Phone 7.x had voice support built into the operating system ' Programs and phone features could be started by voice commands e. g ”Start MvAm>" ' Incoming SMS messages could be read to the user ' The user could compose and send SMS messages - Windows 8 builds on this to allow applications to make use of speech - Applications can speak messages using the Speech Synthesis feature - Applications can be started and given commands - Applications can accept commands using voice input - Speech recognition requires an internet connection, but Speech Synthesis does not
  44. 44. Speech Synthesis
  45. 45. Enabling Speech Synthesis - If an application wishes to use speech output the ID_CAP_SPEECH_RECOGNITION capability must be enabled in WMAppManifest. xm| - The application can also reference the Synthesis namespace Z lD_(AP_SPEE(H_KE(oGNlTloN using windows . Phone . Speech . Synthesi 5;
  46. 46. Simple Speech async void CheeseLiker() { Speechsynthesizer synth = new SpeechSynthesizer‘(), ‘ await synth. SpeakTextAsync("I like cheese. "); ) ' The Speechsynthesizer class provides a simple way to produce speech ° The SpeakTextAsync method speaks the content of the string using the default voice - Note that the method is an asynchronous one, so the calling method must use the async modifier - Speech output does not require a network connection
  47. 47. Selecting a language / / Query for a voice that speaks French, var Frenchvoices : from voice in Insta11edVoices. All where voice. Language : : "Fr-FR" select voice; / / Set the voice as identified by the query, synth . SetVoice(€renchVoices . ElementAt (9) )3 - The default speaking voice is selected automatically from the locale set for the phone - The | nsta| |edVoices class provides a list of all the voices available on the phone - The above code selects a French voice
  48. 48. Demo Speechsemples
  49. 49. Speech Synthesis Markup Language <? xml version '1.e" encoding= "ISO-8859-1"P) <speak versio 1.9" xmlns= httg: [[www. w3.org[2a91[19[synthesis xm1:1ang= "en-US") <p> Your <say-as interpret-as= "ordinal">: lst</ say-as> request was for <say-as interpret—as= "cardina1">1</ say-as) room on <say-as interpret-as= date" Forma mdy">10/19/Z019</ say—as> , arriving at <say—as interpret—a ime" Format= "hms: l2">: lZ:35pm</ say—as>. </ D) </ speak) - You can use Speech Synthesis Markup Language (SSML) to control the spoken output - Change the voice, pitch, rate, volume, pronunciation and other characteristics - Also allows the inclusion of audio files into the spoken output - You can also use the Speech synthesizerto speak the contents of a file
  50. 50. Controlling Applications using Voice Commands
  51. 51. Using Voice command - The Voice Command feature of Windows Phone 7 allowed users to start applications - In Windows Phone 8 the feature has been expanded to allow the user to request data from the application in the start command - The data will allow a particular application page to be selected when the program starts and can also pass request information to that page - To start using Voice Commands you must Create a Voice Command Definition (VCD) file that defines all the spoken commands - The application then calls a method to register the words and phrases the first time it is run
  52. 52. The Fortune Teller Program - The Fortune Teller program will tell your future - You can ask it questions and it will display replies - It could also speak them - Some ofthe spoken commands activate different pages of the application and others are processed by the application when it starts running
  53. 53. The Voice Command. .. <CommandPrefix> Fortune Teller </ CommandPre-Fix) <Example> will I find money </ Example) <Command Name= "showMoney"> <Example> will I find money </ Example) <ListenFor> [Hill I find] {futureMoney) </ ListenFor> <Feedback> Showing {futureMoney) </ Feedback) <Navigate Target= "/moneylxaml"/ > </ Command) <PhraseList Label= "futureMoney"> <Item> money </ Item) <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> - This is the "money" question: ”Fortune Teller Will I find money”
  54. 54. The Voice Command. ..(cnt) <CommandPrefix> Fortune Teller </ CommandPrefix> <Example> will I find money </ Example) 1 <Command Name= "showMoney"> <Example> will I find money </ Example) <ListenFor> [Hill I find] {futureMoney) </ ListenFor> <Feedback> Showing {futureMoney) </ Feedback) <Navigate Target= "/moneylxaml"/ > </ Command) <PhraseList Label= "futureMoney"> <Item> money </ Item) <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> -This1s the phrase the user says to trigger the command -All ofthe Fortune Teller commands start with this phrase
  55. 55. The Voice Command. ..(cnt) <CommandPrefix> Fortune Teller </ CommandP <Examp1e> will I find money </ Example) <Command Name= "showMoney"> <Example> will I find money </ Example) <ListenFor> [will I find] {futureMoney) </ ListenFor> <Feedback> Showing {futureMoney) </ Feedback) <Navigate Tar*get= "/moneyixaml"/ > </ Command) <PhraseList Labe1="futureMoney"> <Item> money </ Item) <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> 5 IL 2;): -This1s examp{e text that w'H be dismayed by the hem for this app as an exampe ofthe commands the app supports
  56. 56. The Voice Command. ..(cnt) <CommandPrefix> Fortune Teller </ CommandPrefi <Example> will I find money <Command Name= "showMoney"> <ExampIe> will I find money / Example) <ListenFor> [will I find] {futureMoney) </ ListenFor> <Feedback> Showing {futureMoney) </ Feedback) <Navigate Tar*get= "/moneyixaml"/ > </ Command) <PhraseList Label= "futureMoney"> <Item> money </ Item) <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> : ,1L, z;1A -This is the command name -This can be obtained from the URL by the application when it starts
  57. 57. The Voice Command. ..(cnt) <CommandPrefix> Fortune Teller </ CommandPre-Fix) <Example> will I find money </ Example) <Command Name= "showMoney"> A '' <Example> will I find money <_ <ListenFor> [Hill I find] {futuneMoney) </ ListenFor> <Feedback> Showing {futureMoney) </ Feedback) <Navigate Target= "/moneylxaml"/ > </ Command) <PhraseList Label= "futureMoney"> <Item> money </ Item) <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> -This is the example for this specific command
  58. 58. The Voice Command. ..(cnt) <CommandPrefix> Fortune Teller </ CommandPr*e-Fix) <Example> will I find money </ Example) <Command Name= "showMoney"> <Example> will I find mone <ListenFor> [will I find] istenFor> <Feedback> Showing {futureM ey) </ Feedback) <Navigate Tar*get= "/moneyixaml"/ > </ Command) <PhraseList Label= "futureMoney"> <Item> money </ Item) <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> £,1t, z;iA -This is the trigger phrase for this command -It can be a sequence of words -The user must prefix this sequence with the words ”Fortune Teiler"
  59. 59. The Voice Command. ..(cnt) <CommandPrefix> Fortune Teller </ CommandPre-Fix) <Example> will I find money </ Example) <Command Name= "showMoney"> <Example> will I find money </ Example) <ListenFor> [Hill I find] {futureMoney). <Feedback> Showing {futureMoney) </ Feedbaco <Navigate Target= "/moneylxaml"/ > </ Command) <PhraseList Label= "futureMoney"> <Item> money </ Item) <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> -This is the phraselist for the command -The user can say any of the words in the phraselist to match this command -The application can determine the phrase used -The phraselist can be changed by the application dynamically
  60. 60. The Voice Command. ..(cnt) <CommandPrefix> Fortune Teller </ CommandPr*e-Fix) <Examp1e> will I find money </ Example) <Command Name= "showMoney"> <ExampIe> will I find money </ Example) <ListenFor> [will I find] {futureM <Feedback> Showing {futureMoney) <Navigate Tar*get= "/moneyixaml"/ > </ Command) <PhraseList Label= "futureMoney"> <Item> money </ Item) <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> 5 IL 2;): -This 1s the spoken feedback from the command -The feedback wiH insert the phrase item used to activate the command
  61. 61. The Voice Command. ..(cnt) <CommandPrefix> Fortune Teller </ CommandPre-Fix) <Example> will I find money </ Example) <Command Name= "showMoney"> <Example> will I find money </ Example) <ListenFor> [Hill I find] {futureMoney) </ ListenFo_rV> <Feedback> Showing {futureMoney) </ "=: ,»rH«~»‘ " ”” <Navigate Target= "/moneyixaml"/ > . </ Command) . <PhraseList Label= "futureMoney"> <Item> money </ Item) <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> -This1s the url for the page to be activated by the command -Commands can go to different pages, or all go to MainPage. xam! ifrequired
  62. 62. The Voice Command. ..(cnt) <CommandPrefix> Fortune Teller </ CommandPre-Fix) <Example> will I find money </ Example) <Command Name= "showMoney"> <Example> will I find money </ Example) <ListenFor> [Hill I find] {futureMoney) </ ListenFor> <Feedback> Showing {futureMoney) </ Feedback) <Navigate Target= "/moneyixaml"/ > A </ Command) <PhraseList Label= "futureMoney">‘ <Item> money </ Item) < <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> -These are the phrases that can be used at the end of the command -The application can modify the phrase list ofa command dynamically -it could give movie times for films by name
  63. 63. Installing a Voice Command async void setupVoiceCommands() { await / oicecommandservice. Insta1lCommandSetsFmmFileAsync( new Uri("ms—appx: /// VCDCommands. xm1", UriKind. Re1at1ve0rAbsolute)); ) ° The VCD file can be loaded from the application or from any URI - In this case it isjust a file that has been added to the project and marked as Content - The VCD can also be changed by the application when it is running ° The voice commands for an application are loaded into the voice command service when the application runs - The application must run at least once to configure the voice commands
  64. 64. Launching Your App With a Voice Command - If the user now presses and holds the Windows button, and says: Fortune Teller, Will Ifind gold? the Phone displays "Showing gold" - It then launches your app and navigates to the page associated with this command, which is / Money . xaml - The query string passed to the page looks like this: "/ wascacammamiMama= shumaneyzmturemonay= ga1a&rem= :m~mnaxza1e11erxw111xza1xmr1naxzago1 d. . . . , . i ‘ l l i l Command Name Phaselist Recognized Whole phrase as It Name Phrase was recognized
  65. 65. Handling Voice Commands if (e. NavigationMode : = System. windows. Navigation, NavigationMode. New) { if (Navigationcontext. QueryString. ContainsKey("voiceCommandName")) { string command = Navigationcontext. QueryString["voiceCommandName"]; switch command) { case "te11Joke": messageTextBlock, Text = "Insert really funny joke here"; break; // Add cases for other commands. default: messageTextBlock, Text : "Sorry, what you said makes no sense. "; break; ) ) - his code runs in the 0nNavigatedTo method of a target page - Can also check for the voice command phrase that was used
  66. 66. Identifying phrases <PhraseList Labe1:"w‘utur*eMoney"> <Item> money </ Item) <Item> riches </ Item) <Item> gold </ Item) </ PhraseList> string moneyPhrase : Navigationcontext. QueryString["futureMoney"]3 - The navigation context can be queried to determine the phrase used to trigger the navigation - In this case the program is selecting between the phrase used in the "riches” question
  67. 67. Demo V0iceRec0der
  68. 68. Review - There are two types of scheduled notifications, Alarm and Reminder - An Alarm allows you to specify a sound file to play when the notification is launched - when you create a Reminder, you can specify a deep link URI - Applications can capture images and video feeds from the camera - Applications can create "Lens” behaviours that are accessed from the in-phone camera - Applications can provide upload behaviors to upload photographs - Applications can use sensors and determine which ofthem are present - The Mediacontent element provides for video playback

Windows Phone 8 progamming

Views

Total views

359

On Slideshare

0

From embeds

0

Number of embeds

1

Actions

Downloads

3

Shares

0

Comments

0

Likes

0

×