SlideShare a Scribd company logo
1 of 19
Download to read offline
i
Voice Response System (VRS) Test
Procedures
Visual Testing and Audio Tuning for EC Polling Stations
4/9/2015
Elections Canada / Élections Canada
Vietorisz, Paul
ii
Table of Contents
1 Overview .........................................................................................................................................................................1
1.1 High Level Statement of Work................................................................................................................................1
1.2 Tools and Applications............................................................................................................................................1
1.3 TTS Extract File Processing Stages...........................................................................................................................2
2 Visual Test Procedures – Stage 1 ....................................................................................................................................4
2.1 Checking for New Polling Addresses from EC’s Sites Operation’s Group...............................................................4
2.1.1 Visual Testing Feedback and Next Steps.........................................................................................................6
3 Audio Tuning Procedures – Stage 2 ................................................................................................................................7
3.1 Preparing excel files (French and English files) for Audio Testing (Nuance)...........................................................7
3.2 Launching the Audio Tuning Environment..............................................................................................................8
3.3 Tuning Instructions ...............................................................................................................................................12
3.3.1 Audio Tuning Feedback and Next Steps........................................................................................................13
4 Description of Primes and Their Responsibilities..........................................................................................................14
5 Address Formatting.......................................................................................................................................................15
5.1 General Address Rules ..........................................................................................................................................15
6 Building Name Translation Rules ..................................................................................................................................16
7 Glossary.........................................................................................................................................................................17
Table of Figures
FIGURE 1 END TO END VRS PROCESSING WORKFLOW...........................................................................................................................................2
FIGURE 2: VRS POST_41 WORKING FOLDER .......................................................................................................................................................3
FIGURE 3: COMMENTS COLUMNS ......................................................................................................................................................................4
FIGURE 4: COPYING A RECORD ..........................................................................................................................................................................5
FIGURE 5: LANGUAGE MAP ..............................................................................................................................................................................6
FIGURE 6: EXAMPLE OF COMMENTED ENGLISH FILE EXTRACT..................................................................................................................................7
FIGURE 7: TAB DELIMITED FILE..........................................................................................................................................................................8
FIGURE 8: VPN CONNECTION WINDOW .............................................................................................................................................................8
FIGURE 9: VPN LOGIN.....................................................................................................................................................................................9
FIGURE 10: MAPPING THE NUANCE INBOX..........................................................................................................................................................9
FIGURE 11: NUANCE INBOX MAPPED ...............................................................................................................................................................10
FIGURE 12: TTS EVALUATION TOOL.................................................................................................................................................................10
FIGURE 13: BASE URL WINDOW.....................................................................................................................................................................11
FIGURE 14: SELECTING THE DESIRED FILE ..........................................................................................................................................................11
FIGURE 15: AUDIO TUNING ENVIRONMENT.......................................................................................................................................................12
1
1 Overview
This section describes the scope of work, tools and applications, and provides an overview of the end-to-end test
process, based on sections 2 and 3.
1.1 High Level Statement of Work
• The scope of the work is to extract text files compiled of EC Polling Station data. Validate the quality of the
information provided in English and French, address any anomalies and then correct them prior to transferring
these text string Locations and Addresses into Voice Segments. Text Sample: St Paul’s church, 230 1st
Avenue,
Toronto
• The text strings are then transferred into Voice Segments. Quality and validation related to speech, in English
and in French occurs. The final step is to load the EC Polling Stations voice segments into the VRS. The Quality
Assurance related to speech is referred to as Tuning. Voice Segment sample: St Paul’s church, 230 1st
Avenue,
Toronto
In addition, the work process requires tracking and logging various work stages.
1.2 Tools and Applications
Stage 1 – tools and applications for visual testing:
• Excel spreadsheet, provided by Sites, listing all of the polling stations (records) to be tested
• Internet Explorer, Google Chrome, or Firefox
Any of these browsers can be used to do the research against each record listed on the Excel spreadsheet
Note: IE can be used without any approval. Approval, however, will be required if you wish to use one of the
other browsers.
Stage 2 – tools and applications for audio tuning:
• TTS Evaluation Tool
This web interface provided by Nuance is used for tuning Text to Speech (TTS) files. Tuning is the process of
listening to voice recordings based on text strings (associated with each polling station or record) compiled of
EC’s polling stations. The Nuance back end translates text to speech while the TTS Evaluation tool provides the
capability or means to listen and improve speech. The voice recordings are available in both official languages.
Tuning is the second stage of TTS file manipulation when considering the end to end work flow.
• Cisco VPN Client
Note: The Cisco VPN Client must be installed on the EC PC workstation as a prerequisite. This software is not
part of EC’s computer base image.
o The VPN Client is used to establish a secure connection channel between an EC IT Telecom workstation
and Nuance’s Tuning server.
o This connection insures access to Nuance’s shared folder where TTS files ready for tuning are saved.
o This connection also insures successful web connection to Nuance’s TTS Evaluation Tool.
o A Nuance provided username and password is required to establish a successful connection via EC’s VPN
Client.
Here are the paths to the “VPN Client install script” and to the “VPN Client Installation video”:
H:Projects162commVRS -TelephonyTSGTelecom Analyst Training Material 2012TTSVPN Client install script
H:Projects162commVRS -TelephonyTSGTelecom Analyst Training Material 2012TTSVPN Client Installation video
2
1.3 TTS Extract File Processing Stages
There are 2 test stages involved in the end to end process of testing a record:
- Stage 1: Visual testing, refer to Section 2 for step by step instructions
- Stage 2: Audio testing, refer to Section 3, for step by step instructions
Beginning with the visual testing in stage 1, VRS testers receive an Excel spreadsheet, referred to as a scrape or extract,
that lists all of the polling stations that need to be tested. Some polling stations may be new while others may have been
modified but in either case all of the polling stations listed will need to be inspected (tested) before Sites can put them
into production, back into the Sites database. Each scrape (also called extract or delta) is represented as a “delta” as
shown in Figure 1. VRS testers examine the information associated with each polling station to determine accuracy.
Some may or may not warrant a comment. Polling stations or “records” from here on, that have no comment means
they passed the visual inspection and can therefore be pushed forward to stage 2, audio testing. However, all of the
commented records that did not pass visual inspection are returned to the Sites team for corrections. The Sites team
examines the comments and decides whether to accept or reject the comments. They will indicate on the spreadsheet
that is returned to the VRS testers, which were accepted and which were rejected. They may use “Yeses” or “noes”:
- Yes to indicate that the comment was accepted and that the record was updated and “fixed”
- No to indicate that the comment was rejected and that the record was left as is and therefore “not fixed”
Sites returns this feedback, that is the same spreadsheet marked with “yeses” and “noes”, to the VRS testers who can
then push records that are “Not Fixed” directly to stage 2 for audio tuning. “Fixed” records however would have to be
visually retested and therefore may or may not be commented against. This is repeated until all records pass visual.
Figure 1 End to End VRS Processing Workflow
3
Records in either stage 1 or stage 2 must be retested until they are deemed good.
Excel file extracts prepared by Sites, referred to as the Delta.xls in Figure 1, for the VRS team are dropped into the
“WEEKLY UPDATES” folder every Friday on a weekly basis in the following path:
H:Projects162commDatabaseTTS Data Extract From Sitespost_41
The “post_41” folder is the VRS team’s working folder that contains all files for visual and audio testing as shown in
Error! Reference source not found..
Figure 2: VRS post_41 Working Folder
Content in the working folder:
- 00 - WEEKLY UPDATES: contains files sent from Sites for VRS testing. These files are also known as “deltas” or
“scrapes”. The files will be named as follows, tts_extract_<month and day>_<year>_<language>.xls, for
example, tts_extract_Apr7_2015_english.xls. A VRS tester can also break up a file if necessary into smaller
manageable chunks, for example: tts_extract_Apr7_2015_english_01, tts_extract_Apr7_2015_english_02, etc. A
VRS tester’s name can also be added to track who is working on which file, for example:
tts_extract_Apr7_2015_english_01_John, tts_extract_Apr7_2015_english_02_Tom.
NOTE: TTS files are deposited every Friday. A TTS file that is empty can be deleted. It may occur that a TTS file
deposited on any given Friday may be empty of content; this is a result of the script running as scheduled but
has found no new Polling Station data to process.
- 01 - TTS to do: files from the WEEKLY UPDATES are moved to this folder for assignment and optional chunking
when necessary. The name of a specific tester can be added to the filename to identify who will be working on
it. A large file can also be broken up into smaller manageable chunks when necessary or practical. For example,
you can take a single file with say 3,000 records and split it up into 10 files each consisting of 300 records.
- 02 - Work in progress: VRS testers choose a file from the “TTS to do” folder and move it to this folder when they
are ready to start testing.
- 03 - Extracts sent to SITES_visual_validation_Done: testers that have completed their visual testing of a file can
then move the file to this folder to indicate that it has been completed and can be sent to sites.
- 04 – Backup: all files sent to Sites after they have been visually tested are backed up to this folder.
- 05 – Misc: used as necessary.
- 06 - Nuance Reports: will contain information to inform Sites of what can or cannot be accepted or fixed.
- TTS Progress Tracking Sheet: files in folder “02- Work in progress” are tracked by this file for the VRS tester’s
purposes and therefore any file a tester chooses to work on must open it from folder 02!
4
2 Visual Test Procedures – Stage 1
2.1 Checking for New Polling Addresses from EC’s Sites Operation’s Group
Follow these instructions to visually inspect the information contained on the French and English Excel
spreadsheets before moving onto the audio testing stage covered in Section 3:
1. Go to the H drive and access: H:Projects162commDatabaseTTS Data Extract From Sitespost_4100 -
WEEKLY UPDATES
Tip: you can also create a shortcut to the “post_41” folder for quick access. Follow instructions according to your
operating system to map a shortcut.
Note: WEEKLY UPDATES is the location for TTS Extract files created each Friday 07:00am via script. The current
contact is Michael Mclean.
2. Select a file, French or English file extract, from the WEEKLY UPDATES folder and move the file to either:
o the “Work in progress” folder, if you are going to start work on it, or to
o the “TTS to do” folder, if you plan to work on it later
Note: you can add your name or other entry to the end of the filename to indicate who is working on which
file. A large file for example could be broken up into smaller chunks whereby the person working on the files
adds his or her name to the end of each of the smaller files to claim the work.
3. Open the spreadsheet you plan to work on from the “Work in progress” folder.
4. Add a comment column to the right of each of the following columns, as shown in Figure 3: TTSSiteNameEnglish,
TTSAddressLineEnglish, and TTSPlaceNameEnglish. The column headers on the French file would be named as
follows: TTSSiteNameFrench, TTSAddressLineFrench, and TTSPlaceNameFrench.
Figure 3: Comments Columns
5. Select a record from the TTSSiteNameEnglish, TTSAddressLineEnglish, TTSPlaceNameEnglish columns and click
Copy. That is, click the cell, preferably starting at the top of the list, under TTSSiteNameEnglish, press the Shift
key and then click the cell in the same row under TTSPlaceNameEnglish. See Error! Reference source not
ound..
5
Figure 4: Copying a Record
6. Open your browser, Internet Explorer or Google Chrome, and Paste the copied record to the Google search
field.
7. Click the search button on the browser or press the return key on your keyboard to get results.
8. Examine the results returned to confirm that the information for the selected record is accurate.
9. If information in any of the 3 columns is inaccurate, add a comment to the appropriate Comment column.
Provide comments when necessary as follows:
o Typo: <include the corrected spelling>
o Please validate. Add this comment if you cannot confirm accuracy from the results returned.
o Research shows: < add the information that you think should be used>
o Street sign shows: <use this comment if the address does not correspond with what is on the street
sign>
o Needs civic address. Make this comment if there is no civic address or if there is a mailing address.
o Needs civic number. Make this comment if the address is missing the civic number or if a highway,
concession, country road, etc. has been entered without a civic number for an exact location.
o Needs site name. Make this comment if there is no site name. Site names are determined by the
Returning Officer. The name they provide may not match your research results and therefore should
generally be accepted, unless missing or otherwise raises much concern.
Note 1: if the address for a particular site is incorrect, for example: the current entry is “78 tenth street” but
you discovered it to be “76 tenth street” your comment should be as follows: Please validate. Research
shows: 76 tenth street. If all of the results consistently show “76 tenth street” then this comment should be
sufficient. However, if you see “76 tenth street” as well as “76 10th
street” then do some further research
using Google Maps to obtain a “street view” of the street sign. If the street sign shows “76 10th
street”, your
comment should be: Street sign shows: 76 10th
street.
Refer to Section 5 “Address Formatting” for complete details regarding addresses.
Note 2: the site names may differ from search results as the information provided is composed by the
Returning Officer (ROs) and should generally be accepted. For example: the RO has “meadowbrook club
house” but your research shows “meadowbrook villas community centre”. In this case, there would be no
need for a comment unless there is a typo or results show something completely different. Any acronyms
should also include spaces between the letters and therefore if not, should warrant a comment.
6
Refer to Section 6 “Building Name Translation Rules” for complete details regarding site names.
Also, depending on the file you are working on, French or English, make sure the information for each record
corresponds with the primary language of the electoral district. For example:
• When working with the English spreadsheet and the community is mostly French, then the
information should also be in French. If the community is English then the information should also
be in English.
• When working with the French spreadsheet and the community is mostly English, then the
information should also be in English. If the community is French then the information should also
be in French.
Figure 5, as per the examples above, indicates which language the information should be in based on
the primary language of the community or electoral district.
Figure 5: Language Map
10. Add your initials, for tracking purposes, to the column after the last Comment column, column “J”.
11. Repeat steps 5 to 10 until all records listed on the spreadsheet have been visually inspected using Google or
other preferred search engines.
12. Once you have completed a spreadsheet, separate all commented records from uncommented records.
a. Create one spreadsheet that has all of the commented records and send it in an email attachment to
the Sites team. Any comments, when Sites provides feedback, that have been rejected means those
records can be prepared for stage 2, audio tuning.
b. Create another spreadsheet that contains all uncommented records and proceed to Section 3.
2.1.1 Visual Testing Feedback and Next Steps
All comments provided by the VRS testers on the Excel spreadsheets are either accepted (fixed) or rejected by the Sites
team. Comments that have been accepted means the respective records have been fixed and therefore has to be
visually retested by being returned to the workflow for a second pass. If a comment is rejected, it means the record is
good and can be moved forward or positioned for audio tuning. Sites will add a comment to each record to indicate
whether their team accepted or rejected a comment:
o Each record that has been accepted and fixed will be identified with a “yes”.
 Accepted comments, that is, fixed records shall be retested before audio tuning.
o Each record that has been rejected will be identified with a “no”.
 Rejected comments means those records can be positioned for audio tuning.
For additional details, refer to Section 1.3.
7
3 Audio Tuning Procedures – Stage 2
3.1 Preparing excel files (French and English files) for Audio Testing (Nuance)
Once the English Excel file and the French Excel file extracts have been visually processed, all records that have not been
commented against can be copied to the Nuance Inbox for audio tuning. Each record represents a polling station
consisting of the following information:
Headers in the English file extract
- Province
- ED_code
- SiteID
- TTSSiteNameEnglish
- TTSAddressLineEnglish
- TTSPlaceNameEnglish
Headers in the French file extract
- Province
- ED_code
- SiteID
- TTSSiteNameFrench
- TTSAddressLineFrench
- TTSPlaceNameFrench
Refer to Error! Reference source not found. for an example of an English file extract.
Figure 6: Example of Commented English File Extract
Each polling station, or call it record, is associated with a unique SiteID number and can therefore be referenced or
searched accordingly. The goal now is to separate, sort, or group all commented records from the uncommented ones.
Once sorted, all commented records shall be sent to Sites (Mariann Canning’s department) for further processing
(corrections) whereas all records that have no comments can be prepared for tuning. To prepare the uncommented
records for tuning means converting them to tab delimited text. Follow these steps to prepare the uncommented
records for audio tuning:
8
1. Block select all uncommented records under columns C, D, E, F,G,H, and I to include SiteID, TTSSiteNameEnglish,
TTSAddressLineEnglish, and TTSPlaceNameEnglish. You can otherwise delete the comment columns, since there
should be no comments at this stage, and block select columns C, D, E, and F.
Note: the SiteID is the item you will have to select in order to play the associated voice segment and listen to it.
2. Copy the selected block.
3. Open Notepad
4. Right-click on Notepad and click Paste. Each record now appears in tab delimited format.
5. Save the file with the following naming convention: tts_extract_MMMDD_YYYY_Pass01_Batch01.language.txt
Figure 7 shows an example of a tab delimited file:
Figure 7: Tab Delimited File
6. Proceed to Section 3.2 to launch the audio tuning environment and to begin audio testing. Section 3.2 consists
of instructions for:
a. establishing a VPN connection to Nuance
b. loading the Nuance Inbox with the tab delimited file
c. Selecting a French or English tab delimited file to launch the Audio Tuning Environment.
3.2 Launching the Audio Tuning Environment
1. Go to Start/Programs/Cisco Systems VPN Client and Select VPN client from the dropdown. The VPN Connection
window pops up as shown in Figure 8.
Figure 8: VPN Connection Window
9
2. Click Connect. The User Authentication window pops up as shown in Figure 9.
Figure 9: VPN Login
3. Enter your Nuance provided username and password and click OK.
4. When the VPN Client is connected, the connection window closes and you will see the lock icon in your system
tray (at the bottom right of the screen).
You will stay connected as long as you are using this program, once you stop using this Program, it will stay
connected for up to 2 hours.
5. Proceed to step 6 if a short cut to the Nuance Inbox on your desktop has already been established otherwise,
map the “Nuance TTS Inbox” following the applicable process used for your operating system to map a network
drive. On Windows 7 for example, follow these sub steps:
a. Right-click Computer from the Start menu and select Map network drive. The Map Network Drive
window pops up as shown in Figure 10.
Figure 10: Mapping the Nuance Inbox
b. Select an available drive from the Drive drop-down.
10
c. Type the following path into the Folder text field: http://mt-empty.nuance.com/tts/ files/inbox
d. Click Finish and wait for the mapping to complete. The following window, Figure 11, showing the
Nuance Inbox, is displayed once the mapping is complete.
Figure 11: Nuance Inbox Mapped
e. Select the Nuance Inbox to drag and drop it onto your desktop, for quick and easy access going forward.
6. Double-click the Nuance Inbox.
7. Paste both copied TTS Extract English and French files into the Nuance Inbox folder.
8. Double-click the TTS Evaluation Tool shown in Figure 12. The “Base URL” window, shown in Figure 13, will pop
up as a result, which could take a few moments so you may have to wait a minute or two before it appears.
Figure 12: TTS Evaluation Tool
11
Figure 13: Base URL Window
9. Click Start on the Base URL pop up. The radio button should be set to “Evaluation Mode” by default.
10. In the next pop up, as shown in Figure 14, you should see both the French and the English TTS files that were
copied to the Nuance Inbox according to step 7.
Figure 14: Selecting the Desired File
11. Select either the English or French file you wish to work on and click Open. The window of the Audio Tuning
Environment is displayed as a result as shown in Figure 15. The left pane of the screen lists the SiteIDs of all the
records associated with the selected file. You are now ready to begin the audio tuning process.
12
Figure 15: Audio Tuning Environment
12. Proceed to Section 3.3 to begin the audio tuning.
3.3 Tuning Instructions
Make sure your headphones are connected to your computer before you begin tuning, that is before you select a record
(SiteID) from the left pane. Follow these instructions to proceed with the audio tuning:
1. Make sure you are wearing your headphones and enlarge the Audio Tuning Environment so that it is full screen.
2. Select a record in the left pane as shown in Figure 15. Selecting a record implies selecting its SiteID. You can see
the information associated with the selected record in the Text box so that you can observe the text while you
listen.
3. Click Play.
4. Listen carefully to the selected record’s voice segment.
5. Select the applicable radio button under the Evaluation Comments based on the following descriptions:
- OK, if the selected voice segment sounds good and is pronounced correctly.
- Bad Pronunciation, if the selected voice segment contains any word that is not pronounced correctly. Means
that a word (or words) is (are) mispronounced.
- Bad Form, if the selected voice segment contains words that run into each other without sufficient pause
between the words or if they cannot be heard. Essentially, something went wrong in the form of the output. ‘St’
might have been ‘saint’ when it should be ‘street’ or something like that. When indicating a ‘bad form’, it is
useful to describe what went wrong in the comment section.
- Other, if the selected voice segment has a typo in the displayed text field.
NOTE: the comment box for the Evaluation Comments below the radio buttons becomes available only when
one of the radio buttons is selected. The Tuning Notes section of the screen is reserved for the follow up
performed by Nuance.
6. Click Next to play the next record listed. Make sure to Save frequently.
NOTE: the font size of the SiteID listed in the left pane for a particular record is reduced when one of the 4 radio
buttons is selected. This will let you easily know which records have and have not been processed.
7. Repeat steps 3 through 6 until the end of the list.
13
8. Click Submit once all of the records listed have been processed. The Submit button is displayed only when all of
the records listed have been processed for the French or English file that was selected according to step 11 in
Section 3.2.
9. Send an email notification to Marc Fabiani so that his team can start the tuning process against the comments
submitted.
10. Repeat steps 11 and 12 in Section 3.2 to complete the audio tuning until both the English and the French files
have been tuned and submitted.
3.3.1 Audio Tuning Feedback and Next Steps
The Nuance team, after a file has been submitted, addresses all Evaluation Comments in the Tuning Notes section.
Tuning Notes descriptions and actions:
- Fixed, the audio has been tweaked and the record therefore has to be retested.
- Disagree, the audio is considered to be good or acceptable and therefore requires no further action. In this case
the audio testers simply select the OK radio button to pass the record.
- Can’t Fix, the audio will be accepted regardless of the comment and therefore requires no further action. In this
case the audio testers simply select the OK radio button to pass the record.
- Unclear, the audio testers comment is not understood by the Nuance team and therefore has to be retested
with a clearer comment.
14
4 Description of Primes and Their Responsibilities
EC Primes are available for support or to report issues encountered during TTS Extract file processing. Here’s a list of
contacts:
• EC’s VPN Client connectivity/authentication issues
• TTS Evaluation Tool/user id’s and passwords
• TTS_Extract file (questions related to content)
• Nuance TTS Evaluation tool (Voice Segments corrections and improvement, Tuning)
• Nuance email notification associated to Tuning
• TTS_Extract file creation script/code
• TTS_Extract file content, Quality Control, Polling Station data anomalies found in TTS_Extract file, EC’s SITES
database
15
5 Address Formatting
The address rules and guidelines described in this section are meant for VRS testers performing Stage 1, visual testing.
These rules and guidelines should serve as a reference for inspecting the address field of each polling station (record)
listed on the Excel spreadsheet submitted by Sites to the VRS testers each week.
5.1 General Address Rules
 The address must be a civic/street address (and not a ‘mailing’ address)
The following are mailing addresses and therefore are not acceptable:
- P.O. Boxes
- General delivery
- Rural Routes
- c.o., c/o (care of), a/s (au soin de)
 A civic/street address is made up of the following fields:
- Civic No/No municipal
- Civic Suffix
- Suite
- Street Name - Name of the Street
- Street Type (Street, avenue, Crescent, rue, chemin etc.)
- Street Direction (North, South, North-East etc.)
For example:
<civic no><civic suffix> <street name> <street type> <street direction> <suite>, for example:
- 987b bank street north, suite 1000 of an English address
- 987b, rue bank nord, suite 1000 of a French address
Note: all entries on the Excel spreadsheet, sent for VRS testing, will be in lower case and must be left that way. However,
the same entry on a Voter’s Information Card (VIC) would be in upper case with the suite number presented first, for
example: 1000 – 987B Bank Street North.
Addresses must be in the language of the electoral district regardless of the province. English and French addresses are
formatted differently (15 Bank Street vs. 15, rue Bank). VRS testers inspecting addresses when a scrape is received shall
make sure addresses are in the appropriate format and according to the language of the electoral district.
Addresses consisting of ‘Highway’, ‘Route’, ‘Concession’ or ‘Region’ as well as ‘County Road’, ‘Range Road’, ‘Township
Road’ are generally acceptable, for example: Highway 17, Route 17, etc. It would be preferable to have a civic number in
front, for example: 267 highway 17. If for example, the entry shows only “highway 17”, your comment should be “Needs
civic number”. If in your research you discover that it is 267 highway 17, your comment should then be “Please validate.
Research shows: 267 highway 17”.
16
6 Building Name Translation Rules
Returning officers are asked to provide the building name in both official languages but are not obliged to do so. The
decision to translate a name is at the discretion of the RO and is based on providing a name for a site which will help
electors find their polling station. Returning follow these guidelinesOutlined below are a few guidelines for translating
the building name for polling sites:
• Building names must be translated when there is an OFFICIAL bilingual name such as for:
o Federal buildings
 i.e. Post Office/Bureau de poste
o Universities and Colleges
• A building name should be translated when the translated name is also used locally
o In Aylmer, Québec the site “La Grange” is also known as “The Barn”.
• Building names should not be translated
o When the name is a well-known landmark
 i.e. the Four Seasons Inn will not be translated as Auberge des 4 saisons or Des 4 saisons Inn
• Building names may be easily translated for:
o Churches
o Schools
o Community Centers
• Examples of building names which are difficult to translate:
o Senior citizens residences; they may be translated as foyer, residence pour personnes âgées or pension.
• Ultimately, the final decision to translate a building name is left to the returning officer as they are considered to
be the ‘owner/expert’ of their polling site information.
17
7 Glossary
- Record: for the purposes of this document, a record also means a polling station. Each row of
information and that is each polling station can also be referred to as a record.
- Voice or speech segmentation: is the process of identifying the boundaries between words, syllables, or
phonemes in spoken natural languages. The quality of synthetic speech played back, using the Nuance
TTS tuning application, against a record is based on these boundaries.
- Scrape: a collection of records gathered from the Sites database that has been prepared and submitted
to the VRS team for testing and tuning.
- Extract: is the same as a scrape and is used in the Excel filename to identify each scrape.
- Delta: each scrape or extract consists of records taken from the Sites database that are either new or
have been modified.
Note: scrape, extract, and delta are generally interchangeable and refer to the Excel spreadsheet
deposited each week by Sites for VRS testing.

More Related Content

Viewers also liked

Cronicas angela
Cronicas angelaCronicas angela
Cronicas angelaAngelabgs
 
Dr. Stacey Schwabenlander - Minnesota State Perspective on Highly Pathogenic ...
Dr. Stacey Schwabenlander - Minnesota State Perspective on Highly Pathogenic ...Dr. Stacey Schwabenlander - Minnesota State Perspective on Highly Pathogenic ...
Dr. Stacey Schwabenlander - Minnesota State Perspective on Highly Pathogenic ...John Blue
 
FAO/WHO follow-up to recommendations of the Joint FAO/WHO Expert Meeting on H...
FAO/WHO follow-up to recommendations of the Joint FAO/WHO Expert Meeting on H...FAO/WHO follow-up to recommendations of the Joint FAO/WHO Expert Meeting on H...
FAO/WHO follow-up to recommendations of the Joint FAO/WHO Expert Meeting on H...FAO
 
Muslim Extremism Historical Overview
Muslim Extremism Historical OverviewMuslim Extremism Historical Overview
Muslim Extremism Historical OverviewGarabet Moumdjian
 

Viewers also liked (9)

Wedding Photos
Wedding Photos Wedding Photos
Wedding Photos
 
Vostro 220
Vostro 220Vostro 220
Vostro 220
 
Cronicas angela
Cronicas angelaCronicas angela
Cronicas angela
 
Notizia oggi30012017
Notizia oggi30012017Notizia oggi30012017
Notizia oggi30012017
 
ISIS vs. Nusra
ISIS vs. NusraISIS vs. Nusra
ISIS vs. Nusra
 
Dr. Stacey Schwabenlander - Minnesota State Perspective on Highly Pathogenic ...
Dr. Stacey Schwabenlander - Minnesota State Perspective on Highly Pathogenic ...Dr. Stacey Schwabenlander - Minnesota State Perspective on Highly Pathogenic ...
Dr. Stacey Schwabenlander - Minnesota State Perspective on Highly Pathogenic ...
 
Blog conade
Blog conadeBlog conade
Blog conade
 
FAO/WHO follow-up to recommendations of the Joint FAO/WHO Expert Meeting on H...
FAO/WHO follow-up to recommendations of the Joint FAO/WHO Expert Meeting on H...FAO/WHO follow-up to recommendations of the Joint FAO/WHO Expert Meeting on H...
FAO/WHO follow-up to recommendations of the Joint FAO/WHO Expert Meeting on H...
 
Muslim Extremism Historical Overview
Muslim Extremism Historical OverviewMuslim Extremism Historical Overview
Muslim Extremism Historical Overview
 

Similar to TTS_Visual_and_Audio_Tuning_Procedures_for _VRS_revision1

Telepresence Testing Approach by Shenick
Telepresence Testing Approach by ShenickTelepresence Testing Approach by Shenick
Telepresence Testing Approach by ShenickIMTC
 
Automated EMV Level 3 Testing - Abrantix Level 3 Player
Automated EMV Level 3 Testing - Abrantix Level 3 PlayerAutomated EMV Level 3 Testing - Abrantix Level 3 Player
Automated EMV Level 3 Testing - Abrantix Level 3 PlayerMartin Gloor
 
ECET 365 Exceptional Education / snaptutorial.com
ECET 365 Exceptional Education / snaptutorial.comECET 365 Exceptional Education / snaptutorial.com
ECET 365 Exceptional Education / snaptutorial.comdonaldzs109
 
Ecet 365 Education Redefined - snaptutorial.com
Ecet 365    Education Redefined - snaptutorial.comEcet 365    Education Redefined - snaptutorial.com
Ecet 365 Education Redefined - snaptutorial.comDavisMurphyC85
 
VoLTE Service Monitoring - VoLTE Voice Call
VoLTE Service Monitoring - VoLTE Voice CallVoLTE Service Monitoring - VoLTE Voice Call
VoLTE Service Monitoring - VoLTE Voice CallJose Gonzalez
 
10135 a 11
10135 a 1110135 a 11
10135 a 11Bố Su
 
Ecet 365 Enhance teaching / snaptutorial.com
Ecet 365   Enhance teaching / snaptutorial.comEcet 365   Enhance teaching / snaptutorial.com
Ecet 365 Enhance teaching / snaptutorial.comDavis116a
 
Digital Television standard part-1-6-2007
Digital Television standard part-1-6-2007Digital Television standard part-1-6-2007
Digital Television standard part-1-6-2007Ashraful Islam
 
Software Localization (L10N) Quality Assurance from the Tester's Perspective
Software Localization (L10N) Quality Assurance from the Tester's PerspectiveSoftware Localization (L10N) Quality Assurance from the Tester's Perspective
Software Localization (L10N) Quality Assurance from the Tester's PerspectiveCarola F. Berger, PhD, Dipl.-Ing., CT
 
RCS Service Monitoring - 1-to-1 Chat
RCS Service Monitoring - 1-to-1 ChatRCS Service Monitoring - 1-to-1 Chat
RCS Service Monitoring - 1-to-1 ChatJose Gonzalez
 
nc_tracking.ppt
nc_tracking.pptnc_tracking.ppt
nc_tracking.pptChemistAnn
 
Controlling interests editors
Controlling interests editorsControlling interests editors
Controlling interests editorseldhoev
 
ISTQB - Foundation Level Syllabus 2011
ISTQB - Foundation Level Syllabus 2011ISTQB - Foundation Level Syllabus 2011
ISTQB - Foundation Level Syllabus 2011Professional Testing
 
ECET 365 Success Begins /newtonhelp.com 
ECET 365 Success Begins /newtonhelp.com ECET 365 Success Begins /newtonhelp.com 
ECET 365 Success Begins /newtonhelp.com myblue134
 
IRJET - Speech to Speech Translation using Encoder Decoder Architecture
IRJET -  	  Speech to Speech Translation using Encoder Decoder ArchitectureIRJET -  	  Speech to Speech Translation using Encoder Decoder Architecture
IRJET - Speech to Speech Translation using Encoder Decoder ArchitectureIRJET Journal
 
Group 4 STS final version
Group 4 STS final versionGroup 4 STS final version
Group 4 STS final versionzenchi0
 
Tekom Netherlands | The evolving landscape of Simplified Technical English b...
Tekom Netherlands | The evolving landscape of Simplified Technical English  b...Tekom Netherlands | The evolving landscape of Simplified Technical English  b...
Tekom Netherlands | The evolving landscape of Simplified Technical English b...Shumin Chen
 

Similar to TTS_Visual_and_Audio_Tuning_Procedures_for _VRS_revision1 (20)

Telepresence Interoperability Testing
Telepresence Interoperability TestingTelepresence Interoperability Testing
Telepresence Interoperability Testing
 
Flex Test
Flex TestFlex Test
Flex Test
 
Telepresence Testing Approach by Shenick
Telepresence Testing Approach by ShenickTelepresence Testing Approach by Shenick
Telepresence Testing Approach by Shenick
 
Automated EMV Level 3 Testing - Abrantix Level 3 Player
Automated EMV Level 3 Testing - Abrantix Level 3 PlayerAutomated EMV Level 3 Testing - Abrantix Level 3 Player
Automated EMV Level 3 Testing - Abrantix Level 3 Player
 
Cobas u411
Cobas u411Cobas u411
Cobas u411
 
ECET 365 Exceptional Education / snaptutorial.com
ECET 365 Exceptional Education / snaptutorial.comECET 365 Exceptional Education / snaptutorial.com
ECET 365 Exceptional Education / snaptutorial.com
 
Ecet 365 Education Redefined - snaptutorial.com
Ecet 365    Education Redefined - snaptutorial.comEcet 365    Education Redefined - snaptutorial.com
Ecet 365 Education Redefined - snaptutorial.com
 
VoLTE Service Monitoring - VoLTE Voice Call
VoLTE Service Monitoring - VoLTE Voice CallVoLTE Service Monitoring - VoLTE Voice Call
VoLTE Service Monitoring - VoLTE Voice Call
 
10135 a 11
10135 a 1110135 a 11
10135 a 11
 
Ecet 365 Enhance teaching / snaptutorial.com
Ecet 365   Enhance teaching / snaptutorial.comEcet 365   Enhance teaching / snaptutorial.com
Ecet 365 Enhance teaching / snaptutorial.com
 
Digital Television standard part-1-6-2007
Digital Television standard part-1-6-2007Digital Television standard part-1-6-2007
Digital Television standard part-1-6-2007
 
Software Localization (L10N) Quality Assurance from the Tester's Perspective
Software Localization (L10N) Quality Assurance from the Tester's PerspectiveSoftware Localization (L10N) Quality Assurance from the Tester's Perspective
Software Localization (L10N) Quality Assurance from the Tester's Perspective
 
RCS Service Monitoring - 1-to-1 Chat
RCS Service Monitoring - 1-to-1 ChatRCS Service Monitoring - 1-to-1 Chat
RCS Service Monitoring - 1-to-1 Chat
 
nc_tracking.ppt
nc_tracking.pptnc_tracking.ppt
nc_tracking.ppt
 
Controlling interests editors
Controlling interests editorsControlling interests editors
Controlling interests editors
 
ISTQB - Foundation Level Syllabus 2011
ISTQB - Foundation Level Syllabus 2011ISTQB - Foundation Level Syllabus 2011
ISTQB - Foundation Level Syllabus 2011
 
ECET 365 Success Begins /newtonhelp.com 
ECET 365 Success Begins /newtonhelp.com ECET 365 Success Begins /newtonhelp.com 
ECET 365 Success Begins /newtonhelp.com 
 
IRJET - Speech to Speech Translation using Encoder Decoder Architecture
IRJET -  	  Speech to Speech Translation using Encoder Decoder ArchitectureIRJET -  	  Speech to Speech Translation using Encoder Decoder Architecture
IRJET - Speech to Speech Translation using Encoder Decoder Architecture
 
Group 4 STS final version
Group 4 STS final versionGroup 4 STS final version
Group 4 STS final version
 
Tekom Netherlands | The evolving landscape of Simplified Technical English b...
Tekom Netherlands | The evolving landscape of Simplified Technical English  b...Tekom Netherlands | The evolving landscape of Simplified Technical English  b...
Tekom Netherlands | The evolving landscape of Simplified Technical English b...
 

TTS_Visual_and_Audio_Tuning_Procedures_for _VRS_revision1

  • 1. i Voice Response System (VRS) Test Procedures Visual Testing and Audio Tuning for EC Polling Stations 4/9/2015 Elections Canada / Élections Canada Vietorisz, Paul
  • 2. ii Table of Contents 1 Overview .........................................................................................................................................................................1 1.1 High Level Statement of Work................................................................................................................................1 1.2 Tools and Applications............................................................................................................................................1 1.3 TTS Extract File Processing Stages...........................................................................................................................2 2 Visual Test Procedures – Stage 1 ....................................................................................................................................4 2.1 Checking for New Polling Addresses from EC’s Sites Operation’s Group...............................................................4 2.1.1 Visual Testing Feedback and Next Steps.........................................................................................................6 3 Audio Tuning Procedures – Stage 2 ................................................................................................................................7 3.1 Preparing excel files (French and English files) for Audio Testing (Nuance)...........................................................7 3.2 Launching the Audio Tuning Environment..............................................................................................................8 3.3 Tuning Instructions ...............................................................................................................................................12 3.3.1 Audio Tuning Feedback and Next Steps........................................................................................................13 4 Description of Primes and Their Responsibilities..........................................................................................................14 5 Address Formatting.......................................................................................................................................................15 5.1 General Address Rules ..........................................................................................................................................15 6 Building Name Translation Rules ..................................................................................................................................16 7 Glossary.........................................................................................................................................................................17 Table of Figures FIGURE 1 END TO END VRS PROCESSING WORKFLOW...........................................................................................................................................2 FIGURE 2: VRS POST_41 WORKING FOLDER .......................................................................................................................................................3 FIGURE 3: COMMENTS COLUMNS ......................................................................................................................................................................4 FIGURE 4: COPYING A RECORD ..........................................................................................................................................................................5 FIGURE 5: LANGUAGE MAP ..............................................................................................................................................................................6 FIGURE 6: EXAMPLE OF COMMENTED ENGLISH FILE EXTRACT..................................................................................................................................7 FIGURE 7: TAB DELIMITED FILE..........................................................................................................................................................................8 FIGURE 8: VPN CONNECTION WINDOW .............................................................................................................................................................8 FIGURE 9: VPN LOGIN.....................................................................................................................................................................................9 FIGURE 10: MAPPING THE NUANCE INBOX..........................................................................................................................................................9 FIGURE 11: NUANCE INBOX MAPPED ...............................................................................................................................................................10 FIGURE 12: TTS EVALUATION TOOL.................................................................................................................................................................10 FIGURE 13: BASE URL WINDOW.....................................................................................................................................................................11 FIGURE 14: SELECTING THE DESIRED FILE ..........................................................................................................................................................11 FIGURE 15: AUDIO TUNING ENVIRONMENT.......................................................................................................................................................12
  • 3. 1 1 Overview This section describes the scope of work, tools and applications, and provides an overview of the end-to-end test process, based on sections 2 and 3. 1.1 High Level Statement of Work • The scope of the work is to extract text files compiled of EC Polling Station data. Validate the quality of the information provided in English and French, address any anomalies and then correct them prior to transferring these text string Locations and Addresses into Voice Segments. Text Sample: St Paul’s church, 230 1st Avenue, Toronto • The text strings are then transferred into Voice Segments. Quality and validation related to speech, in English and in French occurs. The final step is to load the EC Polling Stations voice segments into the VRS. The Quality Assurance related to speech is referred to as Tuning. Voice Segment sample: St Paul’s church, 230 1st Avenue, Toronto In addition, the work process requires tracking and logging various work stages. 1.2 Tools and Applications Stage 1 – tools and applications for visual testing: • Excel spreadsheet, provided by Sites, listing all of the polling stations (records) to be tested • Internet Explorer, Google Chrome, or Firefox Any of these browsers can be used to do the research against each record listed on the Excel spreadsheet Note: IE can be used without any approval. Approval, however, will be required if you wish to use one of the other browsers. Stage 2 – tools and applications for audio tuning: • TTS Evaluation Tool This web interface provided by Nuance is used for tuning Text to Speech (TTS) files. Tuning is the process of listening to voice recordings based on text strings (associated with each polling station or record) compiled of EC’s polling stations. The Nuance back end translates text to speech while the TTS Evaluation tool provides the capability or means to listen and improve speech. The voice recordings are available in both official languages. Tuning is the second stage of TTS file manipulation when considering the end to end work flow. • Cisco VPN Client Note: The Cisco VPN Client must be installed on the EC PC workstation as a prerequisite. This software is not part of EC’s computer base image. o The VPN Client is used to establish a secure connection channel between an EC IT Telecom workstation and Nuance’s Tuning server. o This connection insures access to Nuance’s shared folder where TTS files ready for tuning are saved. o This connection also insures successful web connection to Nuance’s TTS Evaluation Tool. o A Nuance provided username and password is required to establish a successful connection via EC’s VPN Client. Here are the paths to the “VPN Client install script” and to the “VPN Client Installation video”: H:Projects162commVRS -TelephonyTSGTelecom Analyst Training Material 2012TTSVPN Client install script H:Projects162commVRS -TelephonyTSGTelecom Analyst Training Material 2012TTSVPN Client Installation video
  • 4. 2 1.3 TTS Extract File Processing Stages There are 2 test stages involved in the end to end process of testing a record: - Stage 1: Visual testing, refer to Section 2 for step by step instructions - Stage 2: Audio testing, refer to Section 3, for step by step instructions Beginning with the visual testing in stage 1, VRS testers receive an Excel spreadsheet, referred to as a scrape or extract, that lists all of the polling stations that need to be tested. Some polling stations may be new while others may have been modified but in either case all of the polling stations listed will need to be inspected (tested) before Sites can put them into production, back into the Sites database. Each scrape (also called extract or delta) is represented as a “delta” as shown in Figure 1. VRS testers examine the information associated with each polling station to determine accuracy. Some may or may not warrant a comment. Polling stations or “records” from here on, that have no comment means they passed the visual inspection and can therefore be pushed forward to stage 2, audio testing. However, all of the commented records that did not pass visual inspection are returned to the Sites team for corrections. The Sites team examines the comments and decides whether to accept or reject the comments. They will indicate on the spreadsheet that is returned to the VRS testers, which were accepted and which were rejected. They may use “Yeses” or “noes”: - Yes to indicate that the comment was accepted and that the record was updated and “fixed” - No to indicate that the comment was rejected and that the record was left as is and therefore “not fixed” Sites returns this feedback, that is the same spreadsheet marked with “yeses” and “noes”, to the VRS testers who can then push records that are “Not Fixed” directly to stage 2 for audio tuning. “Fixed” records however would have to be visually retested and therefore may or may not be commented against. This is repeated until all records pass visual. Figure 1 End to End VRS Processing Workflow
  • 5. 3 Records in either stage 1 or stage 2 must be retested until they are deemed good. Excel file extracts prepared by Sites, referred to as the Delta.xls in Figure 1, for the VRS team are dropped into the “WEEKLY UPDATES” folder every Friday on a weekly basis in the following path: H:Projects162commDatabaseTTS Data Extract From Sitespost_41 The “post_41” folder is the VRS team’s working folder that contains all files for visual and audio testing as shown in Error! Reference source not found.. Figure 2: VRS post_41 Working Folder Content in the working folder: - 00 - WEEKLY UPDATES: contains files sent from Sites for VRS testing. These files are also known as “deltas” or “scrapes”. The files will be named as follows, tts_extract_<month and day>_<year>_<language>.xls, for example, tts_extract_Apr7_2015_english.xls. A VRS tester can also break up a file if necessary into smaller manageable chunks, for example: tts_extract_Apr7_2015_english_01, tts_extract_Apr7_2015_english_02, etc. A VRS tester’s name can also be added to track who is working on which file, for example: tts_extract_Apr7_2015_english_01_John, tts_extract_Apr7_2015_english_02_Tom. NOTE: TTS files are deposited every Friday. A TTS file that is empty can be deleted. It may occur that a TTS file deposited on any given Friday may be empty of content; this is a result of the script running as scheduled but has found no new Polling Station data to process. - 01 - TTS to do: files from the WEEKLY UPDATES are moved to this folder for assignment and optional chunking when necessary. The name of a specific tester can be added to the filename to identify who will be working on it. A large file can also be broken up into smaller manageable chunks when necessary or practical. For example, you can take a single file with say 3,000 records and split it up into 10 files each consisting of 300 records. - 02 - Work in progress: VRS testers choose a file from the “TTS to do” folder and move it to this folder when they are ready to start testing. - 03 - Extracts sent to SITES_visual_validation_Done: testers that have completed their visual testing of a file can then move the file to this folder to indicate that it has been completed and can be sent to sites. - 04 – Backup: all files sent to Sites after they have been visually tested are backed up to this folder. - 05 – Misc: used as necessary. - 06 - Nuance Reports: will contain information to inform Sites of what can or cannot be accepted or fixed. - TTS Progress Tracking Sheet: files in folder “02- Work in progress” are tracked by this file for the VRS tester’s purposes and therefore any file a tester chooses to work on must open it from folder 02!
  • 6. 4 2 Visual Test Procedures – Stage 1 2.1 Checking for New Polling Addresses from EC’s Sites Operation’s Group Follow these instructions to visually inspect the information contained on the French and English Excel spreadsheets before moving onto the audio testing stage covered in Section 3: 1. Go to the H drive and access: H:Projects162commDatabaseTTS Data Extract From Sitespost_4100 - WEEKLY UPDATES Tip: you can also create a shortcut to the “post_41” folder for quick access. Follow instructions according to your operating system to map a shortcut. Note: WEEKLY UPDATES is the location for TTS Extract files created each Friday 07:00am via script. The current contact is Michael Mclean. 2. Select a file, French or English file extract, from the WEEKLY UPDATES folder and move the file to either: o the “Work in progress” folder, if you are going to start work on it, or to o the “TTS to do” folder, if you plan to work on it later Note: you can add your name or other entry to the end of the filename to indicate who is working on which file. A large file for example could be broken up into smaller chunks whereby the person working on the files adds his or her name to the end of each of the smaller files to claim the work. 3. Open the spreadsheet you plan to work on from the “Work in progress” folder. 4. Add a comment column to the right of each of the following columns, as shown in Figure 3: TTSSiteNameEnglish, TTSAddressLineEnglish, and TTSPlaceNameEnglish. The column headers on the French file would be named as follows: TTSSiteNameFrench, TTSAddressLineFrench, and TTSPlaceNameFrench. Figure 3: Comments Columns 5. Select a record from the TTSSiteNameEnglish, TTSAddressLineEnglish, TTSPlaceNameEnglish columns and click Copy. That is, click the cell, preferably starting at the top of the list, under TTSSiteNameEnglish, press the Shift key and then click the cell in the same row under TTSPlaceNameEnglish. See Error! Reference source not ound..
  • 7. 5 Figure 4: Copying a Record 6. Open your browser, Internet Explorer or Google Chrome, and Paste the copied record to the Google search field. 7. Click the search button on the browser or press the return key on your keyboard to get results. 8. Examine the results returned to confirm that the information for the selected record is accurate. 9. If information in any of the 3 columns is inaccurate, add a comment to the appropriate Comment column. Provide comments when necessary as follows: o Typo: <include the corrected spelling> o Please validate. Add this comment if you cannot confirm accuracy from the results returned. o Research shows: < add the information that you think should be used> o Street sign shows: <use this comment if the address does not correspond with what is on the street sign> o Needs civic address. Make this comment if there is no civic address or if there is a mailing address. o Needs civic number. Make this comment if the address is missing the civic number or if a highway, concession, country road, etc. has been entered without a civic number for an exact location. o Needs site name. Make this comment if there is no site name. Site names are determined by the Returning Officer. The name they provide may not match your research results and therefore should generally be accepted, unless missing or otherwise raises much concern. Note 1: if the address for a particular site is incorrect, for example: the current entry is “78 tenth street” but you discovered it to be “76 tenth street” your comment should be as follows: Please validate. Research shows: 76 tenth street. If all of the results consistently show “76 tenth street” then this comment should be sufficient. However, if you see “76 tenth street” as well as “76 10th street” then do some further research using Google Maps to obtain a “street view” of the street sign. If the street sign shows “76 10th street”, your comment should be: Street sign shows: 76 10th street. Refer to Section 5 “Address Formatting” for complete details regarding addresses. Note 2: the site names may differ from search results as the information provided is composed by the Returning Officer (ROs) and should generally be accepted. For example: the RO has “meadowbrook club house” but your research shows “meadowbrook villas community centre”. In this case, there would be no need for a comment unless there is a typo or results show something completely different. Any acronyms should also include spaces between the letters and therefore if not, should warrant a comment.
  • 8. 6 Refer to Section 6 “Building Name Translation Rules” for complete details regarding site names. Also, depending on the file you are working on, French or English, make sure the information for each record corresponds with the primary language of the electoral district. For example: • When working with the English spreadsheet and the community is mostly French, then the information should also be in French. If the community is English then the information should also be in English. • When working with the French spreadsheet and the community is mostly English, then the information should also be in English. If the community is French then the information should also be in French. Figure 5, as per the examples above, indicates which language the information should be in based on the primary language of the community or electoral district. Figure 5: Language Map 10. Add your initials, for tracking purposes, to the column after the last Comment column, column “J”. 11. Repeat steps 5 to 10 until all records listed on the spreadsheet have been visually inspected using Google or other preferred search engines. 12. Once you have completed a spreadsheet, separate all commented records from uncommented records. a. Create one spreadsheet that has all of the commented records and send it in an email attachment to the Sites team. Any comments, when Sites provides feedback, that have been rejected means those records can be prepared for stage 2, audio tuning. b. Create another spreadsheet that contains all uncommented records and proceed to Section 3. 2.1.1 Visual Testing Feedback and Next Steps All comments provided by the VRS testers on the Excel spreadsheets are either accepted (fixed) or rejected by the Sites team. Comments that have been accepted means the respective records have been fixed and therefore has to be visually retested by being returned to the workflow for a second pass. If a comment is rejected, it means the record is good and can be moved forward or positioned for audio tuning. Sites will add a comment to each record to indicate whether their team accepted or rejected a comment: o Each record that has been accepted and fixed will be identified with a “yes”.  Accepted comments, that is, fixed records shall be retested before audio tuning. o Each record that has been rejected will be identified with a “no”.  Rejected comments means those records can be positioned for audio tuning. For additional details, refer to Section 1.3.
  • 9. 7 3 Audio Tuning Procedures – Stage 2 3.1 Preparing excel files (French and English files) for Audio Testing (Nuance) Once the English Excel file and the French Excel file extracts have been visually processed, all records that have not been commented against can be copied to the Nuance Inbox for audio tuning. Each record represents a polling station consisting of the following information: Headers in the English file extract - Province - ED_code - SiteID - TTSSiteNameEnglish - TTSAddressLineEnglish - TTSPlaceNameEnglish Headers in the French file extract - Province - ED_code - SiteID - TTSSiteNameFrench - TTSAddressLineFrench - TTSPlaceNameFrench Refer to Error! Reference source not found. for an example of an English file extract. Figure 6: Example of Commented English File Extract Each polling station, or call it record, is associated with a unique SiteID number and can therefore be referenced or searched accordingly. The goal now is to separate, sort, or group all commented records from the uncommented ones. Once sorted, all commented records shall be sent to Sites (Mariann Canning’s department) for further processing (corrections) whereas all records that have no comments can be prepared for tuning. To prepare the uncommented records for tuning means converting them to tab delimited text. Follow these steps to prepare the uncommented records for audio tuning:
  • 10. 8 1. Block select all uncommented records under columns C, D, E, F,G,H, and I to include SiteID, TTSSiteNameEnglish, TTSAddressLineEnglish, and TTSPlaceNameEnglish. You can otherwise delete the comment columns, since there should be no comments at this stage, and block select columns C, D, E, and F. Note: the SiteID is the item you will have to select in order to play the associated voice segment and listen to it. 2. Copy the selected block. 3. Open Notepad 4. Right-click on Notepad and click Paste. Each record now appears in tab delimited format. 5. Save the file with the following naming convention: tts_extract_MMMDD_YYYY_Pass01_Batch01.language.txt Figure 7 shows an example of a tab delimited file: Figure 7: Tab Delimited File 6. Proceed to Section 3.2 to launch the audio tuning environment and to begin audio testing. Section 3.2 consists of instructions for: a. establishing a VPN connection to Nuance b. loading the Nuance Inbox with the tab delimited file c. Selecting a French or English tab delimited file to launch the Audio Tuning Environment. 3.2 Launching the Audio Tuning Environment 1. Go to Start/Programs/Cisco Systems VPN Client and Select VPN client from the dropdown. The VPN Connection window pops up as shown in Figure 8. Figure 8: VPN Connection Window
  • 11. 9 2. Click Connect. The User Authentication window pops up as shown in Figure 9. Figure 9: VPN Login 3. Enter your Nuance provided username and password and click OK. 4. When the VPN Client is connected, the connection window closes and you will see the lock icon in your system tray (at the bottom right of the screen). You will stay connected as long as you are using this program, once you stop using this Program, it will stay connected for up to 2 hours. 5. Proceed to step 6 if a short cut to the Nuance Inbox on your desktop has already been established otherwise, map the “Nuance TTS Inbox” following the applicable process used for your operating system to map a network drive. On Windows 7 for example, follow these sub steps: a. Right-click Computer from the Start menu and select Map network drive. The Map Network Drive window pops up as shown in Figure 10. Figure 10: Mapping the Nuance Inbox b. Select an available drive from the Drive drop-down.
  • 12. 10 c. Type the following path into the Folder text field: http://mt-empty.nuance.com/tts/ files/inbox d. Click Finish and wait for the mapping to complete. The following window, Figure 11, showing the Nuance Inbox, is displayed once the mapping is complete. Figure 11: Nuance Inbox Mapped e. Select the Nuance Inbox to drag and drop it onto your desktop, for quick and easy access going forward. 6. Double-click the Nuance Inbox. 7. Paste both copied TTS Extract English and French files into the Nuance Inbox folder. 8. Double-click the TTS Evaluation Tool shown in Figure 12. The “Base URL” window, shown in Figure 13, will pop up as a result, which could take a few moments so you may have to wait a minute or two before it appears. Figure 12: TTS Evaluation Tool
  • 13. 11 Figure 13: Base URL Window 9. Click Start on the Base URL pop up. The radio button should be set to “Evaluation Mode” by default. 10. In the next pop up, as shown in Figure 14, you should see both the French and the English TTS files that were copied to the Nuance Inbox according to step 7. Figure 14: Selecting the Desired File 11. Select either the English or French file you wish to work on and click Open. The window of the Audio Tuning Environment is displayed as a result as shown in Figure 15. The left pane of the screen lists the SiteIDs of all the records associated with the selected file. You are now ready to begin the audio tuning process.
  • 14. 12 Figure 15: Audio Tuning Environment 12. Proceed to Section 3.3 to begin the audio tuning. 3.3 Tuning Instructions Make sure your headphones are connected to your computer before you begin tuning, that is before you select a record (SiteID) from the left pane. Follow these instructions to proceed with the audio tuning: 1. Make sure you are wearing your headphones and enlarge the Audio Tuning Environment so that it is full screen. 2. Select a record in the left pane as shown in Figure 15. Selecting a record implies selecting its SiteID. You can see the information associated with the selected record in the Text box so that you can observe the text while you listen. 3. Click Play. 4. Listen carefully to the selected record’s voice segment. 5. Select the applicable radio button under the Evaluation Comments based on the following descriptions: - OK, if the selected voice segment sounds good and is pronounced correctly. - Bad Pronunciation, if the selected voice segment contains any word that is not pronounced correctly. Means that a word (or words) is (are) mispronounced. - Bad Form, if the selected voice segment contains words that run into each other without sufficient pause between the words or if they cannot be heard. Essentially, something went wrong in the form of the output. ‘St’ might have been ‘saint’ when it should be ‘street’ or something like that. When indicating a ‘bad form’, it is useful to describe what went wrong in the comment section. - Other, if the selected voice segment has a typo in the displayed text field. NOTE: the comment box for the Evaluation Comments below the radio buttons becomes available only when one of the radio buttons is selected. The Tuning Notes section of the screen is reserved for the follow up performed by Nuance. 6. Click Next to play the next record listed. Make sure to Save frequently. NOTE: the font size of the SiteID listed in the left pane for a particular record is reduced when one of the 4 radio buttons is selected. This will let you easily know which records have and have not been processed. 7. Repeat steps 3 through 6 until the end of the list.
  • 15. 13 8. Click Submit once all of the records listed have been processed. The Submit button is displayed only when all of the records listed have been processed for the French or English file that was selected according to step 11 in Section 3.2. 9. Send an email notification to Marc Fabiani so that his team can start the tuning process against the comments submitted. 10. Repeat steps 11 and 12 in Section 3.2 to complete the audio tuning until both the English and the French files have been tuned and submitted. 3.3.1 Audio Tuning Feedback and Next Steps The Nuance team, after a file has been submitted, addresses all Evaluation Comments in the Tuning Notes section. Tuning Notes descriptions and actions: - Fixed, the audio has been tweaked and the record therefore has to be retested. - Disagree, the audio is considered to be good or acceptable and therefore requires no further action. In this case the audio testers simply select the OK radio button to pass the record. - Can’t Fix, the audio will be accepted regardless of the comment and therefore requires no further action. In this case the audio testers simply select the OK radio button to pass the record. - Unclear, the audio testers comment is not understood by the Nuance team and therefore has to be retested with a clearer comment.
  • 16. 14 4 Description of Primes and Their Responsibilities EC Primes are available for support or to report issues encountered during TTS Extract file processing. Here’s a list of contacts: • EC’s VPN Client connectivity/authentication issues • TTS Evaluation Tool/user id’s and passwords • TTS_Extract file (questions related to content) • Nuance TTS Evaluation tool (Voice Segments corrections and improvement, Tuning) • Nuance email notification associated to Tuning • TTS_Extract file creation script/code • TTS_Extract file content, Quality Control, Polling Station data anomalies found in TTS_Extract file, EC’s SITES database
  • 17. 15 5 Address Formatting The address rules and guidelines described in this section are meant for VRS testers performing Stage 1, visual testing. These rules and guidelines should serve as a reference for inspecting the address field of each polling station (record) listed on the Excel spreadsheet submitted by Sites to the VRS testers each week. 5.1 General Address Rules  The address must be a civic/street address (and not a ‘mailing’ address) The following are mailing addresses and therefore are not acceptable: - P.O. Boxes - General delivery - Rural Routes - c.o., c/o (care of), a/s (au soin de)  A civic/street address is made up of the following fields: - Civic No/No municipal - Civic Suffix - Suite - Street Name - Name of the Street - Street Type (Street, avenue, Crescent, rue, chemin etc.) - Street Direction (North, South, North-East etc.) For example: <civic no><civic suffix> <street name> <street type> <street direction> <suite>, for example: - 987b bank street north, suite 1000 of an English address - 987b, rue bank nord, suite 1000 of a French address Note: all entries on the Excel spreadsheet, sent for VRS testing, will be in lower case and must be left that way. However, the same entry on a Voter’s Information Card (VIC) would be in upper case with the suite number presented first, for example: 1000 – 987B Bank Street North. Addresses must be in the language of the electoral district regardless of the province. English and French addresses are formatted differently (15 Bank Street vs. 15, rue Bank). VRS testers inspecting addresses when a scrape is received shall make sure addresses are in the appropriate format and according to the language of the electoral district. Addresses consisting of ‘Highway’, ‘Route’, ‘Concession’ or ‘Region’ as well as ‘County Road’, ‘Range Road’, ‘Township Road’ are generally acceptable, for example: Highway 17, Route 17, etc. It would be preferable to have a civic number in front, for example: 267 highway 17. If for example, the entry shows only “highway 17”, your comment should be “Needs civic number”. If in your research you discover that it is 267 highway 17, your comment should then be “Please validate. Research shows: 267 highway 17”.
  • 18. 16 6 Building Name Translation Rules Returning officers are asked to provide the building name in both official languages but are not obliged to do so. The decision to translate a name is at the discretion of the RO and is based on providing a name for a site which will help electors find their polling station. Returning follow these guidelinesOutlined below are a few guidelines for translating the building name for polling sites: • Building names must be translated when there is an OFFICIAL bilingual name such as for: o Federal buildings  i.e. Post Office/Bureau de poste o Universities and Colleges • A building name should be translated when the translated name is also used locally o In Aylmer, Québec the site “La Grange” is also known as “The Barn”. • Building names should not be translated o When the name is a well-known landmark  i.e. the Four Seasons Inn will not be translated as Auberge des 4 saisons or Des 4 saisons Inn • Building names may be easily translated for: o Churches o Schools o Community Centers • Examples of building names which are difficult to translate: o Senior citizens residences; they may be translated as foyer, residence pour personnes âgées or pension. • Ultimately, the final decision to translate a building name is left to the returning officer as they are considered to be the ‘owner/expert’ of their polling site information.
  • 19. 17 7 Glossary - Record: for the purposes of this document, a record also means a polling station. Each row of information and that is each polling station can also be referred to as a record. - Voice or speech segmentation: is the process of identifying the boundaries between words, syllables, or phonemes in spoken natural languages. The quality of synthetic speech played back, using the Nuance TTS tuning application, against a record is based on these boundaries. - Scrape: a collection of records gathered from the Sites database that has been prepared and submitted to the VRS team for testing and tuning. - Extract: is the same as a scrape and is used in the Excel filename to identify each scrape. - Delta: each scrape or extract consists of records taken from the Sites database that are either new or have been modified. Note: scrape, extract, and delta are generally interchangeable and refer to the Excel spreadsheet deposited each week by Sites for VRS testing.