The document discusses the challenges of accurately transcribing candidate names using speech-to-text software and proposes a CNN-based name recognition model to address this issue. The approach includes extracting known name instances, training the model on these instances, and improving the dataset with audio data for uncommon names. Future steps involve testing the model's accuracy on specific candidates and generating additional data to enhance performance.

1. Candidate Name Recognition
Roshaan Siddiqui and Oliver Diamond

2. The Task
● Speech-to-text software fails to correctly transcribe
candidate names
● Can names be identified directly from audio files?

3. Main Challenges
● Instances originating from different speakers
● Lack of data for uncommon names

4. Method
1. Extract and label known instances of names from dataset
2. Build CNN-based name recognition model
3. Identify phonetically similar instances from transcription
4. Run model on the identified similar instances

5. Training a Convolutional Neural Network (CNN)
Trained
Model
Learns Features
Converts to Spectrogram
Labeled Data (E.g. Biden)

6. Phonetically Similar Instance
Trained
Model
Use-Case
Trump/Biden/Unknown

7. Next Steps
1. Build CNN-based name recognition model to work on Joe Biden
and Donald Trump and test accuracy of the model
2. Build new instances of data by adding background noise to
pre-existing name instances and use Google Text To Speech to
create more instances.
3. Gather audio data for uncommon candidate names and retrain the
CNN-based name recognition model.

8. Questions?

9. Additional Information

10. Metaphone Algorithm + Text Similarly Algorithm
Correct Candidate Spelling Encoding
100%
100%
85%
Direct Match
Direct Match
Possible Match

11. How is our data structured
Donald
Trump
Training Data

12. Phonetically Similar Matches