The Musicologist is a proof of concept of a Conversational Search Interface, exploring what can be done with NLP and Search APIs. This is a talk presented at Paris NLP Meetup, describing the three components behind the Musicologist and what's the value of these new hybrid interfaces. You can find a recording here: https://www.youtube.com/watch?v=3eb4KMNcOII

1. Paul-Louis Nech
Search as a Conversation
R&D Engineer
Public
There is more to search than a searchbox
/in/plnech
@PaulLouisNech
No Dribbble
PLNech

2. Search as a Conversation
Introduction: a dying searchbar
Paul-Louis Nech
Search as a Conversation

3. Search as a Conversation
Paul-Louis Nech

4. Search as a Conversation
> A mobile search interface 📱
> A NLP layer 🤖
> A back-end powered by Algolia ⚡
How does it work?
Paul-Louis Nech
Search as a Conversation

5. 🤖
The NLP layer
Search as a Conversation
Paul-Louis Nech

6. Search as a Conversation
> Built with DialogFlow
> Recognize 2 kinds of intents: Search and Play
entities: Themes and Artists
> Turn a sentence into an intent+parameters 👉
> Turn the search results into a response 👈
🤖 The NLP layer
Paul-Louis Nech
Search as a Conversation

7. > But not enough control over entity recognition
Search as a Conversation
> System-provided or developer-defined
> Automated expansion feature to generalize
🤖 Entity recognition: in DialogFlow
Paul-Louis Nech
Search as a Conversation

8. Search as a Conversation
🤖 Entity recognition: current limitations
> Typo-tolerance
> Pluralisation
> Too precise queries
> Advanced synonyms
Paul-Louis Nech
Search as a Conversation

9. Search as a Conversation
🤖 Entity recognition: handling typos
Paul-Louis Nech
Search as a Conversation

10. Search as a Conversation
🤖 Entity recognition: handling typos
Paul-Louis Nech
Search as a Conversation

11. Search as a Conversation
🤖 Entity recognition: pluralisation
Paul-Louis Nech
Search as a Conversation

12. Search as a Conversation
🤖 Entity recognition: too precise queries
Paul-Louis Nech
Search as a Conversation

13. Search as a Conversation
🤖 Entity recognition: other settings
Paul-Louis Nech
Search as a Conversation

14. 🏭
The back-end
Search as a Conversation
Paul-Louis Nech

15. Search as a Conversation
> The conversational agent’s “relevance engine”
> Uses Algolia to interpret the parameters
and fulfil the intent with relevant content
> Built in Node.js using HAPI
> Hosted on Heroku
🏭 The back-end
Paul-Louis Nech
Search as a Conversation

16. Search as a Conversation
🏭 The back-end: processing parameters
if (req.payload.result.parameters['artistName']) {
artist = req.payload.result.parameters['artistName'];
}
Get parameters from DialogFlow
if (artist !== '') {
searchOptions['restrictSearchableAttributes'] = ['artistName'];
searchQuery = artist;
}
Adapt search options based on the parameters
Paul-Louis Nech
Search as a Conversation

17. Search as a Conversation
🏭 The back-end: formatting search results
if (artistNames.length === 1) {
if (artistIsFoundExact) { // We found the expected artist
this.parameters["artist"] = artistNames[0];
} else { // We found another artist
this.parameters["artistOther"] = artistNames[0];
}
} else { // We found several artists
this.parameters["artistNames"] = artistNames;
}
Derive parameters for the NLP layer’s response
Paul-Louis Nech
Search as a Conversation

18. 📲
The front-end
Search as a Conversation
Paul-Louis Nech

19. Search as a Conversation
> The actual search interface
> Android app, coded in Kotlin
> Uses SpeechRecognizer to process your voice
> Uses TextToSpeech to turn the response into a voice
> Uses InstantSearch Android to display the results
📲 The front-end
Paul-Louis Nech
Search as a Conversation

20. Search as a Conversation
📲 The front-end
textToSpeech.speak(EmojiParser.removeAllEmojis(speech),
TextToSpeech.QUEUE_FLUSH, null, null)
Snackbar.make(button, text ?: speech, duration).show()
Voice and display response with TextToSpeech and SnackBar
aiButton.initialize(AIConfiguration("API_KEY",
ai.api.AIConfiguration.SupportedLanguages.English,
AIConfiguration.RecognitionEngine.System))
Initialise speech recognition
Paul-Louis Nech
Search as a Conversation

21. 💥
The result
Search as a Conversation
Paul-Louis Nech

22. > Many use cases: when do you need your hands
but would enjoy a screen?
Search as a Conversation
> A conversational search interface
> Hands-free interaction and UI adapted for discovery
💥 The result
Paul-Louis Nech
Search as a Conversation

23. Thank you
Paul-Louis Nech
R&D Engineer
/in/plnech
@PaulLouisNech
No Dribbble
PLNech
fork me at github.com/PLNech/Musicologist
read more at alg.li/blog-musicologist