The document discusses adding voice capability to the Jenkins automation server. It describes how humans have long wanted to talk to machines and provides examples. To enable voice control of Jenkins, it would need a programming interface and voice interface using natural language processing. A Convo library is proposed to specify conversations through YAML and handle voice/text input and API calls. Challenges include optimizing for language and training, and designing conversational APIs for humans rather than machines. Contributions to add this capability through the Convo-Jenkins project are welcomed.

1. Having A Talk With Jenkins
Cliffano Subagio - Shine Solutions

2. Cliffano Subagio
@cliffano
Shine Solutions, AustraliaJenkins
(nee Hudson)
User ~2007
Contributor ~2008

3. © 2018 All Rights Reserved. !3
Let’s talk about adding voice capability to Jenkins.

4. © 2018 All Rights Reserved. !4
Humans have always wanted to talk to machines.

5. © 2018 All Rights Reserved. !5
Jean-Luc Picard talking to a spacecraft

6. © 2018 All Rights Reserved. !6
Michael Knight talking to a car

7. © 2018 All Rights Reserved. !7
Jìan-Yáng talking to a fridge

8. © 2018 All Rights Reserved. !8
I have always wanted to talk to Jenkins.

9. © 2018 All Rights Reserved.
Me talking to Jenkins in 2012
!9

10. © 2018 All Rights Reserved. !10
Talking to Jenkins on Firefox via a MacBook
• Speakable Items
• Maps voice input to a command
• plist file format
• No training capability

11. © 2018 All Rights Reserved.
Can we do better in 2018?
!11

12. © 2018 All Rights Reserved. !12
Yes, but first we need a programming interface.

13. © 2018 All Rights Reserved. !13
Jenkins World 2017

14. © 2018 All Rights Reserved. !14
Swaggy Jenkins
• OpenAPI specification
• Covers Jenkins Remote Access API and Blue Ocean API
• Generates API clients and libraries in 70+ technology stacks
• Goal: to broaden out Jenkins integration points

15. © 2018 All Rights Reserved. !15
Next, we need a voice interface.

16. © 2018 All Rights Reserved. !16
Architecture
Natural
language
processing
Voice
(and text)
input
Command
processing
and API calls

17. © 2018 All Rights Reserved.
Me talking to Jenkins in 2018
!17

18. © 2018 All Rights Reserved. !18
Data flow
“Build job ABC”
“Job ABC has been
added to the queue”
{
“queryText”: “Build job ABC”,
“parameters”: {
“jobName”: “ABC”
},
“intent”: {
“displayName”: “build-job”
}
}
POST /job/ABC/build
200 OK
{
“fulfllmentText”: “Job ABC has been added to the queue”,
“payload”: { “google”: { “richResponse”: { “items”: [
“simpleResponse”: {
“textToSpeech”: “Job ABC has been added to the queue”,
“displayText”: “Job ABC has been added to the queue”,
}]}}}
}
{
“inputs": [
{
"intent": "actions.intent.TEXT",
"rawInputs": [
{
"inputType": "VOICE",
"query": “Build job ABC”
}
]}]}
"expectedInputs": [
{
"inputPrompt": {
"richInitialPrompt": {
"items": [ { "simpleResponse": {
"textToSpeech": “Job ABC has been added to the queue"
}
}]}}}]}

19. © 2018 All Rights Reserved. !19
Convo
• node.js library for voice and text conversation
• Specification based
• Generates a Dialogflow agent
• Generates an OpenAPI CloudFunctions middleware
• Maps voice input to OpenAPI call
• Utilises Dialogflow support for multiple languages, devices, and messaging apps
• https://github.com/cliffano/convo

20. © 2018 All Rights Reserved. !20
Conversation specification in YAML?

21. © 2018 All Rights Reserved. !21
Convo Spec - Greetings and unknowns

22. © 2018 All Rights Reserved. !22
Convo Spec - Queries

23. © 2018 All Rights Reserved. !23
Convo Spec - Queries with parameters

24. © 2018 All Rights Reserved. !24
Jenkins can be multilingual.

25. © 2018 All Rights Reserved.
Talking to Jenkins in Japanese
!25

26. © 2018 All Rights Reserved.
Talking to Jenkins in German
!26

27. © 2018 All Rights Reserved.
Talking to Jenkins in Indonesian
!27

28. © 2018 All Rights Reserved. !28
Lessons learned.

29. © 2018 All Rights Reserved. !29
Job names
“Release the Kraken”
“Grill Easter Cracking”
is not

30. © 2018 All Rights Reserved. !30
Project names

31. © 2018 All Rights Reserved. !31
More lessons…
• Optimise for language
• Training, training, training - more phrases
• Need better error handling than “Not found"
• Potential for Convo facts and non-OpenAPI backend

32. © 2018 All Rights Reserved. !32
Most important lesson?Web API is often designed for machines.Conversation API must be designed for humans.

33. © 2018 All Rights Reserved. !33
“Now I can be the weird guy in the corner talking to his Jenkins server.”It might be weird now. But it might be common in the future.

34. © 2018 All Rights Reserved. !34
Contributions are welcome!
Speak Klingon?
Join the effort
http://github.com/cliffano/convo-jenkins

35. © 2018 All Rights Reserved. !35
Questions?

36. Thank you

37. © 2018 All Rights Reserved. !37
Links
• https://github.com/cliffano/convo-jenkins
• https://github.com/cliffano/convo-jenkins-helper
• https://github.com/cliffano/convo

38. © 2018 All Rights Reserved. !38
Credits
• http://s183.photobucket.com/user/TrueX-Ray/media/
Movies%20Caption%20Contest/Movies31a.jpg.html
• http://www.ataricompendium.com/game_library/atari_tube/tube_23.jpg
• https://hologramradio.org/images/The-Impromptu-A-Springboard-Rewrite.jpg
• https://www.youtube.com/watch?v=K3pYZwol6Dc
• https://pics.me.me/you-can-do-better-than-that-memes-com-17633693.png
• https://www.dailydot.com/unclick/release-the-kraken-meme/
• http://www.startrek.com/article/poll-says-series-you-d-want-to-join-is
• https://wiki.jenkins.io/display/JENKINS/Logo