With recent innovations in technology, AI has left the realms of science fiction and is becoming increasingly mainstream. Whether you’re interested in building games or apps, a sound understanding of the practical applications of AI will give you the edge needed to stand out from the crowd and will set you up for success. Through this workshop, you’ll gain hands-on experience in adding AI elements into games and apps built with Unity. Using Microsoft Cognitive Services, a cloud-based AI platform, you’ll get exposure to AI features including computer vision and language recognition. Assets and starting files will be provided for you to work with during the practical components of this 2-hour workshop. This workshop will be taught by Pablo Farias Navarro, founder of Zenva - a leading education platform to learn coding and game making. Since 2012, Pablo has been developing professional-grade courses that have helped over 1 million learners worldwide.

1. Intro to Cloud AI with
Azure and Unity
Pablo Farias Navarro
Zenva founder

2. Project Files
https://gamedevacademy.org/intro-to-cloud-ai-with-microsoft-azure-cognitive-services/

3. Workshop outcomes
What is AI and
Microsoft Azure
Cognitive Services
Using cloud AI in Unity
apps & games
Implementation
details

4. Why?
Impact in daily life Opportunities In-demand skills

6. Roadmap
1. ✅ Introduction
2. ➡ What is AI
3. Microsoft Azure Cognitive Services
4. Unity demos
5. Unity basics
6. Unity in-depth implementation
7. Q&A

7. What is AI?

8. GOFAI
• Explicit definition of symbols
and algorithms
• There is no actual “learning”
• No capacity to abstract
information into new
concepts

9. Machine Learning
• Statistic approach
• Train model with large amounts of data
• Model is able to learn and generalize
• Predict, classify, generate

10. Real-world applications
• Computer vision
• Text and speech recognition
• Recommendation algorithms
• Pattern detection
• Includes all industry verticals

11. Using ML in your apps
Creating and training
a model
Training an existing
model
Using a third-party ML
service

12. Creation of models
• Advanced Math, Statistics and
Computer Science knowledge
• Often done at postgraduate
level
• Really cutting-edge and exciting
field

13. Training of existing models
• Find a suitable model and data
• Free libraries like Tensorflow
• Deployment

14. Third-party ML services
• Can be integrated into any app
• You don’t have to train it, host it
or maintain it
• Microsoft, AWS, Google, and
others
• Black blox, cost $$$

15. Overview
Creating and training
a model
Training an existing
model
Using a third-party ML
service

16. Roadmap
1. ✅ Introduction
2. ✅ What is AI
3. ➡ Microsoft Azure Cognitive Services
4. Unity demos
5. Unity basics
6. Unity in-depth implementation
7. Q&A

17. Microsoft Azure
Cognitive Services

18. • Set of APIs for common AI
needs
• Hosted on Microsoft Azure
• Can be plugged into any app
or game
Overview

19. Supported languages
• C#
• Python
• Java
• JavaScript (Node.js)
• Go
• Direct API calls (language agnostic)

20. Main areas
Decision Language Vision
Speech

21. Workflow
App takes
photo
App sends a
request to
Azure
Azure
processes
image
Azure sends
a response
back to the
App
App receives
image data
Example: https://azure.microsoft.com/en-au/services/cognitive-services/computer-vision/#features

22. JSON
• Data exchange format
• Commonly used when using
APIs
• Easy for humans to write and
read
• Example beautifier:
https://jsonparseronline.com/

23. Creating the service
• Create Azure subscription
• Create Resource Group
• Create Computer Vision
service
• Obtain keys and domain

24. Authentication
• How does Azure know who is
sending the request?
• Keep your credentials safe!
• You can use mine during this
workshop

25. Challenge 1: Run an API call
1. Open a text or code editor (or go to pastebin.com)
2. Go to to the Image Analysis docs, REST API (link in Zoom)
3. Copy the command into the text editor
4. Make the following changes:
• Replace <subscriptionKey> by what’s added on Zoom
• Replace https://westcentralus.api.cognitive.microsoft.com by
5. Open a terminal, paste command and press ENTER:
• Windows: Command Prompt (cmd), NOT Power Shell
• Mac: Terminal
5. Copy response into a JSON beautifier (https://jsonparseronline.com/)
6. Extra credits: replace the image URL by another image

26. Challenge 2: Run an API call
1. Create an Azure Computer Vision Service
2. Obtain service keys
3. Modify the API call from Challenge 1, to utilize

27. Roadmap
1. ✅ Introduction
2. ✅ What is AI
3. ✅ Microsoft Azure Cognitive Services
4. ➡ Unity demos
5. Unity basics
6. Unity in-depth implementation
7. Q&A

28. Unity Demos

29. Overview
Azure Cognitive Services APIS:
• Object detection (image analysis)
• Text recognition (OCR)
• Text to speech
Image sources:
• Webcam
• In-game camera

30. Roadmap
1. ✅ Introduction
2. ✅ What is AI
3. ✅ Microsoft Azure Cognitive Services
4. ✅ Unity demos
5. ➡ Unity basics
6. Unity in-depth implementation
7. Q&A

31. Unity Basics

32. Challenge 3: Re-create Webcam Demo
1. Create a new Scene in Unity
2. Add the following prefabs:
• UICanvas
• WebcamCanvas
• Object Detection
• Text Recognition
• Text to Speech
3. Add subscription keys to Object Detection, Text Recognition and Text to
Speech (select “Webcam” as the source)
4. Setup the events like in Demo 2
5. Make 1 unique change

33. Challenge 4: Expand In-Game Demo
- Add new items to Demo 1 (use the Asset Store)
Look for realistic, free packages.
- Make 1 logic change

34. Roadmap
1. ✅ Introduction
2. ✅ What is AI
3. ✅ Microsoft Azure Cognitive Services
4. ✅ Unity demos
5. ✅ Unity basics
6. ➡ Unity in-depth implementation
7. Q&A

35. Unity In-Depth
Implementation

36. Modules
• UI canvas
• Webcam
• In-game screenshots
• Computer Vision
• Speech

37. Roadmap
1. ✅ Introduction
2. ✅ What is AI
3. ✅ Microsoft Azure Cognitive Services
4. ✅ Unity demos
5. ✅ Unity basics
6. ✅ Unity in-depth implementation
7. ➡ Q&A

38. Q&A

39. Roadmap
1. ✅ Introduction
2. ✅ What is AI
3. ✅ Microsoft Azure Cognitive Services
4. ✅ Unity demos
5. ✅ Unity basics
6. ✅ Unity in-depth implementation
7. ✅ Q&A

40. Thank You
Pablo Farias Navarro
Zenva founder