Slides of my presentation about Image Recognition in WhatsApp chatbot with Azure AI for Azure Community Conference

1. Azure Community Conference 2021
India’s largest Azure Conference
#AzConfDev

2. #AzConfDev
ANANT MAHESHWARI
President, Microsoft India

3. #AzConfDev
Image Recognition in WhatsApp chatbot
with Azure AI
Luis Beltrán

4. #AzConfDev
Luis Beltrán
 Researcher at Tomas Bata University in Zlín,
Czech Republic
 Lecturer at Tecnologico Nacional de Mexico en
Celaya, Mexico
 Microsoft MVP in AI and Developer Technologies

5. #AzConfDev
Agenda
 Introduction
 Cognitive Services
 Twilio API for WhatsApp
 Azure Functions
 Demo

6. #AzConfDev
Introduction

7. #AzConfDev
What is a chatbot?

8. #AzConfDev
How does a chatbot work?

9. #AzConfDev
Computer Vision
Object detection
Brand identification
Face detection
OCR – Optical Character Recognition
Image Analysis

10. #AzConfDev
Historic data
New data
Machine Learning
Algorithm
ML model
Data
Prediction
Training
Prediction
Machine Learning
Machine learning is a branch of AI that focuses on building apps that learn from
data and improve their accuracy over the time without being explicitly
programmed to do so.

11. #AzConfDev
Learning Tasks
Regression
Class A
Class B
Classification Clustering

12. #AzConfDev
Cognitive Services

13. #AzConfDev
Azure Cognitive Services
Perception Comprehension
Computer Vision
Face API
Custom Vision
Text-to-Speech
Speech-to-Text
Speech Translator
Speaker Recognition
Language Understanding
Text Translator
Text Analytics
QnA Maker
Anomaly Detector
Content Moderator
Personalizer
microsoft.com/cognitive
Vision Speech Language Decision

14. #AzConfDev
Computer Vision API
https://aidemos.microsoft.com/computer-vision

15. #AzConfDev
Demo #1: Computer Vision

16. #AzConfDev
LUIS: Language Understanding Intelligent Service
Natural Language Processing
A cloud-based service for
understanding text.
Simple to use and integrate into your
software developments.
No prior AI experience is required.
https://www.luis.ai/
https://aidemos.microsoft.com/luis

17. #AzConfDev

18. #AzConfDev
Demo #2: LUIS

19. #AzConfDev

20. #AzConfDev

21. #AzConfDev

22. #AzConfDev

23. #AzConfDev
Twilio API for WhatsApp

24. #AzConfDev

25. #AzConfDev
WhatsApp Business API
 Boost communication with your customers
around the world so that you can contact them
through WhatsApp in a simple, safe and reliable
way.
 It requires contracting with a global business
solutions provider, available on Facebook's
partner directory.
https://www.whatsapp.com/business/api

26. #AzConfDev
https://www.facebook.com/business/partner-directory/search?platforms=whatsapp&solution_type=messaging

27. #AzConfDev
Twilio
 Communication Platform-as-a-Service
 Access to WhatsApp Business API.
 Simple to use and implement.
 Test and design your app in a sandbox environment.
 Available on several platforms and programming languages
https://www.twilio.com/whatsapp
https://www.twilio.com/docs/whatsapp/api

28. #AzConfDev
Demo #3 – Twilio

29. #AzConfDev

30. #AzConfDev

31. #AzConfDev
Azure Functions

32. #AzConfDev
PaaS
IaaS
On-premises Serverless
How do I architect my app?
Serverless, the cloud native platform
The evolution of application platforms

33. #AzConfDev
Serverless computing
Event-driven / Instant scale Micro-billing
Abstraction of servers
$
Focused on business logic Faster time to market
Reduced DevOps

34. #AzConfDev
Serverless application platform components

35. #AzConfDev
Azure Functions
Code Events + data
Azure Functions

36. #AzConfDev
Functions
Your code runs when a
(previously-specified)
event occurs
Triggers
Blob Storage
Cosmos DB
Event Hub
HTTP
Queues
Service Bus
Timer
Webhook
Bindings
File
Table
Excel
OneDrive
Email
Mobile app
Notification
…
Photo taken and sent r
Upload it to
blob storage
Produce scalated images

37. #AzConfDev
run.csx
public static void Run(byte[] image, string filename,
Stream outputBlob, TraceWriter log)
{
log.Info($"Processing image: {filename}");
var imageBuilder = ImageResizer.ImageBuilder.Current;
imageBuilder.Build(
image, outputBlob,
new ResizeSettings(640, 400, FitMode.Max, null), false);
}
{
"bindings": [
{
"name": "image",
"type": "blobTrigger",
"direction": "in",
"path": "card-input/{filename}.jpg",
"connection": "AzureWebJobsStorage"
},
{
"type": "blob",
"name": "outputBlob",
"path": "card-output/{filename}.jpg",
"connection": "AzureWebJobsStorage",
"direction": "out"
}
]
}
function.json

38. #AzConfDev
Demo #4 – Azure Functions

39. #AzConfDev

40. #AzConfDev

41. #AzConfDev

42. #AzConfDev
Final Demo – Integration of Services

43. #AzConfDev
1
2
3
4a
6
4b
5a
5b
7

44. #AzConfDev
Call to Action
Developing a serverless WhatsApp chatbot
https://dev.to/icebeam7/developing-a-serverless-whatsapp-chatbot-4o72
Enviando mensajes de WhatsApp con Xamarin y Twilio
https://www.luisbeltran.mx/2018/10/15/enviando-mensajes-de-whatsapp-con-
xamarin-y-twilio/
How to Use Image Recognition on Twilio WhatsApp API
https://www.twilio.com/blog/image-recognition-clarifai-twilio-whatsapp-api
Sending and Receiving Media with WhatsApp Messaging on Twilio
https://support.twilio.com/hc/en-us/articles/360017961894-Sending-and-
Receiving-Media-with-WhatsApp-Messaging-on-Twilio

45. #AzConfDev
Towards a responsible AI
 Transparency
 Clarity
 Transfer to a human
 Cultural norms
 Trustworthy
 Privacy
 Data security
 Accessibility
 Responsibility

46. #AzConfDev
Q & A

47. #AzConfDev
Feedback

48. #AzConfDev
THANK YOU!
Image Recognition in WhatsApp
chatbot with Azure AI
Luis Beltrán
Microsoft MVP
about.me/luis-beltran