Slides of my presentation about Serverless Machine Learning using Azure Functions, Twilio APIs, and Cognitive Services for text and image processing of WhatsApp messages at .NET Docs Show weekly community event organized by Microsoft

1. Serverless Machine
Learning
Luis Beltrán

2. Luis Beltrán
• Microsoft MVP in AI and Developer
Technologies
• Researcher at Tomas Bata University in
Zlín, Czech Republic
• Lecturer at Tecnológico Nacional de
México en Celaya
@darkicebea
m
luis@luisbeltran.mx
luisbeltran.mx

3. Agenda
• Intro
• Machine Learning
• Cognitive Services (LUIS, Computer Vision)
• Twilio (WhatsApp API)
• Serverless (Azure Functions)
• Integration (Demo)

4. Intro

5. .NET
Your platform for building anything
Desktop Web Cloud Mobile Gaming IoT AI/ML

6. COMPILERS LANGUAGES
RUNTIME COMPONENTS
LIBRARIES
INFRASTRUCTURE
.NET
TOOLS
VISUAL STUDIO CODE
CLI
VISUAL STUDIO
VISUAL STUDIO FOR MAC
WEB CLOUD MOBILE GAMING IoT AI/ML
DESKTOP

7. Machine Learning

8. Historic data
New data
Machine Learning
Algorithm
ML model
Data
Prediction
Training
Prediction
Machine Learning
Machine learning is a branch of AI that focuses on building
apps that learn from data and improve their accuracy over the
time without being explicitly programmed to do so.

9. Learning Tasks
Regression
Class A
Class B
Classification Clustering

10. Chatbots

11. How does a chatbot work?
Chatbot: What is a Chatbot? Why are Chatbots Important?
https://expertsystem.com/chatbot/

12. Computer Vision
Object detection Brand identification
Face detection
OCR – Optical Character Recognition
Image Analysis

13. Cognitive Services
LUIS + Computer Vision

14. Azure Cognitive Services
Computer Vision
Face API
Custom Vision
Text-to-Speech
Speech-to-Text
Speech Translator
Speaker Recognition
Language Understanding
Text Translator
Text Analytics
QnA Maker
Anomaly Detector
Content Moderator
Personalizer
microsoft.com/cognitive
Vision Speech Language Decision

15. Computer Vision API

16. LUIS: Language Understanding
Intelligent Service
• NLP-as-a-service.
• A cloud-based service for
understanding text.
• Simple to use and
integrate into your
software developments.
• No prior AI experience is
required.
https://www.luis.ai/

18. Demo
Cognitive Services

19. LUIS

20. LUIS

21. LUIS

22. LUIS

25. Computer Vision

26. Twilio

27. WhatsApp Business API
• Boost communication with your
customers around the world so that
you can contact them through
WhatsApp in a simple, safe and
reliable way.
• It requires contracting with a global
business solutions provider, available
on Facebook's partner directory.
https://www.whatsapp.com/business/api

28. https://www.facebook.com/business/partner-directory/search?platforms=whatsapp&solution_type=messaging

29. Twilio
• Communication Platform-as-a-Service
• Access to WhatsApp Business API.
• Simple to use and implement.
• Test and design your app in a sandbox
environment.
• Available on several platforms and
programming languages
https://www.twilio.com/whatsapp
https://www.twilio.com/docs/whatsapp/api

30. Demo
Twilio

34. Serverless
Azure Functions

36. What is Serverless?

37. Serverless application platform
components

38. Azure Functions

39. Your code runs when a
(previously-specified)
event occurs
Blob Storage
Cosmos DB
Event Hub
HTTP
Queues
Service Bus
Timer
Webhook
Photo taken and sent r
Upload it to
blob storage
PP
r
Produce scalated images
File
Table
Excel
OneDrive
Email
Mobile app
Notification
Más…

40. Use Bindings in Your Code
public static void Run(byte[] image, string filename,
Stream outputBlob, TraceWriter log)
{
log.Info($"Processing image: {filename}");
var imageBuilder = ImageResizer.ImageBuilder.Current;
imageBuilder.Build(
image, outputBlob,
new ResizeSettings(640, 400, FitMode.Max, null), false);
}
{
"bindings": [
{
"name": "image",
"type": "blobTrigger",
"direction": "in",
"path": "card-input/{filename}.jpg",
"connection": "AzureWebJobsStorage"
},
{
"type": "blob",
"name": "outputBlob",
"path": "card-output/{filename}.jpg",
"connection": "AzureWebJobsStorage",
"direction": "out"
}
]
}

41. Demo
Azure Functions

46. Final Demo
Integration of Technologies

48. 1
2
3
4a
6
4b
5a
5b
7

49. Call to Action
Developing a serverless WhatsApp chatbot
https://dev.to/icebeam7/developing-a-serverless-whatsapp-chatbot-4o72
Enviando mensajes de WhatsApp con Xamarin y Twilio
https://www.luisbeltran.mx/2018/10/15/enviando-mensajes-de-whatsapp-con-
xamarin-y-twilio/
How to Use Image Recognition on Twilio WhatsApp API
https://www.twilio.com/blog/image-recognition-clarifai-twilio-whatsapp-api
Sending and Receiving Media with WhatsApp Messaging on Twilio
https://support.twilio.com/hc/en-us/articles/360017961894-Sending-and-
Receiving-Media-with-WhatsApp-Messaging-on-Twilio

50. Towards a responsible AI
• Transparency
• Clarity
• Transfer to a human
• Cultural norms
• Trustworthy
• Privacy
• Data security
• Accessibility
• Responsibility

51. Q & A

52. Thanks for joining!
Luis Beltrán
about.me/luis-beltran