This document discusses using artificial intelligence and Azure Cognitive Services with SharePoint. It provides an overview of Azure Cognitive Services capabilities like language detection, sentiment analysis, and image recognition. It then demonstrates how these cognitive services can be used with SharePoint through Power Apps, Microsoft Flow, and the SharePoint Framework to automatically tag and extract metadata from images and text. Examples shown include using cognitive services for image and text analysis, translation, and extracting metadata through Power Apps and updating through Flow.

1. SharePoint with [artificial] intelligence | Adis Jugo
@adisjugo

3. Adis Jugo
Microsoft MVP Office Apps and Services
Microsoft MVP Azure
Microsoft Regional Director
In IT for way too long (first money earned with development in 91)
Still dreaming of a restaurant or a vineyard
Chief Strategy Officer at SysKit Ltd
Mastermind behind European Collaboration Summit (www.collabsummit.eu)
Born in Sarajevo, Bosnia, living in Bingen, Germany
Blogger, speaker, author. adisjugo.com

6. Agenda
• Short intro into Azure Cognitive Services
• Short intro into SharePoint
• Azure Cognitive Services API Explorer
• Using Cognitive Services with SharePoint
• Microsoft PowerApps
• Microsoft Flow
• SharePoint Framework (SPFx)

7. Agenda
• Short intro into Azure Cognitive Services
• Short intro into SharePoint
• Azure Cognitive Services API Explorer
• Using Cognitive Services with SharePoint
• Microsoft PowerApps
• Microsoft Flow
• SharePoint Framework (SPFx)

8. What exactly are we talking about here?
The memex is the name of the hypothetical
proto-hypertext system that Vannevar Bush
described in his 1945 The Atlantic Monthly
article "As We May Think". Bush envisioned
the memex as a device in which individuals
would compress and store all of their books,
records, and communications, "mechanized
so that it may be consulted with exceeding
speed and flexibility". The memex would
provide an "enlarged intimate supplement to
one's memory". The concept of the memex
influenced the development of early hypertext
systems and personal knowledge base
software.

9. 2.5 quintillion
bytes per day

10. 80%
of business relevant information is
unstructured

12. TIFF HTML
…
JPG

15. Azure Cognitive Services:
The AI for poor people ☺

16. Cognitive Services capabilities
Infuse your apps, websites, and bots with human-like intelligence

17. Built-in skills
Cognitive skills
Azure
Databricks
Machine Learning
VMs
Azure Machine
Learning
Custom skills
Your custom
skill goes here!

18. So what do we potentially want to do?
• Language Detection
• Text context understanding
• Sentiment detection (ratings)
• Keyword Extraction (metadata)
• Objects and people extraction (metadata)
• Image understanding
• Image keyword extraction (metadata)
• OCR (search indexing)

19. Cognitive Services capabilities
Infuse your apps, websites, and bots with human-like intelligence
Object, scene, and
activity detection
Text and handwriting
recognition (OCR)
Custom image recognition
Language detection
Text sentiment analysis
Key phrase extraction
Entity recognition
Text translation
Customizable text translation

20. OK, so how do we do it?
SharePoint
image analysis
Limited capability, JSON
outputs
Power
Apps
Directly in Modern
SPForms
Flow
In processes
#SPFx
Highest Flexibility
#SPFx

21. • Any image file is automatically scanned
for object recognition
• identifying geolocation data
• Identifying extractable text images.
• tagged in metadata columns
on each document library
SPOnline, Built in capabilities
https://www.cloudappie.nl/image-analysis-sharepoint-online/

22. OK, so how do we do it?
SharePoint
image analysis
Limited capability, JSON
outputs
Power
Apps
Directly in Modern
SPForms
Flow
In processes
#SPFx
Highest Flexibility
#SPFx
Azure Cognitive Services

23. Demo 1 – Cognitive Services in Azure Portal
• Cognitive Services registration
• Keys
• Cognitive Services REST API Explorer

24. Demo 2 – Using Cognitive Services and
PowerApps for text analytics and autotagging
• In-Form operations
• Using PowerApps to extract and update metadata

25. Demo 3 – Using Cognitive services and
PowerApps for Text Translations
• Bing Translator

26. Demo 4 – Using Cognitive services and
Flow for Image Autotagging
• Background operations
• Using Flow update metadata

27. Thank you for attending! @adisjugo

28. Please give
some
feedback <3