How to analyze text with ml.net and .NET core using the best NLP technology like BERT and GPT. We see the ONNX project and the new ONNX Runtime.

2. Mauro Bennici
C.T.O. & Founder
GHOSTWRITER AI
https://ghostwriter.ai
@maurobennici
https://www.linkedin.com/in/maurobennici/
2

3. GHOSTWRITER AI
Customer conversation analytics
Actionable intelligence from your customers’
interactions

4. Specialties...
4
● Sentiment Analysis
● Semantic Search
● Entities extraction
● Pattern recognition
● Anomaly detection
● Events correlation
● Social Media Management
...and Technologies
● Natural Language Processing (NLP)
● Natural Language Understanding (NLU)
● Machine Learning
● Deep Learning
Scalable and cloud independent
The same expertise and technology fit
different industries, from marketing to
security, insurance, and retail.

5. Our CRM fields, notes,
chats, images, emails,
audio clips are full of
TEXT!
5

6. 66
R&D Production Services Marketing & Sales
Value adding
activities
Inputs Product development Deployment Application Markets
Data
sources
● Customer
segmentation and
targeting
● Customer
behavioral analysis
● Semantic analysis
● Brand
management
● Call center
optimisation
● CX help desk
improvement
● Social media
management
Distribution
● Web
● Social
media
● Email
● Call center
● Others
(surveys,
promotional
events, and
sales reps)
● On-premises
● Cloud
Lead acquisition and brand
awareness
● PPC ads
● events
● inbound
● engagement activities
● nurturing funnel
Sales
● events
● inbound
● engagement activities
● nurturing funnel
● customer support
● Banking, Financial
Services, and Insurance
(BFSI)
● Retail and eCommerce
● Government and Defense
● Healthcare and Life
Sciences
● Manufacturing
● TELCO and IT
● Energy and Utilities
● Media and Entertainment
● Travel and Hospitality
● Automotive and
Transportation
● Others (Education and
Real Estate)
Supplychain
Modeling
Training
of models and
people
Scaling
● Integrations
● Customisations
VALUE-CHAIN

7. TEXT is difficult to analyze!
7

8. 8
https://medium.com/@nitinpanwar98/convolutional-neural-networks-cnn-explanation-and-implementation-8c41be8b51a
https://becominghuman.ai/what-exactly-does-cnn-see-4d436d8e6e52

9. 9

10. 10
https://freecontent.manning.com/deep-learning-for-text/

11. 11
https://upload.wikimedia.org/wikipedia/commons/thumb/9/95/CatVibrissaeFullFace.JPG/1200px-CatVibrissaeFullFace.JPG
CAT
19320
NEKO
GATO
CHAT
GATTO

12. 12
https://upload.wikimedia.org/wikipedia/commons/thumb/9/95/CatVibrissaeFullFace.JPG/1200px-CatVibrissaeFullFace.JPG
https://www.caterpillar.com/en/brands/cat.html
CAT
19320
NEKOGATO CHATGATTO
???

13. 13
https://upload.wikimedia.org/wikipedia/commons/thumb/9/95/CatVibrissaeFullFace.JPG/1200px-CatVibrissaeFullFace.JPG
https://www.caterpillar.com/en/brands/cat.html
There is a cat inside the cat
???

14. 14
● Language
● Tone
● Aspects
● Sentiments
● Entities
● Words
● Grammar
● Context
● etc.
We analyze what
and how people
wrote, as

15. 15
● Context
● Location
● Age
● Job
● Studies
● Intention
● Sentiment
● Personality
● etc.
… to get
Respect privacy and local laws!

16. 16

17. Ibis, redibis, nunquam per bella peribis.
Meaning "you will go, you will return, never in war will you perish".
Ibis, redibis nunquam, per bella peribis.
That is: "you will go, you will never return, in (the) war you will perish".
17
https://en.wikipedia.org/wiki/Ibis_redibis_nunquam_per_bella_peribis
Sibyl: the oracle was without comma. Now, you know why

18. 18
https://imbowei.com/2018/10/27/From-transformer-architecture-to-transfer-learning/
A lot of procedures, models, frameworks!

19. Why ML.NET?
19

20. 20

21. 21
https://dotnet.microsoft.com/apps/machinelearning-ai/ml-dotnet#proven
https://arxiv.org/pdf/1905.05715.pdf

22. 22https://marketplace.visualstudio.com/items?itemName=MLNET.07

23. 23
Code Time

24. 24

25. 25
Code Time

26. 26
https://onnx.ai/

27. 27
Code Time

28. 28
https://microsoft.github.io/onnxruntime/

29. 29https://software.intel.com/content/www/us/en/develop/articles/intel-mkl-dnn-part-1-library-overview-and-installation.html

30. 30
Code Time
https://cloudblogs.microsoft.com/opensource/2020/01/21/microsoft-onnx-open-source-optimizations-transformer-inference-gpu-cpu/

31. 31
Case Studies

32. ghostwriter.ai | 32
SALES CALL OPTIMIZATION
Requirements
Separated (audio or text) conversation for agent/prospect
Training dataset of 1000 phone calls, 20’ each, call result
(ok/ko)
Time
3 months
Activity
AI models to identify key factors that drive to an higher
conversion rate.
‒ Script consistency
‒ Conversation’s rhythm, speak speed, interruptions
‒ Sentiment evaluation (stress, anxiety, happiness, etc.)
‒ Behavioral prediction
‒ Best practices identification
‒ Self-Learning (AI-driven algorithms learn from data
patterns)
Customer Sentiment Avg. speak speed
What we did
Conversation Analysis
We transformed the two audio channel phone calls in text (speech-to-text conversion).
Then, we analyzed the conversations in their elements. We divided the call in three parts: opening, core,
closing.
For each of them, we measured rhythm (how long, how fast, who speak, language, vocabulary, agent to
prospect language cohesion, etc), speak speed, interruptions, who drive the conversation, sentiment
(positive/negative), emotions (stress, excitement, anxiety, interest, irritation, etc).
Case Studies

33. Script Analysis
We analyzed the script, its semantic aspects. Also, we analyzed how the agents use the script, adherence,
changes along the conversation, and conversion of the leads.
Key Elements Relevance
We designed a behavioral prediction, that means, according to all the data collected during a call, the
probability of a lead conversion success.
From that, a diagram that shown the key elements relevance, even broken down by products and other
parameters.
The conversation outcome is in the data, and you can identify and improve key elements that drive
this success.
ghostwriter.ai | 33
SALES CALL OPTIMIZATION
By Factor Relevance
Case Studies
+35%
from script
improvement

34. ghostwriter.ai | 34
SALES CALL OPTIMIZATION
Explorer
Calls Script Rhythm Speed Sentiment Score
Call1
Call2
Call3
What the Customer Got
Case Studies
Full Report
The full report was released.
It measured improvements and suggested actions.
The customer got +35% in conversions thanks to script improvement.
Also, +20% in conversion from all the elements, and a considerable time (and costs) saved from better
coaching.
Best Practices Recommendation
How to improve the script, best agent use case for training and coaching purposes.
Top Agents
Agent Score
Sonia 750
Adam 345
Laura 340
+20%
Conversions
+Hours saved in sales coaching

35. ghostwriter.ai | 35
Requirements
Training dataset of 1000 questions
Their answer set
Knowledge base sources
What we did
AI models to increase ability to find the knowledge the
agents need using words or phrases to return a result set
of answers that best match the request.
‒ Text intent recognition
‒ Knowledge base understanding (NLU)
‒ Classification
‒ Close-text search
‒ Free-text search
‒ Self-learning system
Data
knowledge
AI
ingestion
Agent
question
Answer set
Training
● The agent confirms the answers between 5
options (ordered by confidence).
● If the answer is not in the set, the agent provides
the right solution. The AI learns autonomously.
● Weekly report on 1st positioning correct answers
and confidence increments.
Scrivi il tuo messaggio
HELP DESK
Case Studies

36. ghostwriter.ai | 36
Type your message
Feedback - Select the correct answer
altro⦾
785 - doc firm 228 c⦿
quando l’offerta ricevuta è incompatibile con la tariffa base facente parte del s
1382 - circolare 203⦾
la carta di debito selezionata ha la data di scadenza non compatibile con il pe
1354 - circolare 212 aa⦾
il perimetro di azione secondo cui viene applicato in percentuale la tariffa di ne
863 - doc1⦾
quando si seleziona dal menu a tendina la scadenza si deve verificare che il
1278 - off spec 2bis
le tariffe specifiche con riferimento all’off 22b non hanno la possibilità di essere
⦾
+85%
Confidence
+customer support capabilities
in results
Production
Question/Answer over the established threshold.
The AI recommends the 1st position answer that
the agent use as reply.
or
Question/Answer below the established threshold.
● The agent confirms the answers between 5
options (ordered by confidence).
● If the answer is not in the set, the agent provides
the right solution. The AI learns.
● The AI recommends the 1st position answer that
the agent use as reply
HELP DESK
Case Studies

37. ghostwriter.ai | 37
What the Customer Got
A customized system trained on its knowledge, that auto-learning and manage new/replaced knowledge
base content, and autonomously improves in confidence and accuracy.
Enhances agent
productivity and
helps improve
customer service
Similar use case: email scaling support
Email customer support can get out of control.
Backlog grows, people put pressure to get answers.
Ghostwriter AI Knowledge Search algorithms can
help you to have the job done faster.
The system trained, read the email and compose a
reply.
Data knowledge AI ingestion
Email
Answer set
Training
Production
HELP DESK
Case Studies

38. ghostwriter.ai | 38
Unstructured
data sources
AI
ingestion
Requirements
data access to social insights
What we did
AI models to ingest unstructured data and identify pattern
behaviors.
‒ Social conversation analysis
‒ Data extraction (demographics, psychographics, and
behavioral)
‒ Pattern recognition
‒ Intent recognition
‒ Topic and trends detection
‒ custom report according to defined KPI
TARGET MARKET ANALYSIS
Case Studies

39. ghostwriter.ai info@ghostwriter.ai
TechPartners
Accelerators
Communities
This project has received funding from the European
Union’s Horizon 2020 research and innovation
programme under grant agreement No 886853

40. Mauro Bennici
C.T.O. & Founder
GHOSTWRITER AI
https://ghostwriter.ai
@maurobennici
https://www.linkedin.com/in/maurobennici/
40