10. Takeaways
• Semantically, ChatGPT has no idea. It’s just doing Math.
• Factually, it can occasionally make errors.
• It has awesome natural language understanding and generation skills.
• Prompting and grounding with missing knowledge closes the gaps.
• Generative AI models are “jack of all trades”.
• “Collaborate” with them via prompting, finetuning, humans, models.
• Think “Copilots”, “Assistants”, “Wizards”, “Suggestions”…
12. The Problem of Language modeling
Pre-2017, while CNNs (convolutional neural
networks) worked great for images, RNNs
(recurrent neural networks) for language did not.
RNNs were sequential, error-prone, and did not
capture the language model insights and were
therefore did not output natural looking text.
13. “[Self] Attention is all you need”
• 2017 breakthrough paper titled “Attention is
all you need” introduced the Transformer
model architecture.
• Key idea: Focus on “positional encodings”,
“attention” and “self attention” for the tokens
(integers) representing the words/sub-words.
https://arxiv.org/abs/1706.03762
14. In plain English
Capture a sense of “nearness” by similarity of
use, numerical distance, etc. and relate to
other words (aka tokens) for ordering and
deep understanding of the language model to
compute the probability of the next token in
sequence. (Its just Math)
15. Over-simplified How-it-works
• Assign to each unique word a unique identifier, a number that will serve as a
token to represent that word.
• Note the location of every token relative to every other token.
• Using just token and location—determine the probability of it being adjacent to,
or in the vicinity of, every other word.
• Feed these probabilities into a neural network to build a map of relationships.
• Given any string of words as a prompt, use the neural network to predict the
next word (just like AutoCorrect).
• Based on feedback, adjust the internal parameters of the neural network to
improve its performance.
• Extend the prediction to the next phrase, the next clause, the next sentence, the
next paragraph, and so on, by using feedback to further adjust its internal
parameters.
• Based on the above, generate text responses to user questions and prompts
that reviewers agree are appropriate and useful.
16. Summarizing
• Generative AI language models fundamentally predict
the next words (represented as numerical tokens) in
sequence like auto-complete.
• Words are tokenized into numbers and the model
processes these “tokens” while dealing with context,
nearness, and weights in the neural network and
iterating on them as the model is trained.
17. Adoption landscape and trade-offs
Prompted: Zero or One-shot Prompted: Multiple-shot Finetuned DIY
General/Broad task Narrow task
LLM LLM Specialized Specialized
+
Synthetic data
generation
AI Quality
Evaluation
Inferencing in
production
Usage
Scope
Composition
Lifecycle
Iterative benchmarking and regression testing become critical.
20. Recall the Tech Adoption‘S’Curves
Transitioning from one S-curve to
another is the tricky part for companies.
The bridges go over choppy waters.
The transitions between innovations can
span months and years. Therefore, you
need bridges in form of hybrid systems,
handovers of technology and
customers, and managing of resources
and investment as the landscape shifts.
21. The race in a nutshell
Technological
discontinuity
Era of ferment
Era of incremental change
Dominant
design
emerges
Cost
decreasing
with usage
Chosen
dominant
design
Learn (fast)
Adopt (fast)
Dominate (fast)
Performance
increasing with
usage
Size of
installed
base
Availability of
complimentary
goods
22. First mover(s) pros & cons
1
2
3
Winner(s) take(s) all
Brand amplification and loyalty
Technological leadership
Preemption of scarce assets
Exploitation of buyer switching costs
• Research and development expenses
• Undeveloped supply and distribution channels
• Immature enabling technologies and complements
• Uncertainty of customer requirements
Winner(s) must deal with:
There is no free lunch. Just strategic tradeoffs.
23. Which CEO said what (paraphrasing)
“The one that I think is going to have the fastest direct business loop is going to be around
helping people interact with businesses. I mean, you can imagine a world where over
time every business has an AI agent that basically people can message and interact with
them. … It’s quite human labor intensive for a person to be on the other side of that
interaction.”
“Some of our most successful products were not first to market. They gained momentum
because they solved important user needs and were built on deep technical insights.”
“The most important thing we are doing in AI is trusted and responsible AI. Customers,
particularly large enterprises, are equal parts intrigued and concerned by the
technology’s potential. We’ve all seen the movies and where this can go. We have all
these crazy ideas in our head of what can happen.”
“Many of today’s A.I. chatbots and other generative A.I. tools are part of the “hype cycle,”
while we are focused on the “substance cycle…This is a 10K race while everyone is just 3
steps in.”
“We are making the first moves and if our friends (competition) respond and they will, we
want everyone to know we made them move.”
Amazon
Microsoft
Facebook
Google
Salesforce
25. Most practical customer scenarios
• Written content augmentation and creation: Producing a “draft” output of text
in a desired style and length
• Question answering and discovery: Enabling users to locate answers to input,
based on data and prompt information
• Tone: Text manipulation, to soften language or professionalize text
• Summarization: Offering shortened versions of conversations, articles,
emails and webpages
• Simplification: Breaking down titles, creating outlines and extracting key
content
• Classification of content for specific use cases: Sorting by sentiment, topic,
etc.
• Software coding: Code generation, translation, explanation and verification
https://www.gartner.com/en/topics/generative-ai
29. Advanced Use Cases by Industry
https://www.gartner.com/en/articles/beyond-chatgpt-the-future-of-generative-ai-for-enterprises
“Generative AI has
already been used
to design drugs for
various uses within
months, offering
pharma significant
opportunities to
reduce both the
costs and timeline of
drug discovery.”
30. In a recent Gartner webinar
poll of more than 2,500
executives, 38% indicated
that customer experience
and retention is the primary
purpose of their generative
AI investments. This was
followed by revenue growth
(26%), cost optimization
(17%) and business
continuity (7%).
https://www.gartner.com/en/articles/beyond-chatgpt-the-future-of-generative-ai-for-enterprises
31. Costs: From negligible to many millions
• Free versions of public, openly hosted applications, such as
ChatGPT, or by paying low subscription fees. However, free and
low-cost options come with minimal protection of enterprise data
and associated output risks.
• Larger enterprises and those that desire greater analysis or use of
their own enterprise data with higher levels of security and IP and
privacy. In this instance, costs can be in the millions of dollars.
• Generative AI capabilities will increasingly be built into the
software products you likely use everyday, like Bing, Office 365,
Microsoft 365 Copilot and Google Workspace. Vendors will pass
on costs to customers as part of bundled incremental price
increases. Microsoft recently announced their per-user pricing for
Office 365 CoPilot pricing.
33. Product (Managers)
• Learn up the what/how(AI level) to
update product (what/why)
• More OOB model power => emphasis
on user value and experience
• User generated vs. AI generated content
trade-offs
• More collaborative with higher
accountability bar
• Career opportunity to get into AI (Tech)
35. Business planning
• New underlying economics
• Willingness to pay (WTP) questions
• Pricing models – new units (e.g., tokens)
• Stand-alone vs. bundling with existing
• New and more stakeholders to manage
• Business projections vs. realization
36. User experience design impact*
Collaborative
not
command
AI notices ala
disclaimers
Reverse-
prompt users
Citations and
references
Feedback for
improvement
Address
costs and
speed
https://www.linkedin.com/learning/ux-for-ai-design-practices-for-ai-developers/designing-for-ai
37. Software
Testing/QA
• Learn about AI quality metrics and lifecycle
• Word Error Rate (WER), FactX, ROUGE-L,
Human evaluations
• Precision, Recall, F1-Score, Ground truth
(reference) data
• Learn about development and evaluation
data set creation
• Learn about AI benchmarking and regression
best practices
• Create a GitHub repo, download a public
dataset as your ground truth (reference),
create a test dataset from the internet, GPT,
or your own and publish your findings
38. Recap career tips
Generative AI levels the playing field between tech and non-tech
Opportunity for entry into the tech and AI space
New disciplines like Prompt Engineering are hot
Responsible AI is an example of new careers to learn up and enter the AI industry
Use experience design principles are evolving to meet challenges
For newbies, learn it up while prioritizing your target niche
For experienced ones, decide how and where you want to play
40. Why it is more important today
With Generative AI power also come significantly increased
challenges related to harmful content, manipulation, human-like
behavior, privacy, and more.
The industry is unanimous and for good reasons that responsible
use and AI need to be tied to the hip.
41. Example planning framework
• Policy
• Research
• Engineering
• Fairness
• Reliability and safety
• Privacy and Security
• Inclusiveness
• Transparency
• Accountability
• Operationalization
• Advocacy
• Alliances
• Positioning
Pillars Principles Practices
https://www.microsoft.com/en-us/ai/responsible-ai
42. Responsible Use Product Guidance (example)
• Harm mitigation
• Identify
• Measure
• Mitigate
• Operation
• Transparency note
• Introduction
• Concepts
• Intended use cases
• Considerations when choosing a use case
• Capabilities
• Limitations
• Limited access aka “gating”
• Application
• Review of use case
• Other criteria
• Approvals
• Code of Conduct
• Access requirements
• Content requirements
• Mitigations requirements
Azure OpenAI Responsible Use Guidance (example)
43. Data, privacy, and security
• What data is processed?
• How does the service process data?
• Cover over the wire and in the cloud infrastructure
• Which secure networking options are available
• Regulatory compliance certifications
• Government clouds, Public clouds
Microsoft Azure Compliance Offerings (example)
44. Content safety as a service
https://azure.microsoft.com/en-us/products/ai-services/ai-content-safety
45. Takeaways
• Learn up on the latest and what companies and agencies are up to
• Excellent way to get into AI as a career
• Critical regulatory and compliance space that’s poised to grow
• Solving hard problems while working with AI researchers
