This document discusses how generative AI is being embedded into cloud ecosystems and tools. It notes that AI is appearing in productivity apps, developer tools, search, CRM, analytics, messaging, and ERP systems. The document also discusses issues around data sources, authentication of plugin users, responsibility and privacy considerations for generative AI models, different pricing models between OpenAI and Google Cloud, and experimenting with prompt development.

1. Aavista Oy
Merja Kajava
@mkajava
APIDays Helsinki & North 2023
What Generative AI
Really Means to Cloud
Ecosystems

3. ChatGPT is the tip of the iceberg

4. OpenAI ecosystem

5. OpenAI ecosystem

6. AI is being embedded into tools
Productivity apps
Dev tools
Search CRM
Analytics
Messaging
ERP
API and Plugins
Copilots

7. AI in the cloud ecosystems

8. “We need to start
using AI in our
applications”

9. Where is the data coming from?

10. Data comes from several sources
ERP
Databases
Documents
…
Internet
3rd party databases
…
Public
Private
Data provenance

11. PublishYour API with Plugins
Copilot
Your API
Assistant
Plugin for
Google
Plugin for
ChatGPT
Marketplace
for ChatGPT

12. Plugin is the new API

13. Authenticate Plugin Users
No authentication Service-level
authentication
User-specific
authentication

14. Be responsible
Identify risks
Mind the privacy
Consider IPR
Validate usage via
moderations
Monitor the usage

15. From API to Prompt Development

16. Specify the input and include
format for the response.
PROMPT
Getting Started with Prompt Development
prompt = f"""
Translate the following text to all the Nordic languages and
provide the answer in the same language:
'What is the capital of Sweden?'
Output in Markdown table with headers: Language, Translation,
Answer
"""
response = get_completion(prompt)
Text Files Images
Speech

17. Response is produced and formatted
according to the prompt.
RESPONSE
Responses to Prompts

18. Tuning the Model
def get_completion(prompt, model="gpt-3.5-turbo"):
messages = [{"role": "user", "content": prompt}]
response = openai.ChatCompletion.create(
model=model,
messages=messages,
temperature=0,
)
return response.choices[0].message["content"]
TEMPERATURE
Decide the degree of randomness
of the model's output
FOUNDATION MODEL
Choose the model, for example
GPT 3.5 Turbo, or GPT 4.

19. Pricing models are different
Words from the prompt and response are the
basis for the costs.
Foundation models also have different pricing.
OpenAI API
Prices are per 1,000 tokens.
Token is a piece of word, where
1,000 tokens is about 750 words.
Generative AI on Vertex AI
Prices are per 1,000 characters.
Characters are counted by UTF-8
code points.

20. Be creative
Start experimenting
Build plugins on top of your APIs
Compare prices
Remember the Responsible AI

21. The Data Refinery Company
Merja Kajava
https://www.linkedin.com/in/merjakajava