Build applications with generative AI on Google Cloud! We are going to see in action what Gen App Builder is for developers to build and deploy AI-driven applications. We will explore Model Garden powered experiences, then we are going to learn more about the integration of these generative AI APIs. Vertex AI includes a suite of models that work with code. Together these code models are referred to as the PaLM and Codey APIs. The Vertex AI Codey APIs include the code generation API which supports generating code using a natural language description. We will show strategies for creating prompts that work with the model to generate code. At the end of the session, developers will understand how to innovate with generative AI and develop apps using the generative AI industry trends.

1. MLCon Berlin, November 2023
GenAppsonGoogleCloud
PaLM2andCodeyAPIsinAction
Márton Kodok
Software Architect at REEA.net
Berlin 2023

2. 1. What is Vertex AI?
2. Generative AI / LLM / Foundation Models
3. Exploring Model Garden
4. Code Demo
5. Conclusions
Agenda
Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok

3. ● Google Developer Expert on Cloud technologies (2016→)
● Champion of Google Cloud Innovators program (2021→)
● Among the Top 3 romanians on Stackoverflow 205k reputation
● Crafting Cloud Architecture+ML backends at REEA.net
Articles: martonkodok.medium.com
Twitter: @martonkodok
Slideshare:martonkodok
StackOverflow: pentium10
GitHub: pentium10
Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
About me

4. @martonkodok
Available in 46 languages; 238 countries
Bard - Chat Based AI Tool from Google, Powered by PaLM 2

5. “ demo…
@martonkodok

6. @martonkodok
Bard Extensions

7. @martonkodok
Getting results from Gmail

8. @martonkodok
list the top 10 incidents from email for service name "trademark", group results in table, order
by date, hour descending, extract duration, make an analyses if there is a repeated pattern

9. @martonkodok
Sum my invoices

10. @martonkodok
find all emails from this year from Hidroelectrica that contain "factura" and list all invoice values
for this year, with dates, in a table, order by descending

11. Gen AI and Model Garden on Vertex AI @martonkodok
Google Cloud AI Portfolio
Foundation
Models
Vertex AI
End-to-End ML Platform
Generative Al
App Builder
Text Chat Code Image
Video
Google Cloud Infrastructure - GPUs/TPUs
Contact Center AI Healthcare AI
Discovery AI
Document AI
Conversation AI
Vertex AI
Search
Foundation
Models
Business Users
AI Practitioners
Developers
Audio and
Music
Generative AI
Studio
Generative AI
APIs
Model Garden
Duet Al for
Google Workspace
Duet Al for
Google Cloud

12. @martonkodok
What is
Vertex AI?
Part #1

13. “ VertexAI is a managed ML platform for developers
@martonkodok

14. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
VertexAI:Managed unified ML platform
Fine-tuning 1 click deploy

15. “ You can deploy models on VertexAI
and get a HTTPs Endpointsto do
inference rapidly and reliably.
Generative AI and Model Garden on Vertex AI @martonkodok

16. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
Vertex AI provides tools to build with Gen AI
Model Garden
Generative AI Studio
Open Source
Models
Task Specific
AutoML and APIs
Foundation
Models
Tuning
Adaptive Layers
Prompt Design
Data Science Workbench
Experiment Train Deploy
MLOps
ML Platform

17. Model Garden
Part #2

18. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
Model Garden
Task Specific
AutoML and APIs
Open Source
Models
Foundation
Models
Model Garden

19. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
Model Garden
Foundation Models
Multi-task Large-scale Minimal training

20. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
Language Foundation Models
PaLM 2 for Text
Custom language tasks
PaLM 2 for Chat
Multi-turn conversations with
session context
Codey for
Code Generation
Improve coding and debugging
Chirp
Turning audio containing speech into
formatted text representation
Imagen
Write text prompts to generate new
images or generate new areas of an
existing image.

21. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
Introduction
Generative AI Studio
Console-tool Test generative models Rapid prototyping

22. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
Generative AI Studio

23. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
Custom language Prompt samples
1. Summarization
2. Classification
3. Extraction
4. Writing
5. Ideation
cloud.google.com/vertex-ai/docs/generative-ai/learn/prompt-samples

24. Part #3
Open-source
models

25. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
Model Garden
Open Source Models
Deploy Large-scale Fine-tunable

26. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
Open Source models
53+ other models
Providing open source model ecosystem
enables companies to look for
their particular use case
Falcon-instruct (PEFT)
Popular assistant-style tasks
Falcon-40B-instruct on Vertex AI
Llama 2
Meta's Llama 2 models
on Vertex AI
Code Llama
Designed for general code synthesis and
understanding, designed for Python
More…

27. “ LargeLanguageModelscome with
large sheer (cliff) size problems
…. deployment issues
@martonkodok

28. 1. LLMS have enabled unseen new task paradigm
2. However, LLMs are challenging to deploy to real-world apps due to their largesize
3. 175b LLM requires at least 350GB of GPU memory - using specialized infrastructure
Challenging to serve in practice
Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok

29. “ VertexAI enables model inference via API
Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok

30. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
VertexAI:Managed unified ML platform
Fine-tuning 1 click deploy

31. @martonkodok
Explore AI models with Colab samples
cloud.google.com/vertex-ai/docs/start/explore-models#test-promptsub

32. Demo
Code repository
Part #4

33. Code demo … repository
Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok

34. @martonkodok
Sample code for GenAI on Google Cloud
goo.gle/gen-ai-github

35. @martonkodok
TextGenerationModel

36. @martonkodok
Text Extraction

37. @martonkodok
Text Extraction Results

38. @martonkodok
Text Extraction

39. @martonkodok
Text Extraction

40. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
Introduction
Vertex AI Codey APIs

41. @martonkodok
Accelerate software development with Codey

42. @martonkodok
Codey supporting languages

43. @martonkodok
Fine tune Codey models
1. can use own code
2. improve model quality
3. generate code in language variants to use our standards
4. generate code for custom libraries

44. @martonkodok
Tune language models
cloud.google.com/vertex-ai/
docs/generative-ai/models/tune-model
s

45. Codey demo … repository
Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok

46. Data engineering
Part #5

47. What’s included in VertexAI?
@martonkodok
Data Labeling
AutoML models
DL Environment (DL VM + DL Container)
Prediction
Feature Store Training
Experiments
Data Readiness
Feature
Engineering
Training/
HP-Tuning
Model
Monitoring
Model serving
Understanding/
Tuning
Edge
Model
Management
Notebooks
Pipelines (Orchestration)
Explainable AI
Hybrid AI
Model
Monitoring
Metadata
Vision-Video-NLP-Translate
Models
Datasets
Custom Models
Containers
Python
Endpoints
BigQuery ML
BigQuery Models
Publisher
Pre-trained
models
Model Garden GenAI Studio
Foundation -> LLM Palm 2 API

48. Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
VertexAI:Managed unified ML platform
Fine-tuning 1 click deploy

49. Conclusion
Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok

50. “ At the end of the day, the largest model
is actually not the right answer
Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok

51. Vertex AI: From prompt samples to Fine-tuning
Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok
cloud.google.com/vertex-ai/
docs/generative-ai/models/tune-models
goo.gle/gen-ai-github
cloud.google.com/vertex-ai/
docs/generative-ai/learn/prompt-samples

52. 1. Build with the groundbreaking ML tools that power Google
2. Model Garden provides a curated collection of 100+ models
3. Approachable from the non-ML developer perspective (managed models, fine-tuning training)
4. Accelerate ML with tooling for pre-trained, open source and custom models
5. Deploy to applications with just one-click
6. End-to-end integration for data and AI with build pipelines that outperform and solve complex ML tasks
Vertex AI: Enhanced ML developer experience
Gen Apps on Google Cloud: PaLM2 and Codey APIs in Action @martonkodok

53. slideshare.net/martonkodok
Intro to Model Garden and GenApps @martonkodok

54. Twitter: @martonkodok
Thank you. Q&A.
Reea.net - Integrated web solutions driven by creativity
to deliver projects.
Follow for articles:
martonkodok.medium.com
Slides available on:
slideshare.net/martonkodok