This deck is about what need to be mastered as AI-Powered Java Developers

1. Emily Jiang, Java Champion, FBCS
IBM, STSM, Cloud Native Architect and Advocate
16th May 2024
AI-Powered Java
Developers

2. 2
Java Developers
– You Build It, You Run It
– Dev and Ops together
– IDE
– Build (Jenkins, Containers)
– Deploy (CI/CD)
– Maintain (SBOM, Vulnerabilities)

3. 3
What can Improve Developer
Experience?

4. 4
Remove the tedious tasks!

5. 5
Java Developers need to be AI-
Powered!!!

6. AI-Powered Java Developer Checklist
1. What is AI, LLM?
2. What is RAG?
3. What code assistant tools available?
4. What can help with building AI-Infused apps?

7. AI
Artificial intelligence (AI) is the simulation of human intelligence in
machines that are programmed to think and act like humans.
Learning, reasoning, problem-solving, perception, and
language comprehension are all examples of cognitive abilities.

8. AI History 1956 John McCarthy held a
workshop at Dartmouth on
“artificial intelligence”
1957-1974 AI flourished
2011 IBM Watson won the
game Jeopardy!
Apple released Siri, the first
popular virtual assistant.
2015 OpenAI founded
2020, OpenAI announced
GPT-3
2021, OpenAI introduced
DALL-E
1950 Alan Turing “Computing
Machinery and Intelligence”
1997 IBM Deep Blue beat the
world chess champion, Gary
Kasparov.
8
– https://www.youtube.com/watch?v=056v4OxKwlI

9. GenAI
Generative AI (GenAI) refers to deep-learning models that can
generate high-quality text, images, and other content based on
the data they were trained on.
– https://research.ibm.com/blog/what-is-generative-AI

10. Generative AI
Anything
that creates
new content
Large language model
Great
at text
Foundation
model
Unlabeled
data
Transformer
ChatGPT
inspired interest…
But there is a
bigger concept,
e.g. GPT
Which will
change business
Building blocks of generative AI – Foundation Models
– BERT
– GPT
– Claude
– Cohere
– Stable Diffusion

11. 11
What is LLM?

12. 12
What is LLM?
LLM =Data + Architecture + Training
specialized in Texts

13. Foundation Models
Foundation Model AI system Applications
LaMDA (Google) Bard (Google) AI chat
GPT-4 (OpenAI) ChatGPT (OpenAI) AI Chat
Codex (OpenAI) GitHub copilot (Microsoft) Code generation
AudioLM (Google) MusicLM (Google) Create Music
BLOOM (Hugging Face) Use directly Mutiple NLP tasks. Trained in 46
languages and 13 programming
languages.
LLaMA (Meta) Use directly AI research
DALL-E 2 (OpenAI) Use directly Image creation
– https://www.datacamp.com/blog/what-are-foundation-models

14. – https://github.com/ibm-granite/granite-code-models

15. Comparison of LLMs
– https://research.ibm.com/blog/granite-code-models-open-source

16. Chat
Q&A
Summarization
Summarize info – meeting
minutes, etc
Content Generation
Create email, marketing
materials, etc.
Named Entity
Recognition
Produce audit data
Insight Extraction
Medical diagnose, etc.
Classification
Sort customer complainants,
security vulnerability
classification, etc.
The most common
generative AI tasks
implemented today

17. Issues related to AI
• License
• Audit
• Hallucinations
• Potentially generate bad code
• Security risk
• Lack of innovation
17

18. How to reduce LLM Hallucination
• Domain knowledge gaps
• Data out of date

19. Introducing…
RAG

20. Retrieval Augmented
generation (RAG)
An AI framework for improving the quality
of LLM-generated responses
Grounds a model on additional sources of
knowledge to supplement its internal
representation of information
33
RAG involves three basic steps:
Search for relevant content in your knowledge base
Pull the most relevant content into your model
prompt as context
Send the combined prompt text to the model to
generate output
1
2
3
Significantly elevates level of trust:
• Ensures that the model has access to the
most current and reliable facts
• System becomes "business-aware"
• Sources are known, ensuring output can be
checked for accuracy
• Less likely to make-up a factually inaccurate
responses, with ability to say, "I don't know."

21. RAG
components
Knowledge
Base
Can be any collection of information containing
artifacts such as:
• Internal procedural wiki pages
• Files in GitHub (various formats)
• Messages in a collaboration tool
• Topics in product documentation
• PDF files
• Customer support tickets
• more
Can be any combination of search and content
tools that reliably return relevant content from
a knowledge base (or bases):
• Search and content APIs like GitHub APIs
• Vector databases like Milvus
A generative LLM that suits your use case, prompt
format, and content being pulled in for context
Retriever
Generator

22. Typical RAG process
User Question
Search &
Retrieval
Prompt =
Instructions +
Search
Results +
Question
LLM
Generated
output with
sources
Top search
results

23. Data storage – using embedding and a vector database
Passages
of text
“Embeddings”
New step
Data storage process
(a) Original files to documents
(b) Documents to chunks
(c) Chunks to embeddings
(d) Embeddings to vector store
Vector
database
Semantic vs.
Syntactic match

24. Tasks AI will do for us
• Generate code snippet
• Create tests
• Debugging
• Code review
• Code summarization
• Refactoring
24

25. Some GenAI tools
Chatbot
– Anthropic’s Claude 2
– Google’s Bard
– Meta AI’s Hugging Face Llama 2 Chat
– Microsoft’s Bing Chat
– OpenAI’s ChatGPT
AI code assistant
Github Copilot
Amazon CodeWisperer
Divi AI
Tabnine
Replit
Sourcegraphy Cody
25
– https://www.elegantthemes.com/blog/wordpress/best-ai-coding-assistant#4-tabnine
– https://www.youtube.com/watch?v=TXtnFw9eDmM

26. Introducing…
AI-infused apps
– An app using LLMs

27. 27
Using LangChain4J to simplify
integrating with AI/LLM
capabilities into Java
applications

28. LangChain4J
28
– https://docs.langchain4j.dev/intro

29. LangChain4J example
29
– https://github.com/langchain4j/langchain4j-examples/tree/main/jakartaee-microprofile-example
– https://openliberty.io/blog/2024/04/01/open-liberty-with-langchain4j-example.html

30. AI-Powered Java Developer Checklist
1. What is AI, LLM? √
2. What is RAG? √
3. What code assistant tools available? √
4. What can help with building AI-Infused apps? √

31. 31
Join MicroProfile AI group to create a MicroProfile AI Spec
Monday weekly meeting
5pm CEST
Zoom: https://eclipse.zoom.us/j/83815795087

32. Crucial skills for Java
Developers
• Focus on the architecture
• Innovation
• Serviceability
• AI-infused apps
32

33. 33
AI-Powered Java Developers
won’t be replaced by AI!!!

34. 34
Thank You
Emily Jiang
IBM, Cloud Native Architect and Advocate
emijiang@uk.ibm.com
X/LinkedIn: @emilyfhjiang