Download to read offline
Uploaded byEdward Burns
langchain4j-cdi: Infuse your Jakarta and MicroProfile applications with all the AI
Generative AI burst on to the public scene in November 2022, over ten years after Andrew Ng’s Machine Learning premiered on Coursera. Java developers have long felt like second class citizens, compared to the cool Python kids. LangChain4J changes the game. Java is cool with AI now! This lecture from 30-year industry veteran Ed Burns explores why it took so long for Java developers to have access to easy-to-use AI libraries, compared to Python developers. This session introduces the new langchain4j-cdi repository within the LangChain4j GitHub organization. Combine the power of Jakarta EE with the reach of LangChain4j. All the idioms you know and love from Jakarta now work with all the AI exposed by LangChain4j: injection, validation, persistence, REST and more!
More Related Content
Similar to langchain4j-cdi: Infuse your Jakarta and MicroProfile applications with all the AI
langchain4j-cdi: Infuse your Jakarta and MicroProfile applications with all the AI
- 1. Orlando, FL October 6–9 IBMTechXchange 2025 Session code: 2788 Slide at https://aka.ms/txc-2025 Ed Burns Principal Architect for Java, Microsoft Emily Jiang IBM, STSM, Cloud Native Architect and Advocate langchain4j-cdi: Infuse your Jakarta and MicroProfile applications with all the AI
- 2. Ed Burns professionalbiography Client • NCSA Mosaic (1994) • SGI Cosmo Web Authoring • Sun Netscape 6 OJI Server J2EE JSF (2002) Oracle Java EE Servlet, JSF, Bean Validation, etc. Cloud Java Champion Jakarta EE 11 committer and release coordinator Langchain4j-cdi committer Books
- 3. Emily Jiang Java Champion MicroProfile Technical Lead MicroProfile Spec author Jakarta EE committer Langchain4j-cdi committer Book Author Fellowship BCS TechWomen100
- 4. Agenda 01 02 03 04 05 Background on JakartaEE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation
- 5. Agenda 01 02 03 04 05 Background on JakartaEE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation
- 6. Java SE Activation Java Productsand Specifications Tomcat Jetty Underto w Spring Framework Spring Boot WebSocket Concurrency JSON-B Validation Persistence Messaging Transactions Messaging Servlet Pages Tag Libraries Expression Language Annotations Authorization Faces REST JSON-P Interceptors Dependency Injection CDI Jakarta EE Core Profile Batch Connectors Mail EJB Security EJB Lite Authentication Data Debugging Support WebLogic Server WebSphere Liberty Open Liberty JBoss EAP WildFly WebSphere Traditional Quarkus Micronaut Helidon Jakarta EE Platform and Web Profile (aka Java EE/J2EE JEE) A note on Spring and Jakarta EE “Spring /Tomcat profile” (not an official name) “Spring /Tomcat profile” (not an official name) Config Fault Tolerance Metrics Health JWT RBAC Open API REST Client Telemetry MicroProfile Not Cloud Native Older Cloud Native Newer Cloud Native
- 7.
- 8.
- 9. Ed’s fairly obvious observations Recognize themarket forces that have always been in play. Vendors want adoption. They’ll make it easy to adopt. This is also true with AI.
- 10. James Gosling: Irealized if I wanted bankers to use automatic garbage collection, I needed to give them curly braces rather than lots of irritating silly parentheses (LISP). Packaging a new idea for adoption: deliver it on the back of existing ideas.
- 11. Agenda 01 02 03 04 05 Background on JakartaEE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation
- 12. AI-enhanced Java applications WhyJava? • Leverage skills you and your teams already have • Scalability, Security, Performance • Deeply integrate AI into the logic and flow of your apps AI-enhanced applications • Let users ask questions in natural language about documents, policies, etc. • Translate content between languages dynamically • Summarize meetings or transcripts • Answer user-provided information queries • Create specialized interactive research assistants
- 13. Create an AI-enhancedJava App • Simple LLM use in apps • Use an LLM to interact with provided input • Swap LLMs by changing configuration and/or dependencies • Structured Output • Directly integrate LLM responses with code lib Natural Language UI LLM
- 14. Create an AI-enhancedJava App • Retrieval Augmented Generation (RAG) • Use Retrieval Augmented Generation pattern to extend what LLM knows lib Embedding Computation Docs Natural Language UI LLM Splitting Embedding Computation Vector Store
- 15. What about AIAgents?
- 16. Essential Component ofAI Agents LLM Large language model that can comprehend and generate human language Actions and Tools AI Agents may require access to external tools, datasets, APIs, and automation systems to complete tasks. Tool calling enables LLMs to interface with structured tools, such as granting the model access to information beyond the data used in training. Memory Without memory, AI agents cannot remember the previous conversations. Session based context memory to memorize chat conversations. Structured knowledge bases, vector embeddings databases for long-term memory and retrieval of historical data. 1 6
- 17. Tool Tool Create an AI-enhancedJava App • Java Methods as Tools • Let LLM access tools lib Tool Natural Language UI LLM
- 18. Build AI Agentsusing Langchain4j
- 19. Agenda 01 02 03 05 06 Background on JakartaEE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation J
- 20. Langchain4j • Initially, inspiredby Langchain • A Java library to make interactions with AI models and LLMs • Provides a unified API to avoid the need for learning and implements specific APIs for each of them • Comprehensive toolbox - tools ranging from low-level prompt templating, chat memory management, and function calling to high-level patterns like Agents and RAG • Easy interaction with LLMs and Vector Stores https://github.com/langchain4j/
- 21. Langchain4j Architecture https://docs.langchain4j.dev/intro • Lowlevel: access to all the low-level components such as ChatModel, UserMessage, AiMessage, EmbeddingStore, Embedding, etc • High level: interact with LLMs using high- level APIs like AI Services, which hides all the complexity and boilerplate from you
- 22. Langchain4j AIService https://www.sivalabs.in/langchain4j-ai-services-tutorial/ • Createan interface without an implementation • Use AIServices to retrieve an object of the interface
- 23. Improvement • How toeasily configure with LLMs? • How to make the calling LLMs resilient? • How to add telemetry? https://github.com/langchain4j/
- 24. Introducing Jakarta EEwith AI via Langchain4j-cdi 24
- 25. Langchain4j-cdi with JakartaEE • Declare the AIService via @RegisterAIService (CDI stereotype). • Inject the AIService in other CDI beans via @Inject. Jakarta EE 10 and higher, JDK 17 support and higher. https://github.com/langchain4j/langchainej-cd i
- 26. Langchain4j-cdi with MicroProfile •Configuration (MicroProfile Config) • Low-level components are supported: ChatModel, ChatMesage, ChatMemory, EmbeddingStore, Embedding, tools https://github.com/langchain4j/langchainej-cdi
- 27. Langchain4j-cdi with MicroProfile •Fault Tolerance (MicroProfile Fault Tolerance) • @Timeout, @Fallback, @Retry, @CircuitBreaker, etc. • Observability (MicroProfile Telemetry) • OpenTelemetry AI Spans • OpenTelemetry AI Metrics https://github.com/langchain4j/langchain4j-cdi
- 28.
- 29. Agenda 01 02 03 04 05 Background on JakartaEE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation
- 30.
- 31. But what isMCP?
- 32. But what isMCP? https://modelcontextprotocol.io/
- 33. But what isMCP? https://github.com/modelcontextprotocol/servers • Model Context Protocol (MCP) is an open- source standard for connecting AI applications to external systems. • Using MCP, an AI enabled application can access tools and data from an MCP server. Not bound by LLM’s training cutoff. • An MCP server can be local, communicating over stdio*, or remote, communicating over Streamable-HTTP. MCP servers are composable.
- 34. MCP Architecture •MCP Host– an AI app that uses MCP Clients •MCP Client – a client to connect to an MCP Server •MCP Server – a server that acts as a tool registry and can invoke tools (and other MCP things…) MCP Server LLM AI App / MCP Host JSON-RPC Streaming HTTP MCP Client C1 MCP Client C2 MCP Client C3 MCP Tool T1 MCP Tool T2 Service S1 Service S2 Service S3 REST • MCP Tools should be built to return only the data needed (LLM performance is best with smaller responses) • MCP Tools should be built to return data in a format that is LLM- friendly (JSON is fine, or yaml, or simple text). Structured output can also be used. • MCP Tools should be built to combine common sets of actions into one
- 35.
- 36. Useful resources • LangChain4j •https://github.com/langchain4j/langchain4j • https://docs.langchain4j.dev/intro • https://docs.langchain4j.dev/category/tutorials • LangChain4j-cdi • https://github.com/langchain4j/langchain4j-cdi • Quarkus / LangChain4j • https://docs.quarkiverse.io/quarkus-langchain4j/dev/index.html • Quarkus AI tutorial • https://redhat-developer-demos.github.io/quarkus-tutorial/quarkus-tu torial/17_ai_intro.html • MCP for beginners • https://github.com/microsoft/mcp-for-beginners/ • Full Java GenAI app workshop • https://quarkus.io/quarkus-workshop-langchain4j
- 37. Agenda 01 02 03 04 05 Background on JakartaEE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation
- 38. Q&A Ed Burns Principal Architectfor Java, Microsoft 38 IBM TechXchange | © 2025 IBM Corporation Emily Jiang IBM, STSM, Cloud Native Architect and Advocate emijiang@uk.ibm.com
- 39. Thank you Emily Jiang IBM,STSM, Cloud Native Architect and Advocate emijiang@uk.ibm.com Ed Burns Principal Architect for Java, Microsoft
- 40. Notices and disclaimers IBM,the IBM logo, and ibm.com are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at: www.ibm.com/legal/copytrade.shtml. Certain comments made in this presentation may be characterized as forward looking under the Private Securities Litigation Reform Act of 1995. Forward-looking statements are based on the company’s current assumptions regarding future business and financial performance. Those statements by their nature address matters that are uncertain to different degrees and involve a number of factors that could cause actual results to differ materially. Additional information concerning these factors is contained in the Company’s filings with the SEC. Copies are available from the SEC, from the IBM website, or from IBM Investor Relations. Any forward-looking statement made during this presentation speaks only as of the date on which it is made. The company assumes no obligation to update or revise any forward-looking statements except as required by law; these charts and the associated remarks and comments are integrally related and are intended to be presented and understood together. © 2025 International Business Machines Corporation. All rights reserved. This document is distributed “as is” without any warranty, either express or implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. Case studies and client examples are presented as illustrations of how customers or IBM has used IBM products in production or test environments and the results they may have observed. Actual performance, cost, savings or other results in other operating environments may vary. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. Not all offerings are available in every country in which IBM operates. Any statements regarding IBM’s future direction, intent or product plans are subject to change or withdrawal without notice. IBM TechXchange | © 2025 IBM Corporation 40
Editor's Notes
- #2 Demo slickness disclaimer I’m not a developer advocate I am on the product team My slides and demos are not slick But I do work on the actual products being shown. I am not an AI expert If anyone tells you they are an AI expert, Ask them how they keep up with everything. Because that is the true expertise at this point. Put in a plug for the rockstar and Heather books.
- #3 Demo slickness disclaimer I’m not a developer advocate I am on the product team My slides and demos are not slick But I do work on the actual products being shown. I am not an AI expert If anyone tells you they are an AI expert, Ask them how they keep up with everything. Because that is the true expertise at this point. Put in a plug for the rockstar and Heather books.
- #6 Everything depends on Java SE in one way or another. “Snip corner” shapes are Java products Java products are broken down into “cloud native” (newer and older), and “not cloud native”. Spring Boot depends on Spring Framework. Spring Framework depends on: A subset of specifications from Jakarta EE, showed by the two dashed line boxes in the Jakarta EE oval box. Let’s call this, informally, the “Spring/Tomcat profile”. Spring provides its own proprietary way of handling the concerns addressed by all other aspects covered by Jakarta EE with one small exception: The Dependency Injection specification from the Jakarta EE Core Profile. Spring Framework does depend on this, but they create their own implementation that is tenuously compliant with the specification. Spring Framework depends on a “Servlet Container”. This can be Tomcat or Jetty. Tomcat depends on A subset of specifications from Jakarta EE, showed by the two dashed line boxes in the Jakarta EE oval box. Let’s call this, informally, the “Spring/Tomcat profile”. Micronaut depends on Annotations. WebLogic Server and WebSphere traditional depend on Jakarta EE Platform. Quarkus, Helidon, Liberty and EAP/WildFly depend on: MicroProfile Jakarta EE Web or Platform Profile, depending on circumstances MicroProfile depends on Jakarta EE Core Profile Jakarta EE Core Profile specifications are shown in a different color because some of the things depend on sub-specifications individually.
- #13 Simple LLM use in apps Java apps using an LLM to interact with provided input. Useful for something like a translation or summarization app. Ability to easily swap LLMs by just changing configuration (and possibly dependencies). Useful for development time when I’m trying to find the right LLM for my use case. Structured Output Java apps interacting with LLMs where the LLM responds in a way that is very easy to integrate with code. Useful for AI apps where input or output needs to be converted to a known format to interface with existing code.
- #14 Retrieval Augmented Generation (RAG) Java apps using the Retrieval Augmented Generation pattern. Useful for building chatbots that need access to current or proprietary info not known to LLMs at training time.
- #17 Tools Java apps that use tools. Useful for accessing data or taking actions based on input to an LLM.