DSS_ML_Creating GenAI Apps in Java with SD4J and the ONNX Runtime

Creating GenAI Apps in Java
with SD4J and the ONNX
Runtime
Juarez Barbosa Junior @juarezjunior
Sr Principal Java Developer Evangelist, Oracle

Juarez Barbosa Junior
Senior Principal Java Developer Evangelist
• Coming from Dublin, Ireland
• Developer since 1995 (Java ☕️ 1997)
• 28 years of experience in SW Engineering & DevRel
• Microsoft, Oracle, IBM, Nokia, Unisys, Accenture, startups
• Microsoft Azure Developer Relations Lead
• IBM Watson Tech Evangelist & Cloud Rockstar
• IBM Mobile Tech Evangelist & Global Thought Leader
• Nokia Developers Global Champion
• Java, Python, Cloud, DevOps, SRE, Cloud-native, IoT, AI,
Blockchain, Rust
• Speaker at conferences
• Oracle CloudWorld, Oracle Code, Microsoft Ignite, Microsoft
TechX, jPrime, JCON, DevConf.cz, GeeCon, DevOpsDays,
DeveloperWeek, DevOps Institute, CloudLand, DWX, The
Developer’s Conference (TDC), Sec4Dev, JSNation, NodeConf,
Conf42, Shift Conf, Global Azure, Open-Source Lisbon,
CodeFrenzy, Mêlée Numérique, React Summit, Test.js Summit,
Porto TechHub Conf, Pyjamas, JUGs, meetups, etc.
@juarezjunior
@juarezjunior

Agenda
• Stable Diffusion - Concepts
• Stable Diffusion for Java (SD4J) by
Oracle Labs
• ONNX and ONNX Runtime Extensions
• SD4J - Demo
• Java App Dev with the Oracle Database
• Technical References
• Oracle for Java Developers
Copyright © 2024, Oracle and/or its affiliates

Stable Diffusion - Concepts
• In a nutshell
• Stable Diffusion is a technique under the umbrella of GenAI used to generate
images primarily from textual descriptions.
• A diffusion model predicts the probability distribution of images using a
diffusion process.
• The diffusion process iteratively applies noise to an image, gradually
increasing the noise level and then reversing the process to reconstruct the
original image - denoising with guidance, aligning with the textual
description.
• One of the advantages of Stable Diffusion is its ability to control the diversity
and quality of generated images through the noise level parameter.
• The noise intensity allows users to adjust the image quality and diversity
trade-offs.

Stable Diffusion - Concepts
• In a nutshell
• Stable Diffusion finds applications in various domains, such as art, design, and
entertainment, by providing image synthesis, augmentation, and
manipulation.
• It can run on regular computers with good graphics cards, making it more
accessible.
• Most projects are Open-Source, allowing developers to experiment and fine-
tune the model to meet specific needs.

Stable Diffusion for Java
• SD4J
• A Java implementation of Stable Diffusion (modified port of
a C# implementation) running on top of ONNX Runtime
• Created by Oracle Labs (R&D) as OSS
• It targets ONNX Runtime 1.14
• It supports using CPUs and CUDA GPUs
• It supports SD v1.5, SD v2 and SDXL style models
• It provides a reusable class (com.oracle.labs.mlrg.sd4j.SD4J)
with a clear contract (interface) to allow the creation of
image generation pipelines

ONNX - Open Neural Network Exchange
• ONNX Format
• An open-source format designed to represent machine learning models.
• Expressed as a Protocol Buffers (protobuf) file containing a
computational graph representing the model.
• Models trained in one framework can be converted to the ONNX format
and used in another.
• The ONNX ecosystem includes a wide range of tools and libraries for
model conversion, optimization, inference, and visualization.
• While ONNX can represent models for training, its primary focus is on
inference, which is the process of using a trained model to make
predictions on new data.
• Promotes interoperability (frameworks, tools, runtime environments).
• Also promotes standardization, efficiency (HW), and ecosystem
development.

ONNX - Open Neural Network Exchange
• ONNX Runtime
• The ONNX Runtime is an optimized inference engine,
offering efficient execution of ONNX models with low latency
and high throughput.
• It leverages hardware acceleration capabilities, including GPU
acceleration through CUDA, and optimizations for CPUs.
• It provides tools for quantizing and optimizing ONNX models
to improve performance and reduce memory footprint,
making them more suitable for deployment on resource-
constrained devices.
• It is also cross-platform, as the ONNX Runtime supports
multiple operating systems, including Windows, Linux, and
macOS, making it versatile for deployment across different
target environments.

SD4J – Demo
• Creating GenAI Apps in Java with SD4J (Stable
Diffusion for Java) and the ONNX Runtime
• https://juarezjunior.medium.com/b72385c59329

Oracle Database 23ai Free available for
download now
• Oracle Database 23ai Free accelerates
Oracle’s mission to make it simple to
develop and run all data-driven apps
• Provides developers easy early access to
23ai AppDev features
• Limited database size, memory, and CPU
threads
http://oracle.com/23aiFree
http://oracle.com/23aiFree

SD4J – Demo with the Oracle Database 23ai
• Code walkthrough, SD4J class, and integration with
the Oracle Database

Java App
Dev with
Oracle
Database

Java Database Connectivity with the Oracle
Database
• Java App Development with the Oracle Database
• Converged Database (Vector, Relational, NoSQL, Graph,
Blockchain, XML, Spatial, JSON) and Java App Dev with
Oracle Database
• Oracle JDBC - Support for the Latest Java Versions
• Overview of DB access with Java - JDBC (with Virtual
Threads), GraalVM, Reactive Streams, R2DBC, Reactive
Streams Ingestion Library (RSI), Oracle UCP, Pipelined
Database Operations
• Synchronous and Asynchronous UCs

Overview of Oracle DB Access with Java

Oracle JDBC - Support for the Latest Java
Versions
• Support to the latest Java versions
(11, 17, 21) and GraalVM
• Project Loom - Virtual Threads
support
• Java Virtual Threads and Pipelined
Database Operations
• JDBC Standards - 4.2 and 4.3
• Reactive Streams - Java Flow API
support
• R2DBC, Reactive Streams Ingestion
(RSI)
• Pipelined Database Operations
Data access is critical in mission-
critical applications!

Technical References
• Stable Diffusion
• A latent text-to-image diffusion model - https://github.com/CompVis/stable-diffusion
• Stable Diffusion for Java (SD4J)
• SD4J by Oracle Labs - https://github.com/oracle/sd4j
• ONNX Runtime
• ONNX Runtime - https://onnxruntime.ai/
• Develop Java applications with Oracle Database
• JDBC – https://www.oracle.com/database/technologies/appdev/jdbc.html
• Introduction to Oracle JDBC Driver Support for Virtual Threads - https://bit.ly/3UlNJWP
• Developing an Oracle JDBC app with GraalVM Native Image - https://rb.gy/iy3sgh
• Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC
Driver - https://rb.gy/42dnz5
• Getting Started with the Java library for Reactive Streams Ingestion (RSI) -
https://bit.ly/3rEiRnC
• Introduction to JDBC Reactive Extensions with the Oracle Database 23c Free — Developer
Release – https://rb.gy/qxlrbx
• Pipelined Database Operations - https://rb.gy/iy3sgh

Oracle LiveLabs
Showcasing how Oracle’s solutions can
solve your business problems
500+
free workshops,
available or in
development
3.5 million
people have already visited
LiveLabs
developer.oracle.com/livelabs
learn something new …at your pace!
600+
events run
using LiveLabs
workshops

3 membership tiers
Connect: @oracleace facebook.com/OracleACEs
aceprogram_ww@oracle.com
500+ technical experts &
community leaders helping peers globally
The Oracle ACE Program recognizes & rewards individuals for
their technical & community contributions to the Oracle community
Nominate
yourself or a candidate:
ace.oracle.com/nominate
Learn more - ace.oracle.com
blogs.oracle.com/ace

DSS_ML_Creating GenAI Apps in Java with SD4J and the ONNX Runtime

DSS_ML_Creating GenAI Apps in Java with SD4J and the ONNX Runtime

More Related Content

Similar to DSS_ML_Creating GenAI Apps in Java with SD4J and the ONNX Runtime

More from Juarez Junior

Recently uploaded

DSS_ML_Creating GenAI Apps in Java with SD4J and the ONNX Runtime