Silicon Valley JUG - How to generate customized java 8 code from your database

How to Generate Customized Java 8 Code
From Your Database
Per Minborg
CTO, Speedment, Inc
Emil Forslund
Developer, Speedment, Inc.

Every Decision a Developer Makes is a
Trade-off
The best code is
no code at all

Using Code Generation
• Makes the code efficient
and short
• Modifications are done
once and applied
everywhere
• Minimizes errors
• “DRY” (Don’t Repeat
Yourself) vs. ”WET” (We
Enjoy Typing)
• “Code your code”
But how can we control the generated code?

About Us
Per Minborg
• Founder of several IT companies
• Lives in Palo Alto
• 20 years of Java experience
• 15+ US patents
• Speaker at Java events
• Blog: Minborg’s Java Pot
Emil Forslund
• Java Developer
• Lives in Palo Alto
• 8 years of Java
experience
• Speaker at Java events
• Blog: Age of Java
Spire
• Speedment Open Source mascot
• Lives on GitHub
• 2 years of mascot experience

Agenda
• Problem Description
• Code Generation in Database Applications
• How Does it Work?
• Code Example
• Controlling the Code Generation
• Generate Custom Classes
• Add new Method to Existing Classes
• Code Example
• Additional Features
• Beyond Open Source
• Questions & Answers

Do You Recognize This Code?
Class.forName("org.postgresql.Driver");
try (final Connection conn = DriverManager.getConnection(
"jdbc:postgresql://hostname:port/dbname",
"username",
"password")) {
// Database Logic Here...
}

Why Creating DB-Apps is So Time
Consuming
• Even trivial database operations require a
lot of boilerplate code
• Mixing SQL and Java is error-prone
• ORMs require you to write annotated
POJOs for every table
• Creating even a simple DB app can take
hours

Open-Source Project Speedment
• Stream ORM Java toolkit and
runtime
• Generate domain-model from the
database
• No need for complicated
configurations or setup
• All operations are type-safe
• Data is accessed using Java 8
Streams
• Business friendly Apache 2-
license

Speedment Workflow
customers.stream()
.filter(…)
.filter(…)
.map(…)
.collect(toList());
Step 1: Generate
Code
Step 2: Write Logic Step 3: Run
Application
Step 4: Iterate

Artifacts
• com.speedment:
• runtime
• generator
• tool
• speedment-maven-plugin

So How Do the Generated Code Work?
• Code is organized based on database structure
• Hash-sums make sure user changes are not overwritten
If the DB structure changes, the code is
updated with the press of a button

Entities, Managers and Applications
• An Entity represents a row in a table
• Is a POJO
• Customer
• CustomerImpl
• A Manager represents a table
• Responsible for the CRUD operations
• CustomerManager
• CustomerManagerImpl
• An Application represents the entire project
• Responsible for configuration and settings
• SalesApplication
• SalesApplicationBuilder

Querying the Database using Streams
• Queries are expressed using
the standard Java 8 Stream
API
• Streams are analyzed to
produce high-performance
queries

Expressing Queries as Streams
customers.stream()
.filter(Customer.REGION.equal(Region.NORTH_AMERICA))
.filter(Customer.REGISTERED.greaterOrEqual(startOfYear))
.count();
Standard Stream
API
Generated Enum
Constants
Only 1 value is loaded from
DB
Full Type-Safety
SELECT COUNT('id') FROM 'customer'
WHERE 'customer'.'region' = ‘North America’
AND 'customer'.'registered' >= ‘2016-01-01’;

SELECT * FROM 'customer'
REGION.equal(NORTH_AMERICA)
REGISTERED.greaterOrEqual(2016-01-01)
count()
Sourc
e
Filter
Filter
Term.
Pipeline

REGION.equal(NORTH_AMERICA)
count()
Sourc
e
Filter
Filter
Term.
Pipeline

count()
Sourc
e
Filter
Term.
Pipeline

count()
Sourc
e
Term.
Pipeline

Sourc
e
Pipeline

// Gets the second page of customers in North America
// sorted by name in the form of a JSON array
customers.stream()
.filter(REGION.equal(Region.NORTH_AMERICA))
.sorted(NAME.comparator())
.skip(10)
.limit(10) // JVM from here…
.collect(toJson(encode.allOf(customers)))
[
{”id”:11, ”name”: …},
{…},
…
]

// Supports parallelism on custom executors
// with full control of thread work item layout
customers.stream()
.parallel()
.filter(REGION.equal(Region.NORTH_AMERICA))
.forEach(expensiveOperatation());

Application Example
• Total Count of Customers
• Located in North America
• Registered This Year

Step 1: Getting Speedment using Maven
<plugin>
<groupId>com.speedment</groupId>
<artifactId>speedment-maven-plugin</artifactId>
<version>3.0.3</version>
</plugin>
Generate source files based on database

Step 2: Initializing Speedment
SalesApplication app = new SalesApplicationBuilder()
.withPassword("qwerty")
.build();
CustomerManager customers = app.getOrThrow(CustomerManager.class);
These classes are generated
automatically
Instance is configured using Builder-
pattern
A manager class is generated for every database table

Step 3: Querying
Region fromWhere = Region.NORTH_AMERICA;
Instant fromWhen = Instant.parse("2016-01-01");
long count = customers.stream()
.filter(Customer.REGION.equal(fromWhere))
.filter(Customer.REGISTERED.greaterOrEqual(fromWhen))
.count();

Step 4: Output
System.out.println(
"A total of %d customers from %s " +
"have registered this year.",
count, fromWhere.name()
);

Full Application
public static void main(String… args) {
SalesApplication app = new SalesApplicationBuilder()
.withPassword("qwerty")
.build();
CustomerManager customers = app.getOrThrow(CustomerManager.class);
Region fromWhere = Region.NORTH_AMERICA;
Instant fromWhen = Instant.parse("2016-01-01");
long count = customers.stream()
.filter(Customer.REGION.equal(fromWhere))
.filter(Customer.REGISTERED.greaterOrEqual(fromWhen))
.count();
System.out.println(
"A total of %d customers from %s " +
"have registered this year.",
count, fromWhere.name()
);
}

Output
Pers-MacBook-Pro:~ pemi$ java –jar salesapplication.jar
A total of 354 customers from NORTH_AMERICA have registered this year.

Controlling the Code Generation
• So far we have queried a
database with streams
• We have used code generation
to create entities and managers
• Can it be used for more?

What is Available out of the Box?
• MVC oriented code generation
• Modular design
• Database domain model
• JSON configuration (DSL)
• Java language namer
• Translator and TranslatorDecorator
• Maven goals
• Type mappers

MVC Oriented Code Generation
• Model
• File, Class, Interface, Enum, Field, Method, Constructor,
Type, Generic, Annotation, …
• View
• Renders a model to Java (or another language)
• Set code style using custom views
• Control
• AutoImport, AutoEquals, AutoJavadoc, SetGetAdd, FinalParameters
• Write custom controllers to automate recurring tasks

MVC Oriented Code Generation
• Separation of concerns
• Code generation is type safe
• Catch errors compile time
• Discover methods directly in the IDE
• Reuse code segments and controllers

Database Domain Model
• Project
• Dbms
• Schema
• Table
• Column
• PrimaryKey
• ForeignKey
• Index
List<Table> tables = project.dbmses()
.flatMap(Dbms::schemas)
.flatMap(Schema::tables)
.collect(toList());

JSON Configuration (DSL)
{
"config" : {
"name" : "sales",
"dbmses" : [{
"name" : "db0",
"typeName" : "MySQL",
"ipAddress" : "127.0.0.1",
"username" : "root",
"schemas" : [{
"name" : "sales",
"tables" : [
{ "name" : "city" },
{ "name" : "salesperson" }
]
}]
}]
}
}

Java Language Namer
• Camel caser: converts from “some_db_name” to
“someDbName”
• Java naming conventions: user, User and USER
• Detects keywords like “final”,”static” and escapes them
• Detects collisions
• Pluralizer: bag → bags, entity → entities, fish → fish

Translator and TranslatorDecorator
• Translator
• Renders a DB entity like a Table to a new Class or an Interface
• TranslatorDecorator
• Modifies an existing Class or Interface

Maven Goals
• speedment:tool
• Launches the graphical tool
• Allows customization of configuration model
• Code generation
• speedment:generate
• Code generation without launching the tool
• speedment:reload
• Reloads database metadata without launching the tool
• speedment:clear
• Removes all the generated classes (except manual changes) without launching the
tool

Type Mappers
• Controls how columns are implemented
• Runtime conversion between Database and Java types
java.sql.Timestamp long

Generation vs. Templates
• Separation of concerns
• Easily change code style
• Minimize maintenance
• Maximize reusability

Generate a New Custom Class
1. Create a new Translator
2. Model how the new class
should look
3. Define a Plugin Class
4. Include it in project pom.xml
Example: Generate a Point class

Step 1: Create a New Translator Class
public class PointTranslator extends
AbstractJavaClassTranslator<Project, Class> {
public final static TranslatorKey<Project, Class> POINT_KEY =
new TranslatorKey.of(“generated_point", Class.class);
public PointTranslator(Project document) { super(document, Class::of); }
@Override
protected Class makeCodeGenModel(File file) {
return newBuilder(file, getClassOrInterfaceName())
.forEveryProject((clazz, project) -> {
// Generate Content Here
}).build();
}
@Override
protected String getClassOrInterfaceName() { return ”Point"; }
@Override
protected String getJavadocRepresentText() { return "A 2-dimensional coordinate."; }
}
Every translator is identified by a
TranslatorKey
Name of generated class
Javadoc
Called every time the translator is invoked
forEvery(Project|Dbms|Schema|Table|Column|
…)

Step 2: The makeCodeGenModel -
methodclazz.public_()
.add(Field.of(“x”, int.class)
.private_().final_()
)
.add(Field.of(“y”, int.class)
.private_().final_()
)
.add(Constructor.of().public_()
.add(Field.of(“x”, int.class))
.add(Field.of(“y”, int.class))
.add(“this.x = x;”,
“this.y = y;”
)
)
.add(Method.of(“getX”, int.class).public_()
.add(“return x;”)
)
.add(Method.of(“getY”, int.class).public_()
.add(“return y;”)
)
.call(new AutoEquals<>())
.call(new AutoToString<>());

Step 3: Define a Plugin Class
public class PointPlugin {
@ExecuteBefore(RESOLVED)
void install(CodeGenerationComponent codeGen) {
codeGen.put(
Project.class,
PointTranslator.POINT_KEY,
PointTranslator::new
);
}
}
The key defined earlier
Will execute when Speedment is
being initialized
How the translator is constructed

Step 4: Include it in Project pom.xml
<plugin>
<dependencies>
<dependency>
<groupId>com.example</groupId>
<artifactId>point-plugin</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>
</dependencies>
<configuration>
<components>
<component>com.example.pointplugin.PointPlugin</component>
</components>
</configuration>
</plugin>
This tells Speedment to load the plugin
Make sure our plugin project is on the classpath

Execute
/**
* A 2-dimensional coordinate.
* <p>
* This file is safe to edit. It will not be overwritten by
the code generator.
*
* @author company
*/
public class Point {
private final int x;
private final int y;
public Point(int x, int y) {
this.x = x;
this.y = y;
}
public int getX() {
return x;
}
public int getY() {
return y;
}
The following file is generated: public int hashCode() {
int hashCode = 31;
hashCode += 41 * x;
hashCode += 41 * y;
return hashCode;
}
public Boolean equals(Object other) {
if (this == other) return true;
else if (other == null) return false;
else if (!(other instanceof Point)) {
return false;
}
final Point point = (Point) other;
return x == point.x && y == point.y;
}
public String toString() {
return new StringBuilder(“Point{”)
.append(“x: “).append(x).append(“, “)
.append(“y: “).append(y).append(“}”);
}
}

A More Concrete Example
1. Create a new Translator
2. Model how the new class
should look
3. Define a Plugin Class
4. Include it in project pom.xml
Example: Generate an Enum of
tables in the database

Step 1: Create a New Translator Class
public class TableEnumTranslator extends
AbstractJavaClassTranslator<Project, Enum> {
public final static TranslatorKey<Project, Enum> TABLES_ENUM_KEY =
new TranslatorKey.of(“tables_enum", Enum.class);
public TableEnumTranslator(Project document) { super(document, Enum::of); }
@Override
protected Enum makeCodeGenModel(File file) {
return newBuilder(file, getClassOrInterfaceName())
.forEveryProject((clazz, project) -> {
// Generate Content Here
}).build();
}
@Override
protected String getClassOrInterfaceName() { return ”Tables"; }
@Override
protected String getJavadocRepresentText() { return "An enumeration of tables in the database."; }
}
Every translator is identified by a
TranslatorKey
Name of generated class
Javadoc
Called every time the translator is invoked
forEvery(Project|Dbms|Schema|Table|Column|
…)

Step 2: The makeCodeGenModel -
method
DocumentDbUtil.traverseOver(project, Table.class)
.map(Table::getJavaName)
.map(getSupport().namer()::javaStaticFieldName)
.sorted()
.map(EnumConstant::of)
.forEachOrdered(clazz::add);

Step 3: Define a Plugin Class
public class TableEnumPlugin {
protected void install(CodeGenerationComponent codeGen) {
codeGen.put(
Project.class,
TableEnumTranslator.TABLES_ENUM_KEY,
TableEnumTranslator::new
);
}
}

Step 4: Include it in Project pom.xml
<plugin>
<dependencies>
<dependency>
<groupId>com.example</groupId>
<artifactId>table-enum-plugin</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>
</dependencies>
<configuration>
<components>
<component>com.example.tableenumplugin.TableEnumPlugin</component>
</components>
</configuration>
</plugin>
This tells Speedment to load the plugin
Make sure our plugin project is on the
classpath

Execute
/**
* An enumeration of tables in the database.
* <p>
* This file is safe to edit. It will not be overwritten by the code generator.
*
* @author company
*/
enum Tables {
CITY, SALESPERSON;
}
The following file is generated:

Generate all the Things
• Gson Adapters
• Spring Configuration Files
• REST Controllers
• and much more…

Add New Method to Existing Classes
• So far we have
• Queried a database with generated
classes
• Generated a custom class
• How do we modify the existing
generation of classes?

• Fit Into Existing Class Hierarchy
• Change Naming Conventions
• Optimize Internal Implementations
• Add Custom Methods

Example: Add a getColumnCount
method to generated managers
1. Create a new
TranslatorDecorator class
2. Write code generation logic
3. Add it to Speedment

Step 1: Creating a New Decorator Class
public class ColumnCountDecorator implements TranslatorDecorator<Table, Interface> {
@Override
public void apply(JavaClassTranslator<Table, Interface> translator) {
translator.onMake(builder -> {
builder.forEveryTable((intrf, table) -> {
// Code generation logic goes here
});
});
}
}

Step 2: Write Code Generation Logic
int columnCount = table.columns().count();
intrf.add(Method.of("getColumnCount", int.class)
.default_()
.set(Javadoc.of("Returns the number of columns in this table.")
.add(RETURN.setValue("the column count"))
)
.add("return " + columnCount + ";")
);

Step 3: Add it to Speedment
public final class TableEnumPlugin {
protected void install(CodeGenerationComponent codeGen) {
codeGen.put(
Project.class,
TableEnumTranslator.TABLES_ENUM_KEY,
TableEnumTranslator::new
);
codeGen.add(
Table.class,
StandardTranslatorKey.GENERATED_MANAGER,
new ColumnCountDecorator()
);
}
}
Modify an existing translator key
Our new decorator
The same plugin class as we
created earlier

Execute
When the project is
regenerated, a new method is
added to each manager

Additional Features
• Add GUI Tool components for
custom configuration
• Extend the JSON DSL
dynamically with plugins
• Automate your build
environment with Maven
• Add custom data type
mappers

Additional Features
• Plugin a custom namer
• Generate classes that are related to other domain models
• Generate classes for other languages
• Style the GUI Tool

Database Connectors
Open Source
• MySQL
• PostgreSQL
• MariaDB
Enterprise
• Oracle
• Microsoft SQL server
• DB2
• AS400

Beyond Open-Source
• Speedment Enterprise
• Run your database queries orders of magnitude faster!
• 10x, 100x, 1 000x, 10 000x, …
• Use the same Stream API

How is This Possible?
• No changes to user-written code
• Alternative stream source
• Data is stored in-memory
• Generates optimized serializers
for offheap storage

In-JVM-Memory, JMH Benchmarks(*)
Benchmark Mode Cnt Score Error Units
SpeedmentBenchmark.findStartsWith avgt 200 ≈ 10⁻⁵ s/op
SpeedmentBenchmark.findStartsWithSql avgt 200 ≈ 10⁻⁴ s/op
SpeedmentBenchmark.join avgt 200 0.476 ± 0.005 s/op
SpeedmentBenchmark.joinSql avgt 200 5.174 ± 0.010 s/op
SpeedmentBenchmark.sort avgt 200 ≈ 10⁻⁶ s/op
SpeedmentBenchmark.sortSql avgt 200 ≈ 10⁻⁴ s/op
SpeedmentBenchmark.scrollSorted avgt 200 ≈ 10⁻⁵ s/op
SpeedmentBenchmark.scrollSortedSql avgt 200 24.661 ± 0.670 s/op
SpeedmentBenchmark.count avgt 180 ≈ 10⁻⁸ s/op
SpeedmentBenchmark.countSql avgt 200 5.143 ± 0.012 s/op
(*) Preliminary results, Mac Book Pro, 2.2 GHz i7, 16GB, MySQL 5.7.16 standard
with indexes on all relevant columns.
10x
>10x
100x
2,000,000x
500,000,000x

Speedment Enterprise Example
Courtesy of Extremely Heavy Industries

Free Enterprise Trial
Today we have released version 1.1.1
Get a free trial! Leave your contact information.

Q&A
www.speedment.org
github.com/speedment/speedment
@speedment
www.speedment.com
Linkedin:
Per Minborg
Emil Forslund

Silicon Valley JUG - How to generate customized java 8 code from your database

More Related Content

What's hot

Similar to Silicon Valley JUG - How to generate customized java 8 code from your database

More from Speedment, Inc.

Recently uploaded

Silicon Valley JUG - How to generate customized java 8 code from your database

Editor's Notes