Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to generate customized java 8 code from your database

939 views

Published on

Did you know that database classes, that require many lines of Java and SQL code, may be replaced with a single line of Java 8 code? In this tutorial session you will learn how to use standard Java 8 Streams as an alternative to traditional Object Relational Mappers (ORM). We will use the open-source tool Speedment to show how development speed can be increased and how the application code can be more concise and run faster.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

How to generate customized java 8 code from your database

  1. 1. How to Generate Customized Java 8 Code From Your Database Per Minborg CTO, Speedment, Inc Emil Forslund Developer, Speedment, Inc.
  2. 2. Every Decision a Developer Makes is a Trade-off The best code is no code at all
  3. 3. Using Code Generation • Makes the code efficient and short • Modifications are done once and applied everywhere • Minimizes errors • “DRY” (Don’t Repeat Yourself) vs. ”WET” (We Enjoy Typing) • “Code your code” But how can we control the generated code?
  4. 4. About Us Per Minborg • Founder of several IT companies • Lives in Palo Alto • 20 years of Java experience • 15+ US patents • Speaker at Java events • Blog: Minborg’s Java Pot Emil Forslund • Java Developer • Lives in Palo Alto • 8 years of Java experience • Speaker at Java events • Blog: Age of Java Spire • Speedment Open Source mascot • Lives on GitHub • 2 years of mascot experience
  5. 5. Agenda • Problem Description • Code Generation in Database Applications • How Does it Work? • Code Example • Controlling the Code Generation • Generate Custom Classes • Add new Method to Existing Classes • Code Example • Additional Features • Beyond Open Source • Questions & Answers
  6. 6. Agenda • Problem Description • Code Generation in Database Applications • How Does it Work? • Code Example • Controlling the Code Generation • Generate Custom Classes • Add new Method to Existing Classes • Code Example • Additional Features • Beyond Open Source • Questions & Answers
  7. 7. Do You Recognize This Code? Class.forName("org.postgresql.Driver"); try (final Connection conn = DriverManager.getConnection( "jdbc:postgresql://hostname:port/dbname", "username", "password")) { // Database Logic Here... }
  8. 8. Why Creating DB-Apps is So Time Consuming • Even trivial database operations require a lot of boilerplate code • Mixing SQL and Java is error-prone • ORMs require you to write annotated POJOs for every table • Creating even a simple DB app can take hours
  9. 9. Open-Source Project Speedment • Stream ORM Java toolkit and runtime • Generate domain-model from the database • No need for complicated configurations or setup • All operations are type-safe • Data is accessed using Java 8 Streams • Business friendly Apache 2- license
  10. 10. Speedment on GitHub
  11. 11. Speedment Workflow customers.stream() .filter(…) .filter(…) .map(…) .collect(toList()); Step 1: Generate Code Step 2: Write Logic Step 3: Run Application Step 4: Iterate
  12. 12. Tool
  13. 13. Artifacts • com.speedment: • runtime • generator • tool • speedment-maven-plugin
  14. 14. Agenda • Problem Description • Code Generation in Database Applications • How Does it Work? • Code Example • Controlling the Code Generation • Generate Custom Classes • Add new Method to Existing Classes • Code Example • Additional Features • Beyond Open Source • Questions & Answers
  15. 15. So How Do the Generated Code Work? • Code is organized based on database structure • Hash-sums make sure user changes are not overwritten If the DB structure changes, the code is updated with the press of a button
  16. 16. Entities, Managers and Applications • An Entity represents a row in a table • Is a POJO • Customer • CustomerImpl • A Manager represents a table • Responsible for the CRUD operations • CustomerManager • CustomerManagerImpl • An Application represents the entire project • Responsible for configuration and settings • SalesApplication • SalesApplicationBuilder
  17. 17. Querying the Database using Streams • Queries are expressed using the standard Java 8 Stream API • Streams are analyzed to produce high-performance queries
  18. 18. Expressing Queries as Streams customers.stream() .filter(Customer.REGION.equal(Region.NORTH_AMERICA)) .filter(Customer.REGISTERED.greaterOrEqual(startOfYear)) .count(); Standard Stream API Generated Enum Constants Only 1 value is loaded from DB Full Type-Safety SELECT COUNT('id') FROM 'customer' WHERE 'customer'.'region' = ‘North America’ AND 'customer'.'registered' >= ‘2016-01-01’;
  19. 19. Querying the Database using Streams SELECT * FROM 'customer' REGION.equal(NORTH_AMERICA) REGISTERED.greaterOrEqual(2016-01-01) count() Sourc e Filter Filter Term. Pipeline
  20. 20. Querying the Database using Streams SELECT * FROM 'customer' WHERE 'customer'.'region' = ‘North America’ REGION.equal(NORTH_AMERICA) REGISTERED.greaterOrEqual(2016-01-01) count() Sourc e Filter Filter Term. Pipeline
  21. 21. Querying the Database using Streams SELECT * FROM 'customer' WHERE 'customer'.'region' = ‘North America’ AND 'customer'.'registered' >= ‘2016-01-01’; REGISTERED.greaterOrEqual(2016-01-01) count() Sourc e Filter Term. Pipeline
  22. 22. Querying the Database using Streams SELECT COUNT('id') FROM 'customer' WHERE 'customer'.'region' = ‘North America’ AND 'customer'.'registered' >= ‘2016-01-01’; count() Sourc e Term. Pipeline
  23. 23. Querying the Database using Streams SELECT COUNT('id') FROM 'customer' WHERE 'customer'.'region' = ‘North America’ AND 'customer'.'registered' >= ‘2016-01-01’; Sourc e Pipeline
  24. 24. Expressing Queries as Streams // Gets the second page of customers in North America // sorted by name in the form of a JSON array customers.stream() .filter(REGION.equal(Region.NORTH_AMERICA)) .sorted(NAME.comparator()) .skip(10) .limit(10) // JVM from here… .collect(toJson(encode.allOf(customers))) [ {”id”:11, ”name”: …}, {…}, … ]
  25. 25. Expressing Queries as Streams // Supports parallelism on custom executors // with full control of thread work item layout customers.stream() .parallel() .filter(REGION.equal(Region.NORTH_AMERICA)) .forEach(expensiveOperatation());
  26. 26. Application Example • Total Count of Customers • Located in North America • Registered This Year
  27. 27. Step 1: Getting Speedment using Maven <plugin> <groupId>com.speedment</groupId> <artifactId>speedment-maven-plugin</artifactId> <version>3.0.3</version> </plugin> Generate source files based on database
  28. 28. Step 2: Initializing Speedment SalesApplication app = new SalesApplicationBuilder() .withPassword("qwerty") .build(); CustomerManager customers = app.getOrThrow(CustomerManager.class); These classes are generated automatically Instance is configured using Builder- pattern A manager class is generated for every database table
  29. 29. Step 3: Querying Region fromWhere = Region.NORTH_AMERICA; Instant fromWhen = Instant.parse("2016-01-01"); long count = customers.stream() .filter(Customer.REGION.equal(fromWhere)) .filter(Customer.REGISTERED.greaterOrEqual(fromWhen)) .count();
  30. 30. Step 4: Output System.out.println( "A total of %d customers from %s " + "have registered this year.", count, fromWhere.name() );
  31. 31. Full Application public static void main(String… args) { SalesApplication app = new SalesApplicationBuilder() .withPassword("qwerty") .build(); CustomerManager customers = app.getOrThrow(CustomerManager.class); Region fromWhere = Region.NORTH_AMERICA; Instant fromWhen = Instant.parse("2016-01-01"); long count = customers.stream() .filter(Customer.REGION.equal(fromWhere)) .filter(Customer.REGISTERED.greaterOrEqual(fromWhen)) .count(); System.out.println( "A total of %d customers from %s " + "have registered this year.", count, fromWhere.name() ); }
  32. 32. Output Pers-MacBook-Pro:~ pemi$ java –jar salesapplication.jar A total of 354 customers from NORTH_AMERICA have registered this year.
  33. 33. Agenda • Problem Description • Code Generation in Database Applications • How Does it Work? • Code Example • Controlling the Code Generation • Generate Custom Classes • Add new Method to Existing Classes • Code Example • Additional Features • Beyond Open Source • Questions & Answers
  34. 34. Controlling the Code Generation • So far we have queried a database with streams • We have used code generation to create entities and managers • Can it be used for more?
  35. 35. What is Available out of the Box? • MVC oriented code generation • Modular design • Database domain model • JSON configuration (DSL) • Java language namer • Translator and TranslatorDecorator • Maven goals • Type mappers
  36. 36. MVC Oriented Code Generation • Model • File, Class, Interface, Enum, Field, Method, Constructor, Type, Generic, Annotation, … • View • Renders a model to Java (or another language) • Set code style using custom views • Control • AutoImport, AutoEquals, AutoJavadoc, SetGetAdd, FinalParameters • Write custom controllers to automate recurring tasks
  37. 37. MVC Oriented Code Generation • Separation of concerns • Code generation is type safe • Catch errors compile time • Discover methods directly in the IDE • Reuse code segments and controllers
  38. 38. Modular Design
  39. 39. Database Domain Model • Project • Dbms • Schema • Table • Column • PrimaryKey • ForeignKey • Index List<Table> tables = project.dbmses() .flatMap(Dbms::schemas) .flatMap(Schema::tables) .collect(toList());
  40. 40. JSON Configuration (DSL) { "config" : { "name" : "sales", "dbmses" : [{ "name" : "db0", "typeName" : "MySQL", "ipAddress" : "127.0.0.1", "username" : "root", "schemas" : [{ "name" : "sales", "tables" : [ { "name" : "city" }, { "name" : "salesperson" } ] }] }] } }
  41. 41. Java Language Namer • Camel caser: converts from “some_db_name” to “someDbName” • Java naming conventions: user, User and USER • Detects keywords like “final”,”static” and escapes them • Detects collisions • Pluralizer: bag → bags, entity → entities, fish → fish
  42. 42. Translator and TranslatorDecorator • Translator • Renders a DB entity like a Table to a new Class or an Interface • TranslatorDecorator • Modifies an existing Class or Interface
  43. 43. Maven Goals • speedment:tool • Launches the graphical tool • Allows customization of configuration model • Code generation • speedment:generate • Code generation without launching the tool • speedment:reload • Reloads database metadata without launching the tool • speedment:clear • Removes all the generated classes (except manual changes) without launching the tool
  44. 44. Type Mappers • Controls how columns are implemented • Runtime conversion between Database and Java types java.sql.Timestamp long
  45. 45. Generation vs. Templates • Separation of concerns • Easily change code style • Minimize maintenance • Maximize reusability
  46. 46. Generate a New Custom Class 1. Create a new Translator 2. Model how the new class should look 3. Define a Plugin Class 4. Include it in project pom.xml Example: Generate a Point class
  47. 47. Step 1: Create a New Translator Class public class PointTranslator extends AbstractJavaClassTranslator<Project, Class> { public final static TranslatorKey<Project, Class> POINT_KEY = new TranslatorKey.of(“generated_point", Class.class); public PointTranslator(Project document) { super(document, Class::of); } @Override protected Class makeCodeGenModel(File file) { return newBuilder(file, getClassOrInterfaceName()) .forEveryProject((clazz, project) -> { // Generate Content Here }).build(); } @Override protected String getClassOrInterfaceName() { return ”Point"; } @Override protected String getJavadocRepresentText() { return "A 2-dimensional coordinate."; } } Every translator is identified by a TranslatorKey Name of generated class Javadoc Called every time the translator is invoked forEvery(Project|Dbms|Schema|Table|Column| …)
  48. 48. Step 2: The makeCodeGenModel - methodclazz.public_() .add(Field.of(“x”, int.class) .private_().final_() ) .add(Field.of(“y”, int.class) .private_().final_() ) .add(Constructor.of().public_() .add(Field.of(“x”, int.class)) .add(Field.of(“y”, int.class)) .add(“this.x = x;”, “this.y = y;” ) ) .add(Method.of(“getX”, int.class).public_() .add(“return x;”) ) .add(Method.of(“getY”, int.class).public_() .add(“return y;”) ) .call(new AutoEquals<>()) .call(new AutoToString<>());
  49. 49. Step 3: Define a Plugin Class public class PointPlugin { @ExecuteBefore(RESOLVED) void install(CodeGenerationComponent codeGen) { codeGen.put( Project.class, PointTranslator.POINT_KEY, PointTranslator::new ); } } The key defined earlier Will execute when Speedment is being initialized How the translator is constructed
  50. 50. Step 4: Include it in Project pom.xml <plugin> <groupId>com.speedment</groupId> <artifactId>speedment-maven-plugin</artifactId> <version>3.0.3</version> <dependencies> <dependency> <groupId>com.example</groupId> <artifactId>point-plugin</artifactId> <version>1.0.0-SNAPSHOT</version> </dependency> </dependencies> <configuration> <components> <component>com.example.pointplugin.PointPlugin</component> </components> </configuration> </plugin> This tells Speedment to load the plugin Make sure our plugin project is on the classpath
  51. 51. Execute /** * A 2-dimensional coordinate. * <p> * This file is safe to edit. It will not be overwritten by the code generator. * * @author company */ public class Point { private final int x; private final int y; public Point(int x, int y) { this.x = x; this.y = y; } public int getX() { return x; } public int getY() { return y; } The following file is generated: public int hashCode() { int hashCode = 31; hashCode += 41 * x; hashCode += 41 * y; return hashCode; } public Boolean equals(Object other) { if (this == other) return true; else if (other == null) return false; else if (!(other instanceof Point)) { return false; } final Point point = (Point) other; return x == point.x && y == point.y; } public String toString() { return new StringBuilder(“Point{”) .append(“x: “).append(x).append(“, “) .append(“y: “).append(y).append(“}”); } }
  52. 52. A More Concrete Example 1. Create a new Translator 2. Model how the new class should look 3. Define a Plugin Class 4. Include it in project pom.xml Example: Generate an Enum of tables in the database
  53. 53. Step 1: Create a New Translator Class public class TableEnumTranslator extends AbstractJavaClassTranslator<Project, Enum> { public final static TranslatorKey<Project, Enum> TABLES_ENUM_KEY = new TranslatorKey.of(“tables_enum", Enum.class); public TableEnumTranslator(Project document) { super(document, Enum::of); } @Override protected Enum makeCodeGenModel(File file) { return newBuilder(file, getClassOrInterfaceName()) .forEveryProject((clazz, project) -> { // Generate Content Here }).build(); } @Override protected String getClassOrInterfaceName() { return ”Tables"; } @Override protected String getJavadocRepresentText() { return "An enumeration of tables in the database."; } } Every translator is identified by a TranslatorKey Name of generated class Javadoc Called every time the translator is invoked forEvery(Project|Dbms|Schema|Table|Column| …)
  54. 54. Step 2: The makeCodeGenModel - method DocumentDbUtil.traverseOver(project, Table.class) .map(Table::getJavaName) .map(getSupport().namer()::javaStaticFieldName) .sorted() .map(EnumConstant::of) .forEachOrdered(clazz::add);
  55. 55. Step 3: Define a Plugin Class public class TableEnumPlugin { @ExecuteBefore(RESOLVED) protected void install(CodeGenerationComponent codeGen) { codeGen.put( Project.class, TableEnumTranslator.TABLES_ENUM_KEY, TableEnumTranslator::new ); } }
  56. 56. Step 4: Include it in Project pom.xml <plugin> <groupId>com.speedment</groupId> <artifactId>speedment-maven-plugin</artifactId> <version>3.0.3</version> <dependencies> <dependency> <groupId>com.example</groupId> <artifactId>table-enum-plugin</artifactId> <version>1.0.0-SNAPSHOT</version> </dependency> </dependencies> <configuration> <components> <component>com.example.tableenumplugin.TableEnumPlugin</component> </components> </configuration> </plugin> This tells Speedment to load the plugin Make sure our plugin project is on the classpath
  57. 57. Execute /** * An enumeration of tables in the database. * <p> * This file is safe to edit. It will not be overwritten by the code generator. * * @author company */ enum Tables { CITY, SALESPERSON; } The following file is generated:
  58. 58. Generate all the Things • Gson Adapters • Spring Configuration Files • REST Controllers • and much more…
  59. 59. Ext Speeder
  60. 60. Agenda • Problem Description • Code Generation in Database Applications • How Does it Work? • Code Example • Controlling the Code Generation • Generate Custom Classes • Add new Method to Existing Classes • Code Example • Additional Features • Beyond Open Source • Questions & Answers
  61. 61. Add New Method to Existing Classes • So far we have • Queried a database with generated classes • Generated a custom class • How do we modify the existing generation of classes?
  62. 62. Add New Method to Existing Classes • Fit Into Existing Class Hierarchy • Change Naming Conventions • Optimize Internal Implementations • Add Custom Methods
  63. 63. Add New Method to Existing Classes Example: Add a getColumnCount method to generated managers 1. Create a new TranslatorDecorator class 2. Write code generation logic 3. Add it to Speedment
  64. 64. Agenda • Problem Description • Code Generation in Database Applications • How Does it Work? • Code Example • Controlling the Code Generation • Generate Custom Classes • Add new Method to Existing Classes • Code Example • Additional Features • Beyond Open Source • Questions & Answers
  65. 65. Step 1: Creating a New Decorator Class public class ColumnCountDecorator implements TranslatorDecorator<Table, Interface> { @Override public void apply(JavaClassTranslator<Table, Interface> translator) { translator.onMake(builder -> { builder.forEveryTable((intrf, table) -> { // Code generation logic goes here }); }); } }
  66. 66. Step 2: Write Code Generation Logic int columnCount = table.columns().count(); intrf.add(Method.of("getColumnCount", int.class) .default_() .set(Javadoc.of("Returns the number of columns in this table.") .add(RETURN.setValue("the column count")) ) .add("return " + columnCount + ";") );
  67. 67. Step 3: Add it to Speedment public final class TableEnumPlugin { @ExecuteBefore(RESOLVED) protected void install(CodeGenerationComponent codeGen) { codeGen.put( Project.class, TableEnumTranslator.TABLES_ENUM_KEY, TableEnumTranslator::new ); codeGen.add( Table.class, StandardTranslatorKey.GENERATED_MANAGER, new ColumnCountDecorator() ); } } Modify an existing translator key Our new decorator The same plugin class as we created earlier
  68. 68. Execute When the project is regenerated, a new method is added to each manager
  69. 69. Agenda • Problem Description • Code Generation in Database Applications • How Does it Work? • Code Example • Controlling the Code Generation • Generate Custom Classes • Add new Method to Existing Classes • Code Example • Additional Features • Beyond Open Source • Questions & Answers
  70. 70. Additional Features • Add GUI Tool components for custom configuration • Extend the JSON DSL dynamically with plugins • Automate your build environment with Maven • Add custom data type mappers
  71. 71. Additional Features • Plugin a custom namer • Generate classes that are related to other domain models • Generate classes for other languages • Style the GUI Tool
  72. 72. Database Connectors Open Source • MySQL • PostgreSQL • MariaDB Enterprise • Oracle • Microsoft SQL server • DB2 • AS400
  73. 73. Agenda • Problem Description • Code Generation in Database Applications • How Does it Work? • Code Example • Controlling the Code Generation • Generate Custom Classes • Add new Method to Existing Classes • Code Example • Additional Features • Beyond Open Source • Questions & Answers
  74. 74. Beyond Open-Source • Speedment Enterprise • Run your database queries orders of magnitude faster! • 10x, 100x, 1 000x, 10 000x, … • Use the same Stream API
  75. 75. How is This Possible? • No changes to user-written code • Alternative stream source • Data is stored in-memory • Generates optimized serializers for offheap storage
  76. 76. In-JVM-Memory, JMH Benchmarks(*) Benchmark Mode Cnt Score Error Units SpeedmentBenchmark.findStartsWith avgt 200 ≈ 10⁻⁵ s/op SpeedmentBenchmark.findStartsWithSql avgt 200 ≈ 10⁻⁴ s/op SpeedmentBenchmark.join avgt 200 0.476 ± 0.005 s/op SpeedmentBenchmark.joinSql avgt 200 5.174 ± 0.010 s/op SpeedmentBenchmark.sort avgt 200 ≈ 10⁻⁶ s/op SpeedmentBenchmark.sortSql avgt 200 ≈ 10⁻⁴ s/op SpeedmentBenchmark.scrollSorted avgt 200 ≈ 10⁻⁵ s/op SpeedmentBenchmark.scrollSortedSql avgt 200 24.661 ± 0.670 s/op SpeedmentBenchmark.count avgt 180 ≈ 10⁻⁸ s/op SpeedmentBenchmark.countSql avgt 200 5.143 ± 0.012 s/op (*) Preliminary results, Mac Book Pro, 2.2 GHz i7, 16GB, MySQL 5.7.16 standard with indexes on all relevant columns. 10x >10x 100x 2,000,000x 500,000,000x
  77. 77. Speedment Enterprise Example Courtesy of Extremely Heavy Industries
  78. 78. Free Enterprise Trial Today we have released version 1.1.1 Get a free trial! Leave your contact information.
  79. 79. Q&A www.speedment.org github.com/speedment/speedment @speedment www.speedment.com Linkedin: Per Minborg Emil Forslund

×