JPA/Hibernate Pros/Cons What is alternatives to JPA/Hibernate We select “Requery” Introduce spring-data-requery Use cases

1. Alternatives to JPA
NaverTech talk
debop@coupang.com 2019.01.22

2. Alternatives to JPA

3. Agenda
• What is ORM
• Pros / Cons of JPA
• Current status in Coupang
• Alternatives of JPA
• Slick
• jOOQ
• Exposed
• Requery

4. ORM (Object Relational Mapping)
• OOP’s object graph vs Relational Database
• Focus OOP, not Relational Database
• No matter of RDBMS vendor - Same code
• Hibernate coverage is 95% over traditional SQL statements
• ORM not suitable for data centric application in performance
• Why should you use an ORM?

5. Pros of JPA
• Focus to Java Object graph & OOP
• No need to know relations and constraints of entities
• No need to know specific DB features by various vendor
• No need to know SQL, just use Java API
• Supplement by HQL or JPQL or QueryDSL
• All support for Stateful, Stateless (Default is Stateful)

6. Cons of JPA
• If you knew SQL already, JPA is wired
• Hard to learning (exponential)
• Low performance by stateful and fetch by id
• No suitable for Bulk or Set operations
• Massive Insert, Statistical Summary (Cube …)
• Non-Threadsafe Session - Low throughput
• Need to learn specific JPAVendor (Hibernate, Eclipse Link)
• HQL, @DynamicInsert, @LazyCollection
• 2nd Cache (recommend JCache (JSR-305))

7. Current Status in Coupang
• No Deep Dive
• Missing override (hashCode, equals, toString)
• No using @NatualId
• Bad Policy / Bad Design
• Every entity has Own identifier (Some case no need)
• Poor performance -> Mislead “JPA is bad”
• Apply not suitable case
• Bulk operations, Statistical operations
• No use supplement features
• StatelessSession, 2nd Cache …

8. DB: H2 Memory DB, CPU Core: 12, OS: Linux Mint, JVM: Java 8
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
50,000
10 1000 5000
JPA vs Requery Insert Benchmark
JPA Requery

9. Features in Alternatives of JPA
• Design Principle
• OOP based, Support Multi DBVendor
• No need stateful for Reference object
• Support association, inheritance, converter in JPA
• Performance
• Speed up to Plain SQL
• Stateless
• Support Asynchronous or Reactive
• Support Bulk or Batch operations

10. Slick
ORM for Scala

11. Slick
• Database access library
• Not ORM -> Functional Relational Mapping
• Bring relational model to Scala OOP
• Natural fit ( no impedance mismatch)
• Stateless
• Matures (Slick version 3) (Non-blocking DBIO)
• Essential Slick Book

12. Slick - schema definition
Entity
Schema
Definition

13. Slick - Query
// Get Users with age > 20
Query statement
Async
Operations

14. Slick - DML

15. Slick -Transactions
Define Repository
Async
Execution with
Transaction

16. Slick - Plain SQL
ResultSet to
Tuple
ResultSet to
Entity

17. jOOQ
Java Object Oriented Query

18. jOOQ
• Reflect Database Schema to generate Entity Class
• Typesfe SQL (akaTypesafe MyBatis)
• Database First (Not ORM)
• Stateless
• Need DBMS Owner Authority
• jOOQ vs Hibernate :When to choose which

19. jOOQ - typesafe SQL
SELECT * FROM BOOK
WHERE BOOK.PUBLISHED_IN = 2011
ORDER BY BOOK.TITLE
create.selectFrom(BOOK)
.where(BOOK.PUBLISHED_IN.eq(2011))
.orderBy(BOOK.TITLE)
select().from(t).where(t.a.eq(select(t2.x).from(t2));
// Type-check here: ---------------> ^^^^
select().from(t).where(t.a.eq(any(select(t2.x).from(t2)));
// Type-check here: -------------------> ^^^^
select().from(t).where(t.a.in(select(t2.x).from(t2));
// Type-check here: ---------------> ^^^^
select(t1.a).from(t1).unionAll(select(t2.a).from(t2));
// Type-check here: ----------------> ^^^^
select(t1.a, t1.b).from(t1).union(select(t2.a, t2.b).from(t2));
// Type-check here: -------------------> ^^^^^^^^^^
Predicates
Set operations

20. requery
ORM for Java & Kotlin & Android

21. requery
• No reflection (apt code generation) - Fast instancing
• Fast startup & performance
• Schema generation
• Blocking / Non-blocking API (Reactive with RxJava)
• Support partial object / refresh / upsert
• Custom type converter like JPA
• Compile time entity validation
• Support almost JPA annotations

22. @Entity
abstract class AbstractPerson {
@Key @Generated int id;
@Index("name_index") // table specification
String name;
@OneToMany // relationships 1:1, 1:many, many to many
Set<Phone> phoneNumbers;
@Converter(EmailToStringConverter.class)
Email email;
@PostLoad // lifecycle callbacks
void afterLoad() { updatePeopleList(); }
// getter, setters, equals & hashCode automatically generated into Person.java
}
requery - define entity
Identifier
Entity class
Converter
Listeners

23. Result<Person> query = data
.select(Person.class)
.where(Person.NAME.lower().like("b%"))
.and(Person.AGE.gt(20))
.orderBy(Person.AGE.desc())
.limit(5)
.get();
Observable<Person> observable = data
.select(Person.class)
.orderBy(Person.AGE.desc())
.get()
.observable();
requery - query
Query by Fluent API
Reactive Programming
Cold Observable
Non blocking

24. @Entity(model = "tree")
interface TreeNode {
@get:Key
@get:Generated
val id: Long
@get:Column
var name: String
@get:ManyToOne(cascade = [DELETE])
var parent: TreeNode?
@get:OneToMany(mappedBy = "parent", cascade = [SAVE, DELETE])
val children: MutableSet<TreeNode>
}
requery - self refence by Kotlin
Identifier
Entity class
1:N, N:1
Cascade

25. requery - Blob/Clob usage
class ByteArrayBlobConverter : Converter<ByteArray, Blob> {
override fun getPersistedSize(): Int? = null
override fun getPersistedType(): Class<Blob> = Blob::class.java
override fun getMappedType(): Class<ByteArray> = ByteArray::class.java
override fun convertToMapped(type: Class<out ByteArray>?, value: Blob?): ByteArray? {
return value?.binaryStream?.readBytes()
}
override fun convertToPersisted(value: ByteArray?): Blob? {
return value?.let { SerialBlob(it) }
}
}

26. requery - Blob property
@Entity(model = "kt")
interface BigModel {
@get:Key
@get:Generated
@get:Column(name = "model_id")
val id: Int
@get:Column(name = "model_name")
var name: String?
@get:Convert(value = ByteArrayBlobConverter::class)
@get:Column(name = "model_picture")
var picture: ByteArray?
}
Blob column

27. Exposed
Kotlin SQL Framework

28. Exposed - Kotlin SQL Framework
• Lightweight SQL Library
• Provide two layers of data access
• Typesafe SQL wrapping DSL
• Lightweight Data Access Object
• See : First steps with Kotlin/Exposed
• Cons
• Not mature

29. Exposed - SQL DSL
object Users : Table() {
val id = varchar("id", 10).primaryKey() // Column<String>
val name = varchar("name", length = 50) // Column<String>
val cityId = (integer("city_id") references Cities.id).nullable() // Column<Int?>
}
object Cities : Table() {
val id = integer("id").autoIncrement().primaryKey() // Column<Int>
val name = varchar("name", 50) // Column<String>
}

30. Exposed - SQL DSL
val munichId = Cities.insert {
it[name] = "Munich"
} get Cities.id
Cities.insert { it[name] = "Prague" }
Users.insert {
it[id] = "andrey"
it[name] = "Andrey"
it[cityId] = saintPetersburgId
}
Users.insert {
it[id] = "sergey"
it[name] = "Sergey"
it[cityId] = munichId
}

31. Exposed - SQL DSL - Join
(Users innerJoin Cities)
.slice(Users.name, Cities.name)
.select {
(Users.id.eq("andrey") or Users.name.eq("Sergey")) and
Users.id.eq("sergey") and Users.cityId.eq(Cities.id)
}
.forEach {
println("${it[Users.name]} lives in ${it[Cities.name]}")
}

32. Exposed - SQL DSL - Join 2
((Cities innerJoin Users)
.slice(Cities.name, Users.id.count())
.selectAll()
.groupBy(Cities.name))
.forEach {
val cityName = it[Cities.name]
val userCount = it[Users.id.count()]
if (userCount > 0) {
println("$userCount user(s) live(s) in $cityName")
} else {
println("Nobody lives in $cityName")
}
}

33. Exposed - DAO
object Users : IntIdTable() {
val name = varchar("name", 50).index()
val city = reference("city", Cities)
val age = integer("age")
}
object Cities: IntIdTable() {
val name = varchar("name", 50)
}
Schema Definition
class User(id: EntityID<Int>) : IntEntity(id){
companion object : IntEntityClass<User>(Users)
var name by Users.name
var city by City referencedOn Users.city
var age by Users.age
}
class City(id: EntityID<Int>) : IntEntity(id) {
companion object : IntEntityClass<City>(Cities)
var name by Cities.name
val users by User referrersOn Users.city
}
Entity Definition

34. Exposed - DAO Usage
val munich = City.new {
name = "Munich"
}
User.new {
name = "a"
city = munich
age = 5
}
User.new {
name = "b"
city = munich
age = 27
}
munich.users.joinToString { it.name }
User.find { Users.age.between(18, 60) }
OneTo Many
All user’s name in Munich

35. Conclusion
• Already legacy database exists ? Use only Java
• jOOQ or requery
• Scala only ? -> Slick
• Kotlin only ? -> requery, Exposed
• No matter language? -> requery
• Need Reactive programming? ->
• requery with kotlinx-requery
• kotlinx-rxjava2-jdbc ( we will open March )

36. Requery

37. Agenda
• Requery Overview
• Why Requery
• Define Mapping
• Usage EntityDataStore
• EntityDataStore for Kotlin (Coroutines)
• Introduction of Spring Data Requery

38. Requery Overview
• ORM Library for Java, Kotlin, Android
• No Reflection (vs Hibernate proxy)
• Typed Query language (vs Hibernate Criteria)
• Upsert/Partial objects refresh
• Compile time entity/query validation (vs Hibernate)
• Entity is stateless (vs Hibernate stateful)
• Thread 에 제한 받지 않음 (vs JPA EntityManager)
• Support RxJava, Async Operations, Java 8

39. Why Requery
• Provide benefit of ORM
• Entity Mapping
• Schema Generation
• Compile time error detecting
• Performance
• When bulk job, max 100x than JPA
• REST API - 2~10x throughput
• Support Upsert, Lazy loading …

40. DB: H2 Memory DB, CPU Core: 12, OS: Linux Mint, JVM: Java 8
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
50,000
10 1000 5000
JPA vs Requery Insert Benchmark
JPA Requery

41. Define Entity - Java
@Getter
@Entity(name = "BasicUser", copyable = true)
@Table(name = "basic_user")
public abstract class AbstractBasicUser extends AuditableLongEntity {
@Key
@Generated
protected Long id;
protected String name;
protected String email;
protected LocalDate birthday;
protected Integer age;
@ForeignKey
@OneToOne
protected AbstractBasicLocation address;
@ManyToMany(mappedBy = "members")
protected Set<AbstractBasicGroup> groups;
@Column(unique = true)
protected UUID uuid;

42. Define Entity - Kotlin
@Entity(model = "functional")
interface Person: Persistable {
@get:Key
@get:Generated
val id: Long
@get:Index(value = ["idx_person_name_email"])
var name: String
@get:Index(value = ["idx_person_name_email", "idx_person_email"])
var email: String
var birthday: LocalDate
@get:Column(value = "'empty'")
var description: String?
@get:Nullable
var age: Int?
@get:ForeignKey
@get:OneToOne(mappedBy = "person", cascade = [CascadeAction.DELETE, CascadeAction.SAVE])
var address: Address?
@get:OneToMany(mappedBy = "owner", cascade = [CascadeAction.DELETE, CascadeAction.SAVE])
val phoneNumbers: MutableSet<Phone>
@get:OneToMany
val phoneNumberList: MutableList<Phone>
@get:ManyToMany(mappedBy = "members")
val groups: MutableResult<Group>
@get:ManyToMany(mappedBy = "owners")
val ownedGroups: MutableResult<Group>
@get:ManyToMany(mappedBy = "id")
@get:JunctionTable
val friends: MutableSet<Person>
@get:Lazy
var about: String?
@get:Column(unique = true)
var uuid: UUID
var homepage: URL
var picture: String
}

43. EntityDataStore<Object>
• findByKey
• select / insert / update / upsert / delete
• where / eq, lte, lt, gt, gte, like, in, not …
• groupBy / having / limit / offset
• support SQL Functions
• count, sum, avg, upper, lower …
• raw query

44. @Test
fun `insert user`() {
val user = RandomData.randomUser()
withDb(Models.DEFAULT) {
insert(user)
assertThat(user.id).isGreaterThan(0)
val loaded = select(User::class) where (User::id eq user.id) limit 10
assertThat(loaded.get().first()).isEqualTo(user)
}
}
val result = select(Location::class)
.join(User::class).on(User::location eq Location::id)
.where(User::id eq user.id)
.orderBy(Location::city.desc())
.get()
val result = raw(User::class, "SELECT * FROM Users")
val rowCount = update(UserEntity::class)
.set(UserEntity.ABOUT, "nothing")
.set(UserEntity.AGE, 50)
.where(UserEntity.AGE eq 100)
.get()
.value()
val count = insert(PersonEntity::class, PersonEntity.NAME, PersonEntity.DESCRIPTION)
.query(select(GroupEntity.NAME, GroupEntity.DESCRIPTION))
.get()
.first()
.count()

45. CoroutineEntityDataStore
val store = CoroutineEntityStore(this)
runBlocking {
val users = store.insert(RandomData.randomUsers(10))
users.await().forEach { user ->
assertThat(user.id).isGreaterThan(0)
}
store
.count(UserEntity::class)
.get()
.toDeferred()
.await()
.let {
assertThat(it).isEqualTo(10)
}
}
with(coroutineTemplate) {
val user = randomUser()
// can replace with `withContext { }`
async { insert(user) }.await()
assertThat(user.id).isNotNull()
val group = RandomData.randomGroup()
group.members.add(user)
async { insert(group) }.await()
assertThat(user.groups).hasSize(1)
assertThat(group.members).hasSize(1)
}

46. spring-data-requery
• RequeryOperations
• Wrap EntityDataStore
• RequeryTransactionManager for TransactionManager
• Support Spring @Transactional
• Better performance than spring-data-jpa
• when exists, paging, not load all entities

47. spring-data-requery
• Repository built in SQL
• ByPropertyName Auto generation methods
• @Query for Native SQL Query
• Query By Example
• Not Supported
• Association Path (not specified join method)
• Named parameter in @Query (just use `?`)
(support next version)

48. Setup spring-data-requery
@Configuration
@EnableTransactionManagement
public class RequeryTestConfiguration extends AbstractRequeryConfiguration {
@Override
@Bean
public EntityModel getEntityModel() {
return Models.DEFAULT;
}
@Override
public TableCreationMode getTableCreationMode() {
return TableCreationMode.CREATE_NOT_EXISTS;
}
@Bean
public DataSource dataSource() {
return new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2)
.build();
}
}

49. Provided Beans
@Bean
public io.requery.sql.Configuration requeryConfiguration() {
return new ConfigurationBuilder(dataSource, getEntityModel())
// .useDefaultLogging()
.setEntityCache(new EmptyEntityCache())
.setStatementCacheSize(1024)
.setBatchUpdateSize(100)
.addStatementListener(new LogbackListener())
.build();
}
@Bean
public EntityDataStore<Object> entityDataStore() {
log.info("Create EntityDataStore instance.");
return new EntityDataStore<>(requeryConfiguration());
}
@Bean
public RequeryOperations requeryOperations() {
log.info("Create RequeryTemplate instance.");
return new RequeryTemplate(entityDataStore(), requeryMappingContext());
}
EntityCache 설정Tip :
개발 시에는 EmptyEntityCache,
운영 시에는 Cache2kEntityCache 사용

50. Use @Query in Repository
interface DeclaredQueryRepository extends RequeryRepository<BasicUser, Long> {
@Query("select * from basic_user u where u.email = ?")
BasicUser findByAnnotatedQuery(String email);
@Query("select * from basic_user u where u.email like ?")
List<BasicUser> findAllByEmailMatches(String email);
@Query("select * from basic_user u limit ?")
List<BasicUser> findWithLimits(int limit);
@Query("select * from basic_user u where u.name=? and u.email=? limit 1")
BasicUser findAllBy(String name, String email);
@Query("select u.id, u.name from basic_user u where u.email=?")
List<Tuple> findAllIds(String email);
@Query("select * from basic_user u where u.birthday = ?")
List<BasicUser> findByBirthday(LocalDate birthday);
}

51. Query By Example
BasicUser user = RandomData.randomUser();
user.setName("example");
requeryTemplate.insert(user);
BasicUser exampleUser = new BasicUser();
exampleUser.setName("EXA");
ExampleMatcher matcher = matching()
.withMatcher("name", startsWith().ignoreCase())
.withIgnoreNullValues();
Example<BasicUser> example = Example.of(exampleUser, matcher);
Return<? extends Result<BasicUser>> query = buildQueryByExample(example);
BasicUser foundUser = query.get().firstOrNull();
assertThat(foundUser).isNotNull().isEqualTo(user);

52. Query by Property
List<User> findByFirstnameOrLastname(String firstname, String lastname);
List<User> findByLastnameLikeOrderByFirstnameDesc(String lastname);
List<User> findByLastnameNotLike(String lastname);
List<User> findByLastnameNot(String lastname);
List<User> findByManagerLastname(String name);
List<User> findByColleaguesLastname(String lastname);
List<User> findByLastnameNotNull();
@Query("select u.lastname from SD_User u group by u.lastname")
Page<String> findByLastnameGrouped(Pageable pageable);
long countByLastname(String lastname);
int countUsersByFirstname(String firstname);
boolean existsByLastname(String lastname);
Note: Association Path is not supported
Note: Association Path is not supported

53. Exists
static class ExistsExecution extends JpaQueryExecution {
@Override
protected Object doExecute(AbstractJpaQuery query, Object[] values) {
return !query.createQuery(values).getResultList().isEmpty();
}
}
static class ExistsExecution extends RequeryQueryExecution {
@Override
protected @Nullable Object doExecute(AbstractRequeryQuery query, Object[] values) {
Result<?> result = (Result<?>) query.createQueryElement(values).limit(1).get();
return result.firstOrNull() != null;
}
}
Spring Data JPA - ExistsExecution
Spring Data Requery - ExistsExecution

54. Delete in JPA
static class DeleteExecution extends JpaQueryExecution {
private final EntityManager em;
public DeleteExecution(EntityManager em) {
this.em = em;
}
@Override
protected Object doExecute(AbstractJpaQuery jpaQuery, Object[] values) {
Query query = jpaQuery.createQuery(values);
List<?> resultList = query.getResultList();
for (Object o : resultList) {
em.remove(o);
}
return jpaQuery.getQueryMethod().isCollectionQuery() ? resultList : resultList.size();
}
}
Spring Data JPA - DeleteExecution
Load all entities to delete for
cascading delete

55. Delete in requery
static class DeleteExecution extends RequeryQueryExecution {
private final RequeryOperations operations;
DeleteExecution(@NotNull RequeryOperations operations) {
Assert.notNull(operations, "operations must not be null!");
this.operations = operations;
}
@SuppressWarnings("unchecked")
@Override
protected @Nullable Object doExecute(AbstractRequeryQuery query, Object[] values) {
QueryElement<?> whereClause = query.createQueryElement(values);
QueryElement<?> deleteQuery = applyWhereClause((QueryElement<?>) operations.delete(query.getDomainClass()),
whereClause.getWhereElements());
Scalar<Integer> result = ((QueryElement<? extends Scalar<Integer>>) deleteQuery).get();
return result.value();
}
}
Spring Data Requery - DeleteExecution
Use delete statements directly
cascading depends on DB

56. Future works
• Support Named parameter (ready)
• Support `@Param` in spring data
• Support complecated aggregation operations
• Requery for Apache Phoenix (done)
• CoroutineEntityStore (testing)

57. Resources
• requery.io
• spring-data-requery in github
• 2.0.x for spring-data-commons 2.0.x
• 2.1.x for spring-data-commons 2.1.x

58. Q&A

59. Thank you!