This document discusses using Speedment and Hazelcast to automatically scale out existing SQL databases. Speedment monitors databases for changes and reflects those changes in distributed Hazelcast maps in real-time. It generates code to represent database tables as Java POJOs, managers to interact with the POJOs, and configuration to integrate everything with Hazelcast. Users can then access and modify database data using the Hazelcast maps without changing their code, achieving automatic horizontal scaling of the SQL data. The process involves connecting an existing database, running Speedment to generate the associated code, and adding a few lines to integrate with an existing Hazelcast instance.

1. Speedment and Hazelcast
For Automatic Scale out of Existing SQL
Databases with Hazelcast
Per-Åke Minborg
www.speedment.com

2. The Shared Database Challenge
Database
Hazelcast
Hazelcast Application
Hazelcast Application
Hazelcast Application

3. The Shared Database Challenge
Database
Hazelcast
Hazelcast Application3:rd Party Application
3:rd Party Application
3:rd Party Application
Hazelcast Application
Hazelcast Application

4. The Shared Database Challenge
Database
Speedment
Hazelcast
Hazelcast Application3:rd Party Application
3:rd Party Application
3:rd Party Application
Hazelcast Application
Hazelcast Application

5. Overview
Database Speedment
Hazelcast
Node
Hazelcast
Node
Hazelcast
Node
Hazelcast
Node
INSERT
UPDATE
DELETE

6. Reflects all Data with Hazelcast
Exising SQL data from start
System.out.println(hazelcastMap.size());
-> 300024
Inserts
sql> insert into employees (emp_no, birth_date, first_name, last_name, gender, hire_date)
values (10001, '1953-09-02', 'Georgi', 'Facello', 'M', '1986-06-26');
Employees emp = hazelcastMap.get("10001"); System.out.println(emp);
-> Employees { "emp_no": 10001, "birth_date": "1953-09-02", "first_name": "Georgi", "last_name":
"Facello", "gender": "M", "hire_date": "1986-06-26"}
Updates
sql> update employees set first_name="Georgi2" where emp_no=10001;
Employees emp = hazelcastMap.get("10001"); System.out.println(emp);
-> Employees { "emp_no": 10001, "birth_date": "1953-09-02", "first_name": "Georgi2", "last_name":
"Facello", "gender": "M", "hire_date": "1986-06-26"}
Deletes
sql> delete from employees where emp_no = 10001;
Employees emp = hazelcastMap.get("10001"); System.out.println(emp);
-> null

7. Works with Transactions
SQL> START TRANSACTION;
update employees set first_name="Duangkaew"
where emp_no=10010;
update salaries set salary=salary+10 where
id=12322;
COMMIT;
// Backing hazelcast maps are updated atomically
Employees emp = employeesMap.get("10001");
Salaries sal = salaryMap.get("12322");

8. How to Install
• Download software
• Unzip distribution file
• Start scripts in ”bin” directory

9. Generated Code
§ Database triggers
§ Triggerlog
§ Java POJOs for each selected table
§ Table manager/DAO
§ Hazelcast serialization factories
§ Hazelcast config support
§ Relational database metadata
§ Transaction support wrappers

10. Database Table
mysql> describe employees;
+------------+---------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+---------------+------+-----+---------+----------------+
| emp_no | int(11) | NO | PRI | NULL | auto_increment |
| birth_date | date | NO | | NULL | |
| first_name | varchar(14) | NO | | NULL | |
| last_name | varchar(16) | NO | | NULL | |
| gender | enum('M','F') | NO | | NULL | |
| hire_date | date | NO | | NULL | |
+------------+---------------+------+-----+---------+----------------+

11. POJO Representing a Row
public class Employees extends Struct<Employees>
implements IdentifiedDataSerializable {
private Integer empNo;
private Date birthDate;
private String firstName;
private String lastName;
private String gender;
private Date hireDate;
// Accessors, serialization, deserialization, etc.

12. @Override
public int getFactoryId() {
return EmployeesDataSerializableFactory.FACTORY_ID;
}
@Override
public void writeData(ObjectDataOutput out) throws
IOException {
int sizeOf = (int) sizeOf();
final ByteBuffer bb = ByteBuffer.allocate(sizeOf + 4);
bb.putInt(sizeOf);
writeObject(bb);
out.write(bb.array());
}
POJO Representing a Row

13. POJO Representing a Row
@Override
public void readData(ObjectDataInput in) throws IOException
{
int sizeOf = in.readInt();
final ByteBuffer bb = ByteBuffer.allocate(sizeOf);
in.readFully(bb.array());
try {
readObject(bb);
} catch (final ClassNotFoundException cnfe) {
_logger.error(cnfe.getMessage(), cnfe);
}
}
}

14. Manager/DAO Representing a Table
public class EmployeesMgr extends
Manager<Employees> {
@Override
public Employees getNewObjectInstance() {
return new Employees();
}
// insert, update, delete, finders, size,
etc.

15. Manager/DAO Representing a Table
@Override
public void add(final Employees employees) {
addHazelcast(employees);
}
@Override
public void remove(final Employees employees) {
removeHazelcast(employees);
}
}

16. Manager/DAO Representing a Table
protected void addHazelcast(final Employees employees) {
final String _key = employees._getPrimaryKeyString();
final TransactionalMap<String, Employees>
transactionMap = getHazelcastTransactionalMap();
if (transactionMap != null) {
transactionMap.put(_key, employees);
} else {
hcPrimaryKeyMap.put(_key, employees);
}
}

17. DataSerializableFactory per Table
public class EmployeesDataSerializableFactoryBase
implements DataSerializableFactory {
public static final int FACTORY_ID = 1221169916;
public static final int EMPLOYEES_ID = 0;
@Override
public IdentifiedDataSerializable create(int typeId)
{
return
EmployeesMgr.getInstance().getNewObjectInstance();
}
}

18. Configuration Support Class (One)
public class SpeedmentHazelcastConfig{
public static void addTo(final Config config) {
addTo(config.getSerializationConfig());
}
public static void addTo(final ClientConfig config) {
addTo(config.getSerializationConfig());
}
public static void addTo(final SerializationConfig
serializationConfig) {
serializationConfig.addDataSerializableFactory(
DepartmentsDataSerializableFactory.FACTORY_ID,
new DepartmentsDataSerializableFactory());
serializationConfig.addDataSerializableFactory(
DeptManagerDataSerializableFactory.FACTORY_ID,
new DeptManagerDataSerializableFactory());
serializationConfig.addDataSerializableFactory(
EmployeesDataSerializableFactory.FACTORY_ID,
new EmployeesDataSerializableFactory());
}
}

19. Transaction Support Classes
public class HazelcastTransactionHandlerFactory implements
TransactionHandlerFactory {
private final HazelcastInstance hazelcastInstance;
private final TransactionOptions transactionOptions;
public HazelcastTransactionHandlerFactory(HazelcastInstance
hazelcastInstance, TransactionOptions transactionOptions) {
this.hazelcastInstance = hazelcastInstance;
this.transactionOptions = transactionOptions;
}
@Override
public TransactionHandler newTransactionHandler() {
final TransactionContext context =
hazelcastInstance.newTransactionContext(transactionOptions);;
return new HazelcastTransactionHandler(context);
}
}

20. Transaction Support Classes
public class HazelcastTransactionHandler implements TransactionHandler
{
private final TransactionContext context;
public HazelcastTransactionHandler(TransactionContext context) {
this.context = context;
}
. . .
}

21. Transaction Support Classes
@Override
public void begin() {
context.beginTransaction();
}
@Override
public void commit() {
context.commitTransaction();
}
@Override
public void rollback() {
context.rollbackTransaction();
}
public TransactionContext getContext() {
return context;
}

22. How do you use it?
“Easy as 1, 2, 3...”

23. 1 Connect to your existing SQL DB…

24. …and it is Automatically Analyzed

25. 2 Push play and….

26. Work as you are used to, just add 4 lines of code in your startup class
Config hcConfig = new Config();
// Add optimized serialization Factories for Hazelcast that are generated automatically
SpeedmentHazelcastConfig.addTo(hcConfig);
HazelcastInstance hcInstance = Hazelcast.newHazelcastInstance(hcConfig);
// Tell Speedment what Hazelcast instance to use
ProjectManager.getInstance().putProperty(HazelcastInstance.class, hcInstance);
// Automatically build all Database metadata (e.g. Schema, Tables and Columns)
new TreeBuilder().build();
// Load selected data from the database into the Hazelcast maps and start tracking DB
ProjectManager.getInstance().init();
3 Start Coding with Hazelcast

27. No change in how maps are used
// Get the Hazelcast map for the database table "employees"
IMap<String, Employees> employeesmap =
hcInstance.getMap(EmployeesMgr.getHazelcastMapName());
// Use the Hazelcast map just as you are used to
Employees emp = employeesmap.get("10001");

28. Video

29. Customization
• Use your own beans instead
• Use transactions
• Filter events using predicates
• React on SQL events

30. Performance
Depends mostly on
• The SQL server
• Hazelcast grid configuration and
implementation

31. Business model
• OpenSource coming
• Enterprise license

32. Thank You!
Per-Åke Minborg
minborg@speedment.com
http://minborgsjavapot.blogspot.com/
Twitter: @Pminborg
To download:
sales@speedment.com