OSGi Community Event 2014 Abstract: The main topic of the session is the content of the blog post Modularized Persistence: Development of reusable modules that handle relational persistent data. Additional subjects of the session Reasons why we chose this technology stack instead of JEE Transaction handling with the transaction-helper component (without EJB or Spring) Caching the persistent data based on everit-cache-api More details about the already implemented use-cases (localization, authorization, authentication, etc.) During the session, there will be live examples of: Code generation of Querydsl Metadata classes (same as static metamodel in JPA) Converting a standard query to one that contains authorization logic Speaker's goal Introducing our modules to others so they can: use them as they are start discussions about improvements so others can use them in the future Speaker Bio: Balazs Zsoldos is the co-founder of Everit. He is the leader of the development of Everit OpenSource Components. Developing Java based solutions is not only his job but also his passion. He believes in simplicity. That is why he decided to design and build as many simple, but useful goal-oriented modules as he can. As the base of the stack, he chose OSGi. Balazs does not believe in monoholitic frameworks, therefore all of the solutions that was designed by him can be used separately. In the beginning of his career, Balazs was a big fan of JEE and Spring. After a while, he changed his mind and started to try replacing everything with non-magical solutions that do not contain interceptors, weaving, etc.

1. Modularized Persistence
Balázs Zsoldos

2. History of NoSQL by Mark Madse n, Picture published by Edd Dumbill

3. Blueprint
+
JPA
+
JSF
Declarative
Services
+
Modularized
Persistence
+
JSF
ECM
+
Modularized
Persistence
+
JSF
ECM
+
Modularized
Persistence
+
Modularized
Web
Declarative
Services
+
JPA
+
JSF

4. Stefan Seifert – Apache Sling & Friends Tech Meetup, Berlin 2012

5. JDBC Driver
(DataSourceFactory)

6. DSF Component
(XADataSource)
JDBC Driver
(DataSourceFactory)

8. DBCP Component
(DataSource)
DSF Component
(XADataSource)
JDBC Driver
(DataSourceFactory)

10. ???
DBCP Component
(DataSource)
DSF Component
(XADataSource)
JDBC Driver
(DataSourceFactory)

11. Liquibase DataSource Component
(DataSource)
DBCP Component
(DataSource)
DSF Component
(XADataSource)
JDBC Driver
(DataSourceFactory)

13. <databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS"
logicalFilePath="org.everit.osgi.resource.ri.schema"
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd">
<changeSet id="1.0.0" author="everit">
<createTable tableName="res_resource">
<column name="resource_id" type="bigint" autoIncrement="true">
<constraints primaryKeyName="pk_res_resource" primaryKey="true"
nullable="false" />
</column>
</createTable>
</changeSet>
</databaseChangeLog>

14. Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri";
resource="/META-INF/liquibase/resource.ri.liquibase.xml"
Provide-Capability:
liquibase.schema;
name="org.everit.osgi.resource.ri";
resource="/META-INF/liquibase/resource.ri.liquibase.xml";

15. Business Component Business Component Business Component
Querydsl Querydsl Querydsl
Liquibase DataSource Component
(DataSource)
DBCP Component
(DataSource)
DSF Component
(XADataSource)
JDBC Driver
(DataSourceFactory)

16. Examples from http://querydsl.com

17. Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata

18. <lqmg xmlns="http://everit.org/lqmg"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
defaultPackage="org.everit.osgi.resource.ri.schema.qdsl">
<namingRules>
<classNameRule>
<entity>res_resource</entity>
<class>Resource</class>
<propertyMappings>
<primaryKey>
<name>pk_res_resource</name>
<property>resourcePk</property>
</primaryKey>
</propertyMappings>
</classNameRule>
</namingRules>
</lqmg>

19. Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata

20. Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri";
resource="/META-INF/liquibase/resource.ri.liquibase.xml"
Provide-Capability:
liquibase.schema;
name="org.everit.osgi.resource.ri";
resource="/META-INF/liquibase/resource.ri.liquibase.xml";

21. Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri";
resource="/META-INF/liquibase/resource.ri.liquibase.xml";lqmg.config.re
source="/META-INF/liquibase/resource.ri.lqmg.xml"
Provide-Capability:
liquibase.schema;
name="org.everit.osgi.resource.ri";
resource="/META-INF/liquibase/resource.ri.liquibase.xml";
lqmg.config.resource="/META-INF/liquibase/resource.ri.lqmg.xml"

22. Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata

23. <databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.authorization.ri.schema"
xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd
http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
<include file="eosgi:org.everit.osgi.resource.ri" />
<include file="eosgi:org.everit.osgi.props.ri" />
<include file="eosgi:org.everit.osgi.resource.ri" />
include file="eosgi:org.everit.osgi.props.ri" />
<changeSet id="1.0.0" author="everit">
<createTable tableName="authr_permission">
<column name="authorized_resource_id" type="bigint">
<constraints nullable="false" foreignKeyName="fk_res_r_authr_p_a" referencedTableName="res_resource"
referencedColumnNames="resource_id" />
</column>
<column name="target_resource_id" type="bigint">
<constraints nullable="false" foreignKeyName="fk_res_r_authr_p_t" referencedTableName="res_resource"
referencedColumnNames="resource_id" />
</column>
<column name="action_" type="varchar(255)">
<constraints nullable="false" />
</column>
</createTable>
<addPrimaryKey constraintName="pk_authr_perm" tableName="authr_permission"
columnNames="authorized_resource_id,target_resource_id,action_" />
<createTable tableName="authr_permission_inheritance">
<column name="parent_resource_id" type="bigint">
<constraints nullable="false" foreignKeyName="fk_res_r_authr_pi_p" referencedTableName="res_resource"
referencedColumnNames="resource_id" />
</column>
<column name="child_resource_id" type="bigint">
<constraints nullable="false" foreignKeyName="fk_res_r_authr_pi_c" referencedTableName="res_resource"
referencedColumnNames="resource_id" />
</column>
</createTable>
<addPrimaryKey constraintName="pk_authr_perm_inheritance" tableName="authr_permission_inheritance"
columnNames="parent_resource_id,child_resource_id" />
</changeSet>
</databaseChangeLog>

24. Resource
Authorization
Provide-Capability:
liquibase.schema;
name="org.everit.osgi.resource.ri"; ...
Require-Capability:
liquibase.schema;
filter:=(name=org.everit.osgi.resource.ri)
<databaseChangeLog ...>
<include file="eosgi:org.everit.osgi.resource.ri" />
...
</databaseChangeLog>

25. Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata

26. LQMG
Start embedded
OSGi container
Deploy modules
Start embedded
H2 database
Initialize database
schema
Generate Querydsl
Metadata
Evil magic here!!!
If Capability is not found after
deployments, unsatisfied
bundles are enhanced in the
way that unsatisfied
requirements are marked to be
optional.

27. Use-cases???

28. Monoholitic
Portal
(v5.1.2)
Authorization
Authentication
User
Documents
Articles
SiteMap
Blog
Modularized
Portal
Authorization (v1.1.0)
Authentication (v2.0.0)
User (v1.3.2)
Documents (v1.0.1)
Articles (v3.0.14)
SiteMap (v1.0.3)
Blog (v2.11.0)

29. User
User Address
User Address Country
User Address Country
Company

30. REAL PROJECT!!!

31. Authorization

32. Permission
Resource
● resource_id
● authorized_resource_id
● action
● target_resource_id
Permission Inheritance
● parent_resource_id
● child_resource_id

33. SELECT ...
FROM document d
JOIN document.attachment a
WHERE …
LIMIT 10 OFFSET 1000;
EXISTS(SELECT 1 FROM permission p
WHERE p.authorized_resource_id IN (?, …, ?)
AND p.target_resource_id = d.resource_id
AND action = ?)
SELECT ...
FROM document d
JOIN document.attachment a
WHERE EXISTS(SELECT 1 FROM permission p
WHERE p.authorized_resource_id IN (?, …, ?)
AND p.target_resource_id = d.resource_id
AND action = ?)
AND …
LIMIT 10 OFFSET 1000;

34. @Override
public BooleanExpression authorizationPredicate(final long authorizedResourceId,
final Expression<Long> targetResourceId, final String... actions) {
if (authorizedResourceId == systemResourceId) {
return BooleanTemplate.TRUE;
}
Objects.requireNonNull(targetResourceId, "Parameter targetResourceId must not be null");
validateActionsParameter(actions);
long[] authorizationScope = getAuthorizationScope(authorizedResourceId);
SQLSubQuery subQuery = new SQLSubQuery();
QPermission permission = QPermission.permission;
BooleanExpression authorizedResourceIdPredicate;
if (authorizationScope.length == 1) {
authorizedResourceIdPredicate = permission.authorizedResourceId.eq(authorizationScope[0]);
} else {
// More than one as the scope contains at least one value (other branch)
Long[] authorizationScopeLongArray = new Long[authorizationScope.length];
for (int i = 0, n = authorizationScope.length; i < n; i++) {
if (authorizationScope[i] == systemResourceId) {
return BooleanTemplate.TRUE;
}
authorizationScopeLongArray[i] = authorizationScope[i];
}
authorizedResourceIdPredicate = permission.authorizedResourceId.in(authorizationScopeLongArray);
}
BooleanExpression actionPredicate = null;
if (actions.length == 1) {
actionPredicate = permission.action.eq(actions[0]);
} else {
actionPredicate = permission.action.in(actions);
}
return subQuery.from(permission)
.where(permission.targetResourceId.eq(targetResourceId).and(
actionPredicate.and(authorizedResourceIdPredicate)))
.exists();
}

35. SQLQuery query = new SQLQuery(connection, configuration);
QDocument document = new QDocument("d");
BooleanExpression permissionCheckPredicate = authorizationQdslUtil
.authorizationPredicate(userId, document.resourceId, actions);
List<Long> list = query.from(document).where(permissionCheckPredicate)
.list(document.documentId);

36. Business Component Business Component Business Component
Querydsl Querydsl Querydsl
Liquibase DataSource Component
(DataSource)
DBCP Component
(DataSource)
DSF Component
(XADataSource)
JDBC Driver
(DataSourceFactory)
QuerydslSupport
Querydsl
Configuration
Querydsl
SQLTemplates

37. @Reference(name = "querydslSupport", bind = "setQdsl")
private QuerydslSupport qdsl;
public void setQdsl(QuerydslSupport qdsl) {
public List<Long> listDocumentIds() {
return qdsl.execute((connection, configuration) -> {
SQLQuery query = new SQLQuery(connection, configuration);
QDocument document = new QDocument("d");
BooleanExpression permissionCheckPredicate = authorizationQdslUtil
.authorizationPredicate(a1, document.resourceId, actions);
List<Long> list = query.from(document).where(permissionCheckPredicate)
.list(document.documentId);
}
}
this.qdsl = qdsl;
}

38. Business Component Business Component Business Component
Querydsl Querydsl Querydsl
Liquibase DataSource Component
(DataSource)
DBCP Component
(DataSource)
DSF Component
(XADataSource)
JDBC Driver
(DataSourceFactory)
QuerydslSupport
Querydsl
Configuration
Querydsl
SQLTemplates
Transaction Helper

39. public long saveUser(String firstName, String lastName) {
Objects.requireNonNull(firstName);
Objects.requireNonNull(lastName);
return transactionHelper.required(() -> {
// Logic that should be implemented
return 0l;
});
}

40. Caching
● Available Cache modules: cache-api, cache-infinispan,
cache-noop
● Caching should be done in the persistent module
by the programmer, who knows the business logic
● Caching and table updates should be done within
the same component
● In case bulk update is done in another module,
cache must be cleared

41. OpenSource modules
● Resource
● Authorization
● Authentication
● Property Manager
● Blobstore (Before review)
● Audit (Before review)

42. Roadmap

43. HTML Output
LQMG / Maven
H2
Liquibase schema
(Bundle-Capability)
HTML

45. Manual Schema update
LQMG / Maven
Database
Liquibase schema
(Bundle-Capability)
Generate Schema

46. Manual Schema generation
Webconsole plugin
DataSource
(OSGi service)
Liquibase schema
(Bundle-Capability)
Generate Schema
Custom User / Password

47. Dump SQL file
Webconsole plugin
DataSource
(OSGi service)
Liquibase schema
(Bundle-Capability)
SQL

48. LQMG extension in changelog.xml
<databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS"
logicalFilePath="org.everit.osgi.resource.ri.schema"
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:lqmg="http://everit.org/xml/ns/lqmg"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd">
<changeSet id="1.0.0" author="everit">
<createTable tableName="res_resource" lqmg:class="QResource">
<column name="resource_id" type="bigint" autoIncrement="true"
lqmg:property="resourceId">
<constraints primaryKeyName="pk_res_resource" primaryKey="true"
nullable="false" />
</column>
</createTable>
</changeSet>
</databaseChangeLog>

49. Full Text Search
● Analize the different solutions
– H2 → Lucene
– MySQL → Sphinx
– PostgreSQL → TSearch2
– SQL Server →???
– Oracle → Oracle Text
● Create a common API for Querydsl based query
extension
● Create a module for each database engine

50. Querydsl: http://www.querydsl.com/
Liquibase: http://www.liquibase.org/
Liquibase OSGi Bundle: https://github.com/everit-org/osgi-liquibase-bundle
LQMG: https://github.com/everit-org/osgi-lqmg
LQMG Maven Plugin: http://www.everit.org/lqmg-maven-plugin/
Liquibase DataSource: https://github.com/everit-org/osgi-liquibase-datasource
Querydsl SQLTemplates Component: https://github.com/everit-org/osgi-querydsl-templates
Querydsl Configuration Component: https://github.com/everit-org/osgi-querydsl-configuration
Querydsl Support Component: https://github.com/everit-org/osgi-querydsl-support
Cookbook chapter: http://cookbook.everit.org/persistence/index.html
Download from Maven central
Resource: https://github.com/everit-org/resource-ri
Authorization: https://github.com/everit-org/authorization-ri
Authentication: https://github.com/everit-org/authentication-simple
Property Manager: https://github.com/everit-org/property-manager-ri
Twitter: @EveritOrg