Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Modularized Persistence 
Balázs Zsoldos
History of NoSQL by Mark Madse n, Picture published by Edd Dumbill
Blueprint 
+ 
JPA 
+ 
JSF 
Declarative 
Services 
+ 
Modularized 
Persistence 
+ 
JSF 
ECM 
+ 
Modularized 
Persistence 
+...
Stefan Seifert – Apache Sling & Friends Tech Meetup, Berlin 2012
JDBC Driver 
(DataSourceFactory)
DSF Component 
(XADataSource) 
JDBC Driver 
(DataSourceFactory)
DBCP Component 
(DataSource) 
DSF Component 
(XADataSource) 
JDBC Driver 
(DataSourceFactory)
??? 
DBCP Component 
(DataSource) 
DSF Component 
(XADataSource) 
JDBC Driver 
(DataSourceFactory)
Liquibase DataSource Component 
(DataSource) 
DBCP Component 
(DataSource) 
DSF Component 
(XADataSource) 
JDBC Driver 
(D...
<databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" 
logicalFilePath="org.everit.osgi.resource.ri.schema" 
xmlns=...
Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; 
resource="/META-INF/liquibase/resource.ri.liquib...
Business Component Business Component Business Component 
Querydsl Querydsl Querydsl 
Liquibase DataSource Component 
(Dat...
Examples from http://querydsl.com
Write Liquibase changelog 
Write LQMG file 
Add LQMG to the Capability 
Add inclusions 
Generate Querydsl Metadata
<lqmg xmlns="http://everit.org/lqmg" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
defaultPackage="org.everit.os...
Write Liquibase changelog 
Write LQMG file 
Add LQMG to the Capability 
Add inclusions 
Generate Querydsl Metadata
Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; 
resource="/META-INF/liquibase/resource.ri.liquib...
Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; 
resource="/META-INF/liquibase/resource.ri.liquib...
Write Liquibase changelog 
Write LQMG file 
Add LQMG to the Capability 
Add inclusions 
Generate Querydsl Metadata
<databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.authorization.ri.schema" 
xm...
Resource 
Authorization 
Provide-Capability: 
liquibase.schema; 
name="org.everit.osgi.resource.ri"; ... 
Require-Capabili...
Write Liquibase changelog 
Write LQMG file 
Add LQMG to the Capability 
Add inclusions 
Generate Querydsl Metadata
LQMG 
Start embedded 
OSGi container 
Deploy modules 
Start embedded 
H2 database 
Initialize database 
schema 
Generate Q...
Use-cases???
Monoholitic 
Portal 
(v5.1.2) 
Authorization 
Authentication 
User 
Documents 
Articles 
SiteMap 
Blog 
Modularized 
Porta...
User 
User Address 
User Address Country 
User Address Country 
Company
REAL PROJECT!!!
Authorization
Permission 
Resource 
● resource_id 
● authorized_resource_id 
● action 
● target_resource_id 
Permission Inheritance 
● p...
SELECT ... 
FROM document d 
JOIN document.attachment a 
WHERE … 
LIMIT 10 OFFSET 1000; 
EXISTS(SELECT 1 FROM permission p...
@Override 
public BooleanExpression authorizationPredicate(final long authorizedResourceId, 
final Expression<Long> target...
SQLQuery query = new SQLQuery(connection, configuration); 
QDocument document = new QDocument("d"); 
BooleanExpression per...
Business Component Business Component Business Component 
Querydsl Querydsl Querydsl 
Liquibase DataSource Component 
(Dat...
@Reference(name = "querydslSupport", bind = "setQdsl") 
private QuerydslSupport qdsl; 
public void setQdsl(QuerydslSupport...
Business Component Business Component Business Component 
Querydsl Querydsl Querydsl 
Liquibase DataSource Component 
(Dat...
public long saveUser(String firstName, String lastName) { 
Objects.requireNonNull(firstName); 
Objects.requireNonNull(last...
Caching 
● Available Cache modules: cache-api, cache-infinispan, 
cache-noop 
● Caching should be done in the persistent m...
OpenSource modules 
● Resource 
● Authorization 
● Authentication 
● Property Manager 
● Blobstore (Before review) 
● Audi...
Roadmap
HTML Output 
LQMG / Maven 
H2 
Liquibase schema 
(Bundle-Capability) 
HTML
Manual Schema update 
LQMG / Maven 
Database 
Liquibase schema 
(Bundle-Capability) 
Generate Schema
Manual Schema generation 
Webconsole plugin 
DataSource 
(OSGi service) 
Liquibase schema 
(Bundle-Capability) 
Generate S...
Dump SQL file 
Webconsole plugin 
DataSource 
(OSGi service) 
Liquibase schema 
(Bundle-Capability) 
SQL
LQMG extension in changelog.xml 
<databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" 
logicalFilePath="org.everit...
Full Text Search 
● Analize the different solutions 
– H2 → Lucene 
– MySQL → Sphinx 
– PostgreSQL → TSearch2 
– SQL Serve...
Querydsl: http://www.querydsl.com/ 
Liquibase: http://www.liquibase.org/ 
Liquibase OSGi Bundle: https://github.com/everit...
Modularized Persistence - B Zsoldos
Modularized Persistence - B Zsoldos
Modularized Persistence - B Zsoldos
Modularized Persistence - B Zsoldos
Upcoming SlideShare
Loading in …5
×

Modularized Persistence - B Zsoldos

851 views

Published on

OSGi Community Event 2014

Abstract:
The main topic of the session is the content of the blog post Modularized Persistence: Development of reusable modules that handle relational persistent data.

Additional subjects of the session

Reasons why we chose this technology stack instead of JEE
Transaction handling with the transaction-helper component (without EJB or Spring)
Caching the persistent data based on everit-cache-api
More details about the already implemented use-cases (localization, authorization, authentication, etc.)
During the session, there will be live examples of:

Code generation of Querydsl Metadata classes (same as static metamodel in JPA)
Converting a standard query to one that contains authorization logic
Speaker's goal

Introducing our modules to others so they can:

use them as they are
start discussions about improvements so others can use them in the future
Speaker Bio:
Balazs Zsoldos is the co-founder of Everit. He is the leader of the development of Everit OpenSource Components. Developing Java based solutions is not only his job but also his passion.


He believes in simplicity. That is why he decided to design and build as many simple, but useful goal-oriented modules as he can. As the base of the stack, he chose OSGi.

Balazs does not believe in monoholitic frameworks, therefore all of the solutions that was designed by him can be used separately.

In the beginning of his career, Balazs was a big fan of JEE and Spring. After a while, he changed his mind and started to try replacing everything with non-magical solutions that do not contain interceptors, weaving, etc.

Published in: Technology
  • Be the first to comment

Modularized Persistence - B Zsoldos

  1. 1. Modularized Persistence Balázs Zsoldos
  2. 2. History of NoSQL by Mark Madse n, Picture published by Edd Dumbill
  3. 3. Blueprint + JPA + JSF Declarative Services + Modularized Persistence + JSF ECM + Modularized Persistence + JSF ECM + Modularized Persistence + Modularized Web Declarative Services + JPA + JSF
  4. 4. Stefan Seifert – Apache Sling & Friends Tech Meetup, Berlin 2012
  5. 5. JDBC Driver (DataSourceFactory)
  6. 6. DSF Component (XADataSource) JDBC Driver (DataSourceFactory)
  7. 7. DBCP Component (DataSource) DSF Component (XADataSource) JDBC Driver (DataSourceFactory)
  8. 8. ??? DBCP Component (DataSource) DSF Component (XADataSource) JDBC Driver (DataSourceFactory)
  9. 9. Liquibase DataSource Component (DataSource) DBCP Component (DataSource) DSF Component (XADataSource) JDBC Driver (DataSourceFactory)
  10. 10. <databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.resource.ri.schema" xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd"> <changeSet id="1.0.0" author="everit"> <createTable tableName="res_resource"> <column name="resource_id" type="bigint" autoIncrement="true"> <constraints primaryKeyName="pk_res_resource" primaryKey="true" nullable="false" /> </column> </createTable> </changeSet> </databaseChangeLog>
  11. 11. Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml" Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml";
  12. 12. Business Component Business Component Business Component Querydsl Querydsl Querydsl Liquibase DataSource Component (DataSource) DBCP Component (DataSource) DSF Component (XADataSource) JDBC Driver (DataSourceFactory)
  13. 13. Examples from http://querydsl.com
  14. 14. Write Liquibase changelog Write LQMG file Add LQMG to the Capability Add inclusions Generate Querydsl Metadata
  15. 15. <lqmg xmlns="http://everit.org/lqmg" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" defaultPackage="org.everit.osgi.resource.ri.schema.qdsl"> <namingRules> <classNameRule> <entity>res_resource</entity> <class>Resource</class> <propertyMappings> <primaryKey> <name>pk_res_resource</name> <property>resourcePk</property> </primaryKey> </propertyMappings> </classNameRule> </namingRules> </lqmg>
  16. 16. Write Liquibase changelog Write LQMG file Add LQMG to the Capability Add inclusions Generate Querydsl Metadata
  17. 17. Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml" Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml";
  18. 18. Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml";lqmg.config.re source="/META-INF/liquibase/resource.ri.lqmg.xml" Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml"; lqmg.config.resource="/META-INF/liquibase/resource.ri.lqmg.xml"
  19. 19. Write Liquibase changelog Write LQMG file Add LQMG to the Capability Add inclusions Generate Querydsl Metadata
  20. 20. <databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.authorization.ri.schema" xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd"> <include file="eosgi:org.everit.osgi.resource.ri" /> <include file="eosgi:org.everit.osgi.props.ri" /> <include file="eosgi:org.everit.osgi.resource.ri" /> include file="eosgi:org.everit.osgi.props.ri" /> <changeSet id="1.0.0" author="everit"> <createTable tableName="authr_permission"> <column name="authorized_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_p_a" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> <column name="target_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_p_t" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> <column name="action_" type="varchar(255)"> <constraints nullable="false" /> </column> </createTable> <addPrimaryKey constraintName="pk_authr_perm" tableName="authr_permission" columnNames="authorized_resource_id,target_resource_id,action_" /> <createTable tableName="authr_permission_inheritance"> <column name="parent_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_pi_p" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> <column name="child_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_pi_c" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> </createTable> <addPrimaryKey constraintName="pk_authr_perm_inheritance" tableName="authr_permission_inheritance" columnNames="parent_resource_id,child_resource_id" /> </changeSet> </databaseChangeLog>
  21. 21. Resource Authorization Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; ... Require-Capability: liquibase.schema; filter:=(name=org.everit.osgi.resource.ri) <databaseChangeLog ...> <include file="eosgi:org.everit.osgi.resource.ri" /> ... </databaseChangeLog>
  22. 22. Write Liquibase changelog Write LQMG file Add LQMG to the Capability Add inclusions Generate Querydsl Metadata
  23. 23. LQMG Start embedded OSGi container Deploy modules Start embedded H2 database Initialize database schema Generate Querydsl Metadata Evil magic here!!! If Capability is not found after deployments, unsatisfied bundles are enhanced in the way that unsatisfied requirements are marked to be optional.
  24. 24. Use-cases???
  25. 25. Monoholitic Portal (v5.1.2) Authorization Authentication User Documents Articles SiteMap Blog Modularized Portal Authorization (v1.1.0) Authentication (v2.0.0) User (v1.3.2) Documents (v1.0.1) Articles (v3.0.14) SiteMap (v1.0.3) Blog (v2.11.0)
  26. 26. User User Address User Address Country User Address Country Company
  27. 27. REAL PROJECT!!!
  28. 28. Authorization
  29. 29. Permission Resource ● resource_id ● authorized_resource_id ● action ● target_resource_id Permission Inheritance ● parent_resource_id ● child_resource_id
  30. 30. SELECT ... FROM document d JOIN document.attachment a WHERE … LIMIT 10 OFFSET 1000; EXISTS(SELECT 1 FROM permission p WHERE p.authorized_resource_id IN (?, …, ?) AND p.target_resource_id = d.resource_id AND action = ?) SELECT ... FROM document d JOIN document.attachment a WHERE EXISTS(SELECT 1 FROM permission p WHERE p.authorized_resource_id IN (?, …, ?) AND p.target_resource_id = d.resource_id AND action = ?) AND … LIMIT 10 OFFSET 1000;
  31. 31. @Override public BooleanExpression authorizationPredicate(final long authorizedResourceId, final Expression<Long> targetResourceId, final String... actions) { if (authorizedResourceId == systemResourceId) { return BooleanTemplate.TRUE; } Objects.requireNonNull(targetResourceId, "Parameter targetResourceId must not be null"); validateActionsParameter(actions); long[] authorizationScope = getAuthorizationScope(authorizedResourceId); SQLSubQuery subQuery = new SQLSubQuery(); QPermission permission = QPermission.permission; BooleanExpression authorizedResourceIdPredicate; if (authorizationScope.length == 1) { authorizedResourceIdPredicate = permission.authorizedResourceId.eq(authorizationScope[0]); } else { // More than one as the scope contains at least one value (other branch) Long[] authorizationScopeLongArray = new Long[authorizationScope.length]; for (int i = 0, n = authorizationScope.length; i < n; i++) { if (authorizationScope[i] == systemResourceId) { return BooleanTemplate.TRUE; } authorizationScopeLongArray[i] = authorizationScope[i]; } authorizedResourceIdPredicate = permission.authorizedResourceId.in(authorizationScopeLongArray); } BooleanExpression actionPredicate = null; if (actions.length == 1) { actionPredicate = permission.action.eq(actions[0]); } else { actionPredicate = permission.action.in(actions); } return subQuery.from(permission) .where(permission.targetResourceId.eq(targetResourceId).and( actionPredicate.and(authorizedResourceIdPredicate))) .exists(); }
  32. 32. SQLQuery query = new SQLQuery(connection, configuration); QDocument document = new QDocument("d"); BooleanExpression permissionCheckPredicate = authorizationQdslUtil .authorizationPredicate(userId, document.resourceId, actions); List<Long> list = query.from(document).where(permissionCheckPredicate) .list(document.documentId);
  33. 33. Business Component Business Component Business Component Querydsl Querydsl Querydsl Liquibase DataSource Component (DataSource) DBCP Component (DataSource) DSF Component (XADataSource) JDBC Driver (DataSourceFactory) QuerydslSupport Querydsl Configuration Querydsl SQLTemplates
  34. 34. @Reference(name = "querydslSupport", bind = "setQdsl") private QuerydslSupport qdsl; public void setQdsl(QuerydslSupport qdsl) { public List<Long> listDocumentIds() { return qdsl.execute((connection, configuration) -> { SQLQuery query = new SQLQuery(connection, configuration); QDocument document = new QDocument("d"); BooleanExpression permissionCheckPredicate = authorizationQdslUtil .authorizationPredicate(a1, document.resourceId, actions); List<Long> list = query.from(document).where(permissionCheckPredicate) .list(document.documentId); } } this.qdsl = qdsl; }
  35. 35. Business Component Business Component Business Component Querydsl Querydsl Querydsl Liquibase DataSource Component (DataSource) DBCP Component (DataSource) DSF Component (XADataSource) JDBC Driver (DataSourceFactory) QuerydslSupport Querydsl Configuration Querydsl SQLTemplates Transaction Helper
  36. 36. public long saveUser(String firstName, String lastName) { Objects.requireNonNull(firstName); Objects.requireNonNull(lastName); return transactionHelper.required(() -> { // Logic that should be implemented return 0l; }); }
  37. 37. Caching ● Available Cache modules: cache-api, cache-infinispan, cache-noop ● Caching should be done in the persistent module by the programmer, who knows the business logic ● Caching and table updates should be done within the same component ● In case bulk update is done in another module, cache must be cleared
  38. 38. OpenSource modules ● Resource ● Authorization ● Authentication ● Property Manager ● Blobstore (Before review) ● Audit (Before review)
  39. 39. Roadmap
  40. 40. HTML Output LQMG / Maven H2 Liquibase schema (Bundle-Capability) HTML
  41. 41. Manual Schema update LQMG / Maven Database Liquibase schema (Bundle-Capability) Generate Schema
  42. 42. Manual Schema generation Webconsole plugin DataSource (OSGi service) Liquibase schema (Bundle-Capability) Generate Schema Custom User / Password
  43. 43. Dump SQL file Webconsole plugin DataSource (OSGi service) Liquibase schema (Bundle-Capability) SQL
  44. 44. LQMG extension in changelog.xml <databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.resource.ri.schema" xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:lqmg="http://everit.org/xml/ns/lqmg" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd"> <changeSet id="1.0.0" author="everit"> <createTable tableName="res_resource" lqmg:class="QResource"> <column name="resource_id" type="bigint" autoIncrement="true" lqmg:property="resourceId"> <constraints primaryKeyName="pk_res_resource" primaryKey="true" nullable="false" /> </column> </createTable> </changeSet> </databaseChangeLog>
  45. 45. Full Text Search ● Analize the different solutions – H2 → Lucene – MySQL → Sphinx – PostgreSQL → TSearch2 – SQL Server →??? – Oracle → Oracle Text ● Create a common API for Querydsl based query extension ● Create a module for each database engine
  46. 46. Querydsl: http://www.querydsl.com/ Liquibase: http://www.liquibase.org/ Liquibase OSGi Bundle: https://github.com/everit-org/osgi-liquibase-bundle LQMG: https://github.com/everit-org/osgi-lqmg LQMG Maven Plugin: http://www.everit.org/lqmg-maven-plugin/ Liquibase DataSource: https://github.com/everit-org/osgi-liquibase-datasource Querydsl SQLTemplates Component: https://github.com/everit-org/osgi-querydsl-templates Querydsl Configuration Component: https://github.com/everit-org/osgi-querydsl-configuration Querydsl Support Component: https://github.com/everit-org/osgi-querydsl-support Cookbook chapter: http://cookbook.everit.org/persistence/index.html Download from Maven central Resource: https://github.com/everit-org/resource-ri Authorization: https://github.com/everit-org/authorization-ri Authentication: https://github.com/everit-org/authentication-simple Property Manager: https://github.com/everit-org/property-manager-ri Twitter: @EveritOrg

×