This document proposes a software architecture to address complex and dynamic data modeling challenges. The proposed solution has four main components: [1] An OSGi-based architecture for modularity, reusability and dynamic updates. [2] A graph database (Neo4j) to flexibly store relationships and enable natural queries. [3] A user interface built with AngularJS and D3.js for rich, data-driven visualization. [4] The use of a "mad developer" to implement the architecture. The architecture aims to reduce complexity, support dynamic data and provide a flexible yet user-friendly interface.

1. By Antonio Castellón (as Blue-Infinity consultant) , February, 2014
for Philip Morris International R&D
CIKB
Software Architecture Design Proposal

2. Problem : Data Complex

3. Problem : Data Complex to Model

4. Problem : Dynamic Data ( Uncertainty )
End User requirements and data itself sometimes generate
different types of uncertainty

5. Problem : GUI - User experience
I’m not stupid but
…. this interface
is too
complicated !!!

6. Problem : GUI - Adaptable + Flexible

7. Problem : GUI – Technology + Design
Be careful with awesome solutions that not fit
design and engineering at the same time

8. “The Solution” is a mix of 4 …

9. “The Solution” - Is a mix of …
An Architecture
A set of Data
A cool User Interface
And a mad developer to do it
(joke)

10. “The Solution” – Brick 1
An Architecture

11. Architecture – we aim to
• Reduce the complexity
• To be reusable
• Easy in deployment
• Allows dynamic updates
• To be adaptive
• Fast in responses
• Low memory profile
• To provide security
• …

12. Architecture – The response

13. Architecture – The response
Open Service Gateway initiative
Defines the standard.

14. Architecture – OSGi supported by

15. Architecture – OSGi implemented by …
. . .

16. Architecture – In summary, OSGi goals are …
Service Oriented +
Modular (bundles)
Bundle (x)
Service (x’)
Service (y)
Service (x)

17. Architecture – OSGi : Simple overview
Console Logging Admin …
Web Server
WAB
Application
1
WAB
Application
2
…
Application
Service 1
Application
Service 2
…
…
OSGi Instance 1
JVM
…
Bundles to be
developed for us
Bundles to be
installed

18. A set of Data
“The Solution” – Brick 2

19. Data
NoSQL
( Not Only SQL )

20. Data – NoSQL – Different implementations

21. Data - NoSQL – Comparing data structure
Image from: http://highlyscalable.wordpress.com/2012/03/01/nosql-data-modeling-techniques/

22. Data - NoSQL – Compare
98% of the business
requirements
There is still billions of
nodes and relationships

23. Data – Our selection
Graph
Databases

24. Data – Graph Databases – Why?
Flexible data structure
Doesn’t matter if the relations will change in the future.
Closer match to business logic

25. Data – Graph Databases – Why?
Natural query system
You tell what you want, not how to get it.
with recursive cluster (party, path, depth)
as ( select cast(@userId as character varying),
cast(@userId as character varying), 1
union
(
select (case
when this.party = amc.userA then amc.userB
when this.party = amc.userB then amc.userA
end), (this.path || '.' || (case
when this.party = amc.userA then amc.userB
when this.party = amc.userB then amc.userA
end)), this.depth + 1
from cluster this, chat amc
where ((this.party = amc.userA and
position(amc.userB in this.path) = 0)
or (this.party = amc.userB and position(amc.userA
in this.path) = 0)) AND this.depth < @depth + 1 )
)
select party, path
from cluster
where not exists (
select *
from cluster c2 where cluster.party = c2.party
and (
char_length(cluster.path) > char_length(c2.path)
or (char_length(cluster.path) =
char_length(c2.path)) and (cluster.path > c2.path)
)
)
order by party, path;
SQL = several hours to be executed
VS
START b = node:User(UserId=‘Manolo')
MATCH (b) --(friend)--(friendoffriend)
RETURN count(friendoffriend)
Cypher Language = 635ms

26. Data - Graph Databases – Why?
Fits very well with complex data

27. Data - Graph Databases – Why?
Fits very well with Bio-Informatics
0.9 Billion
relationsips

28. Data – Graph Databases – Why?
Fast Prototyping and development
We don’t need to lose too much time to define the schema (fine-grained).

29. Data - Graph Databases – What is it?
Properties
Labels
Relationships

30. Data - Graph Databases - Implemented by …

31. Data - Graph Databases - Compare
Name API Query
Methods
Consistency Staff (people) /
Community
OrientDB Java Traverser
API, Blueprints,
Rexster
Own SQL-like
Query
Language,
Gremlin
ACID, MVCC 3 / Low
Neo4j Java, Python,
JPython, Ruby,
JRuby,
JavaScript
(Node.js), PHP,
.NET, Django,
Clojure, Spring,
Scala, or REST
(any language)
Cypher
(native/preferre
d), Native Java
APIs (special
cases),
Traverser API,
REST,
Blueprints,
Gremlin
ACID 42 / Very High
DEX Java, C++,
.NET
Native Java, C#
and C++ APIs,
Blueprints,
Gremlin
Consistency,
durability and
partial isolation
and atomicity
5 / ?

32. Data - Graph Databases – Compare

33. Data - Graph Databases - Neo4j customers

34. Data - Graph Database - Neo4j - Partners

35. Data - Graph Database - Neo4j - Licenses

36. “The Solution” – Brick 3
A cool User Interface

37. GUI
+

38. GUI
UI Graphs
Model / View / Controller
( on Browser using Jscript )
JAX-RS (RESTful web services)
JSON responses
On OSGi bundle as a webservice
On Browser client
Data Driven Documents

39. GUI - AngularJS – What is it?
RESTful
+
JSON

40. GUI - D3.js – What is it?

41. GUI - D3.js – Rich and cool interfaces

42. GUI - Examples

43. GUI - Licenses
No requires any payment to use or to modify their code.

44. “The Solution” – The last brick
At least a mad developer to
do it (joke)

45. Architecture – Current draft
KARAF :: OSGi kernel platform
Shell admin
web admin
console
ServiceMix (Optional) :: Enterprise Service Bus
Groovy 2.2.1
Runtime
Jetty Server
8.1.9 Runtime
CIKB
Neo4j 2.0.0
Server
Core ( Business )
Database
connector
CVS
connector
SAW
connector
LIMS
connector
User Portal
UCSD
Connector
XML
Connector
AngularJS + D3.js
…
Admin
Portal
…

46. Thanks you for your attention.
End