The goal of hybrid data architecture is to employ multiple data stores to store your application data in order to maximize your application’s performance, scalability, stability, and extensibility.

1. Hybrid
Data Architecture
Unlock the full potential of
your business-critical applications
Dr. Jayaraj Poroor
http://jayaraj.poroor.com

2. Data architecture defines
Software architecture
“Show me your flowcharts and conceal your tables, and I
shall continue to be mystified. Show me your tables, and I
won't usually need your flowcharts; they'll be obvious.”
Fred Brooks
Computer Pioneer & Turing Award* Winner
* Turing Award is Computer Science equivalent of Nobel Prize

3. Benefits of proper data architecture
● Stability
● Scalability
● Performance
● Security
● Reduced code complexity
● Extensibility

4. Data Architecture: Good vs Bad
Application
Requirements
Wrong Data Architecture
Application
Requirements
Right Data Architecture

5. Not all application data are equal
Basic
Domain
Data
Time-tagged
Log Data
Relationship
Data
Unstructured
Text
Session
Data
The same application may have diverse data requirements.

6. Force fit all data into the same
data store?
● Poor performance/scalability
● Complex application code
○ Slow/buggy
Application’s diverse data
requirements.
A single data store won’t fit all
requirements

7. Hybrid Data Architecture
● Use the right kind of data store for each
different kind of data requirement.

8. Data stores: sheer variety
1. Key-value data stores, e.g., Memcached, Redis, Amazon SimpleDB, Amazon S3.
2. Distributed key-value data stores, e.g., Riak, Amazon DynamoDB.
3. In-process key-value data stores, e.g., Level-DB, Berkeley DB, MapDB.
4. Unstructured/semi-structured search databases, e.g., ElasticSearch, Apache SOLR.
5. Time series databases, e.g., TempoDB, InfluxDB, OpenTSDB.
6. Column family databases, e.g., Apache HBase, Cassandra, Google BigTable (Cloud Datastore).
7. Document databases, e.g. CouchBase, CouchDB, MongoDB, Terrastore, RavenDB.
8. Graph databases, e.g., Neo4J, InfiniteGraph, InfoGrid, OrientDB, FlockDB.
9. SQL databases, e.g., MySQL, MariaDB, PostgreSQL.
10. In-process (embedded) SQL databases, e.g., HSQLDB, Apache Derby, Sqlite.
11. XML databases, e.g., ExistDB, Sedna, BaseX.
12. Logic-based data stores, e.g., Datomic, Jena.

9. About Me
● 19+ years of experience in industry and
research settings.
○ Holds PhD in Computer Science
● Has been architecting software systems
for 14 years.
● Hands-on experience with diverse
programming languages/platforms in
web/mobile/embedded worlds.

10. Hybrid Data Architecture
consultancy services
● Understanding your application requirements and developing a hybrid data
architecture and design that best suites your application needs.
○ I can advise at CTO level or directly work with tech leads.
● Giving hands-on training sessions to the developers to help them come up
to speed on various database technologies.
● Helping developers integrate hybrid data access into the application code.
● Helping your sysadmin get the various databases installed and configured
in your servers or in a cloud platform such as the AWS.

11. Real project cases
(1) Migrating from pure MySQL to hybrid MySQL/Amazon
S3 improved an application’s performance by a factor of
5 and reduced the main transactional database size by
a factor of 50.
(2) Migrating from pure MySQL-based system to a hybrid
architecture of MySQL/ElasticSearch/Neo4J reduced
the application code complexity significantly and
reduced development/testing time.

12. Real project cases (2)
(3) Employing a hybrid architecture of
MySQL/Redis/InfluxDB a highly scalable and stable IoT
back-end application was developed in 3 month’s time
by 2 developers.
(4) Search performance significantly improved for a web
application by employing a hybrid architecture of
MySQL/ElasticSearch.
(5) Employing Mapdb at the client-side greatly enhanced
user experience, allowing disconnected operation.

13. Thank You
Realize the full potential of your business-critical
applications through hybrid data architecture.
jayaraj.poroor@gmail.com
http://jayaraj.poroor.com