Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hybrid data-arch


Published on

The goal of hybrid data architecture is to employ multiple data stores to store your application data in order to maximize your application’s performance, scalability, stability, and extensibility.

Published in: Software
  • Rapidly determine the right choice of databases for your business-critical applications - Shelloid Data Architecture Research Tool (DART) at
    Are you sure you want to  Yes  No
    Your message goes here

Hybrid data-arch

  1. 1. Hybrid Data Architecture Unlock the full potential of your business-critical applications Dr. Jayaraj Poroor
  2. 2. Data architecture defines Software architecture “Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowcharts; they'll be obvious.” Fred Brooks Computer Pioneer & Turing Award* Winner * Turing Award is Computer Science equivalent of Nobel Prize
  3. 3. Benefits of proper data architecture ● Stability ● Scalability ● Performance ● Security ● Reduced code complexity ● Extensibility
  4. 4. Data Architecture: Good vs Bad Application Requirements Wrong Data Architecture Application Requirements Right Data Architecture
  5. 5. Not all application data are equal Basic Domain Data Time-tagged Log Data Relationship Data Unstructured Text Session Data The same application may have diverse data requirements.
  6. 6. Force fit all data into the same data store? ● Poor performance/scalability ● Complex application code ○ Slow/buggy Application’s diverse data requirements. A single data store won’t fit all requirements
  7. 7. Hybrid Data Architecture ● Use the right kind of data store for each different kind of data requirement.
  8. 8. Data stores: sheer variety 1. Key-value data stores, e.g., Memcached, Redis, Amazon SimpleDB, Amazon S3. 2. Distributed key-value data stores, e.g., Riak, Amazon DynamoDB. 3. In-process key-value data stores, e.g., Level-DB, Berkeley DB, MapDB. 4. Unstructured/semi-structured search databases, e.g., ElasticSearch, Apache SOLR. 5. Time series databases, e.g., TempoDB, InfluxDB, OpenTSDB. 6. Column family databases, e.g., Apache HBase, Cassandra, Google BigTable (Cloud Datastore). 7. Document databases, e.g. CouchBase, CouchDB, MongoDB, Terrastore, RavenDB. 8. Graph databases, e.g., Neo4J, InfiniteGraph, InfoGrid, OrientDB, FlockDB. 9. SQL databases, e.g., MySQL, MariaDB, PostgreSQL. 10. In-process (embedded) SQL databases, e.g., HSQLDB, Apache Derby, Sqlite. 11. XML databases, e.g., ExistDB, Sedna, BaseX. 12. Logic-based data stores, e.g., Datomic, Jena.
  9. 9. About Me ● 19+ years of experience in industry and research settings. ○ Holds PhD in Computer Science ● Has been architecting software systems for 14 years. ● Hands-on experience with diverse programming languages/platforms in web/mobile/embedded worlds.
  10. 10. Hybrid Data Architecture consultancy services ● Understanding your application requirements and developing a hybrid data architecture and design that best suites your application needs. ○ I can advise at CTO level or directly work with tech leads. ● Giving hands-on training sessions to the developers to help them come up to speed on various database technologies. ● Helping developers integrate hybrid data access into the application code. ● Helping your sysadmin get the various databases installed and configured in your servers or in a cloud platform such as the AWS.
  11. 11. Real project cases (1) Migrating from pure MySQL to hybrid MySQL/Amazon S3 improved an application’s performance by a factor of 5 and reduced the main transactional database size by a factor of 50. (2) Migrating from pure MySQL-based system to a hybrid architecture of MySQL/ElasticSearch/Neo4J reduced the application code complexity significantly and reduced development/testing time.
  12. 12. Real project cases (2) (3) Employing a hybrid architecture of MySQL/Redis/InfluxDB a highly scalable and stable IoT back-end application was developed in 3 month’s time by 2 developers. (4) Search performance significantly improved for a web application by employing a hybrid architecture of MySQL/ElasticSearch. (5) Employing Mapdb at the client-side greatly enhanced user experience, allowing disconnected operation.
  13. 13. Thank You Realize the full potential of your business-critical applications through hybrid data architecture.