Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NoSQL and Data Modeling for Data Modelers

2,335 views

Published on

Karen Lopez's presentation for data modelers and data architects. Why data modeling is still relevant for big data and NoSQL projects.

Plus 10 tips for data modelers for working on NoSQL projects.

Published in: Data & Analytics
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

NoSQL and Data Modeling for Data Modelers

  1. 1. Big Data, NoSQL & Data Modeling 10 Tips for Data Modeling Success on Modern Data Projects Karen Lopez, InfoAdvisors www.datamodel.com
  2. 2. Data Models – Traditional Process Conceptual (Data) Model Logical Data Model Physical Data Model(s) OLTP OLTP OLTP OLTP OLTP MARTMART OLTP OLTP OLTP Aug 2014©InfoAdvisors - infoadvisors.com
  3. 3. Relational Aug 2014©InfoAdvisors - infoadvisors.com Data Models started with relational modeling, so they look like relational database structures.
  4. 4. But…. That doesn’t mean they can’t be used to model data that goes into a non- relational format. All that formatting happens at build OR consumption time, not requirements time. Aug 2014©InfoAdvisors - infoadvisors.com
  5. 5. The Big Data Story Lots of data Coming at us fast Lots of variety in format & quality We want all the data Highly available “It’s web scale” Aug 2014©InfoAdvisors - infoadvisors.com
  6. 6. What do we really mean by scale? Bringing computing to the data Massively parallel processing Cheap, commodity hardware, but lots of it Optimized for Query/Reads/Questions/Telling stories Aug 2014©InfoAdvisors - infoadvisors.com
  7. 7. We’ve been down this road before… Traditional transactional applications Reporting- optimized tables/structures Data Warehouse / Dimensional Modeling Aug 2014©InfoAdvisors - infoadvisors.com Highly normalized Highly Denormalized
  8. 8. ETL EDW Data Mart Data Mart
  9. 9. Hadoop ETL EDW Analytics Mart Data Mart
  10. 10. NoSQL, Not Only SQL Relational Graph Columnar/Column Family Key Value Document Databases Others Aug 2014©InfoAdvisors - infoadvisors.com
  11. 11. Sample Hive Statement CREATE EXTERNAL TABLE TaxRebateUsage ( state string, zipcode string, agi_class int, n1 int, mars2 int, prep int, n2 int, ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE Aug 2014©InfoAdvisors - infoadvisors.com
  12. 12. Sample JSON/MongoDB Notation Aug 2014©InfoAdvisors - infoadvisors.com
  13. 13. Sample FoundationDB Statement Aug 2014©InfoAdvisors - infoadvisors.com
  14. 14. Sample Cassandra Statement Aug 2014©InfoAdvisors - infoadvisors.com
  15. 15. Sample Vertica Statement Aug 2014©InfoAdvisors - infoadvisors.com
  16. 16. Sample Neo4j Statement Aug 2014©InfoAdvisors - infoadvisors.com
  17. 17. Those weren’t SCHEMALESS…. They had data facts, which had meanings. And sometimes expected formats, precisions, and types. In the NoSQL world, we don’t apply those necessarily at write time, but at read time. SCHEMALESS really is MULTIPLE SCHEMAs (Polyschematic) or VARYING SCHEMAs. Aug 2014©InfoAdvisors - infoadvisors.com
  18. 18. The Big Data Big Lies Schemaless • Schema on Read, not Schema on Write • Polyschematic Big • New data stories • New technologies • Not just volume Aug 2014©InfoAdvisors - infoadvisors.com
  19. 19. 10 Tips For Modeling in a Hybrid World 1. Models require a modeler 2. Data modeling tools are essential 3. There are many types of data models: know which ones you need 4. Modeling does not have to happen at the same time in every project. It should happen at the right time 5. Modeling is not just schema design. Think outside the boxes and lines Aug 2014©InfoAdvisors - infoadvisors.com
  20. 20. 10 Tips for Modeling in a Hybrid World 6. A data model is much more than a diagram 7. You will need training. 8. Team members may not understand modeling. They will need training 9. NoSQL is not one thing. Learn many patterns 10.Modern data architectures are likely hybrid solutions. You can’t just support one part. Aug 2014©InfoAdvisors - infoadvisors.com
  21. 21. What does this mean for data modelers? There will be jobs for traditional, ERD, relational modelers…. ….just like there are still jobs of RPG and COBOL programmers All data has a data story. Many data stories. A good modeler is a an architect at heart – finding the right solution for the data story. Aug 2014©InfoAdvisors - infoadvisors.com
  22. 22. Business Intelligence Journal Look for September 2014 Issue Article on Modern Data Architectures Aug 2014©InfoAdvisors - infoadvisors.com
  23. 23. Thank You! www.infoadvisors.com www.datamodel.com www.dataversity.net community.embarcadero.com #TEAMDATA Aug 2014©InfoAdvisors - infoadvisors.com

×