www.infoadvisors.com Aug 2014 
1 
Big Data, NoSQL & 
Data Modeling 
10 Tips for Data Modeling Success on Modern Data Proje...
www.infoadvisors.com Aug 2014 
2 
POLL: Who Are You? 
©InfoAdvisors - infoadvisors.com Aug 2014 
©InfoAdvisors - infoadvis...
www.infoadvisors.com Aug 2014 
3 
“BIG DATA” 
[x] Vs 
“Data so big it’s awkward to work with” 
Always capitalized Big Data...
www.infoadvisors.com Aug 2014 
4 
Terminology 
ACID 
Atomic, Consistent, Isolated, Durable 
BASE 
Basically available, Sof...
www.infoadvisors.com Aug 2014 
5 
Data Models – Traditional Process 
Conceptual 
(Data) 
Model 
Logical 
Data Model 
MART ...
www.infoadvisors.com Aug 2014 
6 
Traditional Data Modeler Involvement 
Project 
Initiation 
Architecture 
and 
Infrastruc...
www.infoadvisors.com Aug 2014 
7 
What do we really mean by scale? 
Bringing computing to the data 
Massively parallel pro...
www.infoadvisors.com Aug 2014 
8 
We’ve been down this road before… 
Traditional 
transactional 
applications 
Reporting-o...
www.infoadvisors.com Aug 2014 
9 
Hadoop 
ETL 
EDW 
Analytics 
Mart 
Data 
Mart 
NoSQL, Not Only SQL 
Relational Graph 
Co...
www.infoadvisors.com Aug 2014 
10 
Graph Databases 
©InfoAdvisors - infoadvisors.com Aug 2014 
Key Value Pair 
©InfoAdviso...
www.infoadvisors.com Aug 2014 
11 
Columnar 
©InfoAdvisors - infoadvisors.com Aug 2014 
Sample Hive Statement 
CREATE EXTE...
www.infoadvisors.com Aug 2014 
12 
Sample JSON/MongoDB Notation 
©InfoAdvisors - infoadvisors.com Aug 2014 
Sample Foundat...
www.infoadvisors.com Aug 2014 
13 
Sample Cassandra Statement 
©InfoAdvisors - infoadvisors.com Aug 2014 
Sample Vertica S...
www.infoadvisors.com Aug 2014 
14 
Sample Neo4j Statement 
©InfoAdvisors - infoadvisors.com Aug 2014 
What else is differe...
www.infoadvisors.com Aug 2014 
15 
The Big Data Big Lies 
Schemaless 
• Schema on 
Read, not 
Schema on Write 
• Polyschem...
www.infoadvisors.com Aug 2014 
16 
10 Tips For Modeling in a Hybrid World 
1. Models require a modeler 
2. Data modeling t...
www.infoadvisors.com Aug 2014 
17 
Modern Data Modeler Involvement 
Project 
Initiation 
Architecture 
and 
Infrastructure...
www.infoadvisors.com Aug 2014 
18 
Business Intelligence Journal 
Look for September 
Issue Article on Modern 
Data Archit...
Upcoming SlideShare
Loading in …5
×

Data Modeling for Big Data & NoSQL Technologies with Karen Lopez

1,415 views

Published on

Watch the companion webinar for this presentation at http://embt.co/KLopez826. In this webinar, Karen Lopez of InfoAdvisors will cover 10 tips for the modern data architect and resources for coming up to speed on these new approaches. She will share how modern data modeling approaches address both SQL (relational) and NoSQL technologies. We'll look at the role of a data modeler, and how models, processes and data governance processes can add value to enterprise big data and NoSQL development projects.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,415
On SlideShare
0
From Embeds
0
Number of Embeds
29
Actions
Shares
0
Downloads
74
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Data Modeling for Big Data & NoSQL Technologies with Karen Lopez

  1. 1. www.infoadvisors.com Aug 2014 1 Big Data, NoSQL & Data Modeling 10 Tips for Data Modeling Success on Modern Data Projects Karen Lopez, InfoAdvisors www.datamodel.com Karen López Karen has 20+ years of data an information architecture experience on large, multi-project programs. She is a frequent speaker on data modeling, data-driven methodologies and pattern data models. She wants you to love your data. ©InfoAdvisors - infoadvisors.com #TEAMDATA Aug 2014
  2. 2. www.infoadvisors.com Aug 2014 2 POLL: Who Are You? ©InfoAdvisors - infoadvisors.com Aug 2014 ©InfoAdvisors - infoadvisors.com POLL: NoSQL Much? Aug 2014
  3. 3. www.infoadvisors.com Aug 2014 3 “BIG DATA” [x] Vs “Data so big it’s awkward to work with” Always capitalized Big Data A confusing term because it defines what it IS NOT. ©InfoAdvisors - infoadvisors.com Aug 2014 “NoSQL” Scale Not SQL? Not Relational? Not Only Relational? A confusing term because it defines what it IS NOT. ©InfoAdvisors - infoadvisors.com Aug 2014
  4. 4. www.infoadvisors.com Aug 2014 4 Terminology ACID Atomic, Consistent, Isolated, Durable BASE Basically available, Soft state, Eventual consistency Eventual consistency Schemaless Constraints / Have-to/ MUST / OBEY / Rigid / Inflexible ©InfoAdvisors - infoadvisors.com Aug 2014 Relational Tables with rows same columns with the same datatypes with the same constraints with the same domains This is a FEATURE On purpose With many benefits Write-optimized Trans-action-optimized Data integrity Data quality Consistent ©InfoAdvisors - infoadvisors.com Aug 2014
  5. 5. www.infoadvisors.com Aug 2014 5 Data Models – Traditional Process Conceptual (Data) Model Logical Data Model MART MART Physical Data Model(s) OLTP OLTP OLTP OLTP OLTP OLTP OLTP OLTP ©InfoAdvisors - infoadvisors.com Aug 2014 Relational ©InfoAdvisors - infoadvisors.com Aug 2014
  6. 6. www.infoadvisors.com Aug 2014 6 Traditional Data Modeler Involvement Project Initiation Architecture and Infrastructure Design SW Requirements Development Deployment ©InfoAdvisors - infoadvisors.com Aug 2014 The Big Data Story Lots of data Coming at us fast Lots of variety in format & quality We want all the data Highly available “It’s web scale” ©InfoAdvisors - infoadvisors.com Aug 2014
  7. 7. www.infoadvisors.com Aug 2014 7 What do we really mean by scale? Bringing computing to the data Massively parallel processing Cheap, commodity hardware, but lots of it Optimized for Query/Reads/Questions/Telling stories ©InfoAdvisors - infoadvisors.com Aug 2014 Can we fit another buzzword in? Cloud • Enable on-demand scaling • Pay as you go pricing • Click to deploy • Service licensing, not product licensing, if any • Managed by others, not your data center ©InfoAdvisors - infoadvisors.com Aug 2014
  8. 8. www.infoadvisors.com Aug 2014 8 We’ve been down this road before… Traditional transactional applications Reporting-optimized tables/structures Data Warehouse / Dimensional Modeling ©InfoAdvisors - infoadvisors.com Aug 2014 ETL EDW Data Mart Data Mart
  9. 9. www.infoadvisors.com Aug 2014 9 Hadoop ETL EDW Analytics Mart Data Mart NoSQL, Not Only SQL Relational Graph Columnar/Column Family Key Value Document Databases Others ©InfoAdvisors - infoadvisors.com Aug 2014
  10. 10. www.infoadvisors.com Aug 2014 10 Graph Databases ©InfoAdvisors - infoadvisors.com Aug 2014 Key Value Pair ©InfoAdvisors - infoadvisors.com Aug 2014
  11. 11. www.infoadvisors.com Aug 2014 11 Columnar ©InfoAdvisors - infoadvisors.com Aug 2014 Sample Hive Statement CREATE EXTERNAL TABLE TaxRebateUsage ( state string, zipcode string, agi_class int, n1 int, mars2 int, prep int, n2 int, ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE ©InfoAdvisors - infoadvisors.com Aug 2014
  12. 12. www.infoadvisors.com Aug 2014 12 Sample JSON/MongoDB Notation ©InfoAdvisors - infoadvisors.com Aug 2014 Sample FoundationDB Statement ©InfoAdvisors - infoadvisors.com Aug 2014
  13. 13. www.infoadvisors.com Aug 2014 13 Sample Cassandra Statement ©InfoAdvisors - infoadvisors.com Aug 2014 Sample Vertica Statement ©InfoAdvisors - infoadvisors.com Aug 2014
  14. 14. www.infoadvisors.com Aug 2014 14 Sample Neo4j Statement ©InfoAdvisors - infoadvisors.com Aug 2014 What else is different, now? Maturing of high availability technologies Maturing of ____ as a Service business models RDBMS vendors adopting non-relational features Open source software models 100s of Database options ©InfoAdvisors - infoadvisors.com Aug 2014
  15. 15. www.infoadvisors.com Aug 2014 15 The Big Data Big Lies Schemaless • Schema on Read, not Schema on Write • Polyschematic Big • New data stories • New technologies ©InfoAdvisors - infoadvisors.com Aug 2014 Importing Structures ©InfoAdvisors - infoadvisors.com Aug 2014
  16. 16. www.infoadvisors.com Aug 2014 16 10 Tips For Modeling in a Hybrid World 1. Models require a modeler 2. Data modeling tools are essential 3. There are many types of data models: know which ones you need 4. Modeling does not have to happen at the same time in every project. It should happen at the right time 5. Modeling is not just schema design. Think outside the boxes and lines ©InfoAdvisors - infoadvisors.com Aug 2014 10 Tips for Modeling in a Hybrid World 6. A data model is much more than a diagram 7. You will need training. 8. Team members may not understand modeling. They will need training 9. NoSQL is not one thing. Learn many patterns 10.Modern data architectures are likely hybrid solutions. You can’t just support one part. ©InfoAdvisors - infoadvisors.com Aug 2014
  17. 17. www.infoadvisors.com Aug 2014 17 Modern Data Modeler Involvement Project Initiation Architecture and Infrastructure Design SW Requirements Development Deployment ©InfoAdvisors - infoadvisors.com Aug 2014 What does this mean for data modelers? There will be jobs for traditional, ERD, relational modelers…. ….just like there are still jobs of RPG and COBOL programmers All data has a data story. Many data stories. A good modeler is a an architect at heart – finding the right solution for the data story. ©InfoAdvisors - infoadvisors.com Aug 2014
  18. 18. www.infoadvisors.com Aug 2014 18 Business Intelligence Journal Look for September Issue Article on Modern Data Architectures ©InfoAdvisors - infoadvisors.com Aug 2014 Thank You! www.infoadvisors.com www.datamodel.com www.dataversity.net #TEAMDATA ©InfoAdvisors - infoadvisors.com Aug 2014

×