Data	  Modelers	  Save	  Their	  Careers:	  Surviving	  and	  Thriving	  with	  NoSQL	  	  Joe	  Maguire	  Data	  Quality	...
Thesis	  •  RelaIonal	  DBMS’s	  have	  dominated,	  •  ...so	  relaIonal	  modeling	  subsumed	  other	  forms,	  includi...
My	  PerspecIve	  •  Over	  three	  decades	  in	  industry	  •  Career	  is	  a	  three-­‐legged	  stool	  –  Product	  d...
Agenda	  •  History	  •  Current	  Events	  •  Your	  Future	  as	  a	  Data	  Modeler	  •  Q&A	  #Cassandra13	   ©	  2013...
A	  Big-­‐Picture	  Framework	  #Cassandra13	   ©	  2013	  Data	  Quality	  Strategies,	  LLC	   5	  	  Meta-­‐model	  	  ...
Good	  Ideas	  in	  the	  Framework	  •  InformaIon	  Hiding	  –  e.g.,	  conceptual	  excludes	  implementaIon	  details	...
A	  Big-­‐Picture	  Framework,	  distorted	  #Cassandra13	   ©	  2013	  Data	  Quality	  Strategies,	  LLC	   7	  	  Meta-...
How	  the	  DistorIon	  Happens	  •  Tool	  Vendors	  Dismiss	  Conceptual	  Modeling	  – Because	  their	  tools	  cannot...
DistorIons,	  Revisited	  •  Summary	  of	  DistorIons:	  – DistorIon:	  Conceptual	  means	  vague	  – DistorIon:	  Logic...
Agenda	  •  History	  •  Current	  Events	  •  Your	  Future	  as	  a	  Data	  Modeler	  •  Q&A	  #Cassandra13	   ©	  2013...
Current	  Events:	  NoSQL	  •  The	  “Just	  Say	  No”	  InterpretaIon	  #Cassandra13	   ©	  2013	  Data	  Quality	  Strat...
Current	  Events:	  NoSQL	  #Cassandra13	   ©	  2013	  Data	  Quality	  Strategies,	  LLC	   12	  •  The	  “Not	  Only	  S...
Agenda	  •  History	  •  Current	  Events	  •  Your	  Future	  as	  a	  Data	  Modeler	  •  Summary	  •  Q&A	  #Cassandra1...
Your	  Future	  as	  a	  Modeler	  #Cassandra13	   ©	  2013	  Data	  Quality	  Strategies,	  LLC	   14	  •  Remaining	  Re...
A	  New	  (?)	  Modeling	  Framework	  •  Conceptual	  Modeling	  •  Choosing	  a	  Logical	  Meta-­‐model	  •  Logical	  ...
Conceptual	  Modeling	  •  Behaviors	  and	  constructs	  will	  compare	  to	  RelaIonal	  Modeling:	  – Keep	  some	  – ...
Conceptual	  Data	  Model	  Example	  #Cassandra13	   ©	  2013	  Data	  Quality	  Strategies,	  LLC	   17	  
Keep	  Some	  •  Keep	  EnIIes	  •  Keep	  A=ributes	  •  Keep	  RelaIonships	  •  Keep	  IdenIfiers	  •  Keep	  Maximum	  ...
Keep	  EnIIes	  •  Minimum	  Expressiveness	  •  EnIIes,	  Not	  Tables	  – Don’t	  express	  Horizontal	  or	  VerIcal	  ...
Keep	  A=ributes	  •  Honor	  User	  Phenomenon	  – A=ributes	  are	  part	  of	  user	  discourse	  •  A=ributes,	  not	 ...
Keep	  RelaIonships	  •  Minimum	  Expressiveness	  – A=ributes	  are	  part	  of	  user	  discourse	  •  Allow	  many-­‐m...
Keep	  IdenIfiers	  •  IdenIfiers,	  not	  PKs	  – IDs	  are	  not	  moIvated	  by	  computerizaIon,	  but	  by	  typography...
Discard	  Some	  •  Discard	  Foreign	  Keys	  – They’re	  relaIonal	  •  Discard	  Minimum	  Cardinality	  – A	  funcIon	...
Keep/Discard	  Rule	  of	  Thumb	  •  Keep	  – Anything	  that	  helps	  you	  and	  the	  users	  together	  discover	  a...
Conceptual	  Data	  Model	  Examples	  #Cassandra13	   ©	  2013	  Data	  Quality	  Strategies,	  LLC	   25	  
Stress	  Some	  •  Stress	  Consistency	  Requirements	  – RelaIonal	  modelers	  (of	  non-­‐distributed	  databases)	  h...
Change	  Some	  •  Change	  your	  process	  – From	  math-­‐y	  normalizaIon	  to	  English-­‐y	  conversaIon	  with	  us...
A	  New	  Modeling	  Framework	  •  Conceptual	  Modeling	  •  Choosing	  a	  Logical	  Meta-­‐Model	  •  Logical	  Modeli...
Choosing	  a	  Logical	  Meta-­‐Model	  •  Don’t	  Assume	  RelaIonal	  (Duh...)	  •  Don’t	  Assume	  Big	  Table	  •  Lo...
A	  New	  Modeling	  Framework	  •  Conceptual	  Modeling	  •  Choosing	  a	  Logical	  Meta-­‐Model	  •  Logical	  Modeli...
Logical,	  Physical,	  and	  Tool	  Support	  •  Community	  needs	  to	  develop	  a	  roster	  of	  shapes	  – And	  the...
Agenda	  •  History	  •  Current	  Events	  •  Your	  Future	  as	  a	  Data	  Modeler	  •  Summary	  •  Q&A	  #Cassandra1...
Summary	  •  Re-­‐commit	  to	  conceptual	  modeling	  for	  requirements	  analysis	  – Some	  but	  not	  all	  relaIon...
Summary	  •  Remember	  the	  fundamentals,	  so	  that	  you	  can	  make	  informed	  decisions	  about	  relaxing	  the...
Q&A	  •  Joe.Maguire@DataQualityStrategies.com	  •  www.DataQualityStrategies.com	  #Cassandra13	   ©	  2013	  Data	  Qual...
Upcoming SlideShare
Loading in...5
×

C* Summit 2013: Data Modelers Still Have Jobs - Adjusting For the NoSQL Environment by Joe Maguire

856

Published on

Using concrete, real-world examples, the presenter will show the following: How abandoning modeling altogether is a recipe for disaster, even in—or especially in—NoSQL environments; How experienced relational modelers can leverage their skills for NoSQL projects; How the NoSQL context both simplifies and complicates the modeling endeavor.How lessons learned modeling for NoSQL projects can make you a more effective modeler for any kind of project.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
856
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
23
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

C* Summit 2013: Data Modelers Still Have Jobs - Adjusting For the NoSQL Environment by Joe Maguire

  1. 1. Data  Modelers  Save  Their  Careers:  Surviving  and  Thriving  with  NoSQL    Joe  Maguire  Data  Quality  Strategies,  LLC  h=p://www.DataQualityStrategies.com/    ©  2013  Data  Quality  Strategies,  LLC  
  2. 2. Thesis  •  RelaIonal  DBMS’s  have  dominated,  •  ...so  relaIonal  modeling  subsumed  other  forms,  including  conceptual  modeling.  •  As  R-­‐DBMS  wanes,  so  does  relaIonal  modeling  –  and  sadly,  whatever  it  subsumed.  •  Conceptual  modeling  must  be  saved.  •  RelaIonal  modelers  can  step  in  to  save  it...  •  ...with  some  significant  effort.  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   2  
  3. 3. My  PerspecIve  •  Over  three  decades  in  industry  •  Career  is  a  three-­‐legged  stool  –  Product  development  for  soVware  vendors  –  SoluIon  design  for  enterprises  –  Author,  Industry  Analyst,  Thought  Leader    •  Specialize  in    –  Modeling  –  Requirements  analysis  –  Data  architecture  –  Data  quality  •  Joe.Maguire@DataQualityStrategies.com    #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   3  
  4. 4. Agenda  •  History  •  Current  Events  •  Your  Future  as  a  Data  Modeler  •  Q&A  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   4  
  5. 5. A  Big-­‐Picture  Framework  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   5    Meta-­‐model    Data  Perspec1ve  Conceptual   •  EnIIes  •  A=ributes  •  RelaIonships  •  IdenIfiers  Logical   •  Tables  •  Columns  •  Primary  and  foreign  keys  Physical   •  Indexes  •  Table  spaces  •  VerIcal  and  horizontal  parIIoning  •  DenormalizaIons  
  6. 6. Good  Ideas  in  the  Framework  •  InformaIon  Hiding  –  e.g.,  conceptual  excludes  implementaIon  details  •  The  Type/Instance  disIncIon  –  Models  describe  categories,  data  describes  members  •  ApplicaIon/Data  Independence  –  Data  modeling  is  separate  from  process  modeling  •  User  Requirements  ≠  System  Requirements  –  Users  should  not  parIcipate  in  logical  and  physical    •  Model-­‐Driven  Development  –  Forward  and  reverse  engineering  across  model  levels    #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   6  
  7. 7. A  Big-­‐Picture  Framework,  distorted  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   7    Meta-­‐model    Data  Perspec1ve  RelaIonal   •  EnIIes  /  Tables  •  A=ributes  /  Columns  •  RelaIonships  /  FKs  •  IdenIfiers  /  PKs        Physical   •  Indexes  •  Table  spaces  •  VerIcal  and  horizontal  parIIoning  •  DenormalizaIons  
  8. 8. How  the  DistorIon  Happens  •  Tool  Vendors  Dismiss  Conceptual  Modeling  – Because  their  tools  cannot  support  it  anyway  •  Info  Mgmt  Specialists  Confuse  Models  w  Reality  – E.g.,  believing  the  relaIonal  model  suffices  to  describe  the  universe  •  InsItuIonalized  Expediency    – We  know  about  conceptual  modeling,  but  to  save  Ime,  we  combine  it  with  relaIonal  modeling...  – ...then  we  formalize  that  into  our  dev  processes...  – ...and  eventually,  that  becomes  the  “best  pracIces.”  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   8  
  9. 9. DistorIons,  Revisited  •  Summary  of  DistorIons:  – DistorIon:  Conceptual  means  vague  – DistorIon:  Logical  implies  relaIonal  •  Rather  than  XML,  OO,  KV  Store,  Array  Database,  Graph  Database  •  Results  of  DistorIons:  – Two  levels  only:  relaIonal  and  physical  – RelaIonal  modeling  used  for  user  requirements  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   9  
  10. 10. Agenda  •  History  •  Current  Events  •  Your  Future  as  a  Data  Modeler  •  Q&A  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   10  
  11. 11. Current  Events:  NoSQL  •  The  “Just  Say  No”  InterpretaIon  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   11    Meta-­‐model    Data  Perspec1ve  Logical  RelaIonal  •  EnIIes  /  Tables  •  A=ributes  /  Columns  •  RelaIonships  /  FKs  •  IdenIfiers  /  PKs        Physical   NO  LONGER  RELATIONAL:  •  Schemas  Based  on  Big  Table  ImplementaIons  •  Alien  DDL  language  •  Limited  Support  from  Modeling  Tools  
  12. 12. Current  Events:  NoSQL  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   12  •  The  “Not  Only  SQL”  InterpretaIon  – Okay,  so  there  might  be  some  work  for  you  – But  you’re  at  risk  of  being  marginalized        
  13. 13. Agenda  •  History  •  Current  Events  •  Your  Future  as  a  Data  Modeler  •  Summary  •  Q&A  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   13  
  14. 14. Your  Future  as  a  Modeler  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   14  •  Remaining  Relevant  – Selfishly:  Saving  your  career  – Nobly:  Serving  your  client  /  company  /  customer  •  What  you  can  do:  – Wait  for  relaIonal  projects  – Become  a  NoSQL  database  designer  – Help  your  client  choose  data  plasorms  •  That  starts  with  understanding  the  problems  –  which  starts  with  CONCEPTUAL  MODELING.    
  15. 15. A  New  (?)  Modeling  Framework  •  Conceptual  Modeling  •  Choosing  a  Logical  Meta-­‐model  •  Logical  Modeling  •  Physical  Modeling      •  Tool  Support?  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   15  
  16. 16. Conceptual  Modeling  •  Behaviors  and  constructs  will  compare  to  RelaIonal  Modeling:  – Keep  some  – Discard  some  – Stress  some  – Change  some  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   16  
  17. 17. Conceptual  Data  Model  Example  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   17  
  18. 18. Keep  Some  •  Keep  EnIIes  •  Keep  A=ributes  •  Keep  RelaIonships  •  Keep  IdenIfiers  •  Keep  Maximum  Cardinality  of  RelaIonships  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   18  
  19. 19. Keep  EnIIes  •  Minimum  Expressiveness  •  EnIIes,  Not  Tables  – Don’t  express  Horizontal  or  VerIcal  ParIIoning  for  performance  •  But  yes  is  moIvated  by  privacy/security/risk  •  EnIty  names,  not  table  names  – Honor  user  vocabulary,  not  IT  naming  standards  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   19  
  20. 20. Keep  A=ributes  •  Honor  User  Phenomenon  – A=ributes  are  part  of  user  discourse  •  A=ributes,  not  columns  – Worry  about  scale  (nominal,  numeric,  ordinal,  Boolean,  cyclic),  not  data  type  – A=ribute  names,  not  column  names  •  Support  in-­‐progress  models  – During  which  a=ributes  can  become  enIIes  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   20  
  21. 21. Keep  RelaIonships  •  Minimum  Expressiveness  – A=ributes  are  part  of  user  discourse  •  Allow  many-­‐many  and  collecIon  enIIes  – If  the  la=er  seem  strange,  you’ve  been  in  IT  too  long  •  RelaIonships,  not  FKs  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   21  
  22. 22. Keep  IdenIfiers  •  IdenIfiers,  not  PKs  – IDs  are  not  moIvated  by  computerizaIon,  but  by  typography  – IDs  predate  the  informaIon  revoluIon  •  and  the  automoIve  revoluIon,  for  that  ma=er  •  Support  in-­‐process  modeling  – IDs  help  the  modeler  ferret  out  the  homonym  problem  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   22  
  23. 23. Discard  Some  •  Discard  Foreign  Keys  – They’re  relaIonal  •  Discard  Minimum  Cardinality  – A  funcIon  of  process  or  policy,  not  data  – Over-­‐reported  by  users  •  Discard  Most  Constraints  – A  funcIon  of  process  or  policy,  not  data  – Are  over-­‐reported  by  users  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   23  
  24. 24. Keep/Discard  Rule  of  Thumb  •  Keep  – Anything  that  helps  you  and  the  users  together  discover  and  name  the  user  categories  •  Discard  – Anything  else  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   24  
  25. 25. Conceptual  Data  Model  Examples  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   25  
  26. 26. Stress  Some  •  Stress  Consistency  Requirements  – RelaIonal  modelers  (of  non-­‐distributed  databases)  have  not  been  asking  about  these.  •  Stress  Data  Volume  /  Velocity  Requirements  – Can  lead  or  force  your  to  relax  applicaIon-­‐data  independence    #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   26  
  27. 27. Change  Some  •  Change  your  process  – From  math-­‐y  normalizaIon  to  English-­‐y  conversaIon  with  users  – Very  difficult  to  achieve  rigor  conversaIonally    #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   27  •  More  help:  – Mastering  Data  Modeling:  A  User-­‐Driven  Approach    by  Carlis  &  Maguire  – DataStax  Webinar:  25  June  
  28. 28. A  New  Modeling  Framework  •  Conceptual  Modeling  •  Choosing  a  Logical  Meta-­‐Model  •  Logical  Modeling  •  Physical  Modeling      •  Tool  Support?  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   28  
  29. 29. Choosing  a  Logical  Meta-­‐Model  •  Don’t  Assume  RelaIonal  (Duh...)  •  Don’t  Assume  Big  Table  •  Lots  of  Choices  – RelaIonal  – Big  Table  – XML/Document  Database  – Graph  database  – Array  database  – ...    #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   29  
  30. 30. A  New  Modeling  Framework  •  Conceptual  Modeling  •  Choosing  a  Logical  Meta-­‐Model  •  Logical  Modeling  •  Physical  Modeling      •  Tool  Support?  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   30  
  31. 31. Logical,  Physical,  and  Tool  Support  •  Community  needs  to  develop  a  roster  of  shapes  – And  the  a=endant  transformaIons  from  conceptual  shapes  to  Big-­‐Table  shapes  •  During  Logical  Big-­‐Table  modeling,  process  requirements  will  infiltrate  – including  things  like  minimum  cardinality  •  Minimal  support  from  modeling  tools  – Because  few  tools  support  conceptual  modeling  – Because  vendors  have  not  caught  up  to  NoSQL  yet    #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   31  
  32. 32. Agenda  •  History  •  Current  Events  •  Your  Future  as  a  Data  Modeler  •  Summary  •  Q&A  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   32  
  33. 33. Summary  •  Re-­‐commit  to  conceptual  modeling  for  requirements  analysis  – Some  but  not  all  relaIonal-­‐modeling  skills  will  apply  – Must  learn  to  focus  on  user  communicaIon,  not  nerdy  stuff  like  intermediate  normal  forms    #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   33  
  34. 34. Summary  •  Remember  the  fundamentals,  so  that  you  can  make  informed  decisions  about  relaxing  them  –  ApplicaIon-­‐data  independence  –  Consistency  level  as  a  user  requirement  –  DeclaraIve  data  retrieval  (from  informaIon  hiding)  •  AddiIonal  benefits  –  Users  will  like  you  be=er  –  Agile  developers  will  like  you  be=er  –  This  framework  works  in  tradiIonal,  all-­‐SQL  environments    #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   34  
  35. 35. Q&A  •  Joe.Maguire@DataQualityStrategies.com  •  www.DataQualityStrategies.com  #Cassandra13   ©  2013  Data  Quality  Strategies,  LLC   35  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×