Your SlideShare is downloading. ×
0
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Data modelling where did it all go wrong?
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Data modelling where did it all go wrong?

1,389

Published on

Data Modeling - where did it all go wrong? …

Data Modeling - where did it all go wrong?
7 key non-technical reasons why Data Modeling had fallen into disrepute in many organizations.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,389
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Data ModellingWhere did it all go wrong?DAMA London, 15th June 2007Christopher Bradley1 I
  • 2. Contents1. Background2. Seven deadly sins3. Our part in fixing this2 I
  • 3. Audience Poll What’s your role within your organization? Data Architect DBA Manager or Executive Sponsor Business Analyst Consultant Marketing Other3 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 4. 1. Background4 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 5. Background:Data Management growth: 1950-1970 1970-1990 1990-2000 1990-2000 Database development Database operation Data requirements analysis Data modelling Enterprise data management coordination Enterprise data integration Enterprise data stewardship Enterprise data use Explicit focus on data quality Security Compliance Other responsibilities5 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 6. Background:Data Modelling’s promise …. "a single consistent definition of data" "master data records of reference" “reduced development time” “improved data quality” “impact analysis” ……. No brainers? So why is it that in many organisations the benefits of data modelling still need to be “sold” and in others the big benefits simply fail to be delivered?6 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 7. 2. Sevendeadly sins7 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 8. i: Not focusing on benefits Project requirements vs Big picture Reward drives behaviour WIIFM Metrics Evidence Sustained improvement8 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 9. What’s the value X Data M odelling to BP? Company of benefits x body of know ledge - m odels repository. Consistency of cross dom ain data concepts. Eases M aster Data Take-on, Legacy M igration, M I/BI, Application interoperability Reuse of com m on m odels & definitions (including standard industry m odels) Interoperability, & efficiency through com m on approaches Reduction in m aintenance. 9 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 10. Company X: User Survey; Benefits What benefits are you gaining from the Data m odelling service? 80% 79% 77% 70% 70% We are obtaining benefit through use of a 60% common modelling tool 50% 55% 60% 40% We are obtaining benefit through utilisation of a 30% common repository 20% 10% We are obtaining 0% benefit through use of 4% Disagree common standards, guidelines & processes Stongly Agree We are obtaining benefit through re-use of models & artefacts We are obtaining benefit We are not through provision of obtaining any 10 central support & help benefits Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 11. Company Y metricsWhat’s the $ value of Data M odelling to BP? A) Complete representation of requirements M easures • Number of definitions the client takes ow nership of. If the client is w illing to assume responsibility for the maintenance of the definitions, then it is safe to assume the definitions are accurate. • Number of modifications to the model after each review . This is more of a rolling "how w ell is the modelling process going" measure than an end-state measure of how complete the model is. A low er number of post-review modifications is an indicator of a higher degree of completeness. B) Retention of collected information (including re-use) M easures • Number of times portions of a model are referenced (on a w eb page for example). If the model has been published (w hich all should be) and the repository information is easily accessible, the "number of hits" on each entity (for example) can be a gauge of the usefulness of the originally collected information. • Number of entities re-used in subsequent projects. This is as much a measure of the quality of the original analysis (and potentially design) as it is a measure of the amount of re-use. Costs savings for this measure can be calculated based on a "days per entity" number. Total time savings (and related cost savings) w ould be equal to the "days per entity" multiplied by the number of entities re- used • Time to market for projects. Assuming w e w ere able to re-use an existing database for a second application, the time savings could simply be "days per entity" multiplied by the number of tables in the existing database. C) Consistent interface M easures • Review time by entity. The time required to review each entity (or definition) should decrease as the review ers become familiar w ith the consistent style of the model. A side benefit to follow ing a consistent style is that subsequent projects w ill be able to accurately reflect the amount of time required to review a data model in project plans based on the results of past review s. • Amount of time spent during subsequent referral to the model. Just as the number of times the model is subsequently referenced is a measure of the retention theme, the amount of time spent w hen referencing a specific portion of the model is a measure of the consistency. If the model has follow ed a consistent interface, subsequent users of the model should be able to find the required information quickly. 11 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 12. Value of Data Modelling - Company Z• Increased reuse & development efficiency >>> Reduced developm ent tim e (* based upon £10k per new Entity & 46% re-use) $300m• Increased consistency >>> Decreased maintenance (* based upon 22% reduction in # bespoke tables & messages) $75m12 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 13. ii: Forgetting the purpose Top down only? Bottom up & middle out It’s not simply for RDBMS development13 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 14. Why Produce a Data Model?Company Z Top Ten Reasons 1. Capturing Business Requirements 2. Promotes Reuse, Consistency, Quality 3. Bridge Betw een Business and Technology Personnel 4. Assessing Fit of Package Solutions 5. Identify and M anage Redundant Data 6. Sets Context for Project w ithin the Enterprise 7. Interaction Analysis: Compliments Process M odel 8. Pictures Communicate Better than Words 9. Avoid Late Discovery of M issed Requirements14 10. Critical in M anaging Integration Betw een Systems Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 15. Not only for “new” Data Base Systems? SOA: Important in an SoA World. Definition of data & consequently calls to / results from services is vital. Straight through processing can exacerbate the issue • what does the data mean? • which definition of X (e.g. “cost of goods”)? • need to utilise the logical model and ERP models definitions Data Lineage: Repository based Data migration design - Consistency Source to target mapping Reverse engineer & generate ETL Impact analysis ERP: Model Data requirements – aid configuration / fit for purpose evaluation Data Integration Legacy Data take on Master Data integration BI / DW: Model Data requirements in Dimensional Model Reverse engineer BW Info Cubes, BO Universes, …….15 Generate Star / Snowflake / Starflake schemas Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 16. iii: Language & intellectual snobbery The term “ M odelling” often has baggage associated w ith it Use appropriate language & terms for different audiences Banish methodology bigots & dogma Barker / ERD /UM L / OR / etc etc Banish methodology bigots & dogma NEVER air methodology issues in front of users 16 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 17. iv: Discipline Dumbing dow n - It’s not just about picture draw ing! Don’t forget the metadata Training & appropriate NASA Mars Climate Orbiter personnel Identify relevant standards & guidelines Communicate Honesty – it’s not easy!17 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 18. v: Inappropriate positioning Don’t do it just for modellings sake!18 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 19. v: Inappropriate positioning Data modelling performed in isolation – silos DM , PM , DBA ... Left until too late in the lifecycle Speed – too much focus on final 20% to be “ theoretically perfect” DM considered an overhead Charging for M odelling infrastructure Hidden / unpublished models – w hat’s the point! Limited re-use Projects left to ow n devices – “ the train has departed” DM function not resourced appropriately thus models not subject to peer / cross-domain review19 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 20. vi: Failing to adaptPlethora of tools – good usage is more important than choosing the “ best”Forgetting the overall information architecture M aster Data, Transaction data, M I/BI, Unstructured, BDD …Disservice by ERP package vendors COTS Logical Data M odel w ith package?Lack of soft skillsHero seeking cow boys20 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 21. vii: Square pegs & round holesTLA factory – DM , M DM , EDM , EII, CDI, SOA …….The right people in the role? Is being a good modeller enough? Certification coming at last Engaging w ith the business Nobody ow es us a livingCommunicating our successes Do people know w hy this is undertaken?Creating communities of interestLack of “ Selling” skills21 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 22. 3. Our part infixing this22 I
  • 23. Industry CultureDBAs, Data Architects and Executives are different creatures DBA Data Architect Business Executive • Cautious • Analytical • Results-Oriented • Analytical • Structured • “Big Picture” focused • Structured • Passionate • Little Time • Doesn’t like to • “Big Picture” focused • “How is this going to help talk • Likes to Talk me?” • “Just let me • “Let me tell you about • “I don’t care about your code!” my data model!” data model.” • “I don’t have time.” 3NF 23 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 24. Role of the Data ArchitectHow to gain Traction, Budget and Executive buy-in • Be Visible about the program: • Identify key decision-makers in your organization and update them on your project and its value to the organization • Focus on the most important data that is crucial to the business first! Publish that and get buy in before moving on. (e.g. start small with a core set of data) •Monitor the progress of your project and show its value: • Define deliverables, goals and key performance indicators (KPIs) • Start small—focus on core data that is highly visible in the organization. Don’t try to “boil the ocean” initially. • Track and Promote progress that is made • Measure Metrics where possible “Hard data” is easy (# data elements, #end users, money saved, etc.) “Softer data” is important as well (data quality, improved decision-making, etc.) Anecdotal examples help with business/executive users “Did you realize we were using the wrong calculation for Total Revenue?” (based on data definitions) 24 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 25. Communicate EffectivelyProvide Information to uses in their “Language” • Repurpose information into various tools: BI, ETL, DDL, etc. • Publish to the Web • Exploit collaboration tools / SharePoint / Wiki ……. • Business users like Excel, Word, Web toolsDocument Metadata • Data in Context (by Organization, Project, etc.) • Data with DefinitionsProvide the Right Amount of Information • Don’t overwhelm with too much information. For business users, terms and definitions, might be enough. • Cater to your audience. Don’t show DDL to a business user or Business definitions to a DBA.Market, Market, Market! • Provide Visibility to your project. • Talk to teams in the organization that are looking for assistance • Provide short-term results with a subset of information, then move on.25 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 26. Model publishing26 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 27. 27 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 28. Case Study: Web-based information sharing 28 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 29. Company X Data Management Maturity Model Obtaining Limited Delivering broad Quality & Re-use Ideal, Obtaining Optimal Value from Data Operating in “Fire Benefits Level 5 - Optimised Fighting” Mode Level 4 - Managed Undesirable Level 3 - Defined Level 2 - Repeatable AspirationData Principles Level 1 - InitialRecognized Data Ownership Model Data Ownership Model Defined Data Data Ownership Model is Data Ownership Model has does not exist. Data does not exist. Owners Ownership Model implemented for the key data been extended such thatOwnership Owners, if any, evolve commissioned in the exists. Ownership entities. Governance the majority of data entities on their own during short-term for specific Model is loosely process regularly reviews are now governed in a project rollouts (i.e. self projects & initiatives. applied to key data this model and its consistent manner. appointed data owners).As-IsOwnership tends to be entities. application, updating and in form of “Data Teams” To-Be improving as needed. or “Super Users” that manage “all” data.Unique Data definitions Key data defined in the Key data definitions Single set of data definitions Data definitions extended unknown and/or short-term for specific exist to those who exist for the key data beyond just “key” dataDefinitions inconsistent across the projects & initiatives. know where to look. entities. Definitions are entities. Common data business(s). Definitions are not Multiple sets of published to a central definitions used throughout leveraged from project definitions exist location that is accessible to the businesses & functions. to project and changeAs-Is because no other programs, projects and To-Be often. rationalization/standar users in secure manner. dization occurs.Accessible Data repository(s) does Disparate set of data Multiple data A single integrated data Central data repository is not exist. repositories exist as a repositories that repository houses the optimized via standardRepositories result of specific synchronize and/or “record of reference” (single data collection & projects & initiatives. communicate via version of the truth). Other distribution mechanisms. Little or no bespoke interfaces. systems access the RoR Data accessible to other As-Is synch/communication from To-Be the central integrated programs, projects and across these tools. repository. users in secure manner.Lifecycle Complete lack of Short term procedures Limited procedures or Defined & consistent set of Defined & consistent set of procedures or controls or controls for key data controls for key data procedures & ctrls for key procedures & ctrls extendManagement for key data operations operations of create, operations of create, data operations of create, beyond just key data. End- of create, read, update read, update & delete. read, update & delete. read, update & delete. Key to-end automated “create & delete. No warehouse Ltd warehouse & Warehouse/archiving As-Is data is proactively monitored to archive/warehouse” To-Be and/or archiving of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+ that arch’ing/warehousing archiving driven only by defined only for key so processes optimize the life- 29 Complete keyboard char set so that all ordinary characters processes in place. space constraints. data entities. occurs at optimal times. cycle mgmt. of all data.
  • 30. Make it sustainable: Current position Avoid the abyss via investment in “ sustain” activities Visibility Typical Gartner “ hype cycle” Technology Peak of inflated Trough of Slope of enlightenment Plateau of productivity Trigger expectations disillusionment M aturity @ your company30 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+
  • 31. Thank youContact details:Email: chris.bradley@ipl.comTel: +44 (0)7973 184475MSN: chrisbradley@bigfoot.comWeb: www.ipl.com 31 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+ I

×