Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DataEd Webinar: Metadata Strategies

443 views

Published on

The first step toward understanding what data assets mean for your organization is understanding what those assets mean for each other. Metadata—literally, data about data—is one of many Data Management disciplines inherent in good systems development and is perhaps the most mislabeled and misunderstood of the lot. Understanding metadata and its associated technologies as more than just straightforward technological tools can provide powerful insight into the efficiency of organizational practices and can also enable you to combine more sophisticated Data Management techniques in support of larger and more complex business initiatives.In this webinar, we will:Illustrate how to leverage Metadata Management in support of your business strategyDiscuss foundational metadata concepts based on the DAMA Guide to Data Management Book of Knowledge (DAMA DMBOK)Enumerate guiding principles for and lessons previously learned from metadata and its practical uses

Published in: Data & Analytics
  • Be the first to comment

DataEd Webinar: Metadata Strategies

  1. 1. Copyright 2019 by Data Blueprint Slide # 1 Peter Aiken, Ph.D. F o u n d a t i o n a l Strategies M e t a d a t a Practices • DAMA International President 2009-2013 / 2018 • DAMA International Achievement Award 2001 (with Dr. E. F. "Ted" Codd • DAMA International Community Award 2005 • I've been doing this a long time • My work is recognized as useful • Associate Professor of IS (vcu.edu) • Founder, Data Blueprint (datablueprint.com) • DAMA International (dama.org) • CDO Society (iscdo.org) • 11 books and dozens of articles • Experienced w/ 500+ data management practices worldwide • Multi-year immersions – US DoD (DISA/Army/Marines/DLA) – Nokia – Deutsche Bank – Wells Fargo – Walmart … Peter Aiken, Ph.D. PETER AIKEN WITH JUANITA BILLINGS FOREWORD BY JOHN BOTTEGA MONETIZING DATA MANAGEMENT Unlocking the Value in Your Organization’s Most Important Asset. 2Copyright 2019 by Data Blueprint Slide #
  2. 2. About Infogix • Innovating data solutions since 1982 • Data Governance; Data Quality and Data Analytics • Large and mid-size customers world-wide: • Organizations rely on Infogix so they can trust their data • Average customer tenure > 18 years “ I n d u s t r i e s t h a t t h r i v e o n d a t a ”
  3. 3. 1. It is not just the technical metadata • Technical Metadata is generally the easy metadata • Labelling data to preserve context, provenance and semantic relatedness is harder – and more valuable 2. Do not be afraid of the detail • Data has metadata; metadata has metadata (meta metadata); and… 3. Do not forget the governance metadata • Your governance metamodel supports scaling, accountability, transparency, etc. My top 5 Observations from our Customers
  4. 4. 3 4. A good data model is not enough model Traditional enterprise data models can capture many relationships, but is the data they store: •Discoverable? •Understandable? •Accessible? •Usable?
  5. 5. 5. Metadata is critical to the future state “vision” If your organization’s data is to support transformative change You will have more metadata than data! Also without metadata no Ai is going to get this joke!
  6. 6. Copyright 2019 by Data Blueprint Slide # Metadata Strategies: Foundational Practices • Defining metadata in the context of data management – Defining data management – What do we mean by using data as metadata and why is this important? – Specific teachable example using iTunes 1. Metadata is a gerund so don't try to treat it as a noun – Metadata is a use of data, not a type of data 2. Metadata is the language of data governance – Make metadata the language of data governance 3. Treat glossaries/repositories as capabilities not technology – Cyclic approaches do not start with technologies • Metadata building blocks – Many many many resources available to jump-start metadata efforts • Benefits, application & sources – Understand that metadata defines the essence of organizational interoperability • Take Aways, References and Q&A • In the history of language, whenever two words are pasted together to form a combined concept initially, a hyphen links them • With the passage of time, the hyphen is lost. The argument can be made that that time has passed • There is a copyright on the term "metadata," but it has not been enforced • So, the term is "metadata" 4Copyright 2019 by Data Blueprint Slide # Meta data, Meta-data, or metadata?Meta data, Meta-dataMeta data
  7. 7. Investing in Metadata? • How can IT staff convince managers to plan, budget, and apply resources for metadata management? • What is metadata and why is it important? • What technologies are involved? • Internet and intranet technologies are part of the answer and will get the immediate attention of management. 5Copyright 2019 by Data Blueprint Slide # Uses What is data management? 6Copyright 2019 by Data Blueprint Slide # Sources Data Engineering Data Delivery Data Storage Data Assets (optimized) Insight Note: does not well-depict data reuse "Understanding the current and future data needs of an enterprise and making that data effective and efficient in supporting business activities." Reuse Specialized Team Skills Data Governance
  8. 8. Tracking network users and access points is metadata • Your organization's networking group allocates the responsibility for knowing: – All the devices permitted to logon to your network – Locations of all permitted access points • This responsibility belongs to a named individual(s) 7Copyright 2019 by Data Blueprint Slide # Another perspective 8Copyright 2019 by Data Blueprint Slide # HR Head |
 –––––––––––– Section Heads |
 ––––––––––––––––––––– (more) Group Heads |
 –––––––––––––––––––––––––––––––––– (many) Individuals and in this corner - Dave!
  9. 9. The prefix meta- 1. Situated behind: metacarpus. 2. a. Later in time: metestrus. b. At a later stage of development: metanephros. 3. a. Change; transformation: metachromatism. b. Alternation: metagenesis. 4. a. Beyond; transcending; more comprehensive: metalinguistics. b. At a higher state of development: metazoan. 5. Having undergone metamorphosis: metasomatic. 6. a. Derivative or related chemical substance: metaprotein. b. Of or relating to one of three possible isomers of a benzene ring with two attached chemical groups, in which the carbon atoms with attached groups are separated by one unsubstituted carbon atom: meta-dibromobenzene. Definition of the prefix meta- (Emphasis added – source: American Heritage English Dictionary © 1993 Houghton Mifflin). Meta 9Copyright 2019 by Data Blueprint Slide # 4. a. Beyond; transcending; more comprehensive: metalinguistics. b. At a higher state of development: metazoan. Hedy Lamarr • Google celebrated her 101st Birthday on 11/8/2015 – Tablets for fizzy drinks – Improved stop light design • Invention of “frequency hopping” radio – By jumping from one radio frequency to another rapidly, only a receiver that shares the key can find the transmission – Prevent interference with the radio guidance controls of torpedoes • U.S. Patent 2,292,387 (w/ George Antheil) • Associated traffic analysis – Looking at other elements of a communication when you don’t know the actual content – Time/duration of a message – Location of transmitters – Detect specific operator “fists” – Identify the operator and you could identify a specific ship or military unit, locate it with direction finding, and then track its activity over time 10Copyright 2019 by Data Blueprint Slide # https://theconversation.com/how-wwi-codebreakers-taught-your-gas-meter-to-snitch-on-you-29924
  10. 10. 11Copyright 2019 by Data Blueprint Slide # Data About Data Metadata yields … 12Copyright 2019 by Data Blueprint Slide # Sources Data Engineering Data Delivery Data Storage Data Assets (optimized) Specialized Team Skills Data Governance Valuable information about your data assets: • Do we have these specific (or this class of) data assets? • What is the quality of … • What will be the cost to improve this class of data assets? • Can these data assets be provided more granularly? • … (increasing insight) Yes! Not suitable! 35¢/apiece Not easily! Reuse
  11. 11. 13Copyright 2019 by Data Blueprint Slide # DataManagement BodyofKnowledge(DMBoK) 14Copyright 2019 by Data Blueprint Slide # DataManagement BodyofKnowledge(DMBoKV2)
  12. 12. Metadata Management from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International 15Copyright 2019 by Data Blueprint Slide # 16Copyright 2019 by Data Blueprint Slide # • Example: – iTunes Metadata • Insert a recently purchased CD • iTunes can: – Count the number of tracks (25) – Determine the length of each track Example: iTunes Metadata
  13. 13. 17Copyright 2019 by Data Blueprint Slide # • When connected to the Internet iTunes connects to the Gracenote(.com) Media Database and retrieves: – CD Name – Artist – Track Names – Genre – Artwork • Sure would be a pain to type in all this information Example: iTunes Metadata • To organize iTunes – I create a "New Smart Playlist" for Artist's containing "Miles Davis" 18Copyright 2019 by Data Blueprint Slide # Example: iTunes Metadata
  14. 14. Example: iTunes Metadata • Notice I didn't get the desired results • I already had another Miles Davis recording • Must fine-tune the request to get the desired results – Album contains "The complete birth of the cool" • Now I can move the playlist "Miles Davis" to a folder 19Copyright 2019 by Data Blueprint Slide # Example: iTunes Metadata • The same: – Interface – Processing – Data Structures • are applied to – Podcasts – Movies – Books – .pdf files • Economies of scale are enormous 20Copyright 2019 by Data Blueprint Slide # Example: iTunes Metadata
  15. 15. Metadata yields … 21Copyright 2019 by Data Blueprint Slide # Sources Data Engineering Data Delivery Data Storage Data Assets (optimized) Reuse Specialized Team Skills Data Governance Valuable information about your iTunes assets: • Do we have these specific Miles Davis recordings? • Most my played Miles Davis recording • What will be the cost to improve this class of data assets? • Can I listen to the entire album before dinner? • … (increasing insight) Yes! Bitches Brew 2¢/each Not easily! Copyright 2019 by Data Blueprint Slide # Metadata Strategies: Foundational Practices • Defining metadata in the context of data management – Defining data management – What do we mean by using data as metadata and why is this important? – Specific teachable example using iTunes 1. Metadata is a gerund so don't try to treat it as a noun – Metadata is a use of data, not a type of data 2. Metadata is the language of data governance – Make metadata the language of data governance 3. Treat glossaries/repositories as capabilities not technology – Cyclic approaches do not start with technologies • Metadata building blocks – Many many many resources available to jump-start metadata efforts • Benefits, application & sources – Understand that metadata defines the essence of organizational interoperability • Take Aways, References and Q&A
  16. 16. As a topic, Data is ... Wally Easton Playing Piano https://www.youtube.com/watch?v=NNbPxSvII-Q 23Copyright 2019 by Data Blueprint Slide # Complex & detailed • Outsiders do not want to hear about or discuss any aspects of challenges/solutions • Most are unqualified re: architecture/ engineering Taught inconsistently • Focus is on technology • Business impact is not addressed Not well understood • Lack of standards/ poor literacy/ unknown dependencies • (Re)learned by every workgroup 24Copyright 2019 by Data Blueprint Slide # • If others do not understand what you do then you are perceived with a cost bias • If others understand what you do then you can be perceived with a value bias Comprehension by others is critical!
  17. 17. Defining Metadata 25Copyright 2019 by Data Blueprint Slide # Adapted from Brad Melton Where How Who When Data Metadata is any combination of any circle and the data in the center that unlocks the value of the data! What Why Outlook Example 26Copyright 2019 by Data Blueprint Slide # "Outlook" metadata is used to navigate/manage email What: "Subject" How: "Priority" Where: "USERID/Inbox", 
 "USERID/Personal" Why: "Body" When: "Sent" & "Received” • Find the important stuff/weed out junk • Organize for future access/outlook rules • Imagine how managing e-mail (already non-trivial) would change if Outlook did not make use of metadata Who:"To" & "From?"
  18. 18. • Examples of information architecture achievements that happened well before the information age: – Page numbering – Alphabetical order – Table of contents – Indexes – Lexicons – Maps – Diagrams Pre-Information Age Metadata 27Copyright 2019 by Data Blueprint Slide # Example from: How to make sense of any mess by Abby Covert (2014) ISBN: 1500615994 "While we can arrange things with the intent to communicate certain information, we can't actually make information. Our users do that for us." 28Copyright 2019 by Data Blueprint Slide # Remove the structure and things fall apart rapidly
  19. 19. Definitions • Metadata is – Everywhere in every data management activity and integral to all IT systems and applications. – To data what data is to real life. Data reflects real life transactions, events, objects, relationships, etc. Metadata reflects data transactions, events, objects, relations, etc. – The data that describe the structure and workings of an organization’s use of information, and which describe the systems it uses to manage that information. [quote from David Hay's book, page 4] • Data describing various facets of a data asset, for the purpose of improving its usability throughout its life cycle [Gartner 2010] • Metadata unlocks the value of data, and therefore requires management attention [Gartner 2011] • Metadata Management is – The set of processes that ensure proper creation, storage, integration, and control to support associated use of metadata 29Copyright 2019 by Data Blueprint Slide # Analogy: a library card catalog 30Copyright 2019 by Data Blueprint Slide # • Identifies – What books are in the library, and – Where they are located • Search by – Subject area – Author, or – Title • Catalog shows – Author – Subject tags – Publication date and – Revision history • Determine which books will meet the reader’s requirements • Without the catalog, finding things is difficult, time consuming and frustrating from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
  20. 20. 0 500 1000 1500 2000 2500 Manage Positions(2%) Plan Careers (~5%) AdministerTraining (~5%) Plan Successions(~5% Manage Competencies(20%) Recruit Workforce (62%) DevelopWorkforce(29.9%) AdministerWorkforce(28.8%) CompensateEmployees(23.7%) MonitorWorkplace(8.1%) DefineBusiness(4.4%) TargetSystem (3.9%) EDI Manager (.9%) TargetSystem Tools(.3%) Administer Workforce Metadata Uses 31Copyright 2019 by Data Blueprint Slide # 32Copyright 2019 by Data Blueprint Slide #
  21. 21. Why data structure problems have been difficult? 33Copyright 2019 by Data Blueprint Slide # 34Copyright 2019 by Data Blueprint Slide # Student Data Base Master
  22. 22. 35Copyright 2019 by Data Blueprint Slide # Proposed Data Model Metadata … • Isn't – Is not a noun – One persons data is another's metadata • Is more of a verb? – Represents a use of existing facts, rather than a type of data itself • It is a gerund – a form that is derived from a verb but that functions as a noun – e.g., asking in do you mind my asking you? – Describes a use of data - not a type of data • Describes the use of some attributes of data to understand or manage that same data from a different (usually higher) level of abstraction • Value proposition – Is this data worth including within the scope of our metadata practices? 36Copyright 2019 by Data Blueprint Slide #
  23. 23. Keep the proper focus • Wrong question: – Is this metadata? • Right question: – Should we include this data item within the scope of our metadata practices? 37Copyright 2019 by Data Blueprint Slide # Copyright 2019 by Data Blueprint Slide # Metadata Strategies: Foundational Practices • Defining metadata in the context of data management – Defining data management – What do we mean by using data as metadata and why is this important? – Specific teachable example using iTunes 1. Metadata is a gerund so don't try to treat it as a noun – Metadata is a use of data, not a type of data 2. Metadata is the language of data governance – Make metadata the language of data governance 3. Treat glossaries/repositories as capabilities not technology – Cyclic approaches do not start with technologies • Metadata building blocks – Many many many resources available to jump-start metadata efforts • Benefits, application & sources – Understand that metadata defines the essence of organizational interoperability • Take Aways, References and Q&A
  24. 24. Data Data Data Information Fact Meaning Request [Built on definitions from Dan Appleton 1983] Intelligence Strategic Use Data Data Data Data 39Copyright 2019 by Data Blueprint Slide # “You can have data without information, but you cannot have information without data” — Daniel Keys Moran, Science Fiction Writer 1. Each FACT combines with one or more MEANINGS. 2. Each specific FACT and MEANING combination is referred to as a DATUM. 3. An INFORMATION is one or more DATA that are returned in response to a specific REQUEST 4. INFORMATION REUSE is enabled when one FACT is combined with more than one MEANING. 5. INTELLIGENCE is INFORMATION associated with its STRATEGIC USES. 6. DATA/INFORMATION must formally arranged into an ARCHITECTURE. Wisdom & knowledge are often used synonymously Useful Data A Model Defining 3 Important Concepts Data Strategy and Data Governance in Context 40Copyright 2019 by Data Blueprint Slide # Organizational Strategy Data Strategy IT Projects Organizational Operations Data Governance Data asset support for organizational strategy What the data assets do to support strategy How well the data strategy is working Operational feedback How data is delivered by IT How IT supports strategy Other aspects of organizational strategy
  25. 25. Data Strategy and Governance in Strategic Context 41Copyright 2019 by Data Blueprint Slide # Organizational Strategy Data Strategy Data Governance Data asset support for organizational strategy What the data assets do to support strategy How well the data strategy is working (Business Goals) (Metadata) IT Projects How data is delivered by IT Data Governance & Data Stewards 42Copyright 2019 by Data Blueprint Slide # Data Strategy Data Governance What the data assets do to support strategy How well the data strategy is working (Business Goals) (Metadata) Data Stewards What is the most effective use of steward investments? (Metadata) Progress, plans, problems
  26. 26. Domain expertise is less ← | → Domain expertise is greater Roles more formally defined ← |→ Roles less formally defined Encountergoverneddatamoredirectly←|→Encountergoverneddatalessdirectly Moretimeisdedicated←|→Lesstimeisdedicated IT/Systems Development Leadership (data decision makers) Stewards (data trustees) Guidance Decisions Participants/Experts (data subject matter experts) Other Sources/Uses (data makers & consumers) IT/SystemsDevelopment Data/feedback Changes Action R esources Ideas Data/Feedback Components comprising the data community 43Copyright 2019 by Data Blueprint Slide # DIP Implementation 44Copyright 2019 by Data Blueprint Slide # DataLeadership Feedback Feedback Data Governance Data Improvement DataStewards DataCommunityParticipants DataGenerators/DataUsers Data Things Happen Organizational Things Happen DIPs Data Improves Over Time Data Improves As A Result of Focus
  27. 27. 10 DG Worst Practices 1. Buy-in but not Committing: Business vs. IT 2. Ready, Fire, Aim 3. Trying to Solve World Hunger or Boil the Ocean 4. The Goldilocks Syndrome 5. Committee Overload 6. Failure to Implement 7. Not Dealing with Change Management 8. Assuming that Technology Alone is the Answer 9. Not Building Sustainable and Ongoing Processes 10. Ignoring “Data Shadow Systems” 45Copyright 2019 by Data Blueprint Slide # Copyright 2019 by Data Blueprint Slide # Metadata Strategies: Foundational Practices • Defining metadata in the context of data management – Defining data management – What do we mean by using data as metadata and why is this important? – Specific teachable example using iTunes 1. Metadata is a gerund so don't try to treat it as a noun – Metadata is a use of data, not a type of data 2. Metadata is the language of data governance – Make metadata the language of data governance 3. Treat glossaries/repositories as capabilities not technology – Cyclic approaches do not start with technologies • Metadata building blocks – Many many many resources available to jump-start metadata efforts • Benefits, application & sources – Understand that metadata defines the essence of organizational interoperability • Take Aways, References and Q&A
  28. 28. Metadata yields … 47Copyright 2019 by Data Blueprint Slide # Sources Data Engineering Data Delivery Data Storage Data Assets (optimized) Reuse Specialized Team Skills Data Governance Valuable information about your data assets: • Do we have these specific (or this class of) data assets? • What is the quality of … • What will be the cost to improve this class of data assets? • Can these data assets be provided more granularly? • … (increasing insight) Yes! Not suitable! 35¢/apiece Not easily! 48Copyright 2019 by Data Blueprint Slide # Definition of Bed
  29. 29. Purpose statement incorporates motivations A purpose statement describing – Why the organization is maintaining information about this business concept; – Sources of information about it; – A partial list of the attributes or characteristics of the entity; and – Associations with other data items; this one is read as "One room contains zero or many beds." Entity: BED Data Asset Type: Principal Data Entity Purpose: This is a substructure within the Room substructure of the Facility Location. It contains information about beds within rooms. Source: Maintenance Manual for File and Table Data (Software Version 3.0, Release 3.1) Attributes: Bed.Description Bed.Status Bed.Sex.To.Be.Assigned Bed.Reserve.Reason Associations: >0-+ Room Status: Validated 49Copyright 2019 by Data Blueprint Slide # Draft 50Copyright 2019 by Data Blueprint Slide # NTB
  30. 30. Uneven Conversations 51Copyright 2019 by Data Blueprint Slide # 0 25 50 75 100 Customers Vendors Very Knowledgeable Not Knowledgeable Tech-knowledgeGap Student Activities File • Question: – Suppose we want to decrease the price of Swimming from $50 to $40? • Answer: – Change to data items such as ‘swimming’ requires examination of every single record – Will not catch spelling errors – This is usually undesirable and unintended 52Copyright 2019 by Data Blueprint Slide # Row Student ID Activity Fee 2 150 Swiming $50 3 175 Squash $75 4 200 Swimming $40 5 327 Scuba $175 $40
  31. 31. Row Activity Activity Code Fee 1 Skiing TRF $200 2 Squash XGY $75 3 Swimming WRE $40 4 Scuba N2R $150 How Should it be Done? (Joining Tables) 53Copyright 2019 by Data Blueprint Slide # Row Student ID Activity Code 2 150 WRE 3 175 XGY 4 200 WRE 5 327 N2R ... (1 of these provides context for many of these) • Data from the two tables is joined to provide requested information • Student 150 is properly registered for swimming • The inconsistent fee issue with swimming has been resolved • Fee on orange table is a better engineered solution Swimming WRE $40 FTI Metadata Model 54Copyright 2019 by Data Blueprint Slide #
  32. 32. Build Your Own Metadata Repository 55Copyright 2019 by Data Blueprint Slide # Sample Low-tech Repository 56Copyright 2019 by Data Blueprint Slide #
  33. 33. For each column … 57Copyright 2019 by Data Blueprint Slide # Where does this key function as a primary key? 58Copyright 2019 by Data Blueprint Slide #
  34. 34. Where does this key function as a foreign key? 59Copyright 2019 by Data Blueprint Slide # Where does this key function as an attribute? 60Copyright 2019 by Data Blueprint Slide #
  35. 35. Copyright 2019 by Data Blueprint Slide # Metadata Strategies: Foundational Practices • Defining metadata in the context of data management – Defining data management – What do we mean by using data as metadata and why is this important? – Specific teachable example using iTunes 1. Metadata is a gerund so don't try to treat it as a noun – Metadata is a use of data, not a type of data 2. Metadata is the language of data governance – Make metadata the language of data governance 3. Treat glossaries/repositories as capabilities not technology – Cyclic approaches do not start with technologies • Metadata building blocks – Many many many resources available to jump-start metadata efforts • Benefits, application & sources – Understand that metadata defines the essence of organizational interoperability • Take Aways, References and Q&A Architecture • Things – (components) data structures • The functions of the things – (individually) sources and uses of data • How the things interact – (as a system, towards a goal) Efficiencies/effectiveness 62Copyright 2019 by Data Blueprint Slide #
  36. 36. How are components expressed as architectures? 63Copyright 2019 by Data Blueprint Slide # A B C D A B C D A D C B Intricate Dependencies Purposefulness • Details are organized into larger components • Larger components are organized into models • Models are organized into architectures (comprised of architectural components) How are data structures expressed as architectures? • Attributes are organized into entities/objects – Attributes are characteristics of "things" – Entitles/objects are "things" whose information is managed in support of strategy – Examples • Entities/objects are organized into models – Combinations of attributes and entities are structured to represent information requirements – Poorly structured data, constrains organizational information delivery capabilities – Examples • Models are organized into architectures – When building new systems, architectures are used to plan development – More often, data managers do not know what existing architectures are and - therefore - cannot make use of them in support of strategy implementation – Why no examples? 64Copyright 2019 by Data Blueprint Slide # Intricate Dependencies Purposefulness
  37. 37. Design Patterns • Why are the restrooms in the same place in each building? • What about the electrical wiring? • HVAC? Floorplans? ... • Architecture design patterns (spoke and hub, hub of hubs, warehouse, cloud, MDM, changing tires, portal) 65Copyright 2019 by Data Blueprint Slide # http://dmreview.com/article_sub.cfm?articleID=1000941 used with permission Meta Data Models 66Copyright 2019 by Data Blueprint Slide #
  38. 38. Metadata Data Model SCREEN ELEMENT screen element id # data item id # screen element descr. INTERFACE ELEMENT interface element id # data item id # interface element descr. INPUT ELEMENT input element id # data item id # input element descr. OUTPUT ELEMENT output element id # data item id # output element descr. MODEL VIEW model view element id # data item id # model view element des. DEPENDENCY dependency elem id # data item id # process id # dependency description CODE code id # data item id # stored data item # code location INFORMATION information id # data item id # information descr. information request PROCESS process id # data item id # process description USER TYPE user type id # data item id # information id # user type description LOCATION location id # information id # printout element id # process id # stored data items id # user type id # location description PRINTOUT ELEMENT printout element id # data item id # printout element descr. STORED DATA ITEM stored data item id # data item id # location id # stored data description DATA ITEM data item id # data item description 67Copyright 2019 by Data Blueprint Slide # 68Copyright 2019 by Data Blueprint Slide # Business Goals Model Defines the mission of the enterprise, its long-range goals, and the business policies and assumptions that affect its operations. Business Rules Model Records rules that govern the operation of the business and the Business Events that trigger execution of Business Processes. Enterprise Structure Model Defines the scope of the enterprise to be modeled. Assigns a name to the model that serves to qualify each component of the model. Extension Support Model Provides for tactical Information Model extensions to support special tool needs. Info Usage Model Specifies which of the Entity-Relationship Model component instances are used by other Information Model components. Global Text Model Supports recording of extended descriptive text for many of the Information Model components. DB2 Model Refines the definition of a Relational Database design to a DB2-specific design. IMS Structures Model Defines the component structures and elements and the application program views of an IMS Database. Flow Model Specifies which of the Entity Relationship Model component instances are passed between Process Model components. Applications Structure Model Defines the overall scope of an automated Business Application, the components of the application and how they fit together. Data Structures Model Defines the data structures and their elements used in an automated Business Application. Application Build Model Defines the tools, parameters and environment required to build an automated Business Application. Derivations/Constraints Model Records the rules for deriving legal values for instances of Entity-Relationship Model components, and for controlling the use or existence of E-R instance. Entity-Relationship Model Defines the Business Entities, their properties (attributes) and the relationships they have with other Business Entities. Organization/Location Model Records the organization structure and location definitions for use in describing the enterprise. Process Model Defines Business Processes, their sub processes and components. Relational Database Model Describes the components of a Relational Database design in terms common to all SAA relational DBMSs. Test Model Identifies the various file (test procedures, test cases, etc.) affiliated with an automated business Application for use in testing that application. Library Model Records the existence of non-repository files and the role they play in defining and building an automated Business Application. Panel/Screen Model Identifies the Panels and Screens and the fields they contain as elements used in an automated Business Application. Program Elements Model Identifies the various pieces and elements of application program source that serve as input to the application build process. Value Domain Model Defines the data characteristics and allowed values for information items. Strategy Model Records business strategies to resolve problems, address goals, and take advantage of business opportunities. It also records the actions and steps to be taken.Resource/Problem Model Identifies the problems and needs of the enterprise, the projects designed to address those needs, and the resources required. Process Model Extension Support Model Application Structure Model DB2 Model Relational Database Model Global Text Model Strategy Model Derivations/ Constriants Model Application Build Model Test Model Panel/ Screen Model IMS Structure Model Data Structure Model Program Elements Model Business Model Goals Organization/ LocationModel Resource/ Problem Model Enterprise Structure Model Entity- Relationship Model Info Usage Model Value Domain Model Flow Model Business Rules Model Library Model IBM's AD/Cycle Information Model
  39. 39. Metadata for Semistructured Data • Metadata describes both structured and semi-structured data – You cannot convert unstructured data into structured data – Better description: Non-tabular data ➜ tabular data • Semi-structured data – Any data that is not in a database or data file, including documents or other media data • Metadata for semi-structured data exists in many formats, responding to a variety of different requirements • Examples of Metadata repositories describing unstructured data: – Content management applications – University websites – Company intranet sites – Data archives – Electronic journals collections – Community resource lists • Common method for classifying Metadata in unstructured sources is to describe them as descriptive metadata, structural metadata, or administrative metadata • Examples of descriptive metadata: – Catalog information – Thesauri keyword terms • Examples of structural metadata – Dublin Core – Field structures – Format (audio/visual, booklet) – Thesauri keyword labels – XML schemas • Examples of administrative metadata – Source(s) – Integration/update schedule – Access rights – Page relationships (e.g. site navigational design) 69Copyright 2019 by Data Blueprint Slide # from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International Copyright 2019 by Data Blueprint Slide # Metadata Strategies: Foundational Practices • Defining metadata in the context of data management – Defining data management – What do we mean by using data as metadata and why is this important? – Specific teachable example using iTunes 1. Metadata is a gerund so don't try to treat it as a noun – Metadata is a use of data, not a type of data 2. Metadata is the language of data governance – Make metadata the language of data governance 3. Treat glossaries/repositories as capabilities not technology – Cyclic approaches do not start with technologies • Metadata building blocks – Many many many resources available to jump-start metadata efforts • Benefits, application & sources – Understand that metadata defines the essence of organizational interoperability • Take Aways, References and Q&A
  40. 40. • They know you rang a phone sex service at 2:24 am and spoke for 18 minutes. But they don't know what you talked about. • They know you called the suicide prevention hotline from the Golden Gate Bridge. But the topic of the call remains a secret. • They know you spoke with an HIV testing service, then your doctor, then your health insurance company in the same hour. But they don't know what was discussed. • They know you received a call from the local NRA office while it was having a campaign against gun legislation, and then called your senators and congressional representatives immediately after. But the content of those calls remains safe from government intrusion. • They know you called a gynecologist, spoke for a half hour, and then called the local Planned Parenthood's number later that day. But nobody knows what you spoke about. – https://www.eff.org/deeplinks/2013/06/why-metadata-matters 71Copyright 2019 by Data Blueprint Slide # Why Metadata Matters 72Copyright 2019 by Data Blueprint Slide # Shoshana Zuboff, The Age of Surveillance Capitalism Who knows Who decides and Who decides, who decides? Widespread Ignorance
  41. 41. EnveraBusinessValueProposition 73Copyright 2019 by Data Blueprint Slide # 74Copyright 2019 by Data Blueprint Slide # The Real Value of Metadata
  42. 42. • Foundations for Evidence-Based Policymaking Act (H.R._4174,_S._2046) • Purpose: To amend titles 5 and 44, United States Code, 1. to require Federal evaluation activities 2. improve Federal data management, and 3. for other purposes. • Opposition? – FEPA Gives Bureaucrats, Private Parties, Hackers A Data Gold Mine https://truthinamericaneducation.com/privacy-issues-state-longitudinal-data-systems/fepa-gives- bureaucrats-private-parties-hackers-data-gold-mine/ – "A lifelong data citizen surveillance system from coming into fruition" https://youtu.be/Qr14ZmrQGu0 • Passed House 356–17 21-12-2018 • Passed Senate unanimously 21-12-2018 • Signed by the President 14-1-2019 FEPA/H.R. 4174 https://www.congress.gov/bill/115th-congress/house-bill/4174/text 75Copyright 2019 by Data Blueprint Slide # Title I - Federal Evidence-Building Activities • All federal agencies are required to: – Manage its data according to industry best practices – Regularly analyze the data – Use the results to inform policymaking • Future federal decision making process – Specify in advance the open data sets that the policy evaluation will consider – Publish in advance the model that is proposed to be used in the evaluation – Policy may only be changed in ways supportable by the evidence 76Copyright 2019 by Data Blueprint Slide #
  43. 43. Title II - Open Government Data Act • Open – Guidance To Make Data Open By Default – Federal Agency Responsibilities To Make Data Open By Default – Requires all non-sensitive government data to be made available in open and machine-readable formats by default • Data inventory and Federal data catalogue – Comprehensive Data Inventory – Public Data Assets – Federal Data Catalogue 77Copyright 2019 by Data Blueprint Slide # Title II - Open Government Data Act (continued) • Establishes Chief Data Officers (CDO) at federal agencies – Distinct from the CIO role – Nonpolitical – Objective qualifications • Responsibilities – Set standards for data formats, Negotiate terms for data sharing, and develop processes for data publishing; – Coordinate with any agency officials responsible for using, protecting, disseminating, and generating data; – Review the impact of agency IT infrastructure on data accessibility and coordinate with agency Chief Information Officer to reduce barriers that inhibit data accessibility; – Maximize the use of data in the agency to support evidence-based analysis, cybersecurity, and operational improvement; – Acquire and maintain training and certification related to confidential information protection and statistical efficiency; and – Be responsible for overall data lifecycle management. 78Copyright 2019 by Data Blueprint Slide #
  44. 44. Title III - Confidential Information Protection and Statistical Efficiency Act • Aka – Confidential Information Protection and Statistical Efficiency Act of 2018 – Improve public perception of governmental information • Definitions – Agent (towards objective qualifications) – Business data (wide scope) • Coordination – Breaches – Reporting – Processing – Confidence • Statistical Efficiency – Improving interoperability's within Government – Specifically for DSAs: Census, BEA & BLS – Generally: BJS, BTS, ERS, EIA, NASS, NCES, NCHS 79Copyright 2019 by Data Blueprint Slide # 7 Metadata Benefits 1. Increase the value of strategic information (e.g. data warehousing, CRM, SCM, etc.) by providing context for the data, thus aiding analysts in making more effective decisions. 2. Reduce training costs and lower the impact of staff turnover through thorough documentation of data context, history, and origin. 3. Reduce data-oriented research time by assisting business analysts in finding the information they need in a timely manner. 4. Improve communication by bridging the gap between business users and IT professionals, leveraging work done by other teams and increasing confidence in IT system data. 5. Increased speed of system development’s time-to-market by reducing system development life-cycle time. 6. Reduce risk of project failure through better impact analysis at various levels during change management. 7. Identify and reduce redundant data and processes, thereby reducing rework and use of redundant, out-of-data, or incorrect data. 80Copyright 2019 by Data Blueprint Slide # from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
  45. 45. Metadata Strategies: Foundational Practices • Defining metadata in the context of data management – Defining data management – What do we mean by using data as metadata and why is this important? – Specific teachable example using iTunes 1. Metadata is a gerund so don't try to treat it as a noun – Metadata is a use of data, not a type of data 2. Metadata is the language of data governance – Make metadata the language of data governance 3. Treat glossaries/repositories as capabilities not technology – Cyclic approaches do not start with technologies • Metadata building blocks – Many many many resources available to jump-start metadata efforts • Benefits, application & sources – Understand that metadata defines the essence of organizational interoperability • Take Aways, References and Q&A 81Copyright 2019 by Data Blueprint Slide # + = Questions? 82Copyright 2019 by Data Blueprint Slide # It’s your turn! Use the chat feature or Twitter (#dataed) to submit your questions now!
  46. 46. 83Copyright 2019 by Data Blueprint Slide # UsesSources Metadata Governance Metadata Engineering Metadata Delivery Metadata Practices Metadata Storage Specialized Team Skills • 'Data about data' • Metadata unlocks the value of data, and therefore requires management attention [Gartner] • Metadata is less about what and more about how • Metadata must be the language of data governance • Metadata defines the essence of most business challenges • Should we include this data item within the scope of our metadata practices? Metadata Take Aways References & Recommended Reading 84Copyright 2019 by Data Blueprint Slide # from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
  47. 47. References, cont’d 85Copyright 2019 by Data Blueprint Slide # from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International References, cont’d 86Copyright 2019 by Data Blueprint Slide # from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
  48. 48. References, cont’d 87Copyright 2019 by Data Blueprint Slide # from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International Upcoming Events October Webinar: Data Quality Success Stories 8 October 2019 @ 2:PM ET UTC (-4) Data Architecture Summit Data Architecture Bootcamp 14 Oct 2019 @ 8:30 AM CT UTC-5.5 November Webinar: Metadata Strategies 12 Nov 2019 @ 2:PM ET UTC-5 Data Governance Vision Rekindling Data Governance 9 Dec 2019 @ 8:30 PM CT UTC-5.5 December Webinar: Exorcising the Seven Deadly Data Sins 10 Dec 2019 @ 2:00 PM ET UTC-5 Sign up for webinars at: www.datablueprint.com/ webinar-schedule or at www.dataversity.net 88Copyright 2019 by Data Blueprint Slide # Brought to you by:
  49. 49. 10124 W. Broad Street, Suite C Glen Allen, Virginia 23060 804.521.4056 Copyright 2019 by Data Blueprint Slide # 89

×