How to Build and Promote a Successful MDM Solution on a Shoestring

1,362 views

Published on

Implementing a Master Data Management (MDM) sometimes seems like a daunting, expensive proposition. Many MDM efforts end being discredited and discarded in the long run.

A team of two engineers designed, developed, and implemented a MDM in our organization with a small budget. After three years, this MDM is successfully sharing enterprise data to over 40 consumers, and growing in popularity, with minimum maintenance.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,362
On SlideShare
0
From Embeds
0
Number of Embeds
95
Actions
Shares
0
Downloads
24
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Also… Over 27,800 Local congregations Hundreds of global and local information systems Broadcasting network using all types of infrastructure
  • - Governing councils include the First Presidency, Quorum of the Twelve Apostles, and the Presiding Bishopric. Elder Hales: “The wheels of the Church move slowly, but grind really fine.” Information systems have been traditionally built to answer the needs of a specific department or division.
  • - For more information: www.mormon.org or www.lds.org
  • How to Build and Promote a Successful MDM Solution on a Shoestring

    1. 1. How to build and promote a successful MDM solution on a shoestring<br />Pablo Riboldi, PhD<br />Solution Manager for Information Governance & Quality<br />The Church or Jesus Christ of Latter-day Saints<br />
    2. 2. Master Data Management (MDM) is a set of processes, technologies, strategies to securely provide accurate and consistent enterprise data.<br />2<br />© IRI 2011<br />Master Data Management - Definition<br />
    3. 3. Large<br />13 million members in 160 countries<br />53,000 full-time missionaries<br />Materials translated in 83 languages<br />264 centrally managed databases<br />Largest Genealogy system in the world<br />www.lds.org or www.mormon.org<br />© IRI 2010<br />3<br />My Organization<br />
    4. 4. Large<br />Organization (Hierarchical)<br />Governing Council (18 members)<br />Departments (23)<br />Silo Organization<br />Independent information systems<br />www.lds.org or www.mormon.org<br />© IRI 2010<br />4<br />My Organization<br />
    5. 5. Large<br />Organization (Hierarchical)<br />Non Profit<br />Invite people to come to Jesus Christ by<br />Proclaiming the gospel<br />Perfecting the saints<br />Redeeming the dead<br />Clearly defined common purpose and<br />Great people to work with<br />www.lds.org or www.mormon.org<br />© IRI 2010<br />5<br />My Organization<br />
    6. 6. Data Management<br /> Common Method MDM Sharing<br />Consumer Systems<br />Master Data Management Repository<br />MDM<br />Pass-through accounts <br />2800+<br />Views<br />DB Links<br />Systems of Record (Enterprise info)<br />6<br />© IRI 2011<br />
    7. 7. Provide accurate data to Church applications <br />Improve consistency of data reporting across the Church <br />Ensure the security of data <br />Enforce policies for use of data <br />Reduce development time  <br />Eliminate the need to maintain master data in each application <br />Decrease maintenance costs<br />7<br />© IRI 2011<br />MDM Goals<br />
    8. 8. Designate authoritative sources of master data and select stewards <br />Provide interfaces for cleansing and enrichment of data <br />Share master data from the source with other applications <br />Create a common data model and interfaces for using master data in applications <br />Centralize administration of security and data use policies (at the application level) <br />8<br />© IRI 2011<br />MDM Strategy<br />
    9. 9. How to store data in the MDM Repository<br />Relational – Dimensional – Canonical Models<br />Dimensional data models<br />Relational data models<br />ETL<br /><ul><li>Used for data warehouses
    10. 10. Denormalized: Star schema with dimensions (entities), facts (measures)
    11. 11. Optimized for Reporting (ad-hoc queries)
    12. 12. Used for transactional systems
    13. 13. Normalized (at least 3NF, usually at 5NF)
    14. 14. Optimized for CRUD transactions (insert, update, delete)</li></li></ul><li>How to store data in the MDM Repository<br />Relational – Dimensional – Canonical Models<br />Relational data models<br />Canonical data models<br />ETL<br /><ul><li>Used for transactional systems
    15. 15. Normalized (at least 3NF, usually at 5NF)
    16. 16. Optimized for CRUD transactions (insert, update, delete)
    17. 17. Used for reference by other systems
    18. 18. Semi-denormalized (1NF or 2NF)
    19. 19. Optimized for fast access by systems (usually used in dropdowns)</li></li></ul><li>Need to deliver only the information (rows and<br />columns) authorized by the Data Steward…<br />MDM Repository<br />Target Application<br /><ul><li>The target receives only “Active” organizations and it does not receive PARENT_ORG.
    20. 20. Every target receives the same data model.
    21. 21. The target receives the information via Replication, Database Link, or Web Service.</li></li></ul><li>Why use Oracle VPD capabilities?<br />We need to constrain the results at both the row and column level to meet data sharing agreement constraints.<br />The MDM Repository is hosted on an Oracle database.<br />Oracle Virtual Private Database (VPD) permits the creation of policies to control database access at the row and column level.<br />These policies essentially add a dynamic WHERE clause to the SQL statement issued against the table, view, or synonym.<br />
    22. 22. Establishing the repository:<br />Selection of a data source (MDM_User or another source) <br />Publishing the repository:<br />Create MDMR schema with a common data model in each consuming application database <br />Build applications that use foreign keys to the master data instead of copying the data<br />Use Data Integrator to replicate data to MDMR schemas <br />Re-use same code for every application<br />Use VPD to enforce data use policies specific to each application <br />Create a passthrough account on the source<br />Administer MDMR data privileges for each application centrally (Integration team)<br />13<br />© IRI 2011<br />MDM Implementation includes:<br />
    23. 23. Master Data Management – Architecture<br />1) Master data is integrated, cleansed, enriched, and transformed to the MDM data structuresfrom the systems of record.<br />MDM Source<br />Systems of Record (Master data)<br />Organizations<br />Leaders<br />MDM_USER<br />schema<br />Finance<br />Other systems<br />14<br />© IRI 2011<br />
    24. 24. Master Data Management – Architecture<br />1) Master data is integrated, cleansed, enriched, and transformed to the MDM data structuresfrom the systems of record.<br />2) A set of security packages is defined to restrict columns and rows.<br />MDM Source<br />After Login Trigger<br />Systems of Record (Master data)<br />MDM_SEC<br />schema<br />Organizations<br />Leaders<br />MDM_USER<br />schema<br />Finance<br />Other systems<br />15<br />© IRI 2011<br />
    25. 25. 3) The After-login trigger applies security policies to the MDM accounts.<br />Master Data Management – Architecture<br />1) Master data is integrated, cleansed, enriched, and transformed to the MDM data structuresfrom the systems of record.<br />2) A set of security packages is defined to restrict columns and rows.<br />MDM Source<br />Business Objects<br />Data Integrator<br />After Login Trigger<br />APP_PTM<br />Systems of Record (Master data)<br />MDM_SEC<br />schema<br />Organizations<br />Leaders<br />MDM_USER<br />schema<br />Finance<br />Other systems<br />16<br />© IRI 2011<br />
    26. 26. 3) The After-login trigger applies security policies to the MDM accounts.<br />4) Security policies limit the rows and columns available to each MDM account.<br />Master Data Management – Architecture<br />1) Master data is integrated, cleansed, enriched, and transformed to the MDM data structuresfrom the systems of record.<br />2) A set of security packages is defined to restrict columns and rows.<br />MDM Source<br />Business Objects<br />Data Integrator<br />After Login Trigger<br />APP_PTM<br />Systems of Record (Master data)<br />MDM_SEC<br />schema<br />Organizations<br />Leaders<br />MDM_USER<br />schema<br />Finance<br />Other systems<br />17<br />© IRI 2011<br />
    27. 27. 3) The After-login trigger applies security policies to the MDM accounts.<br />4) Security policies limit the rows and columns available to each MDM account.<br />5) Data is published to the MDMR at set frequencies.<br />Master Data Management – Architecture<br />1) Master data is integrated, cleansed, enriched, and transformed to the MDM data structuresfrom the systems of record.<br />2) A set of security packages is defined to restrict columns and rows.<br />Requesting Application Instance<br />MDM Source<br />Business Objects<br />Data Integrator<br />After Login Trigger<br />APP_PTM<br />Systems of Record (Master data)<br />MDM_SEC<br />schema<br />Organizations<br />Leaders<br />MDMR<br />schema<br />MDM_USER<br />schema<br />Finance<br />Other systems<br />18<br />© IRI 2011<br />
    28. 28. 3) The After-login trigger applies security policies to the MDM accounts.<br />6) Requesting App can use IDs as FKs, but should not copy data.<br />.<br />4) Security policies limit the rows and columns available to each MDM account.<br />5) Data is published to the MDMR at set frequencies.<br />Master Data Management – Architecture<br />1) Master data is integrated, cleansed, enriched, and transformed to the MDM data structuresfrom the systems of record.<br />2) A set of security packages is defined to restrict columns and rows.<br />Requesting Application Instance<br />MDM Source<br />Business Objects<br />Data Integrator<br />After Login Trigger<br />APP_PTM<br />Application<br />Systems of Record (Master data)<br />App.<br />schema<br />APP_PTC<br />MDM_SEC<br />schema<br />Organizations<br />Leaders<br />MDMR<br />schema<br />MDM_USER<br />schema<br />Finance<br />Other systems<br />19<br />© IRI 2011<br />
    29. 29. How are the policies defined?<br />A database package is defined for each object to be protected.<br />The database package functions reference information stored in the Application Context of the current database session and returns the SQL fragment to be either:<br />Appended to the Where Clause or<br />Used to determine whether or not to show the column contents in the result.<br />
    30. 30. Sample policy package<br />CREATE OR REPLACE package MDM_ORG_SEC_PKG as<br /> function SET_PREDICATE (OBJECT_SCHEMA VARCHAR2,OBJECT_NAME VARCHAR2) return VARCHAR2; -- row policies<br />...<br /> function VIEW_TIMEZONE (OBJECT_SCHEMA VARCHAR2,OBJECT_NAME VARCHAR2) return VARCHAR2; -- column policies<br />end MDM_ORG_SEC_PKG;<br />/<br />CREATE OR REPLACE package body MDM_ORG_SEC_PKG as<br /> CTX_VALUE VARCHAR2(2000) :=NULL;<br />CTX_NAME VARCHAR2(30) :='MDM_SEC_CTX'; -- defines the application context<br /> function SET_PREDICATE (OBJECT_SCHEMA VARCHAR2,OBJECT_NAME VARCHAR2) return VARCHAR2 as<br /> begin<br /> CTX_VALUE:=NULL;<br /> CTX_VALUE:=SYS_CONTEXT('USERENV','SESSION_USER'); -- identifies the user associated with this session<br /> IF CTX_VALUE = OBJECT_SCHEMA THEN<br />RETURN '1=1'; -- if the user is the same as the object owner, enable all access<br /> END IF;<br /> CTX_VALUE:=NULL;<br /> CTX_VALUE:=SYS_CONTEXT(CTX_NAME,'ORG_PREDICATE',2000); -- gets the where clause fragment to be appended<br /> IF CTX_VALUE IS NULL THEN<br />RETURN '1=0'; -- if there is no information in the context, deny all access<br /> ELSE<br />RETURN CTX_VALUE; -- return the row constraints<br /> END IF; <br /> EXCEPTION<br /> WHEN OTHERS THEN<br /> RETURN '1=0';<br /> end;<br />...<br /> function VIEW_TIMEZONE (OBJECT_SCHEMA VARCHAR2,OBJECT_NAME VARCHAR2) return VARCHAR2 as<br /> begin<br />…<br />
    31. 31. How do we connect the policy to the<br />database object?<br />First, we define the objects and protected column lists in the MDM repository security tables.<br />Then we execute a set of scripts to:<br />Create a stored procedure that will be used to populated the application context when the user connects.<br />Create the application context and tie to the above stored procedure.<br />Tie the VPD policies to the database objects.<br />
    32. 32. MDM Repository Security Tables (part 1)<br />Defines the objects (tables, views, synonyms) to be protected<br />Defines the columns to be protected<br />
    33. 33. MDM Repository Security Tables (part 2)<br />For the specific user, this<br />defines the where clause<br />fragment …<br />and the column sets to include<br />with the default columns.<br />
    34. 34. We’re done!<br />Now, when the SQL statement is executed:<br /> SELECT * FROM MDM_ORG<br /> Only those rows / columns authorized are returned in the result set.<br />
    35. 35. What do we have in our MDM?<br />Reference Data Sets<br />Languages<br />Geopolitical Locations (Countries, Regions, etc.)<br />Currencies<br />Exchange Rates<br />Master Data Sets<br />Organizations<br />Leaders<br />Employees & reporting hierarchy – in progress<br />Physical Facilities (Churches, Temples, Seminaries, etc.) – in progress<br />
    36. 36. How successful is our MDM?<br />We replicate authorized master data to 54 other productions systems (230+ instances) daily.<br />Developers become familiar with the canonical models for master data, which reduces the development cost of using master data.<br />Web services deliver master data from the MDM repository.<br />It takes about 20 minutes to provision master data to a new consumer.<br />Changes in source systems are completely transparent to the consuming systems.<br />Reduce number of DB links to source systems.<br />
    37. 37. Thank you!<br />Questions & Answers<br />Pablo Riboldi<br /> Solution Manager for Information Governance & Quality<br />riboldipj@ldschurch.org<br /> The Church or Jesus Christ of Latter-day Saints<br /> Visit us at www.mormon.org<br />

    ×