• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Data Governance: The Kansas Approach (PPT)
 

Data Governance: The Kansas Approach (PPT)

on

  • 3,940 views

 

Statistics

Views

Total Views
3,940
Views on SlideShare
3,940
Embed Views
0

Actions

Likes
1
Downloads
183
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • These are some of the main challenges we have encountered so far: Playing catch up in a short amount of time: Setting up our infrastructure and processes (work flow & data flow) to “get the ball rolling” Submitting the 03-04 SY and 04-05 SY minimum data sets so we could be “checked off” of that list And also submitting the 05-06 data Another challenge has been managing staff’s conflicting priorities - most of you can probably relate to this Leadership played an important role with this challenge Determining the true source or “single point of truth” of data and subsequently the data owner & steward Working with EDEN folks to work out the “kinks”: EDFacts reports inaccurate File specs with wrong info Clarifications on communications Our data cyles not matching with EDEN data cycles (ie: when data is ready for reporting) Error messages sometimes not very helpful Should always report issues you find to the EDEN PSC so they can get resolved and in the long run make it easier for yourself and for other states. This way we are all helping each other. In the past, KSDE approached federal reporting (PBDMI, etc) as an event But now we approach federal reporting as a process : (a new way of thinking about it – more built into our everyday way of operating)
  • We approached EDEN using Project Management techniques, methodology and tools: The first things we did were to: Set up an infrastructure for our work and data flow processes And we wanted something that was: supported (specific staff were assigned responsibilities) documented (if a staff member left the work would not halt) repeatable (can be used from year to year – don’t have to re-invent the wheel every year) This included: Creating an EDEN Repository database Creating an EDEN Metadata Repository And I attended a training on How to Build & Implement a Data Governance & Data Steward Program. I brought that info back with me and put it to use. We created ways to monitor the project status by using some project management documentation tools including the: EDEN submission plan tool – the spreadsheet workbook where you document when you will submit the files Microsoft project plan with work breakdown structure – I created a detailed project plan and attached our agreed upon work process to each EDEN file Communication matrix document – a document that clearly identifies how the status of the project will be communicated and to who Roles & Responsibilities document – a document that clearly identifies the roles of each player in the project and what their responsibilities are Lots of communication!! EDEN status meeting with core team – this was initially every two weeks but has now moved to a monthly meeting and consists of myself, IT Director, ETL & XML programmers Data Governance Board meeting - monthly Data Steward meeting - monthly Ad-hoc meetings (w/ individual stewards, programmers, etc)
  • We created a metadata management application: The main driver of the application is the EDEN data element name The application is strictly for EDEN metadata You can enter info at two levels: EDEN file level info (directions related to the entire file) EDEN data element level info which includes attributes such as: EDEN data element name EDEN permitted values (ie: M or F) EDEN submission used in (what EDEN file(s) is this element used in) KSDE data owner & steward EDEN definition KSDE definition & business rules KSDE source path (server, database, table, field) – where we will be pulling the data from for reporting Transformations (crosswalk from KSDE value to EDEN value) I created a User’s Guide and did training for the staff that needed to use the application data owners, data stewards, data steward backups, and a few other interested staff
  • This is an illustration of our data flow process: This process corresponds with our work flow process which I am talking about in the next slide so I won’t go into major detail here other than to say what each symbol represents: On the far left – this is our source systems - where we are pulling the data FROM for reporting to EDEN The cylinder on the bottom – represents our metadata repository The box on the bottom right is our reminder to make sure what we are submitting through EDEN is consistent with our other federal reporting. That is why it is so important to have the data stewards involved in the process. The top cylinder is our EDEN repository – where we store the data after extraction from the source system. At this stage it is in the EDEN format (permitted values & correct aggregations). This is also where we review the data for correctness prior to submission. The box that says EDEN submission files represents where the XML programmer pulls the data into an XML file and submits it to EDEN. Ok now lets go to the next slide and look at the work flow process.
  • This may be difficult to read but I left this much info on the screen on purpose. It illustrates all of the work process that take place just to submit ONE file. And all of the communication that needs to happen to be successful in submitting EDEN data. Some dependencies to this work flow: top half of slide represents what steps can be done after the file spec is issued by EDEN the bottom half of the slide represents what steps are done when our source data is ready for reporting WORK FLOW PROCESS EDEN Coordinator - Download file specs from EDEN website EDEN Coordinator - Add core element info to metadata repository EDEN Coordinator (with Director assistance) - Identify data steward EDEN Coordinator - Get copy of specs to steward via email notification Data Steward - Document in metadata (ie: steward / owner, transformation values, source info, business rules / definitions) ____________________________________________________________ Data Steward - Indicates when data is ready for reporting ETL Programmer - Extracts data from source using metadata info and puts into EDEN repository in the EDEN format (transformations & aggregations) EDEN Coordinator - Checks data for valid values (using file specs) Data Steward - Checks data for content accuracy and gives approval for submission to EDEN XML Programmer - Pulls data from EDEN repository and creates XML file XML Programmer - Submits XML file to EDEN If any errors occur they are dealt with accordingly (ie: data corrected, file format corrected, etc) and file resubmitted Don’t be afraid to call EDEN Partner Support Center for assistance, they are very helpful!
  • Data Steward Program: Data stewards are existing staff (our “data experts”). They are already working with the data and in most cases already doing federal reporting. We have simply formalized their role and given them a forum to share their knowledge more easily with each other. The workgroup reports to the Data Governance Board & we have monthly meetings I created a Data Steward Program Manual that includes thing such as: overview & mission scope & responsibilities guiding principles EDEN checklist & data flow diagram data governance hierarchy & issue resolution escalation path Main objectives of the data steward program: Communication & collaboration with each other Data quality – share info and techniques Build capacity for ownership and accountability of data Eliminate the silo effect of working with data Ongoing agenda items for the workgroup include: EDEN update KIDS student level data system update Horizontal data system integration update Enterprise data system update Data quality Issues / concerns data stewards want to discuss To build on this concept, our trainer is working on a data quality certification program that we can extend out to districts & schools. This will help reinforce the importance and responsibility of the role that district and school staff are playing in helping us to achieve a high standard for data quality.
  • The data steward responsibilities are outlined here: Identify & manage the EDEN metadata (and eventually our data warehouse metadata – which we will talk about a little later) As mentioned earlier these are the types of metadata elements they work with: Data Definition & business rules Source info Who is the official data owner & steward What are the valid values for this element Transformation from KSDE value to EDEN value The stewards also: Identify & resolve data quality issues (integrity, timeliness, accuracy, completeness) Communicate data quality issues and problems to individuals that can influence change, as needed Communicate new & changed data requirements to necessary individuals Determine business and security needs of data Define requirements for archiving data Provide input to data analysis Ensure consistency between EDEN reporting and other federal reporting One of the biggest issues has been conflicting priorities for these staff. Currently they are being asked to do additional work. Because in most cases they are still reporting their data “the old way”. But the selling point in getting this off the ground is that once we get to the point were we are reporting through EDEN only (which we have for our special ed data) that this will drastically reduce their burden for federal reporting. Especially since we have set up our work and data flow processes so that they are repeatable and will be the same from year to year (the first year into this process we had a lot of “ramp up” (ie: entering all the metadata info) work also). So in theory each year (as changes to data elements levels off) the EDEN reporting will become easier and metadata will just need to be maintained.
  • Closing remarks: I know this sounds like a lot, but we have been able to accomplished all of this in a little over a year. Started this process in November of 2005 Rehash of major accomplishments: Leadership support was obtained Designated full time EDEN Coordinator Technical & Non-technical infrastructure was set up EDEN repository EDEN metadata repository Data Governance Board & Data Steward Workgroup Data flow & work flow processes initiated Made decision to submit all data via XML Submitted minimum data sets for 03-04 & 04-05 - DONE Submitted 05-06 data - DONE Now working on 06-07 data - made the 2/15 deadline for the two IDEA state level files (2 & 89). Working on end of March deadlines now ANY QUESTIONS?????? THANKS FOR COMING
  • Staffing estimate: .5 FTE – EDEN Data Coordinator .1 FTE – Data Manager .15 FTE (15 persons * 20 hrs = 300 hrs) DGB members .25 FTE – Web (XML) programmer .25 FTE – DBA 2.5 FTE (.25*10) – Data Steward . 25 FTE – Metadata system development 4.0 FTE TOTAL

Data Governance: The Kansas Approach (PPT) Data Governance: The Kansas Approach (PPT) Presentation Transcript

  • Data Governance: The Kansas Approach Education Information Management Advisory Consortium (EIMAC) Spring Meeting May 2007 Presented by: Kathy Gosa Kansas State Department of Education
  • Kansas: The way we were…
    • Independent “silo’s” each collecting and reporting data independently
    • Quality of data collected is unknown and “questionable”
    • Minimal link or consistency in reports
    • No agreement on “authoritative source”
    • No agreement on definitions or policies
    • Inconsistent technologies
    • Work often redundant
    • Security needs not necessarily understood or followed
  • This led to….
    • Challenges in meeting the hundreds of Data Requests we receive
    • Challenges in explaining inconsistencies
    • Difficulty in submitting to PBDMI/EDEN (no data submitted in 04-05 SY)
    • Confusion from schools regarding policies / definitions / etc.
    • Resource constraints – essential enterprise information in the head(s) of a few individuals
  • Add to this….
    • More data!
      • KSDE implemented Kansas Individual Data on Students (KIDS), assigning state IDs to all Kansas students in spring 2005 and collecting student level data as basis for funding, enrollment, federal and state reporting, assessments, and accountability in 2005-2006 school year.
    • Enterprise Data System (including metadata)
      • KSDE received funding from state legislature in 2006 for 3 year project to implement an Enterprise Data System.
  • The dilemma…
    • How can we quickly get on a path of organization and productivity?
    • One part of the answer:
    • Institute Data Governance
    “ When an organization views data as an enterprise asset (transcending the data warehouse and spanning the whole organization), it establishes a … data governance committee that oversees and guides data stewardship across the organization (and may include) Data quality Data architecture Data integration Data warehousing Metadata management Master data management” --Philip Russom, TDWI
  • Why Data Governance?
    • Stepped up regulatory demands
      • Sarbanes-Oxley Act, 2002
      • Data Quality Act, 2002
      • EdFacts / EDEN (PBDMI)
    • Data are becoming critical for decision making.
    • The stakes are getting higher and questionable data quality is unacceptable.
    • The world has changed – no one believes that IT is a superhero!
  • Setting the Stage
    • Learn what we can from business and industry.
      • Professional Training
      • Webinars (on-going)
    • Gain executive buy-in.
      • Focus on ROI and advantages.
      • Communicate in their terms.
      • Propose solutions, not problems.
      • Demonstrate successes!
    • Make Data Governance part of our culture.
      • Takes time and patience.
      • One department at a time!
    Data Governance is a process, not an event!
  • Steps to establishing the Kansas Data Governance Program
    • Determine our approach
    • Establish a structure
    • Explicitly define Roles & Responsibilities
    • Identify individuals for these roles
    • Provide on-going training and capacity building
    • Identify an issue escalation / resolution process
    • Expand, reuse, and improve each year
  • Kansas Approach to Data Governance
    • Customized
      • Learn from industry, but customize for our specific situation and needs
    • Buy-in (vs. Mandated)
      • Mandated is faster and easier to implement, but may be harder to sustain. Also requires the authority to mandate!
      • Buy-in may take more time to implement, but will be more sustainable since will become part of the culture.
    • Project Management
      • Use Project Management techniques to establish the initial processes and track the progress.
      • Evaluate usefulness of Project Management for following years.
    • Define Success (focus on a specific problem)
      • EDEN vs Enterprise Data System vs Data Requests vs Communication vs Master Data Management vs ….
  • Information Security Master Data Management Policy Management Enterprise Data System - Metadata EDEN – Federal Reporting Data Requests Data Quality DATA GOVERNANCE
  • EDEN Challenges
    • Playing catch up
      • Setting up infrastructure and processes (work flow & data flow) to “get the ball rolling”
      • 03-04 SY and 04-05 SY minimum data sets
      • Along with submitting 05-06 data
    • Staff’s conflicting priorities
    • Determining true source of data and subsequently the data owner & steward
    • Working with EDEN folks to work out the “kinks”
    • Previously KSDE approached federal reporting (PBDMI, etc) as an event
    • Now we approach federal reporting as a process
  • Project Management (EDEN Coordinator)
    • Setting up an infrastructure for work flow and data flow that is supported, documented, and repeatable.
      • EDEN Repository
      • EDEN Metadata Repository
      • EDEN Coordinator attended two day training on How to Build & Implement a Data Governance & Data Steward Program
    • Monitor project status and escalate as needed
    • Create & maintain project documentation
      • EDEN submission plan tool
      • Project plan with work breakdown structure
      • Communication matrix document
      • Roles & Responsibilities document
    • Lots of communication!!
      • EDEN status meeting with core team
      • Data Governance Board meeting
      • Data Steward meeting
      • Ad-hoc meetings
  • EDEN Metadata
    • Created a metadata management tool
    • Focused on EDEN data elements only
    • EDEN file level info (directions related to the entire file)
    • EDEN data element level info
      • EDEN data element name
      • EDEN permitted values
      • EDEN submission used in
      • KSDE data owner & steward
      • EDEN definition
      • KSDE definition & business rules
      • KSDE source path (server, database, table, field)
      • Transformations (crosswalk from KSDE value to EDEN value)
    • Training for staff
    • Metadata Repository User’s Guide
  • EDEN Data Flow Processes
  • EDEN Work Flow Process
    • EDEN Coordinator –
      • Download file specs from EDEN website
      • Add elements to metadata repository
      • (with Director assistance) - Identify data steward
      • Get copy of specs to steward
    • Data Steward –
      • Document metadata (ie: steward / owner, transformation values, source info, business rules / definitions)
      • Indicates when data is ready for reporting
    • ETL Programmer - Extracts data from source using metadata info and puts into EDEN repository in the EDEN format (transformations & aggregations)
    • EDEN Coordinator - Checks data for valid values
    • Data Steward - Checks data for content accuracy and gives approval for submission to EDEN
    • XML Programmer –
      • Pulls data from EDEN repository and creates XML file
      • Submits XML file to EDEN
    • If any errors occur they are dealt with accordingly and file resubmitted
  • Data Steward Program & Workgroup
    • Workgroup reports to Data Governance Board
    • Created a Data Steward Program Manual
    • Main objectives
      • Communication & collaboration
      • Data quality
      • Build capacity for ownership and accountability of data
      • Eliminate the silo effect of working with data
    • Ongoing agenda items for workgroup:
      • EDEN update
      • Data Quality
      • Build capacity for other Data Governance initiatives:
        • Student level data system
        • Horizontal data system integration update
        • Enterprise data system update
  • Data Steward Responsibilities
    • Identify & manage metadata
    • Identify & resolve data quality issues (integrity, timeliness, accuracy, completeness)
    • Communicate data quality issues and problems to individuals that can influence change, as needed
    • Communicate new & changed data requirements to necessary individuals
    • Determine business and security needs of data
    • Define requirements for archiving data
    • Provide input to data analysis
    • Ensure consistency between EDEN reporting and other federal reporting
  • EDEN Accomplishments (Year 1)
    • Leadership support obtained
    • Designated full time EDEN Coordinator
    • Established technical & non-technical infrastructure
      • EDEN repository
      • EDEN metadata repository
      • Data Governance Board & Data Steward Workgroup
      • Data flow & work flow processes initiated
    • Made decision to submit all data via XML
    • Submitted minimum data sets for 03-04 & 04-05
    • Submitted 05-06 data
    • Now working on 06-07 data
  • Plans for EDEN – Year 2
    • Kick-off meeting with each department Director and Data Stewards
      • Lessons Learned
      • Schedule and expectations
      • Expected challenges
    • New format for Project Management document
      • Excel vs MS Project
    • Weekly updates to Commissioners
    • Trying new techniques with areas that had challenges
      • group work sessions
      • involve additional staff
    • Unable to re-use as much as expected of what we did last year
      • Early code not optimal
      • Feds made a lot of changes
      • No optionals – deadlines more fixed
  • Staffing!
    • Virtual (vs. Dedicated)
      • Dedicated (all full-time resources) – allows team to focus solely on measurement and improvement of data processes and data quality but requires significant investment from the organization.
      • Virtual (all part-time resources) – a more practical approach for an organization getting started, but other “job” may distract from addressing data governance issues.
    • How many FTEs?
      • For the first year we estimate approximately 4.0 FTE were dedicated to EDEN reporting and Data Governance. (Headcount approximately 14; plus 15 DGB members).
      • For this year we anticipate this may decrease slightly (3.0 FTE).
    • How do we fund the positions?
      • First year partially funded via an NCES Special Task Order; partially funded by tying in to other (funded) projects such as Enterprise Data System.
      • Future years - ???? Until reporting efficiencies are realized this is an issue!
  • Applying EDEN Accomplishments to the Enterprise Information Security Master Data Management Policy Management Enterprise Data System - Metadata EDEN – Federal Reporting Data Requests Data Quality DATA GOVERNANCE
  • Data Governance Structure Data Stewards and Programmers Data Steward Program Coordinator Data Steward Program Director Data Governance Board KSDE Leadership Data Request Review Board Data Governance Board KSDE Leadership Data Stewards & Programmers
  • Data Governance Board
    • Director level decision makers from each department
    • Meet at least monthly
    • Mission: Support Enterprise Data System Initiative as a source of knowledge and advocacy, provide guidance, and ensure buy-in.
    • Actionable agendas
      • Learn from one another (e.g., Demonstrations from each department, updates on EDEN and EDS)
      • Solve problems! (Data Calendar, Policy Management, Data Requests, etc.)
  • Data Request Review Board
    • Provides consistent treatment of data requests.
    • Considers, prioritizes and assigns requests for data.
    • Uses automated escalation and tracking process (FootPrints ® ).
    • Meetings
      • Monthly in person
      • Adhoc virtually
      • Review status of requests and assign priorities to non-urgent requests
  •  
  • Other Roles & Responsibilities
    • Executive Leadership – Commissioner and Deputy Commissioners are responsible to
      • advocate for data governance and data quality; and
      • resolve issues escalated from DGB or Program Director.
    • Data Owners – Department Directors are responsible for applications and their associated data to
      • define and approve access; and
      • identify data security classification.
    • Data Custodian – Director of IT is responsible to
      • ensure safety and integrity of data in custody of IT;
      • implement application and data access controls appropriate for security classification; and
      • provide reasonable safeguards for information resources.
  • Issue Resolution
    • Setting expectations
    • Escalation Process
    Data Stewards and Programmers Data Steward Program Coordinator Data Steward Program Director Data Governance Board KSDE Leadership Issue Escalation and Accountability
  • Master Data Management
    • Initiative to ensure that critical data subjects are managed at the enterprise level (e.g., collected and updated at a single source).
    • DBG assists in identifying appropriate data groups and what source should be the “master”.
    • Currently we are implementing MDM processes for Organization data and core Student Data.
  • Student Data Repository … … LEA District & Schools Student Identifier Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Master Data Management Teacher Assignment & Licensure Budget & Finance Migrant Career & Tech Ed Special Ed Assessments Core Student Data Staff Training & Capacity Building Enterprise Architecture S e c u r i t y Common Authentication – Security Architecture Security & Confidentiality Policies – Security Certificates EDEN MetaData Organizations
  • Enterprise Meta Data
    • Designed based on lessons from EDEN Meta Data
    • Re-use as much meta data as possible
    • Enterprise Meta Data (first version) is specific to Enterprise Data Warehouse
  • Student Data Repository Core Student Data … … LEA District & Schools Student Identifier Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Enterprise Architecture S e c u r i t y Common Authentication – Security Architecture Security & Confidentiality Policies – Security Certificates Definitions Enterprise Meta Data Teacher Assignment & Licensure Budget & Finance Migrant Career & Tech Ed Special Ed Assessments Enterprise Data Warehouse Staff Training & Capacity Building Enterprise MetaData Organizations Data Mart
  • Enterprise Data System
    • Iterative development process
    • Prioritized subjects:
      • Iteration 1 – organizations, students, assessments, accountability
      • Iteration 2 – staff, finance
      • Iteration 3 - programs
    • Data Stewards will be trained regarding use of meta data and business intelligence tools
    • EDEN will become a data mart of the EDS
    • Opportunity to give meaningful data back to LEAs
    • Significant focus on training LEA staff regarding
      • use of meta data
      • use of business intelligence tools
      • effective data use
  • Student Data Repository Core Student Data … … LEA District & Schools Enterprise Data Warehouse Student Identifier Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Enterprise Architecture S e c u r i t y Common Authentication – Security Architecture Security & Confidentiality Policies – Security Certificates Definitions Enterprise Data System: Iteration 1 MetaData Business Rules, Tech Info, Data Quality Enterprise Data Warehouse Integrated Time Variant Cleansed Teacher Assignment & Licensure Budget & Finance Migrant Career & Tech Ed Special Ed Assessments Staff Training & Capacity Building Organizations Cleanse Integrate Transform Load Extraction & Analysis Data Mart Research Data Mart AYP
  • Student Data Repository Core Student Data … … LEA District & Schools Enterprise Data Warehouse Student Identifier Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Enterprise Architecture S e c u r i t y Common Authentication – Security Architecture Security & Confidentiality Policies – Security Certificates Definitions Enterprise Data System: Iterations 2 and 3 Cleanse Integrate Transform Load Extraction & Analysis Data Mart Research Enterprise Data Warehouse Integrated Time Variant Cleansed MetaData Business Rules, Tech Info, Data Quality … Teacher Assignment & Licensure Budget & Finance Migrant Career & Tech Ed Special Ed Assessments Staff Training & Capacity Building Organizations Data Mart State Rpts Data Mart Fed Rpts Data Mart LEA Analysis Data Mart AYP Return Data to the LEAs
  • Collaboration
    • Eases resource constraints
    • Provides consistent message to the field
    • Helps minimize surprises!
    • Promotes perspective that we’re in this together.
  • Policies & Guidance
    • Data Governance Board has adopted this as an initiative:
    • Establishing standard template for documenting (with version control!)
    • Discussing central location for policies
    • Implemented process for public comment
  • Data Quality
    • KIDS Data Quality Certification initiative
    • Data Verification tools and guidance for districts
    • Future version: Accountability Integrated Dashboard
  • Our Data Governance Challenges
    • Linking data and policies
    • Dealing with “distractions”
    • Data quality
    • Changes in the political climate
    • Staffing