Data Governance: The Kansas Approach (PPT)Presentation Transcript
Data Governance: The Kansas Approach Education Information Management Advisory Consortium (EIMAC) Spring Meeting May 2007 Presented by: Kathy Gosa Kansas State Department of Education
Kansas: The way we were…
Independent “silo’s” each collecting and reporting data independently
Quality of data collected is unknown and “questionable”
Minimal link or consistency in reports
No agreement on “authoritative source”
No agreement on definitions or policies
Work often redundant
Security needs not necessarily understood or followed
This led to….
Challenges in meeting the hundreds of Data Requests we receive
Challenges in explaining inconsistencies
Difficulty in submitting to PBDMI/EDEN (no data submitted in 04-05 SY)
Confusion from schools regarding policies / definitions / etc.
Resource constraints – essential enterprise information in the head(s) of a few individuals
Add to this….
KSDE implemented Kansas Individual Data on Students (KIDS), assigning state IDs to all Kansas students in spring 2005 and collecting student level data as basis for funding, enrollment, federal and state reporting, assessments, and accountability in 2005-2006 school year.
Enterprise Data System (including metadata)
KSDE received funding from state legislature in 2006 for 3 year project to implement an Enterprise Data System.
How can we quickly get on a path of organization and productivity?
One part of the answer:
Institute Data Governance
“ When an organization views data as an enterprise asset (transcending the data warehouse and spanning the whole organization), it establishes a … data governance committee that oversees and guides data stewardship across the organization (and may include) Data quality Data architecture Data integration Data warehousing Metadata management Master data management” --Philip Russom, TDWI
Why Data Governance?
Stepped up regulatory demands
Sarbanes-Oxley Act, 2002
Data Quality Act, 2002
EdFacts / EDEN (PBDMI)
Data are becoming critical for decision making.
The stakes are getting higher and questionable data quality is unacceptable.
The world has changed – no one believes that IT is a superhero!
Setting the Stage
Learn what we can from business and industry.
Gain executive buy-in.
Focus on ROI and advantages.
Communicate in their terms.
Propose solutions, not problems.
Make Data Governance part of our culture.
Takes time and patience.
One department at a time!
Data Governance is a process, not an event!
Steps to establishing the Kansas Data Governance Program
Determine our approach
Establish a structure
Explicitly define Roles & Responsibilities
Identify individuals for these roles
Provide on-going training and capacity building
Identify an issue escalation / resolution process
Expand, reuse, and improve each year
Kansas Approach to Data Governance
Learn from industry, but customize for our specific situation and needs
Buy-in (vs. Mandated)
Mandated is faster and easier to implement, but may be harder to sustain. Also requires the authority to mandate!
Buy-in may take more time to implement, but will be more sustainable since will become part of the culture.
Use Project Management techniques to establish the initial processes and track the progress.
Evaluate usefulness of Project Management for following years.
Define Success (focus on a specific problem)
EDEN vs Enterprise Data System vs Data Requests vs Communication vs Master Data Management vs ….
Information Security Master Data Management Policy Management Enterprise Data System - Metadata EDEN – Federal Reporting Data Requests Data Quality DATA GOVERNANCE
Playing catch up
Setting up infrastructure and processes (work flow & data flow) to “get the ball rolling”
03-04 SY and 04-05 SY minimum data sets
Along with submitting 05-06 data
Staff’s conflicting priorities
Determining true source of data and subsequently the data owner & steward
Working with EDEN folks to work out the “kinks”
Previously KSDE approached federal reporting (PBDMI, etc) as an event
Now we approach federal reporting as a process
Project Management (EDEN Coordinator)
Setting up an infrastructure for work flow and data flow that is supported, documented, and repeatable.
EDEN Metadata Repository
EDEN Coordinator attended two day training on How to Build & Implement a Data Governance & Data Steward Program
Monitor project status and escalate as needed
Create & maintain project documentation
EDEN submission plan tool
Project plan with work breakdown structure
Communication matrix document
Roles & Responsibilities document
Lots of communication!!
EDEN status meeting with core team
Data Governance Board meeting
Data Steward meeting
Created a metadata management tool
Focused on EDEN data elements only
EDEN file level info (directions related to the entire file)
EDEN data element level info
EDEN data element name
EDEN permitted values
EDEN submission used in
KSDE data owner & steward
KSDE definition & business rules
KSDE source path (server, database, table, field)
Transformations (crosswalk from KSDE value to EDEN value)
Training for staff
Metadata Repository User’s Guide
EDEN Data Flow Processes
EDEN Work Flow Process
EDEN Coordinator –
Download file specs from EDEN website
Add elements to metadata repository
(with Director assistance) - Identify data steward
ETL Programmer - Extracts data from source using metadata info and puts into EDEN repository in the EDEN format (transformations & aggregations)
EDEN Coordinator - Checks data for valid values
Data Steward - Checks data for content accuracy and gives approval for submission to EDEN
XML Programmer –
Pulls data from EDEN repository and creates XML file
Submits XML file to EDEN
If any errors occur they are dealt with accordingly and file resubmitted
Data Steward Program & Workgroup
Workgroup reports to Data Governance Board
Created a Data Steward Program Manual
Communication & collaboration
Build capacity for ownership and accountability of data
Eliminate the silo effect of working with data
Ongoing agenda items for workgroup:
Build capacity for other Data Governance initiatives:
Student level data system
Horizontal data system integration update
Enterprise data system update
Data Steward Responsibilities
Identify & manage metadata
Identify & resolve data quality issues (integrity, timeliness, accuracy, completeness)
Communicate data quality issues and problems to individuals that can influence change, as needed
Communicate new & changed data requirements to necessary individuals
Determine business and security needs of data
Define requirements for archiving data
Provide input to data analysis
Ensure consistency between EDEN reporting and other federal reporting
EDEN Accomplishments (Year 1)
Leadership support obtained
Designated full time EDEN Coordinator
Established technical & non-technical infrastructure
EDEN metadata repository
Data Governance Board & Data Steward Workgroup
Data flow & work flow processes initiated
Made decision to submit all data via XML
Submitted minimum data sets for 03-04 & 04-05
Submitted 05-06 data
Now working on 06-07 data
Plans for EDEN – Year 2
Kick-off meeting with each department Director and Data Stewards
Schedule and expectations
New format for Project Management document
Excel vs MS Project
Weekly updates to Commissioners
Trying new techniques with areas that had challenges
group work sessions
involve additional staff
Unable to re-use as much as expected of what we did last year
Early code not optimal
Feds made a lot of changes
No optionals – deadlines more fixed
Virtual (vs. Dedicated)
Dedicated (all full-time resources) – allows team to focus solely on measurement and improvement of data processes and data quality but requires significant investment from the organization.
Virtual (all part-time resources) – a more practical approach for an organization getting started, but other “job” may distract from addressing data governance issues.
How many FTEs?
For the first year we estimate approximately 4.0 FTE were dedicated to EDEN reporting and Data Governance. (Headcount approximately 14; plus 15 DGB members).
For this year we anticipate this may decrease slightly (3.0 FTE).
How do we fund the positions?
First year partially funded via an NCES Special Task Order; partially funded by tying in to other (funded) projects such as Enterprise Data System.
Future years - ???? Until reporting efficiencies are realized this is an issue!
Applying EDEN Accomplishments to the Enterprise Information Security Master Data Management Policy Management Enterprise Data System - Metadata EDEN – Federal Reporting Data Requests Data Quality DATA GOVERNANCE
Data Governance Structure Data Stewards and Programmers Data Steward Program Coordinator Data Steward Program Director Data Governance Board KSDE Leadership Data Request Review Board Data Governance Board KSDE Leadership Data Stewards & Programmers
Data Governance Board
Director level decision makers from each department
Meet at least monthly
Mission: Support Enterprise Data System Initiative as a source of knowledge and advocacy, provide guidance, and ensure buy-in.
Learn from one another (e.g., Demonstrations from each department, updates on EDEN and EDS)
Solve problems! (Data Calendar, Policy Management, Data Requests, etc.)
Data Request Review Board
Provides consistent treatment of data requests.
Considers, prioritizes and assigns requests for data.
Uses automated escalation and tracking process (FootPrints ® ).
Monthly in person
Review status of requests and assign priorities to non-urgent requests
Other Roles & Responsibilities
Executive Leadership – Commissioner and Deputy Commissioners are responsible to
advocate for data governance and data quality; and
resolve issues escalated from DGB or Program Director.
Data Owners – Department Directors are responsible for applications and their associated data to
define and approve access; and
identify data security classification.
Data Custodian – Director of IT is responsible to
ensure safety and integrity of data in custody of IT;
implement application and data access controls appropriate for security classification; and
provide reasonable safeguards for information resources.
Data Stewards and Programmers Data Steward Program Coordinator Data Steward Program Director Data Governance Board KSDE Leadership Issue Escalation and Accountability
Master Data Management
Initiative to ensure that critical data subjects are managed at the enterprise level (e.g., collected and updated at a single source).
DBG assists in identifying appropriate data groups and what source should be the “master”.
Currently we are implementing MDM processes for Organization data and core Student Data.
Student Data Repository … … LEA District & Schools Student Identifier Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Master Data Management Teacher Assignment & Licensure Budget & Finance Migrant Career & Tech Ed Special Ed Assessments Core Student Data Staff Training & Capacity Building Enterprise Architecture S e c u r i t y Common Authentication – Security Architecture Security & Confidentiality Policies – Security Certificates EDEN MetaData Organizations
Enterprise Meta Data
Designed based on lessons from EDEN Meta Data
Re-use as much meta data as possible
Enterprise Meta Data (first version) is specific to Enterprise Data Warehouse
Student Data Repository Core Student Data … … LEA District & Schools Student Identifier Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Enterprise Architecture S e c u r i t y Common Authentication – Security Architecture Security & Confidentiality Policies – Security Certificates Definitions Enterprise Meta Data Teacher Assignment & Licensure Budget & Finance Migrant Career & Tech Ed Special Ed Assessments Enterprise Data Warehouse Staff Training & Capacity Building Enterprise MetaData Organizations Data Mart
Data Stewards will be trained regarding use of meta data and business intelligence tools
EDEN will become a data mart of the EDS
Opportunity to give meaningful data back to LEAs
Significant focus on training LEA staff regarding
use of meta data
use of business intelligence tools
effective data use
Student Data Repository Core Student Data … … LEA District & Schools Enterprise Data Warehouse Student Identifier Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Enterprise Architecture S e c u r i t y Common Authentication – Security Architecture Security & Confidentiality Policies – Security Certificates Definitions Enterprise Data System: Iteration 1 MetaData Business Rules, Tech Info, Data Quality Enterprise Data Warehouse Integrated Time Variant Cleansed Teacher Assignment & Licensure Budget & Finance Migrant Career & Tech Ed Special Ed Assessments Staff Training & Capacity Building Organizations Cleanse Integrate Transform Load Extraction & Analysis Data Mart Research Data Mart AYP
Student Data Repository Core Student Data … … LEA District & Schools Enterprise Data Warehouse Student Identifier Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Submission Verification Enterprise Architecture S e c u r i t y Common Authentication – Security Architecture Security & Confidentiality Policies – Security Certificates Definitions Enterprise Data System: Iterations 2 and 3 Cleanse Integrate Transform Load Extraction & Analysis Data Mart Research Enterprise Data Warehouse Integrated Time Variant Cleansed MetaData Business Rules, Tech Info, Data Quality … Teacher Assignment & Licensure Budget & Finance Migrant Career & Tech Ed Special Ed Assessments Staff Training & Capacity Building Organizations Data Mart State Rpts Data Mart Fed Rpts Data Mart LEA Analysis Data Mart AYP Return Data to the LEAs
Eases resource constraints
Provides consistent message to the field
Helps minimize surprises!
Promotes perspective that we’re in this together.
Policies & Guidance
Data Governance Board has adopted this as an initiative:
Establishing standard template for documenting (with version control!)
Discussing central location for policies
Implemented process for public comment
KIDS Data Quality Certification initiative
Data Verification tools and guidance for districts