Decision Suppot System

547 views
422 views

Published on

Slides from a seminar by Lucian Parshall to the NZ Ministry of Education on Thursday 21 January 2010

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
547
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Decision Suppot System

  1. 1. Keep in Perspective “Not everything that counts !can be counted. !And not everything that !can be counted - counts.” Albert Einstein
  2. 2. Keep in Perspective “Not everything that counts !can be counted. !And not everything that !can be counted - counts.”
  3. 3. How important is a DSS?
  4. 4. How important is a DSS? Imagine the CEO of a large enterprise with: 53,800 employees
  5. 5. How important is a DSS? Imagine the CEO of a large enterprise with: 53,800 employees 2,600 branch offices
  6. 6. How important is a DSS? Imagine the CEO of a large enterprise with: 53,800 employees 2,600 branch offices $6.4 billion annual budget
  7. 7. How important is a DSS? Imagine the CEO of a large enterprise with: 53,800 employees 2,600 branch offices $6.4 billion annual budget 758,000 customers
  8. 8. How important is a DSS? Imagine the CEO of a large enterprise with: 53,800 employees 2,600 branch offices $6.4 billion annual budget 758,000 customers What strategic information might this CEO expect to be available?
  9. 9. How important is a DSS? Imagine the CEO of a large enterprise with: 53,800 employees 2,600 branch offices $6.4 billion annual budget 758,000 customers What strategic information might this CEO expect to be available? Would an 18 month delay in finding out how many employees left the company be OK?
  10. 10. How important is a DSS? Imagine the CEO of a large enterprise with: 53,800 employees 2,600 branch offices $6.4 billion annual budget 758,000 customers What strategic information might this CEO expect to be available? Finding out how a customer performed on an evaluation - six months later?
  11. 11. How important is a DSS? Imagine the CEO of a large enterprise with: 53,800 employees 2,600 branch offices $6.4 billion annual budget 758,000 customers What strategic information might this CEO expect to be available? Not knowing the location or the age of the technology in your branch offices?
  12. 12. How important is a DSS? Imagine the CEO of a large enterprise with: 53,800 employees 2,600 branch offices $6.4 billion annual budget 758,000 customers What strategic information might this CEO expect to be available? We find ourselves in the Information Age with an aging information system
  13. 13. What are the needs of the educational community?
  14. 14. Current Education Data Sets Performance Infrastructure Personal Student Finance Foreign
  15. 15. Current Education Data Sets Performance Infrastructure Personal Student Finance Foreign
  16. 16. Current Education Data Sets Performance Infrastructure Personal Student Finance Foreign Promotes narrow decisions based on information extracted from one or two functional data-sets (finance and assessment)
  17. 17. Current Education Data Sets Performance Infrastructure Personal Student Finance Foreign Redundant data entry is common Disconnected data increases resources needed Collections become costly and inefficient
  18. 18. Educational Data Warehousing Performance TID (10 digit) Student Personnel NCLB math TID (10 digit) Admin Unit No. Teacher SS# ........ Supply NCLB read LEA Number Teacher SS# IHE Unit No, ........ ........ Teacher Assign ........ AP score Stud gender ........ SS# PSAT math Stu Grade lvl Type of Cert ........ ........ Stu FTE ........ IHE Endorsement ........ ACT enroll ........ Cert Exp Date Admin Unit No. Finance LEA Number School ........ Infrastructure Per Pupil Admin Unit No. Total Rev Data Partnerships ........ ........ Technology Avg Salary Crime/Safety Foreign Data Gov Data Operating Bdgt Admin Unit No. ........ ........ Admin Unit No. ........ ........ ........ Employment Bld Age Live Births NCES ........ GPS system University Title I Num Arrests Cong Dist
  19. 19. Educational Data Warehousing Performance TID (10 digit) Student Personnel NCLB math a location Admin Unit No. Supply that: TID (10 digit) Teacher SS# NCLB read ........ Integrates information from . . . . . . . . ........ Teacher SS# disparate LEA Number ........ Teacher Assign IHE Unit No, AP score systems into a total view of Cert . . . . . . . . PSAT math . . . and a common Stud gender Stu Grade lvl Type ..... SS# ........ foundation for understanding student ........ Stu FTE ........ ........ Cert Exp Date IHE Endorsement ACT enroll performance and school improvement Admin Unit No. Finance LEA Number School ........ Infrastructure Per Pupil Admin Unit No. Total Rev Data Partnerships ........ ........ Technology Crime/Safety and Foreign Data Avg Salary Operating Bdgt Gov Data Admin Unit No. Provides for Admin Unit No. set. .of. .definitions ........ ........ a common . . . . ........ Becomes the Live .Births. Bld Age sole . source Employment ... . . of reusable data NCES ........ Improves timeliness and utility of reports Title I GPS system University Num Arrests Cong Dist
  20. 20. What is a Data Warehouse?
  21. 21. What is a Data Warehouse? DW is not just storage but the tools to query, analyze and present information on the web.
  22. 22. What is a Data Warehouse? DWs have many definitions - with these similarities: Subject oriented - gives information about a person instead of operations. Integrated - a variety of sources are merged into a whole. Non-volatile - provides users with a consistent picture over specified time periods. Robust architecture - that allows concurrent access by a multiple number of users with frequent queries. Quality data - valid and reliable data that promotes confidence in DW and forms the nucleus of information used by the educational community.
  23. 23. What is a Data Warehouse? DWs have many definitions - with these similarities: Subject oriented - gives information about a person instead of operations. Integrated - a variety of sources are merged into a whole. Non-volatile - provides users with a consistent picture over specified time periods. Robust architecture - that allows concurrent access by a multiple number of users with frequent queries. Quality data - valid and reliable data that promotes confidence in DW and forms the nucleus of information used by the educational community.
  24. 24. What is a Decision Support System? DSS is a process used by the educational community (with support of the data warehouse) that transforms data into a knowledgebase that will support decision-making.
  25. 25. DSS starts with a: What is a Decision Problem + Support System? administration = Data
  26. 26. DSS starts with a: What is a Decision Problem + Support System? administration = Data + dissemination = Information
  27. 27. DSS starts with a: What is a Decision Problem + Support System? administration = Data + dissemination = Information + social discussion = Knowledge
  28. 28. DSS starts with a: What is a Decision Problem + Support System? administration = Data + dissemination = Policy/Action Information + social discussion = + community Knowledge response =
  29. 29. DSS starts with a: What is a Decision Problem + Support System? administration = Data + dissemination = Policy/Action Information + wisdom to ask a more + social complex question = discussion = + community Knowledge response =
  30. 30. 12 Steps to Creating the DSS These steps are a combination of buying and building that depend on time and money
  31. 31. 12 Steps to Creating the DSS Education Community Involvement These steps are a combination of buying and building that depend on time and money
  32. 32. 12 Steps to Creating the DSS Education Community Involvement Conceptual Agreement DRA/DBA Staffing Meta Data DRA Security/Confidentiality Unique ID# Edit check/ETL Time from Operation to Dup Res These steps are a Analysis BI tool DBA combination of buying and building that depend on time and money Data Warehouse Data Mining Information Democracy Training Decision Support System
  33. 33. 12 Steps to Creating the DSS Education Community Involvement Conceptual Agreement DRA/DBA Staffing Meta Data DRA Security/Confidentiality Unique ID# Edit check/ETL Time from Operation to Dup Res Analysis BI tool DBA Looks linear - is multidimensional Data Warehouse Data Mining Information Democracy Training Decision Support System
  34. 34. 12 Steps to Creating the DSS Education Community Involvement Conceptual Agreement DRA/DBA Staffing Meta Data DRA Security/Confidentiality Unique ID# Edit check/ETL Time from Operation to Dup Res Analysis BI tool DBA Factor in the fatigue-fizzle function Data Warehouse Data Mining Information Democracy Training Decision Support System
  35. 35. 12 Steps to Creating the DSS Education Community Involvement Conceptual Agreement DRA/DBA Staffing Meta Data DRA Security/Confidentiality Unique ID# Edit check/ETL Time from Operation to Dup Res Analysis BI tool DBA Factor in the fatigue-fizzle function Data Warehouse Information Escape velocity Data Mining Democracy Training Decision Support System
  36. 36. 12 S S teps DS
  37. 37. Step #1 Concept Formation Initiation Project Phase Planning Execution/ Closeout Control 12 S S teps DS
  38. 38. Building the Framework for the DW and DSS
  39. 39. Building the Framework for the DW and DSS Must have conceptual agreement on the:
  40. 40. Building the Framework for the DW and DSS Must have conceptual agreement on the: Design of support system
  41. 41. Building the Framework for the DW and DSS Must have conceptual agreement on the: Design of support system Policy to protect data
  42. 42. Building the Framework for the DW and DSS Must have conceptual agreement on the: Design of support system Policy to protect data Advisory committee(s)
  43. 43. Building the Framework for the DW and DSS Must have conceptual agreement on the: Design of support system Policy to protect data Advisory committee(s) Buy or build and
  44. 44. Building the Framework for the DW and DSS Must have conceptual agreement on the: Design of support system Policy to protect data Advisory committee(s) Buy or build and Funding
  45. 45. Building the Framework for the DW and DSS Must have conceptual agreement on the: Design of support system Policy to protect data Advisory committee(s) Buy or build and Funding
  46. 46. DSS Design: Best Practice
  47. 47. DSS Design: Best Practice Who? Whom? Where? With? What? School Codes When? Data Warehouse
  48. 48. DSS Design: Best Practice Who? Whom? Where? With? What? Decision Decision Support Support Users Tools Used for: Used how: Operation Data mining Management School Codes Analysis Policy Makers Ad-hoc query Instruction Off-line Research When? manipulation Data Warehouse Foreign data i.e. Employment, Higher Ed
  49. 49. DSS Design: Best Practice Data Democracy Web Interface Who? Whom? Where? With? What? Decision Decision Support Support Users Tools Used for: Used how: Operation Data mining Management School Codes Analysis Policy Makers Ad-hoc query Instruction Off-line Research When? manipulation Data Warehouse Foreign data i.e. Employment, Higher Ed
  50. 50. DSS Design: Best Practice Data Democracy Web Interface Who? Whom? Where? With? What? Decision Decision Support Support Users Tools Used for: Used how: Operation Data mining Management School Codes Analysis Policy Makers Ad-hoc query Instruction Off-line Research When? manipulation Data Warehouse Foreign data i.e. Employment, Higher Ed h y As professionals, we need to make informed W decisions, anticipate their impact on education and design appropriate policy.
  51. 51. Steering Committee Oversight to design of the DSS
  52. 52. Steering Committee Oversight to design of the DSS Local district policy concerns
  53. 53. Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification
  54. 54. Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification Standard reports, and
  55. 55. Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification Standard reports, and Long term funding
  56. 56. Cost Savings? (OCIO-USED) Warehouse/DSS initiative Current costs (paper and mail) Break even 2001 2002 2003 2004 2005 2006
  57. 57. Cost Savings? (OCIO-USED) Warehouse/DSS initiative Current costs (paper and mail) Break even 2001 2002 2003 2004 2005 2006 “We spend a lot of resources on an existing data edifice that isn’t very useful”
  58. 58. 12 S S teps DS
  59. 59. Step #2 DRA & DBA 12 S S teps DS
  60. 60. Partnership on Both Sides of the Keyboard
  61. 61. Partnership on Both Sides of the Keyboard DRA: modifies and DBA: technical enforces standards implementation that sustain the of the data DSS environment - warehouse chairs data environment - managers group chairs IT group
  62. 62. DRA and DBA Collaboration User requirements Feature expectation (DRA) Critical Divergence IT Development Cycle (DBA) 18 Mo 30 Mo Time
  63. 63. DRA and DBA Collaboration User requirements Feature expectation (DRA) Critical Divergence IT Development Cycle (DBA) 18 Mo 30 Mo Time
  64. 64. DRA and DBA Collaboration User requirements Feature expectation (DRA) Critical Divergence IT Development Cycle (DBA) 18 Mo 30 Mo Time
  65. 65. DRA and DBA Collaboration User requirements Feature expectation (DRA) Critical Divergence IT Development Cycle (DBA) 18 Mo 30 Mo Time Outcome of building the DW within time frame: Data Warehouse will run 12-15 years - whereas Current apps last 6-7 years (with patches)
  66. 66. 12 S S teps DS
  67. 67. Step #3 Define the Data 12 S S teps DS
  68. 68. Meta Data Data about the Database in the Data Warehouse
  69. 69. Meta Data Data about the Database in the Data Warehouse to define Meta Data Pupil Personnel break task into Student Meta Data Human Manual logical support Personnel Resources groups Meta Data Manual Finance Meta Data promotes the - Finance Meta Data Office Manual • common understanding by users • data interchange with other agencies Test Performance Meta Data Company Manual School Facilities Meta Data Manual Manager
  70. 70. Meta Data Online Manuals Student Performance Personnel Finance School Infrastructure
  71. 71. Meta Data Online Manuals Student Performance Personnel Finance School Infrastructure Employment Higher Education
  72. 72. Meta Data Online Manuals Name of Field Student Field Number Technical Information Number of characters: (length) SIF name: Blanks: (not accepted, null) XML tag: < > Performance Field type: (alpha, numeric, character) Record position: (35-39) Warehouse name:R/Ecode Warehouse type: VCAR Progam Information Code format: Personnel Definition: Finance Elements (variables): School Date Information Submission: Effective: Reporting Period: Infrastructure Revised: Discontinued: ? Edits Employment Error traps: Fatal Error: Cross field edits: Warning: Historical Information Higher Education Form number replaced: Used for: Statutory requirement: Report number:
  73. 73. Step #4 Maintaining Security and Confidentiality 12 S S teps DS
  74. 74. Protection is both sides of the keyboard
  75. 75. Protection is both sides of the keyboard System Security (DBA) Identification (confident of who) Authentication (confident of source) Authorization (grant access rights) Access control (user profiling) Administration (security procedures) Auditing (monitoring and detection)
  76. 76. Protection is both sides of the keyboard Confidentiality (DRA) Established FERPA policy Unique NSN w/check sum Statistical disclosure (<6) System Security (DBA) Human subject review policy Purge and destruction Identification (confident of who) Set levels of access & audit Authentication (confident of source) Authorization (grant access rights) Access control (user profiling) Administration (security procedures) Auditing (monitoring and detection)
  77. 77. Step #5 Unique Testing ID (NSN) 12 S S teps DS
  78. 78. Test Identification Number: Production Record Warehouse layout layout First name TID (10 digit) Last name First name Date of Birth Last name Gender Date of Birth ……… Gender FTE ……… ……… FTE Grade ……… Race/Ethnic Grade ……… Race/Ethnic ……… ……… ………
  79. 79. Test Identification Number: Production Record Warehouse layout layout TID rules: • Only assigned to one student (is unique). First name TID (10 digit) • Number and name can be confirmed as Last name First name being correct (verified via check sum). Date of Birth Last name • Meets criteria as an identifier (is valid). Gender Date of Birth • Has no intrinsic meaning (is nominal). ……… Gender • Can be substituted for a student’s name FTE ……… (is not personally identifiable). ……… FTE • Permanent over the life-cycle of the Grade ……… student (0-21 for special education). Race/Ethnic Grade • Is returned and used by all local ……… Race/Ethnic education agencies (is ubiquitous). ……… ……… • Issued only by the SEA (is restricted). ……… • Accessible by selected SEA employees only (is confidential).
  80. 80. Test Identification Number: Problems 10 digit Check Sum First name Last name Constant Date of Birth Gender ……… FTE ……… Variables Grade Race/Ethnic ……… ……… ID# Admin Unit #
  81. 81. Test Identification Number: Problems 10 digit Check Sum First name First name Last name Last name Constant Date of Birth Moves Date of Birth Gender Gender ……… ……… FTE FTE ……… ……… Variables Variables Grade Race/Ethnic change Grade Race/Ethnic ……… ……… ……… ……… ID# ID# Admin Unit # Admin Unit #
  82. 82. Test Identification Number: Problems 10 digit Check Sum First name First name Last name Last name Need other constant: Constant Date of Birth Moves Date of Birth Date of Immunization Gender Gender Place of Birth ……… ……… Birth Cert Number FTE FTE ……… ……… Variables Variables Grade Race/Ethnic change Grade Race/Ethnic ……… ……… ……… ……… ID# ID# Admin Unit # Admin Unit #
  83. 83. 42 states use a unique student identifier (DQC) How constructed How issued (NCES) (NCES) ISD Combination (1) of fields (5) Soc Sec Number (8) LEA (9) SEA (20) SSN plus algorithm (1) Other (9) Random School (2) number (8) Other (4)
  84. 84. Crossing over from aggregate to single record Data reliability and validity Aggregate collection Time
  85. 85. Crossing over from aggregate to single record Data reliability and validity Single record collection Aggregate collection Time
  86. 86. Crossing over from aggregate to single record Data reliability and validity Single record collection Aggregate collection Time
  87. 87. Crossing over from aggregate to single record Data reliability and validity Single record collection Aggregate collection Time
  88. 88. Aggregated Data can be Misleading Classrooms: District A Classrooms: District B Class size! Reading Class size! Reading ! 10! 8.04 ! 14! ! 8.1 ! 8! 6.95 ! 6! ! 6.13 ! 13! 7.58 ! 4! ! 3.1 ! 9! 8.81 ! 12! ! 9.13 ! 11! 8.33 ! 7! ! 7.26 ! 14! 9.96 ! 5! ! 4.74 ! 6! 7.24 ! 10! ! 9.14 ! 4! 4.26 ! 8! ! 8.14 ! 12! 10.84 ! 13! ! 8.74 ! 7! 4.82 ! 9! ! 8.77 ! 5! 5.68 ! 11! ! 9.26 Classrooms: District C Classrooms: District D Class size! Reading Class size! Reading ! 10! 7.46 ! 8! 6.58 ! 8! 6.77 ! 8! 5.76 ! 13! 12.74 ! 8! 7.71 ! 9! 7.11 ! 8! 8.84 ! 11! 7.81 ! 8! 8.47 ! 14! 8.84 ! 8! 7.04 ! 6! 6.08 ! 8! 5.25 ! 4! 5.39 ! 19! 12.5 ! 12! 8.15 ! 8! 5.56 ! 7! 6.42 ! 8! 7.91 ! 5! 5.73 ! 8! 6.89
  89. 89. Aggregated Data can be Misleading Classrooms: District A Classrooms: District B Class size! Reading Class size! Reading ! 10! 8.04 ! 14! ! 8.1 ! 8! 6.95 ! 6! ! 6.13 ! 13! 7.58 ! 4! ! 3.1 ! 9! 8.81 ! 12! ! 9.13 ! 11! 8.33 ! 7! ! 7.26 ! 14! 9.96 ! 5! ! 4.74 ! 6! 7.24 ! 10! ! 9.14 ! 4! 4.26 ! 8! ! 8.14 ! 12! 10.84 ! 13! ! 8.74 ! 7! 4.82 ! 9! ! 8.77 ! 5! 5.68 ! 11! ! 9.26 Classrooms: District C Classrooms: District D Class size! Reading Class size! Reading ! 10! ! 8! 7.46 6.77 ! 8! ! 8! 6.58 5.76 ! Avg. classrooms != 11 ! 13! 12.74 ! 8! 7.71 ! Avg. class size != 9.0 ! 9! 7.11 ! 8! 8.84 ! 11! 7.81 ! 8! 8.47 ! Avg. reading score != 7.5 ! 14! ! 6! 8.84 6.08 ! 8! ! 8! 7.04 5.25 Four districts are similar ! 4! 5.39 ! 19! 12.5 ! 12! 8.15 ! 8! 5.56 ! 7! 6.42 ! 8! 7.91 ! 5! 5.73 ! 8! 6.89
  90. 90. Reports using Disaggregated Data 10 10 5 5 District A District B 10 20 10 20 10 10 Individual reading scores 5 5 Four districts are District C District D very different 10 20 10 20
  91. 91. Step #6 Cleaning the Data 12 S S teps DS
  92. 92. Quality Data
  93. 93. Quality Data Reasons for poor quality of data: Absence of definitions Unclear definitions Lack of human resources Inconsistent collections cycles (not ongoing) Insufficient time Inadequate training on entry and data traps Lack of data integration Fear of 'punishment' (look bad syndrome)
  94. 94. Quality Data The key elements that improve the quality of what is being collected include: • Consistency. Data fields must have a standardized definition so that each entity can be collected from each district in a systematic manner. • Timeliness. There is no efficiency in gathering statewide data that reflects a one-time need or an unusual piece of information. Do a survey. • Reliability. The data set should reflect a dependable measurement of every entity from one collection cycle to another (i.e., data has accuracy regardless of who enters it.) • Validity. A data element must reflect a logical and meaningful description of an entity and should not be subject to interpretation (i.e., data has utility to answer the question being asked.)
  95. 95. Step #7 Resolving Duplicates 12 S S teps DS
  96. 96. Thresholds and Assigning ID numbers True False Match Non- match
  97. 97. Thresholds and Assigning ID numbers True False Match is true - are the same student (assign same ID#) Match Pat ! Smith! M! 1/19/60 Pat! T! Smith! M! 1/19/60 Non- match
  98. 98. Thresholds and Assigning ID numbers True False Match is true - are the same student (assign same ID#) Match Pat ! Smith! M! 1/19/60 Pat! T! Smith! M! 1/19/60 Non-match is true - are different students Non- (assign different ID#s) match Pat ! Smith! F! 1/19/60 Pat! T! Smith! ! 1/19/61 Patrick ! Smith ! M! 1/19/60
  99. 99. Thresholds and Assigning ID numbers True False Match is true - are Match is false - are the same student different students (assign same ID#) (assign same ID#) Match Pat ! Smith! M! 1/19/60 Patricia! Smith! F! 1/19/60 Pat! T! Smith! M! 1/19/60 Pat! ! Smith! ! 1/19/60 Non-match is true - are different students Non- (assign different ID#s) match Pat ! Smith! F! 1/19/60 Pat! T! Smith! ! 1/19/61 Patrick ! Smith ! M! 1/19/60
  100. 100. Thresholds and Assigning ID numbers True False Match is true - are Match is false - are the same student different students (assign same ID#) (assign same ID#) Match Pat ! Smith! M! 1/19/60 Patricia! Smith! F! 1/19/60 Pat! T! Smith! M! 1/19/60 Pat! ! Smith! ! 1/19/60 Non-match is true - are Non-match is false - different students are the same student Non- (assign different ID#s) (assign different ID#s) match Pat ! Smith! F! 1/19/60 Pat ! Smith! M! 1/19/60 Pat! T! Smith! ! 1/19/61 Patrick! Smith! ! 1/19/60 Patrick ! Smith ! M! 1/19/60 Pat ! ! Smyth! M! 1/19/60
  101. 101. Thresholds and Assigning ID numbers True False Match is true - are Match is false - are the same student different students (assign same ID#) (assign same ID#) Match Pat ! Smith! M! 1/19/60 Patricia! Smith! F! 1/19/60 Pat! T! Smith! M! 1/19/60 Pat! ! Smith! ! 1/19/60 Non-match is true - are Non-match is false - different students are the same student Non- (assign different ID#s) (assign different ID#s) match Pat ! Smith! F! 1/19/60 Pat ! Smith! M! 1/19/60 Pat! T! Smith! ! 1/19/61 Patrick! Smith! ! 1/19/60 Patrick ! Smith ! M! 1/19/60 Pat ! ! Smyth! M! 1/19/60 Error Creep
  102. 102. Step #8 Select a BI Tool 12 S S teps DS
  103. 103. Task #1: Create Model Software & Hardware
  104. 104. Task #1: Create Model Software & Hardware On scalable, normalized, symmetric multiprocessing architecture
  105. 105. Task #2: Set up a ‘road map’
  106. 106. Task #3: Choose a BI tool
  107. 107. Task #3: Choose a BI tool
  108. 108. Step #9 Data Warehouse 12 S S teps DS
  109. 109. Benefits of DW:
  110. 110. Benefits of DW: Reduction of paper forms Savings from data duplication Best use of technology Sole source of reusable data Common set of definitions Integrated environment of core data Breaks cycle of low quality data Answers that took months take days Reports that took days take minutes
  111. 111. Data Democracy for the Educational Community Ad-hoc Reports Pre- defined Simple - Query Sophisticated one time - ongoing
  112. 112. Data Democracy for the Educational Community Ad-hoc Leg isla rs tive Re searche Ai d es Finance Officers Reports rs ito A ud General Public Reporters Pre- defined Simple - Query Sophisticated one time - ongoing
  113. 113. Data Democracy for the Educational Community Ad-hoc Leg isla rs tive Re searche Finance Ai d es u ll Officers P As system is Reports rs ito used one will A ud find a need to store data not being captured sh General Reporters Pre- P u Public defined Simple - Query Sophisticated one time - ongoing
  114. 114. Push example: one time - pre defined
  115. 115. Push example: one time - pre defined School report card • School Size: small vs. large schools • Spending: percent of budget on staff salary • Safety: rate of expulsions and degree of crime • Technology: ratio of pc's to students & connectivity • Class Size: teacher-student ratio, average size • Staff Turnover: rate and attendance • Advanced Placement: number passing test • Test Scores: gaps in State performance test • College Acceptance Rate: percent taking ACT, PSAT • Graduation/Dropout Rates: number taking GED • Satisfaction: teachers, parents and students
  116. 116. Pull example: ongoing - Ad hoc Significant Usable
  117. 117. Pull example: ongoing - Ad hoc Significant Usable The largest class size in high school is the 9th grade Not No really
  118. 118. Pull example: ongoing - Ad hoc Significant Usable The largest class size in high school is the 9th grade Not No really Some 9th grades have a disproportionate number Possibly No of Hispanics
  119. 119. Pull example: ongoing - Ad hoc Significant Usable The largest class size in high school is the 9th grade Not No really Some 9th grades have a disproportionate number Possibly No of Hispanics Many female Hispanics in the 9th grade are Possibly Yes retained due to poor science skills
  120. 120. Pull example: ongoing - Ad hoc Significant Usable The largest class size in high school is the 9th grade Not No really Some 9th grades have a disproportionate number Possibly No of Hispanics Many female Hispanics in the 9th grade are Possibly Yes retained due to poor science skills Hispanics in the 8th grade had fewer computers in science classrooms and more teachers who do not Yes Yes have a teaching major in science
  121. 121. The DW Backbone: The Sole Authority for the Educational Community NCLB School Accreditation Crime/ Safety Quality Workforce AYP State Report Card Title II (IHE) IDEA Fiscal Trends
  122. 122. The DW Backbone: The Sole Authority for the Educational Community NCLB School Accreditation Crime/ Safety Quality Workforce AYP State Report Card Title II (IHE) IDEA Fiscal Trends
  123. 123. Step #10 Data Mining 12 S S teps DS
  124. 124. Data re-construction
  125. 125. Data re-construction Undirected and exploratory knowledge discovery
  126. 126. Data re-construction Undirected and exploratory knowledge discovery Sequencing: order of patterns or groups
  127. 127. Data re-construction Undirected and exploratory knowledge discovery Framing: using past data to predict trend Sequencing: order of patterns or groups
  128. 128. Data re-construction Undirected and exploratory knowledge discovery Framing: using past data to predict trend Sequencing: order of patterns or groups Clustering: assembling unforeseen groups
  129. 129. Data re-construction Undirected and exploratory knowledge discovery Framing: using past data to predict trend Sequencing: order of patterns or groups Clustering: assembling unforeseen groups Drilling: interactive discovery
  130. 130. Multidimensional Ad-hoc Analysis Student Technology Infrastructure Performance
  131. 131. Multidimensional Single Parent Homes Ad-hoc Analysis Live Births Student Technology Infrastructure Millages Passed Performance
  132. 132. Multidimensional Single Parent Homes Ad-hoc Analysis Live Births Ethnic change Student and growth by enrollment Technology Infrastructure Millages Passed Performance
  133. 133. Multidimensional Single Parent Homes Ad-hoc Analysis Live Births Ethnic change Student and growth by enrollment Technology Infrastructure Millages Performance Passed by gender by PCs Performance
  134. 134. Multidimensional Single Parent Homes Ad-hoc Analysis Live Births Ethnic change Student and growth by enrollment Technology Infrastructure Millages Performance Passed by gender by PCs Trends and Projections Performance
  135. 135. Multidimensional Single Parent Homes Ad-hoc Analysis Live Births Ethnic change Student and growth by enrollment Technology Infrastructure Millages Performance Passed by gender by PCs Trends and Projections Performance Similar districts that passed bonds by month over past 3 yrs by ethnicity by building by grade
  136. 136. Step #11 Conduct Training 12 S S teps DS
  137. 137. The ultimate goal of training is to have everyone who touches the data at every level know what is expected of them, so that the data that are submitted will be the valid and reliable.
  138. 138. Training Training must also include detailed procedures, for example:
  139. 139. Training Training must also include detailed procedures, for example: Who gets notified when an error is discovered and how is the notification done?
  140. 140. Training Training must also include detailed procedures, for example: Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)?
  141. 141. Training Training must also include detailed procedures, for example: Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data?
  142. 142. Training Training must also include detailed procedures, for example: Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user?
  143. 143. Training Training must also include detailed procedures, for example: Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing?
  144. 144. Training Training must also include detailed procedures, for example: Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing? Who receives confirmation that the file has been received as specified?
  145. 145. Training Training must also include detailed procedures, for example: Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing? Who receives confirmation that the file has been received as specified? Who secures the data and maintains confidentiality?
  146. 146. Reallocation of Resources Have Have multiple collections - use once disregard Data collection, Analysis Reporting Decision error checks, support and and clean-up shared data
  147. 147. Reallocation of Resources Have Want Have multiple collections - use once disregard Data collection, Analysis Reporting Decision error checks, support and and clean-up shared data Staff training - shifts from front to back end
  148. 148. Step #12 The DSS 12 S S teps DS
  149. 149. Step #12 The DSS Providing access to critical information for driving, managing, tracking, and measuring institutional policies and goals. 12 S S teps DS
  150. 150. The first decision of the DSS is to make a decision Transactional Cyclical
  151. 151. The first decision of the DSS is to make a decision Transactional Cyclical Realtime Points in time Day to day operations Historical Updates daily/weekly Updates quarterly 7X24 6X18 Read/write Read only Short term data retention Long-term (longitudinal) Mission critical queries Strategic-analytical queries More open access paths More restricted access Standardized reports Adhoc reports Server based Warehouse technology
  152. 152. DSS: Helps Anticipate Issues Problem Anticipation Policy Policy Repercussion Forecasting Problem Reaction
  153. 153. DSS: Helps Anticipate Issues Problem Anticipation Policy Policy Repercussion Forecasting Current Problem Reaction
  154. 154. DSS: Helps Anticipate Issues Problem Anticipation Need to be Policy Policy Repercussion Forecasting Problem Reaction
  155. 155. DSS: Helps Anticipate Issues Problem Anticipation Cannot anticipate with only Need ‘required’ to be data Policy Policy Repercussion Forecasting Problem Reaction
  156. 156. Help Anticipate Impact of Policy: Class Size
  157. 157. Help Anticipate Impact of Policy: Class Size Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?
  158. 158. Help Anticipate Impact of Policy: Class Size Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test? Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?
  159. 159. Help Anticipate Impact of Policy: Class Size Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test? Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs? Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff?
  160. 160. Help Anticipate Impact of Policy: Class Size Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test? Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs? Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff? Infrastructure Issues - Do buildings have the space for additional classrooms?
  161. 161. Help Anticipate Impact of Policy: Class Size Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test? Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs? Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff? Infrastructure Issues - Do buildings have the space for additional classrooms? Trend Issues - Will improved achievement impact employment, graduation or adult life roles?
  162. 162. Impact on State Standards
  163. 163. Impact on State Standards Efficiency of System Inputs Process Outputs
  164. 164. Impact on State Standards Efficiency of System Inputs Process Outputs Input issues: fiscal resources teacher supply building structure technology poverty
  165. 165. Impact on State Standards Efficiency of System Inputs Process Outputs Input issues: fiscal resources teacher supply building structure technology Process issues: poverty crime and safety prof development attendance teacher experience student performance
  166. 166. Impact on State Standards Efficiency of System Inputs Process Outputs Input issues: Output issues: fiscal resources college entrance teacher supply graduate numbers building structure retention rates technology employment Process issues: poverty crime and safety prof development attendance teacher experience student performance
  167. 167. Impact on State Standards Effectiveness of System Efficiency of System Inputs Process Outputs Outcomes Input issues: Output issues: fiscal resources college entrance teacher supply graduate numbers building structure retention rates Outcome issues: employment works with others technology Process issues: acquires information poverty crime and safety understands inter-relationships prof development allocates resources attendance works w/variety of tech teacher experience student performance Impact Policy
  168. 168. Impact on State Standards Effectiveness of System Efficiency of System Inputs Process Outputs Outcomes Output issues: college entrance graduate numbers retention rates Outcome issues: pa ct employment works with others im acquires information i ot ll n ith o nly understands inter-relationships allocates resources W yw ata works w/variety of tech lic re d’ d p o ui req Impact Policy ‘
  169. 169. pa ct im i ot ll n ith o nly W yw ata lic re d’ d p o ui ‘ req
  170. 170. Finding the Balance Required Desired Data Data Social Integration Mandatory Vocational Orientation Measurement in volume Use of Time (amounts, avg., ranks, percents) Daily Living Skills Realistic Mobility Use of Environmental Ques
  171. 171. Finding the Balance Required Desired Data Data Social Integration Mandatory Vocational Orientation Measurement in volume Use of Time (amounts, avg., ranks, percents) Daily Living Skills Realistic Mobility Use of Environmental Ques The DSS must help policy makers find a comfortable balance between acceptable risks and benefits.
  172. 172. Helps in Data Discovery Input Process Output Outcomes Issues Issues Issues Issues General Public Parents Teachers Standards moves from Support Staff efficiency to effectiveness Admin/Boards State Legislators Others
  173. 173. One Last Time Web Front End District Users Upload data formats Public Portal Access Correct duplicate data Predefined Report Cards FERPA requests Limited queries Dashboard/Scorecard
  174. 174. One Last Time Web Front End District Users Security Upload data formats Public Portal Access Correct duplicate data Predefined Report Cards FERPA requests Limited queries Dashboard/Scorecard File ETL: Developer Applications • Student • Assessment • Finance • Professional Student IDs Match & Merge Check Sum Audit (FERPA) Error reports
  175. 175. One Last Time Web Front End District Users Security Upload data formats Public Portal Access Correct duplicate data Predefined Report Cards FERPA requests Limited queries Dashboard/Scorecard File ETL: Developer Applications • Student Reliable/ • Assessment Valid • Finance • Professional Student IDs Match & Merge Check Sum WAREHOUSE Audit (FERPA) GPS Error reports School Meta Codes Data
  176. 176. One Last Time Web Front End District Users Security Upload data formats Public Portal Access Correct duplicate data Predefined Report Cards FERPA requests Limited queries Dashboard/Scorecard File ETL: Developer Applications • Student Reliable/ • Assessment Valid Data Mart • Finance • Professional Student IDs Match & Merge Check Sum WAREHOUSE Audit (FERPA) GPS DoE Users Error reports Generate Report Card School Meta Federal: EDEN, NCLB, IDEA Codes Data Skopus Issue Assessment IDs
  177. 177. Current problem: data rich and information poor
  178. 178. Current problem: data rich and information poor Data Silos Department
  179. 179. Current problem: data rich and information poor Data Gap: Silos Lack of confidence No trust in system Have a low ROI Department Educational Community
  180. 180. Solution Data Democracy Data Warehouse Secure Scalable Flexible Finance Apply information Personnel Meta Data Scho ol and facilitate Meta Data decision-making Manual Meta Data P ent erfor Stu d ta Manual Manual m Meta ance D a Dat Meta al Manu a Manu al Department Educational Community
  181. 181. Solution Data Democracy Data Warehouse Secure Scalable Flexible Finance Apply information Personnel Meta Data Scho ol and facilitate Meta Data decision-making Manual Meta Data P ent erfor Stu d ta Manual Manual m Meta ance D a Dat Meta al Manu a Manu al Department Educational Community
  182. 182. Lucian_Parshall@ameritech.net
  183. 183. Without Data You’re Just Another Person With an Opinion
  184. 184. We find ourselves in an Information Age with an aging information system
  185. 185. Decisions begin with good data
  186. 186. Most of the fun using the DSS is not finding the answer to your question - it’s finding the new questions you don’t have the answers to.

×