Successfully reported this slideshow.

Using Data to Bring People Together

848 views

Published on

Peter Radcliffe
2010 Quality Fair

  • Be the first to comment

  • Be the first to like this

Using Data to Bring People Together

  1. 1. Using Data to Bring People Together Quality Fair 2010 Project and Change Management Collaborators (PCMC) Session Peter M. Radcliffe Executive Director Office of Planning and Analysis University of Minnesota
  2. 2. Overview • Why use data? • Why collaborate? • How can data enhance collaboration? • What are the challenges? • Case study: How has this worked in practice? • How could you apply this in your own work?
  3. 3. How do we make good decisions? Good decisions bring together… • Evidence – What does reality look like? – How do the parts fit together? • Context – How will stakeholders respond? – What resources are available? – What are our goals?
  4. 4. How do we make bad decisions? • Evidence is wrong – Data is inaccurate – Model (understanding of what factors lead to what outcomes) is faulty • Context is wrong – Local culture is different or changing – Resources are unavailable – Contrary to goals of organization
  5. 5. How can data help us? • More comprehensive than our personal experience • More recent than our personal experience • Clearer representation of underlying trends • Challenges us to examine our biases and assumptions
  6. 6. Benefits of collaboration • Sharing of knowledge and expertise – Not all of the information you need for your work can be found in your own area – Your team may lack critical skills or technologies • Improved credibility – Broader collaborations less likely to be perceived as parochial and self-interested
  7. 7. Challenges to collaboration • Problem definition – What situation are we trying to change? – Does evidence suggest the problem is real? • Establishing agreement on outcomes – How do we define success? • Understanding stakeholders and interests – What does each partner need? – Are interests complimentary, opposed, or orthogonal
  8. 8. Types and Uses of Data • Descriptive data – What is the state of the world in this area? • Metrics – What are our goals and standards of progress? • Analysis – How do elements connect in this area? – Can show causation or simply correlation
  9. 9. Data, truth, and stories • Facts may speak for themselves, but they donʼt say anything interesting on their own • Data are summaries of a story – they can help identify some of the critical points, but they are not the story themselves • The full “truth” requires putting observations in context
  10. 10. Anecdotes and data • “The plural of anecdote is data” – Raymond Wolfinger 1 • Anecdotes, like data, are summaries of experiences which highlight the elements of the story that are most relevant for the discussion or deemed most likely to persuade the audience – A data set with a very small “n” (most anecdotes) has very large error bounds on predictions – A large data set is hard to understand without statistics to summarize the information, sacrificing some detail – Combined, they make a fuller, more engaging argument 1 Nelson W. Polsby. PS, Vol. 17, No. 4. (Autumn, 1984), pp. 778-781. Pg. 779: “Raymond Wolfinger's brilliant aphorism ʻthe plural of anecdote is dataʼ never inspired a better or more skilled researcher. I e-mailed Wolfinger last year and got the following response from him: "I said 'The plural of anecdote is data' some time in the 1969-70 academic year while teaching a graduate seminar at Stanford. The occasion was a student's dismissal of a simple factual statement--by another student or me--as a mere anecdote. The quotation was my rejoinder.“
  11. 11. Trust, Bias, and Integrity • “Figures won't lie, but liars will figure” – Charles Grosvenor • Dishonesty with numbers doesnʼt require outright fabrication • Selective use of data sources and definitions can distort descriptions and analyses • Best defense is transparency in data sources and definitions, and use of institutional standards for both whenever possible • "Grosvenor, Charles H."  The Oxford Dictionary of American Quotation. Hugh Rawson and Margaret Miner. Oxford University Press 2008. Oxford Reference Online. Oxford University Press.  University of Minnesota - Twin Cities.  24 January 2010  <http://www.oxfordreference.com/views/ENTRY.html?subview=Main&entry=t251.e757>
  12. 12. Building the relationship • Collaboration involves intruding on another person or officeʼs business • Need to understand their goals and needs, as well as your own • Must demonstrate you are committed to their success, not just yours
  13. 13. Case Study: Recreational Sports and Institutional Research • Recreational sports had information on facility usage collected from U-Card system • Had conducted surveys exploring how students felt about the facilities and the benefits students perceived from using them • Lacked ability to show impact on educational outcomes without assistance • Approached Institutional Research for help
  14. 14. Project Background •Social Interaction •Tintoʼs Theory of Student Departure •Braxton & Hirschy: Communal Potential
  15. 15. Project Background CRF – High Communal Potential Social Interaction Social Integration Persistence Academic Success
  16. 16. Analysis and Preliminary Findings •In 2004, Rec Sports provided OIR with three years of facility usage data •OIR connected that data with retention information, and found a positive relationship •Needed a multivariate model to rule out spurious relationship
  17. 17. Research Questions Is CRF usage associated with better academic performance and increased likelihood of being retained and graduating, above and beyond other important predictors of academic success? First-term GPA First-year retention Graduating within 5 years
  18. 18. Influence of Visits to a CRF on Probability of Returning for a Second Year # of First-term Visits to Campus Recreation Facilities
  19. 19. Significance of collaboration • The story is that campus recreational facilities usage is associated with higher retention, even after controlling for many other factors • Benefits of partnership – Recreational Sports strengthened their case for investment in facilities – Institutional Research gained more understanding of student retention and demonstrated the value of their modeling in making decisions
  20. 20. Roles data played in collaboration • Initial descriptive data and bivariate analysis demonstrated showed promise of effort and garnered support from leadership • Each office needed data and knowledge only the other office could provide • Analysis validated the understanding of how facility usage connected to student success • Institutional metrics identified outcomes of interest
  21. 21. Try this at home! • What stories do you and your colleagues tell and hear about why things happen in your area? • What do those stories tell you about the important concepts to measure? • How could you explore if those experiences can be generalized?
  22. 22. Keep trying… • Who has the information and skills you need to access and assemble that information? – May be in local systems in your or other offices, or in institutional records – You may need to invent a way to collect it • What does your group need to accomplish, and how does that differ from your what your collaborators need? • How would you measure whether each of you is succeeding?
  23. 23. Continuing the conversation • Join the discussion on Moodle – https://moodle.umn.edu/course/view.php?id=11 30 – Will post responses to all questions submitted on the index cards – Ask additional questions on Moodle or contribute to the discussion • Contact Peter: radcl002@umn.edu
  24. 24. Appendix
  25. 25. Metrics and Agreement on Goals • At November 2009 Board of Regents Meeting, the President, Provost, and Executive Director of Planning and Analysis presented a system-wide framework for metrics and key indicators • Framework provides a structure to which activity at all levels of the organization can be aligned
  26. 26. Goals: Mission and Capacity Exceptional Extraordinary Education – Recruit, educate, challenge, and Students graduate outstanding students who become highly motivated lifelong learners, leaders, and global citizens. Transforming the U Pillars Breakthrough Research – Explore new ideas and Mission Exceptional Innovation breakthrough discoveries that address the critical problems and needs of the state, nation, and world. Dynamic Outreach and Service – Connect the Universityʼs Faculty and Staff academic research and teaching as an engine of positive Exceptional change for addressing societyʼs most complex challenges. World-Class Faculty and Staff – Engage exceptional faculty and staff who are innovative, energetic, and dedicated Capacity to the highest standards of excellence. Organization Exceptional Outstanding Organization – Be responsible stewards of resources, focused on service, driven by performance, and known as the best among peers. 26
  27. 27. Strategies and Key Indicators: Education Extraordinary Education – Recruit, educate, challenge, and Goal graduate outstanding students who become highly motivated lifelong learners, leaders, and global citizens. Recruit highly prepared Incoming student preparation students from diverse Student diversity populations Graduation and retention rates Key Indicators Challenge, educate and Placement of graduates Strategies graduate students Student learning outcomes (in process) Ensure affordable Internal support for scholarships access for students of all backgrounds Average net cost for students Student engagement Develop lifelong learners, leaders and global Participation in study abroad and citizens international experiences Student development outcomes (in process) 27
  28. 28. Strategies and Key Indicators: Research Breakthrough Research – Explore new ideas and breakthrough Goal discoveries that address the critical problems and needs of the state, nation, and world. Highly cited research publications Foster an environment of creativity that encourages National academy members and other evolution of dynamic faculty awards fields of inquiry Key Indicators Major research awards, research center Strategies awards and centers of excellence Research expenditures and competitive ranking Develop innovative strategies to accelerate Technology disclosures, licenses and the efficient and effective startups transfer of knowledge for the public good 28
  29. 29. Strategies and Key Indicators: Outreach Dynamic Outreach and Service – Connect the Universityʼs Goal academic research and teaching as an engine of positive change for addressing societyʼs most complex challenges. Promote and secure the Longitudinal changes in communities advancement of the most where the University is actively engaged challenged communities (in development) Key Indicators Strategies Build community partnerships that Active partnerships and assessments of enhance the value and impact (in development) impact of the Universityʼs research and teaching Be a knowledge, information, and human Faculty, staff, and student engagement and capital resource for the community service (in development) betterment of the state, nation, and world 29
  30. 30. Strategies and Key Indicators: Faculty and Staff World-Class Faculty and Staff – Engage exceptional faculty and Goal staff who are innovative, energetic, and dedicated to the highest standards of excellence. Recruit and place Quality of incoming faculty and staff talented and diverse faculty and staff to best Faculty and staff diversity meet organizational Key Indicators needs Supervisor and departmental support satisfaction Strategies Mentor, develop, and train faculty and staff to Employee training and development index (in optimize performance development) Recognize and reward Faculty and staff salary and total outstanding faculty and compensation staff Faculty and staff awards and distinctions Engage and retain Employee engagement index outstanding faculty and staff Employee retention30 index
  31. 31. Strategies and Key Indicators: Organization Outstanding Organization – Be responsible stewards of resources, Goal focused on service, driven by performance, and known as the best among peers. Bond rating: resources and leverage; liquidity Ensure the Universityʼs and operating cushion financial strength Private giving and endowment Be responsible stewards of Carbon footprint resources Facilities Condition Needs Index (FCNI) Strategies Promote performance, Key Indicators External awards to units for performance, process improvement, and quality, and innovation (in development) effective practice Research proposals and awards Foster peer-leading Technology commercialization agreements research competitiveness, productivity, and impact Research space productivity Ensure a safe and secure Workplace injuries environment for the Crime and perceptions of safety University community 31 Faculty and staff satisfaction with support Focus on quality service services
  32. 32. Integrated Metrics Framework University-Wide U-Wide Goals U-Wide Strategies Unit-Level U-Wide Unit-Level Key Indicators Goals Unit-Level Strategies Unit-Level Measures 32
  33. 33. Criteria for decision-making 1. Centrality to mission 2. Quality, productivity, and impact 3. Uniqueness and comparative advantage 4. Enhancement of academic synergies 5. Demand and resources 6. Efficiency and effectiveness 7. Development and leveraging of resources http://www1.umn.edu/systemwide/strategic_positioning/decision.html
  34. 34. Useful URLs • Institutional Research – http://www.irr.umn.edu/ • Management Reporting – http://www.umreports.umn.edu/ • Enterprise Financial System – https://www1.umn.edu/cco/PeopleSoft/v8/fs.shtml • Data Warehouse – https://dw.umn.edu/index.asp
  35. 35. More useful URLs • Accountable to U – http://www.academic.umn.edu/accountability • OVPR Levels and Trends – http://www.oar.umn.edu/trends/index.cfm • Office of Measurement Services – http://oms.umn.edu/oms/index.php • Office of Classroom Management – http://www.classroom.umn.edu

×