Perfecting test with Questionmark


Published on

This paper was presented in Questionmark User Conference in Miami in March 2010

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • All of Dell’s legacy strengths give us permission to play, have informed what services are offered/how services are performed today
  • Perfecting test with Questionmark

    1. 1. Perfecting the Test through question analysis<br />DELL CONFIDENTIAL<br />
    2. 2. About the PresenterNeelov Kar<br />Program Owner for Project Management courses <br />Led the Evaluation Team and introduced QM in Perot Systems<br />Account Manager in Perot Systems since 1998<br />Worked in other organizations in senior positions and has a total of 22 years of IT experience <br />PMP, ISO 9000 Lead Assessor, Six Sigma and Certified Quality Analyst<br />Certified Instructor for Orientation, Leadership Workshop and Project Management courses. <br />Graduation from Friendship University, Moscow and MBA from BIT Mesra<br />DELL CONFIDENTIAL<br />
    3. 3. WELCOME TO DELL SERVICES<br />“Dell Services is an one stop shop for all IT Services delivering its services worldwide in 183 countries with an employee strength of 41000”. <br />DELL CONFIDENTIAL<br />3<br />
    4. 4. Building on years of successful collaboration, together we create Dell Services<br />Serving the public sector, large enterprises and small- and medium-size businesses all over the world<br />More than 41,000 experienced and skilled technology and business-services professionals<br />Approximately $8B revenues from enhanced services and support<br />Best-in-class suite of intelligent, end-to-end services and business solutions to reduce IT complexity and lower costs<br />4<br />Global Presence<br />DELL CONFIDENTIAL<br />
    5. 5. 5<br />The Integrated Services PortfolioFrom product support to business solutions, we can help you achieve your business outcomes<br />DELL CONFIDENTIAL<br />
    6. 6. Dell Services: Foundation of Technology With Industry Capabilities<br />Healthcare<br />Government<br />Commercial<br /><ul><li>Hospitals
    7. 7. Physicians
    8. 8. Health Insurance
    9. 9. Life Sciences
    10. 10. Pharmaceutical
    11. 11. Retail Banking
    12. 12. Payments and Clearing
    13. 13. Insurance
    14. 14. Capital Markets
    15. 15. Manufacturing
    16. 16. Freight and Logistics
    17. 17. Telecommunications
    18. 18. Travel and Transport
    19. 19. Energy
    20. 20. Retail
    21. 21. Technology
    22. 22. Defense
    23. 23. Homeland Security
    24. 24. Intelligence
    25. 25. Civilian
    26. 26. State and Local
    27. 27. Education</li></ul>6<br />DELL CONFIDENTIAL<br />
    28. 28. 7<br />Helping You SucceedSetting Dell apart through our breadth of portfolio, ecosystem of partners and systematic approach to innovation<br />Trusted by you:<br />Over 2 million desktops managed<br />12.8 million desktops and notebooks supported<br />1.4 million servers and storage systems supported<br />Over 5,000 SaaS customers<br />Ranked #1 for Professional Services in North America Healthcare*<br />Ranked #1 in Customer Service**<br /><ul><li>Using customizable technology to deliver business-focused solutions
    29. 29. Leveraging internet technology, we will be at the forefront of cloud and as-a-service delivery
    30. 30. Using standardization to drive down costs and drive up efficiency, from the desktop to the data center
    31. 31. Evolving with the industry to bring you the newest technology and more flexible consumption models</li></ul>*By revenue from Gartner’s Industry IT Services Worldwide Market Share Database; August 14, 2009 **TBR's Q209 Service and Support Customer Satisfaction Study <br />DELL CONFIDENTIAL<br />
    32. 32. Responsible for Design, Development & Delivery of Project Management and Leadership courses for Dell Services across the globe<br />ISO 9000 certified since 2007<br />Awarded “2009 Best Place to Learn” by ASTD Dallas Chapter<br />We use<br />QM for Level 2 Assessment<br />SurveyMonkey for Level 1 feedback<br />Adobe Captivate for CBL development<br />PeopleSoft for Training data management<br />Saba as LMS <br />DELL CONFIDENTIAL<br />8<br />Learning & Development <br />
    33. 33. ADDIE Model<br />9<br />DELL CONFIDENTIAL<br />
    34. 34. Learn to use the different Perception Report to comprehensively analyze the assessment<br />Understand how item analysis Report helps in identifying the badly constructed questions <br />Learn how Dell Services have done the analysis to refine the test<br />DELL CONFIDENTIAL<br />10<br />Learning Objectives<br />
    35. 35. P3MM Fundamentals Course<br />3 Series with 17 VILT Modules<br />Exam after each series<br />Launched in December 2008<br />Cohort 1, 2 & 3 as Pilot, participants were experienced project managers<br />Used Cohort 1, 2 & 3 Exam results and participant feedback to analyze the test questions<br />DELL CONFIDENTIAL<br />11<br />The Case<br />
    36. 36. Done through QM<br />Question Statistics Report<br />Item Analysis Report<br />Indices to look for:<br />Discrimination Index<br />Item Difficulty Level<br />Combination of SurveyMonkey and QM<br />Correlation between Test Score and Demography (Role, Account, BU etc)<br />QM and Course Delivery<br />Correlation between Test Score and Course Content <br />Correlation between Test Score and Instructor<br />Test Analysis<br />DELL CONFIDENTIAL<br />
    37. 37. Completed Schedule 29 <br />80% or over 12 41%<br />Average time 00:20:05 <br />Completed Cost 21 <br />80% or over 4 19%<br />Average time 00:19:43 <br />Completed Scope 21 <br />80% or over 17 80%<br />Average time 00:09:54 <br />Completed All 20 <br />80% or over 10 50%<br />Average time 00:35:57 <br /> Test Results<br />Raised a red flag that there is some problem with Cost Test<br />DELL CONFIDENTIAL<br />
    38. 38. Analyzed Distribution of Answers<br />Categories<br />Majority Correct<br />Polarized<br />Scattered<br />Majority Wrong<br />DELL CONFIDENTIAL<br />
    39. 39. DELL CONFIDENTIAL<br />15<br />Test Analysis Report<br />
    40. 40. DELL CONFIDENTIAL<br />16<br />Question Statistics Report<br />Scattered<br />
    41. 41. DELL CONFIDENTIAL<br />17<br />Question Statistics Report<br />Polarized<br />
    42. 42. DELL CONFIDENTIAL<br />18<br />Item Analysis Report<br />
    43. 43. DELL CONFIDENTIAL<br />19<br />Item Analysis Report<br />
    44. 44. DELL CONFIDENTIAL<br />20<br />Item Analysis Report<br />
    45. 45. DELL CONFIDENTIAL<br />21<br />Item Analysis Report<br />
    46. 46. Analyzing the Question and Feedback<br />DELL CONFIDENTIAL<br />
    47. 47. DELL CONFIDENTIAL<br />23<br />Coaching Report<br />
    48. 48. DELL CONFIDENTIAL<br />24<br />Assessment Overview Report<br />
    49. 49. DELL CONFIDENTIAL<br />25<br />P3MM Fundamentals Course has been stabilized after February 2009 and running smoothly since then………<br />
    50. 50. Thank You<br />26<br />