• Save
ITEM ANALYSIS
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

ITEM ANALYSIS

  • 726 views
Uploaded on

UMT, Test Construction 2, Test Construction II, Item Analysis, Test Analysis, Test Construction, Discrimination Index, Difficulty Index, Waqas A. Khan, Prof. Dr. Abdul Hameed, Abdul Hameed, Dr.......

UMT, Test Construction 2, Test Construction II, Item Analysis, Test Analysis, Test Construction, Discrimination Index, Difficulty Index, Waqas A. Khan, Prof. Dr. Abdul Hameed, Abdul Hameed, Dr. Abdul Hameed, SSH, University of Management and Technology, University of Management & Technology, www.waqas.org

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
726
On Slideshare
721
From Embeds
5
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 5

http://www.waqas.org 5

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. BY: PROF. DR. ABDUL HAMEED ITEM ANALYSIS A PRACTICAL APPROACH WAQAS A. KHAN
  • 2. ABOUT THE INSTRUCTOR • Mr. Waqas A. Khan is a PhD Scholar in Educational Assessment & Evaluation at the University of Management & Technology Lahore. • He holds M.Phil degree in Quality Management, M.Ed and diploma in “Practical Project Management” from Ateneo De Manila University Philippines. He is a Certified Lead Auditor in ISO 9001:2008 Quality Management System. • Currently he is working as Director, Hazara Public School & College Changa Manga. For Details: www.waqas.org | www.hazara.edu.pk
  • 3. ITEM ANALYSIS: WHY? Are the items reliable? Do they measure consistently? What items are most difficult to answer correctly? What items are easy? Are there any poor performing items that need to be discarded?
  • 4. OBJECTIVES 1. Assess quality of the distractors. 2. Assess difficulty of the items. 3. Assess how well an item differentiates between high and low performers. 4. Assess Reliability of the test.
  • 5. TEST DATA ITEM ANALYSIS Paper English Class 9th Boys TOTAL QUESTIONS: 19 TOTAL STUDENTS: 39 Paper Syllabus: 1st Half Book HAZARA PUBLIC SCHOOL & COLLEGE CHANGA MANGA (KASUR) WWW.HAZARA.EDU.PK
  • 6. STEP 1 Students - Questions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 1 C B A B A B D C C C A B C B B B C A 19 B 2 C C B C C B D C A A C C C B B B C C C 3 C B A C C B D C A C C D C B B C B C C 4 B B A A B C D C A A A A B A B A A D 5 C B B B D B D C C A A D C C B B D 6 C B B C D B D C B C A D B A B B C C C 7 C B B D B D C C A D A A B B C B C 8 C B A B C B D C D C A D A B B B C C D 9 A C D C C B D C D C A B C A B B C C D 10 D B B C A B D C D A B B C D C B A C B 11 C B B C C B D C A D C A C A B C C B A 12 C B B B A B D C D D A D D C B B C B A 13 C B A B A B B C D D A D D B B B C A A 14 C D B B D B D C A A D A D C B B C B C C 15 B D A A C B B C C A B C C B A C B B A 16 D C D A D A D C B A D A C A A B A C C 17 C B C A D B D C D A A D B B C A C C D 18 C B A D C B D C A D A D B A C A B 19 C B A B C B D C C C A D C B B B C D 20 C B B B C B D C A C A D B D B A B C D 21 C B A A D B D C D A A D D C B B C C A 22 C B B B C B D C A C C D A A B B C A C 23 C B A A C B B C A C A B A D B C D C D 24 C B C C B D C D A C C A C D B C C B 25 C B B A B D C C D A D A C D C C A B 26 B B D C A B C C B A A B A B B D C D 27 C B D C C B C C B C C D B C A 28 B B B A A C A B C C A C B B C B B D C 29 C B A C C B D C A C C D C A B C B D C 30 B A D B A C A D B C A C A A B B C D C 31 C B D D B D C A C 32 B D D A C B D C D A C A B B B C A C B 33 C B A B A B D C D C A B C C B B C D 34 C B B C D B D C D C A D B A B B C B 35 C B B B A B D C D C A D A A B B C A D 36 C B B C D B D C B C A D B B B B C B C 37 C B B C D B D C A C A D C A B B C B C 38 C B A C C B D C D C A D D B B B C A A 39 C B B C C D D C A C C D C A B C C C C B C C In Excel Sheet write Question Numbers in Columns and Student Numbers in Rows. Fill the response of each student against each question. i-e (A,B,C or D)
  • 7. STEP 2 A B C D 1 1 12 7 10 1 1 1 11 13 25 7 9 14 3 3 4 8 6 5 31 15 14 0 36 3 0 6 2 2 7 8 11 31 26 5 8 6 30 3 1 15 17 1 4 36 6 19 9 4 15 9 2 9 26 17 16 2 3 6 2 12 1 31 2 13 5 2 21 5 3 2 0 4 3 9 In this step we shall check that against each question how many A, B, C and D have been ticked by the students. For this the formula is = COUNTIF(C3:C41, "A") • C3 and C41 is the range of rows in which we have entered the response data. You can change this according to your range of data. • “A” will be changed to “B”, “C” and “D” accordingly.
  • 8. STEP 3 A 2.564 2.564 30.77 17.95 25.64 2.564 2.564 2.564 28.21 33.33 64.1 17.95 23.08 35.9 7.692 7.692 10.26 20.51 15.38 B 12.82 79.49 38.46 35.9 C D 0 92.31 7.692 0 15.38 5.128 5.128 17.95 20.51 28.21 79.49 66.67 12.82 20.51 15.38 76.92 7.692 2.564 38.46 43.59 2.564 10.26 92.31 15.38 48.72 23.08 10.26 38.46 23.08 5.128 23.08 66.67 43.59 41.03 5.128 7.692 15.38 5.128 30.77 2.564 79.49 5.128 33.33 12.82 5.128 53.85 12.82 7.692 5.128 0 10.26 7.692 23.08 We will now find out the percentage of A,B,C and D responses against each question. In this example, total students are 39. So we shall divide each cell with 39 and multiply it with 100 to find the percentage responses. For this the formula is =(C47/39)*100 • C47 is the cell address. 39 is the number of total students and 100 is for %. Cross Check 97.44 97.44 87.18 97.44 100 100 100 100 92.31 100 97.44 100 94.87 94.87 97.44 97.44 100 92.31 94.87 • For cross Checking that we are going right, we can add the four columns and our result shall be 100 or very near to it (EXP)
  • 9. STEP 4 KEY: C B B C A B D C D C A D A A B B C B C • Now we shall put the right answers in one row. “KEY” 1 1 1 0 0 1 1 1 1 0 1 1 0 0 0 1 1 1 0 0 11 2 1 1 0 1 1 0 1 1 0 0 1 1 1 1 1 1 0 0 0 1 0 0 0 1 0 0 0 0 1 1 1 0 1 0 0 0 1 1 10 0 1 1 1 1 0 0 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 0 1 0 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 1 0 1 0 1 1 1 1 1 1 1 0 1 1 0 0 0 1 1 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 1 1 1 1 0 1 0 0 1 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 1 0 0 0 0 1 0 1 1 1 1 1 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0 1 0 1 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 0 1 1 0 0 0 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 0 0 0 0 1 1 1 0 1 1 0 0 0 1 0 0 0 1 0 0 1 0 0 0 0 0 0 0 1 1 1 1 0 0 1 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 0 1 1 0 0 0 0 0 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 0 0 0 1 1 1 1 1 0 1 0 1 1 1 0 0 1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 0 0 1 0 0 0 1 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 1 1 0 1 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 1 1 0 1 1 0 0 0 1 1 0 1 0 1 0 1 1 1 1 1 0 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 0 0 1 0 1 1 0 1 0 1 0 1 0 1 0 0 1 1 1 1 1 1 0 0 0 1 1 1 1 0 1 1 1 1 0 0 1 1 0 0 1 1 0 1 1 0 1 0 0 1 1 0 1 1 1 1 1 1 1 0 0 0 1 0 0 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 1 1 1 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 0 1 1 1 1 0 0 0 1 0 1 1 0 1 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 10 6 11 15 15 13 11 9 11 14 11 10 3 5 9 9 10 10 11 14 8 10 10 9 10 5 11 9 6 5 12 17 16 15 16 13 12 After putting in the “KEY” we shall now ask the computer to mark the student according to the key now. For every right answer the mark given is 1and 0 for vice versa. Formula to do this is =IF(C3="C",1,0) * We are directing computer to mark every C in Q1 as 1 and others 0. In next column replace C with the right KEY ANSWER and paste down.
  • 10. STEP 5 34 35 37 6 7 36 12 22 8 38 33 39 1 5 9 11 13 21 29 2 3 14 19 20 24 25 27 10 17 18 26 30 23 4 31 16 28 32 15 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 0 1 0 1 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 0 0 0 0 1 1 1 1 0 1 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 1 1 0 1 0 0 0 1 0 1 0 0 1 1 0 0 1 1 1 0 0 0 1 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 1 1 0 1 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 0 0 0 0 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 1 1 0 1 1 1 1 0 0 0 0 1 0 1 1 1 0 0 0 1 0 1 1 0 0 0 0 0 0 1 0 0 1 1 0 0 0 0 0 0 0 0 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 0 0 0 1 0 1 0 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 0 0 0 1 1 0 1 1 0 1 1 1 1 1 1 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 1 0 1 0 0 1 1 1 0 1 0 1 1 0 1 0 0 1 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 1 1 1 0 0 0 0 0 1 1 1 1 1 0 0 1 0 0 0 1 0 0 1 1 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 0 0 1 1 1 1 0 0 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 1 0 1 0 1 1 0 1 0 0 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 0 1 0 1 0 0 1 1 1 0 1 1 0 1 0 0 1 0 0 0 0 1 0 1 0 1 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 1 1 1 0 1 0 0 0 1 0 1 0 0 0 0 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 1 0 0 17 16 16 15 15 15 14 14 13 13 12 12 11 11 11 11 11 11 11 10 10 10 10 10 10 10 10 9 9 9 9 9 8 6 6 5 5 5 3 Now we shall arrange the obtained marks in “Descending Order” The purpose of doing this is that we want to arrange the obtained marks of the class on the quantum of ability i-e High Achievers (Ph) Low Achievers (Pl) This will help us in measuring difficulty and discrimination ahead.
  • 11. PH AND PL Ph Pl 1 0.3 1 0.5 0.7 0.1 0.5 0.1 0.2 0.3 1 0.8 1 0.4 1 0.7 0.5 0.1 0.9 0.2 0.9 0.5 1 0.1 0.4 0.3 0.6 0.3 1 0.6 1 0.5 1 0.3 0.5 0.1 0.6 0.3 14.8 6.5 • Now we shall find out the top 25% and bottom 25% for this the way is: Total number of students : 39, 25% of 39 is about 10. • So we will sum the obtained marks of top 10 and bottom 10 students and shall divide this sum with number of these students i-e 10. Formula to do this is. Formula Ph: =SUM(C116:C125)/10 Formula Pl: =SUM(C145:C154)/10
  • 12. DISCRIMINATION & DIFFICULTY Discrimination 0.7 0.5 0.6 0.4 -0.1 0.2 0.6 0.3 0.4 0.7 0.4 0.9 0.1 0.3 0.4 0.5 0.7 0.4 0.3 • For Discrimination value, we shall subtract Ph from Pl. Difficulty 0.769 0.795 0.385 0.385 0.256 0.923 0.795 0.923 0.333 0.487 0.641 0.538 0.231 0.359 0.795 0.667 0.667 0.205 0.41 • For Difficulty we shall add obtained marks in each question and divide it with total number of students. Formula : Discrimination: = (C160-C161) Difficulty: =SUM(C116:C154)/39
  • 13. RELIABILITY To find reliability, we shall now paste the obtained marks data into SPSS Software.
  • 14. RELIABILITY Click Analyze Then Click on Scale Click on Reliability Analysis You will find this answer Case Processing Summary N Cases Valid % 39 a Excluded Total 100.0 0 0.0 39 100.0 a. Listwise deletion based on all variables in the procedure. Reliability Statistics Cronbach 's Alpha N of Items .689 19
  • 15. ANALYSIS RELIABILITY INDEX DESIRED VALUE 0.8 and >
  • 16. DECISION Item Diff. Disc. No. Index Index 1 2 3 4 5 0.8 0.8 0.4 0.4 0.7 0.5 0.6 0.4 6 7 8 9 10 11 12 13 14 15 16 17 18 19 0.3 0.9 0.8 0.9 0.3 0.5 0.6 0.5 0.2 0.4 0.8 0.7 0.7 0.2 0.4 -0.1 0.2 0.6 0.3 0.4 0.7 0.4 0.9 0.1 0.3 0.4 0.5 0.7 0.4 0.3 Distracter Effect % A B C D Decisio n 3 3 31 18 13 79* 38* 36 77* 8 3 38* 5 8 15 5 26* 0 44 31 3 3 92* 8 3 10 3 79* 3 0 92* 5 28 33 64* 18 23* 36* 8 8 10 21 15 15 5 5 18 21 28 79* 67* 13 21* 15 15 49* 23 10 38 23 5 23 67* 44 41* 33* 13 5 54* 13 8 5 0 10 8 23 Select Select Select Select Reject (B) Reject Select Reject (B) Select Select Select Select Reject Select Select Select Select Select Select
  • 17. QUESTIONS? To download presentation and data file Visit www.waqas.org or email at wacas.khan@gmail.com