This document summarizes a research paper that proposes new methods for evaluating student learning achievement and automatically constructing concept maps using fuzzy membership functions and fuzzy rules. It introduces three new methods: 1) Automatically establishing corresponding fuzzy membership functions between loose, strict, and general scores. 2) A new method for student learning achievement evaluation using fuzzy membership functions and fuzzy rules that considers question difficulty, complexity, and importance. 3) A new method for automatically constructing concept maps based on evaluation results and fuzzy reasoning that determines the relevance between concepts. The document provides details on the proposed methods and examples of their application in student evaluation and concept map construction.

1. 利用模糊歸屬函數及模糊規則以評量學生學習成效及建立概念圖之新方法 台灣科技大學／資訊工程研究所 研究生:白世銘 指導教授:陳錫明博士 簡報報告製作者: 葉心寬 （不分系三戊 b9630450）

5. Evaluating Students’ Learning Achievement Using Fuzzy Membership Functions and Fuzzy Rules

6. Two-Phase Concept Map Construction Algorithm

8. Example Rules R1：IF the grade of a strict-type teacher is T1 THEN the grade of a normal-type teacher is F1 R2：IF the grade of a lenient-type teacher is T2 THEN the grade of a normal-type teacher is F2 IF the grade obtained from the lenient-type teacher is 8 then we can get the normal-type score 6.4 𝜇1 (x1)=1100x1 2 Grade membership function of the lenient-type grades, where 0 <x1<10 f1y1=110y1 Grade membership function of the normal-type grades, where 0 <y1<10 𝜇1 (8)=1100 82=0.64 => f1y1=110y1=0.64 => y1=6.4

9. The fuzzy reasoning process 1.0 1.0 𝜇1 (x1) f1y1 0.64 0.5 0.5 8 0 0 5 10 5 10 6.4 lenient-type teacher normal-type teacher

10. But… This method does not present how to construct the grade membership functions of each type grades given by teacher Can not properly be used in real grading systems to solve the subjective judging problem of teachers for fuzzy grading system

11. Weon-and-Kim’s method for education Pointed out that the chief aimshould consider the element for students’ answerscripts evaluation Difficulty Importance Complexity Using fuzzy sets

12. Computation of Response Accuracy With Limited Time Only COR(Pi)=COR(Pi1,Pi2,…,Pim ) Pi denotes the i th question in P on answerscript Pi has several sub-questions Pi1,Pi2,…, Pim 𝜇Pij : the membership grade of the response accuracy of the jth sub-question of Pi 1 means correct、0 means false 𝜇Tij : the membership grade of time that is solve the problem Pij i=1nPi,j=1m𝜇Pij×𝜇Tij COR(Pi)= if v <α if α<γ<𝛽 if 𝛽<v<γ if v>γ 𝜇Tij=1,1−2v−αγ−α 2 2v−γγ−α 2 0 α: the permitted lower limit solving time γ: the permitted upper limit solving time 𝛽=α+γ2

13. S-shaped membership function 𝜇Tij(U) Tij 1.0 0.5 γ 1.0 α 0 𝛽

17. Fuzzy apply… Fuzzy dilation method is used to increase the weight factor Fuzzy concentration method is used to decrease the weight factor

18. Fuzzy membership function Weon and Kim used the following membership functions to evaluate the learning achievement through the response accuracy and the normalized values: “VERY GOOD” = x2, if x=1 “GOOD” = x , if 0<x≤1 “MEDIUM”=2x, if 0<x≤0.5−2x+2,if 0.5<x<1 “BAD”=−x,if 0<x<1 “VERY BAD”=(−x+1)2,if x=0

19. Fuzzy membership function Very Bad Very Good Medium 1.0 Bad Good 0.5 1.0 0 0.5

20. Fuzzy membership function Each question is normalized and the normalized response accuracy is linguistically evaluated by one of the previously defined membership functions after the response accuracy is computed May belong to more than two membership functions However… Because the “difficulty” is a very subjective parameter to adjust the scores of students is not appropriate

21. New method A= Q1⋮QmS1⋯Sna11⋯a1n⋮⋱⋮am1⋯amn Si : students Qi : questions aij: the accuracy rate of the jth student on the jth question tij: the answer-time-rate of the jth student on the jth question T= Q1⋮QmS1⋯Snt11⋯t1n⋮⋱⋮tm1⋯tmn G= Q1⋮Qmg1⋮gm G : a grade matrix storing the score of each question of a student IM= Q1⋮QmImS1⋯ImS5im11⋯im15⋮⋱⋮imm1⋯imm5 imij : the degree of membership of the degree of importance of the ith question Qi belonging to the importance level ImSj C= Q1⋮QmCS1⋯CS5c11⋯c15⋮⋱⋮cm1⋯cm5 imij : the degree of membership of the degree of complexity of the ith question Qi belonging to the importance level ImSj

22. First… Based on the accuracy rate matrix A and the grade Matrix G We can calculate total score of each student And then rank them properly But … If there are any students having the same total grade The proposed method can rank them properly

23. THE PROPOSED METHOD Step 1 : Calculate the average accuracy rate and the average answer-time rate Fuzzify them based on the following five fuzzy sets Calculate their membership grades belonging to each fuzzy set

24. THE PROPOSED METHOD Step 1 : More or less high More or less low high low medium FA= Q1⋮QmFAS1⋯FAS5fa11⋯fa15⋮⋱⋮fam1⋯fam5 More or less long More or less short long medium short 1.0 0.8 FT= Q1⋮QmFTS1⋯FTS5ft11⋯ft15⋮⋱⋮ftm1⋯ftm5 0.6 0.4 0.2 X ０.8 ０.9 1.0 ０.2 ０3 ０4 ０5 ０6 ０.7 ０.1 ０

25. THE PROPOSED METHOD Step 2 : Based on the fuzzy grade matrices FA,FT and fuzzy rules Perform the fuzzy reasoning to evaluate the difficulty of each question D= Q1⋮QmDS1⋯DS5d11⋯d15⋮⋱⋮dm1⋯dm5

26. THE PROPOSED METHOD Step 3 : Based on the difficulty matrices and the complexity matrices Perform the fuzzy reasoning to evaluate the answer-cost of each question CO= Q1⋮QmCoS1⋯CoS5ac11⋯ac15⋮⋱⋮acm1⋯acm5

27. THE PROPOSED METHOD Step 4 : Based on the answer-cost matrices and the importance matrices Perform the fuzzy reasoning to evaluate the adjustment value of each question V= Q1⋮QmVS1⋯VS5v11⋯v15⋮⋱⋮vm1⋯vm5

28. THE PROPOSED METHOD Step 5 : Step 6 : Assume there are k students having the same total grade We construct a new grade matrix EA for these equal grade students EA= Q1⋮QmES1⋯ESkea11⋯ea1k⋮⋱⋮eam1⋯eamk A= Q1⋮QmS1⋯Sna11⋯a1n⋮⋱⋮am1⋯amn aij: the accuracy rate of the jth student on the jthquestion eaij: the accuracy rate of the jth student ESjwith respect to the ith question Qi Based on the adjustment value Calculate the sum of difference for the student with the same total grade

30. Ten students : S1,S2,…,S10 G= Q1Q2Q3Q4Q51015202530 𝑨= the total score : TSj=i=1maij×gi Ex.0.59*10+0.01*15+0.77*20+0.73*25+0.93*30=67.6 S9>S1> S2> S8> S4= S5= S10> S6> S7> S3

31. STEP 1 AvgA1=0.59+0.35+1+0.66+0.11+0.08+0.84+0.23+0.4+0.2410=0.45 AvgT1=0.7+0.4+0.1+1+0.7+0.2+0.7+0.6+0.4+0.910=0.57 AvgA2=0.01+0.27+0.14+0.04+0.88+0.16+0.04+0.22+0.81+0.5310=0.31 AvgT2=1+0+0.9+0.3+1+0.3+0.2+0.8+0+0.310=0.48 AvgA3=0.77+0.69+0.97+0.71+0.17+0.86+0.87+0.42+0.91+0.7410=0.711 AvgT3=0+0.1+0+0.1+0.9+1+0.2+0.3+0.1+0.410=0.31 T= AvgA4=0.73+0.72+0.18+0.16+0.5+0.02+0.32+0.92+0.9+0.2510=0.47 AvgT4=0.2+0.1+0+1+1+0.3+0.4+0.8+0.7+0.510=0.5 AvgA5=0.93+0.49+0.08+0.81+0.65+0.93+0.39+0.51+0.97+0.6110=0.637 AvgT5=0.93+0.49+0.08+0.81+0.65+0.93+0.39+0.51+0.97+0.6110=0.637

32. Step 1 FA = More or less high More or less low high FT = low medium More or less long More or less short long medium short 1.0 0.8 0.6 0.4 0.2 X ０.8 ０.9 1.0 ０.2 ０3 ０4 ０5 ０6 ０.7 ０.1 ０

33. STEP 2 D =

34. STEP 3 CO = C =

35. STEP 4 IM = V =

36. STEP 4 advi=0.1vi1+0.3vi2+0.5vi3+0.7vi4+0.9vi50.1+0.3+0.5+0.7+0.9 adv1=0.1×0.38+0.3×0.38+0.5×0.66+0.7×0.88+0.9×0.750.1+0.3+0.5+0.7+0.9=0.71 centroid method adv2=0.1×0.36+0.3×0.66+0.5×0.66+0.7×0.76+0.9×0.430.1+0.3+0.5+0.7+0.9=0.59 adv3=0.1×0.33+0.3×0.43+0.5×0.76+0.7×0.86+0.9×0.80.1+0.3+0.5+0.7+0.9=0.75 STEP 5 adv4=0.1×0.33+0.3×0.88+0.5×0.68+0.7×0.4+0.9×0.320.1+0.3+0.5+0.7+0.9=0.51 adv5=0.1×0.34+0.3×0.8+0.5×0.68+0.7×0.4+0.9×0.320.1+0.3+0.5+0.7+0.9=0.55 𝑨= EA =

37. SODj=p=1ki=1meaij−eaip×gi×0.5+advi SOD1=0.66−0.11+0.66−0.24×10×(0.5+0.71)+ 0.04−0.88+0.04−0.53×15×0.5+0.59+ 0.71−0.17+0.71−0.74×20×0.5+0.75+ 0.16−0.5+0.16−0.25×25×0.5+0.51+ 0.81−0.65+0.81−0.61×30×0.5+0.55 =3.15 SOD1=3.15 SOD2=−5.3 SOD3=2.15 S9>S1> S2> S8> 𝑺𝟒>𝑺𝟏𝟎>𝑺𝟓 > S6> S7> S3 Step 6

38. Two-Phase Concept Map Construction Algorithm Sue et al. presented this algorithm to automatically construct a concept map of a course by learners’ historical testing records Phase 1 consists of three steps Phase 2 consists of five steps

39. Phase 1 Step 1 : Fuzzify learners’ historical testing records into fuzzy sets based on the fuzzy sets High Low Middle 1.0 0.5 60% 70% 100% 0 40% 20% 10%

40. Phase 1 Step 2 : Sort the total scores of the students in a descending order sequence Divide them into the “High” ,”Middle” and ”Low” groups, respectively, where each of them has 1/3 students Compute the degree of discrimination Diof each test item Compute the degree of difficulty Pi of each test item Delete the test items which have a low discrimination ( < 0.5) Step 3 : Perform fuzzy data mining to obtain fuzzy association rules Di=PiH+PiL PiH=RiHNiH RiH: the summation of the fuzzy grads on test i of each student in the “high” group NiH: the number of students in the “high” group Pi=ML−PiH+PiL2

41. Phase 2 Step 1 : For each association rule, insert the test item and their relation edges into the concept graph Step 2 : Detect whether cycles exist ,If a cycle is found, then remove the edge with a lower confidence from the cycle until no cycle exists Step 3 : Delete independent nodes without edges Step 4 : For each test item node, insert nodes corresponding to learning concepts according to the test item-concept table Step 5 : For each edge between test items, join two connected concept sets for generating the concept relationship edge between concepts by using the join principle to replace the original concept sets

42. But … Uses fuzzy data mining techniques to obtain fuzzy association rules It’s not efficient enough

43. New method Matrix : G= s1⋮smQ1⋯Qng11⋯g1n⋮⋱⋮gm1⋯gmn QC= Q1⋮QnC1⋯Cpgc11⋯gc1p⋮⋱⋮gcn1⋯gcnp Si: students Qi : questions gij: the grade of the ith student with respect to the jth question gij∈ 0,markj markj: the mark allotted to the jth question gcij=1 : the ith question including the jth concept gcij=0 : the ith question not including the jth concept

44. THE PROPOSED METHOD Step 1 : Step 2 : Calculate Score Testing Records Average Highest Lowest Delete the questions that have no discrimination Calculate the entropy of the students’ testing records

45. THE PROPOSED METHOD Step 3 : Step 4: For each grade , calculate the relative percentage Based on the relative percentage fuzzify each one into a fuzzy set lower than the average Near equal to the average Higher than the average Much higher than the average Much lower than the average 1 30% 50% 100% -100% -80% -50% -30% 0% 80%

47. More or less low (ML)

48. Medium (M)

49. More or less high (MH)

51. FUZZY Relative grade table ->Relative relationship degree table

52. THE PROPOSED METHOD Step 6: Construct the concept map 6.1: Calculate the summation of the percentage of each question 6.2: Transform the graph of the relationships of questions into the graph of the relationships of concepts based on the question-concepts matrix QC 6.3: Merge more arrow into one arrow associated with the derived averaging value

54. Five concepts : A,B,C,D,E

55. Ten students : S1,S2,…,S10 G = QC=

56. Step 1 HighQ1=max12,12,12,2,2,2,20,10,10,10=20 AvgQ1=12+12+12+2+2+2+20+10+10+1010=9.2 HighQ2=max18,14,16,8,8,10,5,6,5,3=18 HighQ3=max20,18,14,12,12,8,5,6,5,4=20 AvgQ2=18+14+16+8+8+10+5+6+5+310=9.3 HighQ4=max20,3,4,6,2,2,4,1,1,0=20 HighQ5=max7,7,7,20,12,20,1,5,5,5=20 AvgQ3=20+18+14+12+12+8+5+6+5+410=10.4 AvgQ4=20+3+4+6+2+2+4+1+1+010=4.3 LowQ1=min12,12,12,2,2,2,20,10,10,10=2 LowQ2=min18,14,16,8,8,10,5,6,5,3=3 AvgQ5=7+7+7+20+12+20+1+5+5+510=8.9 LowQ3=min20,18,14,12,12,8,5,6,5,4=4 LowQ4=min20,3,4,6,2,2,4,1,1,0=0 LowQ5=min7,7,7,20,12,20,1,5,5,5=1

57. Step 2 the balance degree of Qj= AvgQj−LowQjHighQj−LowQj EQj= i=1mgij−AvgQj/mHighQj−LowQj+1/2 the balance degree’s range : 35%～65% the entropy range : >40 IF the balance degree and the entropy of a question are both not in the range described above We say the question does not have the discrimination capability -> no discrimination -> delete the jth question

58. Step 2

59. Step 3 RP= s1⋮smQ1⋯Qnrp11⋯rp1n⋮⋱⋮rpm1⋯rpmn G= s1⋮smQ1⋯Qng11⋯g1n⋮⋱⋮gm1⋯gmn RP = rpij=gij−AvgQjHighQj−AvgQj×100% , if gij−AvgQj≥0gij−AvgQjLowQj−AvgQj×−100% , if gij−AvgQj≤0

60. STEP 4 Fuzzy Relative Grade Table for The Students’ Testing Records

61. Step 5 Get the relative relationship degree table Calculate the relationship degreeTRDjkbetween any two adjacent questions Qj and Qk TRDjk=i=1mLjki×0+MLjki×0.2+MLjki×0.5+MHjki×0.8+Hjki×1m More orless high More or less low Low Medium High 1.0 0 Membership functions of each quiz’s grade 0.2 0.8 0.5 1

62. Step 6 SPj=j=1mgij/markjm IF SPj>SPk, then we add an arrow from Qj to Qkassociated with TRDjk into the constructed graph IF SPj<SPk, then we add an arrow from Qkto Qjassociated with TRDjk into the constructed graph 0.63 Q3 Q5 Q1 Q2 0.9 0.64 0.64 0.55 The constructed questions-relationship graph 0.45

63. STEP 6 After transforming the derived questions-relationship graph into the relationships between concepts based on the mapping matrix QC 0.635 0.63 0.615 B 0.45 A 0.635=(0.64+0.63)/20.615=(0.63+0.55+0.63+0.64)/4 C D 0.635 0.9 0.45 E 0.63 0.635 The constructed concept map (concepts-relationship graph)