Language Assessment - Beyond Test-Alternatives Assessment by EFL LearnersEFL Learning
The concept of assemble additional measures of students—portfolios, journals, observations, self-assessments, peer-assessments, and the like—in an effort to triangulate data about students.
Language Assessment - Beyond Test-Alternatives Assessment by EFL LearnersEFL Learning
The concept of assemble additional measures of students—portfolios, journals, observations, self-assessments, peer-assessments, and the like—in an effort to triangulate data about students.
Product Syllabus : product syllabuses are those in which the focus is on the knowledge and skills which learners should gain as a result of instruction.
4.2. process syllabuses are those which focus on the learning experiences themselves.
. Synthetic syllabus: segment the target language into discrete linguistic items.
Different parts of language are taught separately.
4.4 . Analytic Syllabi: focus on the learner and his needs and on the kinds of linguistic
performance necessary to achieve those goals .
4.5. Type A: This type deals with what should be learned in a second language classroom.
4.6. Type B : Consider the question of how a second language should be learned.
Teaching Language Skill: Speaking and WritingUNY Pasca PBI-B
presented by : Musfera NV and Awaliawati W. in RBL class.
source: McDonough, J., Shaw,C., & Masuhara ,H.,
(2013) .Materials and methods in ELT. John
Wiley&Son.
Product Syllabus : product syllabuses are those in which the focus is on the knowledge and skills which learners should gain as a result of instruction.
4.2. process syllabuses are those which focus on the learning experiences themselves.
. Synthetic syllabus: segment the target language into discrete linguistic items.
Different parts of language are taught separately.
4.4 . Analytic Syllabi: focus on the learner and his needs and on the kinds of linguistic
performance necessary to achieve those goals .
4.5. Type A: This type deals with what should be learned in a second language classroom.
4.6. Type B : Consider the question of how a second language should be learned.
Teaching Language Skill: Speaking and WritingUNY Pasca PBI-B
presented by : Musfera NV and Awaliawati W. in RBL class.
source: McDonough, J., Shaw,C., & Masuhara ,H.,
(2013) .Materials and methods in ELT. John
Wiley&Son.
TEDx Manchester: AI & The Future of WorkVolker Hirsch
TEDx Manchester talk on artificial intelligence (AI) and how the ascent of AI and robotics impacts our future work environments.
The video of the talk is now also available here: https://youtu.be/dRw4d2Si8LA
Given the importance of a test, especially if it is used as a decision-making tool, of course a good test is needed. A good test must meet the characteristics (characteristics) of a good test.
An investigation into the characteristics of multiple-choice on the exam results quality in reading tests. This research only surveys teaching methods of teachers. The study focuses on seeking some difficulties met during teaching English about multiple-choice on the exam results quality in Reading tests in class and the reasons why teachers meet those ones.
Similar to Justifying the Use of an English Language Placement Test with an Assessment Use Argument (20)
How to Transform Your Classroom with AR and VRParisa Mehran
Are you looking for ways to keep your students motivated and engaged during English lessons? Using Augmented Reality (AR) and Virtual Reality (VR) is one surefire way to do so. These cutting-edge technologies are more than just gimmicks; you can use them to transform your classroom. With AR, you can bring learning to life, and with VR, you can bring the world to your classroom or take your classroom out into the world. This workshop explores how to use AR and VR to change your classroom into a whole new dimension. To fully participate in this hands-on workshop, please BYOD.
Design, Implementation, and Evaluation of an English Blended CourseParisa Mehran
This presentation outlines the iterative stages involved in designing, implementing, and evaluating a blended course of English for General Academic Purposes (EGAP) at a university in Japan, delivered in the spring semester of 2017 over a period of fifteen weeks. First, the basic Successive Approximation Model (SAM) will be introduced as the guiding instructional design model upon which the course was created. Afterward, the stages of the blended course design will be explicated with a focus upon assessing Japanese students’ English language needs and their e-learning readiness, determining the course overall goals and module learning objectives, optimizing course technologies and the availability of technical support, designing the course syllabus, materials, tasks, and activities, organizing team teaching, as well as managing formative and summative evaluation. Additionally, the way in which the iteration process has allowed for the discovery of some possibilities and problems at the early phases of the blended course design, and the refinements which were made to benefit from the affordable opportunities and to mitigate the difficulties will be discussed. Finally, the Quality Matters Higher Education Rubric and its effectiveness in raising course quality assurance will be reviewed.
“Write 4 Change”: Cultivating Autonomous, Global EFL Learners through BloggingParisa Mehran
The process approach to L2 writing has gained momentum over the past two decades, as this approach encourages autonomy, reflection, collaboration, creativity, and exploration, and enables learners to become more engaged in the writing process. Technology has also grown in popularity in ELT, and blogging for L2 writing has been recommended to promote critical reflection and collaborative interaction within socially mediated online learning platforms. The purpose of this poster presentation is to explore how the application of a collective class blog, titled “Write 4 Change” http://write4change.edublogs.org/, facilitated the L2 writing process, promoted co-construction of L2 knowledge through social interaction, and cultivated autonomy, critical thinking, and global awareness in EFL learners via writing for social change. The tech tools which were incorporated within the blog will also be introduced and demoed. For example, Popplet, a mind-mapping tool used for the planning phase, will be introduced. The use of Google Docs and Draftback for online collaborative writing and e-peer feedback, will be discussed as well.
Physically Banned yet Virtually Connected at EUROCALL2017: How Technology Ove...Parisa Mehran
In the world that we live in now, exclusion is becoming legally permissible, and technology has the power to resist by making bridges and surpassing the offline borders in the online, borderless environment. In this presentation, I will share my visa story and the details of my UK visa denials which eventually banned me from attending EUROCALL2017. Despite being denied, the EUROCALL team helped me overcome the political barriers and be part of EUROCALL2017 with the help of technology. I will discuss how I was provided with opportunities for remote participation especially through Virtually Connecting, which is an open connected learning community that aims to facilitate virtual participation in academic conferences for those who cannot be physically present at conferences. I will also talk about other Open Educational Practices occurred at EUROCALL2017, for example, tweeting and live broadcasting. I will further explain the differences I found between attending a conference onsite and online. Moreover, I will introduce the EUROCALL "Access and Inclusion" working group and the efforts that the committee is making to create more inclusive, diverse, and open educational opportunities. Finally, I would like to speak briefly about my PhD research project, titled Osaka University Global English Online (OUGEO), which I planned to present on at EUROCALL2017.
Debunking Stereotypes about Middle Eastern Women in the EFL Classroom Parisa Mehran
The presenters are both teaching English in the Japanese university context: one is Canadian, and the other is Iranian. In this presentation, we first share our personal narratives focusing on Iran, the first presenter’s homeland, and Pakistan, which the second presenter visited. We then talk about our collaboration in our English classrooms to dispel stereotypes about Middle Eastern women. We introduce our projects (e.g., “I Am More Than A Stereotype: Meet An Iranian Woman”, “Exploring Stereotypes”, and “Write4Change”) aimed at helping our learners reflect upon these existing stereotypes, and the impact of these misconceptions on attitudes towards the identity of these women. We also discuss how technology helps us fulfill our goals of challenging stereotypes.
Building Global Awareness and Responsible World Citizenship through Augmented...Parisa Mehran
English language education no longer focuses merely on building and improving language skills, but also aims to develop independent-thinking learners and socially-responsible global citizens. Teachers play a vital role in helping achieve this. In this poster presentation, we report on our two projects, titled “How Can I Change the World? Bookmarks for Puerto Rico” and “A Virtual Trip to the Unseen Iran”, that we designed to encourage critical thinking, cultivate empathy, dispel stereotypes about the Middle East with the focus on Iran, and develop global awareness in our students. First, we introduce the projects and share the story behind them. Next, we explain what augmented reality (AR) and virtual reality (VR) are and how they can be used to enhance student learning, increase motivation and engagement, and improve the classroom environment. We also discuss how these cutting-edge technologies, also known as “empathy machines”, can be integrated into classroom activities to build global awareness and responsible world citizenship. Finally, we demo how we used augmented and virtual realities in our projects to create active, engaging, three-dimensional learning environments in which students come out of their seats and traveled to Puerto Rico and Iran.
Connecting to Puerto Rico through Augmented and Virtual RealitiesParisa Mehran
After a massive hurricane hit Puerto Rico in September 2017, Antonio Vantaggiato, a professor at the Universidad del Sagrado Corazón in San Juan, started a campaign on social media for his students, asking people to send postcards of encouragement. In response, we designed a corresponding project for our university classes in Japan, titled “How Can I Change the World: Bookmarks for Puerto Rico”. We chose bookmarks as an “educational” item on which our students could write encouraging messages in English, and which could then become a keepsake for the Puerto Rican students. Through the use of virtual reality (VR), our students were able to “travel” to Puerto Rico to experience the devastating results of the hurricane. They were also able to explore the campus of the Universidad del Sagrado Corazón through 3D images. We also utilized augmented reality (AR) to bring our #care4sagrado message to life. Our learning objectives in this project were to teach our EFL Japanese learners to empathize and to become caring global citizens. In this presentation, we demo our AR-generated message and introduce the AR app we used to create it.
Being an Iranian Woman Today イラン人女性として現代に生きるということParisa Mehran
In this presentation, I will talk about the challenges and opportunities of being an Iranian woman today. I will challenge existing stereotypes and misunderstandings about Iranian women by sharing personal narratives and by recounting stories of Iranian women in power and leadership.
この発表では、イラン人女性として現代を生きる上での困難と機会についてお話します。イラン人女性に対しては様々な偏見や誤解がありますが、今日は私の個人的な経験や、リーダーシップを持って力強く生きているイラン人女性たちのことをお話することで、ステレオタイプを打ち破っていきたいと思います。
How Can I Change the World: Postcards for Puerto RicoParisa Mehran
Here are some responses by EFL Japanese learners to the question "How Can I Change the World". The students then learned about Puerto Rico and wrote their messages to the students at Universidad del Sagrado Corazón.
Developing a Blended Course: Why Quality MattersParisa Mehran
This presentation reports on the development, implementation, and evaluation phases of a blended course of English for general academic purposes targeting undergraduate Japanese students at Osaka University. The basic Successive Approximation Model and the Quality Matters Higher Education Rubric were utilized as major references informing course design, development, and quality assurance. Students' perception on the usefulness of the course, the use of learning analytics, and the measurement of learner achievement will also be discussed.
Multimodal e-Feedback in an Online English CourseParisa Mehran
This study aims to explore the use of online interaction platforms and web-based tools to provide multimodal electronic feedback in an online English course. Additionally, the current study has examined how Japanese learners of English perceive the feedback they have received on their online writing and speaking tasks. The perceived usefulness of the provided feedback was also investigated in relation to learner collaboration and sense of presence in the online course.
I Am More Than A Stereotype: Actions and Stories for Diversity Awareness and ...Parisa Mehran
The SIETAR Japan Special Interest Group Living Within Diversity (LiDi) was created in recognition that diversity exists within every cultural group, and that it is necessary to understand this diversity to prevent stereotypes and profiling, as well as to promote respect and inclusion. In today’s world of turmoil and increasing efforts to exclude those who are different and labeled dangerous or inferior, it is imperative that educators, researchers, trainers, and organizations increase efforts to act as agents of change and build environments of tolerance and acceptance; environments in which everyone can feel respected and safe.
LiDi was formed to provide opportunities for SIETAR members to exchange dialogues for facilitating awareness and empathy with Others in Japan and outside its borders. In keeping with this year’s conference theme of “Promoting Equity and Social Change: Acknowledge the Diversity Within”, Living in Diversity (LiDi) Special Interest Group members will share their personal experiences of diversity and social change as well as share ideas for promoting diversity awareness, equity and social change in classroom and training settings. Presenters will share their experiences with participants in a roundtable style workshop, allowing for more personal and deeper discussions. Participants will have opportunities to take active roles in listening and joining various discussions.
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
Honest Reviews of Tim Han LMA Course Program.pptxtimhan337
Personal development courses are widely available today, with each one promising life-changing outcomes. Tim Han’s Life Mastery Achievers (LMA) Course has drawn a lot of interest. In addition to offering my frank assessment of Success Insider’s LMA Course, this piece examines the course’s effects via a variety of Tim Han LMA course reviews and Success Insider comments.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
Acetabularia Information For Class 9 .docxvaibhavrinwa19
Acetabularia acetabulum is a single-celled green alga that in its vegetative state is morphologically differentiated into a basal rhizoid and an axially elongated stalk, which bears whorls of branching hairs. The single diploid nucleus resides in the rhizoid.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
The French Revolution Class 9 Study Material pdf free download
Justifying the Use of an English Language Placement Test with an Assessment Use Argument
1. LOGO
Justifying the Use of an English
Language Placement Test
with an Assessment Use Argument
Presented by:
Parisa Mehran
Alzahra University
Tehran, Iran
2. Placement Tests
Placement test is considered as a fairly high-stakes test (Bachman & Palmer, 1996, 2010),
and the social consequences of placement decisions are of great significance and need to
be investigated, since such decisions can affect the lives of students (Murray, 2001;
Schmitz & delMas, 1991).
Thus, as Brown (1989) emphasizes, it is important to make valid placement decisions to
avoid mismatches that can occur due to inappropriate placement testing.
3. Validity in Language Assessment
Validity has been regarded as the most significant and complex concept in language
assessment, and it has always been under investigation by language testing experts
and researchers (e.g., Bachman, 1990, 2004, 2005; Bachman & Palmer, 2010; Chapelle,
1999; Cronbach & Meehl, 1955; Kane, 2001, 2012, 2013; Lado, 1961; Messick, 1989). As a
result, the conception of validity has undergone a series of reinterpretations
throughout the history of language assessment.
4. Argument-based Approaches to Validity
Argument-based approaches to validity are based on the concept of a validity argument which has
been used in the process of validation for more than twenty years (e.g., Bachman, 2005; Cronbach,
1988, 1989; House, 1980, Kane, 1992; Mislevy, 2003; Mislevy, Steinberg, & Almond, 2002, 2003).
The process is comparable to building a legal case to persuade a judge or a jury. The process of
validation thus becomes ongoing: As long as a test is alive, the collection of relevant evidence is
going to be continued (Bachman & Palmer, 2010). Hence, any kind of relevant evidence is gathered
to show the plausibility of the intended interpretations and uses (Bachman, 2004; Kane, 2012).
5. Purpose of the Study
Using Bachman and Palmer's (2010) AUA as a framework, this study examined the
validity of an English language placement test, which is composed of the Oxford Quick
Placement Test (OQPT) and a follow-up oral examination. The following research
question was addressed:
To what extent are the OQPT and the oral examination justifiable in placing students
appropriately according to Bachman and Palmer's (2010) AUA?
6. Methodology: Participants and Setting
This study was conducted at one of the English language institutes in Tehran, Iran.
Three hundred and thirty-two (332) newcomers to the institute who had to take the
placement test participated in this study, and 15 of them were interviewed. The head of
the institute, three examiners of the placement test, ten teachers, and four experts also
attended the current study.
8. The AUA for Justifying the Placement Test
As Vongpumivitch (2010) and Wang et al. (2012) remark, Bachman and Palmer's (2010) framework
has a top-down approach. That is, the four claims are discussed from the perspective of test
development rather than from that of test use. Therefore, in this study, where the aim is to evaluate
the overall usefulness of an English language placement test, the four claims are presented in the
reverse order from that in Bachman and Palmer (2010).
It should also be mentioned that as Bachman and Palmer (2010) emphasize not all of the warrants
and rebuttals listed by them will necessarily be needed in the AUA for any given test. Moreover,
due to practical research limitations, not all of the warrants and rebuttals have been investigated in
the present study.
9. Claim 4: The assessment records of the OQPT and the oral examination are consistent across
different assessment tasks, different aspects of the assessment procedure, and across different
groups of test takers.
Claim 4: Assessment Records
10. Consistency
The first warrant for this claim is that the procedures for administrating the OQPT and the oral
examination are followed consistently across different occasions and for all test takers:
The observation of how the OQPT and the oral examination were administered as well as the
interviews with the examiners and the head of the institute revealed that there are a set of
administrative procedures which are strictly followed by the test administrators; hence, the
administrative procedures are consistent across different occasions and for all test taker groups.
11. Consistency (cont.)
Another warrant to support the consistency claim involves the scoring criteria and procedures:
The criteria and procedures for rating test takers' performance on the OQPT are well specified and
are adhered to. Since the OQPT is in multiple-choice format, its rating criteria and procedures are
quite objective, and scoring is done based on an answer key.
However, the criteria and procedures for the oral examination are not well specified and are quite
subjective. A set of questions have been devised based on the coursebook. In this sense, the
administration of the oral examination is consistent, yet its scoring process does not follow any
specific procedures. This lack of evidence could be a rebuttal here.
12. Consistency (cont.)
With respect to the warrant of rater training:
Raters undergo training before administrating the placement test.
However, one of the examiners was not satisfied with the training process, and she
believed that what matters is just the examiner's marketing skill to "grab more
customers" for the institute.
13. Consistency (cont.)
To check internal consistency of the items, as another warrant:
Kuder-Richardson formula 20 (KR-20) was used.
The reliability coefficient (KR-20) obtained for the OQPT Version 1 was .93 and for the OQPT Version 2 was
.88 showing that the OQPT has reasonable internal consistency reliability.
Two main test item indices (item difficulty and item discrimination) were used in the test item analysis for the
OQPT.
In terms of difficulty, the items have been ordered from the easiest to the most difficult, and this is in line
with the view of experts, examiners, and test takers. The analysis of item difficulty showed that, by and
large, most of the items were difficult (56% in the Version 1 and 63% in Version 2), and both versions of the
OQPT did not have a fairly acceptable distribution of difficulty.
In terms of item discrimination, the analysis of items demonstrated that the items in the OQPT Version 1
had good amount of discrimination (75%). However, the OQPT Version 2 contained less items of fair
discrimination (46%) in comparison to the first version.
14. Consistency (cont.)
In regard to inter-rater and intra-rater reliability, inconsistencies between and within human
raters are not a source of measurement error because the scoring of the OQPT is done through
an answer key.
In the case of the oral examination, Cronbach's alpha was computed. The alpha was .93 for
inter-rater reliability and .96 for intra-rater reliability indicating that despite the lack of
consistent criteria and procedures for the oral examination, the oral examination has
reasonable internal consistency reliability.
The analysis of the two versions of the OQPT brought a serious rebuttal to the consistency claim.
Cronbach's alpha was calculated and the alpha was .000 indicating that the two versions are not
equivalent. Two experts also remarked that the second version is much more difficult, and that
it cannot be considered as equivalent to the first version.
15. Claim 3: The OQPT scores and the oral examination results can be interpreted as test takers' level
of English proficiency and place them in their appropriate level. Such interpretations are
meaningful, impartial, generalizable, relevant, and sufficient.
Claim 3: Interpretations
16. Meaningfulness
Interpretations about the construct to be assessed are meaningful if they are based on a frame of reference like a
course syllabus, a needs analysis, or a general theory of language ability. The head of the institute and the
examiners believed that the OQPT is a suitable placement test because Oxford University Press is the publisher of
both this test and the coursebook taught in the institute (i.e., English Result). The teaching method followed in the
institute is Communicative Language Teaching (CLT), and speaking is thus the primary focus; therefore, the oral
examination has a significant role in placement testing. However, lack of a listening section in the OQPT can be a
rebuttal to the meaningfulness of the interpretations.
17. Impartiality
To support the impartiality warrant, the assessment items/tasks should be checked for response
formats or content that may either favor or disfavor some test takers, and test takers should be
treated impartially in terms of all aspects of test administration.
As mentioned earlier, interviews with test takers and examiners revealed that due to the lack of
a specific rubric for the oral examination, it was believed that interpretations about the ability to
be assessed were not made without bias against any groups of test takers.
No complaint was made in regard to the appropriateness of the content. Bias and item
sensitivity studies need to be done for deeper analysis.
18. Generalizability
According to the generalizability warrant, the characteristics of the assessment items/tasks (e.g., input,
expected response, type of interaction) as well as the scoring criteria and procedures of the test tasks
should correspond closely to those of the target language use (TLU) domain.
It might be that the items/tasks in the OQPT and the oral examination do not exactly correspond to all
of the TLU tasks; however, the content of the OQPT and the oral examination corresponds to the
content of the textbook taught in the institute. Moreover, in the oral examination, test takers' real world
language performance is examined. Thus, this can to some extent support the generalizability warrant.
Here, it is worth mentioning that some of the teachers, examiners, and experts asserted that a TOEFL or
an IELTS test was a better test, indeed an ideal one, due to having writing and specially listening parts,
for placement testing, but because of time limitations, they could not be used as placement tests.
Consequently, a TOEFL test was given to those who had taken the OQPT, and the results manifested
that the correlation between the OQPT and TOEFL scores was not high (r=.66).
19. Relevance
The forth warrant is relevance according to which the assessment-based interpretations should
provide information that is relevant and helpful for the decision makers to make decisions.
Based on the interviews conducted with examiners, it was revealed that the OQPT scores were
not sufficiently helpful in placement testing. This could have been a rebuttal to the relevance
warrant; yet, the oral examination is in support of this warrant since it is quite helpful for the
examiners to make their placement decisions. As said before, lack of a rubric is a serious
rebuttal.
20. Sufficiency
The fifth warrant demands that the assessment-based interpretations should provide
information that is sufficient for the decision makers to make decisions.
Again, because the process of placement testing includes both a written and an oral
test, sufficient information is obtained to make placement decisions.
21. Claim 2: The placement decisions that are made on the basis of the OQPT scores and the oral
examination results are sensitive to local values and equitable to all stakeholders.
Claim 2: Decisions
22. Values Sensitivity
According to values sensitivity warrant, the existing community values and relevant legal
requirements should be carefully and critically considered in the admission decisions that are to be
made and in determining the relative seriousness of false positive and false negative classification
errors.
The interviews and observations revealed that the process of placement testing does not
guarantee test fairness considerations, and in this phase just absorbing more clients is
important. Hence, it is possible to have potential false positives (i.e., individuals are placed in a
level higher than their actual level) and false negatives (i.e., individuals are placed in a level
lower than their actual level). Usually the latter happens, because it is less risky; nevertheless, if
at the time of placement testing the institute does not have the level appropriate for the test
taker (due to some limitations, such as space, time, lack of students, all the levels are not always
covered), the test taker will be placed in a higher level.
23. Equitability
Due to the subjectivity of the oral examination, it cannot be claimed that the same cut scores
and decision rules are used to classify all students who have applied for the same program, and
no other considerations are used. The economic and practical considerations always exist.
Consequently, test takers and other stakeholders are not fully informed about how decisions will
be made and whether decisions are actually made in the way described to them.
24. Claim 1: The consequences of the placement decisions based on the OQPT scores and the oral
examination results are beneficial to all stakeholders that use the test, including the test takers, the
institution, the teachers, and the supervisor.
Claim 1: Consequences
25. Beneficence
The first warrant is that the consequences of using the assessment that are specific to each
stakeholder will be beneficial.
Some of the test takers were interviewed after their placement test and their attendance in
their classes. On the whole, they were satisfied, although some of them believed that their
level was higher, and that they were placed in a lower level. To them, the reason is basically
cost-effectiveness. Most of the teachers believed that their students were homogenous in the
class; yet, two teachers strongly disagreed and they believed that their classes were not at all
homogenous especially at higher levels.
26. Conclusion
Based on the evidence gathered, this study found that the assessment records of the OQPT and the oral
examination were consistent across different assessment tasks, different aspects of the assessment
procedure, and across different groups of test takers. However, the oral examination required a set of
clear criteria.
Moreover, the analysis of the two versions of the OQPT manifested that their parallelism was under
question which could threaten the consistency of the assessment records.
The findings also indicated that the OQPT scores and the oral examination results could be interpreted
somewhat as test takers' level of English proficiency and could place them in their appropriate levels.
Such interpretations were meaningful, impartial, relevant, and sufficient, although lack of a listening
section in the OQPT and lack of a rubric for the oral examination could be threatening, and
generalizability of the results was to some extent under question.
In addition, the placement decisions were not sensitive to local values and equitable to all stakeholders
due to the subjectivity of the oral examination and the economic considerations of the institute.
Lastly, by and large, the consequences of the placement decisions were beneficial to all stakeholders that
use the test, which is composed of the test takers, the institution, the teachers, and the supervisor.
27. Local Implications
To support the intended test use, it would be helpful to examine the negative evidence that has
been identified in this study and resolve the identified issues or mitigate the potential negative
impact of unresolved issues. For instance, in the case of the current placement test, the oral
examination can be given based on a rubric, a listening section can be added to the written test,
and economic considerations can become less important for the institute, and therefore the
intended uses of the test would become much more justifiable with stronger evidence.
28. The Merits/Demerits of Using an AUA
Finally, this study serves as an illustration of the merits/demerits of using an AUA.
On the whole, the AUA provides a sound framework in which the validity of a test and its use
can be justified and the test developers/users can be accountable for their test.
With the help of the AUA framework, the process of assessment justification becomes more
comprehensive, systematic, and coherent. In fact, one of the merits of an AUA is its clear
articulation about which types of evidence should be collected for which claims or warrants.
However, in the process of assessment justification, an AUA demands that the evaluation of the
test be done at many levels and this needs different types of data and analyses. Thus, in practice,
the complexity of the justification study may be a big challenge for a single researcher.
According to Bachman (2004), items within a fairly narrow range of item difficulty, around .50, are desirable.
Oller (1979) asserts that, for item discrimination, correlations of less than .35 are considered as not being useful for discriminating between participants.