(2022). Cheating on Testing: Cramming on Test Preparation. Language Teaching Research Quarterly, v29 p134-146 Professor Glenn Fulcher's research has influenced language testing and assessment in two ways: in proposing a new understanding of data-driven speaking test design, and in drawing attention to the core identity of language testers. The former has had a powerful impact on practical speaking testing and assessment. The latter has asked a very essential question of what language testers do, exploring philosophical, historical, and social ideas from a pragmatic view of language testing and assessment. In particular, Fulcher (2015)'s chapters on meritocracy and values in language tests explain why we have been using tests throughout history to discern vital roles of language tests and yet why people attempt to cheat on them, in relation to 'three Ms' ideas including meritocracy, material success, and motivation. Within this view, this paper aims to review TOEFL cramming practices on test preparation, using Fulcher (2015)'s notion of cheating. Korean test takers study TOEFL cramming skills…
(2019). Interaction between Language Testing Research and Classroom Testing Practice. Educational Process: International Journal, v8 n1 p59-71 The existing gap between research and practice in language testing has posed a huge challenge to language teachers. In particular, this study intended to examine language testing research and classroom testing activities for their degree of interaction from Iranian EFL teachers' points of view. The analysis drew on the questionnaire developed by Nassaji (2012), which consists of quantitative and qualitative sections. The data were collected from 200 language institute teachers in Bushehr, Iran. The findings indicated that most of the teachers confirmed their familiarity with language testing research, while only a few believed that they were able to conduct or publish testing research. Although the participants acknowledged that the information gained from reading language testing could be valuable for their testing performances, they found their experience more relevant than knowledge gained from language testing research. This study also illustrated that there is a growing…
(2020). Ritualisation of Testing: Problematising High-Stakes English-Language Testing in Bangladesh. Compare: A Journal of Comparative and International Education, v50 n4 p533-553 Although use of high-stakes tests is common across developing societies, very little is known about how these tests are designed, what principles and criteria guide test construction, and what factors influence this process. The present study investigates the development of the English Paper-1 test for the Higher Secondary Certificate examination in Bangladesh, drawing on curriculum policy and test documents, and particularly on the perspectives of test writers and moderators. The findings reveal a range of conservative, compliant and context-responsive approaches that ensure the perpetuation of problematic test design practices and processes. The authors argue that these responses encourage 'ritualistic' design practices which negate concerns about test reliability and validity, and which obscure the basis by which winners and losers are created through the education system. Importantly, social celebration of this ritual does not seem to question the test architecture itself. The…
(2022). Achieving Testing Effects in an Authentic College Classroom. Teaching of Psychology, v49 n2 p164-175 Apr Background: Findings from the testing effect literature suggest several ways to achieve testing effects in an authentic classroom, but few consider instructor workload, equity, and resources that determine feasibility and sustainability of testing effect methods in practice. Objective: To determine elements and procedures from the testing effect literature for practical application, devise a method for feasibly and sustainably implementing testing effect methods in practice, and determine if a simple way to incorporate retrieval practice into an existing introduction to psychology course was sufficient to observe testing effects. Method: Quiz scores of Introductory Psychology sections with and without retrieval practice were compared. Sections with retrieval practice also compared the effects of repeated and new questions on quiz performance. Results: Students with retrieval practice performed significantly better on quizzes than those without. Repeated and new retrieval practice…
(2022). Student Perception of Computer-Based Testing in Kwara State, Nigeria. International Journal of Web-Based Learning and Teaching Technologies, v17 n1 Article 75 Computer Based Testing has become a prominent method in student assessment in Nigeria and student perception on this testing technique is paramount. This study examined the perception of undergraduate students towards computer based testing by comparing several modules studied by Undergraduate students in Nigeria which constitutes a major gap in literature. Results showed that majority of students preferred computer based testing to paper based testing but were not will to adopt this technique in all courses. Results further showed majority of students showed preference to paper based tests in Mathematics more than 50% of students had below average grades when the CBT technique was implemented. Similarly, students showed poor grades in Mathematics, Chemistry and Physics, results further showed the relationship between computer and anxiety and performance in in Mathematics. This study has implications for university administrators in the creation of policies for Computer based testing….
(2022). Ocular-Motor Deception Testing in Civilly Detained Sexually Violent Persons: An Alternative to Post-Conviction Sex Offender Polygraph Testing?. Applied Cognitive Psychology, v36 n1 p32-42 Jan-Feb Post-conviction polygraph testing during sex offender (PCSOT) treatment is common. Ocular-motor deception testing (ODT) uses measures of cognitive load to assess credibility. The accuracy of ODT for discriminating deceptive from truthful response patterns in sexually violent persons was evaluated. Participants chose to 'steal' a voucher of monetary value and try to 'beat the machine,' or leave the voucher and respond truthfully. Compensation was determined by participants' choices and the results of the ODT credibility assessment. Experiment 1 (n = 26) established a base rate of deception to optimize the ODT scoring model. Experiment 2 (n = 74) tested generalizability of the results. Tests of noninferiority found observed accuracy rates were not significantly less than published rates of 80%. Results support use of ODT methods as a potential alternative to PCSOT. Legal and ethical issues regarding the use of deception detection technologies impacting individuals involved in criminal…
(2021). Road Weary? Testing Whether Long Commutes to Testing Sites Explain Deficient Cyber Charter School Academic Performance. Journal of School Choice, v15 n3 p471-481 Cyber charter schools may increase access to a range of educational offerings, but they substantially underperform traditional public schools on measured academic performance, as demonstrated in a range of studies with distinct samples and methods. Artificial testing conditions offer a possible explanation. In contrast to face-to-face students, for cyber charter students standardized testing occurs in unfamiliar locations, often hours from the location of instruction (home), possibly leading to fatigue and lesser performance. Geocoding home locations and testing sites for 2015-18 mathematics, ELA, and science test scores from 5,493 cyber school students in a northern state, we test whether drive times affect performance, controlling for student characteristics. Save for marginal statistical impacts on mathematics tests, we find no evidence that drive times affect cyber student measured academic performance. Implications are discussed….
(2022). Computerized Testing in Reading Comprehension Skill: Investigating Score Interchangeability, Item Review, Age and Gender Stereotypes, ICT Literacy and Computer Attitudes. Education and Information Technologies, v27 n2 p1771-1810 Mar Score interchangeability of Computerized Fixed-Length Linear Testing (henceforth CFLT) and Paper-and-Pencil-Based Testing (henceforth PPBT) has become a controversial issue over the last decade when technology has meaningfully restructured methods of the educational assessment. Given this controversy, various testing guidelines published on computerized testing may be used to investigate the interchangeability of CFLT and PPBT mean scores to corroborate if test takers' testing performance is influenced by the effects of testing administration mode; specifically, if validity and reliability of two versions of the same test are affected. This research was conducted to probe not only score interchangeability across testing modes but also to explore the role of age and gender stereotypes, item review, ICT literacy and attitudes towards computer use as moderator variables in test takers' reading achievement in CFLT. Fifty-eight EFL learners homogeneous in both general English and reading…
(2022). Comparing Effect Sizes and Their Confidence Intervals: A Primer on Equivalence Testing in Educational Research. Journal of New Approaches in Educational Research, v11 n2 p209-225 This paper's objective was to teach the Equivalence Testing applied to Educational Research to emphasize recommendations and to increase quality of research. Equivalence Testing is a technique used to compare effect sizes or means of two different studies to ascertain if they would be statistically equivalent. For making accessible Equivalence Testing, this technique was explained with two examples by conducting manual calculations, using an online calculator, the software R, the software SPSS, and a t table. Furthermore, the software R with an Equivalence Testing code was used, and its results were graphed and discussed with details. Among other recommendations given, Equivalence Testing can be a useful tool for comparing means and effects within certain bounds that could hopefully imply a practical significance to provide meaning to findings. The results of Equivalence Testing can indicate that two treatments' effects are statistically equivalent or not. Thus, the Equivalence…
(2022). Mode Effects in College Admissions Testing and Differential Speededness as a Possible Explanation. Educational Measurement: Issues and Practice, v41 n3 p14-25 Aut As testing programs transition from paper to online testing, they must study mode comparability to support the exchangeability of scores from different testing modes. To that end, a series of three mode comparability studies was conducted during the 2019-2020 academic year with examinees randomly assigned to take the ACT college admissions exam on paper or online. Unlike many prior mode comparability studies, examinees testing online earned higher scores on average than examinees testing on paper, especially on the English, reading, and writing tests. Much of the observed mode effects were accounted for by items near the ends of the tests. Subsequent examination of item omit rates, differential item functioning by mode, and change-point analyses supported the notion that the observed mode effects were related to differential speededness. Specifically, examinees testing online were slightly less speeded, and were therefore able to perform better on items near the ends of the tests….
(2021). Testing Requirement. 50-State Comparison: Vouchers. Education Commission of the StatesSchool vouchers are state-funded programs–often called scholarship programs–that allow students to use public monies to attend a private school. The state provides a set amount of money, typically based on the state's per-pupil amount, for private school tuition. There are currently 27 voucher programs in 16 states and the District of Columbia. This document is part of the "50-State Comparison: Private School Choice" that provides information on state private school choice policies specific to vouchers, education savings accounts, and scholarship tax credits. The document shows the testing requirement for the states with voucher programs. [View the full "50-State Comparison: Private School Choice" here: www.ecs.org/50-state-comparison-private-school-choice/.%5D…
(2022). Consequential Research of Accountability Testing: The Case of the CET. Language Testing in Asia, v12 Article 15 This article examines the changes taken place over the decades after a test-based accountability system was in place and proposes an agenda for consequential research of the testing program. The article starts with a review of theories of test-based accountability (Supovitz, Journal of Educational Change 10:211-227, 2009) and a socially grounded, integrated approach to test validation (Chalhoub-Deville, Language Testing 33:453-472, 2016; Chalhoub-Deville and O'Sullivan, Validity: Theoretical development and integrated arguments, 2020). Following Supovitz's frameworks of accountability testing, the case of the College English Test (CET), a national English language testing program in China, is analyzed to explain the nature of the consequences of accountability testing and show the motivational effects, alignment effects, the informational role and the symbolic values of the accountability testing program. The case analysis highlights the importance of investigating consequences of an…
(2019). Dynamic Multistage Testing: A Highly Efficient and Regulated Adaptive Testing Method. International Journal of Testing, v19 n3 p227-247 This study introduced dynamic multistage testing (dy-MST) as an improvement to existing adaptive testing methods. dy-MST combines the advantages of computerized adaptive testing (CAT) and computerized adaptive multistage testing (ca-MST) to create a highly efficient and regulated adaptive testing method. In the test construction phase, multistage panels are assembled using similar design principles and assembly techniques with ca-MST. In the administration phase, items are adaptively administered from a dynamic interim pool. A large-scale simulation study was conducted to evaluate the merits of dy-MST, and it found that dy-MST significantly reduced test length while maintaining the identical classification accuracy with the full-length tests and meeting all content requirements effectively. Psychometrically, the testing efficiency in dy-MST was comparable to CAT. Operationally, dy-MST allows for holistic pre-administration management of test content directly at the test level. Thus,…
(2020). Intelligent Feedback on Hypothesis Testing. International Journal of Artificial Intelligence in Education, v30 n4 p616-636 Nov Hypothesis testing involves a complex stepwise procedure that is challenging for many students in introductory university statistics courses. In this paper we assess how feedback from an Intelligent Tutoring System can address the logic of hypothesis testing and whether such feedback contributes to first-year social sciences students' proficiency in carrying out hypothesis tests. Feedback design combined elements of the model-tracing and constraint-based modeling paradigms, to address both the individual steps as well as the relations between steps. To evaluate the feedback, students in an experimental group (N = 163) received the designed intelligent feedback in six hypothesis-testing construction tasks, while students in a control group (N = 151) only received stepwise verification feedback in these tasks. Results showed that students receiving intelligent feedback spent more time on the tasks, solved more tasks and made fewer errors than students receiving only verification…
(2021). Testing the Tests for Racism. Academic Questions, v34 n3 p17-27 Against the claim of decreased American racism over the past twenty years have come the audit studies. Throughout much of the modern era, a large number of empirically-minded social scientists have pointed out that racism seems by any objective standard to be declining. However, other scholars argue that anonymous tests show considerable modern-era bias against blacks and other racial minorities. How can both of these results co-exist, across dozens of well-designed studies? To answer this question, Wilfred Reilly reviews the audit studies and finds some of their results obviously do indicate that bias remains a reality within significant sectors of the U.S. employment and housing markets. However, these studies rarely if ever examine rates of prowhite (or pro-POC) bias in higher education, the public sector, and the minority business community; very frequently do not include adjustments for social class or perceived competence; and have not extensively compared the bias faced by…