Frequently Asked Questions About Licensing Exams
CLEAR Exam Review
Eric Werner, M.A.
Question: Are there statistical methods that my board can use to show whether one candidate has copied from another during a multiple-choice test? If so, what actions should we take if the method points to cheating?
Answer: You could use any of several approaches to determine the probability that the similarity between the responses of two candidates occurred by chance alone. In general, you'd begin by assuming that the two candidates performed independently of one another and then you'd use this assumption as the basis for calculating the odds of the response similarity that occurred. Extremely small odds would raise doubt about the correctness of your assumption and may lead you to suspect copying. Sound simple? Well, its not.
Copying detection involves significant statistical, legal, and ethical issues. If you are about to decide whether to use statistical detection methods, or to allow them to be used on your behalf by a testing service, first familiarize yourself with these issues. If you proceed otherwise, you may create more problems than you solve.
In this column I cannot detail all the technical issues involved, but I can list several important ones and direct you to more thorough sources. Consider these points:
Candidates of similar ability should be expected to answer a high percentage of the same questions correctly. Therefore, statistical analysis should focus on the similarity of incorrect responses. Yet even for these, candidates of similar ability may honestly respond the same more often than would be expected by chance alone.
A statistically significant result will not prove that your assumption of independent performance is wrong. The assumption may be correct, and the result may simply represent a rare event.
Even if you reject your original assumption because of the statistical findings, it does not follow that copying is the only remaining way of explaining the similarity. Two candidates may have studied together, attended the same educational institution, used the same texts, completed the same test preparation course, studied the same nonsecured test material from previous exams, followed the same test-taking strategy, and so forth.
Even if all such possibilities are ruled out and you are inclined to believe that copying occurred, you still face the question of whether candidate A might have copied from B, B from A, or each from the other.
Some statistical methods involve the assumption that if a candidate does not know the answer to a question, he or she will guess randomly. This may not be a good assumption because the guesses of most candidates are educated. For this and other reasons, modification of conventional statistical testing may be necessary.
Although statistical evidence can play an important probabilistic role in your attempt to show that copying occurred, it cannot prove your case. Your board should never accuse a candidate of copying or take adverse action against a candidate solely because of statistical findings. Collateral evidence, especially proctor documentation of suspicious behavior, is needed. Since a candidate has a significant interest in his or her professional reputation, you should expect that an accusation of copying will be challenged. Involve your legal counsel in any copying-detection plans you might be developing.
Want to know more? Here are some references:
Angoff, W.H. (1974). The development of statistical indices for detecting cheaters. Journal of the American Statistical Association, 69, 44-47.
Bellezza, F.S., & Bellezza, S.P. (1990). Detection of cheating on multiple-choice tests using error similarity analysis. Unpublished manuscript. Ohio University, Department of Psychology, Athens, Ohio.
Boland, P.J., & Proschan, M. (1990).
The use of statistical evidence in allegations of exam cheating. Chance:
New For Statistics and Computing, 3, 10-15.
Buss, W.G., & Novick, M.r. (1980). The detection of cheating on standardized tests: statistical and legal analysis. Journal Of Law and Education, 9, 1-64.
© 2002 Council on Licensure, Enforcement and Regulation