Innovative Assessment Formats
The following list of resources related to innovative assessment formats was developed for use by psychometricians involved in assessment formats. Some are new formats, while others focus on what is NOT a multiple-choice question. Other examples include a new twist to an older item type or assessment process, e.g., a constructed response or essay question evaluated by a computer grading system (a new approach) or a computerized drag and drop item format delivered with a new twist now called "drag and place.” As another example, Standardized Patients or OSCEs, while not exactly new formats, are being used in perhaps new, more precise ways with video-monitored testing centers.
Strategies and Processes for Developing Innovative Items in Large Scale Assessments
Although this paper focuses on educational testing, it provides information about innovative/alternative items that may also be useful in licensure and certification testing. It describes potential benefits of using innovative/alternative item types and explains how item templates may be used to develop an item bank efficiently.
Designing Templates Based on a Taxonomy of Innovative Items
A resource for understanding the various ways in which items can be innovative/alternative. This paper provides many examples that allow the reader to evaluate the suitability of various innovations for testing in a particular profession or field.
An Overview of Innovative Computer-Based Testing
This paper recaps the dimensions of innovation described in an early paper by Parshall, et al. It addresses potential benefits (vs. MCQ) and risks of adopting innovations.
It includes some terminology used in connection with innovative/alternative item types. This certainly should be used to complement CLEAR's testing glossary.
Measurement Properties of Two Innovative Items Formats in a Computer-Based Test
This is an academic research paper. It is technical, but it offers useful conclusions about figural response and constructed response items in comparison with multiple choice.
This article in the American Journal of Surgery describes a performance assessment of surgical skills and evaluates the reliability and validity of the assessment.
State Assessment Systems: Exploring Best Practices and Innovations: Summary of Two Workshops
Chapter 3. Recent Innovative Assessments
The link leads to a chapter of innovative/alternative assessments in a book on state educational assessments. Very little detail is given about the kinds of tests described, but there is quite a bit of general information regarding potential problems, successes, and stakeholder reactions to innovative/alternative assessments that have been tried in education.
A general description of the characteristics of tasks considered authentic.
Here is information on the Ventriloscope, a tool that produces simulated sounds testing clinical skills in medicine. Only the abstract of the article is shown, but the full article is probably obtainable for those who are motivated to learn more.
This link leads to an audio podcast describing "two innovative assessment methods" employed in evaluating social work students. The entire program is over 45 minutes long, but the introduction suggests that there may be some generalizable information regarding innovative/alternative assessment in a profession.
Other than multiple-choice format
Innovative Item Types Require Innovative Analysis
This paper was presented at NOCA/ICE in 2009. It is a good resource focusing on issues in analyzing innovative/alternative items.
Constructed-Response Test Questions: Why We Use Them; How We Score Them
This is an ETS publication on constructed-response items. It is not too technical and provides a nice overview of the strengths and weaknesses of constructed-response items as well as scoring information.
The following links describe or discuss Extended Matching Questions:
- Extending Matching Items
- Creating an Extended Matching Question Type
- Multiple Choice Questions are Superior to Extended Matching Questions to Identify Medicine and Biomedical Sciences Students Who Perform Poorly
- The Rise of the Extended Matching Question
Innovative Item Types for Computerized Testing
Here is the Parshall, Davey, & Pashley chapter on Innovative Item Types for Computerized Testing
Evaluating Innovative Items for the NCLEX: Part 1 Usability and Pilot Testing
NCSBN's evaluation of items using audio, video, animation, etc.
Computer-Based Assessment: “Intermediate Constraint” Questions and Tasks for Technology Platforms
Scalise's taxonomy of innovative/alternative item types.
Improving the Quality of Innovative Items Types: Four Tasks for Design and Development
Parshall and Harmes on improving quality of innovative/alternative items
Scalise & Gifford on constructing "intermediate constraint" items