Innovative Assessment Formats

The following list of resources related to innovative assessment formats was developed for use by psychometricians involved in assessment formats.  Some are new formats, while others focus on what is NOT a multiple-choice question.   Other examples include a new twist to an older item type or assessment process, e.g., a constructed response or essay question evaluated by a computer grading system (a new approach) or a computerized drag and drop item format delivered with a new twist now called "drag and place.”  As another example, Standardized Patients or OSCEs, while not exactly new formats, are being used in perhaps new, more precise ways with video-monitored testing centers.

 

Strategies and Processes for Developing Innovative Items in Large Scale Assessments

Although this paper focuses on educational testing, it provides information about innovative/alternative items that may also be useful in licensure and certification testing.  It describes potential benefits of using innovative/alternative item types and explains how item templates may be used to develop an item bank efficiently.

 

Designing Templates Based on a Taxonomy of Innovative Items

A resource for understanding the various ways in which items can be innovative/alternative. This paper provides many examples that allow the reader to evaluate the suitability of various innovations for testing in a particular profession or field.

 

An Overview of Innovative Computer-Based Testing

This paper recaps the dimensions of innovation described in an early paper by Parshall, et al.   It addresses potential benefits (vs. MCQ) and risks of adopting innovations.

 

Assessment Glossary    

It includes some terminology used in connection with innovative/alternative item types. This certainly should be used to complement CLEAR's testing glossary. 

 

Measurement Properties of Two Innovative Items Formats in a Computer-Based Test

This is an academic research paper.  It is technical, but it offers useful conclusions about figural response and constructed response items in comparison with multiple choice. 

 

Testing Technology

This article in the American Journal of Surgery describes a performance assessment of surgical skills and evaluates the reliability and validity of the assessment.

 

State Assessment Systems:  Exploring Best Practices and Innovations:  Summary of Two Workshops

Chapter 3.  Recent Innovative Assessments

The link leads to a chapter of innovative/alternative assessments in a book on state educational assessments.  Very little detail is given about the kinds of tests described, but there is quite a bit of general information regarding potential problems, successes, and stakeholder reactions to innovative/alternative assessments that have been tried in education.

 

Authentic Assessment

A general description of the characteristics of tasks considered authentic.

 

The Ventriloscope as an innovative tool for assessing clinical examination skills:  appraisal of a novel method of simulating auscultatory findings

Here is information on the Ventriloscope, a tool that produces simulated sounds testing clinical skills in medicine.  Only the abstract of the article is shown, but the full article is probably obtainable for those who are motivated to learn more.

 

inSocialWork Podcast Series

This link leads to an audio podcast describing "two innovative assessment methods" employed in evaluating social work students. The entire program is over 45 minutes long, but the introduction suggests that there may be some generalizable information regarding innovative/alternative assessment in a profession.

 

Other than multiple-choice format

 

Innovative Item Types Require Innovative Analysis

This paper was presented at NOCA/ICE in 2009.  It is a good resource focusing on issues in analyzing innovative/alternative items.

 

Constructed-Response Test Questions:  Why We Use Them; How We Score Them

This is an ETS publication on constructed-response items.  It is not too technical and provides a nice overview of the strengths and weaknesses of constructed-response items as well as scoring information.

 

The following links describe or discuss Extended Matching Questions:


Innovative Item Types for Computerized Testing

Here is the Parshall, Davey, & Pashley chapter on Innovative Item Types for Computerized Testing

 

Evaluating Innovative Items for the NCLEX:  Part 1 Usability and Pilot Testing

NCSBN's evaluation of items using audio, video, animation, etc.

 

Computer-Based Assessment:  “Intermediate Constraint” Questions and Tasks for Technology Platforms

Scalise's taxonomy of innovative/alternative item types.

 

Improving the Quality of Innovative Items Types:  Four Tasks for Design and Development

Parshall and Harmes on improving quality of innovative/alternative items

 

Computer-Based Assessment in E-Learning:  A Framework for Constructing “Intermediate Constraint” Questions and Tasks for Technology Platforms

Scalise & Gifford on constructing "intermediate constraint" items