Frequently Asked Questions About Licensing Exams

Content validation of examinations

CLEAR Exam Review (Summer 1995)
Norman R. Hertz

Question: What does validity mean in terms of licensing and certification examinations? If an examination is valid, does it mean that candidates who score well on the examination will perform more effectively on the job than a candidate who scores less well?

Answer: The validity referred to in licensing examinations is known as content-related validity. An examination is considered content valid if it is based upon the results of an occupational analysis, sometimes called a job analysis or practice analysis. Content-related validity if based on the premise that a candidate who passes a licensing examination is knowledgeable in the required content of the job. It is an inappropriate interpretation to use the results to predict how effective a candidate will be in the occupation.

In employment contexts, predictive validity can be established demonstrating that a higher score on employment examinations is associated with higher performance on the job. No such relationship can be established in licensing settings. All who pass the licensing examination are considered competent to practice. Examinations of this type are used to make mastery and non-mastery decisions. Any other use of the scores within these categories is inappropriate.

CLEAR Exam Review (Summer 1996)
Norman R. Hertz

Question: In what ways do licensing examinations differ from academic (achievement) and employment examinations?

Answer: First, examinations are designed with regard to their purpose. Licensing examinations are designed to assess candidates' competence after they have completed their education or training and obtained experience. Furthermore, licensing examinations should be designed to assess higher level skills; that is, examinations should not only assess candidates' knowledge, but also whether they are able to apply that knowledge to solve problems or handle situations that they may face in actual practice.

In contrast, academic examinations are designed to measure students' knowledge of material presented in the classroom. The focus of the examination is usually relatively narrow and covers only the material learned in the class--much narrower than the scope of licensing examinations.

Employment examinations are broader in scope than academic examinations but narrower than licensing examinations. Employment examinations are designed to meet job-related standards, as are licensing examinations. However, employment examinations are designed to predict whether a test taker who scores high on an examination will perform well on the job. It is important to make the distinction here that licensing examinations should not be used as a substitute for employment examinations.

The focus of licensing examinations is not to predict success as an employment examination might, but to assure the public that the person who is licensed is qualified to practice without harming the public. By design, licensing examinations do not measure skills critical to business success such as knowledge of office procedures or management practices.

For the qualified candidate who has obtained the education and quality experience, the licensing examination should not be an insurmountable hurdle. The licensing examination is designed to assess candidates' ability to apply the competencies that they gained from their education and experience. Since the examination is applied rather than theoretically based, the licensing examination is much different from a final examination in a college course.

The consequences of an improperly developed licensing examination are much greater than those of an academic or employment examination. If the licensing examination is designed so that unqualified candidates pass the examination the public may be harmed. The costs associated with an improperly developed academic examination may be that the student received a lower grade than was deserved, and for an improperly developed employment examination, there may be a delay in employment. However, improperly developed licensing examinations may result in a candidate's inability to work in his or her desired occupation or profession.

CLEAR Exam Review (Summer 1996)
Norman R. Hertz

Question: We have completed an occupational analysis and used the results to develop a test plan for developing the examination. Doesn't our examination meet validity requirements?

Answer: Not necessarily. You have made a good start but if you want to establish the validity of your examination program, you must do more than conduct an occupational analysis and develop a test plan. A series of steps are required after the test plan has been developed. It is important to understand that an examination does not possess validity. Validity of a licensing examination is inferred if the examination tests the job-related competencies shown to be relevant from the results of an occupational analysis. Evidence for validity must be collected in the course of examination development, administration, scoring, and in establishing the passing score. If proper procedures are not applied consistently in any of these areas, then the validity of the examination may be questioned.

In summary, it is necessary to be vigilant at every step in an examination program in order to ensure the validity of the licensing decision. Validity is a function of doing many things correctly.

Back to index

2002 Council on Licensure, Enforcement and Regulation