Frequently Asked Questions About Licensing Exams

Frequently Asked Questions About Licensing Exams

Developing an occupational/job analysis

CLEAR Exam Review (Winter 1994)
Norman R. Hertz

Question: We have been told that our occupational analysis is obsolete and that we should have a new one performed for our examination program to be considered content-valid. What kind of standards should we apply to evaluate the quality of an occupational analysis?

Answer: The basis for the construction of a content-valid examination is the occupational analysis. An occupational analysis is a comprehensive survey of various tasks and knowledge and skills required to perform a job. The analysis is designed to single out the critical knowledge and skills required by the occupation, their relative importance, and the level of proficiency required for each task.

First, the list of tasks and knowledge and skills that are developed from on-site interviews and follow-up workshops should be comprehensive. The development of the list should continue until no new or additional information can be obtained. Second, the level of specificity should be consistent for all tasks and knowledge and skills statements. Furthermore, the statements should provide sufficient detail so that they are useful during examination construction. Tasks or knowledge and skills statements with only a few words are not adequate because during the examination construction, the examination developers must depend on personal experience to place a context around the statement. Also, in establishing the relative importance of each task and knowledge and skill, evaluators must rely upon their personal experiences. If the statements are too brief or vague, the reliability of the evaluation process is called into question.

There are a number of criteria in professional publications and case law that occupational analysis should adhere to for results suitable to produce examinations that are content-valid for licensure.

The knowledge and skills tested must be important or critical and not peripherally related to effective performance.
Subject matter areas of the examination should be accurately weighted to reflect the relative importance of the attributes that they purport to test.
The level of difficulty of the examination should match the minimal competence level.
Job analysis interviews should cover the full spectrum of tasks performed by licensees.

In summary, licensing boards should examine the quality of occupational analyses to be sure that the results provide a comprehensive description of the profession and meet professional and legal standards.