Frequently Asked Questions About Licensing Exams

Changes/updates to the occupational/job analysis

CLEAR Exam Review (Winter 1990)
Eric Werner, M.A.

Question: Once an analysis of practice (e.g., a job analysis) has been made as the basis for an examination program, is it necessary to update this work periodically? If so, how frequently?

Answer: Many professions change fairly rapidly, especially as a result of technological progress and modification of the boundaries separating related fields. Practice analyses can be very costly and there is no good reason to update them just because a particular period of time has elapsed. However, as a general rule, I suggest that the need for re-analysis be considered no less often than every five years.

CLEAR Exam Review (Winter 1994)
Norman R. Hertz

Question: We have been told that our occupational analysis is obsolete and that we should have a new one performed for our examination program to be considered content-valid. What kind of standards should we apply to evaluate the quality of an occupational analysis?

Answer: The basis for the construction of a content-valid examination is the occupational analysis. An occupational analysis is a comprehensive survey of various tasks and knowledge and skills required to perform a job. The analysis is designed to single out the critical knowledge and skills required by the occupation, their relative importance, and the level of proficiency required for each task.

First, the list of tasks and knowledge and skills that are developed from on-site interviews and follow-up workshops should be comprehensive. The development of the list should continue until no new or additional information can be obtained. Second, the level of specificity should be consistent for all tasks and knowledge and skills statements. Furthermore, the statements should provide sufficient detail so that they are useful during examination construction, the examination developers must depend on personal experience to place a context around the statement. Also, in establishing the relative importance of each task and knowledge and skill, evaluators must rely upon their personal experiences. If the statements are too brief or vague, the reliability of the evaluation process is called into question.

There are a number of criteria professional publications and case law that occupational analysis should adhere to for results suitable to produce examinations that are content-valid for licensure. (Thompson & Thompson, 1982).

In summary, licensing boards should examine the quality of occupational analyses to be sure that the results provide a comprehensive description of the profession and meet professional and legal standards.

CLEAR Exam Review (Winter 1996)
Norman R. Hertz

Question: The scope of practice is specified in detail in our regulations. However, the results from an occupational analysis identified activities that are not mentioned in the regulations but are being performed by the practitioners. What are the implications of the findings for examination development?

Answer: When the scope of practice is written in great detail in the regulations, it becomes very problematic to incorporate changes in actual practice in examinations. It may mean that some of the more recent aspects of practice cannot be tested until the regulations are changed. A long-term solution to the problem may be to change the regulations so they have a provision that can account for evolution of practice.

Professions are evolving rather quickly as a result of technology, managed care, etc. so requiring changes to regulations long before an occupational analysis is completed may result in delays in tests for competencies that practitioners are required to carry out as a part of their job. There are other issues to be considered, however. When the results from an occupational analysis identify activities that are not mentioned in the regulations but are being performed by the practitioners, it may be a result of practitioners performing services outside their accepted scope of practice. Then, information from the results of an occupational analysis should not result in an immediate assumption that practice has changed. Rather, the information should be reviewed by the licensing authority to determine whether regulatory change is warranted.

Back to index

2002 Council on Licensure, Enforcement and Regulation