|
How Were the NAFC Examinations Developed?
The Certification Commission of NAFC the
American College of Certified Forensic Counselors
promotes a minimum competency standard for the forensic counselor profession and
sub specialties,
and serves the public through its quality assurance policy and procedures.
The Certification Commission, retains and works with several consultants, and
develops all of our examinations using test
methods based on the American Psychological Association's Standards, and The
Equal Employment Opportunity Commission's Uniform Guidelines for Employee
Selection Procedures.
Examination development is a long complex process. It required rigorous testing
of the examinations validity. The validity of an examination involves the
demonstration of at least two major qualities:
1). The content of the examination must be job related.
2). The examination covers areas where a lack of knowledge could cause harm to
the public.
Another characteristic that is important when determining the quality of the
examination involves how reliable, or consistent, the examination is.
Reliability is an index of how accurately the examination measures the
candidates skills. A well developed examination is both valid and
reliable.
The NAFC written examinations assesses basic knowledge in area of forensic
counseling. The examination consists of 150 multiple choice questions which take
approximately 2 ˝ hours to
complete.
The multiple choice questions cover the major content areas delineated in the
Role Delineation Studies. The Role Delineation Studies are updated every 3-5 years
to stay abreast of changes within the profession.
The process of validating the content of the NAFC examinations involves a
meticulous step by step process which included:
Step 1. The Role Delineation Study.
Step 2. Development of a test blueprint.
Step 3. Item Development and validation.
Step 4. Pre-testing of all test items.
Step 5. Examination Assembly
Step 6. Review and Revision
Step 7. Content valid examination
Step 8. Determining a cut score
Step 9. Test administration
1. The Role Delineation Studies involve subject matter experts (SMEs) from the
NAFC Certified membership in each certification category to categorize the job
characteristics in the area of their certification. The a validation survey is
developed and sent to approximately 1,000 practicing practioners within their
area of certification.
2. Based on the results of the Role Delineation Study, a test blue print is
developed as a guide to determine the content of the examination.
3. The items or the questions are written by a panel of NAFC certified members
from various areas around the United States, the questions pertain to the
domains, tasks, skills and knowledge required outlined in the test blueprint.
All items had to have at least two verifiable references. In addition, the items
are again reviewed by three different SMEs.
4. Each examination includes field testing items which are not part of the
scoring prescription for the examination. The testing items are reviewed for
performance issues, and if none exist, the items then become available for use
as scored items on the examination.
5. Then the examination is assembled by the SMEs who select the appropriate
number of items identified in the role delineation study in each content area and review the examination to verify the
accuracy of each item.
6. Upon completion of the first five steps in the process, a content valid
examination has then been assembled.
7. The NAFC examination is reviewed annually in order to make sure that it
continues to be a valid measure of the examinee's ability. All previous items
from the previous version are carefully reviewed and statistically analyzed. The
NAFC utilizes a Scantron machine with
Parscore software for its item analysis to determine if a question is
inappropriate or questionable. If they are, they are reviewed.
8. Every valid examination must have a defensible passing score based on the
minimum level of knowledge required to protect the examinee and other
stakeholders. The method utilized by NAFC to determine the cut score is called
the modified Angoff method. This method is the most common method utilized by
certification and licensing boards and is considered to be one of the most
defensible criterion referenced methods for setting passing point scores.
9. NAFC test administration procedures ensure consistent, comfortable testing
conditions for all examinees. Security measures are in place to protect the
examinations integrity and other aspects of administration. All testing
facilities must meet NAFC guidelines including ADA
compliance.
10. The NAFC examinations are scored by NAFC at their home office in Indiana.
Tests are scored by a Scantron scoring machine utilizing Parscore software which
allows for psycho-metric analysis of the examination, including item analysis
which gives information on how
examinees responded to each question for purpose of assessing how difficult the
question is, how the question distinguishes between knowledgeable and
unknowledgeable examinees, verifying that the score key is correct, and how
candidates overall performed on the examination.
In addition to content validity, the examination must be reliable. If the test
is reliable, an examinee should be able to obtain the same or similar score at
two different testing times. Statistical calculations of reliability estimates
range from 0-1. Coefficients above .70 meet
minimum standards. A high reliability for an examination, like that of the NAFC
examinations, is considered by most to .80 or above.
One of the measures of reliability that is used by NAFC is called the Kuder-Richardson
Formula (KR-20) which measures the internal consistency of the examination. It
provides information on the ability of the test to measure a single content
area. The better the examination is in measuring a single content area, the
higher the reliability.
All statistical areas within the NAFC examinations are closely monitored.
All of the procedures utilized in the NAFC examination development are well
accepted for the development of reliable and content valid examinations. Each
step in the test construction is very carefully documented, and reviewed by
subject matter experts as well as external consultants. The NAFC examinations
can be considered valid and reliable with a sound method for determining the
passing scores.
Certification Application
|
