Standard Setting 2025
Final Call to be a Subject Matter Expert for ATCBE Standards
Standard setting is a systematic method for determining a passing score on an exam. It is a formal process that involves establishing a performance standard, such as a passing score. The goal is to identify a score that reflects the performance standard and provides a reasonable basis for pass/fail decisions. This is a short-term commitment in February.
Remote Meeting:
Friday 2/14 (4 hours)
Friday 2/21 (4 hours)
And time to take the exam at your own pace between these meetings
The ATCB will award each SME 25 CEs for participation.
Please complete this form if you are interested in serving as a Subject Matter Expert for Standard Setting.
A bit more about the standard setting:
The process is relatively informal, almost more of a focus group. The NCCA accredits the BC (and the qualifying exam), we use the Modified Angoff process for standard setting. Psychometricians and Assessment experts from Prometric will lead this effort for the ATCB and assist the SMEs throughout.
The process will involve:
1. Define The Minimally Competent Candidate (MCC): the first meeting on 2/14
This concept is the core of the modified-Angoff method, though it is known by various terms or acronyms, including minimally qualified candidates (MQC) or just barely qualified (JBQ). The goal is that we want our exam to separate qualified candidates from those who are not. SMEs define what makes someone qualified (or unqualified!) from a perspective of skills and knowledge. This leads to a conceptual definition of an MCC. We then want to estimate what score this borderline candidate would achieve, which is the goal of the remainder of the study.
2. Ratings: Independently done after the first call.
SMEs read through all the items on the exam form and estimate the percentage of MCCs that would answer each correctly. Prometric will review how to rate with all the SMES.
3. Discussion and round 2 ratings: the final meeting on 2/21
Identify items with the most disagreement (as defined by grouped frequency distributions or standard deviation). For example, a few SMEs thought the question was easy and gave it a high score, but others thought it was hard and gave it a lower score. Items are then rated again based on the discussion. The goal is that there will be a greater consensus.
The psychometricians at Prometric will then take all the data from the standard setting and identify a cut score for the exam. The cut score will be presented to the board for approval and adoption.
