Standard setting is a systematic method for determining a passing score on an exam. It is a formal process that involves establishing a performance standard, such as a passing score. The goal is to identify a score that reflects the performance standard and provides a reasonable basis for pass/fail decisions. This is a short-term commitment in February.
Remote Meeting Schedule:
- Friday, February 14 (4 hours)
- Friday, February 21 (4 hours)
Additionally, participants will have time to take the exam at their own pace between these meetings.
The ATCB will award each Subject Matter Expert (SME) 25 CEs for participation.
Complete this form if you are interested in serving as a Subject Matter Expert for Standard Setting.
About the Standard Setting Process
The process is relatively informal, similar to a focus group. The NCCA accredits the BC (and the qualifying exam), and we use the Modified Angoff process for standard setting. Psychometricians and assessment experts from Prometric will lead this effort for the ATCB and support SMEs throughout.
Process Overview:
1. Define The Minimally Competent Candidate (MCC) – First Meeting (February 14)
This concept is central to the modified-Angoff method and is also known as Minimally Qualified Candidate (MQC) or Just Barely Qualified (JBQ). The goal is to ensure the exam distinguishes between qualified and unqualified candidates. SMEs will define what makes a candidate qualified (or unqualified) based on skills and knowledge, leading to a conceptual definition of an MCC. The next step is estimating the score this borderline candidate would achieve.
2. Independent Ratings – Between Meetings
SMEs will review all exam items and estimate the percentage of MCCs that would answer each correctly. Prometric will provide guidance on how to perform these ratings.
3. Discussion and Round 2 Ratings – Final Meeting (February 21)
The group will identify items with significant disagreement based on frequency distributions or standard deviation. If some SMEs rate a question as easy while others find it difficult, the item will be discussed and rated again to build consensus.
After the standard-setting process, Prometric psychometricians will analyze the data and determine a cut score for the exam. This cut score will then be presented to the ATCB Board for approval and adoption.