Based on your knowledge on exam day....

Blue box on exam results is labeled as “Based on your knowledge on exam day, if you took a similar exam, your score would likely fall within this range”.

Trying to figure out if I’m totally missing something. What they probably should say is “Based on your performance on exam day, your actual knowledge likely falls within this range.” (or maybe not exactly this, but something much more akin to this than what they say; a trial (performance on exam day) tells us something about the distribution (our knowledge), but since it’s a single trial, it doesn’t tell us where that trial falls on the distribution)

Their blue box is essentially assuming candidates performed at their 50th percentile on exam day and that future trials would be evenly distributed around their actual performance. That’s a ridiculous assumption at the individual level. (it’s also assuming that the candidate has ‘average’ variability or variability consistent with their assumptions, but each individuals variability is different)

Another way to look at it is that if you had a 15th percentile performance (for yourself) on exam day, your actual ability is probably at or close to the top of the blue box (i.e., your 50th percentile result). If you had many trials, your performances would generally fall between your exam day performance and 2x to the top of the blue box because you’d have some >50th percentile performances). Or the opposite if you had a 90th percentile performance.

Does anyone else have thoughts on this? And if I’m correct, what should we think about the CFAI exam writers/graders given their inability to get this basic prob/stat stuff right?