Torsten, I am expecting the confidence intervals to be about +/- 7-8% on the actual exam. That would be about my expectation for a 95% confidence interval for an individual taking similar exams. It would imply a standard deviation of about 3.5% - 4% for an individual.
The use is as follows. If your mock scores lead you to believe that you are 8% above the passing score (about 12% above the mean on many mocks for Level 1 ), you would have about a 3% chance of failing if you don’t make game day mistakes like no sleep, forgetting your calculator, food poisoning, panic attack, etc.
i believe the new exam reports show percent correct on the Y axis, rather than percentile. Especially if near the mean, a 1% increase in score might mean much more in percentile , e.g., move you from 54th to 59th or 60th percentile.
MunicipalBob, we essentially agree based on what you wrote in your last paragraph. What I had meant in the longer post pertained to my personal score and not score of a random individual because I am situated in a less volatile area (assuming approximately standard distribution). By saying less volatile area, I mean lower change in percentile ranking as opposed to change in score as opposed to someone closer to the mean. Given that my score (actual hit rate, not percentile rank) is slightly below 90th percentile (less than 0.5mm, lines almost touch each other) I am assuming it is not more than +/- 3 percentile rankings given the less volatile range. Obviously, the change when going up with score would be smaller than change going down (in percentiles).
Therefore, my point still remains - there is a wild discrepancy between CFAI practice problems percentile ranking and actual exam approximate ranking due to reasons highlighted above, at least in my case. If I had been closer to the mean it would be impossible for me to make such a claim.
At the end of the day you are also right assuming a person scored lower on the exam than I did except that we cannot tell at what magnitude will percentile rank would go up/down on the actual thing for that person. It is hard to tell (probably impossible) whether discrepancy between CFAI practice platform percentile and actual CFA exam result.
With respect to the percentile on the CFAI website, I’m highly skeptical of the score and really not focused on that as a benchmark to how I’m performing. People can use notes, reset questions several times, etc., so I think there is some built in bias in that particular metric. Just my two cents.