Pass rate of L1 at 49%and L2 at 55% what for L3?

CFA Level II
#1

2020 may be over but the surprises keep coming back. L1 at 49% and L2 at 55% . What L3 can expect ?

60% ?
65% ?
70%+ ?

1 Like
#2

One possible explanation for the bump in pass rates is because many people who were supposed to take the exam in June had their exam canceled because of covid, and as a result, they had an additional 6 months to prepare, which means come December, some of the candidates were super-ready, and some of them may be repeat test takers. The overall pass rate is not too high because some of the candidates are taking it for the first time.

It’s interesting to note that the 2019 pass rate for level 2 was 44%, which means the pass rate increased by .55/.44 -1 = 25%. The pass rate for cfa level 3 exam in 2019 was 56%, which means a 25% boost would result in a pass rate of 70%. I think the actual pass rate will be lower, however. Somewhere around 64%-68%.

cfa pass rate historical data

edit:
By the way, I tried modeling the ratio of the pass rates for the year 2020 to the pass rates in 2019 (a known constant) as a random variable that is approximately normally distributed with unknown variance. Because there are only two samples available (those for level 1 and 2 exams), this is a T-distribution with 1 degree of freedom. Unfortunately, because the sample size is too low, to get a 95% confidence interval requires using a critical value of 12.706 (which is ridiculously large), resulting in a confidence interval for the mean of the ratio as (.874, 1.571). That translates to a 95% confidence interval for the level 3 pass rates as being in the interval (48.9%, 88.0%). Obviously, that’s not very helpful. We need more data. With two more samples, the sample size would be 4, the degrees of freedom would be 3, and the critical value for the T-distribution would decrease to 3.182 - still large, but far more reasonable. Assuming the sample mean and the unbiased sample deviation remained the same, the corresponding 95% confidence interval for the mean of the ratio would be
(1.161, 1.284), and the 95% confidence interval for the pass rates for level 3 in 2020 would have been (65.0%, 71.9%). Assuming, of course, the normal distribution assumption holds.

More data is needed. Maybe people should petition CFAI to make a level 4 and a level 5 exam?

#3

:grinning: you don’t give up do you ?

#4

I was originally going to try and model the pass rates for level 3 as a beta distribution, and try to use MLE to get estimates for a and b (setting theta to 1) using the most recent historical data. However, because the recent pass rates for level 1 and 2 suggests there will be a jump in pass rates, this method doesn’t work. :slight_smile:

edit:
wait. what if we model the difference between the pass rates for level 3 and the pass rates for level 2 as a random variable? There is some data available, which means we might get more accurate(?) results.
From year 2008 to 2019, the pass rates for level 2 in 2019 was (in percent):
{46, 41, 39, 43, 42, 43, 46, 46, 46, 47, 45, 44}, and for level 3 the pass rates in the same year was (in percent)
{53, 49, 46, 51, 52, 49, 54, 53, 54, 54, 56, 56}

the difference in pass rates for leve 3 and level 2 is
{7, 8, 7, 8, 10, 6, 8, 7, 8, 7, 11, 12}
if we assume the difference is a random variable that is approximately normal with unknown volatiliy, we can use T-distribution. There are 12 samples with sample mean 8.25. With 12 samples, the degrees of freedom is 11 and the critical value for a 95% confidence interval for a T-distribution with 11 degrees of freedom is 2.201. The unbiased sample deviation is 1.815338686
the margin of error is 2.201*1.815338686/sqrt(12) = 1.15341895
A 95% confidence interval for the mean of the difference is 8.25 ± 1.15341895
or (7.09658105, 9.40341895)

Because the pass rate for level 2 in 2020 was 55%, using this method, a 95% confidence for the pass rates for level 3 would be (62.10%, 64.40%)

1 Like
#5

update: the results for CFA L3 is out!

this is significantly less than the T-distribution model predicted

#6

Because your fundamental assumption was wrong. I never asked you to base your estimate on LII results. That is always going to be incorrect in any situation. LII and III are independent event wrt each other. You cannot draw an estimate based on LII pass rate. Even if it did coincide with the actual result it would have been spurious.

#7

What seems surprising though, is that despite the extra 6 months people had to prepare, the pass rates remained largely unchanged. It’s still 56%, just like in 2019. I assumed there would be a significant jump in pass rates, just like for levels 1 and 2, and took the difference to try and take into account said jump. That did not occur.

By the way, it’s possible that the main assumption - that using past data can be used to infer future pass rates - is itself flawed. In this case, even predictions based on L3 pass rates would be incorrect. If this is the case, the pass rates are more like a random walk.

#8

In one of the sub Reddit I read a candidate claiming that he/she had left 60 points of the AM on the table and yet passed.60 points is 33% of the total AM points. He although also claimed to have ‘crushed’ the PM session.

But even then, it surprises me no ends how on earth can one pass having left 60 points blank ? Was it , the MPS was in early 50s? It is getting intriguing with every passing day. With no granular score released there’s no estimation of MPS either. Go figure.

1 Like
#9

On the topic … a small exercise can be done.
We take all pass rates of Level III from 1963( this again may be incorrect for secular reasons) or at least from 1990( so as to have 30 data points) and then start working them:

  1. We check the probability distribution of the data and present the ANOVA table

  2. We create the time series in case data are not normal.

  3. We take at least 2 lags

  4. Before steps 2 and 3 we check for all the hygiene points of the data to establish randomness and eliminate heteroskedasticity( employ white correction, first difference ), sr. Correlation. And covariant stationarity.

  5. In case it is non stationary then we would search for white noise and Random walk

  6. The residual of the data then should be good enough for forecasting one period ahead.

1 Like