I failed band 10 last year and compared my score matrix with other ( few) failed band 10 candidate’s score matrix. Remarkably the score line was around 62-63.333% . So MPS definitely was above 64% 2016 exam for Level 2 but not 70% or above. Yes I used 40/60/80 rule ( tabulated myself) but I don’t think it will produce very significant / different in scores if you use any other methodology. For instance majority of the sections have lower weight , like 6 questions or 12 questions and you don’t have to break you head to infer someone’s score in the particular section. If someone score more than 51% in a 6 questions section , the maximum he/ she could have scored 4 only. FRA and equity may have higher weight but what is the difference once you have crossed the 71%? I don’t think large number of people can score 90 and 100%. Exam year 2013 someone passed with 4 failed sections and that person might have scored 63% at the best . So I don’t buy the argument that MPS should be in the range of 70%. Most of the sections are interconnected ( FRA and Equity, & Derivative and economics and FI) so someone scoring very low in Derivative is not going to score very high in FI or economics ( it means some score 50% in derivatives and scoring 100% in FI is hard to believe).
I agree with a lot of the inference you’re making, but you have to remember that L2 has variability in the number of questions on any particular exam form, so without that information, it makes things less precise.
I think too, that if the CFAI actually did say no one has ever failed with at least a 70% raw score, then this implies that at least some people with just under 70% have failed before, meaning the MPS could have been 70 at some point. I’m not saying the MPS is 70% but I’m saying this argument is more logical because it incorporates more precise data from the source itself rather than low resolution information that we’re not entirely sure how to interpret (again, is >70% out of 100% or relative to some other number such as the average score of the top 10% of examinees?).
I think correlated scores are also probably worth considering as you noted. If someone has most categories <50%, why do we give them the benefit of the doubt of scoring 80% for the section in which they achieved >70%? If someone is consistently showing >70%, I’d think their 50-70% section is closer to 70% than 50%. There are tons of rational arguments for these assumptions, but no one is testing the model before making such exact and certain statements.
It’s really a fruitless discussion, really, because nobody bothers to carry out a proper analysis to test what they’re saying.
someone needs to share a database of mock scores and compare them to 40/60/80 scores to settle this
Huge problem with the mocks scores:
-
people don’t take them at the same point in studying
-
people take a mock that consists of questions they’ve already tackled in old mocks or topic tests
-
people don’t take them under standardized conditions
…I’m sure there are others I’m missing. The other thing is that comparing mock score to 40/60/80 scores would only tell use how well 40/60/80 scores estimate mock scores, but it doesn’t actually tell us how well either allows us to classify passing and failing correctly.
I agree that a database of score matrices would be a step in the right direction. With 200 score matrices and pass fail results for L1, for example, we could use 100 matrices (set A, a “training set”) and results to compute a 40/60/80 MPS. The remaining hundred unused score matrices (set B) can then be used to calculate a 40/60/80 score. This score is then compared to the estimated MPS from set A, and we use this comparison to classify scores from set B as pass or fail. Then, we compare our predictions for pass/fail in set B with the actual pass/fail results of set B. We can then use several statistics to see how well our 40/60/80 MPS classified set B. This is a far better start than what’s posted up on 300 Hours or where ever else, although again, this doesn’t give us as much information on the MPS as it does about the 40/60/80 score as predicting a pass result (which doesn’t have much utility since we know both results at the same time and when you pass, it’s irrelevant).
I don’t really believe in Ethics adjustment to determine the pass or fail. CFA clearly suggests that each question carries equal weight so why do they have to give preference to someone who did well in ethics compared to other candidates who are on the border line?. They may have some adjustment to cover entire exam , not just one area.
I’m only suggesting using mock scores to estimate the reliability of 40/60/80 estimates. I don’t think the issues you bring up should effect that too much. You would then better be able to estimate actual mps scores each year if you had a large enough sample of 40/60/80 scores from the real exam. I assume people like 300hours.com and well connected instructors get hundreds if not thousands of 40/60/80 scores each year.
I agree 100% with ur last paragraph. People underestimate how much volume u must process to earn the Charter. It’s rather sad