In the past month I posted 2 articles covering Level I and Level II learnings from the 2012 exams. In this article the key learnings from Level III are covered.
Below is a visual representation of the sample we used for the Level III analysis. Although we had great participation from the 2012 exam, the difference between candidates was so slight that the MPS range this time is fairly large.
You can see how tight candidates’ scores were in the chart below. The upper and lower boundaries of the vertical lines represent the minimum and maximum scores for each individual candidate, with red columns representing a candidate that failed, and green columns representing candidates that passed.
96% of You Did Worse in the Essay Section.
A feature of the Level III exam that will be new to candidates is the essay format. As most of you will know, the essay format is something that candidates should understand thoroughly and early in their prep process.
In analyzing the results data sample, we made a startling discovery. We found that 96% of all candidates in our sample scored lower than their item-set section.
Understand the essay section early in the game, and minimize that impact.
MPS - Within 49-68%, Probably On the Upper End.
Based on our sample, we were able to deduce a possible range where the MPS lies. Based on further analytics, our hypothesis is that the MPS is most likely on the upper end rather than the lower end. Yes, we know that’s a range wider than the Canyon. The issue that led to the wide range was mainly that the differences between Level III candidates are so slight that there is only small differences (score category wise) for us to work from.
Level III is Highly Competitive. Band Differences Are Very Slight.
Level III is extremely competitive. The candidates have all studied as hard as they can, and it shows in the band differences.
The band difference is defined by the average differences in scores between failing bands, measured either in percentage points or number of questions. Level III has the lowest band difference among all the levels. Level I had a band difference of about 3-4% (or 8-11 questions), and Level II had a band difference of about 2-4% (or 3-5 questions). Level III has a band difference of just 1-2%, or 1-2 item set questions worth.
i.e. You’re probably 1-2 questions away from the next band on average. Again, Level III is close.
Less Topics to Exploit - Most Candidates Cover Everything Thoroughly.
In Level I and Level II, there were some topics where passing candidates did well where failing candidates didn’t. The key learning from this was that failing candidates often did not master topics that were heavyweights in the exam, and thus their overall score suffered.
In Level III, owing to the uneven coverage of topics and the veteran status of candidates, the performance between pass and fail are quite evenly distributed. Everyone’s covering topics as they should - the differences comes from overall thoroughness.
Multiple Problem Areas, But The Ethics Section Still Prevails.
Ethics, consistent with our Level I and Level II observations, saw a solid performance by candidates. However there were quite a few problem areas:
- Alternative Investments
- Fixed Income
- Portfolio Mgmt - Execution
- Portfolio Mgmt - Individual
- Portfolio Mgmt - Institutional
How was your own experience?
For more details on Level III: