Am I the only one or is the type I vs type II error distinction kind of arbitrary. One of the EOC questions talks about a manager getting fired because the company falsely thought he was an underperformer. The answer then says this is a type II error, which is defined as failing to acknowledge that he is an outperformer.
I could have just as well argued that this is a type I error, falsely assuming he is an underperformer.
So we should just memorize, that everything is expressed in terms of outperforming. Either:
we assume someone is an outperformer although he is not (type I)
or
we fail to acknowledge someoen is an outperformer (type II)
These type of errors come from statistics. You should be thinking hypothesis testing here. In one case you reject the null when you shouldnât have (type 1 error, very important not to do this, hence the low acceptable range for such errors 1-5%), in the other you fail to reject the null, when you should have, type 2 error). There is a specific way to set-up statistical tests depending on what you want to (dis)prove, so you cannot just rotate the statements as you wish. When you reject a null, there is very strong evidence that the null is not true, while failing to reject the null does not necessarily prove that the null is true.
We had a very nice analogy in statistics regarding the difference of what you accomplish with statistical test depending on the result. Say you have a murder case and a witness saw the murderer who had blue eyes, and the witness says he had blue eyes. When you catch a suspect, if it turns out he has brown eyes, you can safely reject the null that the suspect is the murderer, you have proven he is not, as he doesnât have blue eyes. But if you catch a suspect and he has blue eyes, you fail to reject the null, but you have proven nothing. You could have caught any of the thousand people with blue eyes. You cannot reject the possibility he is the murderer, as the eye color did not disprove it, but you still need other evidence to prove that heâs the right guy.
Thanks guys for the explanations. Nenorr great analogy, I am familiar with the concept of type I and type II errors from statistics. I get all of that,
All I am saying is that the classification is arbitrary.
type I error could just ast well be:
assume someone is an underperformer although he is not (type I)
we fail to acknowledge someone is an underperformer (type II)
The art of rejecting H0 when is false is the Power of Test.
If you fail to reject H0 when is false, this is (1- Power of test) or Type II Error.
Type 1 error is simply rejecting true H0. Thus, Type 1 error - not fire useless manager in this game which simply might be prescribed to just accidentally decision.
More serious is Type 2 error or fire a good manager.
Thanks Flashback. I think I was a bit vague in my question. My problem is not the statistical background, I get the concept of statistical testing etc. but again, what I am saying is that the classification in this context is arbitrary. I just wanted to clarify that point, and have someone (as Edbert did) say, yes it is arbitrary but that is the way it is, so stick with it.
No further explanation needed from here on out. Thanks everyone.
Type I and II errors are not arbitrary. The selection of the null hypothesis may be somewhat arbitrary on the part of the curriculum, but it is consistently applied.
Guys, I understand type I and II errors, all I am saying that they could have just as well decided to reverse the null and use underperformer instead of outperformer as the relevant metric.
You can define the null as:
H0 = Manager adds no value
Ha = Manager adds value
or:
H0 = Manager adds value
Ha = Manager adds no value
Now we can discuss all day long, which one has is the bigger error, and which one should be avoided (letâs please not do that)
What I am saying is that they have arbitrarily decided to go with one of them, and we need to memorize it.
If one more person feels the need to explain what the difference between type I vs type II error is⌠I am going to make her or him watch my 30 min stand up special from high school, and trust me there are a lot of type I and type II errors in terms of jokes that should not have made the cut.
Currently Iâm preparing a FRM and Hypothesis testing is the most easiest part there regardless this is there 2 steps further challenging than CFAI statistics.
Tartaglia, really doesnât matter how you test hypothesis as long as you know what youâre doing. I have an impression that you are just trying to memorize and simplify the rules with no understanding the concept behind.
I am absolutely comfortable with understanding the concept behind it, what I am saying is the hypothesis is set by the curriculum in a certain way, and if you donât know that definition, you can understand it as much as you want, you wonât be able to answer it correctly:
Example:
Q. If the fund adopted a more relaxed appraisal criteria, the most likely impact would be an increase in:
Type II error only. both types of errors. Type I error only.
Solution C is correct. If the fund relaxes the appraisal criteria, they are more likely to make a Type I error, retaining a poor manager. A and B are incorrect because Type II errors would decrease.
If you donât know that Type I errors is defined as retaining a poor manager you cannot answer this.
If I know what I am doing (as you said), I might say Type 1 error is defined as firing a good manager âŚ
Are what youâre sayinâ above the only facts mentioned in question? BTW, I can recall that H0 was - Manager adds no value as I mentioned before while alternative hypothesis was - Manager adds value. Maybe such construction is a custom in Investment Performance Appraisals area. Since, I donât work there, cannot be sure. My last statement was a general opinion about testing hypothesized value.
Flashback, your definition is actually opposite to what everyone in the forum here said, and what the questions in the curriculum said:
See what someone above posted:
Type I error: keep a bad manager
Type II error: fire a good manager
And this is exactly my point. You have memorized it the other way around, and your answer will be different than someone elseâs answer who memorized it the other way around.
It does not mean your answer is wrong, but for the purpose of the exam, it would be the incorrect choice.
I am not playing a blame game here nor am I questioning any ones abilities (congrats to passing)
All I am saying (since my first post) is that the classification is not something one can figure out, but rather something we need to memorize, they just decided that type I error means: keep a bad manager and NOT fire a good manager. And if you donât know that, you can have your doctorate in statistics, but you are not going to be able to answer that question (unless you guess correctly).
And I just wanted someone (as Edbert already did) tell me:
âYeah dawg, you right about that, but donât matter, stop wasting ya time and get busy memorizing it.â
His words, not mine, at least that is what I remember.
Hey so I get your doubt (I had the same doubt initially), let me keep it simple: Type 1 errors include explicit costs and is an error of commission (doing something that was wrong) Type 2 errors include implicit (opportunity) costs and is an error of omission (not doing what was right).
So the decision to not hire or to fire (and later realising that he was an outperformer) would be a type 2 error (as youâre missing out on it, although thatâs an opportunity cost not an explicit cost affecting your performance directly)
I strongly recommend you refer to the section 2.2, book 6 CFA curriculum Level 3, to deep dive further as its given quite well.