Type I vs. Type II errors

CFA Level I Quant
#1

I’m kinda struggling with the differences of Type I and Type II errors. Every time I think I get it, I look at it again and I’m confused. I think what is most confusing for me is the double/triple negatives that I’m trying to get my head around (i…e. FAILING to REJECT the NULL hypothesis).

Has someone got a good way of explaining the difference in these errors.

Type I error: the rejection of the null hypothesis when it is actually true.

Type II error: the failure to reject the null hypothesis when it is actually true.

If I say took the example of taking a sample from CFA exam results. Let’s just say that we know the std. dev is 10, What we want to disprove is that the mean of the population is 70. We take a sample of 100 scores. So our std. error is 10 / sqrt(100) = 1. We want to prove this to 5% significance (i.e. 2 tailed test, 1.96 std dev on each side, let’s just round to 2 std. dev for simplicity)

So the null hypothesis is u = 70

Type I error as I understand it:

Our sample returns a result of 65, so the mean should lie between 63 and 67, we reject the null hypothesis. The Type I error in this case would be if it turned out that real population mean was actually 69. We rejected the null hypothesis when it was actually true. The chance of this happening is the 5% (i.e. the chance that for a normally distributed population with mean of 69, that we should have been unlucky enough to pick that sample of 65).

Is the above correct before we move on to Type II error?

Type II error as I understand it:

Our sample returns a result of 70, so the mean should lie between 68 and 72. We fail to reject the null hypothesis. The Type II error in this case would be that the real population mean turned out to be 65. We just failed to reject the null hypothesis because of our dodgy sample.

Is the above correct?

I seem to be able to explain the differences, but there is just something that isn’t clicking for me with these explanations. It’s as if type I and type II errors are simply inverses of each other which I know they are not (i.e. P(Type I) is not equal to P(1 - Type II).

Does someone have a good analogy for explaining this that will make this stick. It’s really hurting my brain each time working this out again.

#2

i have to check my notes , but i believe a type II error is failure to reject the null when its actually false.

#3

Yeah, sorry, typo on my part… Should have been.

Type I error: the rejection of the null hypothesis when it is actually true.

Type II error: the failure to reject the null hypothesis when it is actually false.

#4

Try looking it up on the Khan Academy website. I’d be surprised if there isn’t an explanation video there about this.

#5

Type I error: A null hypothesis was rejected when it was true We rejected a null hypothesis mistakenly when it should not have been rejected Type II error: A null hypothesis was not rejected when it was false We were unable to reject the null hypothesis or we failed to reject the null hypothesis when it was wrong.

#6

Here’s an easy way to remember it. This, from Statistics for Dummies.

A man is tried on criminal charges. The man is presumed innocent unless proven guilty.

Type I = An innocent man is found guilty

Type II = A guilty man is set free.

Society generally prefers Type II errors in criminal justice. Thus, Type I errors are worse than Type II errors.

#7

Thanks Hank. Regarding this analogy. Which is the null hypothesis? I’ve seen this analogy before but what confuses me about it is the Type I and Type II error assignment appears dependent on what you use at your null hypothesis.

So in your example, I’m assuming the null hypothesis is that he is innocent (for the Type I error at least). If our null hypothesis had been that he is guilty, then a Type I error would have occurred if “a guilty man is set free”, not Type II.

If the man is presumed innocent until proven guilty, then shouldn’t the null hypothesis be that his is guilty (i.e. the thing we are trying to disprove), and the alternative is that he is innocent?

#8

The null hypothesis is that the man is innocent. The prosecution has to prove its case (the alternate hypothesis). The null is always the default.

Remember that T1 errors are always worse – we sent a guilty man to jail – we hired a hotshot manager because we thought he was good when in fact it was just luck.

T2 errors are less worse – we let a guilty man go free – we passed on a truly good manager by mistake

Don’t worry if you don’t get this. I was totally clueless when I took Level I. Check out videos from KhanAcademy or bionic turtle (they’re free).

#9

Hank moody - your T1 should be sending an innocent man to jail. Sending a guilty man to jail is not an error (T1 or T2).

Clever, regarding your OP, the fact that actual mean is 69 means that you correctly rejected the null (it is not exactly equal to 70 - which is your null hypothesis). The idea is that there is less than 5% chance of having the true mean equal to 70 given that sample mean is 65 (with a standard deviation of 10).

#10

My mistake. It should read:

Remember that T1 errors are always worse – we sent an innocent man to jail – we hired a hotshot manager because we thought he was good when in fact it was just luck.

#11

Here is my 2 cents that may help others with the same question:

Using a fire alarm as an example:

Type I error: Fire alarm goes off without an actual fire.

Type II error: No fire alarm when there _ is _ a fire.

#12

HI. I am in desperate need of help in this particular topic… Ive been struggling with Hypothesis for a couple of days now… :frowning:

#13

try khanacademy or other youtube channels that teach things simply

http://www.youtube.com/watch?v=FHT6e_mdGoU

:stuck_out_tongue:

#14

I would be very grateful If anyone can shed some light on these couple of problems im having in hypothesis testing

  1. How to decide whether to use a one tailed test or a two tailed test

  2. How to find the critical test statistic using the Z table. (the t students table is so much easier)

Like if it mentions 1% level of signifcance… how do you find the corresponding value on the Z table. I know its 2.58 but i keep getting confused while using the Z table.

I know these are amateur questions :stuck_out_tongue: Will be grateful if anyone can help

Thanks!

#15

points to remember:

  • the Null is the hypothesis being tested

  • the Null will always have an equal sign eg x=0, x is less than or equal to 1 etc. if there is no equal sign, then the given hypothesis is actually the alternative hypothesis and your task is to deduce the null for testing purposes

  • Type 1 error occurs when you reject a correct null

  • Type 2 error is where you fail to reject an incorrect null

  • two tailed test is abit difficult to explain so i will just give an example, if null hypothesis is x=0, then this means that the alternative is x is not equal to 0 ie x can either be greater than 0 or less than 0, so there are two tails- the greater than zero tail and the less than zero tail. but if the null hypo is that x is less than or equal to 0, then the alternative hypo is that x is greater than 0. based on this, there is only 1 tail because there is a a lower limit/bound.

#16

I didn’t understand it until I read Intermediate Statistics for Dummies just prior to Level II. Here is a really easy way to cut through the jargon.

In a criminal trial, a defentent is presumed innocent. Society has decided that it is better to let a guilty man go free than send an innocent man to jail. Therefore, the base assumption (the null hypothesis) is that the person is innocent. Your job as the DA is to prove otherwise (the alternate hypothesis).

In a Type I error, an innocent man is found guilty. This is the worst kind of error. In a Type II error, a guilty man is set free. This is a less serious error.

So in finance, the base (or null) assumption is that a particular manager adds no value – that his apparent outperformance is random luck. The manager’s job is to prove that his outperformance actually is the result of skill. This is the alternate assumption.

Type I: You believe the guy adds value when in fact he doesn’t. Type II: The guy actually adds value, but you failed to recognize it.

Learn this now because this comes up again in Level II, and to a lesser extent In Level III.

#17

Thanks for the link to the youtube. Short and sweat, I like it. I will subscribe to this for later. Thanks!

#18

Thanks for the link to the youtube. Short and sweat, I like it. I will subscribe to this for later. Thanks!

#19

yup keep it simple guys… there are two hypothesis (null/alt) and 4 outcomes…find an example that makes sense to you and simply apply the same method over and over again.