They will tell you the confidence level and by the type of hypothesis tested you will need to correctly choose the critical value.
You can be the founder of psychology of statisticians but I don’t want to become guardian of statistics. I think it is not necessary to continue this discussion. I withdraw.
But it is good for the community in this forum if you can answer my questions I asked you.
I’m not saying that these studies are why 0.05 should be the commonly accepted level. I am saying these studies seem to indicate that people tend to be comfortable with this threshold. Therefore, if you see someone saying “…it’s a commonly accepted guideline…” you can understand it’s quite possibly due to human psychology. I don’t disagree that 0.05 is somewhat arbitrary, but it is commonly used and set as a standard in many fields of research (whether it makes sense or not). Keep in mind, I never said that alpha should be 0.05 for all tests (I even clearly explained this later). Also, I could search these studies for you, but that would detract from your learning process. 
I’m glad you agree that there is a tradeoff when selecting alpha. We don’t seem to be saying anything different. (Also, you and I could both pick a different alpha value, and each could be “reasonable” based on our assessment of the situation and needs.) I don’t think that you should have anyone guess what your significance level is-- the book is just sloppy (as we pointed out, so why let it bother you?). As you noticed in my post, I said that alpha depends on the situation and should be carefully chosen to reflect the situation. Unfortunately, many people don’t do this (including the CFA Institute). Statistics isn’t meant to be done with a cookbook approach–doing so will probably lead you astray.
We all recognized that the book isn’t the best source for statistics, so I think there really isn’t anything we disagree on for this.
-
It’s really simple-- these are “generally” acceptable levels for alpha or they want to include some of these for teaching purposes. Most statistics books will tell you that the rule of thumb is an alpha between 0.01 and 0.1, but this can be different depending on the book and the field of research it’s geared towards.
-
If someone asks me to do a test I think of the situation. So, no, I would not automatically put 0.05-- that might be my choice after a bit of thought, though.
You are strong :). I understand you more now.
Anyway, thanks for your reply (not because of your answers, but because you did all what you can).
You are that kind of person that demands a lot but offers a lot less.
Can you prove what you said? Or just because your can not control yourself?
Maybe this phrase makes Harrogath thought that I really profited the “knowledge” of tickersu.
I must explain now what I wrote to be more clearly (I really don’t want to do that, because it will affect the sense of humor of this phrase, and it makes me feel rude). In fact, I pardon his “answers” because I know his limit and he already made all effort.
I’m sorry I couldn’t be more help to you. You seem to have much greater knowledge and experience with this than any of us do. I’m afraid the OP won’t receive adequate help unless you provide it.
You try to accuse me, but what you told is unfounded.
If you want to prove that you can do what you told, ok read what you wrote
Have you shown us these studies? Please don’t worry about my learning process.
and, you wrote
which means what these studies are irrelevant what we are discussing here. Even if they exist, can you tell me what you want to deduce from?
I don’t know whether you are serious or just trolling.
At this point, I’m afraid to ask.
What’s the significance of 0.05 significance?
By Carl Anderson VP, Head of Data and Analytics, WeWork, NYC.
Why do we tend to use a statistical significance level of 0.05? When I teach statistics or mentor colleagues brushing up, I often get the sense that a statistical significance level of α = 0.05 is viewed as some hard and fast threshold, a publishable / not publishable step function. I’ve seen grad students finish up an empirical experiment and groan to find that p = 0.052. Depressed, they head for the pub. I’ve seen the same grad students extend their experiment just long enough for statistical variation to swing in their favor to obtain p = 0.049. Happy, they head for the pub.
Clearly, 0.05 is not the only significance level used. 0.1, 0.01 and some smaller values are common too. This is partly related to field. In my experience, the ecological literature and other fields that are often plagued by small sample sizes are more likely to use 0.1. Engineering and manufacturing where larger samples are easier to obtain tend to use 0.01. Most people in most fields, however, use 0.05. It is indeed the default value in most statistical software applications.
This “standard” 0.05 level is typically associated with Sir R. A. Fisher, a brilliant biologist and statistician that pioneered many areas of statistics, including ANOVA and experimental design. However, the true origins make for a much richer story.
Let’s start, however, with Fisher’s contribution. In Statistical Methods for Research Workers (1925), he states
- The value for which P=0.05, or 1 in 20, is 1.96 or nearly 2; it is convenient to take this point as a limit in judging whether a deviation ought to be considered significant or not. Deviations exceeding twice the standard deviation are thus formally regarded as significant. Using this criterion we should be led to follow up a false indication only once in 22 trials, even if the statistics were the only guide available. Small effects will still escape notice if the data are insufficiently numerous to bring them out, but no lowering of the standard of significance would meet this difficulty.
The next year he states, somewhat loosely,
- … it is convenient to draw the line at about the level at which we can say: “Either there is something in the treatment, or a coincidence has occurred such as does not occur more than once in twenty trials.”…
- If one in twenty does not seem high enough odds, we may, if we prefer it, draw the line at one in fifty (the 2 per cent point), or one in a hundred (the 1 per cent point). Personally, the writer prefers to set a low standard of significance at the 5 per cent point, and ignore entirely all results which fail to reach this level. A scientific fact should be regarded as experimentally established only if a properly designed experiment rarely fails to give this level of significance.
(See http://www.jerrydallal.com/LHSP/p05.htm)
And there you have it. With no theoretical justification, these few sentences drove the standard significance level that we use to this day.
Fisher was not the first to think about this but he was the first to reframe it as a probability in this manner and the first to state this 0.05 value explicitly.
Those two z-values in the first quote, however, hint at a longer history and basis of the different significance levels that we know and love. Cowles & Davis (1982) On the Origins of the .05 level of statistical significance describe a fascinating extended history which reads like a Whos Whos of statistical luminaries: De Moivre, Pearson, Gossett (Student), Laplace, Gauss and others.
Our story really begins in 1818 with Bessel who coined the term “probable error” (well, at least the equivalent in German). Probable error is the semi-interquartle range. That is, ±1PE contains the central 50% of values and is roughly 2/3 of a standard deviation. So, for a uniform distribution ±2PE contains all values but for a standard normal it contains only the central 82% of values. Finally, and crucially to our story,
- ±3PE contains the central ~95% of values. 1 - 0.95 = 0.05
- People like Quetelet and Galton had tended to express variation or errors outside some typical range in terms of ±3PE, even after Pearson coined the term standard deviation.
There you have the basis of 0.05 significance: ±3PE was in common use in the late 1890s and this translates to 0.05. 1 in 20 is easier to interpret for most people than a z value of 2 or in terms of PE (Cowles & Davis, 1982) and thus explains why 0.05 became more popular.
In one paper from the 1890s, Pearson remarks on different p-values obtained as
p = 0.5586 — “thus we may consider the fit remarkably good”
p = 0.28 — “fairly represented”
p = 0.1 — “not very improbable that the observed frequencies are compatible with a random sampling”
p = 0.01 — “this very improbable result”
and here we see the start of different significance levels. 0.1 is a little probable and 0.01 very improbable. 0.05 rests between the two.
Despite this, ±3PE continued to be used as the primary criterion up to the 1920s and is still used in some fields today, especially in physics. It was Fisher that rounded off the probability to 0.05 which in turn, switched from a clean ±2σ to ±1.96σ.
In summary, ±3PE --> ±2σ --> ±1.96σ --> α = 0.05 more accurately describes the evolution of statistical significance.
You two must learn how to make argument. Don’t try to accuse others without proof.
And there are many questions that you can’t answer in order to prove you are right. For example
Really I don’t want to waste my time for you two. I must have spent my time for learning real knowledge
Coastie, what you show us makes sense. And really thank you for this.
I have another link
http://stats.stackexchange.com/questions/132536/how-to-choose-a-confidence-level
Just for knowledge, if you (not just Coastie, but others in this forum) can give us you opinion about what Tim (in this link) said. If you can not, no problem for me (I write that for prudence, because the two may accuse me to profit your knowledge).
Opinion on which part? The confidence level a person chooses to use is rather subjective & depends on the application, unless explicitly stated by regulation.
Thank you for your confirmation.
Your reply will be useful for the two others.
You’re making the mistake of engaging him. The article you posted is a quick google search, as is his stats stackexchange post, but he wasn’t satisfied until someone else did that trivial search for him. Everything that’s been given as a reference is congruent with what I said and with what you are saying: historical reasons, finding a balance that depends on the person and situation, and as I said “other considerations” (which were mentioned in the articles like sample size, effect size, and others)…
tickersu, really I don’t know what is your point. Y ou changed your opinion every post (I just realized that few minutes ago when I read from the beginning of the topic).
Your first post
After that, you disagreed with me and you defended Harrogath 's opinion (about the alpha of 0.05).
Now, you change one more time your opinion.
And I really don’t know where is my mistake. I propose you go to the beginning of the topic and read what you wrote, what I wrote.
Anyway, this topic takes me to much time (in fact, it is a little useful for me because my speed of writing increases considerably :d ) . If you change your mind, there is no problem (Your have the right to change your opinion). If you don’t agree or agree with me, there is no problem neither. And me, I stop here now.
I never changed my statements at all (we just discussed multiple topics). I agree with you that the CFA Institute book is bad for learning statistics. I defended Harrogath to say he was telling the OP not to worry about the book, because the exam will be more clear. I then told you that alpha (and the confidence level) is selected by weighing benefits and costs which depend on the scenario and can vary depending on who is assessing the benefits and costs. I never said alpha 0.05 should be the threshold, but I tried to explain to you why many people do use it as the threshold (whether or not it’s logical). None of these things conflict. I don’t think that I need to reread the thread. Like I said before, I don’t think we are disagreeing on anything. I think if you reread the posts, you’ll see that I said things pertaining to different topics, but none of them conflict with one another.
I don’t know how else to explain it after that. 