Hi, I have been starting to read through quants and just couldn’t understand what the ‘statistically significant’ means. Can anybody help me out?
For example, if the t statistics are NOT in the range of the critical t value, then it is statistically significantly different from zero. Meaning we should reject the null and also conclude that it does not significantly contribute to the subject. But what is this really meaning? Is it simply saying that ‘the sample correlations between X and Y is not a good variable to explain the dependent variable’?
will try to explain as far as possible in English here… (key word being TRY).
you want to say something has a mean of 0.
But the underlying thing you are measuring has its own variance …
so even if you want to say it is zero - you need to account for the variance of the measurement. That variance is provided to you in the form of standard deviation in Statistics world.
so now you say if something lies in the range of 0+1.65 std dev and 0-1.65 std dev - I still consider it to be zero. e.g.
so your t-statistic being in the range of critical t means that what you are measuring lies between MEAN+1.65 std dev and MEAN-1.65 std dev
if it lies in that range - it is still zero.
if it is outside the range - it is NOT zero. (you reject the NULL that the mean is zero - and given that you now brought in the “statistical aspect” by including the std deviation in the measurement) => it is statistically significantly different from zero (hypothesized mean).
I’ll use the example of a hypothesis test with the correlation coefficient.
Say u have some paired data and u find the “r”. Now to prove that it’s an imp value we need to do some hypothesis testing coz well we can’t just simply use it. We need some evidence via stats that it’s good to use. The number may be very high no doubt, but yet needs to be statistically proved that it’s imp.
Hypothesis testing involves the null (Ho) and the alternate hypothesis (Ha).
The null is assumed to be that the r = 0. Ie the correlation is of no use. Its like the default that somethings of no use and we need to prove that something is indeed of use. Hence now the onus is on us to disprove and reject that the Ho is false and on the flip side the alt hypo is true. Ie the r is not = 0 and is the correlation that we’ve found is imp. Of course this is done taking into account the level of significance.
With a certain level of significance ull get a range of values. If the t stat thats been computed is within that range then statistically that value is of no practical use. It’s as good as 0. We want some exceptionally great value thats not encountered in the normal course. That’s when the t stat we’ve calculated is outside the limits decided by the level of significance. It’s then we say yup that the correlation is not 0 (and hence statistically significant).
Just want to point out one mistake in your original post. Rejecting null doe NOT necessarily mean statistically insignificant. It is rather the contrary. If null hypothesis is “b1=0” then rejecting null means b1 is “statistically significant” (usually at 95% level). In this case, we are 95% sure it is not zero. But it could be zero anyway in 5% chance.
In the spirit of pointing out mistakes, you are incorrect that “…we are 95% sure it is not zero. But it could be zero anyway in 5% chance.” We would be 95% confident that it is different from zero, but this does not mean there is a 95% chance it is not zero (or a 5% chance it is zero). The confidence level refers to the long-run performance of the method, not “how sure” we are about where the parameter value is located. This is a common misconception regarding the interpretation of confidence intervals and p-values. The true coefficient is either different from zero or it is not. There is no “in-between” probability statement that can be made with the traditional confidence interval or p-value. Wikipedia has these common misconceptions, but I’d be happy to provide some additional sources if requested. Point #1 directly covers this from a p-value perspective and # 19 covers it a little less directly from the CI perspective.