Does anyone understand this assumption under the linear regression?
R2=RSS/SST
This is a silly question, just ignore it 
But if someone can tell me what are the assumptions for the linear regression model?
I’m not sure that I understand your question correctly. Are you just asking very broadly what the necessary conditions are in order to extract meaningful information from your data using linear regression? (e.g. residuals have an expected value of zero and are normally distributed). Or are you getting at something in particular?
You can just google the classical linear assumptions. But basically it’s something like the following:
-
The model is assumed linear.
-
There is no correlation between any of the independent variables.
-
The residuals are independent of the independent variables.
-
Residuals are independent and identically distributed.
-
Residuals are normally distributed.
-
Residuals are homoskedastic.
Hope this helps you.
Thanks, this is what I’m looking for. I will add the points that I figured out while solving so that it can be useful to others as well.
-
The model is assumed linear.
-
There is no correlation between any of the independent variables.
-
The residuals are independent of the independent variables.
-
Residuals are independent and identically distributed.
-
Residuals are normally distributed.
-
Residuals are homoskedastic.
-
Residuals are mean reverting but are not equal to zero
If you talk to someone with a PhD in statistics, there are only 4 regression assumptions: 1) the mean of the errors is equal to zero; 2) The variance of the errors is constant (homoscedasticity) for all settings of the X variables; 3) the errors are independent of one another; 4) the error distribution is approximately normal.
It is a common misconception that there are any more assumptions for linear regression than the 4 assumptions I mentioned above. Econometricians and non statisticians like to add other statements that aren’t true assumptions of linear regression (when you derive the model and the estimators, or their test statistics, you see exactly what is an assumption and what isn’t). For the purposes of the exam, know the CFA curriculum, but for real life, grab a statistics textbook written by someone with a PhD in statistics.
The CFA quant section was written by a few PhDs too. Maybe they had to dumb it down for us CFA students…
Correlation and Regression by Richard A. DeFusco, PhD, CFA, Dennis W. McLeavey, CFA, Jerald E. Pinto, PhD, CFA, and David E. Runkle, PhD, CFA
I’m pretty sure they would want to assume that the model is linear and that the matrix is full rank. If you add those two to the 4 you listed you recover the 6 I gave.
- A PhD.
Not a single one of those authors has a PhD in statistics. Learning a topic peripherally as part of another degree program doesn’t lead to the same level of understanding had you obtained the knowledge in direct pursuit of that field, generally speaking.
The CFA quant sections are subpar in many aspects.
Since you’ve brought it to the table, what kind of doctorate do you hold?
Linearity is not a formal regression assumption in the sense that the E~i.i.d. N (0, sigma2|X) are assumptions (linearity is actually encompassed in the mean error equal to zero). A full rank matrix is also not a formal assumption of linear regression. The model can be fit with nonunique estimates in the case of perfect collinearity, for example, but this isn’t a formal assumption.
I’ve had this discussion a handful of times with different people who hold a doctorate in statistics. Economics and finance people like to get going about those two additional points but I have yet to see a formally trained and educated statistician consider them real assumptions, nor have I found a statistics textbook/paper (not econometrics, not statistics for finance) that mentions more than the 4 I listed.
Mathematics.
Your statements are misleading. What the statisticians are saying is that you can fit additional models and in those cases you make a different model assumption than a line. Nevertheless, you make a modeling assumption. Anyone, statistician or otherwise, will concede that point: before a model is fit, the form of the model is chosen. To say otherwise is flat out wrong. Otherwise, the term least squares has no meaning (least squares to what?).
Regarding non-uniqueness of solutions, that’s true, if you are comfortable with a space of solutions, then the matrix doesn’t need to be full rank.
THAT ALL SAID, you made the statement that *linear* regression doesn’t have the 6 assumptions that I said. That is what I object to. Linear regression has a specific meaning that’s different from actual least squares regression, so all of your counterpoints to me are not relevant in that context.
Linear regression specifically refers to regression with the model selected to be a line; hence the assumption that the model is a line. To say otherwise is just flat out wrong. So my first assumption is correct. To challenge that assumption is to challenge the underlying model in the first place.
Regarding rank of the matrix, in this context if the matrix is not full rank you no longer have a best fit line, which again is fine but loses the meaning of linear regression. No one is interested in a best fit hyperplane.
Lastly, since you brought it up, I’m guessing you don’t have a statistics PhD (otherwise you would claim the knowledge as your own instead of appealing to authority). While I understand everything you’re saying, you have to keep the context in mind. 1) you are almost certainly misrepresenting what those statisticians say are assumptions of linear regression and 2) general regression is not something that anyone in this board cares about so bringing up that subtly is likely to cause more confusion than anything else.
What area of specialization?
One of these days I want to finish up a PhD in math: algebraic topology (specifically, knot theory). There are a couple of professors at UC Davis who would be good advisors.
Algebraic K-theory, so algebraic topology was a subject I know and love. To make the connection, K_n groups are basically higher homotopy groups (pi_{n+1}). I haven’t done it in a while, I made the switch to finance after I finished the PhD because I didn’t really want to stay in academia. But I always love chatting math. 
I don’t know much about knots though, but if you want to discuss more details feel free to drop me a PM.
Knot theory is awesome. My favorite is the palomar knot. Super strong and simple to remember, in theory at least.
Hahaha I love it!
Refreshing (non sarcastic). So I can understand your perspective about assumptions.
I agree, but isn’t it a tautology to say linear regression assumes E(Y) is linear in X (which is encompassed by the formal assumption of mean error of zero)? Further, many cases of modeling the form of the model evolves after some model is fit. Whereas the assumption of independent errors is given to obtain consistent and unbiased parameter estimates.
Generally, I try to avoid it, and SAS is nice enough to warn you when you accidentally enter all k dummies instead of k-1 for a k-level variable.
I see what you’re saying, sure I agree with that. Least squares regression has the 4 assumptions I mentioned.
Absolutely. I don’t have a PhD in statistics (nor mathematics), so I’m sure you could hold a better technical conversation with a PhD statistician than I could on some of the technical aspects. However, I had shown the CFA text to one of them questioning what I had seen and learned, and he disagreed with what they had written (and I discussed it with another PhD Statistician, with the same result). I’ve never been in a classroom or read a statistics text where they’ve considered those to be true assumptions but that may be the distinction of least squares simple/multiple regression in comparison to a linear regression. Given this, I don’t believe I’m misrepresenting what they said since they said they disagree with the written text. I think it’s fair to bring up that the scope of the curriculum is narrow and frequently incorrect (for a while they interpreted p-values as probability of Ho being true or a specific 95% CI as having a 95% chance of containing the true value). So, I think directing people to the curriculum for the test is fair while pointing out that it wont get them far in real life.
One of the most useful fishing knots I have learned.
The question around tautology, I don’t think so. The statement that E(Y) = AX (which is probably more correct) assumes that the model was originally of the form:
Y = AX + epsilon
So saying that the mean of the epsilons being 0 is the same as saying the model is linear isn’t correct because it’s logic is circular. That’s precisely why I’m being strict on this point. The assumption that the mean of the epsilons is 0 is important, but is insufficient to conclude that Y is linear in X without already assuming it in the first place. I would suspect that said statisticians either already know this implicitly and don’t bother to write it down or they know that the 4 assumptions you stated can be perfectly reasonable to apply in other contexts.
In the case of least squares, that still assumes that the model is of the form:
Y = AX + epsilon
And then from there your assumption that the epsilons are 0 on average allows you to solve for an X (apologies, notation is terrible, but X is the vector of coefficients) by looking at A^T*Y = A^T*AX. Now as you correctly pointed out before, the matrix being full rank is what allows me to take the inverse and get a unique solution. I’m definitely fine to concede that this is not a requirement for a least squares solution to exist. But it does still require the Y to be of the form I stated.
My point regarding the appealing to authority comment is meant to point out that there may be subtleties to what they are saying when they disagree with the CFA text. I by no means stand by what CFA says all the time (see my post regarding I-spreads in that thread below). But the key point is having the proper context to put in perspective what they are saying. I don’t think you’ll find any statistician or stats textbook that will tell you least squares can be solved without using something like Y=AX. It may be in disguise, but it has to be there. And hence I stand by assumption #1 in my original list.
Lastly, on the point of discussing that CFA material isn’t the be-all-end-all, maybe I’m just jaded from years of teaching undergrads, but most people aren’t willing to think beyond what is in front of them, and frankly a lot of very smart people don’t really understand regression to the level that’s already presented by CFA. I suppose I’m saying that you need to crawl before you can walk, while you’re saying “yo, eventually you’ll need to walk” so…I guess that’s just a difference of tastes.
Also, I completely missed the comment regarding the null hypothesis. Eww that’s just awful.
My wife had a finance class a while back that told her Fannie and Freddie were backed by the US Government which, as a mortgage expert, set my blood to a boil. It’s hard, but you do need to sometimes take a good long look at what you are learning. That said, if I’m honest I think CFA has done a pretty good job. At least by Level 3 the items they make you think about are actually things people in finance on the desk think about every day.