Hello, I was hoping someone could please help me out with the last LOS in SS #12. “Interpret the economic meaning of the results of multiple regression analysis, and critique a regression model and its results” Now this is the last one so to me it seems kind of like a summary LOS and a lot of repeated info from all the other LOS’s. There is no part of the reading that explicitly talks of this one, my question is how should I approach mastering this one? Is the first part “economic meaning” just saying can you tell if this regression is useful, if so isnt it just as simple as Yes, if the F-stat is significant, and all the other factors they talk about in the reading. I am blabbing but Is this LOS just asking for me to go through an entire regression and explain all the parts? Thanks I appreciate any help, I am stuck on this one.
Packers, I think the point here is that CFAI is encouraging you to distinguish between statistical significance and economic significance. JoeyD or Maratikus probably can add more on the statistical meaning, which I think, as you said, an F-Test will suffice, but the question here is whether there’s any economic value to the model. I think a fair example would be proving the statistical significance of a widely recognized technical analysis relationship, which due to its wide recognition would yield no significant economic value because many other market participants would already be acting on the model, limiting your value in pursuing the same strategy. Bottom line: I don’t think statistical significance assures economic significance. Anyway, I’d value others’ input.
It is important to see the forest for the trees. MLR helps us to understand relationships but the data will gives us incorrect or fuzzy answers at times. If you have done all the studying, then you know what to look out for. People who trust the data implicitly better have “technique” in applying the tool. Read The Black Swan, by Nassim Taleb, to fully appreciate this. "Bottom line: I don’t think statistical significance assures economic significance. " HiredGuns1 said it quite well. Ask Goldman Sachs about their quant fund. Just know the tools and know they only are good in the hands of a real practitioner who knows when to use them. Basically what they are telling you here is stop staring at the trees. Dave
Doubtful. It’s pretty hard to ask questions about the forest on a multiple choice exam. I pretty much agree with the OP’s assumption above that it is a summary LOS. A regression model probably requires more than statistical significance to be any good. It requires that the results are interpretable, that it makes some kind of intuitive sense, that the assumptions of the modelling are at least reasonably true, etc… The kind of questions that they will ask you on this would be something like after you fit a regression model of GDP on oil prices and then they say “Predict GDP next year if oil prices are $88/barrel”. On a real forest test, they would ask what’s wrong with that prediction but that’s not very CFA-like.
thanks guys that helps
I would go along the lines of what everybody else said too. I would think the basic idea is that even if you have a regression model that is stats significant it might mean nothing. The whole idea of the model is to try to predict future values of something right? Well you might have a model that, although stats significant, might tell you nothing going forward. It might tell you nothing about causality in the future. Kind of like a suprious correlation.
The “Economic vs. Statistical Significance” question probably speaks to the difference between whether you can reject a hypothesis and whether you CARE about the results. Here’s an example: assume that you are testing a time effect (like a January effect) with a dummy vairable regression. You have a large sample, so you have a lot of power. Your coefficient on the dummy variable has a p-value of 0.001, but the coefficent on the dummy variable is 0.002 ==> this indicates that the return in January os 20 basis points higher than in your reference month. So it’s STATISTICALLY significant, but the magnitude of your coefficient (i.e. the abnormal return) is small enough that there’s not any ECONOMIC significance. This happens a lot in large-sample studies - given enough data, you’ll find STATISTICAL significance, even if there isn’t ECONOMIC significance.
i think its kinda hard for us to say that 20bps excess return is economically significant for a pension fund, 20bps excess in a month is huge i doubt they would ask a question like that
Sorry - sloppy exampkle. Make the abnormal return much smaller - say 5 or 10 BPs. If you had a large enough sample size, your standard errors will get small enough that the effect will be statistically significant even though small in real-world terms. I see this a lot in empirical studies - given a large enough sample size, you get very high power in your tests. So, even a very small effect (say 20 BP per year) will be associated with t-statistics of 5 or 10 (so large you don’t need a stat table to see whether it’s significant).