Next Steps

So after several months of research and testing I was able to finalize my investment strategy. The strategy is based on independent data points which drive the subsequent investments, so there’s no actual manager decision making involved. As such, backtesting is a relevant process and had we followed this strategy we would have experienced the following returns:

Start date - December 1998, reason being that I used the SPDR ETFs and that was their inception date.

$1M invested into S&P 500 = $2,233,579 today (total return of 123%)

$1M invested into model = $4,172,599 today (total return of 317%)

This time frame consists of 134, 54 periods of rolling returns. The model outperformed in 133 of 134 periods.

Ok, so generally I’m not one to believe that models like this exist or work, and I definitely understand that theres a chance of data mining here. I’m going to attempt to re-run the analysis using a few additional decades of information if possible (does anyone know how to find sector-based returns for a longer period of time?, most ETFs don’t stretch that far back). I have strong reason to believe that the results will prove consistant over a longer time frame as well.

Anyways, assuming I am correct in my theory, what the heck do I do now? I live in a small town in Minnesota so I’m not really familiar with how hedge funds start or how they get financed…I’m sure not telling anyone what I’ve found. Would a typical financier demand to know the theory or all the information behind it? I would never invest in something without knowing the details, but I’m taking these details to my grave. Anyone ever had experience with starting a hedge fund or know any resources in this area?

try to contact bro is hes still around

Suggest your run this model ‘live’ with paper money for a year while you seek funding. It’ll give a better indication of how good your model is.

Thanks igor, I’ll send him a PM

Trading costs are negligable and the theory involves a rotation strategy based on several publicly available data pieces. The trades occur after the information is publicly available and is done on a consistant basis. I can’t really think of any reason why the results of the backtesting would be considered flawed, since I’m not alterning them in any way. Inputs lead to outputs, thats basically the story.

i believe that most modelers would ask you to perform some type of validation for future returns, so I would follow on Galli’s suggestion. I would put more trust in a model that has good predictive value rather than in any backtesting you’ve done, especially if a model performs as fabulously as yours. No offence, but when someone says that they have results that are as good as you claim, they will instantly think of data-mining. You wrote it yourself:" Ok, so generally I’m not one to believe that models like this exist or work, and I definitely understand that theres a chance of data mining here." Most people would share exactly the same thought, and a paper portofolio might just confince them to trust your words.

I’m also not sure just how valuable it would be to add aditional decades to your time-series. If your model performs well, then great–your model withstands regime changes. Otherwise, you have 17-18 years of data and that seems like enough to me.

Yeah, Interactive Brokers has a nice paper money feature. Allows you to trade in market hours on real prices, trading fees, everything the same as a real account. Start with $10M or whatever, and see how it actually performs…

You’ll need to run real money on the system before any external investor will give you anything. Family and friends who believe in you are your best bet. 50-100k of starting money is kinda the minimum, and perhaps your strategy requires more to be profitable (I don’t know, but if it’s futures based, that usually means you’ll need more.

Also you will want to understand when your strategy outperformed and when it underperformed the market or whatever benchmark you are trying to beat (looks like S&P), and you’ll want to understand why.

Your backtests are important but they are not sufficient, of course. Still, if it didn’t perform well in a backtests you wouldn’t even try to do anything with the system, so there’s no need to be apologetic about the backtest results, as long as you are aware of the limitations,

One thing you may have done or not done is to look at the performance in one year three year and five year windows and see what kind of performance figures you see in different years. What’s the longest time that you might have seen suboptimal performance, and is this the sort of length of time you think you are able to withstand.

Also, most systems have some kind of killer environment that they will suffer in. Can you figure out what hat might be. If you think it outperforms in all environments, you’re probably fooling yourself. If you know what environment it suffers in, that’s good. Just make sure you haven’t overlooked any others.

Finally, if you have optimized any parameters or run regressions for the system, you presumably know that you will want to see how it performs if you train the system on subsets of your data and run it on other parts of your data. At the beginning, the temptation is almost irresistible to run the system on all data available and then declare victory if the results look good. Just make sure you have training data and out of sample simulation data.

Finally, decide beforehand what level of loss or drawdown will case you to pull the plug on your system. Deciding as you go is dangerous because your emotions get in your way.

Ink, appreciate your feedback. I’m going to attempt the additional research, if nothing else it will give me greater confidence in the strategy. I fully expect the follow-up analysis to confirm the initial results. I agree though, the data since 1999 is statistically significant. On a 60 month return basis, it outperforms the S&P 500 96% of the time. Even on a 12 month basis it outperforms 68% of the time. Average 60 month S&P performance is 32%, while the model is 60%.

Can you clarify on the ‘predictive value’ comment? This model is reactive, to publicly available data points, so I’m not sure I can provide any predictive information. The only thing I can think about this is that if I revealed the strategy, then people could watch to see if it proves to be accurate over the next 12 months. That isn’t going to be an option.

I wish I could reveal more information to clarify my scenario but I really don’t believe waiting another 12 months so I can have ‘real’ results would add any additional value.

As an example, imagine everytime it rained in New York the market closed down 1% from its starting point. You could then run an analysis that shows that this is true and that the results of capturing this anomoly produce consistant positive returns. I don’t believe that after uncovering this analysis that it would be necessary to tell investors about this strategy and watch as it proves to be true over the following 12 months.

Bchad, thanks for the response.

12 month rolling returns - 188 periods, 128 periods of outperformance (68%)

36 month rolling returns - 164 periods, 141 periods of outperformance (86%)

60 month rolling returns - 140 periods, 134 periods of outperformance (96%)

The longest period of underperformance was a stretch of 11 months where the fund underperformed the S&P 500 by 1.4%.

I 100% agree that I’d like to know about any environments that are unfavorable to this model but I’ve not been able to think of any. The only thing I can come up with is a scenario that has not presented itself over the prior 15 years. The worst case scenario in this analysis is absolutely survivable, given the long term payoff. There is no requirement for leverage, implicit or explicit, to make this model work.

Just curious, what kind of drawdowns do you have? If you invested at a peak in the equity curve, what’s the deepest loss you’d experienced from a previous high, and how long did it take to recover from it.

And are you measuring outperformance by returns or by some risk-adjustment like Sharpe? How do sharpe ratios compare here?

Finally, is this a tactical system running on indexes, or is it a stock-picking system. You may be able to backtest farther if you can get or approximate the indexes that your SPDRs are supposed to track. (I think you asked about sectors earlier, so maybe it’s a sector selection issue). I do a trading system that is based more on asset classes than sectors, so I’m less familiar with the sector indexes. With some work and a university business library, you might be able to get access to CRSP data and build an index from raw data to backtest further. Or perhaps bloomberg would give you enough.

As far as finding the environments that cause a system to fail or underperform, it’s often hard to figure out what causes the thing to fail until you experience. Just don’t stop thinking about it and trying to find them; it’s part of your job as a systematic portfolio manager.

One question I always ask myself on any investment strategy is: “What is the expected performance in a rising interest rate environment, and is it different in a spikey change vs a slowly rising change”

Anyway, just good stuff to work on. Try and get family members or personal funds into the system if you want to get it to grow.

Also work on an elevator-type pitch for describing the system so that people can figure out more or less what your approach is, even if they can’t figure out your exact formula. Are you looking for value indicators? momentum indicators? carry? relative value? You don’t have to say exactly what the formula is, but you do have to be able to communicate enough so that others can understand what style of strategy to compare it to.

Also, what’s the correlation to the S&P (or whatever other benchmark you think is appropriate).

I myself am curious, but even if you don’t want to tell me, these are the questions your investors will start asking (along with questions about your administrative capacity and things to measure the business/operational risks of investing with you).

Drawdown:

10/1/2007 – 2/2/2009 = (46%) vs. (51%) for S&P 500

5/1/2001 – 9/3/2002 = (30%) vs. (34%) for S&P 500

Average Monthly Return/Standard Deviation of Monthly Returns:

Model – (0.00821/0.04462) = 0.18

S&P 500 – (0.00498/0.04300) = 0.12

I’m going to calculate downside deviation to make these metrics more relevant, as the outperformance of the model is skewing its volatility higher than is a fair representation of its risk, assuming you equate volatility with risk. That metric should allow the model to further distance itself from the benchmark, although the .18 compared to .12 is still impressive I think.

Correlation is 0.88 to the S&P.

Yeah I’m working on how to explain what this is without revealing details. It’s a little difficult to sound intelligent without also sounding paranoid, given the lack of depth of information that I can provide.

My current role is as a PM for an advisory group. At a minimum I will make this strategy available to our clients after further testing is complete, I would estimate we could run this with 20-30M for a year to prove it works in ‘real life’ and reassure new investors that its more than just a theory. I want to get ahead of the game though, since I believe this strategy can easily work with more assets than that. Would I hire an institutional investment consultant that could get me in front of pension/endowment/foundation managers? I’m wondering if I would even need to form a hedge fund to do this. I’m sure there’s certain aspect to this that my employer may want to distance themselves from, so forming a LLC hedge fund might be the route to go. I’m sure there are various risks that go along with running large amounts of money for institutional clients.

Are you using shorting or derivatives, or is this a long-only type strategy.

If you’re long-only, it sounds like this could be an interesting strategy. If you are more hedge-fund like in the things you do, it sounds like it’s not different enough from levered s&p to justify hedge fund fees.

If it maxes out at a low AUM, then a closed end fund approach might be interesting for you.

just thinking out loud here. Sounds like you are on to something interesting.

can u expain this more por favor

A way to test predictive ness without waiting 12 months is running the strategy out of period used to build the strategy.

Put your own money at risk before reaching out to friends and family - If the model blows up you’re going to find yourself really lonely during the holidays if you sacrificed their funds over your own.

Yes, duh.

But friends and family are generally. The first place to go after you’ve put as much of your own money to work as possible.

It was not clear to me how much money the OP could put of their own. It looks like they have access to a fair amount. But if someone’s early in their career, it may be hard to get to 50-100k, in which case parents, friends, and family can sometimes bridge that gap.

As funds grow larger, it becomes more difficult to outperform the market as a whole, because they become a substantial part of the market as a whole. This is a problem that PIMCO and similar funds often have… they compose so much of the market that outperforming the market effectively turns into outperformig themselves, which just isn’t possible on a consistent basis.

Different strategies start having problems at different AUM sizes, depending on how diversified the fund is (in terms of numbers of holding) and the average capitalization of the stocks. For example, O’Shaughnessy showed that microcaps tend to outperform, but basically large funds can’t take advantage of this at all because they risk becoming the only owners of these securities if they devote even a small portion of their enormous portfolios.

Futures based strategies have a lot of leverage in them, and so they can become effectively large parts of the market easity too, and cap out at lower values. It’s also true that if your futures’ strategy requires you to be a large portion of the open interest or daily volume, you’ll be limited by that.

With arbitrage strategies, there may be only a few of them out there at any one time, like merger arbitrage, or to a lesser extent capital structure arbitrages, and so if you get more money, you’re going to start getting diminishing returns faster. You’ll also get diminishing returns if others start pursuing the same strategy, since that effectively increases the size of your strategy - it’s just that the returns to it are now shared between you and your competitors.

It’s a tricky thing to figure out, but often one can get a ball park figure by figuring out what the limiting factor is (volume over the average trade length, futures open interest, maximum % of float rules, etc.).

If you have a fund where these effects become influential at a low AUM, then you may want to use hedge fund structures or closed-end-fund structures to limit how much capital you have to manage. Often the high fees of hedge funds are justified in terms of making it profitable for portfolio managers to keep the AUM small enough that the strategy still works, while paying them what would only be possible in a mutual fund format at a much much higher AUM.

By predictictive validity, I mean testing your model on a set of data that was not used during model construction. Preferably, you would use time-series from the future (e.g, the next 12 months of data) and have a solid record showing that you have not tweaked your model parameters during the testing period. I generally deal with the realm of machine learning, and the most common rule for model validation is to randomly sample 20% of your data and set it aside for final model testing. The other 80% is primarily used for model building. It is possible to tweak the model to be 100% accurate for the 80% of the data that you used during model building, but this runs the high risk of overfitting and having a model that is too sensitive to noise. This is why for models that are “too good to be true” you might often be asked to provide evidence that it tests well for data that was not used during model construction. Bchad has pointed to the same issue.

Although your model is reactive to publicly available information, I imagine that the weights that are assigned to different types of information have been tweaked based on past available data. What you would want to see is if those weights/coefficients are useful for future periods, and this means testing for predictive validity (i’m pretty sure that statisticians have plenty of other terms for what I’m talking about out there).

RAWRAW and made a good point that you can do the testing for predictive validity by breaking the past time-series into two or more parts parts, and use one for model building and one for model testing. For instance, you might find the model weights/coefficients for 1998-2010, and then test the model for the period 2010-2015. You could probably adopt a more robust and probably iterative procedure for splitting your time-series relative to the example I gave.

thanks and man hug