[Sorry for the long post, it took me a while to figure out how to describe the dynamic I was thinking about with respect to bubbles and momentum]
MrSmart is right in that the process I described is more clearly at work in arbitrage strategies.
My comments on bubbles was less of a definitive statment and more of a comment that a similar kind of process may work in trend following strategies and bubble markets: when strategies stop working, it doesn’t mean the strategy is innately bad, it may just be that investors have to be shaken out of the strategy before it starts working again.
For example, value investing is not a genuine arbitrage strategy, because there is no way to lock in a profit, but it has a similar quality in that there are only so many value opportunities out there at any one time and the more people who look for them the fewer there are and lower quality too. Value may seem to underperform in those environments, when the ratio of seekers to opportunities is high, and so in order to perform better, people need to leave that strategy, or seek less aggressively, both of which tends to result in worse performance over the short term. Growth investment strategies may well operate the same way, just likely out of phase.
In momentum strategies and potentially the market bubbles driven by them, the limiting factor isn’t the number of opportunities available (as in arbitrage and value investing), but deployable capital and/or available credit. In trend following, momentum attracts investors which pushes up prices which attracts investors (and the reverse in a decline, though those are more rapid). It’s the supply of capital and credit that runs out, and shakes investors out. Because capital has run out, this is what makes forced selling so prominent in bubble collapses, and why they may be much more damaging than something like merger arbitrage no longer performing well. People are selling to cover losses elsewhere when bubbles pop, so values can drop a lot before anyone finds a bid, whereas when arbitrage strategies underperform, people leave because they think they see better opportunities elsewhere, that makes the underperformance more gentle on markets as a whole.
(Arbitrage strategies that are very sensitive to credit may also have dramatic downturns, if everyone is forced to sell because of increased borrowing costs - and this can result in the unpleasant aspect of discovering that your longs have declined and your shorts have gone up - so don’t be too surprised if a nice long-short balanced portfolio turns to crap in a bubble or a credit crunch, either)
It still begs the question of whether one is better served by a dollar-cost-averaging + buy-and-hold approach or whether it makes sense to hold on while a shaking out is happening. It does seem that if you are going to panic, then you should panic early. If you haven’t panicked early, you may be better served by sticking to your strategy despite pain. It does seem that people routinely underestimate their ability to withstand pain, though, which is why people say they are sticking in for the long term, but so many bail out just as things seem to get better - the worst possible time.
But it’s not necessarily that people are so dumb as to sell because it is the worst possible time. Rather, it becomes the “wost possible time” because that’s when the largest number of people can no longer stand the pain and decide to bail out. Sure, it looks dumb in retrospect, but “worst possible time” is almost by definition the time that the largest number of people are selling. (That’s the insight you may be looking for)
One of my mentors in the macro field once told me: “Sometimes I just ask myself ‘what’s the thing that could happen that will completely screw the largest number of people,’ and I decide that that’s the thing that’s going to happen.” I don’t think it always works out that way, but I think it is a sensible question to ask yourself from time to time, and consider as one of your scenarios. This is a more and more sensible question to ask as the world becomes more unequal, because it will be a strategy of the haves to shake the tree and collect what belongs to everyone else once they can no longer hold on.
I think we’ve been in an environment that is bad for trend-following in general (certainly trend following strategies haven’t done well recently). In my mind what happens is if the recovery seems to be happening quickly, the Fed steps on the brakes to fight inflation. If the economy looks weak, the Fed pumps more cash into it. Depending on how quickly the fed acts on things, this can be bad for trend-followers because it leads to whipsaw action. We haven’t seen it in stocks, but it does seem to be the case in commodities and until recently in real estate.