Market efficiency and market randomness

Water Cooler Investing
#1

I have been spending a significant amount of time intimately involved in markets, for the most part intraday. I have naturally been seeking to answer the classic questions. Are markets efficient? Are markets random? Are certain participants the enemy of true price discovery?

First of all, I find the philosophy of markets to parallel nicely with the philosophy of classical vs. quantum mechanics in physics. At some point (it is believed) deterministic behavior breaks down. I think most can agree, on a day to day time frame, markets are extremely efficient. Why else would they say “buy the rumor, sell on the news”? The market has a way of pricing fundamentals in even before they are pubic knowledge (aside from surprises, of course). My point is, we can agree markets are very efficient for true intrinsic value discovery on a larger time frame. What about intraday? Is that random? Is it efficient?

So, like in physics, do the markets disintegrate into randomness on the intraday level? I’m going to argue no. Markets are NEVER random, only noisy.

What is interesting is that on an intraday timeframe, many participants are not considering the fundamentals at all. I know, because I am learning to become one of them. If that is so, then how is it on a larger timeframe markets are so efficient at pricing in fundamentals?

I have a visual image that represents how I see markets working. There is a large tray filled with sand and iron filings. Under the tray is a magnetic representation of intrinsic value as determined by the fundamentals. The edge of the tray represents the fundamentals themselves. I see intraday trading as the shaking and shifting of that tray. As the sand and iron filings shift around, gradually the magnetic intrinsic value reveals itself as the iron sticks. Intraday trading may not consider fundamentals, but it does concern itself with the process of price discovery. That process is not random.

Under that thinking, there is no price discovery without the action associated with the intraday traders.

So, bottom line is this: Markets are NEVER random since there is always an underlying motive. Markets are efficient because of traders that don’t even consider fundaments, not in spite of them…

I welcome any argument.

#2

pfft

#3

Thanks man. It is a Pretty Freaking Fabulous Theory. laugh

#4

Markets are efficient at time 0 (when a new position is initiated) because of no-arbitrage conditions. Expectations and conditions of the world change which opens the possibility for profitable opportunities (and loss).

#5

i always just envision the intrinsic value as a solid line and the daily trading going on inside a gradient around the line driven based off supply and demand. Daily supply/demand may cause temporary mispricings but it will generally stick inside a pretty narrow trading range. The large moves come into play when the market makes a fundamental shift (i.e. lowering multiples) that change the intrinsic value which the trading range circles around.

Markets are somewhat efficient, we are back to where we started. It would seem almost impossible to make them perfectly efficient due to the sporadic nature of trading and limited liquidity.

#6

“The market has a way of pricing fundamentals in even before they are pubic knowledge”

Really? Because I know a trade is good by how my crotch reacts.

#7

Soros did say that he always knew to close a position when his lower back started hurting from the stress.

Maybe that’s why he’s traded less at age 80 than he did at 40 and 50.

#8

Noise is randomness. Intraday action is very random.

#9

I mean, no you’re totally right today the markets realized all assets are worth 0.37% less than yesterday when they were worth .47% more than the two days they fell .15% after they rose .22% and this all makes perfect sense with transaction fees. It’s just big dumb institutions making sluggish trades after weeks of talks on random days, not a bunch of nerds geeking out by the minute over DCF models. Total noise. Kind of like in September 2008 when the market efficiently decided everything was worth twice what it would be two months later because they’d efficiently assimilated all of the information except for all of the obvious information nobody bothered to assimilate while they were back slapping each other and trusting the rating agencies the banks were paying.

#10

At some level, quantum physics seems to have the only intrinsically random processes that exist. They are intrinsically random in the sense that no amount of observable information or increase of precision on present knowlege can improve your prediction of the future.

Other processes are not intrinsically random (they are the result of cause and effect relations that are deterministic, though often complex). Here, randomness is often inserted to reflect our incomplete information about the world. Things that are potentially knowable, but too difficult to actually collect or know at the precision required, or sometimes (in the case of markets) illegal to act on if known. Sometimes what is unknown is the functional form of the interactions itself, but it is not in doubt that there is some functional form that describes things.

(A special case of this problem is the “sensitive dependence on initial conditions” which generates deterministic chaos in systems subject to feedback loops, of which markets certainly qualify. So perhaps intraday might be described both as chaotic and as random.)

Intraday prices are pretty much all about order flow. There are occasional macro events - earnings announcements, legal judgments, fed actions, a major power invading another, natural disasters, etc. - but those happen too infrequently to explain the minute to minute changes in prices. Instead, it’s about whether Pimco is rebalancing its portfolio in $100 MM or $10MM increments, whether a HFT computer is trying to push up the price to find where the protective stops are, whether Cramer told people to SELL SELL SELL last night and they believed him.

These are things that are potentially knowable in an epistemic sense, though probably not in a practical sense, so we model them as random. They are not intrinsically random the way nuclear radiative decay is, but to us at our trading stations, they might as well be.

Aside from the occasional macro events and company announcements, underlying company fundamentals just don’t change enough from 11am to 11:30am to account for how prices change. It’s order flow.

Sometimes, someone cracks a part of the system - they find an indicator that seems to predict well when people are going to trade on a particular signal. There’s money to be made there. But as the profits increase, the power of that signal then dissipates, and the cost of using it starts to lose its risk-adjusted advantage. Eventually, it may not be possible to use that signal any more. This is the market becoming more efficient and then less efficient. And the more efficient the market actually is, the more it is moved by tiny random things intraday.

The randomness itself can pop out and reappear, not unlike particles in the quantum foam, so there is an interesting parallel there, but I’m not sure how it’s useful or where you can go with it other than to stare and marvel at mathematics made flesh before our eyes.

One interesting point is that some enormous proportion of stock price increases happen between the close and the open. If you were short or out of the market during the day and long the market at night, you’d have some hugely well performing portfolio. But it just doesn’t work after trading costs are factored in. I forget how low they have to be to pay off, but it’s pretty darned low. (I now find myself wondering if a high beta stock would work instead).

#11

Sounds kinky . yes

Personally I don’t like getting too quanty and/or academic. I would say mid-term, markets are not very efficient, “ultra-weak-form EMH”?

I don’t know if short-term moves are random or not. There are reasons behind individual investor moves, but the overall move in the security is the sum of these, and it may end up looking pretty random and/or noisy?? That’s why I don’t day-trade, don’t have any theories on how S/T prices work!

#12

If you add up say 12 or more uniformly random numbers (i.e. in excel: RAND() + RAND() + RAND() + …), what you get is a nearly normal distribution centered on N/2, where N is the number of RAND()s you added. That’s the Central Limit Theorem at work. What’s interesting is that it seems to work with all distributions, provided they have an asymtotically definable mean and variance.

So when we model short term prices as a random walk, often times what we are saying is that “there is a sequence of random or (for all practical purposes) unknown events that is going to happen in between now and whatever time horizon we have set (30 mins… 30 days).” The bet is that all these unknown events will add up to something approximately normal because of this Central Limit Theorem.

The assumptions that are most likely to be broken are that the set of processes have similar enough means and variances that they will add up to some normal distribution (or perhaps something like a T distribution or something). It’s not accurate, but it’s actually not such a bad model, compared to where you started, which was having any idea what’s going to happen at all and therefore listening to your taxi driver or something.

The more problematic assumption is that the events between now and say 30 minutes from now will be uncorrelated. People may be reacting to the same news story, or gloomy weather, or that it’s lunchtime and these may make the summation of random events have a different mean than would be predicted. This doesn’t necessarily mean that your model is wrong, just that the variance is way underestimated.

As you expand your trading window to something more like a swing trade window, the correlation issue may drop off, but the chance of some macro event intervening increases. This is where the jump diffusion models were an innovation.

So modeling unknown information as random processes isn’t a bad idea… it’s just that the temptation to use the math to optimize to a precision that is mathematically attractive but no longer representative then leads people to do math to prove they can do it, rather than to get the best answer to the practical problem.

#13

So it is important, if we want to try to understand or model intraday price behavior, that the assumption be made if it is random or not. If you assume they are random, finding something repetitive (singnals/ indicators) could be expected to produce an edge overtime. This is how many algorithmic trading systems get their edge. Although, even though I believe markets are not random, they are noisy enough that you can model them as if they are.

Now, I want to elaborate more on what I mean by “intraday participants do concern themselves with the process of price discovery”. I have another analogy. The intraday market is a dark room with 40 participants in it. If assuming the market is random, you assume these participants just move around “drunk walk” style and try to model that. If you see markets the way I do, then you add a box of cash hidden somewhere in the room and the participants try to find it. Now modeling the behavior of the participants gets a whole lot different! Certain patterns will certainly emerge. Maybe there will be periods when participants will be very still and listen for where the commotion may be and start headed toward it. Maybe participants start latching onto other ones in attempt to get in with the flow that leads to where the cash box is. Maybe there are participants that violently push others, but then the victims follow the source of the push because that indicates they are close. Anyway, it is all very noisy but deterministic.

The market is an auction. Participants seek to join the flow of price in it’s auctioning process. Below is a 10min candlestick chart of Bund Futures. The white profile is a volume profile for the time period of Oct 5 to Wednesday Oct 19. To the right of the profile is how the market traded on Thursday Oct 20. Looks sort of random, huh? The volume profile shows how much volume was traded at each price. In other words, it shows what prices the market likes, and which ones it does not. Notice how at one point price shot down (hard to see, but the one candle reaches almost to the bottom of the chart), but could not find any value down there so it shot right back into the value area. Then, after the value area had be adequately explored, prices decided to try to find value higher up. Notice how it came to rest in the smaller node at the top. It did not need to shoot back down into the day’s region since it found another place where participants where interested again.

#14

dup

#15

This conversation reminds me of the things I used to debate in my grad program, bright eyed and bushy tailed. This discussion is massively different from any discussion being had by any money manager I’ve encountered. It’s just not particularly useful… or reflective of reality.

#16

It explained the crash in gold pretty well! Market HATES 1310!

#17

Being an anti-academics guy, I tend to think this way. Although I do build Bchad’s random walk stuff into Excel models. Not that I think it is correct/useful, it’s just that we have no better way to model security prices, when such modeling is needed (like for an investor pitch where they want to see scenarios). But I never use that modeling for real world trades. Maybe some quant-fund is doing so…

#18

Quant managers often have these kinds of discussions amongst themselves when they are trying to improve or specify their process.

It would be odd if a fundamental-based investment fund spent much time talking about this, or if quant people talked about this in enough detail that others could reproduce their investment model.

#19

KMD. I liked the image of the dark room with money and others are searching. And that listening to commotions and things can be helpful.

However, in my experience, a lot of these signals look good only in backtests with look-ahead bias. In practice, by the time you have the signal, it is often no longer useful, at least for intraday stuff. That’s basically why lots of models start with a martingale assumption.

Think about it, if you think the pot of gold is where everyone is congregating. Well, they’re already there, so how are you going to get the gold before them?

The image is useful though, because it does suggest other strategies. For example, one might want to invent a commotion so that other searchers congregate somewhere, meanwhile you rush around other places to see if the money pot is there somewhere. That increases the chance that you find the money while others are distracted (assuming that you haven’t fallen for the same trap). I have no doubt that some HFT systems try to do this. In non-computerized systems, it’s called “pump 'n dump” (or “short and distort”). The non-computerized strategy being illegal if it can be proven to be what someone knowingly did.

But ultimately, if the pot of money’s location is unknown to you, and the location of other people is unknown to you, and how fast they can move or travel is unknown, is this really so much different than RAND() + RAND() + RAND() + RAND() + …

Intraday, company fundamentals are pretty much constant (barring the occasional news event). What does change minute to minute are people’s decisions about liquidity needs, and their perception of risks (both absolute and relative to other investment opportunities). There are presumably sophisticated ways to model this kind of stuff, but I am certainly not able to do it without a lot of research time and a bunch data that I can’t afford to buy even if it exists.

#20

I saw a robin hop across a branch the other day and predictably it rained three hours later.