does anyone know which smoothing methods for time series data with outliers are most popular? i know of moving average smoothing and exponential smoothing techniques, but was just wondering if there are any better techniques out there that you guys use. thanks!
i tend to use simple averages especially when i have monthly data and if it’s too volatile to see any trend, i’ll just average the prior 12 (1-year smooth) monthly periods. i also tend to do my own smoothing by manually calculating and manipulating the data and not use the Excel’s “add trendline” function in charts. it depends on your preference too… do you want to put more weight in the more recent periods? then do a weighted-smoothing or an exponential (or any form of it) smoothing. else stick w/ simple. think of a weighted smoothing as a linear weights toward the more recent period, say assigning 75% to the last value, 15% to the 2nd-to-last value, and 5% to the 3rd-to-last… etc and exponential smoothing as like with powers; i.e. x^2, x^3, etc, with the higher powers toward the more recent periods. sorry didn’t answer much i guess… but try looking into weighted-smoothing instead of exponential.
It’s a beautiful world out there of filters and math. It depends on the problem. Can you tell us what you are trying to do?
I agree with Joey. If you explain the problem, you will get more help. For example, sometimes you can identify and kick out outliers before applying EMA, etc but this option might be inappropriate in your case.
It definitely depends on what you’re doing. For the record, I like exponential moving averages because when you use a simple moving average, the MA will jump around a lot when some outlier exits the moving average window, and usually you’d want it to jump only when it enters the window (which is what an EMA does). Joey mentioned filters and stuff. I usually use simple stuff, unless my model can be specified with enough mathematical detail that using more complex things makes sense (unless filtering just means eliminating or weighting outliers, but I think he means more than that.)
thanks guys. what im doing is estimating mean reversion of implied volatility rates using an AR(1) model. but the data is not very clean, and there are some very big outliers. my mean reversion estimate is greater than 200% for one example because of such outliers. even after taking an Moving average smoothing or just manually trying to remove the outliers, the mean reversion is still greater than 100%. however, a Running median smoothing gives me a mean reversion of 10%. which is more like what we are expecting to see, but i need to justify why theres such a big difference with a running median smoothing. and if this is a good approach or if there are any other smoothing techniques that are good that i could use. thanks!
tried smoothies yet?? ROFL ok i know, bad one, but its friday.