Is it as easy as running a regression in excel with returns at time (t) as the dependent variable and the returns at (t-1) as the independent variable and then just checking the coefficient for significance using the t-stat?
No, the t-stats need to be from a Dickey-Fuller table. Ideally you should get a statistical program that will run Augmented Dickey Fuller tests for you, like Stata, SAS or Eviews.