Introduction to stationarity (R)

Time Series¶

A time series is a series of data points captured in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. This post is the first in a series of blogs on time series methods and forecasting.

In this blog, we will discuss stationarity, random walk, deterministic drift and other vocabulary which form as foundation to time series:

Stochastic processes¶

A random or stochastic process is a collection of random variables ordered in time. It is denoted as \(Y_t\). For example, in-time of an employee is a stochastic process. How is in-time a stochastic process? Consider the in-time on a particular day is 9:00 AM. In theory, the in-time could be any particular value which depends on many factors like traffic, work load, weather etc. The figure 9:00 AM is a particular realization of many such possibilities. Therefore, we can say that in-time is a stochastic process whereas the actual values observed are a particular realization (sample) of the process.

Stationary Processes¶

A stochastic process is said to be stationary if the following conditions are met:
1. Mean is constant over time
2. Variance is constant over time
3. Value of the co-variance between two time periods depends only on the distance or gap or lag between the two time periods and not the actual time at which the co variance is computed

This type of process is also called weakly stationary, or co variance stationary, or second-order stationary or wide sense stationary process.

Written mathematically, the conditions are: $$ Mean: E(Y_t) = \mu $$ $$ Variance: var(Y_t) = E(Y_t-\mu)^2 = \sigma^2 $$ $$ Covariance: \gamma_k = E[(Y_y - \mu)(Y_{t+k} - \mu)] $$

Purely random or white noise process¶

A stochastic process is purely random if it has zero mean, constant variance, and is serially uncorrelated. An example of white noise is the error term in a linear regression which has zero mean, constant standard deviation and no auto-correlation.

Simulation¶

For simulating a stationary process, I am creating 100 realizations(samples) and comparing their mean, variance and co-variance. The data for 6 days and 5 realizations is shown:

The mean, variance and co-variance between the samples (realizations) across are as follows:

For a stationary process, the mean, variance and co variance are constant.

Non-stationary Processes¶

If a time series is not stationary, it is called a non-stationary time series. In other words, a non-stationary time series will have a time-varying mean or a time-varying variance or both. Random walk, random walk with drift etc are examples of non-stationary processes.

Random walk¶

Suppose \(\epsilon_t\) is a white noise error term with mean 0 and variance \(σ_2\). Then the series \(Y_t\) is said to be a random walk if $$ Y_t = Y_{t−1} + \epsilon_t $$ In the random walk model, the value of Y at time t is equal to its value at time (t − 1) plus a random shock.
For a random walk, $$ Y_1 = Y_0 + \epsilon_1 $$ $$ Y_2 = Y_1 + \epsilon_2 = Y_0 + \epsilon_1 + \epsilon_2 $$ $$ Y_3 = Y_2 + \epsilon_3 = Y_0 + \epsilon_1 + \epsilon_2 + \epsilon_3 $$ and so on.. In general we could write
$$ Y_t = Y_0 + \sum \epsilon_t $$ As $$ E(Y_t) = E(Y_0 + \sum \epsilon_t) = Y_0 $$ $$ var(Y_t) = t\times \sigma^2 $$
Although the mean is constant with time, the variance is proportional to time.

For simulating a random walk process, I am creating 100 realizations(samples) and comparing their mean, variance and co-variance. The data for 6 days of 5 realizations (samples) is shown:

The mean, variance and covariances between the samples (realizations) across time would look like follows:

From the above plot, the mean of Y is equal to its initial, or starting value, which is constant, but as t increases, its variance increases indefinitely, thus violating a condition of stationarity.

A random walk process is also called as a unit root process.

Random walk with drift¶

If the random walk model predicts that the value at time t will equal the last period's value plus a constant, or drift (\(\delta\)), and a white noise term (\(ε_t\)), then the process is random walk with a drift.
$$ Y_t = \delta + Y_{t−1} + \epsilon_t $$ The mean $$ E(Y_t) = E(Y_0 + \sum \epsilon_t + \delta) = Y_0 + t\times\delta $$ so mean is dependent on time
and the variance $$ var(Y_t) = t\times \sigma^2 $$ is also dependent on time. As random walk with drift violates the conditions of stationary process, it is a non-stationary process.

The mean, variance and the co-variance are all dependent on time.

Unit root stochastic process¶

Unit root stochastic process is another name for Random walk process. A random walk process can be written as $$ Y_t = \rho \times Y_{t−1} + \epsilon_t $$ Where \(\rho = 1\). If \(|\rho| < 1\) then the process represents Markov first order auto regressive model which is stationary. Only for \(\rho = 1\) we get non-stationary. The distribution of mean, variance and co-variance for \(\rho =0.5\) is

Deterministic trend process¶

In the above random walk and random walk with drift, the trend component is stochastic in nature. If instead the trend is deterministic in nature, it will follow a deterministic trend process. $$ Y_t = β_1 + β_2\times t + \epsilon_t$$ In a deterministic trend process, the mean is \(β_1 + β_2\times t\) which is proportional with time, but the variance is constant. This type of process is also called as trend seasonality as subtracting mean of \(Y_t\) from \(Y_t\) will give us a stationary process. This procedure is called de-trending.

A combination of deterministic and stochastic trend could also exist in a process.

Comparison.¶

A comparison of all the processes is shown below:

References¶

1. Basic Econometrics - Damodar N Gujarati (textbook for reference)
2. Business Analytics: The Science of Data-Driven Decision Making - Dinesh Kumar (textbook for reference)
3. Customer Analytics at Flipkart.com - Naveen Bhansali (case study in Harvard Business Review)