Submit your homework report as a pdf via Canvas. You may scan a handwritten report. Later in the course, we will be using Rmarkdown for projects, so you might like to use that also for this homework. In that case, the source code for this assignment is available on github to help get you started. Rmarkdown combines R with Latex, so extra work will be required initially if you are unfamiliar with either of these.
You are reminded that the grading scheme for the homework report puts an emphasis on careful explanation of sources, as explained in the syllabus. A source is any person, website or other information resource that is consulted while carrying out your homework. All sources should be listed. It may be convenient to number your sources so you can refer to them in your report. Be explicit about where you do NOT use sources as well as where you DO. Your report can discuss your sources, what you learned from them, and how your solution makes use of them and goes beyond them. A fully correct solution will not get full points without explicit discussion of sources.
Note that the request for a participation report has been removed from this homework, and is now a separate assignment.
Question 2.1. We investigate two ways to calculate the autocovariance function for AR and MA models. These ways are different from the two ways already demonstrated in the notes, but there is some overlap. The instructions below help you work through the case of a causal AR(1) model, \[X_n = \phi X_{n-1} + \epsilon_n.\] where \(\{\epsilon_n\}\) is white noise with variance \(\sigma^2\), and \(-1<\phi<1\). Assume the process is stationary, i.e., it is initialized with a random draw from its stationary distribution. Show your working for both the approaches A and B explained below. If you want an additional challenge, you can work through the AR(2) or ARMA(1,1) case instead.
A. Using the stochastic difference equation to obtain a difference equation for the autocovariance function (ACF). Start by writing the ACF as \[\gamma_h = \mathrm{Cov}(X_n,X_{n+h})= \mathrm{Cov}(X_n, \phi X_{n+h-1} + \epsilon_{n+h}), \mbox{ for $h>0$}.\] Writing the right hand side in terms of \(\gamma_{h-1}\) leads to an equation which is formally a first order linear homogeneous recurrence relation with constant coefficients. To solve such an equation, we look for solutions of the form \[\gamma_h = A\lambda^h.\] Substituting this general solution into the recurrence relation, together with an initial condition derived from explicitly computing \(\gamma_0\), provides an approach to finding two equations that can be solved for the two unknowns, \(A\) and \(\lambda\).
B. Via a Taylor series calculation of the MA(\(\infty\)) representation. Construct a Taylor series expansion of \(g(x)=(1-\phi x)^{-1}\) of the form \[g(x) = g_0 + g_1 x + g_2 x^2 + g_3 x^3 + \dots\] Do this either by hand or using your favorite math software (if you use software, please say what software you used and what you entered to get the output). Use this Taylor series to write down the MA(\(\infty\)) representation of an AR(1) model. Then, apply the general formula for the autocovariance function of an MA(\(\infty\)) process.
C. Check your work for the specific case of an AR(1) model with \(\phi_1=0.8\) by comparing your formula with the result of the R function ARMAacf.
Question 2.2 Compute the autocovariance function (ACF) of the random walk model. Specifically, find the ACF, \(\gamma_{mn}=\mathrm{Cov}(X_m,X_n)\), for the random walk model specified by \[ X_{n} = X_{n-1}+\epsilon_n,\] where \(\{\epsilon_n\}\) is white noise with variance \(\sigma^2\), and we use the initial value \(X_0=0\).
Reading. We have covered much of the material through to Section 3.3 of Shumway and Stoffer (Time Series Analysis and its Applications, 4th edition). The course notes are intended to be self-contained, and additional reading is therefore optional. Reading this textbook will help to broaden your understanding of these topics.