An interesting application of time series methods to space weather. A range of methods is applied, with a Loess decomposition being preferred to ARIMA modeling.
The project report is missing a title.
The introduction could better clarify and motivate the specific goals of the time series analysis. We don’t model just for the sake of modeling, or forecast just for the sake of forecasting.
There are no equations anywhere in the text. For a technical statistical report, we expect some technical explanation of the models and methods involved.
Numbered captions for figures would help the reader. This is especially true in the context of helping peer reviewers, but the authors knew their report would be read by reviewers.
What is flux1 and flux2? If we don’t
know more about what these are, it is hard to tell whether the analysis
is appropriate.
What is na.approx from zoo? Why is it
appropriate here?
One can guess which are the interpolated values, but it would be good to show them on the plot.
The periodogram for the residuals from the smoothed time series has a peak power driven by the nonparametric detrending process. The team has removed low frequencies, and so there is a peak corresponding to the highest frequencies not removed by the smoothing procedure. This may not be a real scientific property of the data. That could be confirmed by looking at the frequency response function for the detrending operation. Or it can be avoided by calculating the periodogram of the raw data.
The sun’s synodic rotation period is approximately 26 days, which is twice the observed peak frequency in the periodogram. So, perhaps there is a scientific foundation to this frequency. More investigation is needed.
The choice of \(d=1\) for the ARIMA analysis is not well discussed, though indeed that is probably the most reasonable value if one is to do ARIMA. The report spends most of its time developing an interesting alternative approach, so limiting discussion of ARIMA is fine.
The STL decomposition is not explained, and the quantities
t.window and s.window are not
defined.
For “lags extending up to approximately 1750,” it would be better to put in units.
“These bounds are generally indicative of the threshold beyond which autocorrelations are deemed statistically significant.” They provide a 95% coverage under the hypothesis of white noise. As soon as that hypothesis is rejected, they no longer have interpretation.
“We specify seasonal = TRUE due to the presence of
an oscillating pattern.” The model does not have a seasonal
component.
The authors also don’t state what criterion is used to choose the “best model” (ARIMA(2,1,2)).
Without explanation in the report, the source code reveals that
auto.arima was used. In
forecast::auto.arima(max.p=50, max.q=50, seasonal=TRUE, stepwise=FALSE)
with the default seasonal parameters (max.P=2, max.Q=2),
the maximum value of \(p+q+P+Q\) for
the models tried is given by max.order (5 by default). So,
it is likely that the full parameter space described by the authors is
never explored. Using highly automated procedures requires care, and at
the very least the team should let the reader know what was being
used.
MAPE and MASE should be defined.
“MASE is greater than 1, suggesting that the forecasted values are not very accurate.” Whether 1 is big or small should depend on the scale of the problem.
A wide range of frequencies is removed by this detrending. The maximum in the frequency domain may be just the edge where frequencies are no longer removed by the detrending - looking at the frequency response function for the detrending would help address that.
Some plots do not have units of time and frequency.
Residuals have non-constant variance, from the residual time plot.
STL decomposition after detrending has the result that the trend models the residuals from the detrending, which is noise. The seasonal component has small magnitude, since the data do not have substantial annual seasonality.
Predicting each component of an STL separately is an interesting idea. It seems to have some justification here, but the evidence is not enough to be reasonably sure: the only evidence is success at predicting on one particular window, which is a very high-variance evaluation.