Skip to content
Back to projects

dop-usd-backtest

In Development

An honest backtest of USD to DOP forecasts: five models against repeating the last value

Role: Solo developer

Solo · Since Nov 2026 · Public repo

Pythonpandasstatsmodelspytestuv
972
Days of data
83
Rolling origins
13
Tests

A rolling-origin backtest of five forecasters (naive, drift, ETS, ARIMA, 5-day median) on two years of daily USD to DOP rates. The result is that nothing beats repeating the last value, and the project shows how that was measured.

Problem

It is easy to fit an ARIMA and plot a convincing curve. The hard part of forecasting is checking that the model beats doing nothing, without leaking the future into the history it sees.

Solution

At each of 83 weekly origins the model receives only the data up to that day and is scored against the value 1, 7 and 30 days later. Every model is reported as its error difference against the naive forecast, with a block bootstrap confidence interval.

Not done yet

One series, two years, 83 origins. The data comes from a free aggregator, not the Central Bank rate, and has isolated glitches such as 57.72 on 2026-09-29 between days near 59.5. There are no models with external variables, and it is not financial advice.

Architecture highlights

  • ARIMA and ETS have about 10% more error than the naive at a 1-day horizon, and the interval excludes zero. At 7 and 30 days most intervals include zero, so with 83 origins they cannot be told apart from the naive
  • The harness is tested for look-ahead: a spy model records what it receives and it must equal the series prefix. I injected a one-day leak on purpose and five tests failed; an earlier review found that one of my look-ahead tests was not catching it, and a real leak from interpolating the one missing day, both now fixed
  • Origins one week apart share most of a 30-day path, so a plain bootstrap would give intervals that are too narrow. I resample circular blocks of ceil(h/7) origins
  • I did not clean the series. A centered median filter would look two days past each origin, and a backward-looking one cannot flag the most recent day