TradeHouse Fidelity Lab · Guides
Walk-forward validation vs cross-validation
Both methods estimate how a model will do on data it was not fitted to. They differ in how they split time, and in trading that difference decides whether the estimate is honest.
The short answer
| Walk-forward | Standard k-fold CV | Purged / embargoed k-fold | |
|---|---|---|---|
| Train data comes from | Only the past | Past and future folds | Past and future, with overlapping samples removed |
| Leakage risk with serial correlation | Low | High | Reduced |
| Data efficiency | Lower: early periods are only ever training data | High | High |
| Mimics live trading | Yes | No | Partly |
| Typical use | Final check of a frozen rule | Not suitable for overlapping financial labels | Model selection when data is scarce |
Why plain k-fold leaks
K-fold shuffles observations into folds and trains on all folds but one. With independent samples that is fine. Trading labels usually overlap: a 5-day return label for Monday shares four days with Tuesday's label. If Monday is in training and Tuesday is in test, the model has partly seen the answer. Regimes add a second leak: training on 2023 to predict 2021 lets the model learn what happened later.
What purging and embargo fix
Purged cross-validation, described by Marcos López de Prado in Advances in Financial Machine Learning (2018), drops training samples whose label window overlaps the test fold. An embargo also drops a short period right after each test fold, so serial correlation cannot carry information back. This keeps much of k-fold's data efficiency while removing the most obvious leaks.
Why walk-forward still matters
Even purged CV lets the model learn from periods after the test period. Walk-forward never does: fit on a window, freeze the rule, score the next window, roll forward. It is the closest offline imitation of trading the rule live. A common workflow is to use purged CV for model selection and a final walk-forward with the rule frozen as the acceptance test.
How Fidelity Lab uses it
Fidelity Lab runs a point-in-time walk-forward with the threshold frozen on the training split, then checks the day-clustered effective sample size and applies Benjamini-Hochberg at q=0.05 across every rule you submit. If the book is too short to support a conclusion, the verdict is UNTESTABLE rather than a guess.
Try it on your own trades. The free Fidelity Lab tier runs one report a month on up to 5,000 legs: all 11 contamination detectors, cost re-netting and the walk-forward verdict. Run a free report or see pricing.
Frequently asked questions
Is time-series split in scikit-learn the same as walk-forward?
TimeSeriesSplit gives expanding-window train/test splits that only train on the past, which is a form of walk-forward. It does not purge overlapping labels for you.
How many walk-forward windows do I need?
Enough that each test window holds a meaningful number of independent days. Many tiny windows add noise rather than evidence.
Related guides
- Backtest overfitting: a practical guide
- Backtest validation tool: what it should check
- Probability of backtest overfitting (PBO)
- Trading strategy audit service
- Overfitting in backtrader
- Trading strategy robustness tests
- Monte Carlo backtests
- Look-ahead bias in backtests
- Survivorship bias in backtests
- Data snooping in trading
- Out-of-sample testing in trading
- What is backtest overfitting?
- Deflated Sharpe ratio calculator
- Fidelity Lab
Educational content, not investment advice. Updated 2026-10-28.