TradeHouse Fidelity Lab · Guides
Backtest validation tool: what it should check
A backtesting engine tells you what a rule would have done. A validation tool tells you whether you should believe it. They are different jobs, and most backtesting platforms only do the first one well.
What a validation tool should check
- Leakage and look-ahead. Features, fills or outcomes stamped after the decision they inform.
- Cost realism. Fees and slippage from the venue's actual schedule, not a flat assumption.
- Data artefacts. Duplicate rows, crossed books, sign errors on short or NO-side positions, open positions counted as realised.
- Out-of-sample decay. A rule frozen on training data, scored on later data.
- Effective sample size. Independent days, adjusted for autocorrelation, not row counts.
- Multiple testing. A correction for how many rules or cohorts were tried.
- The right metric for the market. A binary contract and a stock should not be graded by the same ruler.
Questions to ask before trusting a validator
- Can it return a negative answer? A tool that always produces a score is not testing anything.
- Does it say when your data is insufficient, rather than inventing a number?
- Does it explain which trades triggered a finding?
- Does it ignore the fee column in your file and use its own schedule, or does it trust whatever you send?
- Is the method published so you can reproduce the result yourself?
How Fidelity Lab works
You upload a CSV or JSON of trades (legs) from any source, including exports from backtesting platforms. Fidelity Lab runs 11 contamination detectors, re-nets costs from a per-venue fee schedule, performs a point-in-time walk-forward with the threshold frozen on the training split, measures day-clustered effective sample size and applies Benjamini-Hochberg at q=0.05. It answers SURVIVES, DISSOLVES (naming whether out-of-sample decay or fees killed it) or UNTESTABLE.
| Fidelity Lab plan | Price |
|---|---|
| Free | $0, one report a month up to 5,000 legs |
| One audit | $99 one-time |
| Five-audit pack | $399 one-time |
| Starter | $49 per month |
| Pro | $149 per month |
Prices are from the Fidelity Lab pricing section. For how it compares with running your own tests, see Fidelity Lab vs doing your own walk-forward.
Try it on your own trades. The free Fidelity Lab tier runs one report a month on up to 5,000 legs: all 11 contamination detectors, cost re-netting and the walk-forward verdict. Run a free report or see pricing.
Frequently asked questions
Does a validation tool replace a backtesting engine?
No. You still need an engine to produce trades. The validator grades the trades it produced.
Will Fidelity Lab tell me what to trade?
No. It reports whether the evidence in your file supports the edge. It is not investment advice.
Related guides
- Backtest overfitting: a practical guide
- Walk-forward validation vs cross-validation
- Probability of backtest overfitting (PBO)
- Trading strategy audit service
- Overfitting in backtrader
- Trading strategy robustness tests
- Monte Carlo backtests
- Look-ahead bias in backtests
- Survivorship bias in backtests
- Data snooping in trading
- Out-of-sample testing in trading
- What is backtest overfitting?
- Deflated Sharpe ratio calculator
- Fidelity Lab
Educational content, not investment advice. Updated 2026-10-28.