# Your winner gets weaker every time you test another version of it — retained record

> Retained reddit record from reddit:algotrading.

Canonical: https://highsignal.app/data/records/fe95c3f7afdd3b33

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:algotrading)  published 2026-10-27 14:35 ingested 2026-10-28 02:39

[original source ↗](https://www.reddit.com/r/algotrading/comments/1xgu80j/your_winner_gets_weaker_every_time_you_test/)

## retained text



```
In the backtest catalog I work with, every strategy carries a Deflated Sharpe score (Bailey and Lopez de Prado). The part that surprises people: a strategy's score can drop without any change to its own returns, just because more variants of other ideas got tested.

I think of it as plain selection. Test 20 versions of an idea with no edge and the best one will still show a decent Sharpe by luck. The Deflated Sharpe asks whether your winner beats the Sharpe you'd expect from the luckiest of all your trials, given how long its record is and how fat its tails are. More trials raise that luck bar, so the count has to include every variant you ran, published or not. Ours counts every strategy variant we've backtested, retired and unpublished ones included. Plain single-fund holds are left out, since nobody searched for those.

History length matters as much as the Sharpe. In our safe-withdrawal table the top 6 rates sit between 14.34% and 19.64%. The first 5 have 18.5 to 52.6 years of history and clear the 0.90 bar. The 6th, at 14.34%, has 7.0 years and gets flagged fragile. Nearly the same headline number as the 5th (14.66%), very different evidence behind it.

*Processing img 395vhzmrt0yh1...*

Our count is still a lower bound. Lookbacks tried in a notebook before a variant got its own row aren't in it, and neither are the ideas other researchers dropped before publishing the paper a strategy came from.

When you count trials, do you include every parameter set you ran, or only the versions you saved as named models?
```



document fetched 2026-10-28 00:45 · canonical https://reddit.com/r/algotrading/comments/1xgu80j/your_winner_gets_weaker_every_time_you_test
