What am I doing wrong?
retained text
Hi, so I would like to share the way I've been trying to get this bot to work. It for sure hasn't been I'll just give a simple break down of how mine is working and feedback is definitely wanted. Everything that passes validation dies out of sample. What am I doing wrong? * **Setup:** ES futures tick data (Databento MBO, Jan–Oct 2026), a decision on every trade from 09:42–16:45 ET using 154 order-flow/price features, limit entries, fixed TP/SL, $6 commission, one position at a time, with daily loss and profit caps. * **Search + validation:** brute-forced 682K single conditions × 4 entries × 80 TP/SL pairs, extended the best to 2–3 conditions, plus LightGBM rules of up to 5 conditions (2.56M candidates). Then Jan–Jun discovery, a Jul–Sep exam with filters (min trades, Wilson win rate above break-even, monthly consistency, drawdown, 0.3 correlation), and a one-shot Sep 15–30 holdout. * **Results:** 173K passed discovery, 850 passed the exam but lost $105K together on the holdout, and the 14 holdout survivors all failed on 3 brand-new days (−$1.5K), which looks like massive multiple testing overfitting. What exactly am I doing wrong? Do I need to show you guys more of what I did as well
document fetched 2026-10-14 00:45 · canonical https://reddit.com/r/algotrading/comments/1x4pf2s/what_am_i_doing_wrong