Sanity-checking a ThetaData SPX options backtest built with ChatGPT—data quality, fills, and overfitting?
retained text
I’m using ChatGPT to help build a backtest using 10 years of minute-by-minute SPX options data downloaded from \*\*ThetaData\*\*. I’d appreciate feedback on both the data and my approach. The way I’ve been prompting ChatGPT is roughly: \> Look for any strategy that could have produced at least a 1,500% cumulative return over 10 years with a maximum drawdown below 40%. These are search targets, not results I’ve verified. I realize that asking it to search for anything that meets a specific return and drawdown target could encourage overfitting or cherry-picking. I want to understand how to structure the process so the results are meaningful. A few questions: \- \*\*ThetaData quality:\*\* Has anyone used its historical SPX options data? What checks do you recommend for missing or stale quotes, timestamp alignment, and correctly distinguishing contracts and settlement types? \- \*\*Midpoint vs. bid/ask:\*\* How realistic is assuming midpoint fills? Should I compare that against buying at the ask and selling at the bid, with commissions and slippage included? How would you handle multi-leg fills and wide spreads? \- \*\*Strategy searching:\*\* How would you structure training, validation, walk-forward testing, and an untouched test period when searching across many strategies? How do you account for the number of strategies and parameter combinations tried? \- \*\*Return and drawdown calculations:\*\* What should I check around position sizing, buying power, compounding, and marking open positions? I want the drawdown calculation to include losses while trades are open. \- \*\*ChatGPT pitfalls:\*\* What mistakes should I audit for in AI-generated backtest code—especially look-ahead bias, unrealistic fills, or code that runs successfully but implements different rules than intended? Is this a reasonable research workflow, and how would you change the prompt or validation process before trusting anything it finds?
document fetched 2026-10-26 00:41 · canonical https://reddit.com/r/algotrading/comments/1xfedfe/sanitychecking_a_thetadata_spx_options_backtest