Pre-registered a wallet score on Polymarket (355k wallets). It passed out of sample, and a mechanism check showed it was measuring the wrong thing
retained text
Looking for critique on the method more than the result. **The question.** Can you find prediction-market wallets that trade on information before the price moves? The obvious score: for each entry, how far the price moved in the wallet's favor 24h later. Average per wallet, shrink toward zero (K = 5 effective entries), turn into a percentile. **Setup, written down before any score was computed:** * entry = a wallet's net position change in one market over one UTC day, $100+ only * move = price of the held side 24h later (needs $100+ of *other* wallets' trades in that hour) minus entry price * sports and short up/down markets excluded (a day after a match the price is the result) * score on Oct 2025 to Mar 2026 (A), test the same wallets on Apr to Sep 2026 (B) * pass = top 10% of A still positive in B, also without the 5 largest events * t-stats clustered by event * every later change logged with a date and a reason **Two data traps we hit before the test:** 1. Related wallets passing shares back and forth in finished tennis matches at made-up prices. One bought at 0.02 and sold to a partner at 0.96 four minutes later. Trade-level, that's a +94c "call". Netting positions per wallet per day removed it. 2. A few $1.50 to $15 trades at 0.15 set the "price 24h later" in a market that settled at 1. Hence the $100 minimum of other wallets' volume. **Out of sample it passed:** |Group from A|Move in B after 24h, cents/entry|t| |:-|:-|:-| |Top 10%|\+0.83|7.9| |Bottom 10%|\-0.76|\-4.7| |All wallets|\-0.01|\-0.2| Holds without the 5 largest events, without weather markets, and for every K from 0 to 100. **The part that changed our mind (exploratory, not pre-registered).** Information should show up as drift: the price keeps moving for hours as news spreads. A good fill shows up immediately and then stops. The top 10% were +0.75c after 1h and +0.83c after 24h. Then we split each wallet's move into first hour vs 1h to 24h drift and checked which one persists A to B (Spearman, 32,867 wallets with 3+ entries on both horizons in both periods): * first hour: 0.138 * drift: 0.050 So the score mostly ranks fill quality. The "knew it first" part barely persists. **Held to settlement, after the fee each wallet actually paid on-chain:** top 10% +1.17c per entry (t 2.9), bottom 10% -0.96c (t -1.7). +1.50c on mostly taker entries, +1.25c on mostly maker entries, so it's not just makers skipping fees. **Where it's weak:** * the settlement result and the 1h/drift split were not pre-registered * on a shorter replication through an API (1,000 markets, May to Sep 2026) the groups did not separate at settlement (top +2.19c, bottom +2.10c). On those mostly longer-dated markets every group drifts up, so you have to compare against all wallets, not zero * dollar-weighted it leans on big positions: the largest 1% of entries hold 42% of net profit We didn't ship the score as an "insider" label, since what repeats is the entry price. Things I'd like pushback on: is netting per UTC day too coarse for intraday traders, and is there a better decomposition than 1h vs drift that doesn't depend on the horizon choice? Aggregated tables, the pre-registration summary and a stdlib script that rebuilds the test: [https://github.com/orcalayer/polymarket-insider-score-backtest](https://github.com/orcalayer/polymarket-insider-score-backtest) Disclosure: I'm one of the people behind OrcaLayer, a Polymarket analytics site.
document fetched 2026-10-25 00:38 · canonical https://reddit.com/r/algotrading/comments/1xebwht/preregistered_a_wallet_score_on_polymarket_355k