I get paid to break AI with maths problems. Honestly, I’m starting to lose now....
retained text
So this is a weird job, but I’ll explain. I studied maths in India, and for a while now I’ve been doing expert work for AI training platforms. Basically I write maths problems, give them to the best models, and if the model solves it, my problem gets rejected. I only get paid when I find something it gets wrong, and then I have to explain *exactly* where its reasoning broke. When I started, this was honestly kind of fun. Now it’s getting hard. Some stuff I’ve noticed: **Normal problems are done.** Calculus, linear algebra, most olympiad-style stuff, forget it. They solve it faster than I can type it out. Stuff that used to work a few months ago just doesn’t any more. **Their proofs** ***look*** **perfect.** That’s the scary part. Everything’s neatly formatted and sounds confident, and then somewhere in the middle there’s a “clearly, it follows that…” and that one line is just wrong. If you don’t check every step, you’d never catch it. **They forget the annoying cases.** n = 0, empty set, dividing by something that can be zero. They nail the general case and quietly ignore the one case where everything breaks. **They trust patterns way too much.** Give them the first few terms of a sequence and they’ll happily “prove” it works forever, even if it breaks at n = 6. **Make up a new definition and they get confused.** If I invent an object and ask about it, they keep drifting back to some familiar concept that sounds similar. Like they can’t let go of what they’ve seen before. **Counterexamples are their weak spot.** Ask them to prove something and they’ll try hard. Ask them to find a counterexample to something that *sounds* true, and a lot of the time they’ll just “prove” the false statement instead. That one still works for me, for now. Honestly, the thing that gets me isn’t any one of these. It’s that none of them feel permanent. Every few months, a whole category of problems I relied on just stops working. I’m chasing a target that keeps moving, and it’s moving fast. Also, something I don’t think most people realise: a lot of the “AI got better at maths” headlines come from people like me, all over the world (a lot of us in India), writing and checking problems by hand. It pays well. But it’s a strange feeling to be paid to find the limits of something that’s getting harder to beat every month. So I’m curious: * If you use AI for maths or technical stuff, where does it still mess up for you? * And once it stops failing at things like this, what do you think people like me end up doing?
document fetched 2026-11-16 00:19 · canonical https://reddit.com/r/ChatGPT/comments/1xx9le8/i_get_paid_to_break_ai_with_maths_problems