Am I stress-testing ChatGPT to its limits, or am I still well within what it should be capable of?
retained text
I've been using ChatGPT extensively across a fairly ridiculous number of projects, and I'm starting to wonder whether I'm genuinely pushing the technology to its limits or whether I'm simply encountering problems it should already be able to handle. Here's a snapshot of what I've been doing. **Building tools:** I've had GPT build browser-based interactive tools, including a slide-based system to help work out business costs and pricing. This is one of the areas where I can really see the potential. Instead of buying software or spending weeks developing something, you describe what you need and start working with it. **Building games:** We've been creating playable games with levels, obstacles, character movement, enemy behaviour and increasingly complicated mechanics. It can be astonishing how quickly something playable appears. But maintaining consistency as the game evolves, fixing bugs without introducing new ones, and following precise instructions can become a battle. **Image generation:** This is probably one of my biggest frustrations. I have an ongoing fictional AI character projects. Getting a good image isn't necessarily difficult. Getting the avatar to maintain the same character, setting, appearance and direction over a long sequence is another matter. It regularly wanders away from what we've established. **E-commerce:** I've built an Etsy store selling AI-generated digital artwork. The production side is relatively straightforward. Making it commercially successful is another matter. So far, the store hasn't been profitable. AI can dramatically reduce the effort of producing something, but it doesn't magically create demand. **Writing books:** I'm working on a novel based on an unusual evolutionary and galactic hypothesis. ChatGPT is surprisingly good as an editor and writing collaborator, particularly when we work one paragraph at a time. Give it too much creative freedom, though, and it can start flattening the original concept. **Research and lead generation:** This has been one of the stronger areas. I've used GPT to research businesses, identify potential customers and partners, and develop personalised commercial outreach. The personalisation has been particularly impressive. We're now working towards automating the research-to-email process. **Website development:** This one annoyed me. When we created a website for my Pelagian Hypothesis project, GPT walked me through the process step by step. What I actually wanted was for it to build the website and help get it uploaded. Sometimes it behaves like a tutor when you need an executor. I don't always want to learn how to do something. Sometimes I want the finished result. **Business strategy:** I've also been using GPT to help build and commercialise an AI voice automation company, covering partner recruitment, sales research, press outreach, pricing and launch planning. It's extremely useful for organising complicated work, although it still needs close supervision. **AI testing AI:** One particularly interesting area has been using AI to evaluate other AI systems. For example, testing voice agents, analysing conversations, identifying failures and assessing whether an agent has handled a real business task correctly. But this raises an obvious question: if an AI can make mistakes when performing a task, how reliably can another AI identify those mistakes? At what point do you need a human to check the checker? **Developing an original scientific hypothesis:** I've been working on something called the Pelagian Hypothesis, exploring the possibility of an intelligent civilisation evolving independently in Earth's oceans, with radically different biology, intelligence and technology. We've examined evolutionary pathways, distributed nervous systems, geothermal energy, underwater engineering and what evidence might support or contradict the idea. This is where GPT can be fascinating as a research partner, but also frustrating. It can help explore unusual possibilities, yet repeatedly tries to bring the discussion back towards familiar assumptions. I want it to challenge the hypothesis scientifically, not automatically reshape it into something more conventional. **Long-term memory and project continuity:** This might actually be the biggest test. I'm not using ChatGPT for isolated questions. I'm running multiple projects over months, with established decisions, approved wording, technical specifications, commercial plans and creative rules. I expect it to remember what we've agreed, recognise when something has changed and avoid undoing previous work. Sometimes it's excellent. Other times it forgets an important decision, introduces something we've already rejected or confidently heads off in the wrong direction. The problem isn't necessarily whether it can do the task. It's whether it can keep doing it consistently over hundreds of interactions. And there's another issue I've noticed. **Does AI naturally pull original ideas towards the average?** If you give it something genuinely unusual, it often seems to want to make it more familiar. An unconventional novel starts drifting towards recognisable science-fiction structures. An original business concept gets pushed towards standard startup advice. A distinctive creative idea gradually becomes something that looks like everything else. It's almost as though the model's instinct is to find the most statistically comfortable version of an idea, rather than preserve what makes it unusual. That might be one of the biggest limitations for creative work: the very thing that makes AI useful — its ability to recognise patterns — can also make it resistant to ideas that deliberately break them. And I think there's a related problem with execution. Sometimes GPT knows perfectly well what you're trying to achieve but still chooses a more conventional approach. It explains when you want it to build. It suggests when you want it to execute. It rewrites when you only asked it to correct something. Which makes me wonder whether the real limitation is intelligence, instruction-following, long-term context or simply how the product is designed to behave. So I'm curious. **Am I actually stress-testing ChatGPT by using it across this many different disciplines, with long-running projects and detailed requirements? Or is this all comfortably within what the technology should be capable of by now?** And has anyone else noticed that the more original your idea is, the more effort you sometimes have to put into stopping AI from making it ordinary?
document fetched 2026-10-28 00:21 · canonical https://reddit.com/r/ChatGPT/comments/1xgoywc/am_i_stresstesting_chatgpt_to_its_limits_or_am_i