← sources

How do you test prompt changes before shipping them ?

source: reddit (reddit:PromptEngineering)published 2026-11-18 10:54ingested 2026-11-19 02:39

retained text

i build ai agents at work and the part i still find messy is changing the system prompt. i fix one thing  and something unrelated breaks  and i usually only find later. Right now i keep small set of conversation and rerun them after  change, then read the output by hand , it works but it is slow.  
what are you doing for this ? proper eval tools , llm-as-judge , just manual check? i am just curious what's actually worked for people in production , not just in theory.

document fetched 2026-11-19 00:40 · canonical https://reddit.com/r/PromptEngineering/comments/1xzcbec/how_do_you_test_prompt_changes_before_shipping