How do you test prompt changes before shipping them ?
retained text
i build ai agents at work and the part i still find messy is changing the system prompt. i fix one thing and something unrelated breaks and i usually only find later. Right now i keep small set of conversation and rerun them after change, then read the output by hand , it works but it is slow. what are you doing for this ? proper eval tools , llm-as-judge , just manual check? i am just curious what's actually worked for people in production , not just in theory.
document fetched 2026-11-19 00:40 · canonical https://reddit.com/r/PromptEngineering/comments/1xzcbec/how_do_you_test_prompt_changes_before_shipping