# How do you evaluate a prompt when you don’t have labeled data? — retained record

> Retained reddit record from reddit:PromptEngineering.

Canonical: https://highsignal.app/data/records/571424e9f88a4986

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:PromptEngineering)  published 2026-11-13 14:47 ingested 2026-11-14 02:41

[original source ↗](https://www.reddit.com/r/PromptEngineering/comments/1xv9s1k/how_do_you_evaluate_a_prompt_when_you_dont_have/)

## retained text



```
I’m scoring candidate work histories against a role spec with an LLM. The inputs are experience entries with titles, dates, and descriptions of what the person did (the descriptions come through Coresignal’s employee data, which is why the model has anything to judge beyond a title). Outputs are a score and a short reason.

The problem is I can’t tell if a prompt change made it better or just different. I have no ground truth. I’ve been reading the outputs by hand, which doesn’t scale, and i’ve tried a second model as a judge, but it mostly agrees with the first one, which feels circular.

What’s a realistic way to build a small eval set? Is it worth hand-labeling a few dozen examples and testing against those, or is there a cheaper method that actually tells you something?


```



document fetched 2026-11-14 00:44 · canonical https://reddit.com/r/PromptEngineering/comments/1xv9s1k/how_do_you_evaluate_a_prompt_when_you_dont_have
