← sources

Luna 6 is a massive downgrade over Luna 5.6. Misses crucial details, less helpful than 5.6 Luna

source: reddit (reddit:OpenAI)published 2026-09-23 05:48ingested 2026-09-24 02:38

retained text

I maintain an agentic reporting tool which uses LLMs and tool calling to report on data from a wide variety of sources. 

Luna 5.6 was working really well for us at low cost, but I was considering upgrading to Luna 6 since it benchmarks slightly better and is 50% cheaper, hoping it would be at least equally as good.

I did some A/B testing using "high" effort level for both:

- On every single eval which involved reviewing written reports and finding relevant items, Luna 6 failed to identify one or more relevant items that should have been included in its answer. Luna 5.6 did not.

- Luna 6 was consistently less helpful, failing to identify how to answer many questions that Luna 5.6 could answer, and providing less useful information in its responses

- Luna 6 appears to try to answer the prompt in the least amount of tokens possible, which makes it less helpful. It will give a vague answer, requiring the user to ask a lot of follow up questions.

For example. "Can you find if there are any items that fit this criteria?"

GPT 5.6 Luna: "I found 7 items, here they are: <Formatted List>

GPT 6 Luna: "Yes I found 7 items". (user has to follow up "Can you list them please?")

If I could use a single word to summarize Luna 6, it's "Lazy". Even on high effort, it would always answer with the bare minimum.

I tried many different use cases and could not find a single instance where 6 Luna provided a superior answer to 5.6 Luna.

I won't be using it for my application, will stick with 5.6 Luna.

document fetched 2026-09-24 00:30 · canonical https://reddit.com/r/OpenAI/comments/1wnxg0n/luna_6_is_a_massive_downgrade_over_luna_56_misses