Why doesn't ChatGPT (and other LLMs!) check facts first? Answers are often hugely misleading
retained text
More often than not ChatGPT will answer a question but will get it wrong **because** it's not fact checked the data first, it takes a question or two for it to admit it got it wrong and to THEN present the correct answer. According to ChatGPT (therefore it may be wrong ......... ) this is for the following reasons: *Training: Models are trained to generate likely answers, not inherently to establish whether every statement is true.* *Verification: Searching the internet doesn't automatically guarantee accuracy. Search results themselves can be wrong or misleading.* *Evaluation: It's relatively easy to test whether an answer sounds good. It's much harder to test whether every factual detail is correct.* *Product design: There is a difference between building a model that can check something and making it reliably do so before answering.* That first point in particular is terribly worrying. **The AI should be checking for accuracy with every answer.** While that won't be terribly important in SOME cases, if somebody asks about a matter that could involve safety or a person's health, it's imperative that the answer is correct. AI's hallucinate, I get that (and that's not a good thing either) but to present facts, particularly on important matters, without thoroughly fact checking first, is appalling. Confidence does not equal correctness, the AI isn't a politician .......
document fetched 2026-11-20 00:20 · canonical https://reddit.com/r/ChatGPT/comments/1y0id5e/why_doesnt_chatgpt_and_other_llms_check_facts