Double-check AI answers — and ask the model to double-check itself
28 episodes in 12 months · 48 total · first in May 2024
The model can be confidently wrong, adjust to you, or answer differently than it did yesterday. So on almost every important answer I say “double-check yourself” — and the system often finds the mistake on its own.
In serious matters — health, money, contracts — spell out a cross-check right in the request and ask it to go through the source material again. The decision and the responsibility still stay with you.
Standing rule. A standing rule: models are still being quietly changed without any announcement, and the same system can answer differently from day to day.
Where to start
On every important answer, write “double-check yourself”, and in matters of health and money ask for links to sources.
For contracts, reports and calculations, make a cross-check the rule: the model checks itself, a human makes the decision.
Do not demand perfection from AI: there is too much nitpicking at it over small mistakes, and people do not notice the strong effect.
“And here, I think, last time we also discussed this, that there's a lot of nitpicking at artificial intelligence, a lot, when, like, remember, earlier everyone was, like: hallucinations, it said something, output some word wrong or, like, skipped some letter. Uh, and not on where people, uh, see a super-strong effect from artificial intelligence.”
When I said it
Quotes
“For me too there was always the question why I personally, every time, on every answer from the system, say: recheck yourself, did you say it right or not. Often the system checks, says: “Oh, I made a mistake,” and gives the data.”
“I strongly recommend to everyone who does serious things, just recently I'm telling, say, one partner of mine: “Listen, you're checking these legal documents, you absolutely must write in the chat, check it for me again, go through the base again for me, right.” And you have to get very used to this.”
“But you start, of course, to trust the system, because you stop being in the mode where you want, where you want to double-check with someone one more time, to take some extra step one more time, and you act on autopilot. And that autopilot can lead to serious consequences.”
“That's why it's important for you to double-check ChatGPT. It's important for you to understand that it can make a mistake even on the simplest question. And that's normal.”