Talk to me

Why does AI give a different answer each time?

A short learning video. No voice, only titles and screens, with music.

Vahid Alamipour

Learning video · 1 min 7 s ·

Video. The same thank-you request is sent twice and comes back in two different words, both fine. Then what to do when it matters: a form, a check against the source, and the checked answer kept. 1 min 7 s, music only, no voice.

What this video shows

The answer comes first, in two lines. It picks each word with some chance. So the words can change each time. On screen, a line of words stops at one spot, three possible words appear, and a small dial picks one. Run it again and the dial picks another.

Then one example. “Write one line to thank a customer.” The first answer: “Thank you for shopping with us today.” The same question again: “Thanks so much for choosing us.” Different words, the same meaning, and both are fine.

The last part is what to do when it matters. If you need the same shape every time, give it a form with fixed boxes. If two answers to a fact disagree, check the source. And keep the version you checked. That is how the small AI apps I build for daily work give a business the same kind of answer every day: a fixed form for the output, and a check before anything important goes out.

What you will learn

  • Why the same question gets different words: the AI picks each next word with a little chance.
  • Why that is usually fine: the wording changes, but a good answer keeps the same meaning.
  • How to get the same shape every time: give it a form, with the boxes it must fill.
  • What to do when two answers disagree on a fact: check the source, and keep the answer you checked.

The numbers behind the picture

Nothing with a digit is on screen. The numbers belong here.

  • 2025: Researchers ran the same question one thousand times at the strictest setting and got eighty different answers; the first hundred or so word pieces were the same every time, then the answers split. With a slower fix on the server, all thousand came out the same. An earlier study found accuracy on the same test moving by up to 15% from one run to the next. (Thinking Machines Lab, 10 September 2025; Atil and others, arXiv, 2024, v5 April 2025)
  • 2026: The dial is moving out of users’ hands. Anthropic’s newest Claude models no longer accept a temperature setting (from May 2026), and Google asks Gemini 3 users to leave it at its default. All three big vendors offer “structured outputs”, a fixed form the answer must fill. A June 2026 study still found an AI judge disagreeing with itself on up to about half of borderline items over twenty runs. (Anthropic API release notes, May to August 2026; Google Gemini 3 guide; vendor structured-output docs; arXiv, June 2026)
  • 2027, my guess: Fewer settings to touch, and more fixed forms and checks built into the apps. The words will keep changing a little; the businesses that rely on AI will check the answers that matter rather than hope for the same wording.

Questions people ask

Why does ChatGPT give different answers to the same question?

Because it writes by picking each next word from a few likely ones, with a little chance in the pick. Ask again and a different word can win early on, and the rest of the sentence follows that path. OpenAI’s own guide says it plainly: the content a model generates “is non-deterministic”.

Is there a setting that makes it give the same answer?

Partly. A setting called temperature used to make the pick less random, but even at its lowest the vendors say results “will not be fully deterministic”. In 2026 Anthropic removed the setting from its newest models and Google asks people to leave it alone on Gemini 3. The reliable ways are a fixed form for the answer and a check.

Is a different answer a wrong answer?

Not usually. Two thank-you lines with different words are both right. It matters when the meaning changes, for example two different prices or dates. Then check the source. The video “Why does AI sometimes make things up?” shows how to give the AI the source and ask where it found the answer.

How do businesses get the same kind of answer every day?

They fix the shape and test the content. The answer fills a form with set boxes, which all three big vendors now support, and important answers are checked against a source or run more than once and compared before they are used.

Further reading

  • Thinking Machines Lab, “Defeating Nondeterminism in LLM Inference”, September 2025. The thousand-run test and why the strictest setting still varies. thinkingmachines.ai
  • Anthropic, Claude glossary (“temperature” and “non-determinism”). platform.claude.com
  • Anthropic, Claude API release notes (the 2026 removal of the temperature setting on new models). platform.claude.com
  • OpenAI, “Text generation” guide. developers.openai.com
  • OpenAI, “Structured outputs”. developers.openai.com
  • Google, Gemini 3 developer guide (leave temperature at its default). ai.google.dev
  • Wang and others, “Self-Consistency Improves Chain of Thought Reasoning in Language Models”, 2022. Ask several times and compare. arxiv.org

Every AI app I build for a business gives its answers a fixed shape and checks the ones that matter before they go out (how I can help). If your team wants answers it can rely on, tell me about the work.

Transcript

Every line of text shown on screen, in order. Nothing is spoken. Each label stays at the bottom of the screen for at least four seconds.

Opening

  • Logo: vai SoftLab (small, no title line)

Scene one: the question

Alone on a dark screen, the biggest text in the film:

  • Why does AI give a different answer each time?

Scene two: the answer

A line fills with faded words and stops at one spot. Three faint words appear above it:

  • thanks
  • thank you
  • cheers

A small dial spins and stops on “thank you”, which drops into the line.

Label: It picks each word with some chance.

The line starts again under the first. This time the dial stops on “thanks”.

Label: So the words can change each time.

Scene three: one example

A chat window with a small grey tag:

  • Chat

Typed, letter by letter, then sent:

  • Write one line to thank a customer.

The answer:

  • Thank you for shopping with us today.

Label: You ask once.

The same question is sent again. A second answer appears beside the first:

  • Thanks so much for choosing us.

Label: You ask again. New words.

A small gold tick appears on each answer.

Label: Both are fine. Same meaning.

Scene four: what to do when it matters

A small form appears with three boxes, and the AI fills them; a second copy fills the same boxes in the same order:

  • Name
  • Thanks
  • Sign off

Label: Need the same shape? Give it a form.

Two short answers side by side end with different words:

  • Friday
  • Monday

A hand points at each word, then taps a document on the right; one line in it lights up, and the first answer gets a gold tick.

Label: Ask twice. If they differ, check.

The hand drags the checked answer into a small folder:

  • Saved

Label: Keep the one you checked.

Scene five: to remember

Dark screen. Two lines:

  • The words change. The meaning should not.
  • When it matters: a form, and a check.

Closing

  • Logo: vai SoftLab
  • vaisoftlab.com

Have an app idea, a stuck app, or a daily job you want an AI agent to do?

Write three lines. I reply within one working day with a plain answer.

Contact See how I can help

More videos

  1. A laptop with a small gold robot face on its screen. The internet icon in its corner is crossed out, and a grey cloud above it has faded. The label reads: It works with no internet.

    · 1 min 7 s video

    Can I run AI on my own computer?

    A short silent video. Yes: a free AI can run on your laptop with no internet, so your files stay with you. It is smaller and slower than online AI.

  2. A chat window on the left with a typed question. On the right, a shelf of small grey file icons under a gold tag that reads Your files; one file, Returns, is gold and open. A page slides from the file across the gap into the chat window. The label reads: It brings the right page in.

    · 1 min 7 s video

    What is RAG?

    A short silent video. RAG means the AI looks in your own files first and answers from the page it finds. One example, and what to do.

  3. A new, empty chat window at sunrise. Beside it, a small open notebook with a gold tag that reads Memory. One gold line, Shop: Blue Door, has slid from the notebook into the top of the chat window. The label reads: It reads the note first.

    · 1 min 10 s video

    Does the AI remember me tomorrow?

    A short silent video. The AI forgets you when the chat ends. Some apps keep a note and read it back next time. You can read, delete or switch it off.