Ask a language model with no data connection what the dealer flip on a major index is right now and it will very often give you a number. The number will be well-formed, within a plausible range and completely invented. This is the single most dangerous behaviour in AI-assisted trading, and it is manageable.
Why it happens
A model is producing the most likely continuation of a text. In a passage where a number belongs, a number is the likely continuation, and the model has seen thousands of examples of figures in this format. Nothing in that process distinguishes recalling from constructing.
Crucially, the invented number carries no marker. It arrives in the same tone as a correct one.
Three things that reduce it
Give it a tool. A model with live data available will generally use it, and an answer built from a tool result is anchored to something real. This is the largest single improvement available and it is mostly a plumbing problem.
Require citation. A system told to name the source of every figure has nowhere to attribute an invented one, and models are noticeably more likely to say they do not know rather than fabricate an unattributable claim.
Make refusal easy. If the feed returns a named error when data is unavailable, the model has a clean thing to report. If it returns zeroes, the model has a number, and it will use it.
What you can check
Two habits catch most of it. Ask which tool produced a figure — a real one has an answer. And ask the same question twice in fresh conversations; grounded answers agree, invented ones drift.
If a figure matters enough to act on, open the source and look. That is true of AI output and it was already true of everything else.