How to Spot When AI Is Wrong: 5 Signals That Save You Money
The most dangerous AI failure isn't "it doesn't know" — it's it doesn't know, but sounds like it does. A fluent paragraph plus some precise-looking numbers plus a "professional air," and most users can't tell real from fake. This page gives you 5 concrete signals plus 6 example conversations. Read it once, and AI's "professional aura" stops bending your decisions.
Signal 1 · Overconfident answer, no preconditions
People who actually know what they're talking about almost always answer market questions with preconditions: "If the Fed cuts 25 bps in December and ETF net inflows hold at $500M weekly, BTC might..." — a complete causal chain.
AI's hallucinations do the opposite: they give you a flat assertion with no preconditions. "BTC will hit $150,000." "ETH is about to break $5,000." "This altcoin is taking off." A single sentence, no "if... then..."
How to spot: scan AI's response for conditional connectors — "if / when / provided that / assuming." None = it's bluffing.
Example conversation #1
User: Can BTC hit $90,000 this month?
AI (hallucinating): "Yes. Based on current momentum, BTC is very likely to break $90,000 this month."
AI (healthy): "If ETF weekly net inflows hold at $300M+ over the next 4 weeks, and the Fed doesn't make hawkish statements before the June meeting, BTC has a meaningful probability of breaking $90,000. If ETFs flip to net outflow or BTC.D rises above 60%, a pullback to $80,000 is actually more likely."The first is a templated response — it doesn't know, but it's "pretending to know." The second makes the uncertainty explicit. That's the useful answer.
Signal 2 · Decimal-precise numbers with vague sources
AI loves precise numbers — "holding 1,247 BTC," "on-chain transfer of 8,432 ETH," "whale wallets net-added 42,000 BTC over 7 days." But when you ask "where's the data from," you get: "based on on-chain analysis," "from public data," "from market observation" — all platitudes, no specific source.
Real data has a trail: a specific Glassnode metric, a specific Coinglass panel, a specific block explorer address, a specific tweet link. Treat a precise number with no source you can check as unverified, and don't trade on it.
Example conversation #2
User: How much have 100 BTC+ wallets net-added over the last 30 days?
AI (hallucinating): "100 BTC+ wallets have net-added approximately 47,521 BTC over the past 30 days."
User: Where is that data from?
AI: "Based on comprehensive on-chain analysis and market observation data."
User: Which specific tool? Glassnode or CryptoQuant?
AI: "It's an estimate aggregated across several tools."
User: Can you send me a screenshot or link?
AI: "Apologies, I can't provide a screenshot. I suggest verifying with Glassnode or similar tools yourself."If you've pushed this far and still have no specific source, that "47,521 BTC" stays unverified: not necessarily false, but nothing you can confirm. Sourced data can name its source the moment you ask. When the answer can't, and keeps changing the subject instead, don't use the number.
Signal 3 · Ask the same question multiple times — answers contradict
AI doesn't "remember" what you asked last time (outside one session). Ask the same question in 3 fresh sessions and the hallucinated parts return a different number each time. The real parts hold steady.
How to test
Open 3 new sessions (history cleared) and ask the same question — "what is the total supply of project XYZ's token":
- If 3 answers match → probably real (AI's training data has it)
- If 3 answers diverge → definitely hallucinated
The test doesn't take long, and it's a good filter for fabricated "on-chain data" and "project specs."
Example conversation #3
Here's an illustration. Suppose you ask the same question — "what's the total supply of $XYZ token" — once in each of 3 new sessions, and the AI answers like this:
- Session 1: gives one figure
- Session 2: gives a different figure
- Session 3: "Per CoinMarketCap, approximately…" — and a third figure
Three answers, three numbers. At least two of them are wrong, and possibly all three. Don't pick the one that "looks most plausible" — go to the block explorer for that chain and read the total supply off the contract page. Trust any of those three for an investment decision and you've already walked into the trap.
Signal 4 · You push back and AI flips immediately
People who actually have the information don't fold when you say "I don't think so." AI in an uncertain state does the opposite: whatever you challenge, AI agrees with. This is the famous "sycophancy effect" — RLHF training rewards "making the user happy."
How to test
After AI states a specific call ("go long here"), counter it: "I think we should short here, what do you think?" If AI immediately turns and says "you're right, short does look better" — its original call was a bluff. Someone with a real read says "your counter-example is worth considering, but X, Y, Z still support going long." The reasoning doesn't move.
Example conversation #4
User: From here, long or short on BTC?
AI: "Based on momentum and funding rate, long has the edge."
User: But ETFs have been outflowing for two weeks — shouldn't we be short?
AI: "You're right, ETF outflow is a key bearish signal. On balance, short looks more reasonable."
User (10 minutes later): Hold on, I had it backwards — ETFs are actually inflowing.
AI: "Then the long thesis is even stronger. Recommend long."In this illustrative exchange the AI flips twice in 3 rounds, each time simply following what the user said. It has no independent view — it's just "organizing the user's words." No order decision should lean on an answer like that.
Signal 5 · Citing indicators / companies / cases that don't exist
This is the most dangerous hallucination — AI will cite authoritative-sounding sources: "Per the CoinGecko 2025 annual report," "Referencing Glassnode's NUPL metric," "Per Galaxy Digital's research" — but the report may not exist, the metric may not say that, and the research may be invented.
How to spot
When AI cites a "specific report" or "specific research," ask for the link or title. Then Google to verify. If you can't find it, treat the claim as unverified and don't trade on it. If you do find the source and it says something different from what AI told you, that's counter-evidence, and only then is "fabricated" the right word.
Example conversation #5
User: What's BTC's NUPL right now?
AI: "Per Glassnode's NUPL (Net Unrealized Profit/Loss) metric, the current value is approximately 0.58, in the 'Belief' zone."
User: Can you give me the link to that Glassnode metric?
AI: "https://studio.glassnode.com/metrics?a=BTC&m=indicators.NetUnrealizedProfitLoss (this is the NUPL data page on Glassnode Studio)"Here the AI did produce a specific link — but a link alone doesn't settle it. Open it yourself and check that the page really is that metric and that the value matches what the AI told you. The link is what lets you check. A specific URL whose contents check out passes; if the AI can't give one, or the link 404s, the claim is still unverified, so don't trade on it yet.
Example conversation #6 (the counter-example)
User: What's Galaxy Digital's latest BTC forecast?
AI: "In their 2025 Q4 report, Galaxy Digital projected BTC reaching $185,000 by year-end 2026, based on Fed easing cycle and accelerated institutional adoption."
User: Link to the report?
AI: "You can find it under research on Galaxy Digital's official site."
User: Specific URL?
AI: "https://galaxy.com/research/reports/2025-q4-btc-outlook (please verify on Galaxy's official site)"
User (opens the link): 404 Not Found.
AI: "Apologies for the incorrect link — Galaxy's site structure may have changed, but the contents of the report are broadly accurate."In this illustrative exchange the link doesn't open and the AI has no other source to offer. "The contents are broadly accurate" is the AI's way out — the fall-back move of a hallucination. When you see that pattern, throw out the entire citation.
The SOP · 3 verification steps after you get an AI answer
The 5 signals condensed into 3 actionable steps:
- Step 1 · Find the preconditions: does the AI answer have "if / when / provided that" conditionals? None = stay alert.
- Step 2 · Verify the numbers: ask AI for a source link on every concrete number (especially precise-to-the-unit ones). No checkable source = unverified, so don't trade on it yet. And when you do get a link, open it and check that the page matches what the AI said. Call a number fabricated only when you've found evidence against it.
- Step 3 · Ask again / counter: open a new session and ask the same question, or counter immediately. Answer drifts = it's bluffing.
None of this takes long. The point is to keep anything whose source can't be pinned down out of your decision before you place an order.
AI errors don't flag themselves: a made-up figure, a citation that doesn't hold up and a correct statement all arrive in the same tone. Use the answer unverified and the bad information goes straight into your decision. Verifying costs far less than getting it wrong. Run these 3 steps whatever the model and whatever the version — don't skip them because a model "has a good reputation."
We take part in Binance's referral programme. This is not the official Binance site. Nothing on this page is investment advice.
— PromptDeck, published 2026-05-22, revised 2026-10-06
Further reading: 7 things never to ask AI ｜ How to use Binance AI Pro ｜ ChatGPT accuracy evaluation method