Your AI Gave You An Answer. Nobody Gave You Intelligence.
We're in South America this week working with a few clients, including a data center build in São Paulo and some time with a defense partner in the Southern Cone. On the way down, a fellow traveler found out what we do and started talking about AI. His view was one we hear constantly now. You ask a model a question, it goes out and pulls everything together, and what comes back is intelligence.
It was a good conversation, and a little strange, because he wasn't wrong about what the tools can do. He was wrong about what the output is. That gap is worth writing about, because there is a whole industry sector making the same mistake at scale.
Most of what's sold as intelligence is information
If you're new to this field, you'll find the word intelligence being misused almost everywhere. Vendors call a feed of indicators intelligence. Platforms call a scraped summary intelligence. AI tools hand back a confident paragraph with no question behind it, no second source, and no analysis, and that gets called intelligence too.
Intelligence is the answer to a question, and usually to a specific set of questions. Information is the raw material collected to help answer those questions. It stays information until someone has analyzed it, checked it against other sources, and tied it back to the question being asked. Many facts about a target don't add up to intelligence on their own. Intelligence starts with a decision somebody has to make and the question that decision depends on.
We wrote about this discipline in depth in our intelligence requirements framework post. Intelligence Requirements set the strategic direction. Priority Intelligence Requirements narrow it to the threats that matter most. Specific Intelligence Requirements turn it into something a collector can go out and answer this week. Everything in that post comes back to one point. If you can't name the question, you aren't doing intelligence work, you're collecting.
A prompt typed into a chat window is a question, but it's rarely the right one. It's usually undirected and has no requirement behind it. And the answer comes back from one source.
Where AI actually fits
AI is useful for this work. It's fast at processing large volumes of open source material, translating it, summarizing it, pulling out entities, and turning a pile of documents into something an analyst can start working with. We use it that way ourselves. It saves real hours.
But look at where that sits in the intelligence cycle. You set requirements, you collect, you process, you analyze, and you disseminate. AI is strong in processing and useful in parts of open source collection. It contributes almost nothing to setting the requirement. It can't vouch for its own output. And it has never stood outside a building at 0500 to watch who shows up for the shift change.
Real intelligence work draws on every kind of source. Open source and cyber sources give you the public footprint, including records, filings, imagery, forums, leaked data, and exposed infrastructure. Technical collection tells you what the network and the devices are actually doing. Human sources and people on the ground tell you what the paperwork and the internet can't. That means the conversation with the site manager, a walk around the perimeter, and the difference between the published security posture and the guard who waves in anyone carrying a clipboard.
In practice this is simple. On a facility assessment, open sources tell us what's supposed to be there, including permits, vendor announcements, imagery, and job postings that list the security stack. The site visit tells us what's actually there. Those two pictures almost never match, and the gap between them is usually where the finding is. An AI summary only knows the first picture, and it will describe it with total confidence.
Five platforms agreeing is not five sources
This is the part that caught our traveler off guard. He said he checks important answers by running them through several AI tools, and if they agree, he trusts the result.
That feels like corroboration, but it isn't. Those models were trained on largely the same public internet and often pull from the same live search results. When they agree, that's usually one source repeated several times. If the original article was wrong, planted, or out of date, all five tools will confidently repeat the same mistake. Agreement between sources that aren't independent tells you nothing about whether a claim is true.
That's why analysis is its own step, and why the techniques behind it matter more now than they did before AI.
Four tests that turn information into something you can act on
Spielmann's 2014 work in the International Journal of Intelligence and CounterIntelligence lays out four techniques built on Richards Heuer's Analysis of Competing Hypotheses. They were written for national security threat analysis, but they apply directly to anyone deciding whether to act on what a tool or a feed just told them.
The zero-based review. Take the view everyone already believes and set aside the fact that it's popular. Then ask whether some other explanation fits the same evidence just as well. If two or more hypotheses are equally supported, the dominant one can't be treated as the answer anymore. In Spielmann's case study, seven other hypotheses explained the same evidence as well as the accepted one. Without any new collection, the favored view dropped from a certainty to a one in eight chance. AI output almost always shows up as the dominant view, because it's fluent, fast, and specific. This test forces you to ask what else that evidence could mean before you commit budget or people to it.
The amplified inconsistency test. Build a matrix with your competing hypotheses across the top and your evidence down the side. For each cell, ask one question. Does this piece of evidence conflict with this hypothesis? Then count the conflicts. The hypothesis with the fewest conflicts is the strongest, even if it isn't the one with the most supporting evidence. Supporting evidence is easy to find for almost any theory, so the useful signal is which explanations the evidence contradicts.
The source vetting double check. Normally you vet the channel. For a human source, you assess the person. For technical collection, you ask whether the collection path could have been compromised. If the channel checks out, the information gets treated as good. Spielmann adds a second step that sets the channel verdict aside and tests the content directly. You compare what the source reported against evidence gathered through unrelated channels. If the source is wrong or lying, those independent sources should contradict it somewhere, because an adversary would have to corrupt all of them at once to keep one lie intact. Your AI platforms fail this test for the reason above, since they don't count as unrelated channels. A satellite image, a public record, and a person who walked the site do count.
The missing data test. Ask what evidence should exist if your hypothesis is true, then compare that list with what you actually have. Every gap becomes a new collection requirement. If the evidence still doesn't turn up after real collection effort, the hypothesis is probably wrong. There's one important exception. If the hypothesis is otherwise strong and only a few things are missing, the target may be deliberately hiding them. AI can't run this test for you, because it only reports what it found and has no sense of what it should have found. On a physical assessment, the missing items are often the most useful ones, such as the camera that should be covering a door and isn't, or the visitor log that should exist and doesn't.
Of the four, we rely most on the zero-based review. The other three improve how you compare competing explanations. The zero-based review is what makes that comparison honest. Without it, the dominant view keeps collecting evidence, and people start treating the amount of evidence as proof. AI makes this problem worse, because it can produce a large amount of supporting material for whatever you asked about in a few seconds.
Why this matters outside the intel community
You don't need to be a government agency for any of this to matter. Every security leader is being sold intelligence right now. That includes threat feeds, dark web monitoring, AI-generated risk briefings, and executive exposure reports. Some of it is excellent. A lot of it is raw collection with no question behind it, no second source, and no analysis. Decisions still get made on it, such as which vendors to cut, which facility gets guards, which executive gets a protection detail, and whether a threat is real enough to act on.
When that material is just information with a professional-looking cover page, the decision is effectively a guess. And the risk is higher than it looks, because a well-formatted report makes a guess feel like a conclusion.
What you can run this week
Pull the last threat or risk report your team acted on and write down, in one sentence, the question it answered. If you can't, or if the question was only written after the report arrived, what you have is information. Next, check whether that question traces back to a real requirement your leadership set, or whether the report shaped your priorities instead of the other way around.
Take the three most important claims in that report and count the independent sources behind each one. Count AI outputs, aggregators, and articles that trace back to the same original as a single source. If any claim that drove a decision rests on one source, you now know how much confidence it actually deserved.
Run a zero-based review on the current consensus view inside your team, whether it's who is targeting you, why an incident happened, or where your biggest exposure is. Put three people in a room, give them only the evidence and not the conclusion, and ask them to build the best competing explanation they can. If they can make an alternative fit just as well, your consensus is weaker than everyone thought.
For one open question, build the inconsistency matrix on a whiteboard. List two or three hypotheses across the top and the evidence down the side, and mark only the conflicts. It takes about an hour, and it will show you which parts of the evidence you've been reading as support when they really aren't.
Write a missing data list for your leading hypothesis, meaning everything that should exist if it's true. Next to each item, write how you would collect it. Pay attention to how many items require someone on site, reviewing records by hand, or talking to a person. That list is the part of your intelligence gap that no tool subscription will close.
Finally, ask whoever produces your intelligence, whether it's internal staff, a vendor, or a platform, two questions. How was this validated, and what evidence would change your conclusion? If the answers are vague, you're being sold collection that's labeled as intelligence.
Where we come in
This is the work we do. We start with the question your leadership actually needs answered and build the requirements from there. Then we collect across every source that bears on it, including open source and cyber collection, technical assessment, and people on the ground who walk the site and talk to the humans involved. We run the analysis that turns all of that into an answer you can act on. It's what sits behind our OSINT investigations, our target and facility assessments, and the intelligence side of our vCISO work. AI is part of how we collect and process. The judgment and validation are done by people.
If your security decisions are resting on reports you can't trace back to a question and a second source, that's the conversation we should have.
We're based in Dallas, Texas, and we go wherever the problem is. Red Cell Security provides cybersecurity and physical security consulting to clients across the US and around the world.
Comments