The difference between a match event and a verdict
A threat intelligence feed is a data stream that regularly distributes lists of indicators of compromise, such as malicious IP addresses, domains, and file hashes. Security vendors and open communities collect them. Once the feed is integrated with NDR, the system automatically compares known indicators against the traffic passing through.
The comparison records only a match event: the fact that an indicator and the traffic lined up. In that respect it is no different from logs and alerts, which show only that something occurred. A match event alone cannot tell what actually passed through that session, or whether normal business communication happened to match the indicator. Adding more feeds does not remove that limit. The longer the list grows, the more alerts pile up whose grounds no one has confirmed, and analysts’ time goes to verifying alerts.
Three requirements for running threat intelligence feeds
An indicator of compromise (IOC) is a technical artifact showing that an attack is imminent or under way, or that a compromise has already occurred. This scope follows the glossary definition of the US National Institute of Standards and Technology (NIST). RFC 9424 from the Internet Engineering Task Force (IETF) extends the observables to the attacker’s tactics and techniques, tools, and infrastructure. To treat a feed as an operational asset, an organization must meet three requirements.
- Verdict: the organization must be able to confirm whether the traffic that matched an indicator is actually malicious. Verdicts based on summary information alone produce two errors at once: normal traffic gets reported as false positives, and disguised malicious activity passes through as false negatives.
- Quality evaluation: the organization must be able to measure which feeds are effective in its operating environment. A feed that gathers indicators from attacks worldwide mixes in many indicators irrelevant to the organization’s industry and region, and indicators aimed at a specific industry sometimes reach global feeds late.
- Retirement: the organization must be able to identify indicators that have lost their validity and remove them from the list. If an IP address registered months ago now belongs to a legitimate service but stays on the list without expiring, that indicator generates false positives instead of detections.
The feed list itself meets none of the three requirements. A list tells only that an indicator exists; it does not record how the indicator behaves in the operating environment.
The measured limits of cross-checking
The usual way to meet the three requirements within the lists themselves is cross-checking: trusting indicators that appear in several feeds and rating those found in only one lower. The rule itself is reasonable, but studies that compared real feeds at scale report that the rule has almost nothing to apply to.
One study collected commercial and open feeds together and measured how many feeds each indicator appeared in. It found that 73% of scan indicators and 88% of brute-force indicators existed in only one feed, and for botnet, exploit, malware, and spam indicators the share exceeded 97% (USENIX Security 2019). For these four types, comparing feeds in pairs showed that more than three-quarters of the pairs had an intersection below 1%. Whichever two feeds one picks, the indicators they hold barely overlap.
A study on feed quality evaluation likewise reports that of roughly 1.38 million indicators, only 6.2% were listed in two or more feeds (ACNS 2020). The same study also measured an average of 21 days between the start of malicious activity and an indicator’s listing in a feed.
Taken together, the two results define the role of cross-checking. Indicators confirmed across several feeds are highly reliable, but they are too few to serve as the first means of verification. With a listing delay on the order of 21 days, a list can hardly vouch for its own freshness. The input for the three requirements comes from data outside the list: the actual content of the traffic that matched the indicator.
Requirement one, verdict: the procedure for verdicts against the raw data
When a match event occurs, the process runs all the way to a raw-data check. Because every alert gets the same handling, false positives and false negatives are both identified at the final verdict stage. [Figure 1] shows the five stages, divided into automated stages and analyst verdict stages.
The automated stages in [Figure 1], confirming the match event and narrowing down the suspicious traffic with logs and metadata, run without human involvement. Analysts handle the stages after that. They retrieve the raw packets of the session to confirm the actual communication content. If the payload shows malicious activity, they rule it a threat; if it proves to be normal business communication, they rule it a false positive. Only cases confirmed as threats move on to blocking and evidence preservation.
When the raw transaction/session data is preserved, this stage becomes a verdict grounded in definitive evidence. To meet the verdict requirement, an organization must first check how much raw data it has preserved, before looking at the quality of its lists.
As verdict history accumulates, it yields more than verdicts: the false-positive rate each feed generates and the lifetime of each indicator type, measured in the operating environment. The quality evaluation requirement uses the former as input, and the retirement requirement uses the latter.
Requirement two, quality evaluation: five axes measured in the operating environment
Organizations evaluate feed quality along five axes. The industry has no single established standard framework; academic work proposes measuring four dimensions, timeliness, sensitivity, originality, and impact (ACNS 2020), or eight metrics (FeedMeter, ICISSP 2024). The five axes in [Table 1] recast that discussion from an operational standpoint.
Each of the five axes draws on different data. Accuracy is the only axis that takes verdict history as input. The other four are calculated from indicator receipt times, the detection history on hand, the overlap rate across feeds, and the additional information that accompanies indicators. A single axis shows only one side of a feed, so the five are evaluated together.
Separately from quality, the integration format needs checking as well. Structured Threat Information Expression (STIX) 2.1 and Trusted Automated Exchange of Intelligence Information (TAXII) 2.1, published as OASIS standards in 2021, are the current specifications. Without confirming the supported formats, even a high-quality feed cannot be integrated into the operating pipeline.
The same history also sets the priority for applying indicators. Applying every indicator at the same priority produces more alerts than the organization can judge. Indicators related to the organization’s industry and assets come first. Indicators linked to attack tactics and techniques, which are more useful in post-detection analysis, move up, and indicators with old creation and update times move down. Giving extra weight to indicators confirmed across several feeds is valid, but because such indicators are few, it does not serve as the main ranking criterion.
The scope of application widens in stages, and indicators of unconfirmed reliability do not go straight into blocking policy. They first apply to detection and monitoring only, to accumulate actual match history; once the raw data has been checked and a verdict reached, they move into blocking policy. The quality evaluation requirement, too, is met from the same input: verdict history.
Requirement three, retirement: lifetimes and aging criteria by indicator type
The organization sets review cycles and retirement signals together, by indicator type. When an attacker changes infrastructure or a domain is recycled, the same value comes to point at a legitimate resource. Keeping indicators that have lost their validity then raises both false positives and the comparison load. The review cycle differs by indicator type. Decay models, which score indicators and lower the score as time passes, and the lifetime policies that feed providers set for themselves both give IP addresses short lifetimes and file hashes long ones.
A long lifetime for hashes, however, does not mean a hash stays useful for detection that long. The verdict that the file a hash points to is malicious stays valid for a long time, but the hash’s usefulness as a detection tool is, if anything, the lowest of all. RFC 9424 notes that an attacker can defeat hash-based detection simply by recompiling code or making small changes to file content, and classifies hashes as the most fragile indicators. If the validity of a verdict and its usefulness for detection are not kept apart, the retirement cycle is miscalculated. [Table 2] summarizes the types.
When the review cycles in [Table 2] are fixed as numbers of days, outside values are hard to adopt as they are. The commonly cited figures rest on different grounds. The paper from the MISP project that proposed the decay model gives 7 days for IP addresses and 60 days for hashes (CIRCL 2018). These are example values to illustrate the model, and the paper itself states that they should be recalibrated through dataset analysis.
MISP’s default intrusion detection model applies the same 120-day lifetime to domains, IP addresses, and URLs. Example values from a paper and values set by default in a tool are different in kind, and no standard number of days per type has been established. The reference values must come from the organization’s own verdict history.
Two policies apply in parallel to the calculated reference values. Time-based expiration is simple to design but risks deleting valid indicators too early. Decay-based retirement reduces that risk by restoring the score to its original value whenever a new observation arrives. MISP’s decay model works this way, with lifetime, decay speed, and threshold as its parameters. The two combine with time-based expiration as the primary condition and the decay score as the secondary one.
Before retirement comes a step that checks past traffic against the indicators scheduled for retirement. Retrospective re-analysis runs newly acquired indicators against stored past packets and sessions, and it works only while the raw data remains within the retention period. If the lookup finds match history, the indicator is archived instead of retired. Deleting an indicator with match history also removes the reference point for later investigations.
Preserving the raw data pays off beyond retirement decisions. With raw data kept at the packet capture (PCAP) level, even when a vulnerability is disclosed later, the organization can determine whether it was compromised before the disclosure. The retirement requirement is met not by a list’s expiration date but by the result of a raw-data lookup.
The grounds for autonomous security operations: an input structure that connects lists to raw data
None of the three requirements can be met with the indicator lists that feeds distribute. Verdicts come from checking the raw data, quality evaluation from the resulting verdict history, and retirement decisions from lookups of past traffic. The three sources differ, but all of them rest on the raw traffic. Meeting all three together requires that the Raw Transaction/Session layer captured when the indicator matched remain preserved at the time of the verdict and at any later lookup.
Only with this structure in place can verdict automation rest on grounds. As attack execution becomes more automated, a cycle in which analysts check and judge indicators one by one cannot keep pace. What becomes necessary is closed-loop automation that runs from detection through analysis and judgment to response, or autonomous security operations. A match event alone does not settle that judgment.
The task of feed operations thus shifts from securing high-quality lists to securing a structure that connects lists to raw data. In autonomous security operations, the advantage comes not from the speed of response but from the completeness of the input.
Frequently asked questions
Q. Does adopting more feeds widen detection coverage?
The number of indicators grows, but unless verdict capacity grows with it, only unconfirmed alerts pile up. Before adding a feed, the recommended order is to build verdict history for the current feeds and evaluate them on the five axes in [Table 1]. Then confirm that the new feed covers ground the existing feeds do not.
Q. How can an organization prepare for needing to recheck a retired indicator later?
Even when an indicator is retired, keep its verdict history and match history separately. As long as the raw traffic remains within the retention period, past windows can be looked up again with new indicators, so the retirement policy and the raw-data retention period must be designed together.
