Classifying False Positives — The FP Triage Process

You deployed a rule. It’s generating alerts. Now what? The first question isn’t “how do I suppress the noise” — it’s “what IS the noise?” Misclassifying an FP type leads to wrong fixes that either don’t reduce noise or accidentally suppress real attacks.

The Three Alert Categories

Every alert that fires belongs to exactly one of three categories:

Alert fires
    │
    ├─ Did it fire for a real attack?
    │   YES → True Positive (TP)
    │           └─ Investigate, respond, close
    │
    ├─ Did the rule logic fire correctly, but for legitimate behavior?
    │   YES → Benign True Positive (BTP)
    │           └─ Add targeted exclusion
    │
    └─ Did the rule fire for something it wasn't designed to detect?
        YES → False Positive (FP)
                └─ Fix rule logic

True Positive

A real threat. An attacker (or red team) performed the technique the rule is designed to detect.

Action: Investigate. Determine scope, impact, lateral movement. Escalate. After closing, update the ADS with what the attack looked like in production.

Benign True Positive (BTP)

The rule fired correctly — the event matched the detection logic as designed — but the behavior is legitimate. Example:

An LSASS-access rule fires for Windows Defender scanning memory
A network connection rule fires for a vulnerability scanner performing authorized scanning
A scheduled task creation rule fires for a software deployment system creating scheduled tasks as part of installation

Critical distinction: The rule is not broken. The logic is correct. The problem is that the rule’s scope is broader than the threat — legitimate behavior shares the same signature.

Action: Add a targeted exclusion for this specific legitimate use case. Do NOT change the rule logic (which would reduce recall). The ADS falsepositives section should list all known BTP sources.

False Positive (FP)

The rule fired for something it wasn’t designed to detect at all — a logic error, a data quality issue, or unexpected field behavior.

Examples:

A rule checking CommandLine|contains: '-enc' fires because a legitimate script uses Remove-Item -Encoding UTF8 (-Enc matches)
A rule targeting EventID: 4624 AND LogonType: 3 fires because the pipeline is incorrectly mapping LogonType values
A rule checking DestinationPort: 4444 fires because an internal monitoring tool coincidentally uses that port

Action: Fix the rule logic. Add more specific conditions. Test against known-good and known-bad data.

The FP Rate Formula

FP Rate = FP alerts / (FP alerts + TP alerts)
         = FP / Total alerts

A rule with:

90 alerts/day: 81 FP + 9 TP → 90% FP rate
After tuning: 30 alerts/day: 6 FP + 24 TP → 20% FP rate

Precision is the inverse: Precision = 1 - FP Rate (approximately, for large volumes)

Both metrics tell the same story — precision is used when measuring improvement, FP rate when describing the problem to stakeholders.

The FP Triage Process

Don’t tune by feel — tune by data.

Step 1: Sample

Pull 20–30 recent alerts from the past 7–14 days. Choose a random sample, not the ones that “look like” FPs or TPs. Sampling bias is the most common tuning mistake.

Step 2: Classify

For each sampled alert, classify it:

Alert	TP / FP / BTP	Evidence	Distinguishing feature
Alert #1	BTP	Windows Defender accessing LSASS	SourceImage = MsMpEng.exe
Alert #2	TP	Confirmed Mimikatz execution	SourceImage = C:\Users…\mimikatz.exe
Alert #3	BTP	Crash reporter (WerFault.exe)	SourceImage = WerFault.exe
Alert #4	FP	LogonType mapping error	LogonType = 0 (no network logon at all)
Alert #5	BTP	EDR agent memory scan	SourceImage = C:\Program Files\SentinelOne…

Step 3: Identify Dominant Sources

Tally the BTP/FP sources. Typically, 80% of noise comes from 2–3 sources. Fix those first.

Sample of 25 alerts:
  2 TP (real attacks, correctly detected)
  12 BTP — Windows Defender (48%)
  8  BTP — SentinelOne agent (32%)
  2  BTP — WerFault.exe (8%)
  1  FP  — LogonType mapping bug (4%)

Three exclusions (Defender, SentinelOne, WerFault) handle 88% of the noise. The LogonType bug needs a rule fix.

Step 4: Write Targeted Exclusions

For each BTP source, write the most specific exclusion that suppresses it without broadening the blind spot:

filter_defender:
  SourceImage: 'C:\Program Files\Windows Defender\MsMpEng.exe'

filter_sentinelone:
  SourceImage|startswith: 'C:\Program Files\SentinelOne\'

filter_werfault:
  SourceImage: 'C:\Windows\System32\WerFault.exe'

Note: full path, not just filename — an attacker could name their process MsMpEng.exe.

Step 5: Validate and Remeasure

After adding exclusions:

Run the rule against the same historical dataset with exclusions applied
Verify TPs still fire (exclusion didn’t accidentally suppress real attacks)
Verify BTP samples no longer fire (exclusion works)
Sample 20 more recent alerts and reclassify — measure new FP rate

Detection Engineering Weekly— Detection Engineering Weekly

Retrieval Practice

You sample 30 alerts from a rule detecting PowerShell with -EncodedCommand. You find: 4 TPs, 18 BTPs (all from a legitimate remote management tool that encodes commands for transport safety), 6 FPs (the rule fires on '-Encoding UTF8' in CommandLine). What is the FP rate for the full alert population, and what are the two different fixes needed?

Reveal answer

FP rate = FP / total = 6/30 = 20% of alerts are FPs. BTP rate = 18/30 = 60% of alerts are BTPs. Total non-TP rate = 80%. Two different fixes: (1) For the 6 FPs: fix the rule logic — add `CommandLine|contains|windash: '-enc '` (with a space after) instead of just '-enc', or use '-EncodedCommand' as the trigger. '-Encoding' will not match '-EncodedCommand' if the space is required. (2) For the 18 BTPs: add a targeted exclusion filtering by the remote management tool's ParentImage or specific binary path. These are separate problems requiring different solutions — merging them risks fixing one while making the other worse.

Retrieval Practice

A colleague argues: 'Our alert volume is too high, let's just increase the threshold — if a rule fires once, ignore it; only alert if it fires 3 times in 10 minutes.' What are the risks of this threshold-based noise reduction?

Reveal answer

Three risks: (1) Low-and-slow attacks that intentionally stay under the threshold (one action per 10 minutes) become completely invisible. The threshold creates an explicit evasion floor. (2) High-value, single-shot techniques (e.g., creating a domain admin account — T1136.002) will never reach a threshold of 3 because an attacker only needs to do it once. Thresholds are appropriate for volume-based techniques (brute force, port scan) but dangerous for technique-specific detections. (3) It doesn't address the root cause — the FP sources. The noise continues, just delayed. The right fix is reducing FP rate via exclusions so each single alert warrants triage, not raising the threshold to hide the noise.