Why is Threat Detection Hard?
(By Anton Chuvakin and originally posted at Anton on Security)
While creating a recent presentation, I needed a slide on “threat detection is hard.” And it got me thinking, why is threat detection so hard for so many organizations today? We can trace the “cyber” threat detection to 1986 (“Cuckoo’s Egg”) and 1987 (first IDS) and perhaps even earlier events (like viruses of the early 1980s). This means we are “celebrating” ~35 years of cyber threat detection.
As I result, I updated my slide to this:
Anton’s Why Is Threat Detection Hard?
Now, let’s talk about it as this can be useful to those organizations that are in the beginning stages of their detection journey.
To start, surely many people think that threat detection is hard because threat actors do not want to be detected (duh!). This is an understandable, but, in my opinion, a naive view. Attackers do need to remain unseen until their goals are accomplished, but the reasons for why they are unseen often have nothing to do with their craft. For sure, this argument does come up for the case of a top-tier actor facing an excellent blue/defense team. However, I’d say that other reasons below play a bigger role for most cases.
Now, my favorite top reason for why threat detection (of most/all forms) is hard: because most organization’s IT is a mess. Think sensitive data all over the place, “rogue” systems and connections, unmanaged systems and components (good argument here), layers of legacy technologies piled on top of each other (think mainframe linked to SOAP API connected to middleware and then to a mobile app). This is just bad terrain for a defender looking to spot the attacker early. BTW, perhaps belated realization of this is what gave rise to so many new asset discovery startups…
Next, despite all the automation (SIEM, UEBA, EDR, SOAR, etc), many detection activities will rely on people (and, as my former favorite co-author would add, process too). For organization in lower tiers of the maturity scale, “people are hard, boxes are easy.” People need hiring, training, retaining, morale improvement etc. Scaling teams is hard for everybody. Threat hunting, naturally, is even more people-centric.
Next, detection runs on data. This does make it substantially different from “block this” or “only allow that” (and, of course, I know that some prevention runs on data too, this is not the point, this is still true). Data needs to come from many sources, some incomplete and some lacking context. Some comments added specific points how lack of context makes detection activities hard. Very often, lacking business context does you in (this comment).
Also, detection activities deliver signals that need to be triaged and confirmed. This partially falls into the above (detection needs people), but also touches on the inherent property of “false positives” and “false negatives.” The “false positives” need to be cleared by more technology (like IDS -> SIEM -> SOAR), people or (most likely) both. There is also overall uncertainty with finding weak signals, whether you do it with rules or with ML. Sadly, teams with traditional IT mindsets often cannot work with uncertainty, inherent in our beloved domain of cyber. Hence “Need detection? Just install a detection tool!” thinking fails spectacularly.
Notice, by the way, that the data argument, the people argument and the triage argument are deeply interconnected. Detection based on incomplete or garbage data and lack of context will make triage harder and will increase the load on people too….
Finally, and this is fun, new one: very often badness detection is about detecting intent, not the activity. Practicality, this equates to intuition and inference yet again, something that again calls for people skills and not machines. An example: here is a connection to port 443 from this IP. Good/bad? Sure, adding context may help (What IP? What else happened? What preceded it?), but it may still prove insufficient in our attempt to deduce intent. Even “known bad” may have a good intent (ever confused a pentester for an attacker?). This does make detection even harder.
Action items? Well, this was more of a musings post, but perhaps this: meditate on your threat detection mindset. Do you crave 100% certainty? Do you expect full automation? Do you have gaps in coverage? Do you over-invest in tools over people and process? Do you think about detection as a product feature and not a process? These and other questions may render better results than some of the tools….
P.S. Cyber security awareness month is here, so perhaps treat this post as my back to basics contribution…
P.P.S. Thanks to Brandon Levene for his ever-insightful comments.
Possibly related posts:
- “Detection Coverage and Detection-in-Depth”
- “Can We Have “Detection as Code”?”
- “Chronicle Road to Detection: Context — Part 1 of 3” (some detection ideas there too)
- Related SANS paper and SANS webinar (featuring the old version of the above visual)