Cyber Event Risk Quantification: 4 Common Issues with Calculating Threat Event Frequency for Web Apps

By Bryan Smith

Bryan Smith is CTO of RiskLens and leads development of the RiskLens enterprise cyber risk quantification platform and other quantitative risk management products and services.

Appropriately quantifying the risk presented by cyber events to your organization is paramount to making good business decisions around cybersecurity, which is the point of using the FAIR model. But, because the model relies on human input, a simple error can lead to incorrect results and, as a result, a misrepresentation of an organization’s cyber risk.

When it comes to the frequency half of the FAIR model, I consider Threat Event Frequency (TEF) king. Yes, in Factor Analysis of Information Risk (FAIR™), all of the ontology is important. But mistakes with Threat Event Frequency act as a multiplier. 

We see this when analysts new to FAIR are working on Threat Event Frequency for web applications. A common use of RiskLens is to prioritize remediation efforts for web applications, and using the FAIR model provides a consistent and contextual way to identify what matters.

If TEF is incorrectly derived, the analysis can run aground. Fortunately, most mistakes made with Threat Event Frequency are simple to correct and easy to avoid.

Threat Event Frequency in FAIR Cyber Risk Analytics

irst, let’s do a quick recap of Threat Event Frequency. “Threat Event Frequency is the probable frequency, within a given time frame, that threat agents will act in a manner that may result in loss (from the FAIR book, Measuring and Managing Information Risk).” That’s a mouthful, so let’s unpack it:

1. Probable frequency — We’re dealing with how often something will happen. In this context, how often bad guys launch cyber attacks on our web application.
2. Given Time Frame — We have to pick a window of time in which we will count events. In RiskLens, we always deal with annualized time. So, one year.
3. Threat Agents — A threat agent (or threat actor) can be many things. A person, a group (think the hacking group Anonymous), malware, etc. The important part is that the agent is acting in a malicious way against your web application (I’m assuming the risk scenario context here is external hackers).
4. May Result in Loss — Just because an attack happened doesn’t mean a loss happened. But it’s important to remember the potential for loss was there.

Keep this definition in mind as we go over these common mistakes and how to avoid them.

More on Threat Event Frequency from the FAIR Institute

Those who are just learning FAIR occasionally confuse Threat Event Frequency (TEF) with Loss Event Frequency (LEF), and this is one of the most-missed questions on the certification exam. There is a strong similarity between the definitions of TEF and LEF. The only difference is that the definition for TEF doesn’t include whether a threat agent’s actions are successful. TEF captures the number of attempts (by a threat actor) to cause harm to an Asset. A common example of a malicious threat event (where harm or abuse is intended) would be the attacker (Threat Actor) who unsuccessfully attacks a web server. Such an attack would be considered a threat event, but not a loss event.

Mistake #1 – Confusing Contact Frequency with Threat Event Frequency

This is by far the most common mistake analysts new to FAIR make. A great example of this is using security logs from a Web Application Firewall (WAF). A WAF can be a great source of data to derive Threat Event Frequency. However, it’s important to understand individual log entries are contact events, not threat events.

To avoid this, the analyst needs to use a consistent method to group contact events into threat events. Not doing so will overstate Threat Event Frequency and make it appear that significant cyber incidents are occurring more often than there actually are. Using concepts such as Imperva’s notion of a Battle Day can help. Just remember that web application contact frequency will likely be grouped into threat events. If you’re not grouping, you’re likely making a mistake.

Mistake #2 – Not Defining Where in the Attack Chain You’re Measuring TEF

For example, if you’re analyzing ransomware risk, we recommend measuring TEF once the ransomware is on an endpoint and not earlier — in other words, before firewalls and filtering introduce noise. Find the spot where the data is easiest to understand and most available. You need to know where to measure so you can define how to measure and achieve consistent measurement; this will also help you align the data you have or see the estimates you need. And, in the above example, it is simpler to measure because it’s always a threat (not a Contact Frequency confusion). This leads us to…

Mistake #3 – Confusing Intent: Is It Really a Cyber Event?

Not all attacks look alike. Actually, not all attacks are attacks. Remember, in our definition above, that the threat actor must be malicious and may cause a loss. Is a scan of a system malicious? Some may say it is, but in FAIR’s definition of TEF, it is not. A scan also doesn’t lead to a loss.

To avoid this, be sure to understand what your data is showing you. Is this a scan (simple contact) or an attack? If it’s a scan, don’t include it in the final Threat Event Frequency. Or include scans as part of an attack, but call it the opening move; just one move among many in a single attack.

Mistake #4 – Not Doing a Smell Test

You’ve crunched the numbers and the data says X. You put the results in and blissfully jump to the next question. Stop. You’re not done — you’ve skipped an important step. Not doing a smell test, or taking a critical look at the results in context of the full risk scenario, is a mistake. A mistake that allows mistakes #1 and #3 to creep into your analysis.

To avoid this, add a “smell test” task to your workflow. A smell test is a simple step that adds critical thinking for quality control. Come back to the answer later, after you’ve gotten your head out of the data. Ask yourself if it still seems realistic. Does it make sense given the context of the risk scenario? If not, go over it once more. Apply the definition to the data again, and see if you fell for a common pitfall and, if you did, correct it accordingly.

If you can avoid these 4 mistakes, your organization will be able to build better plans for incident response based on correct quantification of the likelihood of a cyber event taking place. If you want to learn more about cyber risk quantification and the RiskLens platform, contact us and schedule a demo today.