The Pitfalls of Cyber Risk Assessment

In my previous post, I discussed the ambiguous nature of cyber risk. There was some healthy engagement on the topic, and I thought I would continue from where I left off.

In this two-part post I will explore some shortcomings of the cyber risk assessment practice and follow on by looking at when certain risk techniques should and should not be used.

The Cyber Risk Assessment

I’m going to start off with something that would be familiar to many of you: the risk workshop. For those that are unfamiliar, the basic premise of this risk activity is to bring people together to brainstorm ideas about “bad things that might happen”. These ideas are then typically prioritised on a company risk matrix (heat map) and later organised in a list, to be tracked and periodically communicated.

This process, whether done in a group or an individual setting, is fairly typical of most cyber risk assessment methodologies. A typical cyber risk assessment methodology will look something like this:

  1. Identify risks
  2. Analyse risks using risk matrix
  3. Evaluate them on risk matrix
  4. Track them in risk register

Whilst this may sound promising in concept, in practice it has several flaws. I will focus on two:

·      Identifying risks and judging significance

·      Judging probability & impacts

Identifying risks and judging significance

Whilst we like to think of ourselves as rational thinkers, decade’s worth of behavioural science research has found that we humans struggle to apply logic and sound reasoning to our thought processes. Instead, we base much of our thinking and decisions on our emotions and how we feel. To make matters worse, we have many (often inaccurate) pre-conceived notions about the world and how it works. These biases often misinform our judgement and get in the way of sound reasoning and good decision-making. When performing a risk assessment, these mental shortcomings make it very difficult for us to construct accurate, reliable and credible views on risk. (See Kahneman & Tversky’s work on behaviour economics for more details)

Judging Probabilities and Impacts

Risk assessments too will often lack the quality information needed to make proper assessments about probabilities and impact. Without consistent, accurate and substantiated data to support the risk identification & assessment process, risks along with their impacts and probabilities become merely uninformed (and likely emotional) arbitrary opinions. One tool that exacerbates our (already heavy) predisposition for probability and impact misjudgement and mismanagement of cyber risk is the commonly used risk matrix.

Whilst I will not go into too much detail here, I will say that there is significant research that casts doubt over the usefulness of risk matrices as a prioritisation/assessment tool. According to a 2008 paper on subject by Tony Cox, “use of a risk matrix to categorize risks is not always better than—or even as good as— purely random decision making.” One of the primary criticisms of risk matrices is their use of single point values to represent uncertainty. Indeed, it is highly unlikely that future uncertainty can be forecast with even remote accuracy using a single value.

In part two I will continue this discussion by examining the different situations in which certain cyber risk techniques should, and certainly should not be used.


Popular posts from this blog

The Limitations of Cyber Risk Assessment Tools

Defining Cyber Risk