How To Measure Anything in Cybersecurity Risk (2nd Edition)

by Douglas W. Hubbard
and Richard Seiersen

“Cybersecurity practitioners are overloaded with new threats, new vulnerabilities, new tools, and with constant pressure to keep pace with their organizations’ ever changing needs. To address this complexity and variability, we need the knowledge, mechanisms, and means to accurately prioritize initiatives, measure risk, and evaluate proposed solutions. This book enables the reader to confidently move forward in a cybersecurity landscape that grows in complexity daily.”

Jason Chan, Former VP Information Security Netflix

“Companies can mitigate, transfer, or ignore their cyber risk. But in order to smartly manage it, CISOs must first quantify their risk. This book arms the CISO of the future with the tools needed to make business relevant decisions. These tools will empower leaders to build organizational cybersecurity resilience against an ever-changing cyber risk landscape.”

Vishaal “V8” Hariprasad, CEO, Resilience and Former Active Duty Cyber Effects Operations Officer, US Air Force

“How to Measure Anything in Cybersecurity Risk makes a strong case for quantifying what organizations are up against. This book is equally valuable for technical security practitioners, risk and compliance professionals as well as security leaders. It provides not only concepts but an essential roadmap for measuring risk, developing metrics, and determining return on investment. I recommend anyone looking for a scientific approach for measuring cybersecurity risk to take advantage of Doug Hubbard and Richard Seiersen’s expertise on this topic. “

Gerhard Eschelbeck, Former CISO at Google

“Whether you are a quantified risk skeptic or fan, this book will teach you new ways to think about the problem. I consider it mandatory reading for our field.”

John “Four” Flynn, CISO, Amazon Stores

Forward by Stuart McClure: CEO of Cylance

“What the authors of this book have done is begin to define a framework and a set of algorithms and metrics to do exactly what the industry has long thought impossible, or at least futile: measure security risk.”

Forward by Daniel E. Geer, Jr., ScD.: CISO In-Q-Tel

“It is my pleasure to recommend How to Measure Anything in Cybersecurity Risk….If you have any interest in taking care of yourself, of standing on your own two feet, of knowing where you are, then you owe it to yourself to exhaust this book. Its writing is clear, its pedagogy is straightforward, and its downloadable Excel spreadsheets leave no excuse for not trying.”

New Material For 2nd Edition!

This book is the first of a series of spinoffs from Douglas Hubbard’s successful first book, How To Measure Anything: Finding the Value of “Intangibles” in Business. For future books in this franchise, we were considering titles such as How To Measure Anything in Project Management or industry specific books like How To Measure Anything in Healthcare. All we had to do was pick a good idea from a long list of possibilities.
Cybersecurity risk seemed like an ideal first book for this new series. It is extremely topical and filled with measurement challenges that may often seem impossible. We also believe it is an extremely important topic for personal reasons (as we are credit card users and have medical records, client data, intellectual property, and so on) as well as for the economy as a whole.

Do current risk assessment methods in cybersecurity work? Recent big security breaches have forced business and government to question their validity. Is there a way to fix them? How can risk even be assessed in cybersecurity?

Find the answers to these questions and more in our introduction webinar, How to Measure Anything in Cybersecurity Risk. 

This two-hour webinar will change how you view cybersecurity and give you the tools to begin finding these critical answers – and better protecting your organization. Click on the button below to learn more.

Downloads for Examples and Exercises from the Book Below

Welcome to the website for How to Measure Anything in Cybersecurity Risk.  This is where readers can come to download examples mentioned in the book.  These downloads include spreadsheet examples of the calculations, “Power Tools” and additional calibration exercises.

Chapter 3: Simple One-for-One Substitution Example

This spreadsheet contains examples for the “one-for-one substitution” model descrbed in Chapter 3. There are three tabs.  The first tab shows how tables 3.2 3.3, and 3.4 are computed in the book.  The second tab shows how we add “residual” risk so that we can plot the “loss exceedance curves”  (LEC) as shown in figure 3.3.  Finally, there is a tab showing how multiple portfolios can be added up to to make an aggregate LEC.

Chapter 3: (Updated for 2nd Edition) Rapid Risk Audit and Simple Monte Carlo Example

As explained in Chapter 3 of How to Measure Anything in Cybersecurity Risk, the Rapid Risk Audit is a very basic, yet quantitative assessment of risks. Like qualititative methods such as the Risk Matrix, the inputs can be subjective. But although estimates may be subjective, the quantitative estimates avoid some of the errors introduced by risk matrices. This Rapid Risk Audit proposes a cause-effect taxonomy, but it could be used for any taxonomy. It will produce a total expected annual loss. But it can’t produce a probability of losing various amounts in total. For that, we need the next worksheet, the Monte Carlo simulation.

Chapter 5: (Updated for 2nd Edition) HDR Cybersecurity Survey

This was the survey conducted in 2016 for the first edition of this book.  Now we are making the entire survey free for downloads.

Chapter 6: Decomposition of One-for-One Substitution Model

This is a table showing the calculations used in Figure 6.1 of Chapter 6.  This is one example of how further decomposition could be applied to the Simple One-For-One Substitution Example in chapter 3.  As with the chapter 3 example, every time you hit F9 you get one more random example of the entire portfolio of security events.  The reader is encouraged to use this example to develop different and more detailed decompositions of their own.  We left off the data table to generate scenarios, risk mitigations and the histograms needed for Loss Exceedance Curves.  All of this was shown in the Simple One-For-One Substitution Example and the reader is encouraged to try to add those elements to this model.

Chapter 7: Additional Calibration Questions

Additional calibration tests in case the tests in the book weren’t enough to get you fully calibrated.

Get the REAL calibration training here! HDR provides asynchronous training with self-paced videos so you can practice with proven methods, all while seeing your real-time calibration results on the calibrator dashboard. The data can even be used to optimize estimates on real-life problems with Team Calibrator!

Chapter 7: Expected Distribution of Calibration Answers

These charts are not actually shown in the book but there is a reference in chapter 7 regarding how calibration answers should be distributed if, in fact, all of the test-takers were perfectly calibrated. The tables below show how the scores on the 90% CI test of a perfectly calibrated group would be distributed using the binom.dist() function in Excel for both the 10-question and 20-question tests function in Excel. In reality, uncalibrated groups of test takers will not fall mostly in the green areas as calibrated people do, but will fall mostly in the red areas, which indicate overconfidence.

Chapter 8: Bayesian Threat Intel Example

This is the major data breach example from chapter 8.  The tables below contain the calculations necessary to compute the table on the right from the inputs in the table on the left (in yellow).

Chapter 8: (Updated for 2nd Edition) Bayesian Multifactor Authentication

This is an example of how to use a Bayesian method to update the probability that Multifactor Authentication is working even with very limited observations.

Chapter 9: Beta Dist Example for Three Industries

This shows how the beta distribution could be used to compare breach frequencies based on a few breaches in an industry.  Data from 2014 to the end of 2015 is shown.  You can set “alpha” and “beta” as shown in the book to reflect “hits” and “misses” (i.e., breaches and non-breaches per company per year) to see how the estimate of breach frequency will change with even a single new breach reported.

Chapter 9: (Updated for 2nd Edition) Log Odds Ratios for Simple Lens and SME Aggregation

This spreadsheet has two worksheets. The first is the LOR example in Chapter 9 which provides a quick subsystem risk based on past Lens models created by HDR. The second worksheet shows how the estimates of two SMEs might be combined using a highly simplified LOR method.

Chapter 9: Log Odds Ratio Example

This spreadsheet shows an example of how to use Log Odds Ratios (LOR) to apply several conditions to a single probability. This spreadsheet will estimate the conditional probability of a cybersecurity event given the aggregate effect of several conditions. Further details are given in the areas to the left below.

Appendix A: Selected Distributions

These are the calculations for Appendix A: Selected Distributions. There are several useful random distributions here that apply to a variety of different cybersecurity risks. The Binary (aka Bernoulli) produces a “1” or “0” (which can be used as “event occurred” or “event didn’t occur”). It applies to whether a security event such as a data breach occurred in the first place. The other distributions are more appropriate for types of impacts that result after an event occurs – such as a system outage, a number of records compromised, the costs of legal liabilities, and so on. 

Appendix A: (Updated for 2nd Edition) Probability Distributions in Excel

These are some basic probability distributions we often use in Excel.  These include formulas for the random number generation.  Histograms are plotted for each.