How To Measure Anything in Cybersecurity Risk
by Douglas W. Hubbard
and Richard Seiersen
Forward by Stuart McClure: CEO of Cylance
“What the authors of this book have done is begin to define a framework and a set of algorithms and metrics to do exactly what the industry has long thought impossible, or at least futile: measure security risk.”
Forward by Daniel E. Geer, Jr., ScD.: CISO In-Q-Tel
“It is my pleasure to recommend How to Measure Anything in Cybersecurity Risk….If you have any interest in taking care of yourself, of standing on your own two feet, of knowing where you are, then you owe it to yourself to exhaust this book. Its writing is clear, its pedagogy is straightforward, and its downloadable Excel spreadsheets leave no excuse for not trying.”
“At a time when forecasts tell you a great deal about the forecaster but nothing about the future, comes a practical guide for capturing and articulating risk in the board room with great success.”
—Tim McKnight, CISO, GE; former CISO, Fidelity
“A refreshing voice of reason in cybersecurity risk management. Richard and Douglas successfully rise above noisy security best practices and flashy methods; practitioners have a lot to gain from the clarity within this book’s pages.”
—Vinnie Liu, partner at Bishop Fox; author of Hacking Exposed; former NSA
“I am excited to see a new method of risk management emerging from this book. Shifting from purely qualitative judgments and simplifications to a proven quantitative model that leverages measurements and the expertise of security professionals holds the promise for dramatically shifting how we manage cyber risk.”
—Patrick Heim, head of Trust & Security, Dropbox; former chief trust officer, Salesforce.com
This book is the first of a series of spinoffs from Douglas Hubbard’s successful first book, How To Measure Anything: Finding the Value of “Intangibles” in Business. For future books in this franchise, we were considering titles such as How To Measure Anything in Project Management or industry specific books like How To Measure Anything in Healthcare. All we had to do was pick a good idea from a long list of possibilities.
Cybersecurity risk seemed like an ideal first book for this new series. It is extremely topical and filled with measurement challenges that may often seem impossible. We also believe it is an extremely important topic for personal reasons (as we are credit card users and have medical records, client data, intellectual property, and so on) as well as for the economy as a whole.
Downloads for Examples and Exercises from the Book Below
Welcome to the website for How to Measure Anything in Cybersecurity Risk. This is where readers can come to download examples mentioned in the book. These downloads include spreadsheet examples of the calculations, “Power Tools” and additional calibration exercises.
*Note: Some users may experience download issues while using Google Chrome.
Or try using Firefox, Internet Explorer, Edge, or another browser.
Contact us if the issue persists.
Chapter 3: Simple One-for-One Substitution Example
This spreadsheet contains examples for the “one-for-one substitution” model descrbed in Chapter 3. There are three tabs. The first tab shows how tables 3.2 3.3, and 3.4 are computed in the book. The second tab shows how we add “residual” risk so that we can plot the “loss exceedance curves” (LEC) as shown in figure 3.3. Finally, there is a tab showing how multiple portfolios can be added up to to make an aggregate LEC.
Chapter 6: Decomposition of One-for-One Substitution Model
This is a table showing the calculations used in Figure 6.1 of Chapter 6. This is one example of how further decomposition could be applied to the Simple One-For-One Substitution Example in chapter 3. As with the chapter 3 example, every time you hit F9 you get one more random example of the entire portfolio of security events. The reader is encouraged to use this example to develop different and more detailed decompositions of their own. We left off the data table to generate scenarios, risk mitigations and the histograms needed for Loss Exceedance Curves. All of this was shown in the Simple One-For-One Substitution Example and the reader is encouraged to try to add those elements to this model.
Chapter 7: Calibration Questions
Additional calibration tests in case the tests in the book weren’t enough to get you fully calibrated.
Chapter 7: Expected Distribution of Calibration Answers
These charts are not actually shown in the book but there is a reference in chapter 7 regarding how calibration answers should be distributed if, in fact, all of the test-takers were perfectly calibrated. The tables below show how the scores on the 90% CI test of a perfectly calibrated group would be distributed using the binom.dist() function in Excel for both the 10-question and 20-question tests function in Excel. In reality, uncalibrated groups of test takers will not fall mostly in the green areas as calibrated people do, but will fall mostly in the red areas, which indicate overconfidence.
Chapter 8: Bayesian Threat Intel Example
This is the major data breach example from chapter 8. The tables below contain the calculations necessary to compute the table on the right from the inputs in the table on the left (in yellow).
Chapter 9: Beta Dist Example for Three Industries
This shows how the beta distribution could be used to compare breach frequencies based on a few breaches in an industry. Data from 2014 to the end of 2015 is shown. You can set “alpha” and “beta” as shown in the book to reflect “hits” and “misses” (i.e., breaches and non-breaches per company per year) to see how the estimate of breach frequency will change with even a single new breach reported.
Chapter 9: Log Odds Ratio Example
This spreadsheet shows an example of how to use Log Odds Ratios (LOR) to apply several conditions to a single probability. This spreadsheet will estimate the conditional probability of a cybersecurity event given the aggregate effect of several conditions. Further details are given in the areas to the left below.
Appendix A: Selected Distributions
These are the calculations for Appendix A: Selected Distributions. There are several useful random distributions here that apply to a variety of different cybersecurity risks. The Binary (aka, Bernoulli) produces a “1” or “0” (which can be used as “event occurred” or “event didn’t occur.” It applies to whether a security event such as a data breach occurred on the first place. The other distributions are more appropriate for types of impacts that result after an event occurs – such as a system outage, a number of records compromised, the costs of legal liabilities, and so on. Yellow cells are for user inputs to define the distribution. If you want to simply copy the formula for another sheet, use the formula in the “Random Result” cell. NOTE: The formula for the triangular was wrong in the book! The formula was incorrectly transcribed in the first place and then a last minute change before printing was incorrectly applied to such an extent that what was left was unrecognizable as anything like the triangular distribution. The formula below, however, is correct.