I've been working on a talk tentatively titled "Myths and Lies in InfoSec"
Some of the research I'll be referencing in the talk was inspired by one particular stat: "60% of small businesses go out of business within 6 months of a data breach"
How do we know a stat is fake? 🧵⏲️
Anybody know where this iffy stat came from? I’ve seen it attributed to several organizations, but its actual origin remains murky.
12
25
127
Before we get into hunting for the truth, why do I debunk myths and fake stats in the first place?
I've always had a deep desire to understand how things work, which I think led to computer/IT work, where I did a lot of root-cause analysis.
1
16
My work in IT, I think, naturally led me to security, where the entire industry is one huge discipline and business of root-cause analysis.
That takes us to the flip side - why are fake stats even a thing?
1
2
12
Confirmation bias perhaps? With big mysteries, anything that gets us closer to a better understanding is comforting.
Quoting yourbias.is: "think of your ideas and beliefs as software you're actively trying to find problems with rather than things to be defended"
1
10
When your job is sales, evangelism, thought leading, or any other SME/expert role, there's temptation to reach for stats that suit your narrative, and skip any due diligence or fact-checking process.
Feynman says it well
1
1
14
Comfortable though confirmation bias may be, I won't be wrapping myself in a blanket of lies, so on we go to the debunking process!
👇🤷🤔
1
5
Hunting down the infamous 60% SMBs die after an attack stat
Phase One: What's wrong with this stat?
This one immediately sounded impossible to me. I've been exposed to small business statistics before, and I knew there were MILLIONS of SMBs out there.
1
6
Often, before tracking down the source of the statistic, I want to get some context.
How many small businesses are out there?
In the US alone (where I am), there are over 30 million small businesses. Only 19% of these have employees and there are only 20k "large" employers.
2
1
7
So, over 6 million small US businesses have employees (e.g. not a sole proprietorship), but how many are getting breached?
Wait, first, what's a small business?
1
7
Well, if we consult the SBA's 1000+ line "table of size standards" spreadsheet, we find that if you make less than $2M farming soybeans, you're a small business.
But if you make up to $4M farming mushrooms, you're still a small business!
So THAT'S not straightforward at all...
1
1
7
Verizon's VERIS VCDB has 9134 breach records. Since "small" is complicated, we'll just filter on the dataset's victim.orgsize.Small field.
That leaves us with 2861 records. 2230 with non-US businesses filtered out.
60% of that is 1338. We'll remember this number for later.
2
7
Disclaimer: It's not reasonable to expect that 100% of small business breaches get reported, but we'll work with the data we have. Imperfect context is better than none.
So far, we know there are ~6 million small businesses in the US and roughly 0.04% of them have had incidents
1
6
(BTW, all breaches are incidents, but not all incidents are breaches)
Looking at the 2022 DBIR, it's clear the VCDB is far from comprehensive.
Sadly, most breach data is still treated as confidential, so the general public can't analyze or learn from it directly.
1
6
Alright, with some general stats on small businesses and breaches, let's track down the source of this fake statistic.
Searching on the general text of the statistic, Google returns Cybersecurity Ventures first.
cybersecurityventures.com/60…
1
7
The site it references no longer exists, but is not forgotten.
unbrokerage.com/blog/educati…
The Wayback Machine remembers and redirects it to a blog post that also no longer exists (on withlayr.com), but archive.org still remembers.
web.archive.org/web/20210316…
1
7
This blog post doesn't even contain the 60% stat. It has one that's EVEN MORE unbelievable:
"more than half of all small businesses suffered a cyber breach in 2019"
Depending on how they're defining "small business", that means either 15M or 3M small businesses got hit in 2019
3
1
6
With the most conservative math possible, that's less than 1% of breaches getting reported in 2019, and if combined with the other stat, 1.8 MILLION businesses destroyed by a breach.
Could 1.8M small businesses get hacked into oblivion in 1 year without anyone noticing?
1
7
Oddly, the blog post that doesn't mention the 60% stat cites a CNBC article for the even less credible stat we just discovered.
The CNBC article doesn't mention the crazier stat, but DOES mention the 60% stat! 😵💫
cnbc.com/2019/10/13/cyberatt…
1
5
The CNBC article cites an INC article, which cites the National Cyber Security Alliance, which is where most 60% stat trails ultimately lead.
inc.com/joe-galvin/60-percen…
1
1
8
This is unfortunate, because the NCSA has an article dedicated to retracting this stat, stating:
"This statistic was not generated from NCSA research, and we cannot verify its original source."
staysafeonline.org/resources…
1
11
The NCSA very much did their part in spreading the stat before the retraction, stating it both in an infographic and a press release.
prnewswire.com/news-releases…
1
8
So, is that it? Are we at the bottom of the research rabbithole?
Not quite! Inspecting the press release, we find reference to a Business Insider article, which was contributed by Smallbiztechnology.com, run by a Ramon Ray.
1
6
And this is the bottom, as far as the Internet is concerned. Ray casually mentions the 60% stat at the bottom of the Business Insider piece, with no source for it.
As I began to research Ray, however, I found I was not the first to research this fake stat!
1
5
I discovered a piece published on NextGov by @Joseph_Marks_ titled, "How a Fake Cyber Statistic Raced Through Washington".
Unlike me, Marks is a proper journalist, so naturally he reaches out to Ray and directly asks him what the deal is with the stat:
1
10
Sadly, Ray continued to use the stat, and his website has published several examples of it just in the past 2 years, most recently in late 2021.
smallbiztechnology.com/archi…
1
10
The damage of fake stats like this really can't be understated. Even if used to "get a positive result", fake stats hurt our ability to understand and solve security problems.
1
3
14
This particular stat has been cited in dozens of vendor reports, proposed federal legislation, and continues to regularly show up in marketing materials, blog posts and conference talks.
The NCSA's retraction will do little to discourage reuse.
1
6
This research inspired me to begin compiling a list of all the businesses that have been destroyed by a cybersecurity incident. I've been maintaining it for roughly 8 years now.
There are currently 23 companies on the list. All are small businesses.
docs.google.com/spreadsheets…
1
8
39
Also, if you want to hear more about myths and lies in InfoSec, I chatted a bit with @AllanAlfordinTX about it on The Cyber Ranch Podcast
linkedin.com/feed/update/urn…
Aug 1, 2022 · 3:19 AM UTC
2
1
13
