November 13, 2011

Measuring Cyberfraud, the fall rate of sky, and other metrics from the market for Silver Bullets

The Economist also picks up on the "bursting the bubble" paper from Florencio & Herley:

BIG numbers and online crime go together. One well-worn assertion is that cybercrime revenues exceed those from the global trade in illegal drugs. Another nice round number is the $1 trillion-worth of intellectual property that, one senator claimed earlier this year, cybercriminals snaffle annually.

It is hard to know what to make of these numbers. Online crooks, like their real-world brethren, do not file quarterly reports. In the absence of figures from the practitioners, experts tend to fall back on surveys of victims, often compiled by firms that sell security software. These have a whiff of self interest about them: they are the kind of studies that get press released but not peer reviewed.

A paper by two researchers at Microsoft, Dinei Florencio and Cormac Herley, shows why: because losses are unevenly distributed. ....

So, if the existing numbers are bad (as I posted), where are the good cybercrime numbers? And, of course, how to better measure the real cost of cybercrime?

Well, one way to find better numbers is to ask the criminals. But, this is also flawed. For a start, this only measures their take, not the cost to the victim, which can often be out by a factor of 10:1. Also, "Online crooks, like their real-world brethren, do not file quarterly reports."

Notwithstanding these flaws, it may be better than surveys. And some results are in:

In the latest instalment of a mammoth four-year exercise Chris Kanich of the University of California, San Diego, and colleagues tracked around 20 outfits that use spam to advertise illegal online pharmacies. First they secretly monitored the spammers’ payment systems. Then they obtained logs from one of the servers that power the illegal pharmaceutical sites. They even ordered (and—perhaps surprisingly—received) some of the non-prescription drugs on sale.

Their findings suggest that only two of the 20 or so operators bring in $1m or more per month. The criminals behind fake security software appear to reap similar rewards, say Brett Stone-Gross and colleagues at the University of California, Santa Barbara. Their study, due to be presented at next month’s eCrime 2011 conference in San Diego, puts the annual revenue of each criminal group at a few tens of millions of dollars. As with Mr Kanich’s study, it is not clear how much of this is profit.

OK, so we can guesstimate that each sector - grey market pharma, and grey market anti-virus, or whatever we call the vendors of fake security software to differentiate them from the vendors of exaggerated security software - can do maybe 100m per annum over the lot, assuming a normal industry distribution in a market with free entry.

Which might suggest that phishing is also capped at around that number: 100m per annum across all players. Or might not... Have we got a better number?

Such hauls fall well short of extravagant claims from the security industry that some spammers make millions every day. Stefan Savage, Mr Kanich’s PhD supervisor, says that the security industry sometimes plays “fast and loose” with the numbers, because it has an interest in “telling people that the sky is falling”.

Say hallelujah to that! I'd say the jury is still out, one paper is not enough, and their conclusion aren't easy to extrapolate from. But $100m might be a closer number than a billion.

Posted by iang at November 13, 2011 04:21 AM | TrackBack

a couple years ago I was asked to find public source numbers on the subject. A interesting trivia was that all the major law enforcement websites had public sections for drug related crime ... but the computer crime sections all required authorized access. I did find some supportive (public) evidence but it was in very obscure indirect references.

One significant issue is that cybercrime can include non-internet financial crime that happens to involve dataprocessing. These events tend to be ones that large institutions (especially financial) are extremely adverse about making public. In the 90s financial infrastructure protection meetings, one of the major topics was whether the financial industry ISAC .. aka information sharing, would be subject to FOIA (the FI-ISAC specifically being structured so as to be not subject to FOIA)

Posted by: Lynn Wheeler at November 12, 2011 03:24 PM

While there is a natural desire to have a single number to describe cyber-crime losses, the complexities are such that I'm dubious in attempts to reach that number.

First, we have the problem with combining categories that have different kinds of "losses". For example, a "loss" to an on-line pharma sale seems qualitatively different from a "loss" from a stolen credit card. The former has many kinds of loss components, there is a potential public health loss if the quality of the drugs is sub-standard and there is a potential loss to the brand/patent holder (if we assume that the drug would have been legitimately purchased otherwise) but no real financial loss to the consumer. Indeed, the consumer purchased something and got something... this is a loss in the same way that I feel the money I spent on the Green Lantern movie was a loss. By contrast, a stolen credit card loss represents a real and direct financial loss (although ultimately not to the consumer... most likely to a merchant exploited in reshipping fraud) and also will introduce extraneous losses to the issuer (for new cards and the acquirer for processing chargeback transactions... although these are probably passed to the merchant). Intellectual property losses are even more complex to value.

However, even if we put these loss complexities aside and focus instead on the revenues generated by cybercriminals (within categories for which revenues are well defined... spam, phishing, bank account theft, etc) it is still difficult to extrapolate from one set of measurements to another. For example, the work by Kanich et al. provides an empirical mechanism for estimating of order volumes for a range of big on-line pharma vendors (if you assume ~$150-200/order you get close to ground truth for pharma). Based on this, it seems likely that on-line pharma is in the $100-150M/yr range. However, its unlikely that one can extrapolate from this data to another category like Fake Antivirus. For example, FakeAV can attract a much larger set of customers (larger inherent market for fear than for ED drugs) and has a lower cost structure. Thus, the recent paper by Stone-Gross et al. showed just two FakeAV programs pulling in $45M annually each (even more telling a third program, which was less well managed, only averaged ~$4mm). Similarly, I expect to find few similarities between phishing losses and losses due to banking trojans. Little similarity between credit card losses and those that involve ACH transfers from bank accounts. Each has its own operational complexities which can dramatically impact the revenue that can be streamed through.

If we care about making judgement that can be supported by empirical data I thin we need to couch our discussions within particular criminal ecosystems and the finances therein. Even this is hard, but its a goal I think we have experience with and some hope of achieving.

Posted by: Stefan Savage at November 12, 2011 09:07 PM
Post a comment

Remember personal info?

Hit preview to see your comment as it would be displayed.