Sep 03 2011

Is this really the ‘State of Security’?

Published by at 7:46 am under General,Risk,Security Advisories

I’m not a big fan of opinion polls, especially when the people writing them present them as if they were facts, rather than simply opinions of the people polled.  There’s a huge difference between the reality we live in and the way we perceive that reality.  That’s simply a fact of life, not a criticism of anyone in particular.  But it has a huge impact on the real usefulness of data when it’s based on perception rather than a quantifiable measurement.  And in the information security field, we’ve been working on perception and intuition for far to long and need to start relying on real, measurable data instead.  I have been told I’m too hard on polls, since opinions are valid data points as well, but I’m not so certain.

That’s quite an opening statement for a look at the latest security survey by Symantec, but I wanted to get my own personal biases out of the way before starting.  It also help explain some of my skepticism of the value of the Symantec State of Security 2011 report.  It’s pretty, it’s glossy, it has nice pictures, but it’s still an opinion poll and I always have to wonder how much it’s been affected by the perception of the people who were surveyed, how much they were willing to answer honestly and whether or not they actually knew the answers or just made stuff up.  As I said, I’m not a big fan of opinion polls in security, in large part because I’ve filled out more than a few of them myself.

There’s a lot of white space, large type and big graphs in the the report.  Padding that should have been replaced with more analysis and discussion rather than being wasted.  Which tells me this was probably produced by the marketing department rather than someone in engineering or security.  Marketing might have gotten the analysis right, but the 19 pages would have been boiled down to 8 or 10 pages if it had been written by an engineer instead.  That’s not to say there isn’t some good information in the report, but it does mean there’s a lot of fluff to wade through to get to it.

One of the important tidbits is that, according to the poll, 41% of security professionals feel that security has become more important to their businesses over the last year as opposed to 15% who think it’s decreased in importance.  Given some of the high profile attacks that we’ve seen in the last year, I don’t find that surprising, but I’m still glad to see that what we do is gaining in awareness of management.   41% of respondents also feel that they’re being given more budget, which leads me to ask if the increase in awareness is leading to a greater budget or if an increased budget lead to a feeling of more awareness?  Given how long we’ve been underspending on security, it is good to see some positive movement on this front.

I found the trends that are driving security concerns a little confusing.  According to Symantec, mobile computing, social media and consumerization of IT top the list of concerns; this was explained to me as coming from the newness of the technologies, but I find that hard to swallow.  Smart phones aren’t new, social media isn’t new and consumerization certainly isn’t new.  I know I had to deal with consumer products in the workplace when I was a sysadmin and that’s been nearly a decade.  The first thing I’d point out is that there’s only a 4% difference between the top 6 items in the list and Symantec acknowledges a 5% margin of error in the survey.  Which means that nearly any one of those categories could actually be the biggest security concern.  I’m a little surprised they split different aspects of ‘cloud computing’ into various subcategories such as SaaS, PaaS, public and private cloud, but I mean that in a good way.  It’s so nice to see someone who actually realizes that the ‘Cloud’ isn’t one technology but a collection of very loosely related technologies and implementations. 

I would like know more about how the question concerning significant security threats was posed to the people polled.  Hackers top the list, but there’s also a category for hacktivism, criminals, industrial espionage, targeted attacks and state-sponsored attacks.   I see those all as potentially falling under ‘hacking’ which could mean that there was a flaw in the question asked that biased the results.  I’m also not sure how this perception actually gains us any understanding in the first place.

“71% of respondents saw an attack in the last year…”  Oh boy, that’s a loaded statement.  If only 71% of the companies saw an attack, what were the other 29% doing, because I’m absolutely certain they were attacked, even if it was simply a drive-by attempt.  Were they playing ostrich, with their heads buried in the sand and no detective measures on their network?  Did they have anti-virus and ignore the malicious code that found it’s way into their network or did they not have AV at all?  Were they actually looking at the logs from their IDS or were they ignoring those as well.  I’ve run into more than a few security professionals who’ve said their management didn’t want detective measures  in the environment because detection would mean they’d have to do something about it.  But even I have a hard time believing it was 29% of the companies. 

The one perception I find in this report that I find scary is the measure of what security professionals think they’re doing well.  52% of security professionals polled believe they’re addressing routine security measure effectively.  But that also means 48% of security professionals don’t think they are.  Close to half of us are willing to admit we aren’t doing a good job at the basics.  And that was the highest measurement amongst all the data points.  If half of us admit we aren’t even doing the basics well, is it any wonder that we’ve seen so many breaches in the last couple of years?  Do we even have a chance if half of us admit we don’t have the resources to do the basics?

The recommendations by Symantec are generic and could have come from nearly any security report written in the last few years.  Policy, process, buzzwords don’t help much.  What should have been highlighted is the need to get the basics right, rather than work on policies that most people in your company will never even know exists.  Yes, policy gives us a lever to pry money out of management from time to time, but it doesn’t address the real problems of just being aware of what’s happening on your network.  But that’s probably not what management wants to hear anyway.

Take a little time to read the report, it won’t take you more than 15 minutes to read every word in it.  As with any report there are some nuggets of knowledge to be gained, but question the analysis put forth by Symantec.  I wish they’d included more information about the specific questions asked, because that tells a lot about the biases involved.  I would also like to see hard data points about the points made, rather than just opinions.  But I guess a couple of years of hanging around people like Alex Hutton and Wade Baker, writers of the DBIR, make me value analyzing data over opinion.

[Slashdot] [Digg] [Reddit] [del.icio.us] [Facebook] [Technorati] [Google] [StumbleUpon]

12 responses so far

12 Responses to “Is this really the ‘State of Security’?”

  1. Marc Ruefon 03 Sep 2011 at 8:32 am

    Hello,

    This is a *very* nice summary of scepticism, wich is mandatory when reviewing such a survey. I was asking these things myself for years. And I still wonder why the survey makers don’t try to eliminate these holes of data evaluation. Perhaps it takes too much efforts for survey makers, clients and analysts. But I’d certainly prefer a complex survey than those lose booklets of weak information.

    Regards,

    Marc

  2. poorgradeon 03 Sep 2011 at 9:56 am

    Another poor quality blog post on a topic not remotely qualified to speak authoritatively, except in our own mind.

    No one cares about skepticism, loud, overly opinionated posts about love of claiming how proud about being skeptical, overly opinionated or a loud mouth. its not flattering…save the comment..not in the business to be flattered…spare us the self indulgence.

    write something that is interesting, not just complaints of crap everyone already knows, or at least every at the non technical evangelists reside level in the industry which is not saying much

  3. Martinon 03 Sep 2011 at 10:30 am

    poorgrade,

    If you have such opinions, why hide them behind a Tor exit node? Why not actually identify yourself? And if you think I’m such a poor writer, why do you keep reading?

    You’re entitled to your opinion. I’m entitled to delete the comment as well. And perhaps you could apply some of your commentary to yourself.

  4. Peteron 03 Sep 2011 at 12:36 pm

    I find the data immediately suspect because it comes from Symantec, which sells security software, so they have a vested interest in making the “State of Security” seem as bad as possible. Does anyone really think they would put out a report that said “Attacks down 12%, security is 8th on list of admin concerns, bot nets shut down at record pace.” The questions were probably created to generate the results they wanted.

    You should be VERY skeptical of any survey that comes from someone with a vested interest in a particular outcome.

  5. privacywonkon 03 Sep 2011 at 12:39 pm

    tor…to protect users’ personal freedom, privacy…etc…

    something self proclaimed privacy wonks…..well its understood, blabbed loud enough for long enough…

  6. Martinon 03 Sep 2011 at 2:16 pm

    Heh, poorgrade/peter/whatever, you’re funny. If you think I’m a self-proclaimed anything, you haven’t been paying attention. The only thing I call myself an expert at is PCI, someone else gave me the nickname Cpt. Privacy.

    If you have counterpoints or arguments against what I write, that’s fine, I’ll discuss them. But if your complaint is you don’t like me or my writing style, go someplace else. I don’t expect you, or anyone else, to agree with me or my opinions. But I’m not going to spend any more time on ad hominem attacks, on someone who’s basic comment is “I think you’re ugly and stupid”.

  7. Scott Wrighton 04 Sep 2011 at 2:54 pm

    As a fellow security blogger who was initially inspired by Martin over 3 years ago, I must weigh in.

    Comments should always be welcome on any blog. Anonymity is fine, on the part of the commenter, too. But Martin does not deserve personal attacks. He is one of the most consistent and earnest bloggers I know. I don’t always agree with him, and I am not always into what he talks about; but that’s fine. If I disagree, I try to post a comment. If I am not interested, I move on.

    It’s unfortunate that the commenter above could not be more constructive. I, for one, appreciate hearing other security professionals’ views, and even more so, the views of non-security-industry readers. I just hope the unconstructive language in comments can be minimized in future.

    BTW – i liked this post, Martin, and the editorial views you expressed.

  8. Martinon 05 Sep 2011 at 7:34 am

    Scott, thanks for the support and for letting me know I’m part of what inspired you to write. It’s always great to hear about people who are taking what I do and building upon it.

  9. Geoff Knighton 19 Sep 2011 at 12:24 pm

    Skepticism? If you hate it, Then why is it one of the reason for so many problems in this country? Skepticism or lack there of needs to be more vibrant and out there. Not showcasing poodles. If you dont like the blog, go read someones analysis of a Dog and Pony show

  10. Software Development Serviceson 26 Sep 2011 at 2:17 am

    This article is such an eye-opener. I’m not too satisfied with the current situation and wonder whats in for the future.

  11. Virtualization Securityon 27 Sep 2011 at 11:47 am

    I just don’t think you’ll get everything you want out of a Symantec system, I’d go for more specialized software platforms that offer virtual firewalls and network segmentation. Especially if you have any compliance needs like HIPAA or something, Symantec isn’t gonna cover it..

  12. Davidon 10 Oct 2011 at 7:27 pm

    I agree with you, however it being great that Symantec put out this “statistical analysis” I do feel that the business department had too much to do with how the analysis was laid out, and how the information was shown. The other problem with these is that they can say anything that they want, but never put a definition to the words or data that they are using. Using loaded statements like you said and generally a statistical analysis such as this would have a section in the beginning stating how they completed the survey and an analysis of bias or something else which this does not contain. I also agree with Peter in the fact that Symantec has a vested interest in making the state of security seem bad to get people to buy their products. It would be like Microsoft doing a survey about the market share of operating systems. Too many biases opinions could come from this, and there is just too much fishy marketing going on here.

Trackback URI | Comments RSS

Leave a Reply

%d bloggers like this: