learn unix and linux commands

I've been fascinated by the issue of measuring and improving information security management practices and controls for several years now and despite thinking long and hard about security metrics, I'm still not confident that I even fully understand the problem let alone have anything like a solution. But, that said, it's hard not to poke holes in security metrics proposed by others, usually because most of those who gets into metrics seem to end up creating tedious lists of "Security Things That Can Be Measured" (STTCBM), as if that was what was needed. It isn't.

Take for example a recently-updated NIST standard (Special Publication) on security metrics. The previous version of SP800-55, "Security Metrics Guide for Information Technology Systems" [no longer online], was little more than a catalogue of STTCBM - a long long list, its true, so long in fact that anyone trying to actually use the standard would have been stuffed within the first few pages. The effort and hence costs required to collect, measure and report all those STTCBM would have been a nightmare, in my opinion far far outstripping the value of the metrics. As to how management were expected to interpret the STTBCM and adjust the organization's information security tiller acordingly, I have no idea. It's a classic case of "more data than sense".

The shiny new version of SP800-55, renamed "Performance Measurement Guide for Information Security", takes a rather different tack but is still quite long (80 pages in total, half of which are appendices). I suspect the primary reason for its existence is to suport FISMA (the US Federal Information Security Management Act, essentially a set of information security policies mandated in law for US Government agencies) by imposing a standardized set of metrics that can be used to benchmark agencies and force the laggards to pull their socks up. It remains a highly beurocratic and costly response to a genuine management problem.

Another draft NIST standard, SP800-80 "Guide for Developing Performance Metrics for Information Security", emphasises the process of developing and implementing security metrics. It includes a shorter list of STTCBM ('candidate metrics'), but again takes a database approach with forms in the appendices characterising the metrics by 'metric type', 'frequency of collection' etc., details which, by the way, are organization and implementation-specific and really not that hard for grown-up security managers to figure out for themselves.

I've been fortunate enough to see and comment on the evolving draft standard ISO/IEC 27004. Having made my opinions crystal clear to the authors and committee responsible for '27004, I won't lay into it again in this forum, except to say that infosec practitioners of my acquaintance generally don't need to be told that some metrics (measurements in ISO/IEC-speak) can be "derived" from others by a process commonly known as "arithmetic". Call me a cynic but I honestly don't believe that sums are the primary issue in security metrics.

Today I stumbled across The Metrics Center, a project "to connect people, information, and analytics for the purpose of transforming data into knowledge, action, and ultimately value" (in the context of information security). Sounds great! But a quick look at their Metrics Catalog reveals that it is yet another database of STTCBM. Under the ISO/IEC 27002 section, for example, we find a list of 98 metrics currently, each of which expands to a form with standardised information. Here's a typical extract [sic]:



In relation to information security metrics and management systems, what to measure and why are far more important questions to me than how but while all the standards and initiatives mentioned above list the what and explain the how, none adequately cover the why. Worse still, they don't really explain what to measure: they merely state what could be measured (STTCBM) and in so doing stuff the lists with trivia (simple counts and percentages are all the vogue) while missing out many more creative and valuable measures and information sources (such as employee and industry surveys and numerous excellent web sites detailing current infosec threats, vulnerabilities and incidents etc.).

While the metrics standards are promoting long lists of STTCBM, I'm left struggling to find "a few good measures" for information security, things that are simple for management and others to understand, things that clearly mean something useful and can be used to drive an organisation's information security management system to new heights. As a rehabilitating IT auditor, I'm particularly annoyed by the attitude of some self-acclaimed security metrics experts who insist that everything has to be reduced to numbers. Managers don't manage entirely by the numbers. Informed commentaries in benchmarking reports and security surveys, for example, are every bit as valuable as pie charts and graphs. I'm looking for the information security equivalent of the old "days since a lost time accident" health and safety boards outside the factory gates - something self-evident that immediately resonates with ordinary viewers and has value as security awareness material as well as for numbers-based management. A "Days since the last information security incident" graphic on the corporate intranet, perhaps? If you want to be really posh, click on the graphic to read all about the last incident (what happened, why, how it was discovered and what has been done to prevent recurrence), to explore a breakdown of security incidents by corporate business units/locations/types or whatever. Never mind the "percentage of individuals who are able to assign security privileges for systems and applications who are trained and authorized security administrators". The hard questions are like "How secure are we today compared to yesterday, or compared to our peers?" and "What should we be doing to be more secure tomorrow?". Most of all, "Are we secure enough?".

Like I said, I don't even have all the questions yet, let alone the answers.

Kind regards,
Gary

Gary Hinson
Passionate about security awareness
www.NoticeBored.com Creative awareness materials
www.ISO27001security.com ISO/IEC 27000 standards





More...