Today’s Dilbert inspired me to put some thoughts down concerning Metrics. You have to ask yourself - how does Adams still keep content fresh? It’s genius.
On To Metrics
There’s quite a move towards “metrics” these days. Of course, back in my day, we didn’t have such newfangled things like “measurement” and “statistics” it was all me and Jed out back by the server room with nmap and grep, beef jerky and Mountain Dew. No sir, these days people want dashboards and countin’ stats. They say things like “what gets measured gets done.”
Now I’ve already covered the 3 possible reasons to capture metrics (twice). The question is, “how do we get there?”
And the answer might just be “we don’t get there.” Three reasons:
- Difficulty in coming up with the right risk based metrics. Look, I’ve spent some time with this “risk” thing. ‘taint easy. I think it’s possible, but it’s not as easy as building some clever dashboard that gathers some XML data from some network devices. Heck, we can’t even come up with a logical framework for what risk consists of (oh, wait, maybe we have).
- Folks are interested in precision, not accuracy. I’ve already covered the engineer vs. scientist thing and all the problems with not using stochastic risk methods. All I can say is that if we don’t drop the engineer and accountant world views and start working on problems - good metrics aren’t happening. We’re going to be stuck with quality assurance metrics wrapped around a fillet of ISMS. That’s not science, unless Deming is your Feynman, and W. Edwards just looks to serious to play a song on the bongos about Orange Juice (mp3).
- Our products fail us. Part their fault, part ours. We’re not telling people what we need to see (because we don’t know) and so they’re just throwing information that they can gather at us. Tripwire might be a good exception.
I’m all for the move to metrics, but unless we know why we’re counting what we’re counting, we can’t even have a conversation about what to do with it once we have counted it!
Tags: metrics, compliance, information risk, information, risk, risk management, risk_management, information, security, information security, information_security, governance, compliance, enterprise_architecture
(via RiskAnalys.is)May 16, 2007, 7:22am Comments
