This morning, David Winter from Coleman laid out what he wants as one of the consumers of IT Management. Their needs were the same as you’d expect: we’ve got a ton of IT stuff spitting out data and we need better ways to figure out what’s wrong. We’ve got a CMDB that we fill up with data (with RANCID, which I don’t know about) and archive off-site. But here’s something that goes all whacky, he said, most of these systems can’t do fail-over when the centralized server goes down. All of the little agents out there on the network just stop working. Sure, you can MacGyver that thing somewhere, but it’s not, like, a real feature.
The list went on, but the most interesting point was a point of feeding back knowledge from one part of your whole IT Management suite to other parts. For example, David had kind words for Splunk as a tool for finding events. “Events” are records of (usually) bad things happening on your computers - the email server is down, there were 50 bad logins for an account in 3 minutes (probably someone hacking), or there’s no hard drive space left on the shared drive. Those are simple examples. “Events” can be more complex and esoteric, the equivalent of “the server says it’s tummy hurts and it just puked all over the raised floor.”
Figuring out what to do about that hurt tummy and pouring saw-dust on that puke is a large part of “event management.” Simply finding out about the events is just the start.
So, back to what I was hearing was the interesting point: we’ve got great tools to find these events, but we don’t have much when it comes to feeding how to respond to those events back into the system. So we hear that the server’s tummy hurts. That means to give it some Pepto. But then how can we feed that knowledge back into the other tools in our IT management suite so we don’t have to do it again?
Those with deep pockets have what we call “automation” and “runbooks” now. Otherwise, it’s back to meat-ware: have a person do it.
Previous Topic
