For many years now, I’ve been frustrated by the lack of configuration management maturity in the applications I use. I’m particularly surprised (appalled) that security software nearly always falls short. The problem isn’t only with security software, it’s just that I would expect more from software designed to secure systems. Virtually all security software maintains audit logs, and most can ship those to a log server, but few do more than that. Pick a tool, and chances are you can configure it, but can’t roll back to a previous configuration, compare two configurations, or migrate a configuration changes to another system.
Information Security has three core tenets we refer to as CIA: Confidentiality, Integrity, and Availability. If you are an Infosec professional, you are responsible, directly or indirectly, for ensuring those qualities of the your systems. So what are the core tenets for configuration management? What should we insist that all configurable software support? I propose CAARR.
- Comparability: I should be able to compare the configurations of two different points in time or of two different systems. Ideally, a tool should provide this functionality and offer context. That is, the tool should identify what the differences represent from the user interface perspective, rather than from the software developer’s or the data structure’s perspective. That is a tall order, however. At the very least, the configuration should be exportable as a human-readable flat file, so that configurations may be stored and compared. (Scooter Software’s Beyond Compare is exceptionally good for such comparisons. You can even customize how it performs the comparison – e.g., what it ignores. Microsoft’s free XML Notepad is a great tool for comparing these files if they are XML formatted.)
- Accountability: The software needs to log who did what, and when. As I mentioned above, most security software does this, at least at a rudimentary level. Frequently, however, the logs lack sufficient detail, or only address specific types of changes or events. For example, Symantec Endpoint Protection v11 does a great job logging events on the end users computer, but the management console provides minimal logging of configuration changes. It identifies who made a change and when, but all they tell you about the change itself is what type of change it was, e.g., a policy changed. If the software doesn’t allow you to configure the level of logging detail, it should at least provide you with sufficient information to know what changed, if not explicitly what the change was.
- Annotatability: Many tools lack support for annotations in the configuration or audit log. It is invaluable to be able document why a change was made, who requested or authorized it, when should the change be removed, etc. The annotations should exist right in the configuration itself, adjacent to the changed setting, and the annotations should be visible when are making changes within the tool. That way, when you or someone else is considering changing the configuration, they’ll be aware of previous decisions. For example,
Cisco PIX and ASA firewalls provide a comments field to allow you to document each access control. It is up to you to enter comments every time you make a change – the tool does not enforce it nor remind you. Also, Cisco firewalls lack full annotation support – there are many items that can be created or changed (e.g., NATs) where there is no option for comments. - Rescindability: “Oops!” What do you do when you or a colleague utters that (or worse)? Roll back the changes, of course. Software configurations should be easy to roll back to a previous baseline. Ideally, you should be able to keep multiple configurations stored away, so you can roll back to any previous settings, not just the last one. Whether you identify baselines within the software or simply save configuration files, if you can’t undo changes easily, you’re in for a tough time.
- Repeatability: Do you have multiple installations of this tool? Maybe you have a test installation so you can experiment before putting changes into production. Maybe you have the software installed in locations across the globe. You need to be able to configure a tool and repeat that configuration (or portions of that configuration) elsewhere. You should not have to keep detailed notes about what you’re doing, so you may manually perform those steps on other systems.
So why do most applications fail to meet these requirements? I don’t know, although I think it is partly because programming languages and tools don’t make it easy. You can find frameworks that support logging and frameworks that support configuration files. However, not all frameworks support all five CAARR components. In fact, I’m not sure any do. In a way, this is ironic, because any decent software version control tool (e.g., Subversion, ClearCase, AccuRev, Perforce, Team Foundation, etc.) provides all of these, so developers are (or should be) familiar with the concepts. Tools that store their changes in databases seem to be the least likely to support comparability and repeatability. I presume this is because with a database structure, the developer or framework will need to create the comparability and repeatability functions, and the more tables used to support the configuration, the more complex the coding will be. On the other hand, if the configuration is stored in a flat file, comparability and repeatability are are essentially assured, especially if the file is human-readable and understandable. If the tool is not developed from the onset with CAARR principles in mind, it probably will never be.
The next time you evaluate a software tool, include CAARR in your requirements. Just don’t be surprised if none of the applications you review score well, and you may be forced to compromise. Let the vendors know that the lack of CAARR hurt their candidacy, and hopefully this sad state will begin to improve.
A Stich in Time 