Software Security Model – BSI-MM released

March 12, 2009

Haven’t been posting for a while because work is really busy right now.  More on that, and what I’m up to perhaps in another post but I’ve been asked to have a look at and post about The Building Security In Maturity Model.

I’ve always liked the idea of maturity models – it gives organizations an idea of what other places are doing, and a “where they stand” in the industry (one of the things I’m always asked at the end of project is how did the client compared to “others”, so even post-school/university, people still want to know where on “the curve” they sit!).  The thing that I dislike about maturity models are that they encapsulate a set of activities with associated levels and the assumption is that as you do more of the activities the higher up the levels you get and thus the better you are at whatever the model encapsulates.

That’s not always the truth though.  SEI-CMM was always presented in a way that if you don’t do all the activities for a certain level then you were “stuck” there.  i.e. didn’t do all the activities for level one, but most (if not all) of level two – you’re de-facto level one.  One of the papers my research professor back in the UK (the eminent computer scientist Les Hatton ) was a favorite of mine titled Linux and the CMM [PDF link].  In summary, despite Linux being a rather good bit of software engineering (both process and quality) it is firmly rooted in level 1 – apparently not all of the “activities” are necessary to build good software.  The SEI-CMM has been retired now and we’ve moved on from this idea, but in many cases this view of activities –> levels –> “good/bad” remains.

In BSI-MM there are “levels”, but from what I understand of it you don’t necessarily progress up them (although there’s clearly going to be this thought in some organizations), but rather a set of practices/good ideas/activities that are embodied in the software security initiatives for the organizations(9 of them – all really doing something to improve their software security) that were analyzed to make the model.  To really bring this home you have to see the graph at the end of the FAQ – The “average” maturity for each of the sections in the model are all over the place which means that some companies focus more/less on some activities over others.  Actually digging into the full document itself and reading the introduction/prolog really brings it home of how to best use the model, which I hope more people will do than simply browsing the framework online (how about linking to this text before the model Gary?  I think it might help “set the idea” of how to use the SSF better for casual viewers that don’t want to download and read the entire PDF document – maybe I didn’t find the link on the site though Edit: Ok, I found some at the bottom of the FAQ but I still think it’s a little easy to skip over).

The part for me that I really liked was the section that gave the real stats on who does what.  Like all 9 of the companies did these activities (which made them “core” and probably critical – much like the core competencies in Linux development that made it much more than a level 1 maturity).  Seeing the raw count of how many did each activity was interesting as well – the activities that only one organization was observed doing I would guess that some of it is driven by their operating requirements or compliance needs (perhaps breaking this down by vertical as well would show some interesting correlations?).  One big question as well is answered – “How many security people should I have”.  The answer, at least within the 9 organizations that the model is based on, is 1% of the development staff are dedicated to a “software security group”.  I think there’s there’s an important distinction thought in “development staff” vs “operational staff”, and I believe that this model is only looking at the former so bear that in mind before using this stat too directly which I’m sure is going to be quoted in many a under-resourced IT departments.

In all I think this is a great piece of work and a good insight into the software security practices of companies that are really hard to uncover and even worse to get them to share.  Thanks must go to the authors of this – Gary McGraw, Brian Chess and Sammy Mingues – but also to the organizations that opened up their practices and allowed them to be analyzed, reviewed, categorized and shared.  Information really is power, and this kind of knowledge I fell really does help move our little corner of the industry along for everyone involved that wants to get better and take security engineering seriously.

Posted in Uncategorized