Sunday, December 15, 2013

Tragedy of the commons in software

Unowned resources that are shared in software seem to inevitably end up disorganized and there is unfortunately no easy solution.

A few instances of this I have seen include:
  • Shared libraries
  • Shared revision control repositories
  • Shared database
  • Shared folders
  • Shared log files
  • Shared settings
When different projects are using the same shared resource they often have different needs, goals, rules and cadences.  The resources themselves usually don't provide a way to split up the resource cleanly and one projects ends up spilling over into another one.  Two simple examples would be in a source code repository while one group might name their branches release/minor_release another group might follow build/* and in a shared library one project might declare static objects that eat up ram harming another group that is trying to reduce memory usage.
  The inevitable cleaning up of the shared resource grows to become a monumental and bureaucratic task.  Even what seems like a simple task of who maintains/own something can be a large task and at least once is found to be a guy that is no longer with the project (and by the way it is no longer used).
  Because the resource is shared by many different users it already takes up a decent amount of "stuff" (ram, hd space, bandwidth).  There is an admin team that is in charge of making sure more "stuff" is added when needed and the users inevitably take advantage of this to an extreme example where a user decided to check in a Visual Studio install into the revision control system (deep within the source too).  From their perspective they don't really feel the pain that everyone else might suddenly be burdened with an additional 10GB+.
  Some projects run very lean and clean.  They have very strict rules about how things should work and be stored, but there are many more that don't and as time marches on and new projects are added they end up having cruft all over the place and dependancies across what should have been the project divides.  This abuse of the shared resource ends up hurting all of the projects.  Rules are put in place and even if there is a good reason they are difficult to change.
  What seems inevitable is that slowly some projects start breaking off and using a different resource that is hopefully not shared and in the process the old shared resource gets less attention and is unlikely to ever recover as more and more becomes unmaintained.  Sadly it is often not a big thing, but many small problems that users put up with until one day they realize that abandoning everything will give them a significant boost.
  The only solutions to this problem I am aware of is to first recognize the problem early and second to have a steward, someone whose job it is to rapidly respond to problems and anticipate new ones before users start to leave.  The steward's job include occasionally striking down long held rules about the resource because they are found to be harmful and being the one that causes all the projects pain by forcing a migration.  It is only through these actions that the shared resource can maintain its viability in the long run.

Popular Posts