June 04, 2005

On Consultants and Agile Methods

I caught one of Cedric's entries on XP. Cedric doesn't think XP is used that much, I think it is, though mostly through individual practice adoption. At least he's always maintained a professional, reasonably open attitude when discussing it. On the other hand, a few of Mike Spille's comments struck a nerve with me as being quite reactionary, thus I've written this entry.

In my experience, Agile methods are used widely, and on significant, mission critical projects. I joined BEA consulting recently; every project we lead (in my region, anyway) uses agile approaches to project tracking and prioritization and use many of the XP practices daily. These are projects that run large core systems for multi-billion dollar companies.

There is a disturbing trend among cynics that deride the work of Beck, Fowler, etc. and pan them as the worst lot of opportunistic consultants that have only worked on small scale projects. I think this is an ignorant and generally wrong position, based on a grain of truth. Beck and Fowler have created lasting, large-scale systems running in large enterprises. I personally am aware of some of their past systems, and I know people that maintain them to this day. The grain of truth is that for every Beck or Fowler, there are 50 consultants hawking Agile methods without really getting what they are, how to fit them to a context, and generally causing destruction and chaos in their wake.

These guys have have promoted and articulated some of the most important, practical, and highest impact ideas in programming over the past 15 years:

  • de-emphasizing the role of inheritance in OO and emphasizing the role of protocol / interfaces

  • applying Alexander's work on design patterns to software

  • CRC cards (and responsibility-driven design, which was expanded and promoted by Wirfs-Brock)

  • test-first development, etc.

  • Tom DeMarco's Peopleware has been a classic for 2 decades

  • Waltzing With Bears is IMHO one of the best books on managing risk available, along with Jones' Assessment & Control of Software Risks

DeMarco may not reflect Cedric's experience with software because of his product development focus. It certainly reflects my experience. DeMarco is from the enterprise IT crowd - particualrly defence, finance, and telecom. His discussion of how bad IT managers behave is on the money.

Now, I don't think XP is a complete software development method. It's a collection of very effective practices and a process "in the small". I actually feel the best modern book on software project management is Walker Royce's book "Software Project Management: A Unified Framework", which promotes a reasonably agile approach to the UP.

I've consulted, trained, and mentored people that build financial trading systems, customer information or CRM systems, billing systems, risk management systems, e-commerce sites, multi-terabyte data warehouses. And all of these, when I carried a leadership role, used agile practices -- including frequent releases, continuous integration, pervasive testing, variable priorities & scope, etc. They work well, if fitted to the appropriate context. Were they textbook XP? Of course not. Yet XP has had tremendous positive influence on my practices.

My view is that experience alone in the day-to-day pressures of a dev job does not give one the ability to reflect and think about the bigger picture. Some can, and make wonderful developers or team leads. But compared to many full-time developers, consultants -- by that I don't mean contract employees, I mean people hired to impart knowledge -- tend to have more available downtime to reflect. Thus, they can provide an important contribution. Of course, there are 50 useless consultants for every great consultant. But that's the same with programmers in general, and arguably even managers. It has nothing to do with how much time they spend on a system, there's still that 10:1 productivity ratio. The process, XP , or not, will not save you from a lack of knowledge or skills, or bad management. But it can save you from building the wrong system, or prioritizing the wrong things, or debilitating quality problems at deployment.

Posted by stu at June 4, 2005 12:20 AM