Agile software development practices are often criticized as being suitable only for small, co-located teams of experts working on modest sized projects.  If agile development is truly limited to these perceived boundaries, then it is probably not sustainable.  Software development practices must address large projects, multiple geographies, and a general population of developers if they are to become the basis of a thriving new paradigm.

On the other hand, in The Innovator’s Dilemma, [HarperBusiness, 2000], Clayton Christensen notes that all disruptive technologies start by addressing a market which is considered small and “down-market” from existing technologies.  So if agile practices are a “disruptive technology” compared to traditional software development processes, then it would be quite in character for them to start by addressing small systems.  The questions is, can agile processes grow to address the needs of large systems?

The answer lies in recognizing that the needs of large systems are changing in a way that is best addressed by agile practices.  Over the last decade, corporations have tended to use commercially available systems to address more and more information needs.  Initially, these systems tended to be monolithic and proprietary, but this is changing.  Corporations have found that any monolithic system, even one which is commercially supplied, rapidly becomes a legacy system as new technologies combine with mergers to obsolete any system that cannot adapt to change.  

The result has been a tendency for developers of large systems to prefer the use of commercially supplied components, from API’s to Web Services, for a sizeable portion of any application.  In Building Systems from Commercial Components, [Addison-Wesley, 2001] Kurt Wallnau, et al. point out that large systems cannot be built from software components using traditional development processes.  The marketplace in which component vendors compete dictates that components are complex, they change frequently, and customers pretty much have to take whatever is offered. 

System development in the component marketplace is distinctly different from traditional software development.  To begin with, one does not start by capturing user requirements, but by understanding the capabilities of available components.  The challenge is to bring the user requirements into line with available software, rather than the other way around.  System architecture is generally dictated by the available components and the key architectural task is to create an ‘ensemble’ of components that work well together.  Finally, designing a system to be able to deal with unpredictable change is a fundamental skill when working with commercial components.  If nothing else is certain, the fact that these components will constantly change can be guaranteed.