Within each iteration, stakeholders will shape the direction of the increment for the scope of the iteration. In between iterations, more significant (and less disruptive) adjustments to the scope of the project are possible and often welcome. Stakeholders who are not involved in the progress of the project on a day-to-day basis are required to take part between iterations. That way, the development team also gets important feedback from peripheral stakeholders and can meet their expectations as well. According to authors Gause and Weinberg, this is also called the “Yes, but…” syndrome. Whatever follows the “but…” are changes in the direction of refining the project. The shorter the cycles between the input and the feedback from stakeholders the better. That is one of the motivations to keep the iterations very short.
Besides being able to better manage expectations, the customers and the project team also benefit from iterative-incremental development in many different ways. Nothing is more rewarding than hearing, iteration by iteration, that the team did a good job. Even if they did not, negative feedback allows the team to rectify things in the next iteration or even within the current one. In contrast, in the traditional approach, the verdict comes in so late that the project team cannot take corrective actions. Iterative-incremental development also overcomes requirements paralysis and the issue of ending up with too few requirements. It might sound surprising, but agile, in some respects, introduces more discipline into the process than other project methodologies – such as the discipline to deliver working software in twoweek increments. What customer does not like that?
Other advantages of iterative-incremental development include the following:
The highest risks can be reduced or eliminated in early iterations.
Confidence in planning and estimation increases iteration by iteration.
Based on past iterations, trends for a completion date can be determined.
Completed means completed, not 90 percent done.
Morale is increased through constant feedback.
Iterative-incremental development is a tremendous change to the traditional development processes currently in place in most organizations. However, this degree of change is worth undertaking, as it immediately provides invaluable benefits.
In agile development, progress is measured in working software. For example, a project that focuses on a throwaway prototype during the first iteration and defining requirements in the next iteration has not demonstrated progress, based on the definition of agile. Similar to a pizza, a project must be cut in slices. The difference is, however, that we won’t know in the beginning how many slices we will need and what kind of pizza it will be. We will figure it out as the project continues, slice by slice.
To state it another way, a project will have a few very high-level requirements, and the project team will take one of these requirements (for example, the highest-priority item first) and drill down into more detail. Then the team approaches the more detailed requirements, writes unit tests, designs and programs the code, and executes test cases periodically. At the end of the iteration, one high-level requirement, a slice of the project, is completed. Now, this is where iterativeincremental development really shines. One requirement is converted into working software and can be demonstrated to the customer just two weeks after the project was kicked off. Think about where your waterfall project would be two weeks into the project. Even better, by taking this approach, we opened an early feedback loop to the customer, who can now make changes and help navigate the project in the right direction according to their expectations.
“I am breaking my two-year projects into two major phases: requirements and coding. Does that make my project agile?” The answer to this question is clearly no. Beyond that simple answer, we need to deliver an explanation of what will give a project an agile spin. The various agile processes have some successful patterns in common, which are isolated as key practices. These key practices are so essential to any agile project that they will affect the energy, spirit, and eventually the success of the agile project. These practices are, in no particular order:
This practice is, from a management perspective, the most noticeable change toward an agile approach. Instead of executing every software development cycle (requirements, design, programming, and so on) only once in the entire project (which is the traditional or waterfall way of doing it), iterative development enforces repetition and semi-parallelism of the development activities. That means the activities are extremely narrow and close to one another. They seem to blend, and the order of activities can change.
As a rule of thumb, the shorter the iterations, the better. That’s why agile processes require a time frame for each iteration that is typically between 2 and 6 weeks.
The second aspect of this practice is the actual increment. Whereas the iteration provides the rhythm for the project, the increment demonstrates the actual progress of the project.
Just as the Agile Alliance represents the agile manifesto, so does the Agile Project Leadership Network (APLN) represent the project management declaration of interdependence (PMDOI). The core values of the PMDOI are as follows:
The APLN is a nonprofit organization consisting of a community of project leaders organized into local chapters.
The goals of the Agile Alliance (which you can read more about at www.agilealliance.org) are to promote agile development in the software industry. With approximately 4,000 members in December 2007, the Agile Alliance is the largest nonprofit community of agile professionals in the industry. The alliance promotes agile methodologies that comply with the agile manifesto, offers funds to members who promote agile development, offers a library of agile publications, and announces local events in the industry. The flagship of the Agile Alliance is the yearly Agile conference. This five-day conference offers several programs, depending on your interest, and a variety of speakers. The interest in this conference seems boundless. Attendance increased from 675 participants in 2005 to 1,100 in 2006 and to more than 1,600 in 2007. Every year, the conference is sold out, and it seems the topics create a degree of interest limited only by the size of the venue. I agree, large does not necessarily mean better quality, but I believe the attendance records demonstrate that agile development has become a mainstream approach.
The agile manifesto (which you can read at http://www.agilemanifesto.org) is the result of a meeting at the Snowbird ski resort in Utah in 2001. Prior to that date, the individual agile processes were referred to as lightweight. I think it was a good idea renaming it to agile because lightweight could have given the impression that it is easy to do and that heavy things were left out. Lightweight could also lead someone to believe that it was incomplete. Once you implement agile development practices, you will see that doing so can actually be difficult and that agile practices are not incomplete. The word agile, however, presents a challenge for the community. Nobody likes to admit that they are not agile, so some developers who call themselves agile are, in reality, not agile in our definition of the practices.
Despite any small misgivings about the name, all 17 participants at the ski resort defined the process and signed the manifesto, which was to become the measure of agility in the years to come. I remember the release of the manifesto, which immediately gave the industry a tangible definition of agile and ground rules for adding new ideas in the future. Still today, the manifesto provides clear direction and is used to discuss and compare agile methodologies. More important in my opinion, the manifesto provides one common roof for all agilists, whatever their favorite agile methodology might be. Here are the core values of the manifesto:
Individuals and interaction take precedence over processes and tools.
Working software takes precedence over comprehensive documentation.
Customer collaboration takes precedence over contract negotiation.
Responding to change takes precedence over following a plan
By the end of 2007, more than 4,700 professionals across the information technology (IT) industry had agreed to and signed the manifesto. Among the 17 authors of the manifesto were representatives from the Scrum, Extreme Programming, DSDM, and crystal clear methodologies.
The father of the crystal clear process is Alistair Cockburn. Crystal clear, like the other agile processes, is rooted in iterative-incremental development. Alistair’s work is also heavily focused on humans, invention, and the idea of developing software as a corporate game. Interesting in this process is the emphasis of product and methodology fine-tuning at the beginning and in the middle of the iteration. This approach enables every project to evolve not only the system deliverables but also the chosen process itself. The ceremony of project documentation is also delegated to the project level.
Crystal clear also requires the team members to be in close proximity, the identification of real users, and that the team use basic code-versioning tooling. A major differentiator from other processes is the consideration of project communication. The larger the team, the more communication has to be factored in. Whereas a team of 5 or 10 members might easily fit into a team room with very effective communication channels, this setup might not be applicable for a team of 50 or more members. Based on the size of the team, different flavors of the crystal clear family can be adopted.
Listing the Unified Process (UP) here as an agile development process is not entirely correct. Compared with the other processes, the Unified Process is a descriptive process rather than an empirical one. That means the UP describes in text, like a hyperlinked version of a book, what particular roles are required to do, when they do it, and how they do it. As with any book, changes to it are more challenging to redistribute. Although the Unified Process is based on architecture-centric, iterative-incremental development principles, the project phases, disciplines, and relationships between roles and responsibilities provide less flexibility. On the other hand, it is exactly the somewhat fixed description of the process that appeals to large organizations that need to comply with a variety of standards and need to outline, for example, a companywide policy for a software development process. Nonetheless, the UP provides a tremendous step forward from the traditional waterfall process (large, separated, and sequenced software engineering phases) and is often an intermediate step between traditional processes and agile processes. There are two noteworthy flavors of the Unified Process:
1. IBM Rational Unified Process (RUP)
The RUP is a result of merging three methodologies (Booch, Objectory, and OMT) during the early and mid-1990s into one unified approach. After Rational was acquired by IBM in 2003, an eclipse-based1 process authoring tool called the IBM Rational Method Composer (RMC) was developed. The eclipse framework provides a consistent platform for all toolmakers to develop and deploy software products. By using such a framework, developers can organize different tools under one umbrella and view them through perspectives. Therefore, RUP can easily be modified using the RMC.
OpenUP is an open-source process released in October 2006. A year earlier, IBM Rational donated a significant amount of its “RUP for Small Projects” to the eclipse community (which you can learn more about at http://www.eclipse.org/epf) with the goal of developing a simple iterative-incremental process framework for small projects. Like the commercial version of RUP, OpenUP comes with an eclipse-based free process authoring tool called the Eclipse Process Framework (EPF).
Dynamic Systems Development Method
Developed in the mid-1990s, the Dynamic Systems Development Method (DSDM) has its roots in Rapid Application Development (RAD), an iterative-incremental process model that uses prototypes at each stage of development. Compared with agile development, which strives for working software at the end of each iteration, the prototypes might be incomplete and not functioning. The prototypes do provide, however, a great way of including all stakeholders early in the requirements work, because the prototypes might be enough to get feedback—for example, from end users. Prototypes can be created for all aspects of the system, including its architecture. In reality, they are often used with graphical user interfaces. DSDM consists of the following nine principles:
1. Active user involvement
2. Addressing business needs
3. Baselining of high-level scope
4. Communication and collaboration among all stakeholders
5. Frequent delivery
6. Team decision making
7. Integrated testing
8. Iterative-incremental development
9. Reversible changes throughout development
Lean development, originated by Bob Charette, applies the principles of lean manufacturing to software development. The result is a kit of 22 tools. The names of these tools still reflect their manufacturing origin – for example, “eliminate waste.” Mary and Tom Poppendieck are leading advocates of lean development in the agile software development industry, having spoken and written extensively about it.
Extreme Programming (XP) was developed by Kent Beck, Ward Cunningham, and Ron Jeffries during the 1990s as a set of dynamic programming practices. Today, XP is the most often adopted agile methodology in the high-technology industry. The most noticeable practices of XP are pair programming and test-driven development. Although XP provides planning practices for project management, it is often seen as an agile engineering process.
Ken Schwaber and Jeff Sutherland, who developed Scrum in the 1990s, define it as a framework for agile project management rather than an agile process. Scrum has its origins in lean manufacturing, iterative-incremental development, and the Smalltalk engineering tools. Scrum provides a simple set of rules. Aside from these simple rules, Scrum is extremely flexible and adaptive to emerging situations. Scrum is a term used in rugby and not an acronym. In a nutshell, Scrum is quite different from existing project management practices. First, the role of a traditional project manager is shared among three different roles: the product owner, the scrum master, and the team. Second, two different backlogs are used to manage scope: the product backlog, which captures the scope of the product, and the sprint backlog, which contains the detail work for the current iteration. A sprint, which is the Scrum synonym for an iteration, is four weeks long. The entire Scrum team meets daily for 15 minutes so that each member can give other team members a quick update. Scrum is very popular in the agile industry.