Agile and the long arm of history

For many Agilists, the emergence of Agile development models such as SCRUM is a recent event and that IT history was all about the dreaded waterfall development model.

It is true that, for most of the 1980's, standards such as Mil Spec 2167A which was a US Defense standard that demanded the waterfall approach for all Defense contracts (this Mil Spec was also followed by governments in the UK and Australia) tended to dominate the world of software engineering. Going back further into history, system development models such as SDM/70 and Method1 that emerged in the 1970's also were focussed on the waterfall model which had been borrowed from construction.

However, by the early 1980's a number of alternative development models started to emerge that confronted the inherent inability of the waterfall model to respond to change as well as the inordinate length of the development process. In 1982, the Australian Department of Defence abandoned a project called SSR which had been in the analysis phase for over 10 years as they blindly followed Mil Specs! As a side note, once SSR was cancelled a consulting company called Mincom was engaged and had a prototype of SSR within 6 months based on a package they had developed for another industry. Early Agile.

Certainly by 1985/6 the impact of deregulation and global capitalism driven by Ronald Reagan and Margaret Thatcher had begun to be felt world-wide and the rate of change experienced by projects and organizations had begun to increase. This fuelled the initial debate about waterfall. Agilists, please note, this was 1985!

By the mid-1980's, techniques such as Time-boxing where the project was broken into releases of 6 months or so, Joint Application Development (JAD), Rapid Application Development (RAD), Joint Requirements Planning (JRP) and Production Prototyping begun to emerge and gain some acceptance. All these models tried to increase the level of involvement of stakeholders in the system development process while, at the same time, speeding up the development process.

However, a number of factors limited their impact. First, while 4th GL languages were available, the most common development technology was still main frame centric and not suited for rapid iteration, Secondly, the power base of IT was still dominant and many traditionalists did not believe that business should be involved in projects, Third, the rate of change was relatively low and the business imperative for fast delivery was not as intense as it is today. Finally, the cultural impact of these "new" techniques was completely ignored .. see next blog.

The key point here is that the emergence of Agile models is NOT a revolution but rather the result of an evolution that started some 30 years ago.

We believe that Agile was inevitable and like any evolutionary process cannot be stopped and will continue to evolve.

We are close but not there yet.

So what is Agile culture?

The traditional project management and development culture coupled with the use and abuse of "expert" power lead to an approach to both project development, project management and sponsor relationships based on an inward looking and technology-focussed set of behaviors that reflected a value system.

Values plus behaviors = culture.

As we discussed in earlier blogs, this traditional value system had already been established for hundreds of years in construction, engineering and similar professions.

The key values of this culture were:

• Closed - stakeholders (including sponsors) were treated as "users" who either got in the way of technical people delivering or worse, as luddites who resisted the change;
• Dishonesty - critical documents such as Business Cases were treated as a bureaucratic hurdle and risks, estimates (costs and benefits) were manipulated or copied with minimum effort to get the project approved so the technical people could get in with it;
• Distrust - a set of behaviors such as checking up or "are we there yet?", haggling about estimates and designing complex reporting systems all sent a message that people were to be treated as children not professionals;
• Lack of courage - it became common for project managers and technical people to avoid confrontation and negotiation by accepting imposed deadlines and budget constraints which could be negotiated at the latest possible stage. This form of ambushing was/is so common that every senior manager we have worked with actually expects it to happen. Worse, by agreeing to imposed deadlines and other constraints, the project manager often forced many internal service providers such as H.R. and Infrastructure to have to work within these constaints;
• Technical focus - the inward focus of projects meant that issues such as the true costs of projects and support and the realization of benefits were either ignored or treated as after thoughts.

We are not saying that these values and the related behaviors were deliberately designed but rather they evolved and like the frog placed in cold water that is brought to the boil, we didn't notice the rising heat of the culture.

We weren't looking and we weren't paying attention. In fact, any discussion about culture was seen as "soft" or "flakey".

Agile values and the related behaviors are the exact opposite.

• Open - stakeholders (including sponsors) are completely involved in the planning and delivery of their projects. RAP (RApid Planning) and Agile Development are open and transparent enabling stakeholders to get closer and have control of their projects;
• Honesty - critical documents such as Business Cases are treated as critical information that link the business problem to the technical solution. Risks, estimates (costs and benefits) are honestly created and "not knowing the answer" is more acceptable to "making the answer up";
• Trust - team members, stakeholders and sponsors are treated as intelligent professionals who will make good decisions and fully support their commitments to colleagues and to the project's big picture;
• Courage - Being able to say "No" appropriately and with justification and being able to admit mistakes as well as not knowing everything are supported as signs of strengths not weakness. Not punishing project managers when projects go "Red" but rather asking what help is required to make it "Green" are some of the examples of this value;
• Money focus - projects are about true value add and that all costs must be measured and managed. The realization of financial benefits is the key measure of success not time. Use of "intangible" benefits is not allowed and a true partnership between the project manager and team who deliver the capability and the sponsor and stakeholders who must use the capability for benefits is created at the beginning of the project.

In our consulting experience, these Agile values really challenge some of the more commonly accepted behaviours and "artifacts" of prevailing project management. Just as Agile development no longer uses GANTT charts, Agile project management requires new tools and artifacts. We'll talk about some of these later in our Agile Conversations blog.

Meanwhile check out http://onegreenerday.blogspot.com/2009/06/integrating-agile_19.html . This is a really awesome example of how social networking technology can open things up .. in this case .. the Agile Conference in Amsterdam in June 2009.

Agile Culture - the real Agile issue

When traditional project management (GANTT charts, scope, etc) begun to be adopted in the early-70's by business and IT projects from construction and engineering disciplines not only did we borrow their tools but covertly we adopted their culture. The "expert knows what is best for you" values that were well established in architecture, design and construction became the cultural norm for IT. For example, calling a banking manager with 20 years of experience and an MBA a "user" is a perfect example of how IT had adopted the "expert knows best" attitude. Some 40 years after I began my IT and project management journey, I still hear a statement we often said in 1969. "The problem with my users is that they don't know what they want and they change their minds all the time!"

In effect, traditional project management and development approaches typically exhibited a closed, distrusting and often dishonest set of values towards those ##xx users and senior management. Stakeholders were asked to sign-off documents such as Business Cases and requirements without any open participation in their creation. In many cases, the users were only involved in testing and documentation activities (you know, the actvities IT people don't like).

As we work to assist people and organizations become more Agile, the most important conversation we must have with them is a cultural conversation.

All Agile models are based on openness, honesty, trust and courage.

These values fundamentally change the behavior of project managers and developers as well as fundamentally change how business experts and senior business people sponsoring projects interact, relate and behave to each other.

Agile is not about techniques such as XP, Scrum and so on.

It is about behaving differently.

It's the culture .. always the culture

Researchers have shown that culture is the outcome of a complex series of relationships between values or beliefs and behaviors that are the manifestation of those beliefs. These values and behaviors are also evident in the various rituals and symbols we follow and believe in.

For example, if you believe (value) that a fair and decent society should look after those in the society who are struggling to earn a living or are living in poverty, then you would support paying taxes for a social welfare net that provides training, financial support and other for those people. If you saw someone eating out of a rubbish bin, you would probably give that person some money for a proper meal. Supporting social welfare and giving some money are behaviors that reflect your value system.

However, if you believed that everyone is born equal and has an equal chance to be a President, for example, and that people are solely responsible for their own situation and the person eating out of the rubbish bin had simply made bad choices and was responsible for their own situation, you would not give money and probably not support higher taxes for social welfare. Different values, different behaviors.

What we also know is that for most of us culture is something we are not taught but something we slowly learn and absorb as children. Culture is everywhere. In our parents' and friends' behaviors, television, papers, the political system, the schools we go to. Like fish don't understand the sea they swim in, for many of us, culture is simply the sea we swim in. We rarely need to discuss or understand it and when culture becomes a public issue most people can't define it precisely.

In our next blog, we'll look at organization and project management cultures. This is the key to understanding Agile and how difficult it can be to successfully implement Agile.

Why Agile is so important - Part 2

A quick search on Google will show how Agile (in all its flavors) has become very "in, hot, cool, faddish". I had a conversation recently with a long-term bureaucrat who believed that "Agile Bureaucracy" was a possible option for bloated and non-responsive government service! Even EDS with its 100,000 plus people has an Agile model. It won't be long before all major multi-national consultancies will have instant Agile experts.

So, what do we see as Agile.

In our client base, Agile models include three related and evolving elements. The first is Agile Sponsorship/Governance. This covers the linking of change and projects to the organization's Strategic Plan or Outcomes as well as prioritizing projects, executive management of major projects and the management of benefits realization and on-going support. The second element is Agile Project Management (often termed Extreme or Radical) which involves an open and inclusive approach to stakeholder control of their projects and the use of RApid Planning (RAP) sessions developed by our company over the past 20 years. Finally, there is Agile Development (SCRUM, DSDM, etc) which has evolved from RAD, timeboxing and JAD approaches in the mid-1980's. Agile Development involves highly-focussed development cycles of a month or less.

However, all these three elements are bound by the principles of Simplicity and Transparency.

In our next Blog, we'll begin to explore the underlying cultural challenge facing those people and organizations who wish to really implement Agile.

Why Agile is so important - Part 1

As we face the worst economic and social meltdown in our life, we have been working with some innovative organizations who are seeing the current situation as an opportunity to re-evaluate and re-assess their business and project models. As identified in the February 2009 issue of Harvard Business Review by management gurus such as Hamel, Mintzberg and Kelly, the prevailing management models, which date back to the late 19th Century, have run their course. Most companies are adopting the good old hunker down, wind down the bunker cover and hope this all goes away so we can get back to normal. As Einstein said " Insanity is doing the same thing over and over again and expecting different results".

Over the next few weeks, we'll be outlining how Agile models can and are redefining how business and business change can bring the same benefits to service organizations as the Lean or Toyota models brought benefits to some car manufacturers.

We look forward to this new form of communicating with new and existing friends and colleagues

Rob and Camille