I’ve recently had the opportunity to be part of two similar projects each being run using a different methodology.

Both projects ran for a year, with the second project a continuation of the first. Let’s look at the various aspects and results of the projects before we look for some conclusions.

Methodology

Project A was run using an agile approach – 2 weekly sprints, with a demo to our internal customer and the rest of the company at the end of each sprint, careful time tracking, burn down charts and a project backlog were all used by the development team.

As the technology was new to the team a large part of the initial time was spent investigating and evaluating the technology. Many prototypes were produced to test the possible solutions and remove some of the risk. Once the favoured technology was chosen a few weeks were put aside to produce a small internal application. This proved both a useful demonstration of what could be achieved and as a way of getting lots of silly “first time” mistakes out of the way in this system rather than the production project. It was a great learning experience for the team and with the tight time scales and sense of achievement it was also the start of getting the team to gel.

Project B took a more traditional big bang approach. The year was divided up into 3 stages, development, alpha testing and beta testing. About 6 months was development, with 2 months for alpha and 1 month for beta testing. The remaining 3 months were allocated as contingency.

A lot of the risk was removed from this project as it was a continuation of Project A and so used the same technology stack.

However, the project did still push the boundaries of what had been previously achieved with a web browser so a number of investigations into technology were still required. These were largely done the same as before, but as there were fewer investigations to perform and the technologies themselves less of an unknown the investigations took less time than those in Project A.

Team composition

Project A was led by an experienced and senior member of the team. He was mainly tasked with the day to day management of the team and with removing any technical road blocks that might be encountered along the way. There were 3 other members of the team, one other experienced member of the team (but not rated by some of his peers) and a junior working on his first project in the industry. The final member of the team was a graphical designer, also working on his first professional role.

Project B was again led by an experienced and senior member of the team although not the same leader as in Project A. There were 3 others on the team, the same junior developer and graphical designer from Project A, but this time they had 1 years experience. The other member of the team was a developer with 5 years of experience and highly rated by his peers.

Outcome

Project A was considered by the technical team to be a great success. They had risen to the challenge of new technologies, new agile approach and new team members and had delivered a functional, stable and coherent software base that be the basis for the next 5 years of development in this area.

Changes in direction, the need for sales demo’s and additional functionality were all handled with minimum fuss by simply adjusting the goals of the next sprint.

Unfortunately, the business side were somewhat disappointed with the project. They saw the project as being delivered too slowly, with little ambition and lacking innovation.

Project B was considered by the technical team to be another success, but lacked some of the pioneering spirit of project A. They just didn’t have the same buzz about them, the urgency was gone, it felt more of a slog. At the beginning of the project they had taken a small gamble by deciding to write a large part of the solution themselves instead of using the best open source component available. That gamble paid off – the end result was amazing, above and beyond what might have been hoped for initially. I believe this showed the value in trusting the team and allowing those individuals in the team to take that trust and deliver what they believed could be possible – the motivation from that trust and desire to be proved right is a winning combination (although they obviously need the skills as well).

As might be expected from a year long project the initial requirements changed over time. This was roughly catered for by the inclusion of 3 months contingency in the original plan. However, the resentment at being expected to somehow magic up additional functionality was stronger on this project, especially as the final deadline loomed and the contingency had all been used.

This time the business side were very pleased with the results of the project. They got the innovative update they asked for, with minimal interaction and monitoring of the development team and demo’s and project completion were all completed on time. They even thrown some extra requirements at the team and were pleased that these had all been accommodated.

Summary

So what can we learn from this experience? My personal beliefs tell me that being agile is good and I’m not alone. In fact, the results of these projects just confirm those beliefs. Being agile allowed changes to the project to slot right in, with suitable discussions on the impact of making those changes in direction.

The feedback cycle was short – delivering a demo every 2 weeks was tough, but each time they made it they were rewarded with a sense of achievement and renewed desire to nail the next one.

The architecture of the project grew naturally without the need for lots of extensive design process up front (which to be fair applies equally to project B).

The risks were greatly reduced, by showing the project every 2 weeks, progress was known and the final deliverable could be shaped throughout the year as more knowledge was gained about the market, feedback from users was acquired and thoughts matured about what could be possible.

In comparison, Project B struggled with some of the changes in requirements. They were more focused on delivering what had originally been asked for and resisting change. It was difficult to have suitable feedback on the consequences of making changes to the requirements – the suggested changes were all good, so the main response was simply to suck it up! (They could have chosen to not deliver instead, but that wouldn’t really please anyone).

They lacked some of the buzz from continual pressure and regular small achievements which led to my earlier comments of the project seeming more of a slog.

The risk of delivering the wrong solution was large. With no feedback until the project neared completion, the end results could have been very bad. It was largely down to the skill of the team that they had a good sense of what was required and how to make it happen. That led them to make the right choices themselves and bring the project to a successful conclusion. However, they certainly suffered from the large amount of feedback that turned up near the end of the project and demanded their time exactly when they didn’t have any left.

Conclusions

One interesting observation was that the technical people involved in the projects all considered their projects successful. Does everyone in the team always think that their projects are successful? What would an unsuccessful outcome look like to them? Why were their thoughts on the success of the project different to the business side?

Firstly, this relates to something described in almost any discussion of projects – defining success. Both projects neglected to really define success from a business point of view. They had their own vision, but somehow project A failed to meet the success criteria of the business, whereas Project B didn’t.

Another area often talked about being key to success is communication.  In each project the communication within the team was excellent (although progress was less known in Project B so was harder to communicate).  It was the communication out of the development team that varied drastically between the projects.  It would seem that more communication is not always a good thing especially if the message being communicated is not what the recipient wants to hear.

In each project, the business wanted the project delivered earlier than the development team thought feasible.  Project B got this message out early, took the flack and then consistently stuck to the same completion date.  It got to the point where it became pointless for the business to ask for status as they were always given the same completion date.  The team either didn’t know their own status (which is a difficult thing to know when the project is so large) or would lie to the business about the true status.

Project A always knew their status and couldn’t hide it – every 2 weeks there they were showing the world what they had done.  This caused a biweekly reminder to the business that they weren’t getting the software when they wanted it and, as they didn’t always understand what they were being demonstrated, a lack of confidence in the team
delivering at all.  Accusations of a lack of focus on what was important (to the business instead of technically) and of not being committed to achieve were frequent.  A lack of understanding of the agile process caused friction with the business wanting a guaranteed
completion date and the team wanting a more free flowing let’s see how it goes and we’ll know when we’re done when we get there.  Perhaps I could have described the communication coming from the team as marketing.

Project B did a better job in marketing itself.  It gave the business what it wanted – a cast iron guarantee of delivery date (which wasn’t the date the business wanted and even the new date turned out to be bogus as they missed it anyway).  They then stuck to that date religiously demonstrating huge belief in their abilities and commitment to the cause.

Project A told the truth – they had no guarantees, but would work towards whatever they were asked to in the pursuit of an excellent product.  When everyone agreed the product was awesome (or had enough good stuff to be useful) the product could ship and the team could move to the next version or end the project.  This just didn’t work – the message was wrong, the target market was wrong, the success of the project depended on it and this mistake was costly.

So, I still believe in agile – even more so, but the results also don’t lie – Project B gave better business results than Project A (and when the business is happy then we are all happy).  My conclusions were also that the team in Project A didn’t do much wrong – they followed the agile advice – they involved the customer all along the way – but the customer didn’t follow the script.  The customer seemed annoyed that they had to spend time every 2 weeks seeing a project they wanted yesterday taking longer than they thought it should.  They didn’t appreciate the opportunity to steer the project, to reduce the risk and to develop great technology.  They were supposed to say “that’s great, I can use that now, that is so much better than projects I’ve been used to in the past”, but all they could say was “when will it be finished? I need it NOW”.

Do I really need to fix the customer?  Can we be agile without a customer?  Can we communicate better and get the customer on board?

Only time will tell.  We’ll carry on doing, evaluating, and trying to improve (or rinse and repeat as the TDD folks would say).