New Chapters

The book I have been working on (and published on the web) has mutated into something very different. The Agile Leadership stuff is now only part of the book. In response to feedback from a several readers, the main focus has shifted, and I have published new intro chapters exploring this shift.

From Management to Leadership

We hear a lot about Project Management, but Agile works best with Project Leadership.

To help me explain this, for the past couple of weeks, I have been digging deeply into the difference between management and leadership.

This has helped a few new book chapters take shape, and one of these is now published (in draft form) here.

Before Agile

I have now added a few more chapters to the book, most recently an overview of methodologies before Agile

Agile Practices are Classic Management Practices

I have now published rough versions of the first three chapters of my book.

A lot of folks have said that Agile is just grounded in common sense. The problem is that common sense isn't that common until you have heard it and applied it. A lot of "common sense" goes against the tugs of human nature. Agile gives you a small number of principles and practices that keep you on the straight and narrow - they stop natural tendencies from pulling you in the wrong direction.

It turns out that a lot of the principles and practices of Agile are not new at all - we just lost track of them by sticking to ivory tower methodologies which looked good in theory but, not surprisingly, didn't actually work in practice.

If we forget the fancy theories, we can turn back to the management classics. These proved themselves time and again in the real world. A great example is the classic books on Time Management, which say "Do the highest priority things first!".

My book relates Agile methodologies back to these classics, and pulls out the essence of why and how they work.

More to come.

Book Writing

For the past few months I have been working for a large and very well known ".com" to help them "go Agile".

The folks on the ground - doing the actual project work - were really enthusiastic, and took to many of the practices quickly.

Unfortunately, my attempts to get senior management "buy in" were a dismal failure. In short, they said they wanted to "go Agile" to make development faster and more predictable, but without altering their own measurements and processes.

After three months of banging my head against the wall, I left the company. Senior management was surprised - whereas the actual teams were not.

Here, then, was my major revelation: Without senior management buy-in you are never going to be fully effective with the introduction of Agile practices. Sure, you can work "under the radar" with some of the practices (e.g. much of XP), but to really get the benefits you need commitment from the very top.

How do we get that commitment? Well, this is what I am figuring out now. One problem I have seen (and not just at the famous ".com") is that you can talk about the benefits of Agile approaches until you are blue in the face, but if it conflicts with the senior management's world view you aren't going to get their commitment. Top management doesn't care about unit tests, or sprints, or product backlog, but about results, about predictability, about beating the competition and grabbing market-share.

The closest I have seen to expressing things this way is Scrum - it cuts down much of the geek-speak and focuses on business benefits. However, it still assumes you already have the buy in you need. Lean Software Development is pretty good too in focusing on the benefits rather than the techniques, but the only response I got to that was "there is no waste in our organization!". There is also David Anderson's excellent book on Agile Management, but that is more about how to apply modern management theories to software development, rather than how to convince senior management to make a culture shift.

My aim is to push these things even further: To understand how geeks like me can be persuasive at the top levels in big firms. At the ripe old age of 43 I have now quit my job and am completely focused on understanding how big bosses think. What are their fears and desires? What compels them to make huge shifts in their world-view and and commit to major corporate change.

In short, I am re-thinking the whole Agile perspective in terms that have meaning to senior executives. My mission is to unravel how to get executive buy in.

Hopefully, this will be interesting to others too. I will be writing up my progress in uncovering this stuff on this blog, and also on my website: http://www.anthonylauder.com/

Feedback would be very welcome. In fact, without feedback, I fear my investigations will drift off into theory. Your feedback will keep me grounded in reality - keeping my head out of the clouds and my feet firmly on the ground.

Ultimately, I would like to produce a book: firstly because I am deeply passionate about this topic, and secondly because I feel this is an area where there hasn't been much written so far (let me know if I am wrong!). That, however, is secondary to uncovering and sharing the knowledge itself.

This is how to contact me: anthony [at] anthonylauder.com

Cheers
Anthony

Successful Project

It has now been about ten months since my main client hired a new boss to sort them out. He seems to have made great improvements. The downside, personally, is that he now feels that the project in which I have been lead developer is sufficiently complete that they can do without me. I have agreed to work with them for a couple more months helping them become independent. Personally, I believe that the project still has much to do, but at the same time it is good that the client sees the project as a success, and that they are now confident they can proceed with it on their own.

This has been, perhaps, my most successful project to date - and certainly the one of which I am most proud. Several things worked well: we followed an agile approach with Crystal as its philosophy, and Extreme Programming as its practice, and this delivered on its promises. Having said that, we didn't do much pair programming - and that was perhaps a serious omission, since each developer became an expert in only one area of the system. Perhaps more importantly, we only has an on-site customer for about twenty percent of the whole time. This meant that we guessed requirements far too often. Furthermore, we released frequently: at least once per month, but received nowhere near enough feedback along the way. If I could change one thing, then, it would have been far more involvement by the customer's decision makers and business experts - rather than just their technical staff.

Comparing this project to earlier ones, I see that upper management can (and did) make a great difference to a project. If a team is just left to "get on with it" then it is highly unlikely they will magically materialise something that adds business value. Upper management needs to help everybody stay focused on delivering clearly articulated business value, communicative on progress and plans, and collaborative in steering towards them. Collaboration, I have learned, is not just about sticking people in the same room and telling them to work together, but about ensuring that people really care about and are committed to the success of the project, and put in personal effort to ensure that one another is as effective as possible. "Team players", then, are not hired or born, but cultivated.

The end of this particular project has actually caused me to reflect on a few of the project's successes:

Unlike just about every other project I have worked on over the past 20 years, this is one I am actually quite proud of. There have been ups and downs along the way, but overall I sense that the project has moved forward with a regular heartbeat, has resulted in a very happy client, and has taught me a tremendous amount both technically and professionally.

In fact, the project has been so inspiring, that I am going to take the core concepts from the project, and rework them over the coming months into a product with much wider appeal. As a sneak preview, I will be developing in the first instance several technologies that will simplify considerably the construction of CRUD applications. Although many developers look down at CRUD applications, I have found that non-trivial ones are very challenging, and very valuable to businesses.

It won't be until the end of the year that I start full time on this, since I am deep in the process of complete hand-over of my current client work. Starting in the new year, though, I will be developing pretty rapidly and full-time, and then releasing my output as open source software.

One vital point is that this is not intended to be a technology that can be thrown at any project to make it successful. Rather, the intent is to help reduce the technical burdens that plague many projects, so that developers and business experts can spend more time on communication and collaboration in working out business-value-laden requirements. The developers will still have to be technical, but should become more productive, freeing them to spend more time on systems analysis with customers. Furthermore, the technology will, I hope, match well with the agile principles of beings able to keep pace with requirements change, so that the collaborations between developers and business experts can remain ongoing and fruitful. That is, as well as eliminate big-up-front-design, we can let go of big-up-front-requirements-analysis.

The ultimate aim is then to establish a reputation as an effective collaborator in the development of practical software, which helps companies focus on delivering business value rather than being held back with technical distractions. From this, I hope to establish a new direction in my consultancy career - helping people to use the technologies I have released, in conjunction with several others that bring clear benefit. The focus will be always be on understanding evolving business value and how that maps to changing requirements in the software systems supporting that business value. It will be, for me, a fresh challenge, and I hope, for future clients, a rewarding experience.

Why?

I have worked on several agile projects and found they went really well. We talked to the customers, found out what they wanted at each stage (and even let them change their mind sometimes), delivered frequently, and proved it all worked with a fine body of tests.

These projects were clearly successful – so maybe agile methodologies were the answer. I did have one niggling concern, though. I noticed it was always the best people we put onto these projects – not just in terms of their programming abilities, but also in the analytical ability of working out with customers what the requirements were (albeit only in small scale increments).

These developers, then, had somehow gained the requirements gathering skills seen in analysts of old – they weren't drawing lots of code-level diagrams, nor were they simply writing down what the customer said. Rather, they were doing real analysis of their interactions with customers.

I often wondered what would have happened if the folks involved were not so highly skilled in their analysis abilities. Eventually, I actually found the answer (at least in a single project which shall remain nameless to protect the guilty). A customer asked a team with which I was working to develop some business software over two years. We were to deliver it incrementally – taking an agile approach – with deliverables every month or two. For each increment, the team negotiated requirements just-in-time with business folks, gave demonstrations to the customer, developed lots of tests to prove the software worked properly, and was given the thumbs up at every stage that we were doing a great job.

Then, after two years (and about 15 releases) we were told "that isn't what we wanted at all".

Well, what a knock to the head that was. How could we have been “doing a great job” throughout the whole project, and then have delivered the wrong thing at the end? We had been "fully agile" we had customer involvement and approval at every step. It didn't add up.

It took me a while, but eventually I realised that we were taking customer "thumbs up" as a sign that we were doing the right thing, whereas the customer was taking our own frequent deliverables as a sign that the project has a heart beat rather than stalling, and hence was in no need of interference. As one business person later said to me “I am am too old to understand all this – we leave it to you younger technical guys”. Nobody was questioning whether we were actually building the right system. All the little pieces were correct, but the overall system was completely wrong.

Eventually, for this and many other reasons, the company hired a new very senior manager to oversee most of the business operations. He turned out to be just what was needed: somebody with the authority and insight to keep asking "is this strategically correct"? That is, he remained very much focused on continually asking not "can we make a computer system that meets tactical obligations?" but more importantly "what are the strategic business needs towards which our computer systems should contribute?"

His questions were quite simple, but they caused a major shake-up. The two year agile project was frozen (a euphemism for canceled). The project started again, but this time with a small group of very experienced people from both the business side and the technical side, continually driving requirements decisions from strategic business needs. That is, not just asking "What and How?" but also "Why?"

I see this as the real heart of the problem in many of the projects in which I have been involved. We were forever trying to get people to pin down the "what, and how?" requirements of the computer systems in a specifications that we could then translate into code. This is not analysis, this is transcription.

Analysis – the part developers rarely have the background and (perhaps more importantly) the authority to do – is the art of asking "Why?" all the time.