I sometimes get involved in discussions about whether agile teams need project managers.
I have a very strong view on this – NO, we do not need project managers, unless of course, we do, in which case YES.
Project managers are nice people and we should let them hang out with the team if they want to. But there is a conflict in an agile world, where some styles of project management come undone because:
I hope to show this by reviewing my own history in running projects and then look at how different project managers might need to adjust their approach when working with empowered teams.
Then in another article, I will explore the problem created by product management and how this upsets good PMs as well.
Then finally I hope to show some positive patterns or approaches where I believe we can gain real benefits from having a PM in an agile project.
The problem with self-managing teams
My early projects were run by artisans rather than project managers
When I first worked on projects I was really just the senior guy leading a team of peers. We shared a related skill set and we were all clear that we were working on the same thing.
My skill was superannuation (pension calculations and things) and data conversion. I am not sure if these are recognised crafts with their own guilds, but we were experts in our field.
People called me the PM but I was really just an experienced guy leading some less experienced people. We managed our work based on our craft. Or sometimes a less experienced guy following the lead of a more experienced artisan.
We had no product owner. Well, in theory, we had someone who asked us to do the work. But that person trusted us because we were the experts in our field – we told them what was needed as often as they told us.
As the senior craftsman, I did some work and delegated some. I found that the best way to delegate work was to talk about what to do, then agree together who would do what, and then get on with it.
I know empowerment is good, but there were many times when I let junior people learn on the job. This meant that they got some things done in an ineffective way because I thought they would figure it out and learn how to handle things next time.
I prioritised learning over efficiency without ever checking with our “product owner.” This is good in that it builds capability but also challenging in that it impacts delivery.
Other times I took shortcuts because my client needed something fast, or I told them that we needed to do things properly and that “this will take a while – we need to do it properly.” In these situations, I made decisions about value based on “the needs of my craft” rather than letting the customer decide.
I began to learn more about running projects
After a time, I learned that we needed to have some structure to our “decision rights” rather than just saying “let me think – yep do that.”
I learned to use, very roughly, this decision framework:
We did not actually have an iteration manager, a ScrumMaster, a project manager, a dedicated product owner, or any other project related role base. We just had “the crew,” the “experienced crew member,” “impacted customers” and “what we need to do.”
Our customers loved us but to be honest they often followed our advice on how to get things done, and sometimes even on what needed to be done.
I like to think we were an agile team even though some people have told me that we were more “a good team” than a “proper agile one.”
Things changed when I ran scary projects
A little down the track, I became a real project manager and I managed some highly skilled, highly technical people.
They knew a lot more about their work than I did, so I could not tell them how to do it. Naturally, I was still accountable for the outcome even though they were more across the work than me.
Thankfully, a mentor told me what project management is really about:
"Project management is not about knowing how to run the project, we just need “one throat to strangle” when it goes wrong. So we strangle you and you can decide who to take down with you."
I am not sure that is really aligned with agile concepts such as “safe to fail” but it did clarify my role for me.
This was helpful advice because I soon learned I was to blame for anything that happened on my watch. I guess blame is a bad word – maybe I mean accountable.
Interestingly, I was never thanked when everything worked really well – I was just there to take the hit for the team when it did not.
This was good though – I learned that the team do excellent work if they know they can take risks and that you will be there to take the hit. My customers also trusted the team to try new things because they knew I was on top of things.
Actually, I was never fully on top of things, but I did have excellent instincts, even early on, to know when to hold my breath and when to get to the bottom of things. I also seemed to know when to go along with things and when to make a tough call that the team did not want to make.
I learned that if you cannot trust the team then you are doomed. There is no point in trying to tell a technical expert how to do their job if they are really technical and you are not.
If the project manager is telling the technical experts how to do their job then they are doomed
So far so good, but I also learned something else.
If the experts in the team cannot explain their thinking to you in simple, dumb, project manager terms, then they are about to make a terrible mistake and you are all in trouble, even if nobody knows it yet.
So I learned to ask really dumb questions until I was confident people knew what they were doing.
If the technical experts cannot explain their thinking to you, it is generally not their communication skills that are lacking. It is usually an indication that they are not clear on the problem they are tackling.
I found that quite often, several talented people understood different parts of the puzzle, but nobody really knew what the whole thing looked like.
I also learned that frequently, everyone thought that they understood the problem because there was no disagreement. But there was actually no disagreement because nobody understood the problem well enough to disagree.
When experts in different fields work together you get excellent results if they communicate, but things go south if they sit in their own silos solving different problems and then try to bring their solutions together.
So even though I was the PM, my real role was a combination of:
Figurehead or shield to allow people to express risky views; and
Boring annoying guy who forced people into long discussions and dress rehearsals rather than leaving people alone to do their work.
Times were good. I always turned out to be a lucky project manager throughout my career.
The need for rhythm
With some experience behind me, I also found that as the PM, I strongly influenced the rhythm of the project. I could stumble about and get in the way or I could slowly get the project team moving to the same “beat.”
Success usually followed a rhythm, while a lack of rhythm usually meant that something was wrong.
I learned to listen to my gut and stop things when they started to get out of whack. People never wanted me to stop things to have even more meetings and investigations, but a project that was out of whack (unstable and jittery) always got worse if I did not intervene.
When I discovered agile project management I got really excited. But when someone explained scrum to me, I thought it was an anti-climax. I thought they were talking about project management 101.
When someone said I should become a scrum master, I replied that I was already the project manager and I was perfectly happy. So I told them to go away.
Strangely enough, I have done all sorts of roles in agile teams but somehow I have never been a scrum master. It is kind of my dark secret as an agile coach.
Big scary things are similar but different
Anyway, I started to work on larger projects – sometimes they were called programs or “BST’s” or “Big Scary Things,” or something similar.
I made the transition easily because I never really knew how to get things done and I never really managed the team. This was good news because there is less of this involved at a more senior level.
You need to coordinate the teams but they actually manage their own work. So need to put in place feedback loops so you find out the things you should know about and people in the teams know what is going on. Then you intervene by exception or just harass people when you feel bored.
Of course, I learned some new things as I ran big scary things. This is a summary of the key lessons involved when I managed multiple teams:
So I managed my BST’s using a combination of “management by walking around,” lots of meetings and huge numbers of dependency battles … and focusing only on the bit that seemed to be in the most trouble, or the most urgent.
But then I learned yet another lesson. As soon as a project gets big, you run out of resources. You either cannot get people a computer (which is embarrassing), you have only one DBA or one interior decorator, or you start to run out of time and money.
Strangely, this is where I found my real strength a project manager – scrounging around to get people what they needed.
In agile people talk about removing impediments but this was nothing compared to my skill at asking for favors and acquiring what we needed through dubious and unconventional means.
While I wanted to be like a great general driving the army toward success, I was more like a character called Radar O’Reilly in a show called Mash. I could see trouble coming and I always found people what they needed to get the job done. Then they all worked out how to get the job done.
My teams got better equipment, they got extensions on deadlines and they even got permission to go into production in change freezes when nobody else could. We had staff from call centres volunteer to come and do testing and we had support teams like brand and legal coming to talk to the team about what they could help with.
I am not sure if this is how projects are really meant to be run, but this is how I learned to run them.
All along the journey though, I realise now that teams were managing their own work. I was facilitating the debates and acquiring the things that the teams needed.
Working with product owners
I realise now that learned to secretly become the project owner. I would take accountability for the scope and more importantly for the value we were delivering.
I often wrote the business case and where I did not, I created the mandate for the team and made the final call on what we would and would not deliver.
Sure, I asked permission of the sponsor or the client, but they usually relied on my team’s recommendations, because we really knew what was going on and they trusted us. Plus I controlled the resources and therefore controlled the ability to execute.
It all seemed to work pretty well when we went agile, even though I was probably in violation of the structure and the approach we would establish in an agile team. As the project manager, I was in many ways the PO and in some ways the team facilitator.
There was a minor issue when I worked on “agile” projects
The problem came when I first worked with product owners.
Actually, while we called them the product “owner,” I was treating them as more of an SME (subject matter expert). They were like a local guide that gave us information to act on, or they were someone with valued opinions. But they were not in control.
In fact, if they became too demanding or unreasonable then I automatically became the shield of steel that protected the team from their potential madness. This is a role that I enjoyed and that I was good at.
You might realise though, this is more than a little different from the traditional definition of “product owner”
Some business people liked this because it meant that they could express opinions to both me and the sponsor, without having to make the final decision. The team liked it because they could get one answer from me and one from the product owner and choose the best one.
But businesspeople felt stressed because they felt like they did not understand their role. More correctly though, they did not understand the incorrect role that I said they had.
Other product owners pushed back and I eventually learned that I should back down. But in hindsight maybe they should not have had to push. Maybe I should have said – yep you are the one.
But when I got out of the way I learned that some product owners made decisions without taking into account what the experts thought. This was a disaster.
The expert would give some arcane advise that the PO ignored. Then the PO would give clear directions that seemed wrong to the expert, so the expert would ignore them and do what they thought was right.
Or worse the experts would start deploying paperwork and authorization requests to protect themselves. This is the opposite of collaboration in a self-organising team.
So I came out of retirement and became the annoying PM who we thought we didn’t need – but who made the PO and the team collaborate. I was really just a mitigation for bad collaboration. Then, once again, I came to realise that mitigating bad collaboration was different from building good collaboration.
It seems obvious now, but I somehow missed that the product owner is actually the owner when I was the PM and that they and the team should get along without me if they want to collaborate and self-organise. I honestly thought I was being agile, and therefore I missed some pretty obvious things.
So that was my journey.
There is no room for bad project managers in an agile setting
Some project managers learned their craft by taking a different journey.
Some project managers, who I will call bad project managers, learned to avoid being strangled when things went wrong.
That sounds sensible, but the problem is that they learned to avoid trouble by either blaming the team or by creating piles of paperwork that protected them from being blamed if the project got into trouble.
These PM’s claimed to be working towards success, but they always positioned themselves to be ready for failure. And when things went wrong the PM was safe, but people in the team were often in peril.
If you are positioning yourself for failure, making sure that you are ready to blame the team or the customer, then it is hard to convince the team to commit to success
I actually find that good PM’s can fall into this trap, not because they are bad themselves, but the environment they are in has taught them to adopt bad habits.
However, whether they are nice people or not, they are an obstacle to self-management.
As a result, these PM’s will not like agile – because it is hard to blame a self-managing team unless you let it self-manage, in which case you lose control of it, so they stop protecting you and your fate is tied to theirs.
To make matters worse, they will not produce the paperwork you want unless you really make a scene over it … and it is hard to hide behind paperwork that does not exist.
Instead, the way the team works in an agile project creates transparency and open (healthy) disagreement. Trying to stop this happening breaks both the mindset of agile and the successful implementation of all the practices.
Bad PMs try to control the team without being accountable, rather than trusting the team and still being accountable. They try to control the communication and maintain power and most dangerously of all, they ask the experts to sign off on what they are doing without having the long and arduous conversations that lead to real understanding.
This always leads to trouble.
I am not even going to be polite – they are just bad project managers.
Agile teams can mitigate them for a while but real progress means removing the obstacles the team has – and in this case, the PM is the obstacle.
The good news is that these bad project managers usually evolve or leave;
Good project managers who are reformed paperholics
In many organisations, there is a lot of management going on to align projects with the organisation's management of projects. People know this is true but never realise how true it is until they are involved in managing it.
So, unlike the bad project managers who use paperwork to protect themselves, the reformed paperholic will manage all the paperwork so as to minimise the harm it causes the team, while still gaining some of the benefits the organisation wants to achieve from good planning and governance.
That makes them useful to both the team and the organisation. But there is a problem. Even when organisations call themselves agile, I sometimes feel that they are saying “we are agile and you can be too as long as you do all this paperwork and lock things in so you don’t self-organise too much.”
This is important because unlike scrum masters and others who have never understood all the paperwork, these reformed paperholics tackle all the drama and keep all the paperwork out of the way of the team so they self-organise.
This can only go on for so long though because the PM is producing the reporting without the reporting being built into the way the team is working. And the PM is justifying decisions that have already been made while making them look like approved recommendations.
They are also trying to manage budgets, dependencies and scope with tools that are not aligned with the way the team is working. This is hard work.
Anyway, we can ignore the bad project managers because Karma will remove them. But the reformed paperholics are stuck enabling a self-managing team while managing organisation processes that are designed to stop that happening.
They can do this for a while, but eventually, they will be squashed by the paperwork, or they will fall off the wagon and go back to trying to protect themselves. This means that they are bad project managers again and they become an obstacle to deal with.
Good project managers who struggle with self-managing teams
There is another group of project managers who accept that they are to blame if the project fails, but have learned lessons that I did not learn.
They have learned that they can deliver projects through sheer willpower and extreme focus.
Let me explain what this looks like:
The sheer willpower of the project manager helps to bend reality to the will of the project goals and somehow drive things through; and
The extraordinary focus of the PM enables them to understand every aspect of the project so that everything moves together and problems are tackled before they get out of hand
I don’t think these PM’s are bad, I think they are excellent and they often deliver extraordinary results.
But the problem with self-managing teams (or self-organising teams) is that they are managing themselves. So they neither want nor need a PM who is managing the details of the project with great control and precision.
Also, self-managing teams need to deliver through their own willpower and focus, not through outsourcing that focus to someone else.
This suggests that the PM should fail in an agile world. But I have noticed a counter-intuitive pattern. Let's look at what happens when the team starts to go agile.
Firstly, we tell the PM to use a self-managing team and we tell the team that they are all accountable for the outcome. This is all good and it is important.
But to begin with, the team do not know how to work together in this way. Even if they have done it before, they do not know each other too well and so they struggle.
The PM sees the team struggle and soon realizes that the project is slipping. Their instinct is to step in and help the team.
At the same time, the team are worried. They also feel that the project is slipping. So they are relieved when someone steps up and takes command.
Everyone is comfortable. In fact, the team learn to trust the PM and they start to see results so the customer is happy.
This would be OK in a very predictable world. In that world, the PM could start with detailed planning up front then move their focus to execution. They can apply their extreme focus and supernatural willpower to one thing at a time.
Unfortunately, in an agile team, planning and execution are happening at the same time.
So the PM gets great results to start with and everyone thinks “agile” is working well. But in fact, “agile” is not happening. Rather, we have freed up a good PM to deliver the project through sheer willpower and constant management of everything.
This approach works well for a time and so we continue to run with it. But then things go wrong. I referred to this in an earlier article about heroes and agile.
The PM is eventually unable to manage the complexity of the project and teams are half self-managing and half being chased up all the time. In other words, they cannot self-manage.
Eventually, things resolve themselves in one of two ways:
The PM starts out driving the project with sheer willpower but then starts to ease up as things get moving. People start to take control of their own destinies and the PM focuses on clearing the road for them; or
As things get worse, the PM does more of what they know. They continue to add passion and hard work. The PM gets burned out and everyone feels sorry for them. They fall over eventually. But the team is not ready to take on the full ownership of their work that is needed to self-manage. So the PM and the team get fried, both trying to both partially seize control and partially trying to collaborate.
So these guys can really do well in an agile setting – but they often really struggle to hand the reins over to the team. This is not because they fear agile, but because the team want them to stay in control and they believe that they should.
Is there a point to this long story?
The problem for project managers is that they have learned different approaches that have served them well in the past.
Some, who have been a senior craftsman in their role, see agile as a continuation of that. But in fact, they need to learn to adapt to managing the integration of multiple crafts,
Agile is about collaboration of cross-functional, self-managing teams. The PM needs to learn to challenge and trust people whose expertise is alien to them. They also need to support other crafts at the expense of their own. This can be a steep learning curve.
Some PMs, like me, will have operated in a “sort-of” agile fashion in the past because of the necessity of the types of projects that they ran. They will see agile as obvious but will actually miss the subtleties because they will assume it is just the same as what they were doing before.
The bad project managers will see agile as a threat. We might tell them that it will be great for them, but this is not true. In reality, agile should be seen as a threat by bad project managers.
Agile teams need transparency and empowerment or they cannot self-manage. But transparency and empowerment make it hard for the bad PM to remain dodgy and seemingly in control.
Reformed paperholics will fill a gap that often exists between the way the organisation rolls and the way the team roll. This will seem like important work but it is often only a temporary fix. Worse, it often starts to require more and more from both the PM and the team, until a large amount of the focus is on mitigating the organisation, rather than leveraging the vast resources that the organisation can make available.
Finally, the hardworking PMs who deliver through willpower and excellent organisational skills will be very valuable to agile teams in the short-term, but risk taking the burden of self-management off the teams.
This will work for a while but in the end, it holds the teams back from growing and it puts an unacceptable workload on the PM. In fact, the workload keeps increasing until something breaks or the PM changes their approach.
So the problem here is NOT that PMs are bad for agile teams. The problem is that experienced PM’s need to unlearn a lot of what made them successful in order to thrive in an agile environment.
More specifically, the PM must stop “managing” the project and let (even demand) that the team members manage the project. They must either unleash the empowered teams or admit that they are not going to be fully agile.
Posted by James King
Copyright © James King
Your details have been submitted and we will be in touch.