- About Us
- Contact Us
- Course Calendar
Now that agile has gone mainstream, I often find that people are concerned about the performance of the members of an agile team. In particular, people seem interested in how we measure the product owner and team facilitator roles (ScrumMaster, agile coach, celebrity post-it note artiste etc).
When working through this, I came up with some “useful” metrics, but this still seemed to leave a gap, which led me to wonder if it is the metrics we need, or an understanding of what we mean by “self-organising teams, served by scrum servants and product guides.”
In fact, one interesting conversation began when one of the crew asked if there was such a thing as a “senior ScrumMaster.” I told them no. ScrumMasters become more junior and more like team servants until they become a team lacky, or retire from work to become an agile coach.
This seemed to get people upset. Especially because the agile coaches told me that they were not retired – they were working really hard, but just weren’t sure what it was that they were working on. Anyway, that is a different topic – we agreed that agile coaches are really there to help people learn to be more effective as a team and everyone was happy on that point.
For now, though, I thought I would share my own logic on metrics for PO and ScrumMaster types in order to see if it leads to an agreement or grave concerns. I will also let you be the judge of whether this is just a fun article or if it is hinting at a gap that we can fill in the understanding people have of the roles.
Inspect and adapt and other good things
In agile teams, we love to inspect things and improve our performance. Even the humble user story can have multiple metrics inflicted on it – cycle time, control charts, ageing, velocity and maybe more. It follows then that we would also apply this to our people – we can inspect and adapt in order to help them improve.
There is one problem though – they do not operate in isolation. In fact, the best evidence of a good PO and a good ScrumMaster is that the team is able to quickly and easily add value without the intervention of the PO and ScrumMaster. So team effectiveness without intervention is probably the best metric.
In the early days, the team needs guidance, but as time goes on, presumably, the PO and the ScrumMaster are creating greater clarity and learning, which means the team become clear on what to focus their energy on, how to measure success, how to work together and how to manage their own work.
A rating for evolving irrelevance
If the goal is to build a self-managed team that is independent and effective, then it follows that a good metric would be to assess how long it is until the PO and ScrumMaster are completely irrelevant. Ultimately, when they have become completely irrelevant, both the PO and the ScrumMaster have been of exceptional service, but if they are not yet irrelevant then perhaps we could estimate the days left until they expect to become irrelevant. I have included such a metric here:
But what about the rest of the team
Agile work is team-based, so we cannot expect the PO and ScrumMaster to make themselves irrelevant without help. So maybe we can look at some leading indicators across the team, to see if they are supporting the emerging irrelevance.
Collaboration driven irrelevance
The good news is that I have done some research into this very topic – maybe the ScrumMaster is needed because the team needs help with collaboration. This means that they are still relevant as long as the team hopes to either improve their collaboration or to at least have some help facilitating it.
This, in turn, means that the ScrumMaster cannot become completely irrelevant until each team member is completely collaborative.
So now we have a metrics hierarchy – If I rate each crew member on how collaborative they are then I can move them all to 100% collaborative, at which point the ScrumMaster is completely irrelevant.
To drive the collaboration score for each team member we will need to tackle the following drivers:
So first the ScrumMaster must confirm that people want to collaborate. If not then the will not be effective. But once people are willing to try, the ScrumMaster chips away at building the factors that lead to their irrelevance.
A minimum viable metric
Measuring days to irrelevance is good, but seems difficult. Ultimately you can adopt the following formula, but people always seem dubious. The predicted days to irrelevance is actually the greater of (90 days or the number of days the person has already been in their role). So if someone is new, we should see them becoming irrelevant in about 90 days. But if someone has been in the role 45 days, then we still predict that they have 90 days to go. If they have been in the role for 180 days, then we can expect them to stick around for another 180 days.
But most people don’t believe that is a real metric, so you can try this one:
Or if you want more detailed metrics – you can replace these two questions with the set of drivers we apply to team members:
Irrelevance based on autonomy
So measuring the increasing irrelevance of the ScrumMaster is easy, but what about the PO?
It’s a similar journey – the PO exists because we do not trust that the team would work on the right things if they were not there, or we don’t trust that the organisation would give the team permission to do what they want. In other words, we are not fully ready to make the team truly autonomous.
So perhaps we should measure whether the team members can act with autonomy or not. In theory, autonomy is good, but not if we don’t trust the team or more politely we don’t yet think they are ready for full autonomy. The key driver here is alignment, followed by understanding and finally competence.
If the PO does not believe that the team are aligned to the goal, then the PO cannot step away. Once the PO is clear that the whole team is aligned to the right goal, then the PO starts to become less relevant. OK – the first metric for a good PO is to measure the clarity of the goal and the commitment of the team to achieving it.
So far so good. But some teams appear incapable of breaking their own work down or understanding how to achieve the goal without a PO to guide them. In this case, the PO, or a mini-me PO is still needed. So the second driver of team autonomy is the ability to recognise, prioritise and focus on valuable work.
Finally, and this is controversial, the PO will probably not be willing to move away if he or she doubts the competence of the team. In theory, the team might be clear on the vision, clear on the priority and completely inept at doing the work.
I have been asked what the PO should do if they do not believe the team is competent at achieving the goal but the answer is a bit sad. Either you are the wrong PO, or you need to change the mission to one which includes time for the team to learn, or you have the wrong team.
So the final driver for the PO is to measure whether the team are confident and trusted to do the work well. If they are not, then the PO must either seek resources that are needed or prioritise team learning and growth.
OK then, just like the ScrumMaster needs to have a team that is willing to interact, the PO needs a team who are willing to become competent. Once the team is willing and able to become competent, then the PO can begin to become irrelevant. After that, the metrics are similar to the ScrumMaster.
If we do not want to measure days or percentage irrelevance then we can use the following minimum viable metrics:
So - will this work?
I have spoken to people about these metrics and some are quite happy, even though the metrics seem informal. Others seem to want more robust metrics of team alignment and collaborations but I am not sure that is necessary. Others want more robust metrics on the value the team is creating (revenue, joy, etc), which seems fair enough. I think we should focus on that more than the individual performance.
Some people don’t like the idea of the PO and ScrumMaster becoming irrelevant, they want to believe that the team will always need them. The aim then will be to have a single point sensitive PO/ScrumMaster who is micromanaging the team. In that case, I guess you can just reverse the metrics and measure team dependency and helplessness. The metrics will still work, I’m just not sure if you have the right role descriptions.
The chief cause of distress seems to be that many people do not agree that the performance of the PO and ScrumMaster is derived from team performance. They want to be able to say that the PO and ScrumMaster can do good work as individuals, even if the team completely fails to add value and completely fails to collaborate.
I think the whole point of both roles is to make the team (or group of teams) more effective and more self-managing. But if you want to measure something else then maybe measure “number of meetings attended per week” or “delightfulness of the velocity chart” or something.
More seriously though – I think it is possible to measure the performance of the PO and the ScrumMaster and I see value in doing so. But the challenge is to shift the mindset from personal outcomes to “supporting the team to deliver outcomes” and then to shift again from “managing the team to deliver outcomes” to “teaching or enabling the team to achieve outcomes.”
I think that is the sticking point. It’s not the metrics that are hard, it is seeking agreement on the point of the role and the value that it should create. In my view the point of both roles is to unleash the potential of the people in the team and then get out of the way – but I am not sure if you agree.
Posted by James King
Copyright © James King