Imperfect #8: Team Performance Basics
Successful team, successful manager.
Some thoughts today on how team health, business objectives and technical indicators coalesce into a measure of team (and manager) performance.
But first, a word. I’m anxious about what’s currently happening in Ukraine, and I’ve mixed feelings about the West’s answer. Other humanitarian crises continue to rage but have fallen off the news. As a father, I cannot image leaving my family to go fight in a war, or having a child leave to do the same. And it fills me with a terrible sadness to think of all those who can’t sleep at home anymore. And yet that is the reality of millions of people right now.
Compared to that, the troubles of engineering managers seem positively minute.
But, unable to singlehandedly changing the course of world events, each of us has to find the resolve to go to work every day and do what we can for one another in whichever capacity that is.
For better or worse, the world keeps turning. We might be struggling with our own feelings about it, but we still have jobs to do. As managers, we have the ability to make things a bit better for one another, and there’s merit to that.
But it can be difficult to know if we’re being successful. And that difficulty can leave us flailing. As a human being with emotions, I find it hard to support others when I myself am struggling. So I thought today I’d share some ideas on how to measure our own performance, so that we can calm down a bit and focus on getting those measurements up for the benefit of our teams and companies.
There’s a certain serenity in knowing if we’re doing well, or badly, and being able to tell where the issues are. It brings back a sense of control. Control over something small, but maybe that’s enough to stop us spinning out into the pit of despair.
Interview question: how do you know you’re being successful?
To be honest, I had been managing people for a few years when I arrived at a solid sense of what to look for in terms of success indicators. Some of the criteria I didn’t arrive until recently, and I’m sure a year from now I’d write this differently still. A recent conversation with my good friend Nikita Vershinin helped crystallise my thoughts into a short list that you can use as a cheatsheet:
Team: do people generally do well in your team(s)?
Retention
Job satisfaction / pulse surveys
Senior : non-senior ratio
Business: is your output driving business outcomes?
Core, long-lived KPIs
Success metrics
OKR
Operations: is your everyday operation conductive to high quality and productivity?
DORA metrics
Let’s look at them one by one.
Like what you’re reading so far? Subscribe to receive original writing on tech leadership in your inbox!
Team
You’re doing well if people want to be in your team(s). The following factors can tell you if that’s happening. As an Engineering Manager, you’ll be able to influence some of these factors. As a Director or VP/CTO, you’ll influence all of them. Remember, though, these are only a means to an end. The end is to deliver business value through working software. Retaining talent and domain knowledge helps achieve that. Any time that money or time (which is money) needs to be spent on retention and job satisfaction, it needs to be put through the lens of how it helps achieve our ends.
Retention
Do you lose a lot of people to other teams or companies, as compared to other managers? If you don’t know, ask HR. If your numbers are bad, focus hard on job satisfaction. Generally that’s career progression, sense of purpose, learning, challenge, compensation. Remember that each person will have their own goals and motivators, so make sure to spend time with your reports to learn what they want. Anything you can’t improve, make sure to flag and bring up with whoever’s above you in the chain - and be specific with what you need.
Job satisfaction
Related to retention. Pulse surveys are a good tool to have an abstract idea about how your team is feeling. Take them with a grain of salt, but understand that as far as higher management goes, those numbers are your metrics. Engage HR to understand your results and leverage your knowledge of individual team members to identify actions points to improve things in the trenches. Some times, people need to move to different projects they find more interesting. Other times, they need a raise. Or a laptop that works.
Senior to non-senior ratio
Teams without enough senior engineers can struggle to make decisions and meet quality standards. Worse, learning is slow in those teams, as there is no one to learn from and people are forced to discover everything from experimentation and first principles. Which is fine a lot of the time, but a few failures in a row can sap all drive out of the most motivated people. It’s good to have a few experienced hands around to point people in the right direction. Velocity is motivating in itself.
Business performance
I’ve seen a lot of CEOs admire web pages and mobile apps, but not many browsing GitHub. Very few of us have jobs because software needs to be written. The need is usually for some kind of business value, and software just happens to be the way to make it happen.
We can have the healthiest, happiest, most productive teams in the world. If they’re consistently delivering the wrong product, we still have problems. So it pays to watch our business metrics and be mindful of the product development side of things.
Project success metrics
Any initiative needs to have measurable success criteria in order to make sense. Without success criteria, we can’t know if our time was well spent. Of course, much development happens without any and this is a tragedy.
Be wary of environments where neither engineers nor PMs can identify success criteria for the initiatives they develop. The latter are betraying a lack of accountability, the former a lack of understanding.
Simply ask “what’s the number we’re looking to move here?”. If no numbers are being tracked, it will be impossible to spot a change. In that case, please insist on implement tracking of those numbers as part of the project. Sometimes, a periodic database query is enough. It just needs to be possible to chart progress over time.
Long lived KPIs
The smaller the company, the harder this can be, but it’s very useful for teams to have very few, very clear, very long-lived Key Performance Indicators. Maybe an API team can have KPIs related to traffic, latency and request duration. A team responsible for a checkout flow in an online store can have KPIs about checkout completion, page load times or upsells. These eternal KPIs synthesise the answer to the question: is the team fulfilling its mission? Of course, one has to have been selected for them to be possible.
In small tech groups, it’s hard for teams to have very few KPIs, since there’s a lot of multitasking going on and people own many things at once. In that case it helps to define those KPIs at product or service level. As the company grows and teams become more specialised on fewer and fewer products, they inherit their KPIs.
OKR - Objectives and Key Results
Love them or hate them, they seem to be here to stay. So better grab a copy of Measure What Matters, lots of coffee, and roll with it.
In organisations that use the OKR approach, we can rest assured they’re getting brought up in management performance reviews. In organisations that don’t, some other kind of objective tracking approach will be used. Either way, someone is going to be paying attention to whether goals are being achieved. And if no one isn’t, that might be an issue for our own career advancement, as well as our reports’, as it will become harder to demonstrate how much our team’s efforts flow into the entire company’s results. I’ve seen that kind of situation and it isn’t good - the solution there is to keep track of things by ourselves and be vocal about our findings.
In any case, this is related to project success metrics except it’s usually tracked vertically per team, department, unit and company. Project success metrics, by contrast, are tracked horizontally down the time axis, that is to say, per project.
Be on top of that.
Technical operations
The metrics that junior managers wish were the main ones. I’m sorry to disappoint. Once again, very few CEOs ever ask how many deployments were done today. They tend to be more interested in whether we’re hitting our conversion goals in the checkout funnel. Or in why the site’s down again.
That’s where these come in. Good technical metrics aren’t an end in themselves, but they can be a leading indicator of quality. A team that deploys often, without breaking production, is more likely to be able to deliver working software fast than one that doesn’t. That can mean beating a competitor to market, and the CEO will very much care about that. Technical quality is a precondition for reliably successful product delivery.
In my career as manager I’ve considered lots of different metrics, but the industry seems to be converging on the findings of the DORA group, so I’ll list them here:
Deployment frequency: how often do you ship working code
Lead time for change: how long it takes for a commit to go live
Change failure rate: how often do deployments break production
Time to restore service: how quickly are production breaks fixed
The thing about these metrics is that they’re either very easy to gather if everything’s automated, or very hard if nothing is.
If our code goes through CI/CD pipelines with automated QA, our alerting systems trigger incident responses on their own, which get managed with software all the way to resolution, all that can feed data into graphs that show the DORA metrics with very little extra work.
If we need manual QA before going live, with no information being gathered anywhere about the manual parts, if there’s no automation after pager duty alerts or any ticketing system in place, it can be impractical to keep track anything. Events can still be tracked manually, but from experience that’s work that easily gets deprioritised.
So the advice there is to treat the feasibility of DORA as a technical operations smell. If it’s hard to implement, that in itself can indicate deeper quality issues, so definitely work on those.
Closing words
Successful team, successful manager. This is true all the way up the chain. Engineering Managers in charge of single teams must ensure their teams have all the conditions to be healthy, and operate at high levels of quality, while meeting business expectations. The higher up you go, the more focus needs to be on business goals, with competent technical people leaders in charge of smaller units.
Hopefully you find this useful, and it helps you reach some sense of how you’re doing and, if not well, what to do next. Clarity makes good managers, and good managers make good experiences for people at work. And we could all use some of that.