Using the “Who” process for hiring developers and scrum masters

By Sarah

1 August 2016


This blog post by Clarion Coughlan was first published on

Hands up if you find hiring one of the most painful parts of your job? Perhaps you’re new to it and you lack confidence; perhaps your existing process isn’t working as well as it should; or perhaps you’re just plain worn out by the sheer volume of applicants you have to screen and interview. Well, I hear you — having experienced all of these things as a line manager and hiring manager.

A new way of hiring

We recently trialled a new process for hiring at Boost based on the book “Who that gave us greater rigour, speed, confidence and ultimately a higher caliber of job candidates to interview and hire. I thought I would share our experiences with you because I was somewhat surprised by how applicable “Who” is for hiring in tech. At first glance, “Who” seems to be directed at C-level recruitment. Can it be used for developer and scrum master roles? Is it compatible with our Agile values? The answer is, “Yes!”.

How tech companies are hiring currently

I recently spoke to the hiring managers from Wellington’s best known tech companies to find out more about how we hire as an industry.

It’s commonly understood that who you hire is the most important decision you can make as a company. And it’s common practice for hiring activities to be undertaken by the most senior members of your team. Hiring is undoubtedly a big investment.

You may be surprised to learn that we all share similar approaches to hiring, and it looks something like this:

  1. CV screening — Usually conducted by non-technical staff who find technical CVs incredibly difficult to discern.
  2. Casual chat— Usually undertaken by a hiring manager to validate what is said in the CV is true. A quick assessment for cultural fit and technical skilfulness is also made.
  3. Interviews for cultural fit and technical skilfulness — The most promising candidates are invited to in-depth interviews involving senior managers and tech leads lasting some 2–4+ hours.
  4. Team meet & greet — The leading candidate is given a chance to socialise with, and even interviewed by, future team mates.
  5. An offer is made.

What the best hiring managers know

Tech companies globally are currently preoccupied with two concerns in recruitment — the scarcity of talent and lack of diversity in tech.

The hiring managers I met with know that hiring is not an art. And yet we see so many companies approaching it as such. The lack of process resulting from this mindset creates conscious and unconscious bias, which in turn narrows the talent pool.

Great hiring managers know that good process and a data-driven approach is your best bet against bias and pointless interviews. Without it, you risk excluding promising candidates, and wasting time on interviewing people who are not right for the job.

Outstanding hiring managers know to use hiring teams that are themselves diverse, and who are really good at hiring because they are trained to hire for the long-term good of the company.

In my mind, “Who” offers a process that is compatible with addressing these big concerns that we have in tech recruitment.

Let’s get specific about “Who”

Boost largely follows the process set out in “Who: The A Method for Hiring”. Everything referenced below is taken from it. Please note that I have decided to only focus on the interviewing aspects of the process. So I have skipped the way we reference check and sell the role. So I highly suggest that you read “Who” to find out more 🙂


At Boost, we decided that we were only interested in hiring people with the right skills, requisite intellect and growth mindset to join us. This is a subtle but vital shift in our approach. I’ve seen companies inadvertently highlight technical skills over other important competencies because they assess this rigorously in structured tech interviews, while using less robust means to determine “cultural fit” — the catch-all for other capabilities.

To do this, we created Scorecards which outlined the mission, outcomes and competencies for the roles. What’s interesting about our Scorecards is what we’ve omitted — specific languages, technologies and methodologies; education and qualifications; and years of experience — criteria I have seen in more traditional job descriptions. Instead we used competencies that very much aligned with our Agile values such as the one for “growth mindset”:

Has a growth mindset — understands that competency is not fixed but is enhanced through dedication and hard work. Demonstrates a love of learning and resilience to adversity that is essential for great accomplishment.

As applications came in for the developer and scrum master roles, and throughout every stage of the hiring process, we assessed them against these Scorecards. The results were somewhat startlingly as we consistently interviewed great people that blew us away!

First interview: Skype screening

If a candidate looked promising against the Scorecard for the role, we invited them to a Skype call. Skype calls are preferred over in-person meetings because most applicants are screened out at this stage. So you don’t want to spend too much time on these. The call is conducted by one hiring manager and takes no more than 30 minutes. During this call, we asked four questions:

  • What are your career goals?
  • What are you really good at professionally?
  • What are you not good at or not interested in doing professionally?
  • Who were your last five bosses, and how will they each rate your performance on a scale of 1–10 when we talk to them?

After the interview, we screened out everyone that didn’t strongly match the Scorecard.

Second interview: Topgrading

If a candidate looked promising after the initial Skype screening, we invited them to the next stage which is a topgrading interview. It’s best to do this in-person. However, Skype calls work just as well for people who can’t make an on-site meeting. We divided the candidate’s CV into chapters. Each chapter could be a single job, or a group of jobs. The interview was then conducted by two hiring managers and takes about 1.5 hours. We walked the candidate through their career history chronologically — starting with the oldest jobs. And we asked them five questions for each chapter in their career:

  • What were you hired to do?
  • What accomplishments are you most proud of?
  • What were some of the low points during that job?
  • Who were the people you worked with?
  • Why did you leave the job?
  • After the interview, we once again screened out everyone that didn’t strongly match the Scorecard.

Final interview: Technical

If a candidate looked promising after the topgrading interview, we invited them to the final interview, which is a technical interview and deep dive into their skillfulness for the job. Once again, it’s best to do this in-person, on-site.

Prior to the interview, we prepared a challenge for the candidate. For example, we asked potential scrum masters to do a lightning talk on any aspect of Agile. Whatever the challenge, we tried to make it fun and relevant to the role. We also prepared good questions that helped us to fully comprehend the candidate’s technical capabilities. We understood that poor preparation would result in a poor interview where we didn’t learn much.

As with the previous interviews, we screened out everyone that didn’t strongly match the Scorecard.

And in the spirit of continuous improvement, we reviewed the way we conducted our technical interviews, keeping what is working and changing what isn’t working for the next interview.

Who to hire?

Think of the three interviews as a funnel that is wide at the top and narrow at the bottom with less and less candidates flowing through. For example, our recent developer job vacancy generated 72 applications. 16 people were invited to a Skype call, six people progressed to the topgrading interview, three moved onwards to the technical interview, and one developer was finally hired.

We knew who to hire when:

  • We were 90 percent or more confident that the candidate could get the job done because his or her skills matched the outcomes on our Scorecards.
  • We were 90 percent or more confident that the candidate would be a good fit because his or her will matched the mission and competencies of the role.

Wait. Are you nuts?

But wait. These questions are so generic and these are very technical roles. How can you possibly discern anything useful from them and hire a great developer or scrum master?

Trust me, you do.

Think of these questions as scaffolding for great conversation. When a candidate responds to, “What accomplishments are you most proud of?”, you need to get curious and excited by asking follow-up questions until you are clear about what that person is really saying. Hiring may not be an art but you do need to learn the art of having a great conversation. And that’s where trained and experienced hiring teams come in. You cannot avoid having a technical conversation about technical roles! So if you do care about specific languages, technologies and methodologies; education and qualifications; and years of experience etc — don’t worry too much: It all gets revealed without it being overemphasised. What is highlighted are other importance competencies which make for a great hire.

Are you nuts? You’ve published your interview questions and process online. People will cheat and trick you!

Funny. We don’t think of interviews as a test where we set traps for people and see which ones they fall into. We see the hiring process as a shared journey. And as hiring managers, we really want to give great candidates a chance to shine. We want to truly collaborate with them throughout the process and have meaningful and honest conversations about whether or not we should work with each other. And that’s what we’ve managed to achieve with the help of “Who”.

Related links

Test-drive Boost — Collaborative hiring events that let candidates try Boost before they sign up

Make a bigger impact tomorrow

Talk to us today