Start with clear scoring criteria
Laying out the scoring criteria in the coding challenge before it’s assigned serves two purposes: reducing individual bias and structuring the post-challenge discussion. You won’t be able to fully eliminate subjectivity in the interview process, but having predetermined scoring factors laid out will certainly limit it. Slack, for instance, takes a transparent approach to grading coding challenges:
The exercise is graded against a rigorous set of over 30 predetermined criteria. We’re looking for code that is clean, readable, performant, and maintainable … We do this to limit bias. If there are clear criteria, variations that might impact score but have nothing to do with the candidate (such as if the grader is having a good day) are less likely to influence the outcome.
That’s not to say you need to detail all of the criteria in the instructions, such as stating the candidate will be graded on how clean their code is — having purposefully vague instructions can be valuable to gain insight into the candidate’s thought process. In agency roles, for example, requirements can often be vague and assumptions can lead to costly consequences, so the questions candidates ask can be one of their performance factors.
On the other hand, confusing or misleading instructions (e.g., saying something is optional when it’s a key scoring consideration) can add time to the exercise or send candidates down the wrong path for the project, hurting your ability to assess the interviewee’s skills and adding time for both developer and code reviewer. Providing thoughtful guidelines will drive better results for both interviewer and interviewee.
Align the challenge to the role
One of the most frustrating things for a candidate to experience is receiving a take-home challenge that’s either peripheral or irrelevant to the role they’re interviewing for. For a QA developer candidate, it could make sense to provide a historical version of your company’s iOS game and have them set up some test scripts — that same exercise for an API integration engineer wouldn’t be as appropriate: they’ll question whether the focus of the role might be different than what’s been described or if the interviewer doesn’t understand the position.
You want your potential hires to have a sense of what day-to-day responsibilities will look like and hope that they feel both excited and capable to take on the role, whether that’s setting up test scripts or an infrastructure to parse API data, or designing out a basic web page.
Don’t set unnecessary parameters
The reason why interviewers turn to exercises like pair programming interviews and take-home challenges is because these provide a more realistic environment than a whiteboard. That means it’s important to make the exercise as realistic as possible, ideally with input from developers on your team.
If you’re setting explicit limitations, such as not allowing the candidate to use generators to set up an app or restricting them to a certain framework, make sure there’s a good reason for it since it will add time to an already time-intensive project. Any guardrails you set should help provide additional insight into the candidate.
For example, does the team work within a limited tech stack and needs the candidate to be comfortable with that stack immediately? Do they want to see how the candidate picks up something unfamiliar? Or is it most valuable to understand how the candidate approaches the challenges using the frameworks, libraries and packages they’re most familiar with?
As Atlassian describes in their approach to coding challenges, there’s no right or wrong to any of these, but when setting up rules and shortcut limitations, know why you’re doing so.
Allow them to add it to their portfolio
Some of the pushback from candidates on coding challenges is that it’s “free work”. There are several arguments against that logic, though: a challenge won’t be used as actual work-product, and candidates don’t get paid for any other part of the interview process, which can be as long as a take-home assignment.
But you don’t want your candidates to feel like they’ll put several hours of work down a black hole if they don’t get the job. Saying at the outset that they can add the project to their GitHub profile can add goodwill, though you should probably add the caveat to leave your company name off unless they don’t plan on offering the same challenge again.
It’s also important to consider candidates’ overall portfolios. While many won’t be able to share their past proprietary work, any individual work or contributions to open-source projects can help your hiring manager better evaluate and anticipate the quality of their work.
Discuss the challenge
Finally, it’s good practice to discuss the coding challenge after the interview: can the candidate justify all their code and explain their approach, or did they do a Google search and paste some code from a thread without a full understanding of it?
From the candidate’s perspective, whether they move forward after the assignment or not, they’ll want some post-mortem of it. After all, they likely spent several hours on the project and will want to know the impact it had in the interview process. Offering this feedback could prevent candidates from leaving negative reviews of the interview process because they don’t fully understand it.
This is where the scoring criteria come in — if the candidate is offered a position with your company and the challenge was aligned to the role, it can offer insight into how performance is measured. If the candidate doesn’t move forward in the process, it’s just as useful for them to understand why not. At the very least, this feedback can help candidates better tackle their next coding challenge, and can even help them identify a skills gap that may have an impact on future positions they apply for.
Take-home challenges may always be controversial. But by building a coding challenge (or using a pre-built assignment) that emphasizes a positive candidate experience, you can make the process far less painful for you and talent alike.