Whiteboarding and live-coding exercises in tech interviews are counterproductive

It’s no secret that the tech sector currently (2021) uses whiteboard-style interviewing techniques quite often. This can be through the use of actual whiteboards, or services such as CoderPad.

But there is a strong body of science that shows these types of interviews don’t give reliable predictions of candidates’ skills or ability to perform a given role.

The study Does Stress Impact Technical Interview Performance? (Behroozi et al. 2020) compares a “traditional tech interview” where candidates solve a problem on a whiteboard, while explaining their thought processes, against solutions from candidates who solve the problem completely on their own in a private room.

The researchers found that people who took the traditional interview performed only half as well as people that were able to interview in private. Overall:

• Participants in the public setting provided significantly lower scores.
• Participants experienced higher stress levels and higher cognitive load in the public setting.
• In the public setting 61.5% of the participants failed the task, compared with 36.3% in the private setting.
• Interestingly, a post-hoc analysis revealed that in the public setting, no women (𝑛=5) successfully solved their task; however, in the private setting, all women (𝑛=4) successfully solved their task—even providing the most optimal solution in two cases.

GitLab has experienced the same effect with their hiring process, which they’ve written about in The trouble with technical interviews? They aren’t like the job you’re interviewing for (2020):

At GitLab, we found that live coding exercises don’t accurately represent engineering capability. Oftentimes, a recent computer science graduate will outperform a more senior candidate with a lot of valuable experience. In summary, live coding exercises will often disadvantage more senior candidates, people who are nervous in high-pressure situations (read: everyone), and advantages more junior engineers or people who have practiced live coding.

What to do instead

Both of the following research papers offer multiple guidelines to improve this style of interviews. Given that these are open-access papers, I’ll quote their headings and leave you to read the guideline in full directly from source.

• Hiring is Broken: What Do Developers Say About Technical Interviews?

  1. Use rudimentary questions for screening
  2. Share the interview description in advance
  3. Offer alternative interview formats
  4. Use a real problem
  5. Solve problems as colleagues, not as examiners

• Does Stress Impact Technical Interview Performance?

  1. Use retrospective think-aloud for accessing explanation skills
  2. Evaluate the kinds of stress necessary for position
  3. Provide accessible alternatives
  4. Consider impacts on talent and diversity

In addition, GitLab’s blog post from earlier explains the process they adopted, which allows candidates to make changes in an actual repository in order to get the full experience of exploring a codebase, checking CI, committing changes and making a merge request.

References