7 Tips for Using “Problem Solving” Interviews to Screen Candidates
October 30, 2018
Business case interviews have been a mainstay in many industries, particularly consulting and investment banking, for decades. Technology companies use coding and “algorithm” interviews extensively in evaluating software developers—in fact, many technology companies use little else when evaluating engineering talent.
This form of interviewing, which we can generally refer to as “problem solving” interviews, seeks to uncover a candidate’s ability to think through challenging problems in real-time by presenting them with a novel business or technical problem to tackle. These interviews have an important role to play in evaluating candidates for certain types of roles, but they do have limitations. For example, they are prone to generating false negatives—rejections of candidates who could actually be quite capable of performing well in the job. This can happen for a variety of reasons: nervousness, introversion, or simply good old-fashioned “brain farts.”
That’s why problem solving interviews are best used (a) when you are able to give candidates multiple opportunities across interviews, (b) when there is an ample supply of strong candidates (i.e., the cost of a false negative is lower) or (c) when the candidate mix is more junior (i.e., they have less professional history to benchmark).
Problem solving interviews are quite useful when used appropriately, and I strongly recommend them as a supplemental lens to evaluate candidates who will be hired into an intellectually or analytically demanding role. Based on my experience, here are seven important guidelines for making them work:
1. Use a problem that’s relevant
Ensure the problem you are presenting to the candidate is highly relevant to the role they are interviewing for. That means avoiding random brain teasers or riddles—stick with problems that closely resemble those they can expect to face on the job. In fact, the best problem solving questions I have seen are often directly derived from actual work in the company. Not only does this ensure you are testing relevant skills, but it also gives the candidate some direct exposure to the kinds of problems your company solves every day.
2. Ensure there’s more than one way to solve it
Ideally, choose a problem in which there are multiple potential paths to solve the problem. It is common in software development “coding” interviews to present a problem that can be coded up in several different ways, some being (for example) more flexible, scalable or coefficient than others. This allows you to make finer distinctions between “good” vs. “great” candidates, and increases the probability that weaker candidates will still have a positive experience with you.
3. Use phasing to structure the problem
Whenever possible, structure the problem in distinct phases or steps, allowing you to see different elements of the candidate’s thinking process. Many such interviews (both business and technical) are set up in three steps—(1) diagnose the problem, (2) design the best solution and (3) implement/execute on it.
4. Make it interactive
Stick with problems that can be broken into components and discussed interactively. Encourage your candidate to explain their reasoning and talk through the trade-offs they are making as they go along. Much of the data you will gather about a candidate will come from their process, not just their solution. An overly “black box” approach robs you of this valuable information.
5. Create safety nets
Make sure you have plenty of “nudges” handy for candidates who may struggle with one or more parts of the problem. Doing so ensures you avoid awkwardness with particularly weak candidates, and ensures all candidates come away feeling as good as possible about their performance.
6. Maintain rapport – and stay positive
When asking behavioral questions, great rapport comes from showing curiosity about the candidate him/herself. In problem solving interviews, that same curiosity can be channeled into your collaborative exploration of the problem with the candidate. Even though you may have presented this problem dozens of times before, it is important to maintain believable passion and interest throughout the interview. Similarly, be very careful not to express disapproval, judgment or frustration if the candidate gets off track—candidates can detect this and it can taint the entire hiring process.
7. Standardize your problem solving interviews
Use a standard set of problem solving interviews across your team in order to maintain objectivity and consistency. Get clear on what constitutes top 10% vs. merely top 50% performance on a given problem, and track candidates’ performance over time to ensure you are well-calibrated. Periodically check Glassdoor to ensure your questions have not inadvertently leaked—this is a frequent problem that can create an obvious bias.
Problem solving interviews are not the be-all, end-all approach to vetting candidates, even in roles with a heavy analytic or problem solving component. They should be viewed as one of many acceptable interview formats, and should be used as a supplement to a deep understanding of your candidate’s unique history.
When misused, problem solving interviews can deliver “noisy” data and can create a bad candidate experience. But when deployed appropriately, they can provide insights into an individual’s intellectual capabilities and thought processes that no other interview format can capture as directly.
Need to train your teams on hiring and interviewing best practices? Visit our website at www.burtonadvisorsllc.com.
Jordan Burton is the founder of Burton Advisors LLC, a boutique talent advisory firm serving a select group of high growth companies and their investors to help them identify, attract, hire and retain top talent at all levels. He was formerly a Partner at ghSMART& Company and a Case Team Leader at Bain & Company, serving clients in the high technology and private equity industries. His website can be found at www.burtonadvisorsllc.com.
To receive blog posts like this one straight in your inbox, subscribe to the blog news.