How to do a good technical interview

(I gave this talk at the FlixTech Summit convention in late June, here's my attempt to transcribe it. The video should be posted to YouTube soon, when it is I'll link to it. Here are my slides.)

Technical interviews are one of the most important and difficult things we do, yet often there's a haphazard approach to conducting them, with predictable uneven results. A few numbers from the Net:

Only 25% of technical interviewees are consistent in their performance.  - interviewing.io

Strong performers mess up technical interviews 22% of the time.  - interviewing.io

Most interviews can only explain 14% of an employee's performance  - Wired 

Obviously we need to improve our technical interviewing!

Interview Goals

The technical interview is one step in the process that usually starts with a CV entering our recruiting pipeline, and hopefully ends with an excellent new team member being hired. It's probably the most important step in this process:

• It gives us the chance to deeply explore the candidate's technical knowledge and maturity
• It allows us to assess the candidate's compatibility with our company's values and culture
• It allows an early assessment of the candidate's career level for the given role
• It also allows us to leave a strong impression of our company behind with the candidate, even if they're not successful (this is really important for building up a long-term image of our organization, helping future referrals and making our recruitment pipeline stronger)

Preparing

A good technical interview is a properly prepared interview. We're all busy with our normal work, and finding 10 or 15 minutes to get ready for a technical interview can seem hard. But it's well-invested time. Some things are quite obvious: study the candidate's CV, make sure you're on time for the interview, make an effort to learn the candidate's name (and figure out how to pronounce it).

Studying the CV can reveal a candidate's potential strengths and weaknesses, and suggest lines of questioning

Plan the questions you're going to ask. Normally you should have a partner for the interview, this makes it easy for one person to take notes, and to observe the candidate closely. Organize the questions, group them by subject matter. It makes a bad impression if you randomly jump from a questions about algorithms to a question about OOP, to a question about Java language fundamentals. Figure out who will ask which questions. I know this takes time, and sometimes I don't manage it myself, but it's important.

Make sure you ask good questions - you learn more, and you give a better impression of your company. What makes for good questions?

• Layer the complexity - start off with something easy, then drill down into the topic. For example, you can ask about the Java Collections API interfaces, then ask about the implementations of Map, then ask how a hash table is implemented (if that's your beer).
• Don't ask trivial questions ("How many bits is a Java short?" Come on, who even uses shorts these days?). A right or wrong answer to a trivial question rarely reveals essential information about a candidate's knowledge. 
• Do ask questions that have real-world relevance for you. What could be better than having a candidate solve a problem you have? We once interviewed a candidate for a front end developer job, at the time we were contemplating migrating from AngularJS 1.x to Angular 2. So we asked the candidate, and learned a ton about what we needed to consider.
• Don't be afraid to teach the interviewee. This has (at least) two possible benefits: the chance to see if a good candidate is also open for learning, and the chance to leave a strong impression of your company with a candidate you won't hire. Once I saw a colleague spend about 20 minutes in an interview trying to help an unsuccessful candidate solve a fairly easy SQL problem. That was kind of extreme (maybe he was trying to test his own teaching ability), but the candidate surely left us with a good impression.

Structure the Interview

An interview should have at least a rough structure, this will help you navigate the interview and make the candidate feel more at ease.

Starting the interview

• Introduce yourselves briefly, maybe sharing a fun fact about yourself (I'm a life master in the American Contract Bridge League, for example)
• Then let the candidate introduce themselves. Let them talk about their job history. Do they remember everything well? Are they convincing about their contribution to past teams? In my experience strong developers remember well projects and products they worked on five years ago (or longer)
• Describe the product(s) your team is working on, the working environment, the cool technologies you're using, how we live Agile principles
• Be enthusiastic about your company!

Warning!

Don't ask anything personal about a candidate: family status, religion, age, whatever. In the EU this would be a violation of privacy laws (and maybe even in the US). We don't want to get sued!

The heart of the interview

It's often a good idea to start off with some easier warm-up questions. This helps the interviewee feel more comfortable (there are no bonus points for making a candidate nervous). Ask some detailed questions about their previous jobs (see Preparing), e.g. "When you built that retail web application, how did you deal with performance issues in the persistence layer?" Let the candidate know what's coming next ("Let's move on to some questions exploring your knowledge of OOP principles"). 

If it turns out after a fair number of questions that the candidate is not going to make it, you can try to shorten the interview somewhat; be careful not to be rude in the process. Finishing the interview after only 30 minutes would likely come across as rude! Remember, it's important that candidates we're not going to hire also leave with a favorable impression of our company, word gets around really quickly about good and bad interview experiences.

On the other hand, if the candidate is very strong, you can start to shift gently to "sell" mode, emphasizing the advantages of joining your team and company.

Wrapping it up

Be sure to leave the candidate time at the end to ask their questions. Did they ask good questions? "How many vacation days do I get?" or "How big is the bonus?" are not good questions, or at least show where someone's primary motivation lies. 

Avoid making an explicit commitment or rejection - you should leave that up to HR, this is what they're paid for.

Don't forget to thank the candidate for their time and wish them a good day (or evening)!

Tips and Techniques 

• Keep a friendly, respectful tone throughout. You don't need to come across like a schoolteacher or play good cop / bad cop (I've seen all that before, you probably have too)
• Make sure someone is taking notes. This helps document any decision, and also helps those in the next round of the process (if any).
• Speak clearly, avoid any jargon associated with your team's domain. You can't expect an applicant to know your DSL.
• Focus on what they've done in their career, not just what they know. I always recall Joel Spolsky's description of a good developer: "Smart, gets things done."
• Don't be swayed by your early reactions to the candidate, whether positive or negative. It's easy to be impressed by someone's personality in the first five minutes and then neglect to notice that there's no depth to their technical knowledge. Or to decide someone's arrogant after five minutes and miss the fact that they were merely nervous (nervousness can easily masquerade as arrogance). Challenge your early assumptions actively!
• Give them feedback, especially to good answers. It shouldn't be a mystery to them how they're doing (but you don't need to give strong negative feedback).
• Don't let the candidate evade answers or even take control of the interview. This can happen easily; interrupt if necessary and bring the interviewee back on track. If they can't be brought on track, that's a very large red flag.
• If they seem to have no technical depth, try to drill down.
• Be aware of the difference between open and closed questions. A closed question can usually be answered with a few words, or yes/no, like, "Do you have experience working with Hibernate?" An open question is more complicated and elicits opinions, like, "What are the advantages and disadvantages of using an ORM framework?" Beware of the candidate who transforms a closed question to an open question!
• Observe the candidate's body language, what does it tell you? For remote interviews: are they Googling? That actually happened to me once in an interview with a working student.
• What's the quality of their questions at the end of the interview (see Wrapping it up)?

Evaluating the Results

It's a good idea if the interviewers independently summarize the strengths and weaknesses of the candidate. How did they react when they didn't know the answer? Were they honest about it, or did they try to fake their way through? How were their soft skills? Sometimes your gut feelings can speak for you. We once had a candidate where the team was strongly in favor of hiring after the technical interview. I had a manager's interview with the candidate, and was irritated by the candidate's habit of responding to every question I asked with, "If you really want to know the truth, then..." I felt like saying, "Damn it, I do want to know the truth!" Of course I didn't say that. There was nothing else obviously wrong with the candidate, but I had a strange feeling. We made an offer, which was accepted - and then three days later the candidate dropped us for another company. Should have listened to my gut feeling!

Simple thumb voting about the candidate usually brings clarity. Any thumbs down vote should be a veto (unless discussion changes that); there needs to be at least one (if not two) strong up votes. It's not worth going after candidates where the consensus opinion is wishy-washy.