Coding Interviews are Stupid

But the recruiting pipeline situation at fortune 500 companies is acutally much more dire than that. Not only are coding interviews for top science and engineering jobs stupid, they actively weed out your smartest candidates.

Let me explain, no there is too much, set me sum things up...

First, a bit of context. I have a Ph.D. from a top university in Robotics that was seven years and $200,000 in the making. For the past fifteen years I have been working as a Robotics Research Scientist at some of the World's top robotics companies and research institutes. I have always had a policy that I talk to all recruiters from major companies and I pretty much always say yes to interviews. I do this because I want to stay up on what roles are available and because I want to keep my interview skills in top gear for when a better job comes down the pipe. All that to say, I have probably done 50 interviews in the past 15 years for pretty much every robotics company you can think of.

At least 90% of these interview processes have a coding interview or code screen component. The format of that coding skill evaluation is always the same, an interviewer spends about 45 minutes asking you to solve toy coding problems. Things like, how can you sort this array of characters? How can you find the minimum cost path on this toy occupancy map of ints? How can you find the longest set of ones in this bool array after removing exactly one? And on and on. There are a number of websites dedicated to inflicting this process on as many job applicants as possible like Hacker Rank, Leet Code, Coder Pad, and so on.

I have been told many many times that the company is not going to extend an offer at this time because I did not pass the coding interview or that the role requires more coding experience. Rejection wouldn't be so bad other than the waste of time, but there is a problem. I have been writing C++ software at the highest possible proficiency for 15 years. If you have an iRobot product, then chances are some of my code is operating in your house right now. I have programmed for every target architecture micro-controllers from 8bit to 32bit, SoCs, FPGAs, Intel, AMD, Nvidia Cuda, ... Essentially, if it can be programmed, I can program it. Hard stop. Any language and any architecture.

Here is another huge problem with this recruiting paradigm. I am not a software engineer. I have a Ph.D. in electrical engineering, neuroscience, and mathematics. I am a scientist at the highest level. Why does it matter if I can write a bubble sort algorithm in 30 minutes in some web interface with a nearly asleep interviewer? Here's the thing. Anyone can solve toy coding problems in any language with no experience at all if they have a web browser, search engine, and patients. I would estimate any average high-school senior could solve the coding test at any top company. The only variable is how long will it take them. If you've done it a bunch of times by practicing on the code interview websites, you can probably do it in ten minutes and if you have never done it before it could take you 8 hours, but pretty much any human of average intelligence can solve these toy coding problems in less than a day even if they are complete novices with no advanced training.

So why are we weeding out scientists because they don't do toy coding problems fast enough? You are not hiring scientists to solve toy coding problems. In fact you don't even hire software engineers to solve toy problems. No one needs to solve toy problems ever, unless they find it fun. There is no need or value to that skill whatsoever. You hire scientists and engineers because you have a problem that has never been solved and you need their advanced training to bring skills of physics, mathematics, systems theory, ... to do something that has never been done or to do it in a way that has never been done.

Now don't get me wrong, I'm not saying it is easy to evaluate potential candidates. Far from it. I have been on the hiring and interviewing committees for hundreds of positions and honestly sometimes we get a productive new hire and sometimes we get a dud. The dud ratio is not insignificant. I have certainly hired candidates and then regretted it once I saw that they just don't get work done.

This process has another unintended consequence of selecting for toy problem masters, which are really just people who spent a thousand hours on leetcode.com solving toy problems. The problem is that does not give them advanced training of any kind or qualify them in any way to solve hard problems.

So what is a better way? Well first of all, do not give toy coding tests and certainly don't give really hard coding problems as homework as part of an interview process. The best method of interviewing I've found is to simply have an open ended conversation with the candidate for the time allotted. Letting the conversation go where it will asking about all the relevant skills and experience. I've found this format naturally evolves as a breadth first search until a topic of interest is found and then proceeds as a depth search on those topics. Probably the only reason this technique is not common in interview pipelines is if you don't ask all the candidates the same questions, it can be hard to compare their answers so you can sort them based on some heuristics. It also requires that the interviewer has the same level of experience and skill as the position they are trying to fill, which is often not the case.

So what is a better way? Select the smartest person as the team manager and only the team manager needs to interview candidates. Sit them down for an hour or two and discuss the various skills required seeing how familiar they are with the skills and getting project details for prior similar work they've done or what familiarity they have with the theory when they haven't. And that is massively important. The reason you want to hire smart scientists and engineers with formal university level training is that they are able to work on hard problems that they have never seen before and that is what you want. You want to select for candidates with top level specialized training, relevant work history, passion for the problems at hand, deep theoretical familiarity, and the ability to adapt themselves to solve fundamentally difficult tasks. Honestly people who do well on toy coding exercises should probably be excluded because they spent time learning toy coding exercises when they should have been studying deep scientific principles.