So what is the best technical interview in my view? That honour goes to two game developers Avalanche Studios and Dovetail Games (others may also use similar test but these are the 2 I know of) whose tests go something along the lines of:

1. Send candidate code for a retro game clone, e.g. Pacman or Asteroids.
2. Tell them to get it running on their machine and fix any bugs encountered.
3. a) If interested in creative coding: improve the game in any way they see fit, e.g. new features, enhanced movement, etc.
   b) If more interested in core tech: Profile the code, then make it run 20% faster (10% is fairly easy, 20% shows skill, more than 20% shows real mastery)
4. Send the result back

The reasons I find these tests superior to all the other technical tests I’ve heard of are as follows:

• The tests are pragmatic.
• The tests are real.

Does it matter if a programmer has memorised the technical standards for every programming language invented? Does it matter if a programmer can recite the Gang of Four or every bit trick under the sun? How is that programmer actually going to work with your code base? What will they add? Will they make a mess of things? Will they have the ability to work unguided? Will they respect the architecture and coding standard? Will they know when is the right time to throw the coding standard out of the window?

There is a place for developers who can fathom ideas never undertaken before but that is not the norm of development. Innovation rarely comes in bounding leaps, it usually comes from careful iteration on a theme. For those reasons, I find this type of pragmatic, real world approach to tests by far the most useful way of determining whether someone will be up to the technical challenge of working in an existing team.

In my previous post I talked about how there are still some very bad interview processes being used by top firms when hiring developers. In the last two weeks I have seen many of my former colleagues going for interviews with a wide array of firms, including some of the best technology and financial companies in the world. This has given me a great opportunity to compare and contrast the hiring process in its current state and has led me to two further observations.

The first is quite obvious – the range and quality of hiring processes is as varied as there are firms. Some are very old fashioned, some are incredibly fastidious, while occasionally, a firm will surprise me by creating a programming test that is both novel and directly attributable to the kind of real world work one of their developers might be expected to do on a daily basis. Congratulations to those companies for taking the initiative – everyone else, you’ve got some competitor research to do if you don’t want to lose out in the hiring game. Remember the candidate is interviewing you as much as you are interviewing them.

The second observation is one that starts to show my age. Some of the tests I’m seeing  over my colleagues’ shoulders are very familiar indeed. In fact one was almost an exact facsimile of a programming test I sat over 10 years ago. Another of the tests – one that stood out from the crowd for its pragmatic approach – was almost identical to a test a friend once described to me he’d sat in Sweden. A quick dig around the wonders of social networks informed me that in both of these cases, I could directly trace someone I’ve previously worked with to these firms at some point in the last 3 years. I can only assume they took their coding tests with them when they moved, ensuring that the lineage lives on. Just please, don’t take the bad tests with you when you move on and, whatever you do, don’t let them get into the cloud else we’ll never be rid of them!

I’ve been doing a round of interviews recently, trying to replace a contract that has recently come to an end and I’ve come to a conclusion – I’m really good at my job but I’m terrible at interviews!

Well, not all interviews. There are some interviews I’m great at. The ones where a real conversation can be had about a diverse range of topics and in which the answers can have some nuance to them. The kind of interviews where I feel I have come out knowing a bit more than I went in with regardless of which side of the table I’m sat. I like those interviews.

I hear you walk good. Let me tie your shoelaces together, blindfold you and spin you round a few times. Now show me your walk.

The kind of interviews I’m terrible at all have a very particular rhythm to them. The meeting starts, pleasantries are exchanged (mostly, although some technical interviewers I’ve met don’t even give their names) and everyone settles into their chairs. A sheet of paper and a pen are slid across the table. My heart sinks, this is going to be one of *those* interviews which have the very black and white questions.

Am I perhaps putting pride before a fall? Shouldn’t a candidate expect to jump through some hoops to get the job? Isn’t this all just good due diligence? Don’t most programming problems come down to boolean logic and therefore the answers can be deemed correct or not? Well no, I suspect that the only interview questions with objectively correct answer are the cases which are trivial. Those questions annoy me.

Question – “In C++, what is wrong with this reverse function?”

Answer – “It’s not std::reverse?”

“No!”

Answer – “It’s in the global namespace?”

Correct Answer – “No! There is no test associated with it and there are loads of bugs!”

More Correct Answer – “Why did someone write all this codes and all these tests to bloat our code base with a worse version of the C++ Standard Library? Who do we employ who is better than Howard Hinnant and why aren’t they contributing pull requests to llvm?”

Question – “Write an algorithm to count all the pairs in an array, matching a predicate, the 2 test sets are 5 elements long… with permanent pen and a single side of A4!”

Answer – “Er… vaguely this? I usually hit F7 about now and run some tests while I have a little think about the wider structure (or it’s already been gulped).”

Correct Answer – “No! There’s a bug! And it uses O(n^2) complexity!”

Question – “How would you optimise it?”

Answer – “Er… does this code really need optimising? What’s the real world usage of this algorithm? How often does it get run? Is this ever going to have more than 5 values passed into it? If this is all we’re doing can’t we just go perfectly wide over all cores for every combination and accumulate to an atomic variable? Can I just Map-Reduce it? Use a GPGPU?”

Correct Answer – “No! This was a set of unique numbers! You could have optimised for that!”

“Er… nothing in the brief said this was guaranteed to be a set… why isn’t it sorted? Is this really all you ask people to find out if they’re worth giving a job to? Where are the architectural system design questions? How much did you say the hourly rate for this position was?”

Don’t you have infinite monkeys somewhere that can do this?

The older I get, the more these trivial pen and paper questions irritate me. I don’t go as far as to say all these things in the interview but I probably don’t hide my inner monologue very well.  I have had a fair number of jobs now and hired a fair few people myself, yet not a single one of those jobs started with someone giving me a permanent pen and paper coding test.

My Dad used to write code on punch cards. To compile it he had to catch a train to London. I write more lines of code per day than my Dad did.

We are all standing on the shoulders of giants. Alan Turing gave the world its greatest tool since the wheel and it is the height of arrogance to consider that any code is written in a bubble. If you are not assessing the capability of  your developers to write code in a meaningful environment, in context to other code and the data it transforms, then you are failing to do your due diligence and opening yourself up to immense risk.

Hiring is a two way process with the candidate trying to assess the quality of the development team just as much as the interviewer is trying to assess the quality of the candidate. The goal of an interview is to assess the depth and breadth of a candidate’s understanding and ability to apply principles in practice, not to find out how much someone enjoys toy puzzles.

So, hiring managers, I have one thing to ask. Next time you take the effort to look at a CV,  check the references and talk to a candidate on the phone. Don’t then call them in for a pen and paper test of the most trivial nature and waste their time and yours. And even if you change nothing else about your tests, at the very least give the candidate a text editor and a delete key and assess whether they can even type!