Designing the perfect programming test
If you haven’t already, read my previous article on the subject. As follow up from the aforementioned article, you may not need a test. But if you were, this article is a guide as to what that may look like, and how you may design it.
The PR review
There are so many skills required to review a PR effectively, not just technical know how, but also soft skills. You need to know how to phrase your comments, and come across clearly. You need to know when to pick your battles or not, what’s worth commenting on, or not.
These are all typically things you won’t get out of someone in a standard programming test. Generally you’re given a task that a dev guru could pass with flying colours, but would that dev guru ask if you’re okay if you came into work teary eyed? Working with teammates isn’t just based around your technical skills, but your soft skills too.
Savants may solve any complex problem you throw at them, but does anyone want to work with them?
To pull off the PR review test, we need to create a small codebase which has a few glaring problems, and a few more that a dev is introducing. The task for the applicant (person we’re interviewing) is to identify the problems that this new dev is introducing. We’re also going to be testing not just on identifying the problems, but on what they propose as a solution.
The beauty of this test is that you can configure it to whatever role you’re wanting. There’s no dependency on programming language, or problems. If you want to test an applicant on reusability then have the PR involve a few instances of repeated code. If you want to test on clean architecture, then have the PR involve a few wonky solutions to the structure of the app.
Regardless this will test the developer’s ability to read through a codebase, and start to understand how things are laid out. Not just writing code, but reading, and understanding it are crucial too.
The Pomodoro timer (Verbal programming)
Firstly, thanks to my friend, and coworker Tony who introduced me to this as an interviewing test.
As a bit of background, the Pomodoro technique is a way to structure the way you work. You work in intervals, and take breaks in-between. Here’s the wiki if you want to read more.
This programming test is fully verbal, no code needing to be written, although you could use a whiteboard if you wanted. To administer this test you provide background to what the Pomodoro technique is, as I have above, then ask the applicant how they would go about implementing their own version of it.
The beauty of this test is that you can solve it across any programming language, frontend, or backend. The versatility of its use is one of it’s main pros.
Imagine a scenario where you have a React frontend developer (hello it’s me) and you ask them to start talking about implementing it. You can begin by saying “How would you build the pomodoro timer?”. As they continue describing their implementation you can start to pepper in more questions, or hold off if you feel like they’re struggling.
“How would you store the countdown timer?”
“How would you implement pausing the timer?”
“How could you make it so other people can see whether you’re on a break, or working so people know not to disturb you?”
There’s so many lines of questioning you can go down. You can be more specific, or less depending how well the applicant is answering questions.
The hands-off test
The hands-off test utilises a developer’s interpretation of loose requirements. Rather than just looking at the ‘desired solution’ you likely have on-hand for a generic programming test, this provides a way for the applicant’s creativity to come through.
At the end of the day, some companies will just want to see the applicant write some code. So how can we get an applicant to undertake a test within the least amount of time, to show off the most about their way of working?
Easy. Don’t provide hard technical requirements, and tell them to not take longer than an hour on it.
For example, provide a loose list of points for the developer to hit. It could be something like
- Create an app
- It should query for data (from an external source)
- Render that data in a scrollable list
The beauty of this test is that we’re letting the developer take any route they’re comfortable with to get the desired result. There’s no it has to be X way, and look Y way, just do whatever you’re happy with. Although it’s a test, you’re most likely to get code close to how the developer authentically writes code.
To further customise this test to emphasise it being comfortable, and approachable. You could provide a list of possible loose technical requirements and tell the developer to reach an amount out of the total, e.g. meet three out of the five requirements.
Provided the test has gone well, you then get the chance to ask the developer about why they did things in a certain way. And get to see their creativity, and personality flare in how they went about the test.
End
At the end of the day, no test is perfect, we can only aim for perfection. Feedback is critical, ask the developers who take the test what they thought about it.
Get your own developers to take the test, see how they handle it. Was it too easy? Too hard? Confusing? Ambiguous?
What do you think of these approaches?
What’s the best programming test you’ve done?
What would make you not take a programming test?
