On Tech Challenges
Coding Challenge
The company I am with, Offgrid-Electric, has evolved our hiring process over time to try to ensure we can find a diverse and strong distributed team. There are many interesting processes related to hiring a distributed team, but I will quickly focus on the coding challenge, technical challenge, or whatever folks are calling it these days.
image by FotografieLink
Offer Challenge Options
This post is kicked off by the thread below.
We accept any that the applicant is most comfortable with... Share existing code and walk us through what was done, pairing, code exercise take home, or tech challenge live on a call. I find some people are more comfortable with one method or another so we try to do anything
— Dan Mayer (@danmayer) December 18, 2017
Folks followed up asking for more details on other challenge options we provide and why, which I will try to breakdown below.
What is the point?
First off why have a coding exercise in person, take-home, or other? The goal is to allow a candidate to demonstrate skills beyond being great to talk with during interviews. That they can back up the conversation with problem-solving skills that match the team’s expectations. There are far too many posts and stories joking about how few developers can code, which I will skip over, but often a team wants a way to help ensure that a candidate has skills that will make them successful if they join. At OGE we try to allow for the widest possible options to prove out that a candidate would be technically qualified or the position.
Tech Challenges Options
We offered a few standard options and have added more and evolved our challenges over time.
Take Home Challenge: We have someone on our team develop a challenge for any job description we are hiring for. The challenge is prepared before we even post a job opening. In general, we have someone on the team develop the challenge and complete it themselves to ensure they fully understand the challenge and have a good estimate of the time required to complete it (generally 2-4 hrs). This falls somewhere into the category of building a small application that does X, Y, and Z given this sample data set. Given folks likely have a current job we give plenty of time to work on the challenge and include instructions for how to submit the challenge and things we expect to see.
Pair Programming: We had some folks talk to us that enjoy pair programming and find it a good way to understand the way our current team works and thinks. We offer to pair on the take-home challenge and work on it with the candidate at a scheduled time, or to work on some of the code we are currently working on, or to pair on some code that the candidate has recently been working on. Some folks are very comfortable with this kind of exercise while others get very nervous pairing with someone they are just getting to know. Some folks have paired a lot in the past, or it might be a fairly new experience for them. Unless your team practices 100% pair programming, I believe you should offer some alternatives to just pairing. I like having a take-home challenge that can also be offered as a pairing exercise.
Open Source Work: This is a great option for folks that already have a lot of publicly available code they are proud of and showcases their skills. There is no reason to make someone spend extra hours of their time when you could have them walk you through code they are passionate about and have already been working on. I have enjoyed learning about some candidates projects even if it wasn’t a great fit and the enthusiasm for something they have recently worked on in their own time shows through. It is important to have other options beyond just open source. While this is a great alternative, it can’t be the primary or expected way to check off the technical challenge. You can find more detailed reasoning about this on various diversity hiring posts, but not everyone has the same privilege to spend extra time outside of work on open source, some jobs don’t allow it, it isn’t everyone’s cup of tea.
Technical Questioning: While I prefer to do something with code like the options listed above, sometimes that isn’t the best fit for whatever reason. In these cases, we have done some technical white boarding type interviews over video chat. In my interview, at OGE, I walked through how to split out a monolith application into smaller component applications and discussed the kind of DevOps you need to support such transitions. This format has also worked well for QA interviews. I try to avoid trick questions / brain teasers. Asking data modeling questions tends to work pretty well over a whiteboard for an example of good technical questions.
Choose your own Adventure
Taking this even further, given our only goal with this part of the interview process is to come away with some confidence that the person is technically capable in a set of skills… I often ask the candidate after presenting our standard challenge options listed above if that have any other way they would like to prove out technical skills. I will list some things candidates have suggested or done before.
Technical Writing / Educational Materials: We have had some folks apply that have created online materials for courses. Sharing video series that teach programming concepts, blog posts detailing how to use some technology, or videos of talks given at a conference covering technically relevant material.
School Projects / Papers: If you are talking to someone recently coming out of university they might have some great projects they could share with you. If it is a group project just get into specifics about what pieces they directly contributed. We recently had a candidate submit their Ph.D. thesis as a way to confirm their technical skill.
Code Reading / Walkthrough: A few candidates that didn’t have open source code but as consultants had learned to quickly look at projects and distill how they worked. In this case, they were also curious to see how our systems worked. After some discussion, we walked through sections of our companies source code with the candidate reading the code and explaining what it did and the recommendations they would have to improve that section of code. While this is a bit more challenging on the interviewer’s side, it can give candidates a great picture of the kind of code and the quality of the project they will be getting involved in. If the person can quickly jump in and absorb parts of the project’s domain and explain it back to you, it can give you the confidence you need to move forward.
Comfortable Candidates Can Share Their Skills
In the end, it comes down to just making sure you set a candidate up to show you their best work. You don’t want to setup a situation that will have a great candidate so nervous they don’t show what they are capable of. Being a globally distributed team, in the end, I don’t expect all our team to work the same way, so I wouldn’t expect all candidates to succeed in the same style of challenges. You can always find a developer that loves to pair or hates it, or gets very nervous working out a problem on a whiteboard or could lecture in front of a whiteboard for hours on end. All of that is fine, let em shine.