Collaboration > evaluation: Why we pay SRE candidates to interview all-day

on

As a team of (mostly) engineers we understand why a growing consensus think the process of hiring for tech is broken. Between us, we’ve interviewed hundreds of times and have our fair share of lacklustre, or downright terrible, interview experiences. When interviewing for SRE it becomes particularly difficult we’re looking for qualities like empathy and teamwork that rarely come across in a typical whiteboard interview.

We try to solve this by having a full day SRE “interview”, where we invite the candidate in to work with us as a colleague for a whole day. We work on real problems, and we pay the candidate for their time (or offer to make an equivalent donation to a charity of their choice). It’s more work (for the candidate and for us) but it’s the best way to find out if we can work together because it’s realistic. It lets our whole SRE team find out what someone is like to work with, and it gives the candidate a complete and honest view of our people, our tech and our processes. What follows is an outline of how we do this at Hosted Graphite, what it tells us about a candidate and how the day feels from an applicant’s perspective.

Regular expressions
Nobody learns anything from a traditional tech interview (source: https://xkcd.com/208/)

At most companies, SRE interviews are made up of a screening call followed by an in-person whiteboard problem-solving interview. These exercises are a useful part of an interview process but don’t always reveal much about how someone will tackle a real world problem. Additionally, from the candidate’s perspective this type of interview doesn’t give them any real insight into what we’re like as a team and what it’s actually like to work here.

That’s why spending a day together as part of the hiring process makes more sense for everybody. It allows us to work on projects that are already in our backlog to see how the candidate approaches real problems and works with the team. The idea is that the code the candidate works on will potentially end up running in production, as opposed to being some throwaway code on a whiteboard. It’s also a more relaxed atmosphere so it helps to reduce normal interview nerves as much as possible. Because we’re working together to solve some real problems, the focus shifts from evaluation to collaboration, which is a big part of our job and very difficult to address in a standard type interview. We don’t just want to see if you can do the job, we want to see if we can do it together.

This approach would seem to make sense from an applicant’s perspective too. Below, one of our SREs Dave Fennell talks about what the day was like for him:

Coming in for my full day interview, I was definitely nervous. Not quite the jitters I’d had at the first interview, but the idea of coming in for a full day’s work with a team of people I’d only met briefly was a little bit daunting. With no experience of the product, my major concern was being able to contribute something meaningful in such a short time. 

The day opened with grabbing coffee and getting to meet the team in a less stressful setting (compared to the previous interview). Following this, I started my first task making some small code changes to an existing project, a nice way to get my feet wet and deal with a relatively confined task with well defined goals. This was a good opportunity for me to see how the team works, as I was working while ’embedded’ in the team the same as everyone else. Although I was lacking some context for the work, it was a good way to see the dynamics of the team first hand and to decide if it was a place where I’d fit in. During that morning, there was an issue with a server that was flagged, talked about and dealt with in quick succession with some pretty solid teamwork – that definitely left me with a positive impression that stuck with me after the end of the day. The morning’s tasks were followed up by a tasty team lunch (in our case company-wide given the numbers) which was good fun and another chance to get to know people and decide if Hosted Graphite was a place I’d like to work.

The final task of the day was a much more general proof of concept, based around the idea of continuous deployment coming from a webhook on source control. At the end of the day, I was pretty surprised to see that my work from the morning was going to end up in production that day, coming from a corporate environment with strict release cycles on the order of weeks, if not months, it was a pretty good feeling and did a lot for improving my confidence.

The day was time consuming and it’s pretty inconvenient to book time off work (although I was compensated by Hosted Graphite). However, the insight into how the company works is incredibly valuable and I’d imagine in some cases could save a lot of time for a candidate who isn’t looking for the kind of team or work environment that we have here. While I came out of other interviews feeling like I didn’t accurately show off my abilities and skills, the full day interview gave a much stronger impression of how I actually work compared to the extreme time pressures and stress of the regular-style interview.

The bottom line is that being judged, and making a judgement about somebody is never going to be easy. It’s harder still when the usual approach to interview gives little real indication of a candidate’s practical skills as an engineer, and how they work on a team. Spending a day working on a real problem together is the best approach for us, and it seems to make sense for the candidate too.

If this sounds like an interesting challenge, why not send an application? We’re hiring.