Reverse interview

2022-03-04 Permalink

Joel test is rather outdated. In 2022 you’ll be hard pressed to find a company that doesn’t ask candidates to code on an interview. Everybody will use source control and a bug tracker, but not necesserily in the right way.

The following are my updated criteria at evaluating a programmer’s experience at a company:

  1. Can you checkout all your code in one step?
  2. Can you build and test in one step?
  3. Are your build and tests deterministic?
  4. Can you build and test offline?
  5. Is your build warning-free?
  6. Is most of your code written in one statically typed language?
  7. Can your software run without any virtualization layers?
  8. Do you conduct code reviews?
  9. Are your communication channels quiet?
  10. Do new hires tinker with your code on their first day?

These are more stringent than Joel’s. Even a score of 5/10 is good, though you should be aiming for all 10.

Can you checkout all your code in one step?

If your code is spread across multiple repositories, then it is not grepable. When a programmer wants to find a definition of X, or all uses of X, they have no practical way of doing so. This increases surface area, and renders large scale refactorings impossible.

With multiple repositories atomic commits fly out of the window too. The alternative—specifying dependencies between repositories by hand—is a sure way to insanity. Every single change would have to be backward compatible across repository boundaries, whereas testing becomes impractical due to the combinatorial explosion of different version combinations.

Use version control for its intended purpose. A revision hash should uniquely identify the versions of the different components within your system.

This is necessary for ‘bisect’.

Can you build and test in one step?

Automation eliminates the possibility of different people following different procedures or making mistakes. There should be a single source of truth. The developer shouldn’t discover that they ‘broke something for somebody’ because those followed a different procedure.

The ‘one step’ shall apply directly to a clean checkout, not only an incremental build.

This is necessary for ‘bisect’.

Are your build and tests deterministic?

A source code revision shall be in a clear state of being ‘good’ or ‘broken’. A build shall not break because some cloud service is inaccessible, or was updated to a newer version that’s incompatible with the code being built. It shall not fail due to a build tool randomly crashing, or due to build steps executed in the wrong order.

It’s frustrating for the programmer and a waste of their time to rebuild in order to ensure that an obscure error wasn’t a fluke. A failure of a build or a test must be an indication that programmer’s changes are at fault, and that it’s their responsibility to fix them.

This is necessary for ‘bisect’.

Can you build and test offline?

Since deterministic builds preclude any access to the network, you may just as well provide your programmers with the necessary tools installed on their local machines. A programmer should be able to do most of their work while traveling at 900 km/h over the Atlantic, or during a network outage. Moreover, a moderately sized codebase (10 MLOC) should fully build and test overnight on that said programmer’s machine.

Is your build warning-free?

Build and tests should produce a clear binary answer. In particular there should be no ‘warnings’ or ‘lint failures’, as those are quickly learnt to be ignored, thus lowering the signal to noise ratio.

Instead, you should elect specific warnings and coding conventions to be treated as errors, throughout the code, with no exceptions. There shouldn’t be a way to ignore or override those at the commit level, other than fixing the code. If there’s even a slight chance of a violation being triggered by a perfectly valid code, then it shouldn’t be enabled in the first place.

Stylistic conventions (like whitespace, capitalization, etc...) are bad candidates for enforcement. Different parts of your code may call for different formatting; strict rules trade consistency for readability. Additionally, a programmer shouldn’t spend their time on superficial matters like fixing a line that’s just a tad too long.

Is most of your code written in one statically typed language?

Dynamically typed languages are ‘write only’ languages. They provide little tools for programmers to reason about the code, or automatic tools to do even basic sanity checks.

In addition to being statically typed, there should be only one primary programming language in a codebase. The integration between different languages is rarely friction-free, making code reuse across language boundaries problematic. In addition, switching between different tools and languages costs precious brain-cycles.

Can your software run without any virtualization layers?

Virtualization shall be seen as a security measure or a compatibility layer. And it should be left at that. It shall not be a requirement to build, test, or run your software.

It’s only legit to use one virtualization layer per target platform to satisfy items 2 and 3, and only if cross-compilation isn’t an option. If the host platform can easily satisfy the dependencies of your software (by having the necessary packages pre-installed from public repositories), then the overhead and complexity of virtualization is unjustified.

OK: using a Windows VM to build with MSVC from a Unix machine.

Not OK: docker fetch a debian container, use it to build a custom version of emscripten, create a new image with that version installed, and then use it to build other parts of your codebase within that container, all while running postgresql and nginx in their own containers.

Do you conduct code reviews?

Code reviews ensure that there’s more than one programmer who understands the code. Reviewers should understand what changes were made, why they were made, and how they achieve their goal. A superficial ‘compliance with coding style’ check misses the point of reviews. A reviewer shall be ready to take ownership of the code if the original author is unavailable.

If you practice pair-programming, you already pass this item.

Are your communication channels quiet?

While Joel focuses on in-person distractions in his "Do programmers have quiet working conditions?" item, the same principles apply to remote work too.

A programmer shall not receive notifications (through email, chat, calls) about anything that doesn’t require their immediate attention. For example, build success or failure shall notify only the programmers that triggered them. A bug from QA shall only reach the programmer who’s responsible for that component. By keeping a high signal to noise ratio you reduce the chances that something important is missed, as well as the time that critical issues are handled.

For your company to scale, the communication overhead shall tend to O(N) rather than O(N2). E.g. customer support or QA shall report problems through a bug tracker and assign those to team leaders or maintainers of the respective components, rather than mailing the entire developers team.

Do new-hires tinker with your code on their first day?

Joel’s "Do you do hallway usability testing?" item guarantees that new-hires don’t need a lengthy training to understand your code or product. If you answered YES to items 1-4 above, there is no obstacle to assigning them a task as soon as they have it all up and running.

This hands-on approach is an efficient way for them to get started at their role, as well as for you to see how they integrate with the team.