"work on the weekend for free" aka move the risk (of wasted resources) to the employee. Excellent advice.
How about fostering a working culture were employees get a chance of trying out new things during office hours? Especially since AI is rising productivity so much.
I've been considering very unconventional techniques for interviews. Like requiring candidates participate in a team based multiplayer game as part of the interview process. Completely out of left field without warning.
Assuming the applicant has zero experience with gaming, this would be an incredible opportunity to evaluate how well they cope with uncertainty under pressure and adapt to unusual conditions. If the applicant is already a grandmaster on the ladder, this becomes an excellent test of their humility and teamwork.
If the applicant gets shitty about the mere idea of playing a game as part of a "serious" job interview, I think I would have all the information I need to make the right decision. If you can't get a little bit flexible in this new world, I don't want you on my team anymore.
What AI has done is that we can do 10x more work a week, while only delivering 2x to production.
Right now, there are 4 people (and me) working on the same idea in 5 different ways - we're all training small language models, the goal is to get smaller or fewer active parameters.
By next week, we'll have lost on 3 different approaches. I know my approach is a dead-end already, but I'm not ready to give up till the pencils down.
The people I work with are great, because they will see the better solution by the end of it, whether they proposed it or someone else did.
A couple of them have been in the industry for less than 5 years and I feel like their slope is better than mine, which means I'm going to have a good time when they're fully up to speed :)
In the new world, normalised interviews are going away, because business value is not normalized. This means that domain based interviewing and signals are going to make a comeback.
It would make sense to build your career in a single domain, since your built up expertize will transfer to other companies within that domain.
Far fewer switches will happen for seniors across domains.
I choose to argue the complete inverse. In the age of AI there is increasing value on being a reasonably well rounded generalist. Exposure to different paradigms and possibilities matters significantly more than depth within a single purview.
From a career risk perspective, this is also the right answer. Domains disappear, though they frequently morph. Cross domain knowledge allows the individual to apply lessons and patterns they've learned elsewhere.
The heart of the coding interview is to get signal on a candidates’ ability to solve a problem without all of the inputs given to them, see how they and explain their thought process as they do it.
Because of AI, what it means to “not have all of the inputs” has changed as anyone can cut and paste a Leetcode problem to a chatbot and solve it.
I think we are going to iterate a bit to figure out a better signal than the typical coding interview. For junior engineers, I think coding questions will move towards MBA style problems when someone is given more of an open ended question with a codebase and you see what the candidate does without context to reduce the problem.
I used to have a stack of books with references notes on my desk. The internet came and I could search and even stack-overflow improved this. Doing a whiteboard riddle from rote knowledge is something I always fail. That particular solution I maybe used it once or twice in my 30 years as SWE. Why would I remember this. I do remember where I can find the solution for it.
Find an easy to understand product, like a step above a todo list app. Share a spec and have them code it in their own IDE using LLM. Focus on speed. Then have them extended it, adding some features. Then ask them to show you where certain functionality exists in the code and to explain the architecture of how it is built. Ask them where it is weak and could be improved.
That's it, that's the interview. SWEs are getting quite good at the prompting, but fast navigation and understanding of the code is the key.
Anyone that can effectively translate a spec to code with minimal handholding shows product awareness. Anyone who can produce a working app can prompt AI reasonably. Anyone who can revise their design on the fly to support new features is someone with flexible thinking and some humility. Anyone who can navigate and explain what AI built is someone that understands how to build a mental map of a codebase. Anyone who can do all of this in an interview is someone who can handle pressure, work quickly and communicate.
You are welcome to keep trotting out linked list traversals and sliding window problems if you like, but leetcode is now (and maybe always has been) about culture fit more than an ability to build software.
The way you describe your ideal interview is how I conducted leet code style interviews. Ask a problem with a simple algorithm and then ask extensions until the time ran out. This worked pretty well.
For interviews where candidates are using AI I think you can supercharge this approach, but I do not think there is value in turning it into a code navigation puzzle as there is little value in that can be evaluated in the ~45 min you have for coding in a 1 hr interview slot. Ultimately in many software engineering jobs these days we need to understand how to build quality systems without looking into the underlying system except as necessary. I am not sure how to best evaluate this judgement in an AI assisted interview
"work on the weekend for free" aka move the risk (of wasted resources) to the employee. Excellent advice.
How about fostering a working culture were employees get a chance of trying out new things during office hours? Especially since AI is rising productivity so much.
I've been considering very unconventional techniques for interviews. Like requiring candidates participate in a team based multiplayer game as part of the interview process. Completely out of left field without warning.
Assuming the applicant has zero experience with gaming, this would be an incredible opportunity to evaluate how well they cope with uncertainty under pressure and adapt to unusual conditions. If the applicant is already a grandmaster on the ladder, this becomes an excellent test of their humility and teamwork.
If the applicant gets shitty about the mere idea of playing a game as part of a "serious" job interview, I think I would have all the information I need to make the right decision. If you can't get a little bit flexible in this new world, I don't want you on my team anymore.
Why a multiplayer game when you can do the same test in a nicer environment at a local golf course?
This Ousterhout lecture lives in my brain (this one and the one about scar tissue).
"A Little Bit of Slope Makes Up For a Lot of Y-Intercept"
https://www.youtube.com/watch?v=mE98Is1quwY
> Pick a radical idea
> Build it over a weekend:
What AI has done is that we can do 10x more work a week, while only delivering 2x to production.
Right now, there are 4 people (and me) working on the same idea in 5 different ways - we're all training small language models, the goal is to get smaller or fewer active parameters.
By next week, we'll have lost on 3 different approaches. I know my approach is a dead-end already, but I'm not ready to give up till the pencils down.
The people I work with are great, because they will see the better solution by the end of it, whether they proposed it or someone else did.
A couple of them have been in the industry for less than 5 years and I feel like their slope is better than mine, which means I'm going to have a good time when they're fully up to speed :)
Thank you for that link. I started feeling rusty and needed something to shake off.
On a side note, I did founded my startup this year, and had that bug of doubt back of my head, but oh well…
In the new world, normalised interviews are going away, because business value is not normalized. This means that domain based interviewing and signals are going to make a comeback.
It would make sense to build your career in a single domain, since your built up expertize will transfer to other companies within that domain.
Far fewer switches will happen for seniors across domains.
I choose to argue the complete inverse. In the age of AI there is increasing value on being a reasonably well rounded generalist. Exposure to different paradigms and possibilities matters significantly more than depth within a single purview.
From a career risk perspective, this is also the right answer. Domains disappear, though they frequently morph. Cross domain knowledge allows the individual to apply lessons and patterns they've learned elsewhere.
The heart of the coding interview is to get signal on a candidates’ ability to solve a problem without all of the inputs given to them, see how they and explain their thought process as they do it.
Because of AI, what it means to “not have all of the inputs” has changed as anyone can cut and paste a Leetcode problem to a chatbot and solve it.
I think we are going to iterate a bit to figure out a better signal than the typical coding interview. For junior engineers, I think coding questions will move towards MBA style problems when someone is given more of an open ended question with a codebase and you see what the candidate does without context to reduce the problem.
I used to have a stack of books with references notes on my desk. The internet came and I could search and even stack-overflow improved this. Doing a whiteboard riddle from rote knowledge is something I always fail. That particular solution I maybe used it once or twice in my 30 years as SWE. Why would I remember this. I do remember where I can find the solution for it.
While the approach makes sense, I have yet to see a quantifiable and normalised way to replace Leetcode interviews.
It’s probably too soon to claim “death of the coding interview”.
I have, and it is surprisingly straightforward.
Find an easy to understand product, like a step above a todo list app. Share a spec and have them code it in their own IDE using LLM. Focus on speed. Then have them extended it, adding some features. Then ask them to show you where certain functionality exists in the code and to explain the architecture of how it is built. Ask them where it is weak and could be improved.
That's it, that's the interview. SWEs are getting quite good at the prompting, but fast navigation and understanding of the code is the key.
Anyone that can effectively translate a spec to code with minimal handholding shows product awareness. Anyone who can produce a working app can prompt AI reasonably. Anyone who can revise their design on the fly to support new features is someone with flexible thinking and some humility. Anyone who can navigate and explain what AI built is someone that understands how to build a mental map of a codebase. Anyone who can do all of this in an interview is someone who can handle pressure, work quickly and communicate.
You are welcome to keep trotting out linked list traversals and sliding window problems if you like, but leetcode is now (and maybe always has been) about culture fit more than an ability to build software.
The way you describe your ideal interview is how I conducted leet code style interviews. Ask a problem with a simple algorithm and then ask extensions until the time ran out. This worked pretty well.
For interviews where candidates are using AI I think you can supercharge this approach, but I do not think there is value in turning it into a code navigation puzzle as there is little value in that can be evaluated in the ~45 min you have for coding in a 1 hr interview slot. Ultimately in many software engineering jobs these days we need to understand how to build quality systems without looking into the underlying system except as necessary. I am not sure how to best evaluate this judgement in an AI assisted interview
Ai assisted interviews are pretty good and can replace leetcode parroting pretty well.