There's no better feeling than building something open source and watching it take off. Nine months ago, I built a simple hiring agent to solve one very real problem.
Things it is not: It's not an ATS. We don't use it to screen our open roles. Our customers don't use it either.
Here's what it is: Every year at HackerRank, we get 50,000 to 60,000 intern applications. No human can read that many resumes well. So I built something to rank them, helping me decide which resumes to read first.
[This was before we built AI Interviewer (Chakra) to automate the first round of interviews, so candidates are no longer rejected based on their resumes alone.]
Two things worth clarifying since I've seen them come up in this thread:
The default model is gemma3:4b because it's what runs locally on most laptops - no cloud API needed. Actual resumes are evaluated using a top Gemini model. The repo ships with a demo config, not the production one.
The cutoff score was set very low — the system was designed to rank resumes, not reject them. Only resumes at the very bottom of the distribution were filtered out. The vast majority passed through to human review, where the real decisions were made.
Over the last week, it's taken on a life of its own. People are cloning it, running their own resumes through it, opening issues, sending PRs.
I contributed to open source a lot in college. Somewhere along the way, I drifted away from it. This week reminded me how good that feeling is. This thread has also given me more ideas than I expected. The critiques here are sharp and I'm already thinking about how to act on them. Improvements are coming.
You know you're not writing for LinkedIn? So platitudes about drifting away, watching your project "succeed" by being really popular, is not relevant to the main concerns pushed by this piece. Particularly brushing off the non deterministic score calculation.
I'm a bit disappointed to see "The critiques here are sharp", a Claude tell, in a response which (to me) is trying to subtly argue that hackerrank is not overly reliant on LLMs.
I'm not sure if your intent was to come across as having written this yourself, but it did not have the effect of improving my perception that this approach is flawed.
I was also disappointed that you didn't address the variability in scores. I'm inferring that you believe the larger model takes care of the main observation in the post, but I don't really see you directly addressing the points.
There is variability in scores and that's expected given we are eventually using a LLM to score. At least, when I used it 7 months ago, the only way I could avoid it was by keeping the cutoff score low (as low as 10 or 20).
Reading this thread, I'm hoping to minimize the variability even further (even though I know it can't be fully removed).
Saw this comment at the top with 0 replies and thought “How is that possible??” and then saw the “0 minutes ago” timestamp. Only on HN can you stumble into the comments section just moments after a CTO, founder, author, etc. left unfiltered remarks about the exact topic of the post. Never change HN.
You have to write code at some point, but you also have to think about what you are doing. Especially in time-constraints environments, there is a big risk doing a lot of code which ends up totally useless because it was not well thought, and did not do what was intended, etc...
More important than unit test and all that trivial stuff, what matters a lot in my experience is the ability to write for changes. That's really what's difficult about most programming tasks: how to deal with changing specs, change of deployment environments, etc...
Don't agree on Unit test being trivial. Infact, it is one of the things that aid you with changing specs and code. It helps you change your code with confidence. That really matters.
I appreciate your overall point, had just one correction. thanks.
I should have explained what I mean by trivial: I think any competent programmer knows the value of testing. So in that sense saying that you should write tests is like saying you should write good code - it is true, but not very insightful.
In a time constraint environment, you often need to make a choice between more tests and more features, etc... I have seen a lot of people who said their code was great because it was well tested, but then the tests depend so much on the implementation that it actually goes against better code (since refactoring may mean breaking a lot of unit tests). So the more interesting question is: what to test, how, and how far. For example, my own rule for testing is that in general, I don't test something if testing it takes more code than the feature itself (except for regression testing, and things which really play well with testing, like parsers and the likes, and code which needs to run on many different platforms).
It is also useful to recognize when testing has value, and to avoid overestimating its actual benefits. Too often, people don't really think about failures when testing, or how the API it exercices can be used in a straightforward manner or not. For example, I don't get so much the value of TDD - I find it much more useful to think in terms of API usecases, for which small tests are often not well suited. Sure, designing for easy testing can help, but you can get the same benefit without TDD in my opinion.
Finally, too often, I see "this library is well tested" sold as a feature in open source projects - but test coverage is not a feature. Sure, everything else being equal, I would prefer a well tested library to a not well tested library. But everything else is not equal: there is a constant tradeoff between designing better API, getting a better documentation or installation process and testing.
There's no better feeling than building something open source and watching it take off. Nine months ago, I built a simple hiring agent to solve one very real problem.
Things it is not: It's not an ATS. We don't use it to screen our open roles. Our customers don't use it either.
Here's what it is: Every year at HackerRank, we get 50,000 to 60,000 intern applications. No human can read that many resumes well. So I built something to rank them, helping me decide which resumes to read first.
[This was before we built AI Interviewer (Chakra) to automate the first round of interviews, so candidates are no longer rejected based on their resumes alone.]
Two things worth clarifying since I've seen them come up in this thread:
The default model is gemma3:4b because it's what runs locally on most laptops - no cloud API needed. Actual resumes are evaluated using a top Gemini model. The repo ships with a demo config, not the production one.
The cutoff score was set very low — the system was designed to rank resumes, not reject them. Only resumes at the very bottom of the distribution were filtered out. The vast majority passed through to human review, where the real decisions were made.
Over the last week, it's taken on a life of its own. People are cloning it, running their own resumes through it, opening issues, sending PRs.
I contributed to open source a lot in college. Somewhere along the way, I drifted away from it. This week reminded me how good that feeling is. This thread has also given me more ideas than I expected. The critiques here are sharp and I'm already thinking about how to act on them. Improvements are coming.