Danielle Curry shared this with you through Refery. Danielle works with the company’s hiring team and can put you forward.
Evals Researcher, Rewards and Environments
You would build the evaluation systems that decide which data is worth training on and whether the models and agents are actually getting better.
Sound like you?
Two minutes. Danielle hears the same minute. Nothing goes to the company until you say so.
The company
Post-training and evaluation lab, profitable at four months, San Francisco
The company works on the data and infrastructure problems behind frontier model training and deployed AI agents: post-training research and infrastructure with frontier labs, and models and agents deployed inside very large enterprises and government customers. Under four months in, it is already profitable with seven figures of revenue realized, has taken no priced round, and is a team of ten out of well-known AI companies and frontier-lab teams, hiring several people on each of three founding seats.
- Under 4 months
- old, and already profitable, which is rare for an AI lab
- 10 people
- out of well-known AI companies and frontier-lab teams, planning to double by year end
- 7 figures
- of revenue realized, more contracted, no priced round
- 3 seats
- all founding, several hires on each, founding-level equity
Drafted from the brief with the company, its clients and its investors taken out. The name comes with the first conversation.
Why join
Evaluation is the bottleneck
As agent workflows get longer, judging whether an agent did the job correctly and efficiently is the open problem. Here it is the whole job, tied directly to a training pipeline rather than an offline benchmark.
Join for the upside
Founding-level equity, a bonus tied to the customer deployments your evals serve, and a team that hires for ownership over cash. The research seats carry the higher bar and are treated as the higher-level roles.
What you would do
- Rank training value before compute is spent: work out which tasks in a dataset are worth training on and build a system that ranks every task by expected training value.
- Build scoring that runs constantly and cheaply, tuned to each customer's definition of a good outcome rather than generic correctness.
- Catch drift before it is a problem: notice when real customer requests have moved far enough from the test set that it no longer describes the job, and rebuild the evals to match.
- Design rewards: checkers a model cannot game, model-as-judge and rubric graders, and evaluation of long-horizon agent trajectories.
- Sit at the centre of the post-training loop: decide before compute is spent, and say afterwards whether a change actually helped.
The pay
$150k–$350k base, with equity
The band is set by experience and seniority and is shown on this page. On top of base there is founding-level equity and a bonus tied to customer deployments, and the equity schedule is confirmed with your offer.
Is this you?
Tap what is true. 6 of 7 and it is worth a chat.
Good to know
Things the hiring team told us that are not in the posting.
- Hybrid in San Francisco. The team counts six days of work a week, with three or four of them typically in the office; the rest depends on customer work and travel.
- New York candidates who spend the first three to six months in San Francisco, and senior candidates elsewhere in the US who spend one or two months on site, are considered case by case.
- No new visa petitions right now. OPT with time remaining and H-1B transfers are considered. A strong candidate outside the US may start remotely while relocation is worked out.
- Equity is founding-level for early hires, around 0.5% to 1% at the junior end and more for senior hires, plus a bonus tied to the customer deployment you work on. Health, dental and vision are covered.
- The stack is deliberately not a filter. The team wants systems engineers who can pick up anything.
- Several hires are planned on this seat, not one.
- At least a year of full-time experience is the floor on this seat; one to five years is typical, and a paper is welcome but not required.
- Nice to have: model-as-judge systems, automated graders or rubric design, and eval infrastructure tied directly to a training pipeline.
How they interview
- 1Intro call with the hiring teamThirty minutes. Weekend and evening slots exist if weekdays are hard.
- 2Behavioural and technical screenA few hours of technical work, sometimes with a take-home, then a debrief with the team on what you did.
- 3Work trialA flexible mini work trial of around 48 hours, onsite. When the screen goes very well it is sometimes replaced by meeting the team in person.
- 4References
- 5OfferTwo to three weeks end to end as the team puts it, about four weeks on their written process.
Decision typically inside 21 days of the first call. Refery relays the read after each step.
Refery introduces people to companies through people who already know them. This page is not indexed or listed; the company’s name comes with the first conversation. Questions: lily@refery.io.