Save application

Artificial Analysis

20 days ago

Member of Technical Staff (MTS), Language Model Evaluations

Onsite · San Francisco, CA United States

hacker newswho is hiring

hnwhoishiring

Match & cover letter

Create a profile to see your match and get a cover letter for this role.

Create profile and match

Source: hnwhoishiring

Never pay or share passwords or codes with an employer. How to spot a fake job

AI TL;DR

You will work on building evaluation datasets and benchmarks for language models in San Francisco. This role involves hands-on technical tasks to enhance model performance.

AI match

Solid listing · 60/100 against the market band on this page.

Interview prep for this job

  1. What would you ship in the first month as Member of Technical Staff (MTS), Language Model Evaluations at Artificial Analysis?
  2. Which tradeoff in the description would you ask about first?
  3. How do you explain a production incident to this team?
  4. What would you measure in the first two weeks?
  5. Which part of your last project maps to this stack?

Five questions are free. The rest are Pro. See Pro

Similar roles

  • No similar roles on the board yet.

Job description

You will work on building evaluation datasets and benchmarks for language models in San Francisco. This role involves hands-on technical tasks to enhance model performance.

Details

  • San Francisco, work from office

The work

  • Build frontier evals including datasets and benchmarks
  • Develop harnesses for evaluations