Save application

castaigroupinc

7 days ago

Senior ML Engineer | Kimchi (LLM Inference Optimization)

Onsite · Austria; France; Germany; Italy; Netherlands; Poland; Spain; United Kingdom

pythonnodeawskubernetesgcpazurepytorch technology

greenhouse

Match & cover letter

Create a profile to see your match and get a cover letter for this role.

Create profile and match

Source: greenhouse Verified source

Never pay or share passwords or codes with an employer. How to spot a fake job

AI TL;DR

You will optimize LLM inference performance for customers while leading technical direction at Cast AI. This role requires deep understanding of both machine learning systems and infrastructure.

AI match

Clear role · 80/100 against the market band on this page.

Interview prep for this job

  1. What would you ship in the first month as Senior ML Engineer | Kimchi (LLM Inference Optimization) at castaigroupinc?
  2. Which tradeoff in the description would you ask about first?
  3. How do you explain a production incident to this team?
  4. What would you measure in the first two weeks?
  5. Which part of your last project maps to this stack?

Five questions are free. The rest are Pro. See Pro

Similar roles

Job description

You will optimize LLM inference performance for customers while leading technical direction at Cast AI. This role requires deep understanding of both machine learning systems and infrastructure.

Details

  • Location: Austria, France, Germany, Italy, Netherlands, Poland, Spain, United Kingdom
  • 5+ years of experience in ML systems
  • Skills: Python, vLLM, SGLang, TensorRT-LLM, Kubernetes, AWS, GCP, Azure

The work

  • Push throughput through kernel-level tuning and continuous batching
  • Cut latency by profiling and fixing bottlenecks
  • Optimize KV cache utilization for better throughput
  • Quantize models without quality regression