Save application

castaigroupinc

5 days ago

Senior ML Engineer | Kimchi (Inference Stack Optimization)

Onsite · Austria; France; Germany; Italy; Netherlands; Poland; Spain; United Kingdom Austria; France; Germany; Italy; Netherlands; Poland; Spain; United Kingdom

pythonnodekubernetespytorch technology

greenhouse

Match & cover letter

Create a profile to see your match and get a cover letter for this role.

Create profile and match

Source: greenhouse Verified source

Never pay or share passwords or codes with an employer. How to spot a fake job

AI TL;DR

You will optimize the inference stack to improve throughput and reduce latency for our AI models. This role involves hands-on work with real ML systems and requires a strong technical background in infrastructure.

AI match

Clear role · 80/100 against the market band on this page.

Interview prep for this job

  1. What would you ship in the first month as Senior ML Engineer | Kimchi (Inference Stack Optimization) at castaigroupinc?
  2. Which tradeoff in the description would you ask about first?
  3. How do you explain a production incident to this team?
  4. What would you measure in the first two weeks?
  5. Which part of your last project maps to this stack?

Five questions are free. The rest are Pro. See Pro

Similar roles

Job description

You will optimize the inference stack to improve throughput and reduce latency for our AI models. This role involves hands-on work with real ML systems and requires a strong technical background in infrastructure.

Details

  • Location: Austria, France, Germany, Italy, Netherlands, Poland, Spain, United Kingdom
  • Salary: Competitive, depending on experience
  • Experience: 5+ years in ML systems
  • Skills: Python, Kubernetes, PyTorch, vLLM, SGLang, TensorRT-LLM

The work

  • Optimize throughput through continuous batching and scheduler tuning
  • Reduce latency by profiling and addressing bottlenecks
  • Enhance KV cache performance with advanced caching strategies
  • Minimize cold start times for large checkpoints