Senior MLOps Engineer
We are seeking a Senior MLOps Engineer to steer the technical vision of our Training and Inference Optimization team. In this high-impact role, you will architect the infrastructure that powers our next-generation AI models. You will bridge the gap between systems programming and machine learning, optimizing large-scale LLM training via NVIDIA NeMo and building ultra high-throughput serving systems using vLLM, TensorRT-LLM, and SGLang.
Your mission is to ensure our models are not only state-of-the-art but also production hardened, cost-efficient, and performant at scale.
Key Responsibilities
- Training Infrastructure: Architect and maintain scalable distributed training pipelines using NVIDIA NeMo/Nemotron/Megatron-Bridge. You will optimize GPU utilization, manage complex checkpointing strategies, and implement automated fault tolerance for long-running jobs.
- Inference Orchestration: Lead the deployment of LLMs using vLLM, TensorRT-LLM, or SGLang. You will implement and tune cutting-edge techniques - including PagedAttention, continuous batching, and advanced quantization (AWQ/FP8) to maximize throughput and minimize TPOT (Time Per Output Token).
- Workload Orchestration: Utilize SLURM/Flyte/Ray/SkyPilot to manage and scale ML workloads across diverse cloud providers and on-prem clusters, ensuring seamless resource shifting and cost-effective execution.
- Lifecycle Management: Standardize model tracking, versioning, and transition workflows using MLflow (or similar tool), ensuring reproducible training runs and a clear path from research to production.
- Performance Engineering: Conduct deep-dive profiling and bottleneck analysis across the full stack - from CUDA kernels and NCCL collective communications to Python-level orchestration.
- Efficiency & Cost Governance: Monitor and optimize cloud and on-prem GPU expenditures through intelligent scaling policies and high-density resource packing.
- Technical Leadership: Set the bar for engineering excellence. You will drive the roadmap, perform rigorous code reviews, and mentor junior and mid-level engineers.
Required Qualifications
- Experience : 5+ years in MLOps, DevOps, or Software Engineering, with a minimum of 2 years dedicated to LLM infrastructure.
- Deep Learning Ecosystem : Expert-level proficiency with PyTorch and the NVIDIA stack (CUDA, NCCL, Triton).
- Specialized Tooling : Hands-on experience with NVIDIA NeMo (or Megatron-Bridge) for distributed training and at least two of the following for serving: vLLM, TensorRT-LLM, or SGLang.
- Orchestration & Lifecycle : Proven experience with SLURM/Flyte/Ray/SkyPilot for cluster management and MLflow (or similar tool) for experiment and model management.
- Infrastructure : Deep expertise in Kubernetes and K8s operators (e.G., KubeRay, MPI Operator, or Run:ai).
- Systems Programming : Mastery of Python and a functional understanding of C++ or Rust for performance-critical components.
- Next-Gen Hardware : Familiarity with high-performance networking (InfiniBand/RoCE) and NVIDIA H200/B200 (Blackwell) architectures.
By applying to this role you understand that we may collect your personal data and store and process it on our systems. For more information please see our Privacy Notice (