Member of Technical Staff, ML Engineer (Inference & Performance)

This job was selected by the SpringCube team to help AI, Data and Cloud Engineers discover relevant San Francisco Bay Area employers. Sign up to view the full employer details and apply directly with the hiring company.

Company Overview

A Physical AI infrastructure company building foundational technology for embodied intelligence and robotics. The company partners with robotics and world model research teams to advance systems that can perceive, understand, and operate in the physical world.

Responsibilities

• Own model serving end to end, designing and operating production inference infrastructure using Ray Serve and PyTorch across a heterogeneous GPU fleet.
• Optimize throughput and GPU utilization through batching strategies, scheduling, concurrency, queueing, and workload analysis.
• Improve model memory efficiency through quantization, precision selection, memory layout, activation management, and cache optimization while validating that model quality is preserved.
• Perform compiler, operator fusion, CUDA, and Triton kernel-level optimization when profiling demonstrates meaningful performance opportunities.
• Support high-volume video and multimodal inference workloads, evaluation systems, and reinforcement learning environments requiring large numbers of fast rollouts.
• Build reproducible benchmarking systems to measure performance consistently and identify regressions before they reach production.
• Operate inference infrastructure in production, including autoscaling, fault tolerance, graceful degradation, observability, and workload-level cost measurement.
• Bring partner models onto the company’s infrastructure, including packaging, throughput optimization, memory fitting, validation, and performance comparison against local environments.
• Continuously profile production workloads to identify actual performance bottlenecks and prioritize optimization efforts based on measurable impact.
• Improve inference efficiency across throughput, latency, hardware utilization, reliability, and cost.

Qualifications

• 3+ years of experience building production machine learning systems.
• Deep proficiency in Python and PyTorch.
• Experience owning inference performance for production workloads and demonstrating improvements through quantitative benchmarking.
• Strong understanding of GPU execution, including memory hierarchy and bandwidth, kernel launch overhead, occupancy, and execution bottlenecks.
• Experience operating distributed serving or compute frameworks such as Ray or Kubernetes under production workloads.
• Ability to measure and optimize systems based on throughput, latency, utilization, and cost.
• Strong judgment when determining which performance optimizations are worthwhile and which should be left unchanged.
• Experience working with production machine learning infrastructure and performance-critical systems.

Nice to Have

• Experience with CUDA, Triton, or compiler-level optimization, including torch.compile, TensorRT, XLA, or similar technologies.
• Experience with video or multimodal inference at scale, including hardware decoding, preprocessing, and batching variable-length inputs.
• Production experience with quantization, distillation, speculative methods, or related model optimization techniques.
• Experience with Ray Serve, vLLM, TensorRT-LLM, or comparable model serving frameworks.
• Experience serving diffusion models, video generation models, or vision-language-action models.
• Open-source contributions to model serving, machine learning infrastructure, or performance-focused systems used by other engineers.

Disclaimer

SpringCube curates tech job listings from various company websites to support tech professionals globally.

1. No Endorsement: SpringCube does not endorse or recommend any company or position listed.
2. No Client Relationship: Listing a position does not mean SpringCube has a client, partnership, or employment relationship with the hiring company.
3. To Apply: Use the Apply button to apply directly through the relevant hiring company’s application process.
4. No Liability: SpringCube is not responsible for the hiring process, employment terms, recruitment decisions, or outcomes associated with any listed position.