Back to Job Listings

ML Infra Engineer, Data Systems

San Francisco

SpringCube

Full-time - Senior Engineer

Healthcare, Biotech, Pharma & Life Sciences

Posted 1 week ago

Disclosed upon interview

Contact Employer
  • Share:
Send Feedback
Report This Job

Job Description

This job was selected by the SpringCube team to help AI, Data and Cloud Engineers discover relevant San Francisco Bay Area employers. Sign up to view the full employer details and apply directly with the hiring company.

Company Overview

The company develops general-purpose artificial intelligence systems designed to operate in the physical world. Its multidisciplinary team of engineers, scientists, and robotics specialists builds foundation models and learning algorithms for robotics and other physically actuated devices. Its infrastructure organization develops the data platforms, training systems, evaluation pipelines, and engineering tools required to support large-scale machine learning research.

Job Description

The ML Infra Engineer, Data Systems will design, build, and operate the data infrastructure that supports large-scale robot learning. This role focuses on developing reliable, high-performance systems that connect raw data sources with model training and evaluation workflows.

Working at the intersection of distributed systems, storage technologies, and machine learning infrastructure, the engineer will build scalable data ingestion, processing, storage, and management systems. The position will help research and engineering teams work efficiently with massive multimodal datasets while maintaining data quality, performance, and operational reliability.

Responsibilities

  • Data Ingestion and Processing: Design and develop high-throughput pipelines to validate, transform, and extract features from raw multimodal data.
  • Batch and Streaming Systems: Build and operate large-scale batch and streaming workflows that process massive datasets efficiently.
  • Storage Systems: Design object storage architectures, metadata systems, and efficient data access patterns. Evaluate file formats based on performance, scalability, and operational requirements.
  • Data Lifecycle Management: Develop systems for data backfills, dataset reconstruction, garbage collection, and large-scale data transformations.
  • Training-Time Performance Optimization: Improve data loaders, sharding, prefetching, caching, and data throughput to reduce the time between data ingestion and model training.
  • Metadata and Indexing: Build scalable metadata storage and indexing systems for datasets, annotations, and training artifacts.
  • Large-Scale Data Movement: Develop efficient methods for transferring petabytes of data across clusters and computing environments.
  • Operational Reliability: Implement observability, validation mechanisms, and safeguards to detect and prevent silent data quality regressions.
  • Cross-Functional Collaboration: Work closely with researchers, engineers, and robotics specialists to translate evolving data requirements into reliable and scalable infrastructure.

Qualifications

  • Strong software engineering fundamentals.
  • Experience building distributed systems or large-scale data pipelines.
  • Ability to reason about system performance, memory utilization, input/output operations, and storage efficiency.
  • Familiarity with batch processing and/or streaming data systems.
  • Experience with object storage systems and understanding of data format trade-offs.
  • Strong ownership mindset, with the ability to design, build, operate, and continuously improve systems end-to-end.
  • Ability to collaborate closely with researchers and support fast-moving technical projects.

Preferred Qualifications

  • Experience with large-scale machine learning training pipelines or data loading systems.
  • Knowledge of columnar storage formats or custom data formats.
  • Experience with distributed data and processing technologies such as ClickHouse, Ray, Flink, or Spark.
  • Hands-on experience managing and operating petabyte-scale datasets.
  • Demonstrated ability to identify, debug, and resolve performance bottlenecks in data-intensive systems.

Team Overview

The Infrastructure organization develops the foundational systems required for large-scale machine learning and robotics research. Its responsibilities include training infrastructure, data platforms, evaluation pipelines, and tools that enable researchers and robotics specialists to work safely and efficiently with massive datasets.

Disclaimer

SpringCube curates tech job listings from various company websites to support tech professionals globally.

  1. No Endorsement: Job ads on SpringCube do not imply endorsement of their authenticity or quality.
  2. No Client Relationship: This company is not a client of SpringCube unless stated.
  3. To Apply: Click the “Apply” button to be redirected to the hiring company’s application page for this job.
  4. No Liability: SpringCube is not liable for inaccuracies.
‹