Staff Software Engineer, ML Data Infrastructure, Autonomy

This job was selected by the SpringCube team to help AI, Data and Cloud Engineers discover relevant San Francisco Bay Area employers. Sign up to view the full employer details and apply directly with the hiring company.

Company Overview

An automotive technology company focused on developing electric vehicles and advanced software systems for autonomous driving and vehicle technology. The organization combines vehicle engineering, artificial intelligence, data infrastructure, and software to develop connected electric vehicles and technology platforms designed for next-generation transportation.

Responsibilities

• Own the unified data layer for autonomy, providing a single interface across document metadata, high-cardinality columnar analytics, real-time search, and object-stored sensor data.
• Design the architecture for the data lake, metadata access systems, and APIs that provide engineers with a unified data access layer.
• Define and own data access and format strategies, including canonical production log formats and future ML-native columnar and vector access approaches.
• Define canonical schemas, manage schema evolution, and lead migration strategies across data systems.
• Design and build batch and streaming pipelines for evaluation data and the underlying storage infrastructure, supporting large volumes and diverse metrics from on-road and simulation runs.
• Build indexing and discovery systems for fast semantic and metadata search across fleet data, including scenario tagging, event indexing, embedding-based similarity search, and engineer-facing query interfaces.
• Improve data mining tools that use machine learning techniques to help engineering teams discover rare and long-tail scenarios at fleet scale.
• Develop well-documented tools and APIs that allow engineers to find, filter, and materialize required datasets without building custom pipelines.
• Establish data correctness practices covering schema validation, producer-consumer contracts, freshness and completeness monitoring, and automated alerting.

Qualifications

• Bachelor’s degree in Computer Science, Electrical Engineering, or a related field, or equivalent professional experience.
• 8+ years of software engineering experience, with a strong focus on data infrastructure or data platform development.
• 5+ years of experience owning large-scale data platforms for machine learning or analytics workloads, including storage and data access architecture.
• 5+ years of experience with columnar and analytical data stores such as ClickHouse, Pinot, Druid, BigQuery, Databricks, or Snowflake, including high-cardinality data modeling and query performance optimization.
• 3+ years of experience designing storage and query layers for large volumes of heterogeneous metrics, including KPIs and performance telemetry.
• 3+ years of experience with document or NoSQL databases, including schema design, indexing, sharding, and large-scale operational considerations.
• 3+ years of experience with modern data and table formats such as Parquet, Arrow, Iceberg, and Delta.
• 3+ years of experience building production-scale batch and streaming pipelines with orchestration and data quality controls.
• 3+ years of hands-on Python development, plus production experience with Go, C++, or Rust for performance-sensitive components.
• 3+ years of experience with queueing and event-driven systems such as SQS, Kafka, Kinesis, or equivalent technologies in production ingestion environments.
• 2+ years of experience with production monitoring and alerting for data pipelines using tools such as Prometheus, Grafana, Datadog, or CloudWatch.
• Experience implementing automated data integrity and freshness checks.
• 2+ years of experience migrating production data platforms between storage or data-format architectures while maintaining ongoing service availability.
• Demonstrated engineering leadership, including setting technical direction, defining priorities and timelines, serving as a technical lead, and mentoring engineers.
• Ability to translate ambiguous, high-level requirements into detailed system designs and independently drive implementation through completion.
• Strong technical expertise and a demonstrated ability to raise engineering standards across broader technical organizations.
• Experience designing schemas and interfaces that other engineering teams depend on and that remain effective as requirements evolve.

Disclaimer

SpringCube curates tech job listings from various company websites to support tech professionals globally.

1. No Endorsement: SpringCube does not endorse or recommend any company or position listed.
2. No Client Relationship: Listing a position does not mean SpringCube has a client, partnership, or employment relationship with the hiring company.
3. To Apply: Use the Apply button to apply directly through the relevant hiring company’s application process.
4. No Liability: SpringCube is not responsible for the hiring process, employment terms, recruitment decisions, or outcomes associated with any listed position.