Senior Data Engineer

This job was selected by the SpringCube team to help AI, Data and Cloud Engineers discover relevant San Francisco Bay Area employers. Sign up to view the full employer details and apply directly with the hiring company.

Company Overview
A rapidly growing advertising technology company is transforming TV advertising through the intelligent application of AI, machine learning, sophisticated media buying, and proprietary analytics. Its platform combines advanced media buying with data-driven measurement to help brands reach customers through linear and streaming TV advertising.

The company has received recognition across the advertising technology industry and is backed by experienced leaders from major technology and media organizations.

Summary
The Senior Data Engineer will join the Reporting and Measurement team to design, build, and own large-scale data pipelines that transform high-volume raw data into curated, production-grade datasets powering reporting and measurement products.

This role will take data products from requirements and technical design through implementation, validation, and production rollout. The position will work closely with data science, backend and frontend engineering, and product teams to deliver reliable and scalable data solutions.

Responsibilities

• Design, build, and operate large-scale production ETL pipelines
• Develop data products from requirements through technical specifications, implementation, validation, and production deployment
• Build and maintain high-volume data pipelines for reporting and measurement products
• Partner with data science teams on data methodologies and analytical requirements
• Collaborate with backend and frontend engineers on data-serving layers
• Work with product teams to understand requirements and deliver production-ready datasets
• Design curated and gold-layer datasets for analytics and downstream services
• Implement strong data validation, reconciliation, and methodology comparison processes
• Monitor and improve data quality and pipeline reliability
• Design and maintain workflow orchestration, including DAGs, backfills, and failure recovery
• Debug distributed data systems and perform performance and memory optimization
• Prepare technical specifications, documentation, and incident reports
• Stay engaged with emerging agentic developer workflows, including agent harnesses and agent-loop engineering

Qualifications

• 5+ years of experience designing, building, and operating production ETL pipelines at scale
• Strong programming skills in Python or a similar programming language
• Advanced SQL skills
• Deep hands-on experience with Spark/PySpark
• Experience working with Databricks and Delta Lake
• Experience with workflow orchestration tools such as Airflow
• Strong data modeling skills for analytics, including curated/gold-layer datasets, schema design, partitioning, and data contracts
• Strong understanding of data quality, validation, and reconciliation
• Experience debugging distributed data systems, including performance tuning, memory issues, and optimization
• Strong written communication skills
• Ability to collaborate effectively across data science, engineering, and product teams
• Interest in emerging agentic development paradigms and how they are changing software development

Preferred Qualifications

• Experience working with OLAP data stores such as ClickHouse
• Experience building or testing agentic developer workflows
• Experience working with large-scale reporting or measurement datasets

Disclaimer
SpringCube curates tech job listings from various company websites to support tech professionals globally.

1. No Endorsement: Job ads on SpringCube do not imply endorsement of their authenticity or quality.
2. No Client Relationship: This company is not a client of SpringCube unless stated.
3. To Apply: Click the “Apply” button to be redirected to the hiring company’s application page for this job.
4. No Liability: SpringCube is not liable for inaccuracies.