Senior Data Engineer

This job was selected by the SpringCube team to help AI, Data and Cloud Engineers discover relevant San Francisco Bay Area employers. Sign up to view the full employer details and apply directly with the hiring company.

Company Overview

A rapidly growing AI-native technology company is transforming the insurance industry through an intelligent platform designed to modernize the full policy lifecycle. The organization combines agentic AI, advanced machine learning, and modern software infrastructure to address complex insurance workflows and deliver solutions that fundamentally change how insurance operations are performed.

The organization has raised significant funding and is expanding its global operations while building a highly advanced AI-focused engineering environment. Its technology platform is designed from the ground up to integrate AI into core workflows rather than simply adding AI capabilities to legacy systems.

The organization is seeking a Senior Data Engineer to join its small, high-impact data engineering team within the AI/ML organization. This role will focus on building the infrastructure, data pipelines, and internal frameworks that enable machine learning engineers to develop, deploy, and iterate on AI-powered features.

The successful candidate will collaborate closely with machine learning, analytics, product, data science, and software engineering teams to ensure that data infrastructure and tooling are reliable, scalable, maintainable, and aligned with the needs of an AI-native platform.

Key Responsibilities

• Collaborate with Data Science, Product Managers, and Software Engineers to build robust ETL pipelines that support compelling user-facing product features.
• Design, develop, and maintain scalable data pipelines and data infrastructure.
• Contribute to architecture decisions that improve the reliability, scalability, and maintainability of data systems.
• Develop and enhance observability tooling to monitor data pipelines and infrastructure.
• Contribute to data quality initiatives and establish practices that ensure accurate and reliable data.
• Help develop scalable internal frameworks for managing prompt engineering pipelines and other AI workflows.
• Build infrastructure and tooling that enables machine learning engineers to efficiently develop, deploy, and iterate on AI-powered features.
• Promote and enforce engineering best practices across the AI/ML organization.
• Support high standards for code quality, testing, documentation, and maintainability.
• Collaborate with cross-functional teams to understand business and data requirements and translate them into effective technical solutions.
• Develop high-impact data products and pipelines in a fast-paced environment.
• Help improve engineering velocity through reusable tools, frameworks, and automation.

Required Qualifications

• 5+ years of experience in data engineering, backend engineering, or related roles focused on data infrastructure.
• Proven experience designing, building, and maintaining scalable data pipelines.
• Experience with workflow orchestration tools such as Airflow, Dagster, or Prefect.
• Strong proficiency in SQL and Python.
• Experience with software development practices including version control, CI/CD, and automated testing.
• Experience with dbt and data testing strategies.
• Familiarity with cloud infrastructure, preferably AWS or GCP.
• Experience with infrastructure-as-code tools such as Terraform.
• Ability to work effectively with version control and CI/CD systems.
• Strong understanding of data infrastructure, pipeline reliability, and operational best practices.
• Ability to work closely with business stakeholders to understand data requirements and translate them into scalable solutions.
• Proven ability to design and deliver high-impact data products and pipelines in fast-paced environments.
• Strong communication, collaboration, and problem-solving skills.
• Ability to navigate ambiguity and take ownership of complex technical challenges.

Preferred Qualifications

• Experience with TypeScript or a similar programming language.
• Prior experience working in or adjacent to insurance, fintech, or risk modeling.
• Experience with MLOps or supporting AI/ML-driven products.
• Experience building internal tools or frameworks that improve engineering team velocity.
• Contributions to open-source data tools or active involvement in the data engineering community.
• Familiarity with AI/ML workflows and infrastructure supporting machine learning development and deployment.

Disclaimer

SpringCube curates tech job listings from various company websites to support tech professionals globally.

1.No Endorsement: Job ads on SpringCube do not imply endorsement of their authenticity or quality.

2.No Client Relationship: This company is not a client of SpringCube unless stated.

3.To Apply: Click the “Apply” button to be redirected to the hiring company’s application page for this job.

4.No Liability: SpringCube is not liable for inaccuracies.