Senior Data Engineer

This job was selected by the SpringCube team to help AI, Data and Cloud Engineers discover relevant San Francisco Bay Area employers. Sign up to view the full employer details and apply directly with the hiring company.

Company Overview

A growing technology organization is focused on transforming how organizations perceive and use people data. Its data platform serves as a core foundation for products, data science initiatives, and customer-facing solutions. The organization emphasizes communication, transparency, customer success, leverage, integrity, and trust while fostering close collaboration across product, data science, and engineering teams.

The organization is seeking a Senior Data Engineer to join its data engineering team and help build and maintain the core data assets that support the broader technology platform. The role will involve designing and developing data pipelines for massive datasets, taking data systems from initial development through exploration, production, and customer use.

The successful candidate will have strong experience with distributed data processing, object-oriented programming, cloud-based data engineering, and large-scale data systems. This position requires the ability to work independently while collaborating closely with data science, product, and engineering teams.

Key Responsibilities

• Construct data pipelines that ingest and merge billions of individual entities into core data assets.
• Design, develop, and maintain scalable data pipelines for massive datasets.
• Work closely with data science teams to run machine learning models across billions of data points.
• Build supporting data services and applications to orchestrate and monitor data systems.
• Take ownership of data engineering projects from inception and exploration through production and customer use.
• Design and implement distributed data processing solutions for large-scale datasets.
• Collaborate with product, data science, and engineering teams to develop reliable and scalable data solutions.
• Make appropriate technical trade-offs between quality, complexity, and speed of delivery.
• Simplify complex technical problems into practical and maintainable solutions.
• Contribute to the development of cloud-native data engineering infrastructure and systems.

Technology Stack

• Cloud Platform: Google Cloud Platform (GCP)
• Programming Languages: Java, Python, Kotlin
• Data Warehouse & Databases: BigQuery, PostgreSQL, Scylla/Cassandra
• Distributed Processing: Dataflow, Apache Beam, Apache Spark
• Orchestration: Airflow

Required Qualifications

• 4–8 years of professional experience in data engineering.
• Significant experience working with Apache Beam, Apache Spark, Apache Flink, or MapReduce.
• Strong object-oriented programming skills in a JVM-based language.
• Expert-level understanding of distributed data processing.
• Familiarity with different data stores, their characteristics, differences, and appropriate use cases.
• Experience working at a company with fewer than 200 employees.
• Strong written and verbal communication skills.
• Ability to simplify complex problems into clear and practical solutions.
• Strong sense of ownership and responsibility while maintaining effective team collaboration and alignment.
• Ability to make informed trade-offs between quality, complexity, and speed of delivery.

Preferred Qualifications

• Proven experience taking large-scale projects from ideation through production.
• Experience leading greenfield data engineering projects.
• Working knowledge of cloud-native data engineering infrastructure.
• Experience designing and implementing scalable data platforms and distributed systems.

Disclaimer

SpringCube curates tech job listings from various company websites to support tech professionals globally.

1.No Endorsement: Job ads on SpringCube do not imply endorsement of their authenticity or quality.

2.No Client Relationship: This company is not a client of SpringCube unless stated.

3.To Apply: Click the “Apply” button to be redirected to the hiring company’s application page for this job.

4.No Liability: SpringCube is not liable for inaccuracies.**