Job Description
This job was selected by the SpringCube team to help AI, Data and Cloud Engineers discover relevant San Francisco Bay Area employers. Sign up to view the full employer details and apply directly with the hiring company.
Company Overview
A mission-driven healthcare technology organization is developing advanced technology designed to help improve healthcare delivery and patient outcomes. Its platform analyzes comprehensive medical records to identify diagnoses and treatment opportunities that may otherwise be missed, supporting healthcare professionals at the point of care, during chart review, and through population-wide screening.
Key Responsibilities
- Collect, model, and consolidate data into the data platform to support analytics and research initiatives.
- Design, build, and evolve data models and pipelines that reliably transform and deliver data to downstream consumers.
- Own data quality in collaboration with engineering teams and ensure datasets are trustworthy and production-ready.
- Partner closely with product teams to deliver analytics and actionable insights to internal and external stakeholders.
- Own the reliability and day-to-day operation of the data platform and its pipelines.
- Implement proactive monitoring, alerting, and operational management practices.
- Analyze and optimize Spark workloads and partitioning strategies to improve performance and manage infrastructure costs.
- Adapt data pipelines and platform components to upstream breaking changes.
- Develop transparent, code-driven data systems rather than relying on black-box services.
- Help architect and continuously improve the data infrastructure supporting the platform.
- Participate in on-call operational support for owned systems.
Required Qualifications
- Bachelor’s degree in Computer Science, Mathematics, Statistics, or a related field, or equivalent practical experience.
- 5+ years of experience in data engineering roles.
- 3+ years of experience using PySpark to build data pipelines.
- 3+ years of experience working with public cloud technologies, preferably AWS services such as S3, EMR, or Athena.
- Strong proficiency in Python and SQL.
- Hands-on experience across the full data stack, with particular depth in data modeling and pipeline design.
- Practical experience with LLM-assisted development and an understanding of its capabilities and limitations.
- Willingness to participate in on-call operational support for owned systems.
- Strong understanding of data quality, reliability, production deployment, and operational data engineering practices.
Preferred Qualifications
- Experience with one or more of the following technologies: Apache Iceberg, Dagster, ClickHouse, PostgreSQL, FastAPI, or Metabase.
- Experience working with healthcare data.
- Familiarity with HIPAA compliance and healthcare data de-identification.
- Familiarity with open healthcare data standards such as OMOP CDM.
- Experience building and supporting data pipelines for production workflows.
- Experience with data validation, deployment, monitoring, and ongoing performance evaluation.
- Experience optimizing distributed data processing workloads and cloud infrastructure costs.
Technology Stack
- Amazon S3
- Apache Iceberg
- Amazon EMR
- PySpark
- Dagster
- Kubernetes
- ClickHouse
- PostgreSQL
- FastAPI
- Metabase
Hybrid Work & Location Requirements
- Candidates must be authorized to work in the United States without visa sponsorship.
- Candidates should be located within the New York City, Los Angeles, or San Francisco metropolitan areas.
- Engineers are expected to work from the office on Tuesdays and Thursdays.
- Additional in-office attendance may be required during onboarding periods and team onsite weeks, which may occur up to once per month.
- Relocation assistance may be available for candidates who do not currently reside in the NYC metropolitan area.
- Preference is given to candidates within commuting distance of the organization’s offices.
- Employees who prefer working more regularly from Los Angeles or Manhattan offices may have access to catered lunches and additional workplace benefits.
- Hybrid employees may work from locations outside their designated home office for up to 6 weeks per year.
Compensation
The expected base salary range for this position is $165,000 to $220,000, with additional equity opportunities. Compensation may vary based on factors including skills, qualifications, location, and experience.
Disclaimer
SpringCube curates tech job listings from various company websites to support tech professionals globally.
1.No Endorsement: Job ads on SpringCube do not imply endorsement of their authenticity or quality.
2.No Client Relationship: This company is not a client of SpringCube unless stated.
3.To Apply: Click the “Apply” button to be redirected to the hiring company’s application page for this job.
4.No Liability: SpringCube is not liable for inaccuracies.**
