Job Description
This job was selected by the SpringCube team to help AI, Data and Cloud Engineers discover relevant San Francisco Bay Area employers. Sign up to view the full employer details and apply directly with the hiring company.
Company Overview
A leading organization is seeking an experienced data engineering professional to help build and advance scalable, cloud-based data platforms that support enterprise analytics, operational reporting, and advanced data use cases. The organization’s data environment focuses on modern cloud technologies, enterprise data integration, governance, and reliable data delivery.
The organization is seeking a Lead Data Engineer to architect, build, and lead the development of scalable cloud-based data platforms. The role will provide technical leadership in designing and optimizing ETL/ELT frameworks using Azure data services, integrating information from ERP, CRM, operational systems, APIs, and third-party sources, and establishing robust data models within a modern Lakehouse architecture.
Key Responsibilities
- Lead the design, development, and optimization of scalable data pipelines supporting data ingestion, transformation, and enterprise-wide consumption.
- Architect and implement enterprise-grade ETL/ELT frameworks using Azure Fabric or comparable cloud data platforms.
- Oversee and optimize data integrations from ERP systems such as NetSuite and SAP, CRM platforms such as Salesforce, internal systems, APIs, and third-party data sources.
- Design and govern high-quality, scalable data models supporting analytics, reporting, operational systems, and advanced data use cases.
- Partner with Data Architects to define and implement Lakehouse patterns, Delta Lake strategies, medallion architecture, and domain-driven design principles.
- Establish and enforce data quality frameworks, validation standards, lineage tracking, and observability practices.
- Drive performance optimization, scalability, reliability, and cloud cost governance.
- Provide technical leadership and mentorship to data engineers.
- Conduct technical and architecture reviews while enforcing engineering best practices.
- Collaborate with analysts, application teams, architects, and business stakeholders to translate requirements into scalable data solutions.
- Lead Master Data Management (MDM), metadata management, governance, and data standardization initiatives.
- Oversee CI/CD automation, DevOps integration, testing frameworks, and monitoring strategies for data workflows.
- Evaluate emerging technologies and recommend platform improvements aligned with enterprise technology strategy.
Required Qualifications
- 10+ years of experience in data engineering, data architecture, or related roles.
- Proven experience leading large-scale data platform initiatives in cloud environments.
- Extensive hands-on experience with Azure data services, including Azure Data Lake, Data Factory, Fabric, Synapse, or similar technologies.
- Advanced proficiency in SQL and Python.
- Experience with Spark or other distributed data processing frameworks.
- Deep experience designing and implementing enterprise ETL/ELT frameworks.
- Strong expertise in data modeling, including dimensional modeling, star schema, and Lakehouse/Delta modeling.
- Experience integrating complex enterprise systems, including ERP, CRM, and operational platforms.
- Strong understanding of data governance, metadata management, MDM, and data quality frameworks.
- Experience with performance tuning, workload optimization, scalability, and cloud cost management.
- Demonstrated ability to lead technical teams, conduct architecture reviews, and mentor engineers.
- Strong problem-solving, debugging, and system design skills.
- Ability to collaborate effectively with technical and business stakeholders.
- Willingness to travel up to 5% depending on business needs.
Preferred Qualifications
- Experience with Delta Tables, Snowflake, Synapse, or comparable cloud data platforms.
- Experience with event-driven and streaming architectures, including Kafka, Event Hub, and streaming pipelines.
- Familiarity with finance, operations, energy, or ERP-driven data domains.
- Experience designing API-based data integrations and modern integration patterns.
- Azure certifications such as Data Engineer Associate, Solutions Architect, or equivalent.
- Experience enabling analytics teams, data science workflows, or machine learning pipelines.
- Experience implementing enterprise data security and compliance frameworks.
Disclaimer
SpringCube curates tech job listings from various company websites to support tech professionals globally.
1.No Endorsement: Job ads on SpringCube do not imply endorsement of their authenticity or quality.
2.No Client Relationship: This company is not a client of SpringCube unless stated.
3.To Apply: Click the “Apply” button to be redirected to the hiring company’s application page for this job.
4.No Liability: SpringCube is not liable for inaccuracies.
