Back to Job Listings

Sr Databricks Data Engineer

San Francisco

SpringCube

Full-time - Senior Engineer

IT Services & Consulting

Posted 2 weeks ago

Disclosed upon interview

Contact Employer
  • Share:
Send Feedback
Report This Job

Job Description

The SpringCube team curated the following job opportunity to help you in your job search. Explore the position below to find your next career move.

Company Overview

A leading global professional services organization helps enterprises transform technology platforms, modernize data environments, and unlock business value through innovation. The company partners with organizations to design, build, and optimize cloud-based data solutions that support analytics, artificial intelligence, and large-scale digital transformation initiatives across industries.

The organization is seeking a Sr Databricks Data Engineer to design, build, and optimize cloud-based data engineering solutions that support enterprise-scale transformation. This role will work closely with business and technology leaders to modernize data platforms, enable advanced analytics and AI use cases, and drive measurable business outcomes.

The successful candidate will play a key role in delivering scalable Databricks solutions that enhance performance, expand digital capabilities, and help organizations make data-driven decisions at scale. This position offers the opportunity to work on complex cloud data engineering initiatives while contributing to innovation and modernization efforts.

Key Responsibilities

  • Establish, document, and promote best practices for data architecture, integration, and data modeling.
  • Design, develop, and maintain robust data pipelines and data architectures that support large-scale enterprise data requirements.
  • Lead initiatives focused on improving data quality, operational efficiency, and process scalability.
  • Evaluate, pilot, and integrate emerging big data and analytics technologies to drive innovation.
  • Lead, mentor, and develop teams of data engineers and architects, fostering technical growth and effective project delivery.
  • Design and implement data governance, security, and compliance strategies for modern cloud data ecosystems.
  • Communicate technical concepts and business value to executives, business stakeholders, and technology teams.
  • Oversee the implementation of CI/CD practices using tools such as Azure DevOps, AWS CodePipeline, Jenkins, TFS, or PowerShell.
  • Collaborate across business and technology functions to solve complex data modernization and platform engineering challenges.

Required Qualifications

  • Bachelor’s degree in Computer Science, Engineering, or a related field.
  • 5+ years of hands-on experience in data engineering with a focus on Databricks on AWS, Microsoft Azure, or Google Cloud Platform (GCP).
  • Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed computing platforms.
  • Experience with data warehousing, third normal form (3NF), dimensional modeling, enterprise data lakes, incremental data loading, and metadata-driven ingestion and data quality frameworks using PySpark.
  • 1+ year of experience leading complex cross-functional data projects and technical teams.
  • Hands-on experience with Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data engineering workloads.
  • Ability to travel up to 50% based on project and client requirements.
  • Strong written and verbal communication skills.
  • Ability to work independently and collaboratively in team environments.
  • Strong project leadership, stakeholder management, and relationship-building skills.
  • Excellent organizational skills with the ability to manage multiple priorities in a fast-paced environment.
  • Strong attention to detail and commitment to delivering high-quality solutions.

Preferred Qualifications

  • Master’s degree in Computer Science, Engineering, or a related field.
  • Experience across AWS, Azure, and GCP cloud ecosystems and associated big data services.
  • Experience tuning and optimizing performance in Databricks and Apache Spark environments.
  • Experience with Databricks Lakeflow.
  • Experience developing or supporting artificial intelligence and machine learning solutions.

Disclaimer

SpringCube curates tech job listings from various company websites to support tech professionals globally.

  1. No Endorsement: Job ads on SpringCube do not imply endorsement of their authenticity or quality.
  2. No Client Relationship: This company is not a client of SpringCube unless stated.
  3. To Apply: Click the Apply button to be redirected to the hiring company’s application page for this job.
  4. No Liability: SpringCube is not liable for inaccuracies.