Back to Job Listings

Data Engineer – Databricks, ETL & Big Data Pipelines

SpringCube

Full time - Senior Associate/ Asst Manager

Aerospace, Aviation & Airlines

Singapore ( Remote )

Published 4 weeks ago

Salary: SGD5,000 - SGD10,000

Contact Employer
  • Share:
Send Feedback
Report This Job

Job Description

The SpringCube team curated the following job opportunity to help you in your job search. Explore the position below to find your next career move.

Data Engineer – Databricks, ETL & Big Data Pipelines

Company Overview
A leading DataTech company is transforming the $170B Aircraft Asset Management Industry by leveraging AI-driven solutions and advanced data models. The company optimizes airworthiness and maintenance data management, turning unstructured technical records into actionable insights, helping Airlines, Lessors, and MROs save millions in operational inefficiencies.

Job Description
The Data Engineer will be responsible for designing, building, and maintaining scalable, efficient, and reliable data infrastructure within the Databricks environment. You will collaborate with the data team to optimize ETL processes, develop data pipelines, and ensure high-quality data for analytics and machine learning.

Key Responsibilities

  • Data Pipeline Development: Design, build, and maintain data pipelines for diverse data sources and destinations, ensuring scalability, reliability, and performance.
  • ETL Process Optimization: Develop ETL processes for data integration and transformation from multiple sources.
  • Data Quality & Monitoring: Implement checks and monitoring systems to ensure the health and quality of data pipelines.
  • Collaboration: Work with data analysts and data scientists to provide clean, usable data for further analysis.
  • Automation: Automate the deployment, management, and monitoring of data pipelines using Databricks APIs, Python, Apache Airflow, and other relevant tools.
  • Optimization: Continuously optimize and tune the performance of data pipelines to handle large-scale data efficiently.
  • Databricks Expertise: Utilize Databricks notebooks, clusters, jobs, and workflows to develop and manage data processes.
  • System Integration: Integrate Databricks with other data processing tools and systems, ensuring seamless data flow across platforms.

Job Requirements

  • Education: Bachelor’s degree in Computer Science, Information Technology, or related field.
  • Experience: Minimum of 3 years as a Data Engineer with expertise in data pipelines and big data processing.
  • Programming: Proficient in Python, Java, Scala, and SQL.
  • Big Data Frameworks: Hands-on experience with Apache Spark, Hadoop, or Flink.
  • Databricks: Strong experience with Databricks, including Databricks SQL, Delta, and ML Flow.
  • Cloud Tools: Familiar with AWS Glue, Azure Data Factory, Google Cloud Dataflow, or other cloud-based data processing tools.
  • Data Modeling: Knowledge of data modeling, warehousing, and integration.
  • Version Control: Experience with Git and collaborative software development.
  • Communication: Strong analytical, problem-solving, and teamwork skills.

Preferred Qualifications

  • Advanced Degree: Master’s degree in Computer Science, Information Technology, or related field.
  • Distributed Data Systems: Experience with Apache Kafka, Apache Storm, or similar distributed systems.
  • Containerization: Familiarity with Docker, Kubernetes, or container orchestration tools.
  • Visualization Tools: Experience with tools like Tableau, Power BI, or QlikView for data visualization.

Disclaimer
SpringCube curates tech job listings from various company websites to support tech professionals in Singapore during these challenging times.

  1. No Endorsement: Job ads on SpringCube do not imply endorsement of their authenticity or quality.
  2. No Client Relationship: This company is not a client of SpringCube unless stated.
  3. To Apply: Click the “Apply” button to be redirected to the hiring company’s application page for this job.
  4. No Liability: SpringCube is not liable for inaccuracies.