Job Description
This job was selected by the SpringCube team to help AI, Data and Cloud Engineers discover relevant San Francisco Bay Area employers. Sign up to view the full employer details and apply directly with the hiring company.
Company Overview
The hiring organization is a fast-growing advanced analytics consulting firm focused on Machine Learning, Data Science, Artificial Intelligence, and data-driven business solutions. The organization works with Fortune 500 companies to help generate business value from data and has received recognition from established market research firms. Its global analytics team brings together technical professionals with expertise across data engineering, machine learning, cloud technologies, and emerging AI solutions.
Responsibilities
- Design and develop distributed data systems and large-scale data warehouses.
- Build and maintain scalable batch and real-time data pipelines.
- Develop robust pipelines for document ingestion, indexing, and retrieval to support RAG solutions.
- Architect data platforms capable of processing large-scale, including petabyte-level, datasets.
- Build data ingestion, transformation, extraction, and loading processes across diverse data sources.
- Curate and collect data from traditional and non-traditional sources.
- Develop and maintain data persistence and retrieval solutions using relational databases, vector databases, buckets, graph databases, and knowledge graphs.
- Integrate external databases, APIs, and knowledge graphs into RAG systems to improve contextualization and response generation.
- Work with graph technologies, graph algorithms, LLMs, optimization algorithms, relational databases, and diverse data formats.
- Build and maintain knowledge graphs and ontologies for knowledge retrieval applications.
- Develop schema-level constructs, including higher-level classes, property inheritance, and related ontology structures.
- Work with Open Cypher and graph-based retrieval technologies.
- Conduct experiments to evaluate RAG workflows, analyze results, and continuously improve system performance.
Data Warehousing and Databases
- Experience with Snowflake, NoSQL databases, and Neo4j.
- Experience with relational databases, vector databases, graph databases, knowledge graphs, and other data persistence technologies.
Data Pipelines
- Strong experience with Apache Airflow.
- Experience building and deploying batch and real-time pipelines.
- Experience with document ingestion, indexing, retrieval, and data processing workflows.
Cloud Platforms
- Familiarity with AWS services such as S3, RDS, Lambda, AWS Batch, SageMaker Processing Jobs, CloudFormation, SageMaker, and OpenSearch.
- Alternatively, experience with GCP services such as Vertex AI RAG, data pipelines, BigQuery, and GKE.
Operating Systems and Development Tools
- Experience with Linux.
- Experience with GitHub.
- Proficiency with VS Code.
Engineering Practices
- Experience with software testing, deployment automation, DevOps, and SysOps.
- Experience designing scalable and reliable infrastructure.
- Understanding of monitoring, optimization, security, and operational best practices.
Communication and Collaboration
- Strong presentation and communication skills.
- Experience working effectively with onshore and offshore teams.
- Strong analytical, problem-solving, and collaboration abilities.
Desired Skills
- Experience with Hadoop and Apache Spark.
- Proficiency with Streamlit and dashboard development.
- Experience building and maintaining internal APIs.
- Basic understanding of machine learning concepts.
- Familiarity with Generative AI tools and techniques.
- Experience creating and retrieving knowledge graphs.
- Experience managing vector databases.
- Experience working with unstructured data, including audio, video, images, and text.
- Experience with AWS, particularly SageMaker, Lambda, and OpenSearch.
- Experience with Airflow DAGs, AutoSys, and CronJobs.
- Expertise in CI/CD using Jenkins and GitHub Actions.
- Advanced skills in Terraform and CloudFormation.
- Knowledge of Docker and Kubernetes.
- Experience monitoring and optimizing system performance, reliability, and security.
- Understanding of cloud security best practices.
- Experience designing and managing scalable infrastructure.
- Knowledge of disaster recovery and business continuity planning.
- Ability to stay current with emerging technologies, industry trends, and engineering methodologies.
- Ability to work effectively within collaborative team environments.
Disclaimer
SpringCube curates tech job listings from various company websites to support tech professionals in Singapore.
1.No Endorsement: Job ads on SpringCube do not imply endorsement of their authenticity or quality.
2.No Client Relationship: This company is not a client of SpringCube unless stated.
3.To Apply: Click the “Apply” button to be redirected to the hiring company’s application page for this job.
4.No Liability: SpringCube is not liable for inaccuracies.
