Key Skills and Competencies

pyspark,big data , AWS , feature engineering , cloud computing , sql , python , spark , big data , api , flask , streamlit , azure databricks

Work Experience

  • November 2022 - Present
    VISA, India

    Staff Data Engineer

    ● Leading fraud detection featureengineering, designing multi-platformframeworks thatboosted processing efficiency by 800% and managingend-to-end model deployment.
    ● Pioneered aGenerative AI application that improved productivity by30%.
    ● Invented and architected the patented Record Query Estimator, anindustry first solution that accurately predicts table record counts,boosting query efficiency andreporting by70%.
    ● Formulated an advancedscheduler framework for complex workflows beyondthe existing market solutions, enhancing timeefficiency by 40%.
    ● Spearheadedcompany’s PySpark revenue code revamp, boosting performance by 60%.

  • November 2021 - November 2022
    Nagarro, india

    Senior Data Engineer

    ● Directed the design and deployment of AWS-based ETL pipelines,integrating data from SharePoint, Oracle, and Redshift into Athena withschema validation, delivering30% cost savings.
    ● Architected a JSON ingestion pipelinefor Workday data, leveraging acustom-built generic utility that streamlined parsing and modelling,cuttingproject costs by 60%.
    ● Developed Generic revenue-modelling utility, automated end-to-endcalculations, eliminating manual processes, cost and time by95%.

  • June 2017 - November 2021
    Accenture, india

    Application Development senior analyst

    ● Invented and patented Generic Data Parsing Utility to automateXML/JSON schema derivation andparsing, enabling normalized tablepopulation for 40+ enterprise applications, saving multi-milliondollars.
    ●Streamlined end-to-end real-time pipelines on AWS using Kinesis, Firehose,EMR, Lambda, Step Functions, and CloudWatch.
    ●Led on-prem to cloud migration on Azure Databricks and AWS, buildingpipelines and integrating S3, Kinesis, Athena, Lambda, Step Functions, EMR,and CloudWatch.
    ●Developed QA-Tool UI with Flask, Python, Spark, and HTML to test raw vsparsed data, reducing testing from 25–30 days to minutes.
    ●Built XML Comparison Utility to compare XML at any level with detailedExcel reports, saving up to 95% of comparison time and cost.
    ● Invented automated schema migration utility for moving on-prem tablesto Athena, cutting manual effort, testing, and migration time by 97%.

SpringCube is a technology recruitment firm and tech-focused career platform. Since 2001, we’ve connected top talent with leading companies across Singapore and Southeast Asia.

Quick Links
Contact Us
Linkedin
Tell us how we can improve
feedback@springcube.com

Copyright 2024 ©SpringCube Asia.

Linkedin
‹
 
Required 'Candidate' login to applying this job. Click here to logout And try again
 
 

Account Activation

Before you can login, you must activate your account with the code sent to your email address. If you did not receive this email, please check your junk/spam folder. Click here to resend the activation email. If you entered an incorrect email address, you will need to re-register with the correct email address.

✕

You are already logged in as a .

Logout