| | | 20,000+ Fresh Resumes Monthly | |
|
|
| |
Click here or scroll down to respond to this candidateCandidate's Name
Phone : PHONE NUMBER AVAILABLE Linkedin:Email : EMAIL AVAILABLE https://LINKEDIN LINK AVAILABLE SUMMARYData Engineer with 8+ years of experience specializing in Cloud Computing, Data Integration, ETL, and Data Visualization. Proficient in Python, PySpark and SQL with a strong focus on Big Data concepts and data modeling. Experienced in optimizing data pipelines and achieving up to 40% performance enhancement. Skilled in both SQL and NoSQL databases including Oracle, DB2, SQL Server, and MongoDB. Adept at developing robust data solutions using Agile methodology and delivering high-quality, timely insights to drive business decisions.SKILLS: Programming Languages : Python, SQL, PL/SQL, UNIX Scripts, XQUERY Python Libraries : Pandas, NumPy, PySpark, SciPy, Matplotlib Big Data Ecosystem : HDFS, MapReduce, Hive, PIG, Spark SQL, Sqoop ETL Tools : IBM INFOSPHERE DATASTAGE v8.5, v9.1, Talend BI Tools : Qlikview, OBIEE, Tableau, Quicksight, Looker Databases : Oracle, DB2, SQL Server, Netezza, Redshift, Teradata, BigQuery DevOps : CI/CD, WCNP, Stage gates, Docker, Kubernetes No-SQL Databases : MongoDB Cloud Platforms : GCP, AWS, Azure Orchestrator/scheduler : Airflow, StonebranchPROFESSIONAL EXPERIENCE:Walmart Inc, USAData Engineer IIIFebruary 2023 Current Ideated, and led a scrum team of 5 in developing an optimized and efficient tax data pipeline using Python and PySpark, enhancing data flow efficiency by 30%. Enhanced data processing performance using advanced PySpark techniques, including DataFrame operations and Spark SQL, resulting in a 40% reduction in processing time for tax data. Achieved near real-time tax data ingestion and processing with minimal latency. Leveraged Airflow to orchestrate and schedule workflows. Worked with Event driven integration with GCS to create more dynamic and responsive data pipelines. Automated the generation of end-of-month tax reports, reducing manual effort by 60% and improving report accuracy by 25%. Wrote optimized and efficient SQL queries within PySpark to extract and manipulate tax data, improving query performance and reducing execution times by 30%. Implemented robust error handling and logging mechanisms, ensuring reliable and timely report delivery to higher leadership. Collaborated with cross-functional teams and stakeholders to understand tax data requirements and deliver robust data solutions. Ensured data quality and integrity by doing data validation and error-checking routines in Python and PySpark, maintaining data accuracy and reducing errors by 35%. Optimized cloud costs by managing GCS resources effectively, implementing lifecycle policies and monitoring usage to reduce storage costs by 20%. Designed automated reporting solutions using Python, integrating with GCS and looker to deliver timely and accurate end-of- day tax reports, reducing reporting time by 40%.Technologies used: Python, PySpark, GCS, BigQuery, Airflow, Looker, Looper, Concord, Azure ADLS, DPaaS Groupon Inc, GermanyFinancial Data EngineerMarch 2019 August 2019 Developed an efficient International Finance Data pipeline using python, pySpark and AWS Teradata vantage, which improved the efficiency by 40%. Lead a team in creating informative Tableau visualizations for fintech analysis, speeding up decision-making by 15%. Worked with business users and data analysts to design and implement data integration workflow from various sources into AWS, ensuring consistency and availability of financial data while reducing integration time by 20%. Optimized data model achieving an 80% performance improvement. Resolved Performance issues by optimizing codebase thorough code reviews. Monitored project status closely, suggesting design changes that saved budget. Technologies used Python, AWS Teradata Vantage, Talend, Stonebranch Amazon Development Centre, IndiaSenior Business Intelligence DeveloperFebruary 2017 September 2018 Developed workflow for Daily Incremental loads using python scripts and AWS Redshift connectors. Designed and managed cloud eco system in AWS. Managed AWS cluster, set up and configure EC2 instances, allocation of Redshift clusters and monitoring the services being used and keep track of the performance. Designed Data Pipelines with Datanet Jobs and SQL Scripts which will extract large data from various sources and transformed to fit the cloud data warehouse model. Developed T50 and T90 Metrics for performance tuning of the SQL queries and Data Models. Developed Quicksight dashboard for Vendor holdback Metrics and Debit Balance Metrics using KPIs derived from Business users.Technologies used: Python, SQL, AWS services, Redshift, Oracle, OBIEE, Quick sight, Datanet, and Djs MINDTREE Limited, IndiaSenior Data EngineerMarch 2013 February 2017 Designed ETL jobs using IBM infosphere DataStage 9.1 to Extract, Transform and load Data in to staging, ODS and EDW. Built the Generic Code to avoid redundancy and reused the components using job parameters and Environment Variables. Designed and developed database mode using various Partitioning (Round-Robin, Hash, Entire, Same, modulus etc.) and Collection (Round- Robin, Ordered and Sorted Merge) techniques. Delivered an Entire module handling ETL for xml data from real time message Queues. Optimized the code to improve the performance by tuning the queries and DataStage jobs by following best practices and reviewed the teams code as part of peer review. Migrated Jobs from XML input stage to Distributed Transaction Stage with XQUERIES to yield high performance in real time services with message queues.Technologies used: IBM Infosphere DataStage v9.1, QlikViewv11.2, DB2, Netezza, SQL Server and UNIX EDUCATION: The University of Central Missouri, Warrensburg, Missouri, August 2021 December 2022 Jawaharlal Nehru Technological University, Hyderabad, August 2008 June 2012 |
