Candidate Information Title Data Engineer Quality Target Location US-VA-Front Royal Email Available with paid plan Phone Available with paid plan	20,000+ Fresh Resumes Monthly     View Phone Numbers     Receive Resume E-mail Alerts     Post Jobs Free     Link your Free Jobs Page     ... and much more Register on Jobvertise Free
Search 2 million Resumes Keywords: City or Zip:	Related Resumes Data Engineer Big Herndon, VA Data Engineer Big Herndon, VA Software Quality Assurance Engineer Centreville, VA SR. Data Analyst/ Data Engineer Leesburg, VA Big Data Engineer Chantilly, VA Quality Assurance Engineer Manassas, VA Data Engineer Migration Herndon, VA

SUMMARY Data Engineer with over 4 years of experience in designing, implementing, and optimizing data ingestion, processing, and storage solutions across various platforms, including AWS and GCP. Proficient in developing scalable data ingestion pipelines using Apache Kafka, AWS Kinesis, and AWS Lambda, processing over 50 million real-time events daily. Leveraged tools such as Power BI, Amazon QuickSight, and Python libraries (Pandas, NumPy, Matplotlib) for data analysis, visualization, and providing actionable insights. Implemented data quality checks, monitoring, and data profiling frameworks, reducing data quality issues and ensuring data integrity and reliability. Expertise in utilizing big data technologies like Apache Spark, Hadoop, Hive, and Sqoop to handle and process large volumes of structured and unstructured data, resulting in significant processing time reductions. Built real-time data ingestion pipelines using Apache Kafka and AWS Kinesis, processing millions of events daily, and integrating NoSQL databases like MongoDB and Cassandra for high-throughput data storage. Designed and maintained centralized data warehouses on AWS Redshift, enabling self-service analytics for business users and enhancing data-driven decision-making. Collaborated effectively with data scientists, analysts, and stakeholders to translate business requirements into technical solutions, demonstrating strong communication and problem-solving skills. Effective communicator with excellent problem-solving abilities, experienced in writing test cases, implementing test classes, and managing error handling to ensure software quality and reliability. SKILLSMethodologies: SDLC, Agile, WaterfallProgramming Language: Python, SQL, MATLAB, R, Scala Packages: NumPy, Pandas, Matplotlib, SciPy, Scikit-learn, TensorFlow, Seaborn Visualization Tools: Tableau, Power BI, Advanced Excel (Pivot Tables, VLOOKUP), Amazon Quick sight IDEs: Visual Studio Code, PyCharm, Jupyter Notebook Cloud Platforms: Amazon Web Services (IAM, S3, VPC, EC2, Athena, AWS Glue, Lambda, cloud watch, EMR, Redshift, Sage Maker), Google Cloud PlatformDatabase: MySQL, Oracle SQL, PL/SQL, PostgreSQL, MongoDB Data Engineering Concept Apache Spark, Apache Hadoop, Apache Kafka, Apache Beam, Apache Airflow, ETL/ELT Other Technical Skills: SSIS, SSRS, SSAS, Docker, Kubernetes, Jenkins, Terraform, Informatica, Talend, Amazon Redshift, Snowflake, Alteryx, Google Big Query, Data Quality and Governance, Machine Learning, Big Data, Grafana, Prometheus, Data Mining, Data Visualization, Data warehousing, Data transformation, Critical Thinking, Communication Skills, Presentation Skills, Problem-SolvingVersion Control Tools: Git, GitHubOperating Systems: Windows, Linux, Mac OSPROFESSIONAL EXPERIENCEData Engineer Wells Fargo, MD Oct 2022  Present Developed a scalable data ingestion pipeline using Apache Kafka, AWS Kinesis, and AWS Lambda to process and load over 50 million real-time events per day from various sources into an AWS Data Lake on Amazon S3. Developed and optimized Sqoop jobs to import and store 10 TB of data daily in HDFS and Hive, utilizing data transformation scripts with Pig against unstructured data. Nookesh VDATA ENGINEERMaryland, USA PHONE NUMBER AVAILABLE EMAIL AVAILABLE Designed and implemented a distributed data processing framework using Apache Spark on AWS EMR to perform batch and real-time analytics on structured and unstructured data, resulting in a 40% reduction in processing time. Built and maintained a centralized data warehouse on AWS Redshift, incorporating data from multiple on-premises and cloud sources, enabling self-service analytics for over 100 business users. Implemented data quality checks and monitoring using AWS Glue, AWS Lambda, and Amazon CloudWatch to ensure data integrity and reliability, reducing data quality issues by 60%. Created robust ETL/ELT processes using Python and Apache Airflow, resulting in a 30% increase in data processing efficiency. Integrated MongoDB and Cassandra for real-time and high-throughput data storage requirements, optimizing the performance and scalability of NoSQL databases within the data architecture. Collaborated with data scientists and analysts to understand business requirements and translate them into technical solutions, enabling data-driven decision-making across multiple business units. Automated data processing workflows using AWS Data Pipeline and AWS Step Functions, reducing manual effort, and ensuring timely delivery of analytical insights. Implemented data security and access control measures using AWS Identity and Access Management (IAM), AWS Key Management Service (KMS), and AWS Lake Formation, ensuring compliance with data privacy and security regulations. Leveraged Power BI and Amazon Quick Sight for ad-hoc data querying and interactive data visualization, empowering business users with self-service analytics capabilities. Implemented monitoring and alerting systems for data pipelines using Grafana and Prometheus, reducing downtime by 20%.Data Engineer Accenture, India Sept 2019  Dec 2021 Architected a real-time fraud detection system using Kafka and PySpark, processing 1 million transactions per hour with 99.9% accuracy. Developed Airflow DAGs to orchestrate daily ETL processes, reducing pipeline failures by 40% and improving data freshness by 6 hours. Optimized Hive queries for customer segmentation analysis, decreasing runtime by 35% for tables exceeding 100 million rows, implemented a data lake solution on Google Cloud Platform, migrating 75 TB of historical data from on- premise systems with 99.99% data integrity. Engineered a Spark-based data quality framework, automatically validating 500+ data quality rules daily, catching 95% of anomalies before production. Utilized PySpark and Python libraries (Pandas, NumPy) to build a churn prediction model, improving accuracy by 18% over legacy systems. Developed SSIS packages for nightly data mart updates, processing 30 GB of incremental data with 99.9% reliability. Designed and implemented REST APIs using Python Flask, enabling real-time data access for 50+ downstream applications. Created an Elasticsearch-based search solution for customer service, reducing average query time by 60% and improving user experience. Implemented k-means clustering using PySpark to segment 10 million customers, leading to a 25% increase in marketing campaign effectiveness. Optimized Big Query performance by implementing partitioning and clustering, reducing query costs by 40% and improving response times by 50%. Utilized GCP's Dataflow to create streaming ETL pipelines, ingesting 10,000 events per second from IoT devices with sub-second latency. Developed a custom Hadoop MapReduce job to perform complex aggregations on 5 years of transaction data, reducing processing time from 24 hours to 4 hours. EDUCATIONMaster of Science in Information Systems  University of Maryland, Baltimore County, Maryland, USA Bachelor of Technology in Electrical and Electronics Engineering  Amrita University, Bangalore, India CERTIFICATIONAWS Certified Solutions Architect - Associate