| 20,000+ Fresh Resumes Monthly | |
|
|
| | Click here or scroll down to respond to this candidateCandidate's Name
Sr. Data EngineerEMAIL AVAILABLE PHONE NUMBER AVAILABLELinkedIn: LINKEDIN LINK AVAILABLESUMMARYResults-driven Data Engineer with 5+ years of experience in designing, developing, and deploying large-scale distributed systems. Expertise in Python, R, and Hadoop, with a strong background in scalable solutions and ETL processes. Experienced in cloud platforms like AWS, Azure, Snowflake, and Databricks. Skilled in data visualization with Tableau and Power BI, database management, and leveraging Python and R for data analysis and machine learning. Effective in collaborating with cross- functional teams and presenting insights to both technical and non-technical stakeholders. SKILLSSoftware Development: SDLC, Agile, WaterfallProgramming: Python, R, SQL, PL/SQL, Java, Scala, Shell Scripting, HTML Data Visualization: Tableau, Power BI, SSRS, Excel Cloud Platforms: AWS, Azure, GCP, SnowflakeBig Data: Hadoop, MapReduce, Hive, Spark, Kafka, Pig, Sqoop, PySpark Databases: Oracle, SQL Server, MySQL, Cassandra, Teradata, PostgreSQL, MongoDB, HBase, SnowflakeMachine Learning: Logistic Regression, Decision Trees, Random Forest, KNN, PCA, Linear Regression, Nave BayesOperating Systems: Windows, LinuxPROFESSIONAL EXPERIENCENorthern Trust, Chicago, ILAzure Data Engineer October 2023 - PresentSpearheaded the architecture and implementation of scalable data solutions utilizing Azure services, ensuring optimal performance and reliability.Designed and optimized ETL workflows in Azure Data Factory, enhancing data processing efficiency and reducing latency.Directed seamless data migration from legacy systems to Azure using Azure Data Factory, safeguarding data integrity and consistency.Administered Azure Virtual Machines, VNet configurations, and Azure Storage Accounts, ensuring high availability and secure data management.Streamlined deployment processes by implementing containerized solutions using Azure Kubernetes Services (AKS) and Docker, accelerating delivery timelines.Enhanced data accessibility and usability through seamless integration of REST APIs with Northern Trust UI.Developed and maintained comprehensive Grafana dashboards for real-time monitoring and actionable insights, linked to Azure App Insights, SQL databases, and Managed Instances.Implemented robust data encryption strategies with SecupI and Collibra within Azure, aligning with regulatory compliance and data protection standards.Automated deployment processes using Azure pipelines, minimizing manual intervention and ensuring consistency across production environments.Conducted in-depth Azure log analysis for proactive performance monitoring and resource optimization.Established and managed CI/CD pipelines in Azure DevOps, streamlining the code deployment process and reducing downtime.Environment: Azure Data Factory, Databricks, Azure Function App, Azure Batch Service, Application Insights, Logic Apps, Azure Virtual Machines, Azure Storage Accounts, Azure Kubernetes Services (AKS), Docker, SQL Databases (ADLS, SQL Database), ETL Workflows, Grafana, Data Encryption (SecupI, Collibra), Azure Log Analysis, Alert Creation, Azure Monitoring, CI/CD Pipelines (Azure DevOps)Kaiser Permanente, OhioAWS Data Engineer November 2021 - October 2023Engineered and established an Enterprise Data Lake, catering to diverse use cases such as storage, processing, analytics, and reporting on large-scale, dynamic datasets using AWS services.Developed and optimized ETL processes using AWS Glue, transforming and migrating data from sources like S3, Redshift, and RDS.Automated data migration and processing pipelines leveraging AWS Lambda, Kinesis, and Database Migration Service (DMS), ensuring seamless integration and real-time data availability.Utilized AWS Glue DataBrew for data preparation, ensuring data quality, and AWS Athena for executing complex queries and driving business insights.Implemented real-time data processing and analytics using Kinesis Data Streams, Firehose, and Analytics, feeding data into S3, DynamoDB, and Redshift.Designed and managed PySpark jobs in AWS Glue for complex data transformations, optimizing performance and scalability.Established monitoring, alerts, and logging mechanisms using CloudWatch, enabling proactive issue detection and resolution across AWS services.Conducted a comprehensive architecture assessment and implementation of AWS services like EMR, Redshift, and S3, enhancing data processing and storage capabilities.Leveraged AWS DMS for seamless migration of databases from on-premise to cloud, ensuring high availability and minimal disruption.Delivered interactive reports and dashboards using AWS QuickSight, providing actionable insights for business decision-making.Environment: AWS Glue, S3, IAM, EC2, RDS, Redshift, EC2, Lambda, Boto3, DynamoDB, Apache Spark, Kinesis, Athena, Hive, Sqoop, PythonOpenText, Hyderabad, IndiaData Engineer May 2019 - May 2021Designed and implemented scalable data solutions on AWS, utilizing services like Amazon S3, Redshift, Glue, and EMR for efficient data storage, processing, and analysis.Built and optimized data pipelines using AWS Data Pipeline, Glue, and Databricks to streamline data ingestion, transformation, and loading from various sources.Developed big data processing workflows on Databricks, leveraging Apache Spark for high- performance data analytics and real-time processing.Managed Spark clusters on AWS Databricks, enhancing processing efficiency and reducing computation costs through optimized cluster configuration.Implemented streaming analytics and real-time data processing solutions using AWS Kinesis and Databricks Structured Streaming.Developed robust Spark applications using Scala for advanced data transformations, data exploration, and machine learning model development.Enabled seamless data ingestion and processing through Kafka integration, feeding data streams into Spark for analysis and real-time decision-making.Automated ETL processes on AWS using Talend and Glue, facilitating seamless data migration and transformation across environments.Implemented data warehousing solutions on Redshift, optimizing SQL scripts and query performance for large-scale data analytics.Proficiently managed data security through encryption, IAM policies, and network security controls, ensuring compliance with industry standards.Collaborated closely with cross-functional teams in Agile Scrum environments to deliver high- quality data solutions within defined timelines and budgets. Environment: Agile Scrum, Spark, Scala, Hive, Kafka, Python, AWS (EC2, S3, EBS, ELB, RDS, SNS, SQS, VPC, CloudFormation, CloudWatch, ELK Stack), Jenkins EDUCATIONUniversity of Memphis, Memphis, TNMasters in Information Systems Aug 2021 Dec 2022 GPA: 3.20/4.0Mallareddy Engineering College, Hyderabad, IndiaBachelors in Information Technology Aug 2017 May 2021 GPA: 3.1/4.0CERTIFICATIONSMicrosoft Certified: Azure Data Engineer Associate (May 2024) Python for Everybody (Coursera) (Dec 2019) |