| | | 20,000+ Fresh Resumes Monthly | |
|
|
| |
Click here or scroll down to respond to this candidate AJEETHEmail: EMAIL AVAILABLE PH: PHONE NUMBER AVAILABLESr, Data EngineerProfessional Summary An Enthusiastic Data Engineer with specializing in Snowflake and Databricks to create, build, install, test, and maintain highly scalable data management systems, ensure that systems satisfy business needs, create high-performance algorithms, and develop data set processes. I'm seeking for opportunities to use my knowledge and abilities to develop innovative solutions for corporate clients.
Deployed Instances, provisioned EC2, S3 bucket, Configured Security groups and Hadoop eco system for Cloudera in AWS.
Experience in using distributed computing architectures like AWS products (e.g., EC2) and working on raw data migration to Amazon cloud into S3 and performed refined data processing.
Using snowflake cloud data warehouse and AWS S3 bucket to integrate data from multiple sources, including loading nested JSON formatted data into snowflake table.
Created AWS Lambda, EC2 instances provisioning on AWS environment and implemented security groups, administered Amazon VPC's.
Significant experience in software engineering and data engineering roles, with expertise in Snowflake and Databricks.
Perform analysis of large, complex data sets and assist with data management processes in academic research using Jupyter Notebooks with Apache Spark, Python, R, SAS, and Snowflake Datawarehouse on AWS.
Development and maintenance of data pipeline on Azure Analytics platform using Azure Databricks, PySpark, Python, Pandas and NumPy libraries.
Proficient in design and development of various dashboards, reports utilizing Tableau Visualizations like bar graphs, line diagrams, pareto charts, funnel charts, scatter plots, pie-charts, donut charts, bubble charts, funnel charts, heat maps, tree maps according to the end user requirements.
Designed and developed Power BI graphical and visualization solutions with business requirement documents and plans for creating interactive dashboards
Strong understanding of databases and big data software technologies, particularly Snowflake and Databricks. Worked on research and analysis on data sources in support of data discovery of OLAP cubes. Experience with data modeling, schema design, and SQL development.
Created databases on RDS and loaded the data from AWS S3 to RDS SQL Server. Experience with creating APIs and created security groups for accessing the APIs externally.
Converted the Code from Scala to PySpark in DHF (Data Harmonization Framework) AND Migrated the Code and DHF_UI from DHF 1.0 to DHF 2.1.
Analyzed the requirements and framed the business logic for the ETL process. Identified and designed process flows to transform data and populate the target databases.Professional ExperienceAWS Data EngineerMaximus, Denver, CO July 2022 to PresentResponsibilities:
Designed and developed Security Framework to provide fine grained access to objects in AWS S3 using AWS Lambda, DynamoDB.
Extracted data and load into HDFS using Sqoop commands and scheduling Map/Reduce jobs on Hadoop.
Implemented Lambda to configure Dynamo DB Autoscaling feature and implemented Data Access Layer to access AWS DynamoDB data.
Loaded data into S3 buckets using AWS Glue and PySpark. Involved in filtering data stored in S3 buckets using Elasticsearch and loaded data into Hive external tables.
Experience in Migrating existing databases from on-premises to AWS Redshift using various AWS Services.
Developed the PySpark code for AWS Glue Jobs and EMR.
Migrated an existing on-premises application to AWS, using services like EC2 and S3 for data processing and storage, leveraging Snowflake and Databricks for seamless migration and data processing. Migrated an existing on-premises application to AWS. Used AWS services like EC2 and S3 for data sets processing and storage.
Utilized Talend Administration Center (TAC), Job Server, and Studio for ETL processes. Was responsible for creating on-demand tables on S3 files using Lambda Functions and AWS Glue using Python and PySpark.
Transformed the data using AWS Glue dynamic frames with PySpark, catalogued the transformed the data using Crawlers and scheduled the job and crawler using workflow feature.
Used Data Integration to manage data with speed and scalability using the Apache Spark Engine and AWS Databricks.
Loaded data into S3 buckets using AWS Glue and PySpark, and filtered data stored in S3 buckets using Elasticsearch, demonstrating expertise in utilizing Snowflake and Databricks for data integration and processing. Experienced in designing, built, and deploying and utilizing almost all the AWS stack (Including EC2, S3,), focusing on high- availability, fault tolerance, and auto-scaling.
Extracted structured data from multiple relational data sources as Data Frames in Spark SQL on Databricks.
Responsible for loading data from the internal server and the Snowflake data warehouse into S3 buckets.
Performed the migration of large data sets to Databricks (Spark), create and administer cluster, load data, configure data pipelines, loading data from Oracle to Databricks.
Created Databrick notebooks to streamline and curate the data from various business use cases.
Worked on migrating existing on-premises application to AWS Redshift. Used AWS services like EC2 and S3 for processing and storage.
Assist with the development and review of technical and end user documentation including ETL workflows, research, and data analysis.
Used Informatica for jobs involving data transformation and integration, creating and implementing workflows to automate ETL procedures.Environment: AWS, Snowflake, Python SQL, PostgreSQL, PySpark, PL/SQL, UNIX Shell Script, EC2, Spark, Databricks, AWS GLUE, Redshift, ETL, Data Mining, Informatica, Talend.Data EngineerCentene Corporation St Louis, Missouri Aug 2020 to June 2022Responsibilities:
Responsible for validation of Target data in Data Warehouse which are Transformed, loaded using Hadoop Big data.
Implemented Informatica workflows for data extraction, transformation, and loading (ETL) processes to provide seamless connection with the Snowflake data warehouse. Validated target data in Data Warehouse transformed and loaded using Hadoop Big Data, demonstrating strong understanding of databases and big data software technologies, particularly Snowflake and Databricks. Designed the ETL process from various sources into Hadoop/HDFS for analysis and further processing of data modules.
Have good experience working with Azure BLOB and Data Lake storage and loading data into Azure SQL Synapse analytics (DW).
Worked on Amazon Web service (AWS) to integrate EMR with Spark and S3 storage and Snowflake.
Using Spark, performed various transformations and actions and the result data is saved back to HDFS from there to target database Snowflake.
Designed, developed, tested, implemented, and supported Data Warehousing ETL using Talend and Hadoop Technologies, incorporating Snowflake and Databricks for optimized data processing. Extensively worked with MySQL for identifying required tables and views to export into HDFS.
Engaged directly with IT to understand their key challenges and demonstrate and price solutions that fit their needs for PaaS and Iaas based solutions.
Responsible for creating Hive tables on top of HDFS and developed Hive Queries to analyze the data. Staged data by persisting to Hive and connected Tableau with Spark cluster and developed dashboards.
Setup databases in AWS using RDS and configured backups for S3 bucket, utilizing Snowflake and Databricks for efficient data storage and management. Design, develop, test, implement and support of Data Warehousing ETL using Talend and Hadoop Technologies and utilized Matplotlib and Seaborn for data visualization. Created Build and Release for multiple projects (modules) in production environment using Visual Studio Team Services (VSTS).
Have Knowledge regarding Stream sets which are pipelines used for the Injecting data into Raw layer from Oracle Source.
Used Terraform scripts to Automate Instances for Manual Instances that were launched before.
Environment: AWS, Snowflake, Python SQL, PostgreSQL, PySpark, PL/SQL, UNIX Shell Script, EC2, Spark, Databricks, AWS GLUE, Redshift, ETL, Data Mining, Informatica, Talend.Truist Bank, Charlotte, NC July 2018 to July 2020AWS/ Python DeveloperResponsibilities: Built models and tools to extract value from large amounts of data in the environment, utilizing Snowflake and Databricks for data analysis and processing. Creating and launching EC2 instances using AMI s of Linux, Ubuntu, Windows and wrote shell scripts to bootstrap instance. Built models and tools to extract value from large amounts of data in the environment, utilizing Snowflake and Databricks for data analysis and processing. Setup database in AWS using RDS and configuring backups for S3 bucket. Involved in Data Collection, Data Extraction, Data Pre-Processing, Feature Engineering, Dimensionality Reduction, Algorithm Implementation, Back Testing and Validation. Analyzed cloud infrastructure and recommended improvements for performance gains and cost efficiencies, incorporating Snowflake and Databricks for data-driven insights and optimizations. Imported reviews data by into Python environment using Beautiful Soup and re (regex) library in Python. Identified solutions to strategic business problems high-level modeling, statistical analysis techniques Developed complex jobs in Talend ETL for device analysis and maintained workflows. The model was deployed using Sagemaker(AWS) for production use. Worked and delivered results in an agile environment. Also Worked on Data Migration from IBM Netezza to AWS Redshift.Worked on shell scripting to Automate stored procedures in AWS Redshift environmentEnvironment: Python (NumPy, Pandas, Matplotlib, boto3, Hive, Tableau, AWS Sagemaker, AWS Lambda, AWS Cloud formation, AWS Redshift, shell scripting, Snowflake, Azure Databricks, Data Mining, Informatica, TalendPython DeveloperCybage Software Private Limited Hyd India Jan 2017 to June 2018Responsibilities:
Developed various MYSQL database queries from Python using Python-MySQL connector and MySQLdb package, integrating Snowflake and Databricks for efficient data querying and processing. Worked with tools like Jenkins to implement build automation.
Development of the company s internal CI system, providing a comprehensive API for CI/CD.
Associated with various phases of Software Development Life Cycle (SDLC) of the application like requirement gathering, Design, Analysis and Code development.
Worked with team of developers on Python applications for RISK management.
Generated Python Django Forms to record data of online users. Used Python and Django creating graphics, XML processing, data exchange and business logic implementation.
Designed and developed the UI of the website using HTML, XHTML, AJAX, CSS, and JavaScript. Developed and tested many features for dashboard using Python, Java, Bootstrap, CSS, JavaScript, and J Query.
Developed custom Jenkins jobs/pipelines that contained Bash shell scripts utilizing the AWS CLI to automate infrastructure provisioning.
Experience in writing Sub Queries, Stored Procedures, Triggers, Cursors, and Functions on My SQL and Post gre SQL database.
Wrote and executed various MYSQL database queries from python using Python-MySQL connector and My SQLdb package.
Provided financial deliverables for funding agencies, created dashboards, scorecards, views, pivot tables, and charts, utilizing Snowflake and Databricks for data-driven insights and reporting. Performed Compatibility testing of applications for dynamic and static content of browsers using HTML Ids and X Path in Selenium.
Worked on Integration of Selenium RC/Web Driver with existing API to test Framework.
Cleaned data and processed third party spending data into manoeuvrable deliverables within specific formats with Excel macros and python libraries. Used TDD (Test driven development) methodology.
Environment: Python, Django, Mongo DB, Snowflake, Pandas, Java, J Query, Zookeeper, Databricks, My SQL, Linux, Ajax, JavaScript, Apache, JIRA, Cassandra, HTML5 and CSS, Angular JS, Backbone JS.Data AnalystYana Software Private Limited Hyderabad, India July 2015 to Dec 2016Responsibilities:
One of the top five banks in the US is Wells Fargo. When it lends money out,the bank earns more money than when it borrows. Consumer banking and Lending, Commercial Banking, Corporate and Investment Banking, and Wealth & Investment Management are the four business segments operated by Wells Fargo. With Tableau desktop, engaged in the process of producing various reports in response to business requests and ad hoc requirements. Provide financial deliverables for funding agencies, including budgeting reports, proof of concept documents, and prototypes of solutions. For analysis, monitoring, management, and a better understanding of the business performance measures, create various dashboards and visualizations. For further data analysis, I created dashboards, scorecards, views, pivot tables, and charts. Establish and convey the project's scope, key deliverables, and anticipated client needs. Build solution-driven dashboards by creating several chart types in Tableau and Power BI Desktop, such as cross tabs, Heat/Geo/Tree maps, pie/bar charts/circle views/Lines/Area charts, Scatter plots, bullet Graphs, and Histograms. Connect to numerous data sources (Microsoft SQL Server, Oracle, MS Excel, CSV) to create reports and visualizations for users in Tableau and Power BI. With both Tableau and Excel, there are calculated fields, sorting, grouping, live connections, and in-memory features. Accountable for managing the delivery, gathering needs, and interacting with business stakeholders. The Financial Report SF 425 for federal funds should be prepared and submitted periodically. Create and maintain connections with important suppliers and evaluate their performance using information based on cost, quality, and service level indicators that are in line with client demands and business procurement rules and procedures. To enable sole source purchases, competitive source selections, and performance reporting, evaluate cost and pricing. Lead pricing meetings with project managers to price out projects, calculate incentive budgets, revenue sharing across teams and overall profit margins. Including cost analysis, estimates to completions (ETC), budgetary limitations, and trend analysis, management is responsible for ensuring the financial performance of projects. Used exclusive accounting software to normalize confidential financial accounts from private enterprises in accordance with GAAP requirements. To develop the organization's yearly operational plan, I oversaw the collection and analysis of multi-year business-line estimates. |
