| | | 20,000+ Fresh Resumes Monthly | |
|
|
| | Related Resumes Senior big data engineer San Marcos, TX Big Data Developer(Scala, Python, Kafka, Databricks) Austin, TX Data Engineer Software Development Georgetown, TX Data Scientist Software Engineer Austin, TX Data Engineer Big San Antonio, TX Software Engineer Data Management Georgetown, TX Scrum Master Big Data Cedar Park, TX |
|
Click here or scroll down to respond to this candidateCandidate's Name
Big Data / Hadoop AdministratorEMAIL AVAILABLEPHONE NUMBER AVAILABLESUMMARYAround 8 years of IT experience as a Big Data / Hadoop Administrator, adaptively working/consulting on different Hadoop environments as per business requirements. Played roles of Senior Data Engineer and Senior Software Engineer.Worked on various Hadoop platforms such as CDH, CDP, Azure stack, and HDP.Certified in Az 900 Microsoft Azure fundamentals.Experienced in working on deployment, maintenance, and support projects.Experienced in working with several Hadoop-related services such as Kafka, HDFS, Yarn, Hive, Map Reduce, Hive, Livy, Tez, HBase, Spark, Knox, Kerberos, NIFI, Atlas, Ranger, ADLS, zeppelin, Kafka, Sentry, LDAP.Experienced in upgrading Hadoop clusters.Hardworking and willing to take initiative to work on blockers or difficult tasks.Excellent Mathematical skills and Logical Ability.Highly motivated to learn any software skills or tools which are required, on the go.Excellent Communication Skills and can quickly adapt to any situation.Efficient Researcher and an excellent Team Player.Excellent at documenting and maintaining run books for future reference.Good collaboration with several teams to share expertise and resolve issues quickly.Implemented automation for log rotations and sending alert emails in case of service failure on Hadoop clusters.Experienced in communicating with Vendors to resolve service-level bugs.Client-focused, good at understanding business needs and targets and to provide client acceptable solutions.Adhere to plan, quality, schedules of assignment and bring in innovative solutions.TECHNICAL SKILLSHadoop PlatformsCloudera, Hortonworks, Azure stackHadoop / Big Data ServicesKafka, HDFS, Yarn, Hive, Map Reduce, Hive, Tez, HBase, Spark, Knox, Kerberos, NIFI, Atlas, Ranger, ADLS, zeppelin, Sentry, Kafka, LDAP, Pig, ScoopCloud PlatformsAzure, AWS, GCPLanguagesPython, Java, SQL, Spark, XML, C, C++Version ControlGit, SVNOperating SystemsWindows, MacOS, Unix, Ubuntu, RHEL and several other flavors of LinuxNotebooksZeppelin, Jupyter, PyCharmAzure StackADB, ADF, Synapse, Purview, Immuta, Azure Active Directory, ADLSOther toolsPower BI, MS Office, Tableau, MATLAB, LaTeX, ADO, ServiceNow, DBeaver, Ansible, ODBC, Shell ScriptingPROFESSIONAL EXPERIENCEIMR Soft Jan 2023 Till DateClient: VISAProject #6: Data Streaming Platform Production SupportRole: Kafka / Hadoop AdministratorThe scope of this project is to maintain, monitor and support multiple Kafka and Hadoop clusters (HDP 3.1.5 / confluent) in VISA data streaming production environment.Roles and Responsibilities:Monitor the clusters using Grafana, Email alerts, and Incidents.Maintenance planning for security Patching and infrastructure changes.Analyze the existing clusters and provide cluster-hardening solutions.Attending deployment meetings where we are required and provide inputs as needed.Update Wiki Pages and maintenance runbooks along with issue analysis for major issues.Provide long-term fixes for recurring issues.Prepare maintenance calendars and handovers.Automate repetitive tasks/maintenances and integrate with an internal UI.Following up on pending issues with application/dev/external teams.Join swat calls in cases where there is downtime and resolve if there is any kind of platform issues on the go.Responsible for troubleshooting and providing solutions to Kafka and Hadoop platform issues.Responsible for OAC and HA testing to onboard new clusters.Larsen & Toubro Infotech Limited March 2021 Sep 2022Client: Ernst & Young Global LimitedProject #5: Trusted Data Fabric (TDF)Role: Hadoop Administrator DevOps SupportThe scope of the project (TDF) is to deploy, maintain and support several multi-tenant Hadoop Clusters for multiple clients to ingest, transform, visualize, and analyze the client or third-party data as per the business needs.Roles and Responsibilities:Debug / Troubleshoot any issues in Hadoop Environment (HDP 3.1.5) including services like HDFS, Yarn, Hive, Map Reduce, Hive, Tez, HBase, Spark, Knox, Kerberos, NIFI, Atlas, Ranger, ADLS, zeppelin.Troubleshoot access and data refresh-related issues in data visualization tools like DBeaver, and PowerBI.Installation of Python environments and packages as per requirement on Hadoop clusters.Renewing SSL certificates as part of maintenance to avoid any application access outages.Extensive calls with solution teams to consult on planning and deliverables.Monitor and resolve any service failure in Dev, UAT and Prod by using Grafana, Ambari, Cloudbreak and Azure Portal.Provide 24/7 support to tickets raised in ServiceNow and Azure DevOps.User access management to Git Repositories in Azure DevOps.Document and create wiki pages on debugging or troubleshooting procedures for future reference.SPOC for onboarding solution teams or clients onto TDF and setup the environment.User access management using Kerberos, LDAP, and Ranger.Consult the clients on any new tools integration and deployment with TDF environment.Code deployments in Dev, UAT and Prod as per client requirement.SPOC for code / pipeline promotions to higher environments.Extensive debugging of code or service failure in Hive, Spark and NIFI.Maintain optimal capacity management using YARN on Hadoop clusters to avoid any job failure or overloading issues.Perform log-level root cause analysis for issues within the scope of the project and provide a fix.Extensive calls with vendors (Cloudera and Microsoft) on any service level bugs to fix them on priority.Deploy / Patch the jar files provided by vendors as part of version upgrades or bug fixes.Redirect NIFI pipelines to different clusters as per requirement to avoid critical failures due to cluster consumption overload.ADLS access management, whitelisting IP address.Troubleshoot data ingestion-related issues and report RCA for the issue.Troubleshoot application Web UI failures for Zeppelin and NIFI.Hive Database / Table creation and tagging as per requirement.Resolve Hive table lock and table compaction issues.HDFS file system management.Implement automation ideas like log rotations, sending alerts in case of an outage or service failures, and migration of NIFI pipelines.Good knowledge of Synapse, Immuta, Purview, Azure Data Factory, and Azure Databricks.Perform Hadoop cluster deployments manually and by using Ansible Tower.Stop or decommission Hadoop clusters as per business needs.Attend scrum calls and team connects to discuss pending tickets to resolve any blockers.Connect with various internal teams like Networking team, Core DevOps team, Ingestion team, API teams, Infosec team, etc. to address any related issues and get their expertise.Provide inputs on impact analysis and facilitate the migration of current Hadoop clusters to Azure stack.Run GSL (Global Semantic Layer) pipelines as per the requirement to replicate metadata across regions.Analyze average cluster capacity usage by each client to plan for deploying new clusters as per necessity.Responsible for collaborating with different teams and clients to understand business expectations and resolve any queries of business.Larsen & Toubro Infotech Limited Oct 2020 March 2021Client: Ernst & Young Global LimitedProject #4: EY LensRole: Hadoop AdministratorThe Scope of the project is to plan, design, and deploy CDP clusters in Dev, UAT, and Prod. The project objective is to migrate from HDP to CDP with minimal impact on business.Roles and Responsibilities:Analyze the current HDP cluster CPU and Memory historical consumption to plan for designing CDP clusters.Compare and provide detailed behavioral changes and impact analysis reports for migrating to CDP.Provide design recommendations for CDP clusters.Provide detailed Network Security Recommendations for CDP clusters.Services include HDFS, YARN, Hive, Oozie, Spark, Kerberos, Kafka, NIFI, Ranger, Sentry, Knox.Deployed CDP cluster on AWS as a Proof of concept and to do testing.Presented a detailed plan to the client on the design and deployment plan for Dev, UAT, and Prod environments.Successfully deployed CDP clusters within the timeframe of the client and facilitated the data migration process.Post deployments test the CDP clusters for any unpredicted behavioral changes.Provide plan to decommission the HDP clusters after getting sign off from the client.Client: Helmerich & Payne, INC Dec 2018 Sep 2020Project #3: Data Driven AnalyticsRole: Hadoop AdministratorRoles and Responsibilities:Administration support for PROD cluster installed with CDH5X version.Responsible for all configuration changes and applying patches on the cluster.Performance tuning on MapReduce and setting up a cluster with HDFS and MapReduce.Configured Kerberos security to enable a Secure Hadoop cluster in the PROD/DEV environment.Configuring SQOOP to import data from external database MYSQL.Migrated data from the existing cluster to the newly built cluster through the dist-cp utility.Work with users to resolve issues related to access and jobs running on the cluster.Commissioning and decommissioning worker nodes.Imported/exported data from RDBMS to HDFS using Data Ingestion tools like Sqoop.Used Fair Scheduler to manage Map Reduce jobs so that each job gets roughly the same amount of CPU time.Recovering Nodes from node failures and troubleshooting common Hadoop cluster issues.Involved in creating Hive Internal/External tables, loading with data and troubleshooting with Hive jobs.Worked on configuring security for Hadoop Cluster, managing and scheduling jobs on a Hadoop Cluster.Experienced in managing and reviewing Hadoop log files.Managing nodes on Hadoop cluster connectivity and security.Client: Standard Chartered Bank Sep 2017 Nov 2018Project #2: Data ManageRole: Hadoop AdministratorRoles and Responsibilities:Worked on Provisioning, managing, and monitoring Hadoop clusters for Cloudera Distribution of Hadoop with the Cloudera Manager.Helping the Application team with job failure on MapReduce and spark application job.Configured various property files like core-site.xml, hdfs-site.xml, mapred-site.xml and hadoop-env.sh based upon the job requirement.Helping developers in importing and exporting data into HDFS using Sqoop.Worked on managing and reviewing Hadoop log files.Monitoring cluster performance using Cloudera Manager.Start/stop/restart different services both manually and from the Cloudera Manager Interface.Keeping the cluster/services in maintenance mode while stopping and upgrading.Supported MapReduce Programs running on the cluster. Involved in loading data from UNIX file system to HDFS.Document any changes made to configurations or services to maintain a record.Involved in creating Hive tables, loading data, and writing Hive queries.Client: PC World Computers PVT LTD Sep 2015 Aug 2017Project #1: L1 SupportRole: Linux Administrator supportRoles and Responsibilities:Working on a ticketing system using JIRA.Monitoring the performance of the applications on servers.Updating and troubleshooting the Linux OS and software-related problems Commissioning and decommissioning on servers.Preparing backup reports.Troubleshooting and system maintenance in Red Hat Linux Installation and configuration of RedHat Linux.Installation of Python environments and packages as required.Document process flow for users.Installation of various RPM packages.Remote administration using SSH. User management.Server health monitoring using PS, uptime, df h, free, etc.EDUCATIONBachelors in Electronics and Communication EngineeringCBIT, Osmania University (2015) |
