Quantcast

Big Data Engineer Resume Northville, MI
Resumes | Register

Candidate Information
Name Available: Register for Free
Title Big Data Engineer
Target Location US-MI-Northville
Email Available with paid plan
Phone Available with paid plan
20,000+ Fresh Resumes Monthly
    View Phone Numbers
    Receive Resume E-mail Alerts
    Post Jobs Free
    Link your Free Jobs Page
    ... and much more

Register on Jobvertise Free

Search 2 million Resumes
Keywords:
City or Zip:
Related Resumes
Click here or scroll down to respond to this candidate
Candidate's Name
Status: US CitizenPHONE NUMBER AVAILABLE; EMAIL AVAILABLEBig Data EngineerDesigner, Development, Test and Production Support of Big Data ApplicationsA collaborative engineering professional with 17 years of substantial experience in designing, developing, executing and supporting applications for complex business problems involving big data solutions, data warehousing, near to real-time Analytics and BI solutions. Known for using the right tools when and where they make sense and created an intuitive architecture that helped organizations effectively analyze data and create Machine Learning (ML)/Artificial Intelligence (AI) solutions to enhance business opportunities.Core CompetencyData Lake (CDP):Proven history of building data processing systems and serving as an expert in data warehousing solutions while working with a variety of database technologies, structure and unstructured files (social media, campaigns & inventory), vsam files, flat files designing and optimizing while loading into data lake/data warehouses/data marts and HIPAA X12 EDI transaction setsExperience architecting data models of highly scalable, distributed systems using different tools as well as designing and optimizing down streams. Able to integrate and convert to Big Data technologies as team-member and also lead Team.Data warehousing:BI/Analytics:Tools and Technologies:Database and Tools: HDFS, NoSQL, Hue, Jupyter Notebook, Python, Pypark, Teradata Studio, HDFS, Apache Hive, Kudu, Impala, Informatica (Power Center, BDM Developer, Power Exchange, IDQ), DataStage, Alteryx, Talend, HQL, Sqoop, Cloudera, Collibra, Data Mapping tool, Erwin & Power Designer, Power BI, Cognos (certified), Alteryx, Tableau, Oracle, MS SQL Server, Teradata, DB2, Netezza,AWS Cloud,Redshift,EMR,EC2.Jupyter Notebook.Data Analysis: Consulted with business partners and data architects and made recommendations to improve the effectiveness of Big Data systems for descriptive analytics systems. Integrated new tools and developed technology frameworks/prototypes to accelerate the data integration process and empower the deployment into Hadoop data lake. Identify gap analysis in the existing data models in different environments in order to integrate into data lake. Data visualizing of the data with goal of discovering useful information, informing and supporting decision making.Tools:Data Transformation: Experience hands-on designing high level & low design documents Mapping documents, design data flow diagram and process improvements for the applications by implementing and optimizing data transformation processes in the Hadoop and Informatica ecosystems. Able to consolidate, validate and cleanse data from a vast range of sources- from applications and databases to files and validate the target data once loaded applying Audit Base Control (Reconciliation) and process restart ability.Data Collection/Catalog/Mapping: Capable of extracting the data from existing different databases/files. Experience in designing and implementing fast and efficient data acquisition using Big Data processing techniques and tools and quickly discover data that matters so that can generate impactful insights that drive business value. Using Mapping Tools to help business and technology data mappers to identify the upstream and downstream attributes in order to identify if any transformation logic applied. Using the catalog tool identifying the fully managed and highly scalable data discovery and metadata management service.Data Model: Designed the Logical and physical data mart models to organize the elements of data and standardize how they relate to other entities.Business Intelligence: Designed and Developed reports for data analysis andData Visualization for different strategies, data analysis, reports and dashboarddevelopment to help organizations to make more data-driven decisions.Scrum: Played as team member focus on creating stories that meet the Definition of Done, causing the removal of impediments to the team process ensuring all theevents take place and are positive, productive and kept within the time box sprintcompletion. Backlog grooming, Retrospective and Informing the stakeholders forupcoming production deployment and feedback after deployment.Tools: Jira, Microsoft Team.Strong communication skills, including the ability to convey analytic insights effectively to both IT and business team.Past experience leading the team onsite and offshore to deliver against the goals in the data environment.Exposure to Cloud envs (AWS & Microsoft azure) and hands on NoSQL databasesPast experience leading as Team Lead and training/mentoring less experienced personnelDocumentation for data solutions (data models, data dictionaries, data catalog, data process flows and architecture diagrams)EDUCATION:Bachelor of Engineering  Mechanical -1990Kavikulguru Institute of Technology and Science, Ramtek, Maharashtra, IndiaPost Graduate Diploma in Personal Management & Industrial Relations.1992Kakatiya University, Warangal, Telangana, India.TRAINING AND CERTIFICATESTraining Informatica certificate on Cloud Mapping Designer basic transformation.Training Informatica certificate on Data Quality Running DQ Exception in Hadoop.Training Informatica certificate on Cloud Data warehouse and cloud data lake.Informatica Certification on Cloud Integration for Application ModernizationInformatica Certification on Cloud Lakehouse Data Management Foundation.Training Certificate on Power Center Developer II from InformaticaTraining Cloudera certificate on Data Analyst, & Developer training on Spark & Hadoop.Training Talend certificate on Talend Studio for Data Integration- ELT.IBM Cognos Certified BI Authors & Training IBM certificate on DataStageTraining certificate on ERP Systems (Baan -Logistics) trained from NITIE, Mumbai, India.Training workshop on Scrum Alliance from http://3back.comWorkshops on Data marts by Ralph Kimball and AWS ProductsPython Boot Camp & Cassandra Workshop- 8 weeks programmed hands on experience for data analysis.Big Data University Courses & Certifications Awarded.(i)Hadoop & HBase for Real-time Access to Big Data certification. (ii)Accessing Hadoop Data Using Hive certification (iii) Moving Data into Hadoop certification. (iv)Big Data, MapReduce and Yarn certification (v) Training and pass certification in python 101.GM Learning & training certifications(i) Installation of Hadoop clusters and cloudera Manager(ii) Ecosystem for Hadoop (iii)Apache Hadoop (iv)Data Factory with Hive, Pig, Ozzie & Hue (v) Data Flow for the Hadoop Ecosystem (vi) Data Repository with Sqoop.Agile Project Management with Scrum.Training Netezza certification of Completion  NPS Appliance Usage 5.0Training MapR certificate on Essentials & Data Analysts courses.Training certificate from Pluralsight on Snowflake and AWS ProductsTraining certificate from plural sight in Microsoft Azure fundamental course AZ-900AWS Data bricks cloud data lake Development workshop.RECOGNITIONRecognized as Best Employee from IT Department -- BBVA Compass -2009.Professional ExperienceAvaility.com. April 2021-PresentData Service Providers facilitates billions of clinical, administrative, and financial transactions annually with tools for patient eligibility and benefits, claims status and authorization to facilitate the business side of health care for providers, health insurance and Members.Responsibilities:--Data Analysis, Daily handling Data Reconciliation process, loading the History data and patching missing data in different domains (Claims, Claim status, HL7, Eligibility Benefits, Transactions, ERA and HCSR)--DLQ (Dead Letter Queue data  error handling data analysis) and Patching scheduling on DAG Airflow schedular.---Loaded the history data into different stages in AWS S3 buckets and finally into Redshift database---Handling the Billing transactions to bill to the customers monthly for all the Domains.---Production Support Jobs on AWS Cloud -Apache DAG Airflow, Informatica Power Center -Batch and Realtime Applications.---Development ETL Informatica mappings/sessions/Worklets/Workflows and scheduling on Jams schedular tool.Highlights:Production support, Design, Development, ETL Mapping document, Data analysis, Bug fixing issues, Data Quality and Data patching process.Using Informatica PowerCenter, RDBMS (Oracle, SQL Server), AWS Cloud (Redshift, EMR, EC2,S3,DynamoDB,CloudShell)Jupyter Notebook), Python, pyspark and Scala (Knowledge)Knowledges on Health HIPAA X12 EDI transaction sets.M&T Bank: Technical Engineer: October 2019- April 2021Credit Risk System (CRS): In-progressCredit Risk System for Commercial Loans that currently exists in Microsoft SQL Server database. Business business wanted to migrate into Big Data Platform for Machine Learning and Artificial Intelligence solutions.Highlights:Involved in current (understand /design/development/load process/), hands on development and reviewed code and support.Core bdm mappings (take recent and gm projects in nice way),Social media, customer experience,OEM,Inventory,Data Gap analysis on existing data mart and data lake data to identify the entities, attributes, analyzing and profiling, analysis on the raw data for important decisions by identifying various facts and trends for downstream.Creating Data Model in different Layers (raw/stage/int/consumer and extract)Design and develop Big Data applications for processing commercial loans data.Integrated Hadoop into traditional ETL, accelerating the extraction, transformation, and loading into different layers and aggregate data creating the Files for stakeholders.Scheduling and creating workflows and objects.Customer Profitability: CDP/BDM UpgradeExisting application integrated different sources: commercial loans, small business, direct deposits, dealer floor plan and Mortgage loans in Hadoop ecosystem built on Cloudera platform with BDM informatica for downstream stakeholders reports and for OFSAA applicationHighlights:Coordinating with the Production team for source daily ebcdic files into downstream envs (dev/test/cert/) to test Jobs.Coordinating with the Production team for source daily ebcdic files into downstream envs (dev/test/cert/) to test Jobs.Hands on executing BDM mappings, validating the data at each individual stages with production w.r.t specific business date.Optimized ETL Jobs to improve the performanceData validation coordinating with the Data Analysts team for approval to deploy.Created change control process for production deployment Support Level 1 & 2 proactive monitoring daily batch jobs, resolving the tickets raised meeting the SLAs and informing if any issues to stakeholders.Accountable to deliver the applications in Agile environment on time delivery through agile methodology principles.Problem solving mindset working in an agile environment.Knowledge sharing among peers for process improvement.Credit Lifecycle Management (CLM): Operational Data Store (ODS):This application is integrated up streams daily batch files processing into Oracle, Teradata databases and CRM Sales force with all the financial credit information (commercial, retail and construction loans) in order to track and monitor downstream reporting in order to improve the business.Highlights:Identified gaps in the data processes and drove the improvements via continuous improvement loop (Data process) for performance and scalability.Worked on the transformation logic (bug fixes), in the existing jobs.Data analysis for new source to integrate, providing the details to Business ownerDesigned the Data Models at different stages.Data flow diagram, Design document and ETL mapping specification coordinating with Data Analyst team.Designed, development, test and deploying into production.Change control process for production deployment.GENERAL MOTORS: April 2016- September 2019IT Data Engineer/Data Intelligence AnalystMarket Investment Optimization (MIO):Migrating into the Hadoop ecosystem from existing Microsoft access and Excel. based on the external vendors landing the files for Marketing strategy analysis for the stakeholders reporting based on running the quarterly campaigns.Highlights:Integrated into Hadoop ecosystem using Apache HiveQL into traditional ETL, accelerating the extraction, transformation, and loading of the massive structured and unstructured Google, JD Edwards & Adobe data (social media and campaigns -impression, clicks and execution) into HDFS using Hive Data warehouse tables into different layers Raw & Staging.Data analysis by improving data quality, reliability and efficiency of individual source components for downstream processes.Created Models in different layers and ensured architecture meets the business requirements in HDFS and Hive tables.Loaded the aggregate data into a reporting Layer for reporting and ad-hoc analysis, which to improve processes for low latency and operating costs and offset the rising cost of existing programming.High- and Low-Level design frameworks, data flow diagram and ETL Mapping Specification, Unit Testing Templates.Worked closely with data architects, solution architects business and data analystsWeekly Meetings with Stakeholders and followed Agile process for any changesGlobal Business Service (GBS):Ad-hoc projects for Analytics.Highlights:Design, development, Testing and production support with Design documents (ETL Mapping specifications) for the below projects worked in Agile Methodology.Indirect material Inventory: Ingested and integrated from all 54 for plants North America Region process loading into Teradata/Aster Database and HDFS to Hive and query using BigSQL and also using IBM Big insights web console for import and export Files into HDFS for Analytical forecasting patterns.Dealer Optimization analysis (DrMo) monthly North America sales analysis & CRM MavenRecall- Effectiveness: Reporting the vehicle warranty claims information as per campaigns, type of service labor operation occurred on vehicle at dealer shop in order gauge the different level of campaigns.Dealer Risk Management and Optimization: Identifying incentive and warranty program, sales and service provided by the dealers for cars.GENERAL MOTORS: May 2013- March 2016Global Warranty Management (GWM):Upgrade project for Dealers claim charges for warranty vehicles for any vehicle related repairs/maintenance, integrating with upstream Files, SAP-IDOC and Oracle databases using the DataStage ETL daily batch process.Highlights: Technical LeadImplemented migration for several applications from See-beyond to DataStage 11.3 by providing guidance to team members on all aspects of software development compliance with design technical frameworks and enabling the coding standards.Provided accurate estimates, identify dependencies and risks and own all development activities.Support cross function activities involving data governance and analysis, mapping document specifications, functional and test cases templatesInstall Operating systems (Solaris, Linux), data stage software installation, databases, setup SAP integration and exciting hands on ETL jobs by comparing with existing and new platforms for downstream deployment into test/pre-prod and production.Collaboration with stakeholders, report progress and escalate any issues to their attentionEngage SME for upstream and downstream interfaces to complete UAT for deployment into the Prod environment.ITAC SOLUTIONS, Birmingham, AL: March2011-May2013Client: BBVA Compass Bank, Birmingham, ALHighlights: ETL Consultant: Individual and Team Member contributor in Setting up the dev principles and delivery standards. Designed, build ETL Mapping spec, Testing & supporting the ETL Deployed jobs for below projects:(i)Enterprise Information System (EIS)  DM (ii)Basel 3-New sources Integration into EDW. (iii) CRM-DM (iv)Non-Consumer Revenue Portfolio-DM (v) E3 (Mortgage & Insurance) -DM (vi)Conversion Project Windows to AIX ETL Jobs.ROSE INTERNATIONAL, Jefferson City, MO: Sept 2010March 2011Client: Bank of America, Charlotte, NCHighlights: Technical AnalystTechnology Conversion Project: Enterprise Credit Risk Project for POC using tool: Talend, Data Quality Mpx (ELT), Netezza (Database), Informatica (Power Center), Subversion, Perforce Tools.BBVA COMPASS BANK, Birmingham, AL: March 2008  Sept 2010Highlights: Sr. Lead Programmer Analyst /ETL ArchitectDesigned, Development and Administration tasks for different applications.Analysis on the source system and created etl mapping specifications and if any enhancements for the existing source systems.Requirements review, design of ETL process flow designs/architecture/framework, track progress, code/design documents to ensure data quality to meet standardsTrack of production ETL loads, identify SLA issues and inefficient jobs and metadata daily/weekly/monthly reports.Administrative activities for Informatica (Power Center) integration and Repository Services, Informatica IDQ support on AIX, Solaris and windows-support installation and applying hotfixes.New Development/Support/Maintenance for Data Marts for Credit Risk (CRDB), Collections Data Mart, Credit Origination and Navigation, Retail Incentive (RTLINC), Anti Money Laundering (AML-DIS Files to Mantas), Visa Payroll source into EDW, Treasury Management System (TMS), SBA and CPS.DAIMLERCHRYSLER FINANCIAL SERVICES, Detroit, MIMarch 2003  March 2008Highlights: ETL ConsultantSPOT (Single Point of Truth)  Data WarehouseDesign, Development, testing using ETL in order to consolidate reports from data stored in over 50 different operations systems for DCS business leaders and key decision-makers throughout North and South America.Production Support for daily batch Jobs.MBC Commercial Vehicle MORIS
(Management Operational Reporting Intranet Solution) Data Mart ProjectCreated data flow diagram & ETL Mapping specification document.Designed, development, testing data validation and support the application taking care of complete ETL ProcessCreated change request process for deployment to Dev to test to Prod env.,Worked closely with stakeholders, BI Team, solution architect and Data Architect.Dealer Profitability  Truck Finance  Data Mart ProjectDeveloped a process for operations management to better analyze the profitability of their Portfolios use to enhance managements ability to price business profitability from a different perspective, allowing management to identify top sales performers.Opti load Conversion  Data Warehouse:Since its evolution in DCS organization, Opti load replacement to replace the current Production Opti load jobs with Informatica Power Center / Power Connect (Striva) tools.Highlights:Requirements gathering and designing Logical and Physical Model using Power Designer 9.0Designed Star Schema Model for Dealer ROE Project and comprehensive Data Dictionary Documents. ETL Mapping Specification Documents for Development.Developed hands on mappings (@200+) / mapplets using Informatica  Power center (Designer) sessions & workflow using Workflow Manager.Involved in testing of mappings, which developed extensively using Debugger.Informatica Administrative (users, Groups and Privileges, folder organization and backups), Documented the Operational Manuals for the developedInvolved in Installation / Upgrade of software Installation PowerCenter / Power Exchange 5.1.2.

Respond to this candidate
Your Message
Please type the code shown in the image:

Note: Responding to this resume will create an account on our partner site postjobfree.com
Register for Free on Jobvertise