Azure Databricks Engineer

responsibilities

Build end-to-end direct capabilities.

Create and maintain optimal data pipeline architecture.

Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources

Use analytics for capitalizing on the data for making decisions and achieving better outcomes for the business.

Derive insights to differentiate member and team member experiences.

Collaborate with cross-functional teams.

Analyze and define with product teams the data migration and data integration strategies.

Apply experience in analytics, data visualization, and modeling to find solutions for a variety of business and technical problems.

Querying and analyzing small and large data sets to discover patterns and deliver meaningful insights.

Integrate source systems with information management solutions and target systems for automated migration processes.

Create proofs-of-concept to demonstrate the viability of solutions under consideration.

#qualification

10 years of professional software or data engineering experience.

5 years of strong experience with Azure services, including Azure Data Factory and Databricks.

Hands on Experience on PySpark and Python.

Hands on Experience on SQL.

1 year of professional Apache Spark data processing application design and development experience.

Understanding of Lakehouse Architecture is required.

Solid foundation in data structures, algorithms, design patterns, and strong analytical and problemsolving skills.

Previous Client Interaction Experience

Nice to have Scala but not required.

Azure App Services/Function App development experience with .NET/C# is a plus.

Interested candidates can share your updated resume on 8669124373

Regards

HR Mrunal