Data Engineer

Job Summary:

The Data Engineer will play a crucial role in designing, building, and maintaining our data infrastructure. This position will leverage expertise in Python, PySpark , and SQL to streamline data flows and enable advanced analytics, ultimately driving data-driven decision-making across the organization.

Key Responsibilities:

• Design and implement robust data pipelines using Python, PySpark, and SQL to support various analytics and reporting needs.
• Collaborate with data scientists and analysts to understand data requirements and ensure data quality and integrity.
• Utilize Azure Synapse to optimize data storage solutions and enhance data processing capabilities.
• Develop and manage ETL processes using Databricks to efficiently transform and load data from multiple sources.
• Monitor and troubleshoot data workflows to ensure seamless operation and prompt resolution of issues.
• Document data processes and maintain metadata repositories to enhance data governance practices.

Requirements:

• Minimum of 8 years of experience in data engineering or related roles.
• Proficiency in Python and PySpark for building scalable data pipelines.
• Strong SQL skills for data manipulation and querying.
• Experience with cloud platforms, particularly Azure, including Azure Synapse Analytics.
• Familiarity with data architecture and design principles.
• Ability to work collaboratively in a fast-paced environment with cross-functional teams.
• Strong analytical skills and attention to detail.