Lead Databricks Engineer

Experience: 10-15 Years
 

 

About EXL

EXL is a leading global analytics and digital operations company that partners with organizations to drive business transformation through data, AI, and advanced technology solutions. We are looking for passionate Databricks Engineers who thrive in building modern, scalable, and high-performance data platforms.

Job Summary

We are seeking experienced Databricks Engineers with strong expertise in Databricks, PySpark, Python, and SQL to design, develop, and optimize enterprise-scale data engineering solutions. The ideal candidate will have hands-on experience building robust ETL/ELT pipelines, processing large datasets, and delivering high-quality data solutions in cloud environments.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks.
• Build high-performance data processing solutions using PySpark, Python, and SQL.
• Process and transform large volumes of structured and semi-structured data.
• Optimize Spark jobs, SQL queries, and data pipelines for performance, scalability, and reliability.
• Develop and maintain Delta Lake tables following modern data engineering best practices.
• Collaborate with cross-functional teams to understand business requirements and deliver data-driven solutions.
• Perform data validation, troubleshooting, root cause analysis, and production support.
• Follow coding standards, testing methodologies, CI/CD practices, and Agile development processes.

Required Skills

• Strong hands-on experience with Databricks
• Proficiency in PySpark, Python, and SQL
• Experience in designing and developing ETL/ELT pipelines
• Strong understanding of Apache Spark architecture and performance tuning
• Experience working with Delta Lake
• Excellent analytical, debugging, and problem-solving skills

Preferred Skills

• Experience with Azure Databricks or AWS Databricks
• Exposure to Azure Data Factory (ADF), Apache Airflow , or other cloud-based data platforms
• Knowledge of Git, CI/CD pipelines , and Agile methodologies

Experience: 10 –15 Years
Company: EXL