Senior Data Engineer (Databricks)

Senior Data Engineer (Databricks)

• About the Role
• We are seeking an experienced Senior Data Engineer with strong hands-on expertise in Databricks to design, build, optimize, and support enterprise-scale data pipelines. This individual will play a key role in expanding and maturing our modern data platform, ensuring high-quality, reliable data is available for analytics, reporting, and downstream applications.

Responsibilities

• Design, develop, and maintain scalable, production-grade ETL/ELT pipelines using Databricks, PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows.
• Build and optimize data ingestion pipelines for structured, semi-structured, and streaming data across Bronze, Silver, and Gold layers.
• Develop reusable data engineering frameworks, transformation logic, and data quality validations.
• Leverage Databricks AI Genie and other native platform capabilities to improve development efficiency and accelerate delivery.
• Implement best practices for orchestration, monitoring, alerting, automation, and pipeline reliability.
• Optimize Spark jobs and SQL queries for scalability and performance.
• Design and implement dimensional data models (star and snowflake schemas) to support analytical workloads.
• Apply normalization and denormalization techniques where appropriate to balance performance and usability.
• Build physical data models aligned with Delta Lake and Medallion Architecture.
• Ensure data quality, integrity, governance, and compliance with enterprise standards.
• Collaborate with cross-functional teams to deliver scalable, high-performing data solutions.

• Required Qualifications
• 7–10+ years of experience in data engineering.
• Proven experience designing, building, and deploying enterprise-scale Databricks solutions.
• Strong expertise with Databricks, PySpark, Spark SQL, Delta Lake, Delta Live Tables, Unity Catalog, and Databricks Workflows.
• Strong understanding of Medallion Architecture (Bronze, Silver, and Gold).
• Advanced SQL development and performance tuning experience.
• Experience with AWS, Azure, or GCP.
• Strong understanding of modern lakehouse architecture and distributed data processing.
• Experience with CI/CD pipelines, Git, and job orchestration tools.
• Deep understanding of dimensional data modeling and data warehousing concepts.
• Experience developing scalable, maintainable ETL/ELT frameworks in production environments.

• Preferred Qualifications
• Experience working with streaming data pipelines.
• Familiarity with enterprise data governance and security best practices.
• Experience building reusable data engineering frameworks and automation solutions.
• Strong problem-solving skills with the ability to optimize large-scale data workloads.