Lead Data Engineer (Databricks)

• Databricks Lead / Architect
• Charlotte, NC/ New York
• 12 Months Contract

Job Description:

We are seeking an experienced Databricks Lead/Architect with strong expertise in CCAR (Comprehensive Capital Analysis and Review) and modern data engineering platforms. The ideal candidate will lead the design, development, and implementation of scalable data solutions using Databricks while supporting regulatory reporting and risk management initiatives within the banking and financial services domain.

Key Responsibilities:

• Design and architect enterprise-scale data solutions using Databricks.
• Lead data engineering teams in developing and optimizing data pipelines.
• Build and maintain ETL/ELT processes using Databricks, Spark, and Python.
• Work closely with Risk, Finance, and Regulatory Reporting teams to support CCAR requirements.
• Design data models and data quality frameworks for regulatory reporting.
• Optimize performance, scalability, and reliability of Databricks workloads.
• Collaborate with business stakeholders, architects, and development teams.
• Ensure compliance with banking regulatory and governance standards.
• Mentor junior engineers and provide technical leadership.

Required Skills:

• 10+ years of Data Engineering/Data Architecture experience.
• Strong hands-on experience with Databricks and Apache Spark.
• Experience with CCAR, Risk Management, Regulatory Reporting, or Finance Data.
• Proficiency in Python, SQL, and ETL development.
• Experience with cloud platforms (AWS, Azure, or GCP).
• Strong understanding of Data Warehousing and Data Lake architectures.
• Excellent communication and stakeholder management skills.

Preferred Skills:

• Experience with Delta Lake, Unity Catalog, and CI/CD pipelines.
• Knowledge of Basel, Stress Testing, or other regulatory reporting frameworks.
• Banking or Financial Services industry experience.