Job Title: Databricks Technical Lead
Location: Irvine, CA (Onsite)
Pay Rate: $65–$70/hr on W2
Experience: 10–15+ Years
Employment Type: W2 Contract
Job Summary
We are seeking an experienced Databricks Technical Lead to lead the design, development, and implementation of scalable data engineering solutions. The ideal candidate will have strong expertise in Databricks, Apache Spark, Python, SQL, cloud data platforms, and modern data architecture. This role requires technical leadership, hands-on development, performance optimization, and collaboration with business and technical stakeholders.
Key Responsibilities
Lead the design and development of scalable data pipelines using Databricks and Apache Spark.
Develop and optimize ETL/ELT workflows using Python, PySpark, and SQL.
Design and implement data lakehouse architectures and Delta Lake solutions.
Build, maintain, and optimize batch and streaming data processing pipelines.
Work with cloud platforms such as Azure or AWS to develop reliable data engineering solutions.
Implement data quality checks, error handling, monitoring, and performance tuning.
Lead technical discussions, code reviews, and solution design sessions.
Mentor developers and establish coding standards and best practices.
Collaborate with data architects, business analysts, and cross-functional teams to translate requirements into technical solutions.
Ensure data security, governance, scalability, and reliability across data platforms.
Required Skills
10–15+ years of experience in IT, data engineering, or data platform development.
Strong hands-on experience with Databricks and Apache Spark.
Expertise in Python, PySpark, and advanced SQL.
Experience with Delta Lake, Lakehouse architecture, and data modeling.
Strong understanding of ETL/ELT processes and large-scale data processing.
Experience developing and optimizing complex data pipelines.
Hands-on experience with Azure or AWS cloud data services.
Knowledge of Git, CI/CD pipelines, and automated deployment practices.
Strong troubleshooting, analytical, and performance-tuning skills.
Proven experience leading technical teams and delivering enterprise-level data solutions.
Preferred Qualifications
Experience with Databricks Unity Catalog, workflow orchestration, and governance.
Knowledge of data warehousing and modern data architecture patterns.
Experience with streaming technologies such as Kafka or Structured Streaming.
Familiarity with data security, access controls, and regulatory compliance.
Excellent communication, stakeholder management, and technical leadership skills.
Work Location
This is an onsite position in Irvine, California. Candidates must be willing to work onsite and be eligible for W2 employment.