Senior Data Engineer

• Job Title: Data Engineer (Azure | Databricks | PySpark | Snowflake)
• Location: Plano, TX (Hybrid/Onsite as per client requirement)
• Duration: Long-Term Contract
• Client: USAA / IBM

• Job Description
• We are seeking an experienced Data Engineer to join our team supporting enterprise-scale data engineering initiatives for a leading financial services client.
• The ideal candidate will have strong experience building scalable cloud-based data platforms using Microsoft Azure technologies, Databricks, PySpark, and Snowflake. You will design and develop high-performance ETL/ELT pipelines, optimize large-scale Spark workloads, and deliver analytics-ready datasets supporting business intelligence, reporting, and advanced analytics.

Responsibilities

• Design and develop scalable ETL/ELT pipelines using Azure Data Factory (ADF)
• Build high-performance data processing solutions using Azure Databricks and PySpark
• Develop and optimize Delta Lake architectures using Medallion Architecture
• Build and maintain Azure Synapse Analytics solutions
• Develop data pipelines using ADLS Gen2
• Design and optimize Snowflake data warehouse solutions
• Improve Spark performance using partitioning, caching, broadcast joins, AQE, and Z-Ordering
• Develop reusable frameworks using Python and SQL
• Implement CI/CD pipelines using Azure DevOps and Git
• Work with business analysts, architects, and data scientists to deliver production-ready data solutions
• Support Agile development and production deployments

• Required Skills
• Azure Data Factory (ADF)
• Azure Databricks
• PySpark
• Apache Spark
• Azure Synapse Analytics
• Azure Data Lake Storage (ADLS Gen2)
• Snowflake
• SQL
• Python
• Delta Lake
• Medallion Architecture
• Structured Streaming
• Spark SQL
• Azure DevOps
• Git
• ETL / ELT
• Data Warehousing
• Performance Tuning
• Data Modeling