Data Engineer

MUST:

• 5 years minimum as data engineer
• Very strong with pipelines with one of these: Synapse OR Azure data factory/data bricks
• Very strong with writing notebooks and very strong with spark
• Very strong with data Lakehouse development
• Understands how to build out a Datawarehouse, temp tables
• Strong coding in either python or sql
• Understand Source code, git for source code
• Experience with either: Azure devops or Jira
• proven experience building robust scalable ETL/ELT pipelines
• Comes from a regulated environment – banking is ideal
• Experience working with various sources for data: with SFTP integrations, api’s, direct database connections

PLUS:

• Powerbi exposure
• Fabric- nice to have from ETL standpoint
• Banking

DAY TO DAY:

Our client is building out a new “align team” to help build a new datawarehouse. The Senior Data Engineer will build, develop, and maintain robust, scalable ETL/ELT pipelines using Azure Synapse, Azure Data Factory, Databricks, Spark, Python, and SQL. The Data Engineer will write and optimize Spark notebooks to transform and process large volumes of data within a data lakehouse environment. The Data Engineer will design and develop data warehouse solutions, including data models, temporary tables, data transformations, and integrations. This person will collaborate with business and operations teams to understand requirements and develop scalable data engineering solutions that support business needs.

You will participate in the full database development lifecycle, including design, development, testing, deployment, and ongoing maintenance. The Data Engineer will develop and maintain integrations using SFTP, APIs, and direct database connections while ensuring data quality, security, scalability, and reliability. The Data Engineer will use Git and Azure DevOps or Jira to manage source code, track development work, and support development processes. The Data Engineer will troubleshoot and optimize pipelines, contribute to database development standards and best practices, and work within a regulated environment to ensure compliance with applicable requirements. The Data Engineer will also mentor and provide technical guidance to fellow data engineers and may support Power BI and Microsoft Fabric initiatives as needed.