Senior Data Engineer

• About the Company
• We're looking for a Senior Data Engineer to design, build, and operate modern data platforms at scale. You'll work across Azure, Microsoft Fabric, and AWS — building reliable pipelines, performant data models, and self-service analytics that drive real business decisions. A core part of this role is containerized workload execution: designing and deploying pipeline jobs using Azure Container Apps Jobs for scheduled and event-driven processing. This is a hands-on role for an engineer who thrives in ambiguity, values clean architecture, and moves comfortably between strategy and implementation.

• About the Role
• Data Platform & Architecture
• Design and build scalable data pipelines using Python and cloud-native orchestration tools (Azure Data Factory, Azure Container Apps Jobs, Fabric Data Pipelines)
• Architect solutions across Microsoft Fabric Warehouses, Azure SQL, and AWS (S3, Redshift), selecting the right tool for the workload
• Implement Medallion/layered architecture (Bronze → Silver → Gold) for structured, governed data delivery
• Manage and optimize large-scale data warehouse environments for performance, cost, and maintainability

Responsibilities

• Develop Python-based ETL/ELT pipelines ingesting data from APIs, flat files, databases, and SaaS platforms
• Build and deploy containerized jobs using Azure Container Apps Jobs, including scheduling, scaling rules, secrets management (Key Vault), and Container Registry integration
• Maintain data movement between on-premises SQL Server and cloud targets
• Design idempotent, fault-tolerant pipeline patterns with robust logging, alerting, and retry logic
• Partner with analytics teams to deliver clean, well-documented data models for Power BI and similar BI tools

Qualifications

• 5–8 years of hands-on data engineering in production environments
• Proven track record delivering data platforms on Azure and/or AWS
• Experience migrating legacy on-premises infrastructure to cloud-native solutions
• Hands-on experience with Azure Container Apps Jobs or a comparable containerized job execution platform

• Required Skills
• Python — production-grade pipelines, transformations, and automation
• SQL — advanced T-SQL and ANSI SQL; SQL Server, Azure SQL, Redshift, and Fabric query engines
• Microsoft Fabric — Warehouses, Lakehouses, Data Pipelines, OneLake
• Azure — ADF, Azure SQL, ADLS, Key Vault, Container Apps Jobs, Container Registry
• Containerization — Docker image development, container registry management, Container Apps Jobs (scheduling, scaling, secret injection)
• AWS — S3 for data lake storage, Redshift for cloud warehousing
• Data Modeling — dimensional modeling, star/snowflake schema, ER modeling for OLTP and OLAP
• DevOps — Git, GitHub, PR workflows, CI/CD pipelines
• Visualization — Power BI, Tableau

• Preferred Skills
• Microsoft Fabric in production (Fabric Warehouses, OneLake integration)
• dbt or similar transformation frameworks
• Power BI semantic model development, dataset optimization, DirectQuery/Import tradeoffs
• Agile/Scrum experience
• Cloud certifications: DP-203 (Azure Data Engineer), AWS Certified Data Analytics, or equivalent
• Bachelor's degree in Computer Science, Information Systems, Data Science, or a related field — or equivalent professional experience