Data Engineer - Intermediate

About The Project

Our client is looking for an experienced Data Product Analyst to support a major Government of Alberta initiative focused on modernizing environmental and natural resource regulatory processes through advanced data engineering and analytics.

As a Data Product Analyst, you will play a critical role in developing analytics-ready datasets, modern Azure-based data pipelines, and secure enterprise data products that support regulatory oversight and future AI initiatives.

Key Responsibilities

• Design and implement scalable, secure, and high-performance data architecture using Microsoft Azure.
• Develop and maintain enterprise data pipelines using Azure Data Factory, Azure Databricks, Delta Lake, and Azure Synapse Analytics.
• Build and manage Azure Data Lake Storage Gen2 solutions for structured and unstructured data.
• Design, optimize, and automate ETL/ELT processes for enterprise data integration.
• Integrate data from multiple enterprise systems including ServiceNow, APIs, geospatial platforms, and other external data sources.
• Develop robust data models and semantic layers for reporting, analytics, and machine learning.
• Write efficient Python and SQL code for data transformation, cleansing, enrichment, and automation.
• Design secure APIs and data services using Azure API Management.
• Implement data governance, metadata management, lineage, classification, and quality controls.
• Ensure compliance with privacy and security standards including FOIP, GDPR, encryption, masking, and role-based access controls.
• Monitor, troubleshoot, and optimize enterprise data pipelines for reliability and performance.
• Utilize AI-assisted development tools for code generation, testing, optimization, monitoring, and documentation.
• Design standardized, analytics-ready datasets to support advanced reporting and future AI use cases.
• Collaborate with Data Architects, business stakeholders, and cross-functional technical teams.

Must-Have Qualifications

• Post-secondary degree, diploma, or certificate in Computer Science or a related field.
• 5+ years of hands-on experience with Python and SQL for Data Engineering.
• 3+ years of experience building scalable data pipelines using Azure Databricks, Delta Lake, Azure Data Factory, and Azure Synapse Analytics.
• 3+ years of experience designing analytics-ready data platforms that support business intelligence and reporting.
• 3+ years of experience with Azure Infrastructure, Azure Services, and Azure Authentication.
• 2+ years of experience in data governance, metadata management, and security within Azure/Databricks environments.
• 4+ years of experience using GitHub/Git for version control and collaborative development.
• 1+ year of experience utilizing AI-driven code generation, analysis, and automation tools.

Nice-to-Have Qualifications

• Experience performing business requirements analysis for enterprise data initiatives.
• Strong knowledge of Microsoft SQL Server and database development.
• Experience with Microsoft Fabric.
• Experience building data products within the Government of Alberta (GoA) Cloud Environment.
• Experience designing scalable ETL pipelines.
• Experience designing and integrating RESTful APIs.
• Experience working with ServiceNow data integrations in Azure environments.
• Experience with scalable messaging and queueing technologies.
• Experience collaborating with cross-functional software development teams throughout the SDLC.