Senior Azure Data Engineer

Job Title:Azure Data Engineer 8 years

Location:New York - Onsite

Employment Type:Full-Time

About the Role:We are seeking a highly skilled and experienced Azure Data Engineer to design, develop, implement, and maintain scalable cloud-based data engineering solutions using the Microsoft Azure ecosystem.

Key Responsibilities:

• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory (ADF), Azure Databricks, Python, PySpark, and SQL.
• Develop and optimize cloud-based data solutions using Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, Azure SQL Database, and Azure Blob Storage.
• Build robust batch and real-time data pipelines for ingesting, transforming, cleansing, and integrating structured and unstructured data from multiple sources.
• Develop complex SQL queries, stored procedures, views, functions, and performance optimization strategies for large-scale data processing and analytics.
• Implement data transformation and processing frameworks using Databricks, Apache Spark, PySpark, and distributed computing technologies.
• Design and implement data warehouse anddimensional modeling solutions , including Star Schema, Snowflake Schema, Fact and Dimension tables, SCD Type 1/Type 2,and data marts.
• Perform data integration across relational databases, APIs, flat files, SaaS applications, enterprise systems, and cloud data sources.
• Implement data quality, validation, reconciliation, error handling, logging, monitoring, and alerting mechanisms across enterprise data pipelines.
• Develop reusable and parameterized Azure Data Factory pipelines, including triggers, linked services, datasets, integration runtimes, and pipeline orchestration.
• Implement CI/CD and DevOps practices data engineering solutions using Azure DevOps, Git, YAML pipelines,and automated deployment processes.
• Apply Azure security, governance , access control, encryption, and data protection best practices using technologies such as Azure Key Vault, Microsoft Entra ID, RBAC, and managed identities.

Required Skills \& Qualifications:

• 8 years of experience in Data Engineering,Data Warehousing, ETL/ELT, and cloud-based data platforms, with strong hands-on experience in Microsoft Azure.
• Strong expertise in Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2), Azure SQL Database, and Azure Blob Storage.
• Advanced proficiency in SQL and experience with query optimization, stored procedures, views, functions, indexing, and performance tuning.
• Strong programming experience with Python andPySpark, including data processing, transformation, automation, and distributed computing.
• Hands-on experience developing ETL/ELT pipelines, data ingestion frameworks, data integration, data transformation, and pipeline orchestration.
• Strong knowledge of Apache Spark, Databricks, Delta Lake, Lakehouse Architecture, andlarge-scale data processing.
• Experience with Data Warehousing, Dimensional Modeling, Star Schema, Snowflake Schema, Fact/Dimension modeling, SCD Type 1/Type 2, andenterprise data architecture.