Senior Data Engineer

• Role: Senior Data Engineer
• Location: Ottawa/Toronto (Hybrid)
• Must have : SQL+ Datawarehouse+ Banking domain
• Job Description:
• This position is for a Senior Data engineer with a background in Python, Pyspark, SQL and data warehousing for enterprise level systems. The position calls for someone that is comfortable working with business users along with business analyst expertise.
• Major Responsibilities:
• · Build and optimize data pipelines for efficient data ingestion, transformation and loading from various sources while ensuring data quality and integrity.
• · Design, develop, and deploy Spark program in databricks environment to process and analyze large volumes of data.
• · Experience of Delta Lake, DWH, Data Integration, Cloud, Design and Data Modelling.
• · Proficient in developing programs in Python and SQL
• · Experience with Data warehouse Dimensional data modeling.
• · Working with event based/streaming technologies to ingest and process data.
• · Working with structured, semi structured and unstructured data.
• · Optimize Databricks jobs for performance and scalability to handle big data workloads.
• · Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.
• · Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions.
• · Proficient writing SQL queries and programming including stored procedures and reverse engineering existing process.
• · Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards.

Skills:

• · 5+ years - SQL Server based development of large datasets
• · 5+ years with Experience with developing and deploying ETL pipelines using Databricks Pyspark.
• · Experience in any cloud data warehouse like Synapse, Big Query, Redshift, Snowflake.
• · Experience in Data warehousing - OLTP, OLAP, Dimensions, Facts, and Data modeling.
• · Previous experience leading an enterprise-wide Cloud Data Platform migration with strong architectural and design skills.

• Education
• • Minimally a BA degree within an engineering and/or computer science discipline
• • Master’s degree strongly preferred