Job Role: Data Scala Engineering
Location: NYC, NY - In-person Interview
Duration; 12 Months
Job Description:
lead the migration of complex regulatory reporting pipelines from legacy ETL, Autosys, SQL, and Teradata platforms to a modern Databricks and Snowflake environment on Azure. The ideal candidate will have strong expertise in Scala, Spark, Databricks, Snowflake, SQL, and Airflow, with proven experience delivering large-scale cloud data migration projects.
Core Data Engineering
Scala (strong production development experience)
Apache Spark (DataFrame API, joins, window functions, partitioning, optimization)
Databricks (Serverless Compute, Unity Catalog, Asset Bundles, CLI)
Snowflake (schema design, Spark-Snowflake connector, performance tuning)
Advanced SQL (complex script analysis and migration)

Responsibilities include designing, developing, testing, and reconciling production data pipelines, ensuring parity between legacy and target platforms, implementing data quality controls, and supporting CI/CD and DevOps best practices. Candidates should have experience with Databricks (Unity Catalog, Asset Bundles, CLI), Snowflake, Azure services, Airflow orchestration, automated testing (ScalaTest/TDD), and data reconciliation frameworks.
Preferred candidates will have prior experience in financial services/regulatory reporting, legacy ETL modernization (Autosys/Informatica/Teradata), and modern data engineering practices such as Medallion Architecture, lineage, schema evolution, observability, and idempotent pipelines.

Data Scala Engineer

Similar Engineering Jobs