Job Summary
We are seeking an experienced Databricks Architect with proven hands-on experience leading or executing Greenplum-to-Databricks migration projects. The ideal candidate will have strong expertise in Databricks architecture, data migration, data warehouse modernization, and large-scale data platform transformations.
The candidate will be responsible for architecting and supporting the migration of data, workloads, ETL processes, and data pipelines from Greenplum to Databricks, ensuring a smooth, scalable, and efficient transition.
Key Responsibilities
Lead the architecture and technical execution of Greenplum-to-Databricks migration initiatives.
Assess existing Greenplum databases, schemas, tables, workloads, stored procedures, and ETL processes.
Develop migration strategies and roadmaps for transitioning data and workloads from Greenplum to Databricks.
Design scalable Databricks Lakehouse architecture to support migrated workloads.
Migrate and optimize data pipelines, SQL queries, ETL/ELT processes, and data models for the Databricks environment.
Identify migration challenges, dependencies, and performance considerations during the transition.
Ensure data accuracy, integrity, security, and performance throughout the migration process.
Work closely with data engineers, architects, developers, and business stakeholders to plan and execute migration activities.
Provide technical leadership and guidance on Databricks best practices and migration methodologies.
Troubleshoot and resolve migration-related issues and optimize migrated workloads for performance and scalability.
Develop technical documentation, migration plans, architecture diagrams, and implementation guidelines.
Required Skills & Qualifications
Mandatory: Proven experience migrating data and workloads from Greenplum to Databricks.
Strong experience as a Databricks Architect, Data Architect, or Senior Data Engineer.
Hands-on expertise with Databricks and Lakehouse architecture.
Strong knowledge of Apache Spark, PySpark, and SQL.
Experience with data warehouse modernization and large-scale data migration.
Strong understanding of ETL/ELT pipelines, data modeling, and data integration.
Experience analyzing and migrating Greenplum databases and workloads.
Experience with cloud platforms such as Azure, AWS, or GCP.
Strong problem-solving and communication skills.
Preferred Skills
Experience with Delta Lake and Unity Catalog.
Experience migrating Greenplum SQL, stored procedures, and ETL workloads to Databricks.
Knowledge of Databricks Workflows and data pipeline orchestration.
Experience with cloud data migration and modernization programs.
Experience with performance tuning and optimization of migrated workloads.

Data Architect

Similar Other Jobs

Data Modeler With Insurance - W2 only

Data Modeler/ McLean, VA (Onsite)- Contractor

Associate Director, Business Intelligence

Manager, Data Enablement

Enterprise Account Executive - US | AI-native Marketing Analytics SaaS | Remote (US)

General Superintendent (Data Center)