Databricks Data Engineer (Only USC)

• Job Title :- Databricks Data Engineer
• Location :- Dallas, TX/ Phoenix, AZ

Note :- We need only USC for this position.

• Note :- Must Have any one of these certifications
• DataBricks Associates
• DataBricks Professional

• Role Summary
• A Data Engineer responsible for designing, building, and maintaining scalable data pipelines and ETL/ELT workflows on the Databricks platform, enabling reliable, high-quality data for analytics, reporting, and downstream business use.

• Key Responsibilities
• Design, develop, and optimize data pipelines using Databricks and Apache Spark.
• Build and maintain ETL/ELT workflows to ingest data from multiple structured/unstructured sources.
• Implement data models and pipelines using the Medallion Architecture (Bronze/Silver/Gold layers).
• Work with Delta Lake for data versioning, ACID transactions, and schema management.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.
• Ensure data quality, governance, and security across pipelines.
• Monitor, troubleshoot, and resolve pipeline failures and performance bottlenecks.
• Implement CI/CD practices for data pipeline deployment.
• Document data flows, architecture, and technical processes.

• Required Experience & Skills
• Strong hands-on experience with Databricks and Apache Spark (PySpark/Scala).
• Proficiency in SQL and data modeling concepts.
• Experience with Delta Lake and Medallion Architecture.
• Experience with a cloud platform (Azure, AWS, or GCP) — Databricks-native services (e.g., Azure Data Factory, AWS Glue).
• Programming skills in Python (and/or Scala).
• Experience with workflow orchestration tools (e.g., Databricks Workflows, Airflow).
• Understanding of data warehousing concepts and ETL/ELT design patterns.
• Experience with version control (Git) and CI/CD pipelines.
• Familiarity with data governance/security practices (Unity Catalog, RBAC).

• Preferred / Good-to-Have
• Experience migrating legacy systems (Hadoop, Teradata, on-prem ETL tools) to Databricks.
• Exposure to streaming data (Structured Streaming, Kafka, Event Hubs/Kinesis).
• Knowledge of MLflow or basic ML pipeline integration.
• Experience with BI tools (Power BI, Tableau) for downstream consumption.