Data Engineer

• Position: Data Engineer
• Location: Sunnyvale, CA (Onsite) – ex-apple and local
• Duration: Fulltime

Job Description

We are seeking an experienced Data Engineer to design, build, and optimize scalable data pipelines and analytics infrastructure. The ideal candidate has hands-on expertise with Snowflake for cloud data warehousing and Apache Doris for real-time OLAP analytics and is comfortable working across the full data lifecycle — from ingestion to transformation to serving high-performance queries for analytics and BI teams.

Responsibilities

• - Design, build, and maintain robust, scalable ETL/ELT pipelines to ingest structured and semi-structured data from multiple sources into Snowflake and Apache Doris.
• - Architect and optimize data models, schemas, and partitioning strategies in Snowflake and Doris to support high-throughput, low-latency analytical workloads.
• - Tune query performance, indexing, materialized views, and storage layout in Doris for real-time OLAP use cases.
• - Manage Snowflake warehouse sizing, cost optimization, resource monitors, and query performance tuning.
• - Build and maintain data orchestration workflows (e.g., Airflow, dbt, or similar) to automate pipeline scheduling, dependency management, and monitoring.
• - Implement data quality checks, validation frameworks, and observability/alerting for pipeline reliability.
• - Collaborate with data analysts, data scientists, and BI teams to understand reporting requirements and translate them into performant data models.
• - Support migration and integration projects between transactional systems, data lakes, Snowflake, and Doris.
• - Ensure data governance, security, and access control best practices (RBAC, data masking, encryption) across platforms.
• - Document data architecture, pipeline logic, and operational runbooks.

Required Qualifications

• - 8+ years of experience as a Data Engineer or in a similar role.
• - Hands-on production experience with Snowflake — including SQL scripting, warehouse management, Snowpipe, Streams/Tasks, and performance tuning.
• - Strong SQL skills and experience with data modelling (star/snowflake schemas, dimensional modelling).
• - Proficiency in Python commonly used in data engineering.
• - Experience with workflow orchestration tools (Apache Airflow, dbt, Dagster, or similar).
• - Familiarity with streaming/messaging systems (Kafka, Pulsar, or similar) for real-time data ingestion.
• - Solid understanding of data warehousing concepts, OLAP vs. OLTP design, and distributed systems fundamentals.
• - Experience with version control (Git) and CI/CD practices for data pipelines.

Preferred Qualifications

• - Experience migrating workloads from traditional data warehouses to Snowflake and/or Doris.
• - Familiarity with cloud platforms (AWS, GCP, or Azure) and infrastructure-as-code tools (Terraform, CloudFormation).
• - Experience with containerization/orchestration (Docker, Kubernetes).
• - Knowledge of BI/visualization tools (Tableau, Looker, Superset, Power BI).
• - Background in cost optimization and capacity planning for large-scale analytical systems.