Data Engineer – AWS Lakehouse (Mandarin Required)

Job description:

• About the Role
• We are looking for a mid-level Data Engineer to join our Data Platform team and take ownership of building and scaling our AWS-based data lakehouse. You will architect and deliver robust, production-grade data pipelines, work closely with data scientists, analytics engineers, and product teams, and set the technical direction for how data flows across the organization. This is a hands-on engineering role — you will write production code in Java and Python every day, while also contributing to platform design decisions, mentoring junior engineers, and driving best practices around data quality, reliability, and governance.
• Key Responsibilities
• Data Lakehouse Development
• Build and extend medallion-architecture data lakehouse layers (Bronze /
• Silver / Gold) on AWS S3 using the Apache Iceberg table format.
• Develop and maintain high-throughput ETL/ELT pipelines using AWS
• Glue, EMR (Spark), and Lambda.
• Implement schema evolution, partitioning strategies, and compaction
• processes for Iceberg tables to optimize storage and query performance.
• Write production-quality pipeline code in Java and Python, following team
• conventions for structure, testing, and maintainability.
• Real-Time & Batch Streaming
• Build and operate event-driven data pipelines using Amazon Kinesis Data
• Streams, Kinesis Firehose, or Apache Kafka (MSK).
• Implement exactly-once or at-least-once processing semantics for
• streaming workloads using Apache Flink or Spark Structured Streaming on EMR.
• AWS Platform Engineering
• Contribute infrastructure-as-code definitions using AWS CDK or
• Terraform for repeatable, auditable data platform deployments.
• Monitor and help tune cost and performance across AWS services
• including S3, Glue, Athena, Redshift Spectrum, EMR, Lambda, Step Functions, and
• EventBridge.
• Follow platform security standards in day-to-day work: IAM least-
• privilege policies, KMS encryption, and VPC networking.
• Maintain and extend CI/CD pipelines for data workloads using AWS
• CodePipeline, GitHub Actions, or equivalent.
• Data Quality & Governance
• Add data quality checks using frameworks such as Great Expectations or
• Deequ, and integrate validation steps into pipeline orchestration.
• Help maintain and uphold data contracts between producing and
• consuming systems.
• Contribute to data cataloguing and lineage tracking using AWS Glue Data
• Catalog or Apache Atlas.
• Collaboration & Ways of Working
• Partner with data scientists, ML engineers, and analysts to understand data
• requirements and deliver performant, well-documented datasets.
• Take an active part in code reviews, design discussions, and pair
• programming, and support junior engineers where you can.
• Document your pipeline designs and decisions, and contribute to the
• internal engineering knowledge base.
• Required Qualifications
• Experience
• 3+ years of professional data engineering experience, with at least 1–2
• years on AWS cloud platforms.
• Experience building and supporting production data pipelines, ideally on
• large datasets with defined freshness or reliability SLAs.
• Working knowledge of data lakehouse concepts — medallion pattern and
• open table formats (Iceberg preferred; Delta Lake or Hudi acceptable).
• Programming Languages
• Java: Working proficiency in Java (8+) for Spark jobs and pipeline
• components, with familiarity with Maven or Gradle build systems.
• Python: Solid Python 3 skills for AWS Glue scripts, orchestration logic,
• data quality checks, and automation tooling. Experience with pandas, PySpark, and
• boto3. Strong skills in one language and a willingness to ramp up on the other is
• acceptable.
• AWS Core Services
• Storage & Compute: S3, Glue (jobs, crawlers, Data Catalog), EMR
• (Spark/Flink), Lambda, EC2.
• Streaming: Kinesis Data Streams, Kinesis Firehose, or MSK (Managed
• Kafka).
• Orchestration: Step Functions, MWAA (Managed Airflow), or
• EventBridge Scheduler.
• Querying: Athena, Redshift, or Redshift Spectrum.
• Security & Governance: Comfortable working within IAM, KMS, Secrets
• Manager, and VPC setups.
• DevOps: Exposure to AWS CDK or CloudFormation, and to CodePipeline
• or equivalent CI/CD tools.
• Data Processing Frameworks
• Apache Spark (PySpark and/or Spark Java API) — distributed
• transformations and a working grasp of performance tuning.
• Apache Iceberg — reading and writing tables, time travel, and basic table
• maintenance.
• SQL — strong SQL for data transformation, including window functions,
• CTEs, and query tuning.
• Must be Chinese Mandarin fluent.
• Preferred Qualifications
• AWS Certified Data Engineer – Associate or AWS Certified Solutions
• Architect certification.
• Experience with dbt for SQL-based transformation layers on top of the
• lakehouse.
• Familiarity with ML platform integration: feature stores (SageMaker
• Feature Store), model serving data needs, or MLflow experiment tracking.
• Experience with real-time OLAP engines such as Apache Druid or
• ClickHouse.
• Experience with Lake Formation fine-grained access control, or with data
• cataloguing and lineage tooling.
• Exposure to data mesh or data product thinking — domain ownership and
• data contracts.
• Tech Stack at a Glance
• Languages
• Java/ Python 3
• Cloud Platform
• AWS (S3, Glue, EMR, Kinesis, Athena, Lambda, Step Functions, Lake Formation, CDK)
• Processing
• Apache Spark, Apache Flink, Spark Structured Streaming
• Table Format
• Apache Iceberg (primary), Delta Lake / Hudi (familiarity)
• Streaming
• Amazon Kinesis, MSK (Kafka), Kinesis Firehose
• Orchestration
• Apache Airflow (MWAA), AWS Step Functions
• IaC & CI/CD
• AWS CDK / Terraform, GitHub Actions / CodePipeline
• Job Type: Full-time
• Benefits:

• 401(k)
• 401(k) matching
• Dental insurance
• Health insurance
• Life insurance
• Paid time off
• Parental leave
• Retirement plan
• Vision insurance