Data Engineer
📍 Location: New York, NY, US
(Hybrid)
🏢 Industry: Financial Services
💼 Work Setting: Hybrid

Are you passionate about building cloud-native data platforms, architecting large-scale data pipelines, and leading the development of modern analytics ecosystems that power business and customer experiences?
We are seeking a Senior Data Engineer to design, build, and optimize scalable cloud-first data solutions, lakehouse architectures, real-time streaming platforms, and enterprise data products. This role combines technical leadership, cloud engineering, distributed computing, data architecture, governance, and mentorship while driving best practices across modern data platforms.
The ideal candidate brings expertise in Python, SQL, Spark, Databricks, Snowflake, cloud platforms, streaming architectures, and large-scale data engineering with a passion for innovation and engineering excellence.

Key Responsibilities
Cloud Data Architecture & Engineering
Design, develop, test, and deploy scalable cloud-native data solutions.
Build modern lakehouse architectures supporting analytics and machine learning workloads.
Develop highly available and resilient data platforms.
Create end-to-end solutions that support growing business and customer demands.
Ensure systems are scalable, maintainable, and operationally efficient.
Focus Areas
Data Architecture
Data Engineering
Cloud Data Platforms
Lakehouse Architecture
Enterprise Analytics
Data Pipeline Development
Design and implement batch and real-time data pipelines.
Build distributed processing frameworks for large-scale workloads.
Develop ingestion, transformation, and delivery processes.
Optimize pipelines for performance, reliability, and scalability.
Support integration across diverse data sources and business domains.
Responsibilities
ETL / ELT Development
Streaming Data Pipelines
Data Integration
Data Transformation
Pipeline Automation
Technical Leadership & Mentorship
Lead and influence cross-functional engineering teams.
Establish and promote data engineering best practices and design standards.
Mentor engineers, analysts, and technical contributors.
Drive code quality, maintainability, and architectural consistency.
Support continuous learning and technical excellence.
Leadership Areas
Technical Coaching
Architecture Reviews
Engineering Standards
Design Patterns
Code Quality
Cloud & Distributed Data Processing
Build and support large-scale distributed workloads using technologies such as:
Apache Spark
Databricks
AWS EMR
AWS Glue
Hadoop Ecosystem
Responsibilities
Distributed Computing
Data Processing Optimization
Large-Scale Analytics
Cloud-Native Engineering
Performance Tuning
Data Warehousing & Analytics Platforms
Design and manage enterprise data warehousing solutions.
Develop scalable analytical data models.
Support reporting, business intelligence, and data science initiatives.
Optimize query performance and storage strategies.
Deliver trusted datasets for downstream consumers.
Technologies
Snowflake
Amazon Redshift
Databricks
Data Warehouse Architectures
Real-Time & Streaming Analytics
Design and operate real-time streaming applications.
Implement event-driven data architectures.
Enable timely delivery of business-critical insights.
Support high-volume data processing environments.
Ensure reliability of streaming data ecosystems.
Technologies
Kafka
Spark Streaming
Event Processing Frameworks
Real-Time Analytics Platforms
Data Governance, Quality & Security
Implement strong data governance and security controls.
Ensure encryption of data at rest and in transit.
Design fine-grained authorization and access management controls.
Maintain data quality, lineage, and compliance standards.
Promote responsible and secure use of enterprise data.
Areas of Focus
Data Governance
Data Security
Data Quality
Compliance
Access Control
Data Products & Reusability
Build reusable data assets and products serving multiple business domains.
Establish standardized frameworks and engineering patterns.
Promote self-service analytics capabilities.
Improve discoverability and usability of enterprise data.
Drive long-term platform sustainability and efficiency.
Collaboration & Stakeholder Engagement
Partner closely with:
Product Managers
Software Engineers
Data Scientists
Data Analysts
Business Stakeholders
Responsibilities
Translate business needs into scalable technical solutions.
Communicate technical concepts to non-technical audiences.
Drive alignment across teams.
Support strategic decision-making through data solutions.
Observability & Reliability
Implement monitoring and observability frameworks.
Establish proactive alerting and operational visibility.
Analyze performance bottlenecks and reliability issues.
Improve platform stability and operational excellence.
Support incident resolution and root-cause analysis.
Technologies
Airflow
Dagster
Splunk
Monte Carlo
Observability Platforms
Innovation & Emerging Technologies
Stay current with evolving cloud, analytics, and data technologies.
Participate in engineering communities and knowledge sharing.
Evaluate emerging tools and frameworks.
Leverage AI-powered development tools to enhance engineering productivity.
Drive innovation through experimentation and continuous improvement.
Required Qualifications
Education
Bachelor's Degree in:
Computer Science
Engineering
Mathematics
Statistics
Analytics
Operations Research
Economics
Related Quantitative Field
Experience
4+ years of application development experience.
2+ years working with distributed data systems.
2+ years of SQL experience.
2+ years of experience with Python, Java, or Scala.
2+ years designing and developing data pipelines.
1+ year of data modeling and end-to-end data solution design experience.
Experience with relational and non-relational databases.

Preferred Qualifications
Education
Master's Degree in:
Computer Science
Data Engineering
Information Systems
Related Technical Discipline
Technical Experience
7+ years of application or data engineering experience.
4+ years supporting public cloud environments:
AWS
Azure
Google Cloud Platform
4+ years building distributed computing workloads.
4+ years designing and operating streaming data applications.
4+ years with enterprise data warehousing solutions.
4+ years working with NoSQL databases and semi-structured data.
Experience developing reusable enterprise data products.
Experience within Agile engineering environments.
Technical Skills
Programming Languages
Python
SQL
Scala
Java
Big Data & Distributed Processing
Apache Spark
Databricks
Hadoop
EMR
AWS Glue
Cloud Platforms
AWS
Azure
Google Cloud Platform
Streaming Technologies
Kafka
Spark Streaming
Event-Driven Architectures
Data Warehousing
Snowflake
Redshift
Lakehouse Platforms
Databases
MongoDB
Cassandra
DynamoDB
Relational Databases
NoSQL Databases
DevOps & Data Operations
Airflow
Dagster
CI/CD
Observability Tools
Automated Testing
Developer Productivity
GitHub Copilot
AI-Assisted Development Tools
Professional Competencies
Technical Leadership
Problem Solving
Strategic Thinking
Collaboration
Communication Skills
Mentorship
Innovation
Continuous Learning
Stakeholder Management

Core Competencies
Data Engineering
Data Architecture
Cloud Data Platforms
Databricks
Snowflake
Apache Spark
Python
SQL
Data Pipelines
Streaming Data
Kafka
AWS
Azure
GCP
Data Warehousing
Lakehouse Architecture
NoSQL Databases
Data Governance
Data Security
Analytics Engineering

Data Engineer

Similar Engineering Jobs

Data Engineer Intern, Product Analytics (Summer 2027)

Software Engineer, Systems ML

Data engineer

Lead Data Engineer

Data Engineer opportunity in New York

Sr. Data Engineer