AWS Data Engineer

• Data engineer
• Client: Charter Communications
• Location: Greenwood Village, CO (3 days onsite & 2 days remote per week)
• Duration: base contract; long-term extensions
• Work Authorization: Any Visa but need W2 only
• Interview: 1 virtual, 1 in-person

Notes:

**Local candidates are preferred.**

**Relocation candidates are accepted (however, MUST attend an in-person final round interviewer)**

Main Skillset:

• Spark/Scala
• Python
• AWS (S3, Glue, Athena, EMR)
• Advanced SQL
• Airflow
• Large-scale datasets
• Batch pipelines, data ingestion pipelines, etc…
• Data Lake/Lakehouse concepts

MAJOR DUTIES AND RESPONSIBILITIES

• Design, develop, and maintain scalable ETL pipelines using Apache Spark (Scala) to ingest, transform, and load network data into the IIA Data Lake.
• Onboard new data sources (network telemetry, syslogs, SNMP traps, device configuration data, ticketing systems) by building ingestion pipelines from raw source to query-ready format.
• Implement monitoring and alerting solutions to ensure data pipeline reliability and performance.
• Develop and manage deployment pipelines to facilitate continuous integration and delivery of data engineering solutions.
• Manage and optimize data storage solutions, including distributed file systems, relational databases, flat files, and external sources accessed via API.
• Implement data quality checks, validation rules, and automated testing to ensure pipeline reliability and data integrity.
• Optimize pipeline performance for large-scale data processing (billions of events per day) across batch and mini-batch processing patterns.
• Manage and evolve data schemas, partitioning strategies, and storage formats to support efficient querying and downstream consumption.
• Support data backfills and recovery when upstream issues or schema changes require reprocessing.
• Collaborate across teams to ensure data solutions align with existing production architectures and business requirements.
• Work with data scientists and agent developers to understand data requirements and deliver datasets that support anomaly detection models and AI agent workflows.
• Provide technical guidance on data engineering best practices and methodologies.
• Document and communicate data engineering processes and standards to business-intelligence, data, and analytics professionals with varied backgrounds.
• Continuously evaluate and improve data engineering tools and approaches to enhance performance and efficiency.
• Perform other duties as required.
• REQUIRED QUALIFICATIONS
• Skills/Abilities and Knowledge
• Ability to read, write, speak and understand English
• Strong communication and collaboration skills
• Proficiency in Python with experience in distributed data processing (Spark preferred, willingness to learn Scala acceptable)
• Strong experience with Apache Spark for distributed data processing
• Proficiency in building and maintaining ETL pipelines at scale
• Experience with AWS services: S3, Glue, Athena, EMR
• Strong understanding of relational databases and SQL
• Knowledge of data architecture, data warehousing, partitioning strategies, and columnar storage formats (e.g., Parquet)
• Experience implementing data quality checks and validation frameworks
• Experience with workflow orchestration tools (Airflow preferred)
• Proficiency with Linux-based operating systems and shell scripting
• Experience with Git-based version control and collaborative development workflows
• Demonstrated ability and desire to continually expand skill set, and learn from and teach others
• PREFERRED QUALIFICATIONS
• Skills/Abilities and Knowledge
• Experience with streaming or mini-batch data processing (Spark Streaming, structured streaming, or similar)
• Experience with Apache Kafka or similar messaging/streaming platforms
• Experience with NoSQL databases
• Experience in the telecommunications industry or other large-scale network operations environments
• Familiarity with network data sources: telemetry, syslogs, SNMP traps, device configuration data
• Experience with data integration via REST APIs and cloud SDKs (e.g., boto3)
• Experience writing automated tests for data pipelines
• Knowledge of text analysis or log parsing techniques
• Education
• Bachelor's degree in Computer Science, Mathematics, Statistics, Engineering, or related field, or relevant experience

• Related Experience
• Bachelor's degree: 5+ years of data engineering experience
• Master's degree: 3+ years of data engineering experience
• WORKING CONDITIONS
• Hybrid (3 days in office and 2 days remote)