Data Engineer

About The Company

For over 25 years, Edgesource Corporation has established itself as an innovative leader in providing advanced technology solutions to a diverse range of federal, state, and commercial clients. Our core mission is to deliver impactful and tailored technical services that support national security, cybersecurity, and intelligence operations. We serve prestigious clients including the Department of Defense (DOD), Department of Homeland Security (DHS), Department of State (DOS), the U.S. Intelligence Community, and Law Enforcement agencies, both domestically and internationally. Edgesource specializes in delivering boutique technical solutions, supporting critical missions such as counter-unmanned aerial systems (CUAS) and cybersecurity threats, positioning us as a trusted partner in national security efforts. Committed to quality and excellence, we are ISO 9001:2015 certified and CMMI Level 3 appraised, emphasizing our dedication to continuous improvement and operational excellence.

About The Role

We are seeking a highly skilled Data Engineer to join our dynamic team. The successful candidate will play a pivotal role in designing, developing, and maintaining complex data pipelines and workflows for a custom application. This role requires advanced expertise in Python programming, familiarity with Java, and a solid understanding of data security, privacy, governance, and compliance principles. The Data Engineer will be responsible for building scalable, reliable, and efficient data processing systems, leveraging industry-standard tools and cloud technologies. The position offers an exciting opportunity to work on mission-critical projects, including large-scale data migrations, platform modernization, and integration of AI/ML services. Collaboration with stakeholders to understand data requirements and translating them into robust technical solutions will be key to success in this role.

Qualifications

• Active TS/SCI Full Scope Polygraph clearance
• Minimum of 5 years of professional experience in data engineering or related fields
• Proficiency in building production data pipelines and ETL workflows at scale
• Expertise in Apache Spark and PySpark for distributed data processing
• Advanced Python programming skills, including data manipulation libraries such as Pandas and NumPy
• Strong understanding of data security, privacy, governance, and compliance standards
• Experience with workflow orchestration tools such as Apache Airflow or AWS Step Functions
• Hands-on experience with containerization technologies like Docker or Podman
• Proficiency with AWS services including S3, Lambda, and Step Functions
• Experience working with PostgreSQL and MySQL databases, including performance tuning and schema design
• Solid SQL skills with experience in query optimization for complex analytical workloads
• Experience with version control systems such as Git and CI/CD pipelines
• Ability to collaborate effectively with stakeholders to gather requirements and develop solutions
• Strong problem-solving skills for troubleshooting data quality issues and pipeline failures

Responsibilities

• Design, develop, and maintain end-to-end data pipelines leveraging Python and Spark technologies
• Configure and manage Spark jobs within orchestration tools to ensure reliable deployment
• Containerize and deploy data applications in cloud environments, primarily AWS
• Optimize database performance through schema design and query tuning in MySQL and PostgreSQL
• Implement data security, privacy, and governance best practices across data workflows
• Track data lineage and manage data catalogs to ensure data integrity and compliance
• Automate data processing tasks using Bash scripting and other automation tools
• Integrate AI/ML models and services into data pipelines for enhanced analytics capabilities
• Collaborate with stakeholders to understand data requirements, assess feasibility, and propose solutions
• Monitor and troubleshoot data pipelines, resolving issues related to data quality and performance
• Contribute to documentation, best practices, and design patterns for data engineering efforts
• Support large-scale data migration and platform modernization initiatives

Benefits

• Flexible Paid Time Off policy and 11 paid holidays
• Flexible work schedules including remote and hybrid options
• Comprehensive medical, dental, and vision insurance plans
• Flexible Spending Account (FSA) options
• 401(k) retirement plan with company matching
• Tuition reimbursement and professional development support
• Commuter benefits to assist with transportation costs
• Bonus and employee referral programs
• Opportunities for career growth and advancement within the company

Equal Opportunity

Edgesource is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, disability, age, sexual orientation, gender identity, national origin, veteran status, or genetic information. We are committed to fostering an inclusive environment and providing reasonable accommodations to individuals with disabilities.