Data Engineer

Stack

Responsibilities

• ▹design and build scalable production data pipelines (Spark, Python, SQL, Databricks)
• ▹integrate and transform health, financial and operational data
• ▹develop data models and analytics datasets
• ▹build and maintain API-based integrations
• ▹monitor, troubleshoot and optimize scheduled workflows
• ▹implement data validation, quality, lineage and governance
• ▹contribute to automated testing, code reviews and documentation

Requirements

• ▹3+ years of data engineering, software engineering or related technical experience
• ▹strong Python and SQL proficiency
• ▹experience with Apache Spark or similar distributed processing frameworks
• ▹experience with Databricks or a similar modern data platform
• ▹experience building production data pipelines and ETL/ELT processes
• ▹data modeling knowledge
• ▹experience integrating data through REST APIs
• ▹experience with Git-based development workflows

Nice to have

• ▹AWS cloud experience
• ▹experience working with very large datasets

Soft skills

About the company

540 is a technology company focused on U.S. federal government contracts, delivering data modernization solutions.