Big Data & Backend Engineer – Spatial ETL (m/f/d)

About Project IRIS Project

IRIS is a proprietary applied research initiative processing massive amounts of infrastructure and spatial data. We bridge the gap between fragmented public data and actionable infrastructure intelligence. When standard APIs reach their limits and entire national registries must be processed, you step in. In this role, you are the architect for heavy data loads and robust backend pipelines.

Your Focus & Responsibilities

• Bulk Processing: Automate the download and processing of massive data dumps (e.g., corporate registers, industrial emission databases like PRTR, energy grids).
• ETL Engineering: Program high-performance ETL pipelines that decompress, parse, and transfer complex archives into our central PostgreSQL architecture.
• Database Governance: Ensure idempotency, deduplication, and reliable monitoring during weekly multi-million row updates.
• Container Orchestration: Deploy and maintain isolated Docker environments for all ingestion tasks.

What We Are Looking For

• Technical Background: Degree in Computer Science, Software Engineering, or extensive practical backend industry experience.
• Core Technical Skills: Expertise in processing relational mass data, strong Python skills (SQLAlchemy, Pandas), and deep PostgreSQL performance tuning.
• Mindset: A clean-code engineering approach focused on system stability and scalability.
• Eligibility: Open to local applicants and international candidates (Chancenkarte / Opportunity Card supported).
• Languages: Professional working proficiency in English (required); German language skills are a plus.

What We Offer