Data Integration Engineer (m/w/d) – Schwerpunkt Streaming & Confluent
Core
Building and operating a central data integration platform using Confluent Cloud (Apache Kafka) as the real-time data hub, connecting diverse data sources and enabling data flow to Azure Databricks Lakehouse.
Role type
Data Integration Engineer (Streaming & Event-Driven Architecture)
Builds
Real-time data pipelines, CDC connectors, and automated integration processes for a central enterprise data platform.
Domain
Retail & Food distribution, Cloud Data Engineering, Event-Driven Architecture
Required skills
Apache Kafka (Confluent Cloud), Change Data Capture (CDC), Data Modeling (Data Vault), ELT processes, Python, Azure Cloud, SQL, Data Security & Compliance (GDPR)
Preferred skills
Custom Connector development, Schema Registry, Avro, Debezium, Azure Databricks (PySpark), Pipeline Monitoring & Observability
Technologies
Confluent Cloud, Apache Kafka, Azure Databricks, Python, PySpark, SQL, Debezium, Avro
Responsibilities
Connect data sources to Confluent Cloud via CDC and custom connectors; Design, standardize, and automate integration processes; Operate and maintain integration pipelines; Implement security and compliance measures; Build reporting and monitoring components; Model integrated data for the Lakehouse.
Seniority
Mid-level, hands-on IC