Data Engineer
Core
Design, develop, and maintain reliable ETL/ELT batch and streaming data pipelines, build and iterate data warehouses/lakes, and implement data quality rules for diverse data sources.
Role type
Senior Data Engineer
Builds
Data pipelines, data warehouses, data lakes, and reusable datasets
Domain
Cloud data engineering, real-time streaming, data governance
Deliverable
production ML models
Required skills
SQL, Python, Kafka, Flink, Airflow, dimensional modeling, data quality monitoring, cloud cost optimization
Preferred skills
Snowflake, BigQuery, Redshift, ML feature engineering, metadata platform construction
Technologies
AWS, Azure, GCP, Kafka, Flink, Pulsar, Airflow, Dagster, Snowflake, BigQuery, Redshift
Responsibilities
Design and deploy ETL/ELT pipelines, optimize complex SQL queries, implement data quality rules and anomaly detection, collaborate with analysts and DS on data solutions, manage workflow orchestration, participate in data governance and security
Seniority
Senior, hands-on IC