Senior Data Engineer, Data Lake
Core
Build and maintain data ingestion and transformation pipelines to support a centralized, trusted corporate data lake for decision-making.
Role type
Intermediate Data Engineer (Data Lake)
Builds
Data ingestion and transformation pipelines, data quality checks, and documentation for datasets.
Domain
Financial services / Data Engineering
Required skills
Python, PySpark/SparkSQL, SQL, Airflow DAGs, Data pipeline concepts (ETL/ELT), Data modeling (Kimball, medallion layers), Git/GitHub workflow
Preferred skills
Delta Lake, Terraform, BI integration (Superset), CDC patterns, Data quality concepts
Technologies
Databricks, Apache Spark, Airflow (MWAA), Amazon S3, AWS (IAM, CloudWatch), Great Expectations
Responsibilities
Develop, test, and maintain data pipelines; Build and modify Airflow DAGs; Write and optimize SQL queries; Support data quality by implementing monitoring checks; Participate in code reviews; Investigate pipeline failures; Write and maintain documentation.
Seniority
Intermediate, hands-on IC