Senior Data Engineer
Core
Build and operate the infrastructure that moves data reliably across a career marketplace and AI products, including workflow orchestration, data ingestion, and streaming pipelines.
Role type
Senior IC data platform engineer (infrastructure & orchestration)
Builds
Workflow orchestration platform (Airflow/Astronomer), data ingestion pipelines, streaming infrastructure, and data warehouse (BigQuery) foundations.
Domain
AI data business, career marketplace, cloud infrastructure
Deliverable
production ML models
Required skills
Python, production data/distributed systems, Airflow (scheduling, deployment, scaling), event-driven/streaming data infrastructure, CDC, cloud infrastructure (Kubernetes, Terraform, IAM), CI/CD, observability
Preferred skills
GCP services (Pub/Sub, Dataflow, Datastream, BigQuery, GKE), Astronomer, Apache Beam, Terraform, Spacelift, Datadog, dbt, Spark, Dataproc
Technologies
Airflow, Astronomer, Pub/Sub, Dataflow, Beam, Datastream, BigQuery, Kubernetes, Terraform, GCP
Responsibilities
Lead technical direction and roadmap for Airflow/Astronomer platform; build paved paths for workflow authoring, testing, and deployment; design and operate streaming/CDC pipelines; ensure data delivery resilience (retries, duplicates, schema changes); improve cloud foundation with K8s, Terraform, and cost management; lead cross-team design decisions on data contracts; mentor engineers; participate in on-call and incident response; build foundations for AI workloads (batch inference, evaluation datasets).
Seniority
Senior, hands-on IC with technical leadership
