Data Engineer
Core
Design and build scalable data pipelines to process, transform, and load archival data for public access and discovery.
Role type
Data Engineer
Builds
Scalable ETL/ELT pipelines, data enrichment tools, and integrated data models for UK public records.
Domain
Digital Archiving / Public Sector
Required skills
Python, SQL, Shell scripting, XML, JSON, RDF, RDBMS, NoSQL, graph databases, data modeling, schema design, data cleansing, automation
Preferred skills
Experience with linked data stores, ontologies, emerging data technologies
Technologies
Python, SQL, Shell, Talend, XSLT, XML, CSV, JSON, RDF, RDBMS, NoSQL, graph databases
Responsibilities
Design and develop scalable data pipelines; Automate data cleansing, validation, and enrichment; Develop and document schemas and models; Integrate data from diverse sources; Collaborate with archivists and developers to deliver data solutions.
Seniority
Mid-level, hands-on IC