Data Engineer (Content Processing)
Core
Architect intelligent data pipelines to ingest, process, and transform hundreds of thousands of unstructured documents daily, generating vector embeddings and hybrid search systems for national security threat detection.
Role type
Senior Data Engineer (AI/Unstructured Data)
Builds
High-throughput data pipelines, vector embedding systems, hybrid retrieval indexes, and multimedia analysis tools for security analytics.
Domain
National Security / Cybersecurity / Unstructured Data Processing
Deliverable
production ML models | infrastructure
Required skills
Scalable ETL/ELT pipeline design, unstructured data processing, vector embedding generation, hybrid search indexing, multimedia processing, cloud-native architecture, code quality & observability
Preferred skills
Vector databases, LLM integration, semantic search frameworks, container orchestration (Docker/Kubernetes), computer vision libraries
Technologies
Vector databases, LLMs, Docker, Kubernetes, CI/CD tools
Responsibilities
Architect and maintain high-throughput pipelines for unstructured data ingestion; Develop transformation logic for content parsing and vector embedding generation; Build and support hybrid keyword and vector search indexing systems; Engineer scalable solutions for video analysis and image recognition; Uphold data quality standards and system reliability; Collaborate with data scientists and product managers to deliver data services.
Seniority
Senior, hands-on IC