数据处理架构师-AI Data
Core
Optimize high-frequency operators and batch processing pipelines for large model pre-training data processing, covering content understanding, model labeling, data cleaning, and format conversion to improve throughput, stability, and resource utilization.
Role type
Senior IC data processing architecture engineer (large model pre-training)
Builds
Scalable data processing platforms and optimized inference pipelines for large language models
Domain
Artificial Intelligence / Large Language Models / High-Performance Computing
Deliverable
production ML models
Required skills
Python, C++, Go, Rust, GPU inference optimization, CUDA, TensorRT-LLM, vLLM, SGLang, model quantization, distillation, pruning, Spark, Ray, Flink, distributed computing, profiling, benchmarking, performance analysis
Preferred skills
Large model pre-training data processing experience, offline model inference optimization, model compression implementation, complex offline task stability governance
Technologies
CUDA, TensorRT-LLM, vLLM, SGLang, Spark, Ray, Flink
Responsibilities
Optimize batch processing chains for data cleaning and format conversion; Develop reusable optimization capabilities for GPU inference and task scheduling; Establish evaluation systems for cost, throughput, and stability of inference acceleration solutions.
Seniority
Senior, hands-on IC