ML-QA Engineer
Core
Designing and maintaining automated test frameworks for AI-driven, agentic applications to ensure quality, safety, and reliability under real-world conditions.
Role type
Senior ML-QA Engineer (Agentic/AI Applications)
Builds
Automated test frameworks, CI/CD integration, and validation tools for intelligent applications
Domain
Artificial Intelligence / Agentic Systems / Software Quality Assurance
Deliverable
production ML models | product features
Required skills
Python, test automation frameworks (PyTest, Selenium, Playwright), CI/CD integration, API testing (REST, Postman, Swagger), chaos testing, data validation pipeline testing, root cause analysis, scenario-based testing
Preferred skills
Observability tools (Datadog, Grafana, AWS CloudWatch), microservices/serverless architecture testing, security and guardrail testing, performance/load testing for AI endpoints
Technologies
Python, PyTest, Selenium, Playwright, Postman, Swagger, GitHub Actions, Jenkins, GitLab CI, AWS, Datadog, Grafana, CloudWatch
Responsibilities
Design and maintain automated test frameworks and scripts; Integrate tests into CI/CD pipelines; Collaborate with developers to identify and resolve defects; Define test plans and acceptance criteria for agentic features; Verify observability and explainability of agent behavior; Perform chaos testing to validate decision-making under uncertainty; Test human-in-the-loop workflows; Conduct root cause analysis of production issues
Seniority
Senior, hands-on IC