This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a AI QA Trainer - LLM Evaluation based in Italy.
This role offers the opportunity to contribute directly to the evolution of advanced AI systems by improving their reliability, accuracy, and safety.

You will work on evaluating large language models through structured testing, quality analysis, and adversarial scenarios.

The position combines AI quality assurance, data evaluation, and technical problem-solving to strengthen model performance.

You will design evaluation frameworks, identify failure patterns, and help improve AI reasoning capabilities across diverse use cases.

Working with cutting-edge AI technologies, you will influence how future systems handle information, reasoning, and real-world tasks.

This is a remote contract opportunity for professionals passionate about AI quality, testing, and responsible model development.

Accountabilities
The role focuses on ensuring the quality, reliability,



and safety of advanced AI models through comprehensive evaluation and testing processes. You will collaborate on identifying weaknesses, improving evaluation methodologies, and creating actionable insights that enhance model performance.
- Evaluate large language models across areas such as factual accuracy, reasoning quality, hallucination detection, safety, and reliability.
- Design and execute test plans, regression suites, and evaluation frameworks to measure model performance.
- Identify and document model failures, including prompt vulnerabilities, incorrect outputs, bias issues, and reasoning inconsistencies.
- Develop clear evaluation rubrics, pass/fail criteria, and quality benchmarks for AI system assessment.
- Perform adversarial testing, red-teaming exercises, and robustness evaluations to identify potential risks.
- Verify grounding, retrieval-augmented generation outputs, tool usage accuracy, and workflow reliability.
- An

📌 AI QA Trainer (Italia)
🏢 Jobgether
📍 Italia