This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an Evaluations Team Lead based in Italy.

As an Evaluations Team Lead, you will shape how advanced AI models are evaluated, validated, and improved in production environments. You will lead the development of a shared evaluation platform that enables research, engineering, and customer-facing teams to measure model performance with confidence. Your work will establish rigorous standards for benchmarking, statistical validation, data quality, and reproducibility. Combining hands-on technical leadership with people management, you will build a high-performing team while remaining closely involved in code, experiments, and methodology. You will assess predictive quality, inference latency, and cost efficiency to guide research priorities and product releases. This is an opportunity to define evaluation practices from the ground up and directly influence the reliability and impact of next-generation enterprise AI.

n

Accountabilities:





• Build and maintain a shared evaluation platform that supports research, engineering, and Applied AI teams, enabling independent benchmarking and investigation of model performance.

• Develop and continuously improve benchmark suites, curated datasets, and model comparisons to assess predictive quality across a broad range of business use cases.

• Establish rigorous evaluation standards covering statistical metrics, validation strategies, data splits, leakage prevention, calibration, uncertainty estimation, and benchmark contamination.

• Design reproducible evaluation pipelines with version-controlled inputs, datasets, and artifacts, integrating automated regression checks into research and release workflows.

• Benchmark models against competing approaches using fair and consistent protocols for hyperparameter tuning, data access, computational resources, and end-to-end latency measurement.

• Evaluate pre

📌 Evaluations Team Lead (Italia)
🏢 Jobgether
📍 Italia