h3About The Role /h3pSenior data annotation is a Mid-Senior, full-time remote role focused on training data quality for AI/ML systems—labeling, RLHF, prompt evaluation, and QA evaluation that directly improve model performance. /ppYou will help build reliable LLM training pipelines by applying annotation guidelines compliance, auditing datasets, and refining instructions for human feedback. /ph3What You Will Do /h3ulliOwn and execute complex annotation tasks across NLP and computer vision annotation (e.g., named entity recognition, classification, ranking, span labeling) /liliPerform RLHF workflows (preference comparisons, rationale checks, policy-aligned scoring) and prompt evaluation for LLM training pipelines /liliRun QA evaluation using sampling plans, inter-annotator agreement checks, and targeted error analysis /liliMaintain annotation guidelines compliance by documenting decisions,



escalating ambiguous cases, and updating rubrics /liliPartner with stakeholders across AI labs, tech startups, BPO teams, and annotation vendors to deliver production-ready datasets on schedule /li /ulh3Required Qualifications /h3ulliProduction experience in data annotation/data labeling or QA evaluation for AI/ML systems /liliStrong understanding of training data quality and guideline-driven labeling /liliFamiliarity with RLHF and large language model evaluation concepts (prompt evaluation, preference ranking) /liliHigh attention to detail, consistent decision-making, and comfort with ambiguous edge cases /liliClear written communication and precise documentation habits /li /ul #J-18808-Ljbffr

📌 Senior Data Annotation Jobs in Rome (Italia)
🏢 RemoExperts
📍 Italia