Rex.zone is hiring a Mid-Senior Senior Data Annotator to deliver high-accuracy training data and evaluations for AI/ML systems, including large language models and computer vision models. This is a FULL_TIME , Remote role focused on training data quality, guideline compliance, and scalable labeling operations.

What You Will Work On

- RLHF preference ranking and pairwise comparisons to support LLM training pipelines
- Prompt evaluation (helpfulness, factuality, instruction-following, harmlessness) using rubric-based scoring
- QA evaluation via audits, sampling, error analysis, and adjudication to reduce defect rates
- Content safety labeling across policy categories and moderation taxonomies
- NLP annotation such as named entity recognition (NER) and taxonomy-driven text classification
- Computer vision annotation as needed (bounding boxes, segmentation masks, attribute tagging)

Key Responsibilities

- Own high-complexity annotation queues and deliver consistent, auditable labels
- Lead guideline clarifications, document edge cases,



and propose rubric improvements to reduce ambiguity
- Track throughput and quality metrics (accuracy, agreement rate, rework rate) and recommend process improvements
- Collaborate with QA and data operations teams through calibration sessions and review cadences

Required Qualifications

- Mid-senior experience in data annotation/data labeling operations with strong QA evaluation rigor
- Demonstrated annotation guidelines compliance and controlled labeling workflows
- Familiarity with RLHF evaluation concepts and prompt evaluation methods
- Experience with NLP tasks (NER, classification) and comfort with computer vision annotation or willingness to learn
- Strong written communication for decision rationale and edge-case documentation

How To Apply Apply via Rex.zone with a brief summary of your annotation background (NLP, computer vision, content safety), QA evaluation experience, and any guideline/rubric improvements you have led.

📌 Senior Data Annotator Jobs in Rome (Remote) (Roma)
🏢 Rex.zone
📍 Roma