Position: LLM - AI Quality Analyst Personalization - Italian

Type: Hourly contract

Compensation: $11.5 - $15.3/hour

Location: Remote

Commitment: 20 hours/week

Role Responsibilities

- Evaluate AI model responses for personalization quality including grounding, integration, and helpfulness
- Design and execute multi-turn prompts based on personal context to test AI capabilities
- Analyze responses for hallucinations, incorrect personalization, and flawed inferences
- Perform side-by-side (SxS) comparison of model outputs to determine quality and effectiveness
- Write clear and structured rationales for response evaluations and rankings
- Provide detailed feedback and annotations to improve AI model performance
- Maintain strict data hygiene and ensure accurate documentation of evaluations

Requirements:

- Strong proficiency in Italian with excellent reading and writing skills




- Experience in data annotation, AI evaluation, content moderation, or similar roles
- Strong analytical thinking with the ability to evaluate nuanced AI responses
- Ability to design creative multi-turn prompts based on personal context
- Excellent written communication and attention to detail
- Full-time availability with flexibility and ability to overlap with PST time zone
- Bachelor s degree or equivalent experience in a relevant analytical field

Application Process:

- Apply/Easy Apply and check email for application form
- Fill Google form
- Assessment Link (After shortlisting)
- Screener
- 1/3 assessments mandatory
- Language vetting

📌 AI Data Annotator (Italian) | $15.3/hr Remote (Italia)
🏢 Crossing Hurdles
📍 Italia