Position: LLM - AI Quality Analyst Personalization - Italian
Type: Hourly contract
Compensation: $11 - $15/hour
Location: Remote
Commitment: 30–40 hours/week
Role Responsibilities
Evaluate AI model responses for personalization quality including grounding, integration, and helpfulness
Design and execute multi-turn prompts based on personal context to test AI capabilities
Analyze responses for hallucinations, incorrect personalization, and flawed inferences
Perform side‑by‑side (SxS) comparison of model outputs to determine quality and effectiveness
Write clear and structured rationales for response evaluations and rankings
Provide detailed feedback and annotations to improve AI model performance
Maintain strict data hygiene and ensure accurate documentation of evaluations
Requirements
Strong proficiency in Italian with excellent reading and writing skills
Experience in data annotation, AI evaluation, content moderation, or similar roles
Strong analytical thinking with the ability to evaluate nuanced AI responses
Ability to design creative multi-turn prompts based on personal context
Excellent written communication and attention to detail
Full‑time availability with flexibility and ability to overlap with PST time zone
Bachelor’s degree or equivalent experience in a relevant analytical field
#J-18808-Ljbffr