LLM – AI Quality Analyst (Personalization)
Employment Type:
Short-Term Contract (2 Months)Work Model:
RemoteStart Date:
ImmediateOpen Positions:
50Experience Required:
Minimum 1 YearCompensation:
USD $15 per hourSchedule:
Full-time (30–40 hrs/week) with minimum 4-hour overlap with PSTRole OverviewWe are seeking professionals to evaluate AI-generated personalized responses. In this role, you will assess how effectively AI uses contextual and personal data to deliver relevant, accurate, and helpful responses. This position combines analytical evaluation with creative prompt design to test personalization quality.Key ResponsibilitiesDesign and execute multi-turn conversational prompts based on personal contextEvaluate AI-generated responses for accuracy, relevance, and personalization qualityCompare model responses side-by-side (SxS) and rank effectivenessIdentify grounding issues, incorrect inferences, or misleading outputsAssess integration of personal data for natural and helpful responsesProvide structured feedback and annotations to improve AI performanceMaintain strict data handling and confidentiality standardsEnsure evaluation data hygiene and compliance with project guidelinesRequired QualificationsHigh proficiency in reading and writing EnglishMinimum 1 year experience in customer support, content moderation, AI evaluation, or similar roleStrong analytical thinking and decision-making abilityExperience evaluating nuanced or ambiguous digital contentWillingness to use primary personal Google account with enabled data sourcesFull-time availability with PST time zone overlapReliable laptop/desktop and stable internet connectionAbility to work independently in a remote environmentPreferred BackgroundExperience in data annotation, AI quality evaluation, or content reviewFamiliarity with digital support environments (chat, email, platforms)Experience handling escalations or customer-facing quality review tasksBachelor’s degree (BS/BA) or equivalent experience in a relevant analytical fieldKey SkillsPersonalization quality evaluationPrompt design and conversational testingAnalytical reasoning and structured feedback writingAttention to detail and quality judgmentClear written communicationOffer DetailsContractor engagement (2 months)30 or 40 hours per week optionsMinimum 4 hours daily overlap with PSTGlobal 24-hour operations teamSelection ProcessJob Interest FormOnline Assessment (complete within 24 hours)Pre-onboarding discussion