* Evaluate LLM-generated responses across diverse general topics
* Conduct fact-checking using trusted public sources and external tools
* Annotate strengths, weaknesses, and factual inaccuracies in model outputs
* Assess reasoning quality, clarity, tone, and completeness
* Ensure responses align with conversational guidelines and system standards
* Apply structured taxonomies, benchmarks, and evaluation frameworks consistently
Position: Generalist – English & German
Type: Hourly contract
Compensation: $39.78 per hour
Location: Remote
Commitment: Full-time or Part-time
Role Responsibilities
* Evaluate LLM-generated responses across diverse general topics
* Conduct fact-checking using trusted public sources and external tools
* Annotate strengths, weaknesses, and factual inaccuracies in model outputs
* Assess reasoning quality, clarity, tone, and completeness
* Ensure responses align with conversational guidelines and system standards
* Apply structured taxonomies, benchmarks, and evaluation frameworks consistently
Requirements
* Bachelor’s degree
* Native or ILR 5 / CEFR C2 fluency in German
* Fluency in English (written and spoken)
* Strong writing skills with ability to provide nuanced feedback
* Experience using large language models (LLMs)
* Strong analytical thinking and attention to detail
* Comfort working across varied topics and structured evaluation frameworks
* Strong college-level mathematics skills
Application Process (Takes 20 Mins)
* Upload your resume.
* Complete an interview.
* Submit a short form.
#J-18808-Ljbffr