Shapd AI
Mathematics & Reasoning Evaluator
$45 - $95/hourRequired Skills
About Shapd AI
Shapd AI is an expert-led AI data and evaluation company. We connect skilled professionals with AI teams that need reliable human judgment for task authoring, model evaluation, red teaming, rubric design and data quality workflows.
Our work turns real-world expertise into high-signal training data, evaluations and feedback loops that help AI systems learn, reason and perform more reliably.
Job Title: Mathematics & Reasoning Evaluator
Job Type: Contractor (Flexible)
Location: Remote
Job Summary
Design and evaluate mathematical reasoning tasks that measure how AI models solve multi-step problems, explain logic and handle ambiguity.
In this role, you will apply your domain expertise to help AI teams test capabilities, identify failure modes and build more reliable evaluation workflows. The work is structured, remote and reviewed against clear quality standards.
What You Will Do
- Create original reasoning tasks across algebra, calculus, probability, discrete math or adjacent STEM topics.
- Assess model solutions for correctness, skipped reasoning, invalid assumptions and clarity.
- Write anchored rubrics that make complex mathematical evaluation more consistent.
Required Skills and Qualifications
- Strong academic or professional background in mathematics, statistics, physics, engineering or a related field.
- Ability to identify subtle reasoning errors and communicate corrections clearly.
- Careful attention to notation, assumptions and final-answer validation.
Preferred Qualifications
- Teaching, tutoring, research, assessment design or contest math experience.
- Experience reviewing AI-generated math or STEM content.
Process
- Apply to the role and submit your basic profile details.
- Complete a structured screening or calibration step if required.
- Shapd AI reviews your background, availability and domain fit.
- If selected, you receive project-specific onboarding instructions.
Compensation Structure
Compensation depends on project requirements, task complexity and quality standards. Pay ranges are listed for planning purposes and may vary by project scope.
Start Timeline and Availability
Some projects begin quickly, while others depend on client timelines and expert matching. Applying to a role does not guarantee immediate placement, but it helps us evaluate your profile for relevant opportunities.