AI Evaluation

---

Model

All AI evaluation uses claude-sonnet-4-6 from Anthropic.

---

Automated Screening Evaluation

Triggered after POST /v1/screening/:token/submit. Runs as an inline async call (Phase 1A) or via BullMQ job queue (Phase 6 hardening).

Service: src/services/screeningEvaluationService.ts

Steps:

1. Fetch Interview with stageData.screeningResponses
2. Resolve effective questions:
   • If Interview.stageOverrides.screeningConfig is set → use those
   • Otherwise → use JobOpening.stages[stageIndex].screeningConfig
3. Build prompt: job description + all Q&A pairs
4. Call Claude API → parse response into screeningAiReport shape
5. Compute candidateAggregateScore (0–100)
6. Interview.findByIdAndUpdate(id, { 'stageData.screeningAiReport': report, candidateAggregateScore: score })

Stored (recruiter-only):

screeningAiReport: {
  score: number;        // 0–100
  summary: string;
  strengths: string[];
  concerns: string[];
  recommendation: 'proceed' | 'reject' | 'review';
  generatedAt: Date;
}

Exposed to candidate: Only candidateAggregateScore (single 0–100 number). No individual question scores, no summary, no recommendation.

---

AI-Assisted Technical Interview

stageTypeKey: 'technical_ai_assisted'

Service: src/services/aiInterviewerService.ts

• Conducts multi-turn Q&A via WebSocket or HTTP streaming
• Initial question: ScenarioQuestion.context
• Follow-up generation: based on followUpGuide + candidate's previous answer
• Stores each turn in Interview.stageData.aiTechnicalResponses
• On session completion: generates aiReport with criteriaScores from evaluationRubric

---

AI Conversational Interview

stageTypeKey: 'ai_conversational'

• Real-time voice agent (TTS + STT integration — Deepgram or similar)
• Stores turns in Interview.stageData.conversationalTurns
• audioUrl stored per turn (recruiter-only — never exposed to candidate)
• On session completion: generates aiReport

Model: claude-sonnet-4-6 for Q&A generation; TTS/STT via Deepgram or similar for voice.

---

AI Report Schema (Recruiter-Only)

Used by technical_ai_assisted and ai_conversational stages:

aiReport: {
  overallScore: number;
  summary: string;
  criteriaScores: { criterion: string; score: number; notes: string }[];
  recommendation: 'proceed' | 'reject' | 'review';
  generatedAt: Date;
}

Never exposed to candidates. Candidates see only their own Q&A transcript + candidateAggregateScore.

---

AI Chatbot in Live Sessions

For live_1on1 stages with toolsConfig.aiChatbot: true, the live room has an AI assistant panel (existing feature). This is separate from the AI interviewer — it's an assistant for the human interviewer, not an autonomous session conductor.

---

Private Question Push (Live Sessions)

During a live_1on1 session, the interviewer can push questions to the candidate's screen:

1. Interviewer calls GET /v1/interviews/:id/private-questions (auth-gated — recruiter JWT only)
2. Returns [{ _id, text, interviewerHints, expectedAnswerGuide }]
3. Interviewer clicks "Push to candidate" → emits socket event push_question
4. Candidate's screen shows the pushed question text in a visible card

The interviewerHints and expectedAnswerGuide fields are never sent to the candidate's browser.

---

Phase Delivery

Phase	AI Feature
Phase 1A	Automated screening evaluation (screeningAiReport, candidateAggregateScore)
Phase 1B	AI chatbot in live rooms (existing), private question push
Phase 4	AI technical interviewer, AI conversational interviewer
Phase 6	AI evaluation moved to BullMQ job queue for reliability