ai-generation-safety
GitHub审查并加固PANaCEa中Gemini API集成、回退机制及输出验证，确保AI生成内容（如试题、临床模拟）的临床准确性与安全性。适用于所有涉及Gemini调用的开发、故障排查及新功能审计场景。
Trigger Scenarios
Install
npx skills add NeverSight/learn-skills.dev --skill ai-generation-safety -g -y
SKILL.md
Frontmatter
{
"name": "ai-generation-safety",
"description": "Review and harden AI generation pipelines, Gemini API integration, fallback paths, and output validation in PANaCEa. Use this skill whenever working on question generation, AI tutoring, Ghost Grader behavioral analysis, OSCE simulation, content enrichment, or any code that calls Gemini — even if the user just says 'the AI is returning bad results' or 'generation is failing'. Also use when adding new AI features, reviewing rate limiting, or auditing clinical accuracy of generated content."
}
AI Generation Safety in PANaCEa
PANaCEa relies on Google Gemini for question generation, behavioral analysis, tutoring, and clinical simulation. Every AI-generated output is presented to PA students and may influence learning or clinical reasoning. This skill ensures all AI integrations are validated, rate-limited, fallback-safe, and clinically sound.
Core Principle
No AI output reaches a student without validation and a safe local fallback. If Gemini is unavailable, the system must degrade gracefully.
PANaCEa AI Surface Area
1. Question Generation
Files:
functions/api/questions/generate-batch.ts— batch generation endpointfunctions/api/_shared/question-generator.ts— Gemini calls for single questionsfunctions/api/_shared/aiQuestionService.ts— orchestration layerlib/verified-question-generator.ts— client-side verification
Models: gemini-2.5-flash (default)
Outputs: Questions, vignettes, options, explanations, difficulty levels, distractor rationale
Risk: Hallucinated drug interactions, incorrect diagnosis, malformed JSON, distractor validity
2. Ghost Grader (Behavioral Confidence Analysis)
Files:
functions/api/_shared/analyzeBehaviorGemini.ts— main implementationlib/implicit-metrics.ts— fallback local derivation
Models: gemini-2.0-flash-exp
Inputs: Time-to-first-click, answer switches, hover duration, correctness, dwell time
Outputs: Confidence (0–1), implied FSRS rating (1–4)
Fallback: deriveContinuousRating() — local Bayesian accumulation (low-cost backup)
Status: GOOD — has robust fallback; Gemini fail → local metrics (lines 134–150)
3. Clinical Tutor
Files:
functions/api/tutor/chat.ts— deep-think tutoring endpoint- Uses extended thinking + Google Search grounding
Models: gemini-3-flash-preview (primary), fallback gemini-2.5-flash
Features: Thought signatures preserved across turns, Socratic dialogue, PANCE blueprint focus
Rate limit: 10 req/min per ai_endpoints config
Risk: Outdated clinical guidelines, hallucinated citations, missing thought signature propagation
4. OSCE Live Simulation
Files:
functions/api/osce/live-engine.ts— ephemeral token generationfunctions/api/osce/chat.ts— ongoing encounter chat- Client-side WebSocket to Gemini (real-time audio + text)
Models: gemini-2.0-flash-exp
Features: Voice + text, dynamic persona (pain/mood), tools (get_current_vitals, reveal_lab_result)
Fallback: None currently (real-time WebSocket; graceful degrade to text-only)
Risk: Missed clinical reasoning cues, persona hallucination, tool hallucination
5. Embedding & Semantic Search
Files: lib/services/ (content enrichment via embeddings)
Model: text-embedding-005
Risk: Dimension mismatch, stale cached vectors, poor retrieval for niche conditions
6. Rate Limiting & Cost Control
Files:
functions/api/_shared/rateLimiter.ts— sliding window limiterRATE_LIMITS.ai_endpoints— 10 req/min
Status: Per-user tracking via auth ID; KV fallback for distributed rate limiting
Validation Checklist
A. Schema & Type Safety
- All Gemini responses parsed via
zodschema (e.g.,geminiBehaviorOutputSchema) - JSON parsing wrapped in try-catch; fallback on parse error
-
safeParse()used; never trust raw JSON - Response fields bounded: string length limits, array sizes capped
Red flags:
- Direct
JSON.parse()without error handling - No
zodvalidation - Unbounded arrays or deeply nested objects from Gemini
B. Question Distractor Quality
File: lib/distractorValidation.ts
Check before persisting to PreGeneratedQuestion:
- Each distractor is plausible for a slightly different patient
- No "obviously wrong" options (e.g., "Pregnancy" as an answer option for a male patient)
- Distractor count matches question type (MC: 3 options, typically 1 correct)
- No duplicate options
- Rationale for each distractor explains why it's wrong for THIS stem
Common failure: Generic drug-interaction distractors that work for many questions (low signal)
C. Clinical Accuracy Signals
- Drug names match FDA/Micromedex (watch for Gemini hallucination: "amoxicillin-clavulanic acid" vs "amoxicillin-clavulanate")
- Dosages align with PA formulary or PANCE reference
- Lab reference ranges use US units
- Diagnosis criteria match DSM-5, ICD-10, or board standard
- Procedure complications are real (not made up)
Critical: Run spot-checks on 10% of batch questions against known sources (Micromedex, UP TO DATE, PANCE Q-Bank)
D. JSON Safety
- Response
responseMimeType: 'application/json'enabled for Ghost Grader and question gen - Empty response bodies handled (Gemini sometimes returns
candidates: []) - Malformed JSON captured with context logging
Fallback Design Patterns
GOOD: Ghost Grader (Resilient)
try {
const result = await analyzeBehaviorGemini(env, params);
return result; // { confidence, impliedRating, rawText }
} catch (error) {
console.warn('[AI_SAFETY] Gemini API failed. Attempting local fallback.');
const fallbackMetrics = deriveContinuousRating({...});
return { confidence: fallbackMetrics.confidence, impliedRating: fallbackMetrics.discreteRating };
}
Status: Two-tier fallback (Gemini → local heuristics). Student always gets a confidence score.
NEEDS WORK: Question Generation (No Fallback)
Current: generateQuestionsWithGemini() fails → returns empty array → batch aborts
Improvement: Reserve pre-generated pool; if generation fails, draw from backfill questions
NEEDS WORK: OSCE Live (No Fallback)
Current: WebSocket disconnect → encounter paused
Improvement: Fall back to text-only chat on Gemini unavailability; continue assessment
Rate Limiting & Cost Audit
Current Config
ai_endpoints: { maxRequests: 10, windowSeconds: 60 } // 10 req/min per user
Cost Drivers
| Feature | Model | Est. Input Tokens | Output Tokens | Cost/Call | Limits |
|---|---|---|---|---|---|
| Question Gen | gemini-2.5-flash | 2000–3000 | 500–800 | $0.01–0.02 | 50 q/batch max |
| Ghost Grader | gemini-2.0-flash-exp | 500–800 | 100–200 | ~$0.001 | 10 req/min |
| Tutor (thinking) | gemini-3-flash-preview | 5000–8000 | 1000–2000 | $0.10–0.30 | 10 req/min |
| OSCE Real-time | gemini-2.0-flash-exp | Streaming | Streaming | $0.10–0.50/min | Depends on WebSocket duration |
Audit Checklist
- All Gemini calls include rate limit check before request
- Failed rate-limit hits return 429 with
Retry-Afterheader - Batch size capped at 50 questions (prevent runaway generation)
- Tutor sessions have max token budget per turn (2048 tokens default)
- OSCE WebSocket closed after 15 min inactivity
- Cost tracking logged:
logger.info('Gemini call', { model, inputTokens, outputTokens, cost })
Common Failure Modes & Fixes
1. Gemini 429 (Rate Limit) Cascade
Problem: Student retries question generation every 2 seconds → 30 failed calls → cascade
Fix:
- Exponential backoff in client (use
retry-afterheader) - Server queues excess generation jobs; returns
{ queued: true, eta: 30s } - See:
functions/api/_shared/rateLimiter.ts
2. Malformed JSON from Gemini
Problem: generationConfig: { responseMimeType: 'application/json' } set, but Gemini returns markdown or plain text
Symptoms:
JSON.parse()throwszodvalidation fails silently- Fallback code path never exercised
Fix:
- Always use
zod.safeParse(); log parse failures - Wrap Gemini calls in timeout handler (
fetchWithTimeout) - Unit test response parsing with 10 real Gemini responses
3. Hallucinated Medical Facts
Problem: Gemini generates plausible-sounding but incorrect distractor: "Vancomycin dosage: 2000 mg QID" (should be 15–20 mg/kg Q8–12H)
Fix:
- Spot-check 10% of batches against Micromedex
- Flagging system: if distractor matches known incorrect drug dosage, escalate to review
- Implement automated check: validate drug names + dosages against trusted API (RxNorm)
4. Embedding Dimension Mismatch
Problem: text-embedding-005 returns 768-dim vectors; code expects 1536
Fix:
- Store embedding model name in metadata
- On retrieval, verify dimension matches; if not, re-embed
5. Missing Thought Signature (Tutor)
Problem: gemini-3-flash-preview returns thought signature, but client doesn't pass it back in next turn → model loses reasoning
Fix:
- Schema requires
thoughtSignature?: stringon history turns - Client extracts from previous model response:
data.candidates?.[0]?.content?.parts?.find(p => p.thoughtSignature) - Always include in next request:
previousThoughtSignatures: [sig]orparts: [{ thoughtSignature: sig }]
6. Distractor Validation Failures
Problem: validateDistractors() rejects a batch; batch is discarded; student sees no new questions
Fix:
- Log rejection reason with full question + options for review
- Implement retry with adjusted prompt (e.g., "avoid drug-interaction distractors; use comorbidity-based distractors")
- Partial batch success: return valid questions, queue invalid ones for re-generation
Model Routing & Selection
When to Use Each Model
| Feature | Model | Why | Fallback |
|---|---|---|---|
| Question Gen | gemini-2.5-flash | Fast, cheap, good for structured output | gemini-2.0-flash-exp |
| Ghost Grader | gemini-2.0-flash-exp | Reliable JSON, low latency | local deriveContinuousRating() |
| Tutor (Deep Think) | gemini-3-flash-preview | Extended thinking for reasoning | gemini-2.5-flash (no thinking) |
| OSCE Real-time | gemini-2.0-flash-exp | Low latency for streaming voice | Text-only fallback |
| Embeddings | text-embedding-005 | 768 dims, clinical domain tuned | — |
Model Availability Handling
async function callGeminiWithFallback(
primary: string,
fallback: string,
prompt: string
): Promise<string> {
try {
return await callGemini(primary, prompt);
} catch (e) {
if (e.status === 404 || e.message.includes('not found')) {
console.warn(`Primary model ${primary} unavailable. Trying fallback ${fallback}.`);
return await callGemini(fallback, prompt);
}
throw e;
}
}
Files to Inspect First
When user reports AI issue:
functions/api/_shared/analyzeBehaviorGemini.ts(Ghost Grader; easiest to test locally)functions/api/_shared/question-generator.ts(prompt engineering + schema)functions/api/_shared/rateLimiter.ts(check rate limit config)lib/distractorValidation.ts(distractor rules)functions/api/tutor/chat.ts(if tutor issue; check thought signature handling)functions/api/osce/live-engine.ts(if OSCE issue; check tool definitions)
To add fallback:
- Edit API endpoint to call fallback function in
catch()block - Update types to allow
fallback: trueflag in response - Test offline scenario in Vitest
Composes With
- clinical-safety-review — audit generated clinical content (distractor rationale, explanations) for tone, accuracy, and patient safety tone
- clinical-content-gen — enrichment via embeddings; ensure embedding model and dimension match
- cf-edge-api — ensure Gemini calls use
context.env.GEMINI_API_KEY, not rawprocess.env - panacea-verify — after changes, run
npm testto ensure schema parsing and fallback paths are covered
Version History
- e0220ca Current 2026-07-05 22:11
