llm-architect

GitHub

专注LLM系统架构设计,涵盖模型选型、RAG管道构建、微调策略及生产环境部署优化。提供性能、成本与安全平衡方案,适用于大规模语言模型应用的全链路设计与基础设施规划。

data/skills-md/404kidwiz/claude-supercode-skills/llm-architect/SKILL.md NeverSight/learn-skills.dev

Trigger Scenarios

需要设计端到端LLM系统架构 进行模型选型与推理服务基础设施规划 实施RAG检索增强生成管道 优化LLM推理成本与延迟

Install

npx skills add NeverSight/learn-skills.dev --skill llm-architect -g -y
More Options

Non-standard path

npx skills add https://github.com/NeverSight/learn-skills.dev/tree/main/data/skills-md/404kidwiz/claude-supercode-skills/llm-architect -g -y

Use without installing

npx skills use NeverSight/learn-skills.dev@llm-architect

指定 Agent (Claude Code)

npx skills add NeverSight/learn-skills.dev --skill llm-architect -a claude-code -g -y

安装 repo 全部 skill

npx skills add NeverSight/learn-skills.dev --all -g -y

预览 repo 内 skill

npx skills add NeverSight/learn-skills.dev --list

SKILL.md

Frontmatter
{
    "name": "llm-architect",
    "description": "Use when user needs LLM system architecture, model deployment, optimization strategies, and production serving infrastructure. Designs scalable large language model applications with focus on performance, cost efficiency, and safety."
}

LLM Architect

Purpose

Provides expert large language model system architecture for designing, deploying, and optimizing LLM applications at scale. Specializes in model selection, RAG (Retrieval Augmented Generation) pipelines, fine-tuning strategies, serving infrastructure, cost optimization, and safety guardrails for production LLM systems.

When to Use

  • Designing end-to-end LLM systems from requirements to production
  • Selecting models and serving infrastructure for specific use cases
  • Implementing RAG (Retrieval Augmented Generation) pipelines
  • Optimizing LLM costs while maintaining quality thresholds
  • Building safety guardrails and compliance mechanisms
  • Planning fine-tuning vs RAG vs prompt engineering strategies
  • Scaling LLM inference for high-throughput applications

Quick Start

Invoke this skill when:

  • Designing end-to-end LLM systems from requirements to production
  • Selecting models and serving infrastructure for specific use cases
  • Implementing RAG (Retrieval Augmented Generation) pipelines
  • Optimizing LLM costs while maintaining quality thresholds
  • Building safety guardrails and compliance mechanisms

Do NOT invoke when:

  • Simple API integration exists (use backend-developer instead)
  • Only prompt engineering needed without architecture decisions
  • Training foundation models from scratch (almost always wrong approach)
  • Generic ML tasks unrelated to language models (use ml-engineer)

Decision Framework

Model Selection Quick Guide

Requirement Recommended Approach
Latency <100ms Small fine-tuned model (7B quantized)
Latency <2s, budget unlimited Claude 3 Opus / GPT-4
Latency <2s, domain-specific Claude 3 Sonnet fine-tuned
Latency <2s, cost-sensitive Claude 3 Haiku
Batch/async acceptable Batch API, cheapest tier

RAG vs Fine-Tuning Decision Tree

Need to customize LLM behavior?
│
├─ Need domain-specific knowledge?
│  ├─ Knowledge changes frequently?
│  │  └─ RAG (Retrieval Augmented Generation)
│  └─ Knowledge is static?
│     └─ Fine-tuning OR RAG (test both)
│
├─ Need specific output format/style?
│  ├─ Can describe in prompt?
│  │  └─ Prompt engineering (try first)
│  └─ Format too complex for prompt?
│     └─ Fine-tuning
│
└─ Need latency <100ms?
   └─ Fine-tuned small model (7B-13B)

Architecture Pattern

[Client] → [API Gateway + Rate Limiting]
              ↓
         [Request Router]
          (Route by intent/complexity)
              ↓
    ┌────────┴────────┐
    ↓                 ↓
[Fast Model]    [Powerful Model]
(Haiku/Small)   (Sonnet/Large)
    ↓                 ↓
[Cache Layer] ← [Response Aggregator]
    ↓
[Logging & Monitoring]
    ↓
[Response to Client]

Core Workflow: Design LLM System

1. Requirements Gathering

Ask these questions:

  • Latency: What's the P95 response time requirement?
  • Scale: Expected requests/day and growth trajectory?
  • Accuracy: What's the minimum acceptable quality? (measurable metric)
  • Cost: Budget constraints? ($/request or $/month)
  • Data: Existing datasets for evaluation? Sensitivity level?
  • Compliance: Regulatory requirements? (HIPAA, GDPR, SOC2, etc.)

2. Model Selection

def select_model(requirements):
    if requirements.latency_p95 < 100:  # milliseconds
        if requirements.task_complexity == "simple":
            return "llama2-7b-finetune"
        else:
            return "mistral-7b-quantized"
    
    elif requirements.latency_p95 < 2000:
        if requirements.budget == "unlimited":
            return "claude-3-opus"
        elif requirements.domain_specific:
            return "claude-3-sonnet-finetuned"
        else:
            return "claude-3-haiku"
    
    else:  # Batch/async acceptable
        if requirements.accuracy_critical:
            return "gpt-4-with-ensemble"
        else:
            return "batch-api-cheapest-tier"

3. Prototype & Evaluate

# Run benchmark on eval dataset
python scripts/evaluate_model.py \
  --model claude-3-sonnet \
  --dataset data/eval_1000_examples.jsonl \
  --metrics accuracy,latency,cost

# Expected output:
# Accuracy: 94.3%
# P95 Latency: 1,245ms
# Cost per 1K requests: $2.15

4. Iteration Checklist

  • Latency P95 meets requirement? If no → optimize serving (quantization, caching)
  • Accuracy meets threshold? If no → improve prompts, fine-tune, or upgrade model
  • Cost within budget? If no → aggressive caching, smaller model routing, batching
  • Safety guardrails tested? If no → add content filters, PII detection
  • Monitoring dashboards live? If no → set up Prometheus + Grafana
  • Runbook documented? If no → document common failures and fixes

Cost Optimization Strategies

Strategy Savings When to Use
Semantic caching 40-80% 60%+ similar queries
Multi-model routing 30-50% Mixed complexity queries
Prompt compression 10-20% Long context inputs
Batching 20-40% Async-tolerant workloads
Smaller model cascade 40-60% Simple queries first

Safety Checklist

  • Content filtering tested against adversarial examples
  • PII detection and redaction validated
  • Prompt injection defenses in place
  • Output validation rules implemented
  • Audit logging configured for all requests
  • Compliance requirements documented and validated

Red Flags - When to Escalate

Observation Action
Accuracy <80% after prompt iteration Consider fine-tuning
Latency 2x requirement Review infrastructure
Cost >2x budget Aggressive caching/routing
Hallucination rate >5% Add RAG or stronger guardrails
Safety bypass detected Immediate security review

Quick Reference: Performance Targets

Metric Target Critical
P95 Latency <2x requirement <3x requirement
Accuracy >90% >80%
Cache Hit Rate >60% >40%
Error Rate <1% <5%
Cost/1K requests Within budget <150% budget

Additional Resources

  • Detailed Technical Reference: See REFERENCE.md

    • RAG implementation workflow
    • Semantic caching patterns
    • Deployment configurations
  • Code Examples & Patterns: See EXAMPLES.md

    • Anti-patterns (fine-tuning when prompting suffices, no fallback)
    • Quality checklist for LLM systems
    • Resilient LLM call patterns

Version History

  • e0220ca Current 2026-07-05 21:15

Same Skill Collection

data/skills-md/00prabalk00/claude-skills/knowledge-base-gap-finder/SKILL.md
data/skills-md/01000001-01001110/agent-jira-skills/jira-agile/SKILL.md
data/skills-md/01000001-01001110/agent-jira-skills/jira-auth/SKILL.md
data/skills-md/01000001-01001110/agent-jira-skills/jira-issues/SKILL.md
data/skills-md/01000001-01001110/agent-jira-skills/jira-project-management/SKILL.md
data/skills-md/01000001-01001110/agent-jira-skills/jira-projects/SKILL.md
data/skills-md/01000001-01001110/agent-jira-skills/jira-safe/SKILL.md
data/skills-md/01000001-01001110/agent-jira-skills/jira-search/SKILL.md
data/skills-md/01000001-01001110/agent-jira-skills/jira-spaces/SKILL.md
data/skills-md/01000001-01001110/agent-jira-skills/jira-transitions/SKILL.md
data/skills-md/0731coderlee-sudo/wechat-publisher/wechat-publisher/SKILL.md
data/skills-md/0froq/skills/conventionalcommits/SKILL.md
data/skills-md/0froq/skills/nuxt/SKILL.md
data/skills-md/0froq/skills/oq/SKILL.md
data/skills-md/0froq/skills/pinia/SKILL.md
data/skills-md/0froq/skills/pnpm/SKILL.md
data/skills-md/0froq/skills/slidev/SKILL.md
data/skills-md/0froq/skills/tsdown/SKILL.md
data/skills-md/0froq/skills/turborepo/SKILL.md
data/skills-md/0froq/skills/unocss/SKILL.md
data/skills-md/0froq/skills/vitepress/SKILL.md
data/skills-md/0froq/skills/vitest/SKILL.md
data/skills-md/0froq/skills/vue-best-practices/SKILL.md
data/skills-md/0froq/skills/vue-router-best-practices/SKILL.md
data/skills-md/0froq/skills/vue-testing-best-practices/SKILL.md
data/skills-md/0froq/skills/vue/SKILL.md
data/skills-md/0froq/skills/vueuse-functions/SKILL.md
data/skills-md/0froq/skills/web-design-guidelines/SKILL.md
data/skills-md/0juano/agent-skills/bondterminal-x402/SKILL.md
data/skills-md/0juano/agent-skills/edgeone-pages-deploy/SKILL.md
data/skills-md/0juano/agent-skills/ley-ar/SKILL.md
data/skills-md/0juano/agent-skills/ticktick/SKILL.md
data/skills-md/0juano/agent-skills/x-image-cards/SKILL.md
data/skills-md/0juano/x-image-cards/x-image-cards/SKILL.md
data/skills-md/0x0funky/agent-sprite-forge/generate2dsprite/SKILL.md
data/skills-md/0x0funky/agent-sprite-forge/video2dsprite/SKILL.md
data/skills-md/0x2e/superpowers/brainstorming/SKILL.md
data/skills-md/0x2e/superpowers/dispatching-parallel-agents/SKILL.md
data/skills-md/0x2e/superpowers/executing-plans/SKILL.md
data/skills-md/0x2e/superpowers/finishing-a-development-branch/SKILL.md
data/skills-md/0x2e/superpowers/receiving-code-review/SKILL.md
data/skills-md/0x2e/superpowers/requesting-code-review/SKILL.md
data/skills-md/0x2e/superpowers/subagent-driven-development/SKILL.md
data/skills-md/0x2e/superpowers/systematic-debugging/SKILL.md
data/skills-md/0x2e/superpowers/test-driven-development/SKILL.md
data/skills-md/0x2e/superpowers/using-git-worktrees/SKILL.md
data/skills-md/0x2e/superpowers/using-superpowers/SKILL.md
data/skills-md/0x2e/superpowers/verification-before-completion/SKILL.md
data/skills-md/0x2e/superpowers/writing-plans/SKILL.md
data/skills-md/0x2e/superpowers/writing-skills/SKILL.md

Metadata

Files
0
Version
e4a0f95
Hash
6719ed00
Indexed
2026-07-05 21:15

Home - Wiki
Copyright © 2011-2026 iteam. Current version is 2.155.2. UTC+08:00, 2026-08-05 18:34
浙ICP备14020137号-1 $Map of visitor$