Agent Skills › waybarrios/opencode-power-pack › huggingface-local-models

huggingface-local-models

GitHub

提供在本地通过 llama.cpp 运行 Hugging Face GGUF 模型的完整工作流，涵盖模型搜索、量化选择、服务器启动及兼容性转换。

skills/huggingface-local-models/SKILL.md waybarrios/opencode-power-pack

触发场景

用户询问如何在本地部署大语言模型需要查找或下载特定格式的 GGUF 模型文件配置 llama-cli 或 llama-server 进行本地推理

安装

npx skills add waybarrios/opencode-power-pack --skill huggingface-local-models -g -y

更多选项

不安装直接使用

npx skills use waybarrios/opencode-power-pack@huggingface-local-models

指定 Agent (Claude Code)

npx skills add waybarrios/opencode-power-pack --skill huggingface-local-models -a claude-code -g -y

安装 repo 全部 skill

npx skills add waybarrios/opencode-power-pack --all -g -y

预览 repo 内 skill

npx skills add waybarrios/opencode-power-pack --list

SKILL.md

Frontmatter

{
    "name": "huggingface-local-models",
    "license": "Apache-2.0 (modified; see UPSTREAMS.json)",
    "description": "Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact GGUF file lookup, conversion, and OpenAI-compatible local serving."
}

Hugging Face Local Models

Search the Hugging Face Hub for llama.cpp-compatible GGUF repos, choose the right quant, and launch the model with llama-cli or llama-server.

Default Workflow

Search the Hub with apps=llama.cpp.
Open https://huggingface.co/<repo>?local-app=llama.cpp.
Prefer the exact HF local-app snippet and quant recommendation when it is visible.
Confirm exact .gguf filenames with https://huggingface.co/api/models/<repo>/tree/main?recursive=true.
Launch with llama-cli -hf <repo>:<QUANT> or llama-server -hf <repo>:<QUANT>.
Fall back to --hf-repo plus --hf-file when the repo uses custom file naming.
Convert from Transformers weights only if the repo does not already expose GGUF files.

Quick Start

Install llama.cpp

brew install llama.cpp
winget install llama.cpp

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
make

Authenticate for gated repos

hf auth login

Search the Hub

https://huggingface.co/models?apps=llama.cpp&sort=trending
https://huggingface.co/models?search=Qwen3.6&apps=llama.cpp&sort=trending
https://huggingface.co/models?search=<term>&apps=llama.cpp&num_parameters=min:0,max:24B&sort=trending

Run directly from the Hub

llama-cli -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M

Run an exact GGUF file

llama-server \
    --hf-repo unsloth/Qwen3.6-35B-A3B-GGUF \
    --hf-file Qwen3.6-35B-A3B-UD-Q4_K_M.gguf \
    -c 4096

Convert only when no GGUF is available

hf download <repo-without-gguf> --local-dir ./model-src
python convert_hf_to_gguf.py ./model-src \
    --outfile model-f16.gguf \
    --outtype f16
llama-quantize model-f16.gguf model-q4_k_m.gguf Q4_K_M

Smoke test a local server

With llama-server running (as started above), verify it responds:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer no-key" \
  -d '{
    "messages": [
      {"role": "user", "content": "Write a limerick about exception handling"}
    ]
  }'

Quant Choice

Prefer the exact quant that HF marks as compatible on the ?local-app=llama.cpp page.
Keep repo-native labels such as UD-Q4_K_M instead of normalizing them.
Default to Q4_K_M unless the repo page or hardware profile suggests otherwise.
Prefer Q5_K_M or Q6_K for code or technical workloads when memory allows.
Consider Q3_K_M, Q4_K_S, or repo-specific IQ / UD-* variants for tighter RAM or VRAM budgets.
Treat mmproj-*.gguf files as projector weights, not the main checkpoint.

Load References

Read hub-discovery.md for URL-first workflows, model search, tree API extraction, and command reconstruction.
Read quantization.md for format tables, model scaling, quality tradeoffs, and imatrix.
Read hardware.md for Metal, CUDA, ROCm, or CPU build and acceleration details.

Resources

llama.cpp: https://github.com/ggml-org/llama.cpp
Hugging Face GGUF + llama.cpp docs: https://huggingface.co/docs/hub/gguf-llamacpp
Hugging Face Local Apps docs: https://huggingface.co/docs/hub/main/local-apps
Hugging Face Local Agents docs: https://huggingface.co/docs/hub/agents-local
GGUF converter Space: https://huggingface.co/spaces/ggml-org/gguf-my-repo