类库
› rvLLM
solidsf-inc/rvLLM
rvLLM是用Rust实现的高性能Gemma 4大模型推理引擎。它通过原生代码处理加载、调度和KV缓存，支持NVIDIA CUDA和Apple Metal后端。提供兼容OpenAI格式的HTTP API服务器，适用于需要低延迟和高效率的本地或私有化部署场景。
技术栈
v3/crates/rvllm-attention rust
查看全部依赖 (4)
依赖
libloading
0.8
rvllm-core
rvllm-kernels
rvllm-mem
v3/crates/rvllm-bench rust
查看全部依赖 (7)
依赖
rvllm-core
rvllm-cutlass
rvllm-fused
rvllm-kernels
rvllm-loader
rvllm-mem
rvllm-runtime
v3/crates/rvllm-core rust
查看全部依赖 (4)
依赖
half
serde
serde_json
thiserror
v3/crates/rvllm-cutlass rust
查看全部依赖 (3)
依赖
rvllm-core
rvllm-kernels
rvllm-mem
v3/crates/rvllm-fused rust
查看全部依赖 (4)
依赖
half
rvllm-core
rvllm-kernels
rvllm-mem
v3/crates/rvllm-graph rust
查看全部依赖 (3)
依赖
rvllm-core
rvllm-mem
rvllm-metadata
v3/crates/rvllm-imageio rust
查看全部依赖 (1)
开发依赖
base64
0.22
v3/crates/rvllm-invariants rust
查看全部依赖 (1)
开发依赖
toml
0.9
v3/crates/rvllm-kernels rust
查看全部依赖 (2)
依赖
rvllm-core
rvllm-mem
v3/crates/rvllm-loader rust
查看全部依赖 (11)
依赖
half
lru
0.16.3
memmap2
0.9.11
parking_lot
0.12
rayon
1.10
rvllm-core
rvllm-mem
rvllm-metal
safetensors
0.4
serde
serde_json
v3/crates/rvllm-mcp rust
查看全部依赖 (4)
依赖
serde
serde_json
tracing
tracing-subscriber
v3/crates/rvllm-mem rust
查看全部依赖 (2)
依赖
half
rvllm-core
v3/crates/rvllm-metadata rust
查看全部依赖 (3)
依赖
rvllm-core
rvllm-mem
sha2
0.10
v3/crates/rvllm-metal rust
查看全部依赖 (4)
依赖
half
rvllm-core
=0.3.0
thiserror
tracing
v3/crates/rvllm-runtime rust
查看全部依赖 (12)
依赖
rvllm-attention
rvllm-core
rvllm-cutlass
rvllm-fused
rvllm-graph
rvllm-kernels
rvllm-loader
rvllm-mem
rvllm-metadata
rvllm-sampling
serde_json
tracing
v3/crates/rvllm-sampling rust
查看全部依赖 (4)
依赖
rvllm-core
rvllm-fused
rvllm-kernels
rvllm-mem
v3/crates/rvllm-serve rust
查看全部依赖 (4)
依赖
rvllm-core
rvllm-kernels
rvllm-loader
rvllm-mem
v3/crates/rvllm-vision rust
查看全部依赖 (5)
依赖
anyhow
=1.0.103
candle-core
=0.10.2
candle-nn
=0.10.2
serde
serde_json
截图
