首页 话题 library logo 文库 码库 小摊
更多
类库 › kimi-k3-in-c
FareedKhan-dev

FareedKhan-dev/kimi-k3-in-c

这是一个在纯C99环境下运行的Kimi K3大模型推理引擎。无需GPU、BLAS或外部框架，仅用单CPU和8GB内存即可运行2.78万亿参数模型。代码轻量便携，旨在展示极致优化能力，主要面向Linux x86-64平台。

7,542 1,211 7,542 9
在 GitHub 上查看
FareedKhan-dev/kimi-k3-in-c

标签

avx2 c99 cpu-inference deep-learning from-scratch inference-engine kimi-k3 linear-attention llm llm-inference machine-learning memory-efficient mixture-of-experts moe mxfp4 quantization simd systems-programming transformer zero-dependencies

截图

A small resident working set on top, the model itself on NVMe underneath, and a few labelled pipes between them
Four steps from a server cluster down to an ordinary laptop, with the same output at both ends
What each preset actually costs in memory
The naive requirement: every parameter resident at bf16
One token wakes 16 experts and leaves 880 asleep
Only 16 of 896 experts fire per layer, so most of the model sleeps
The always-active set: 113.49 GB at bf16, everything else is streamable
Four reductions, and the output is identical at both ends
One binary, four kinds of machine, and one identical answer
A 176 KB binary that runs a 1.56 TB model
safetensors: one length, one header, then raw bytes at known offsets
Index the shard, read the exact bytes on demand, then drop the pages
96 shards, 1,560,936,091,448 bytes, verified one file at a time
One-based layer indices, and 92 and 93 are both MLA by design
Refuse rather than guess, because a guessed field gives you a different model
Every case goes through a file, never through argv
Four files in, byte-identical files back out
MXFP4: a 4-bit nibble scaled by one 8-bit exponent per 32 weights
One byte carries two weights, and the low nibble is the even one
Half a byte per weight plus the shared scale gives one expert exactly
The low nibble is the EVEN element, and reversing it is silently wrong
Half a byte per weight saves 4 TB, and never dequantizing saves 194 GB a token
RMSNorm with epsilon inside the square root, accumulated in double
A fixed reduction order, so scalar and AVX2 agree bit for bit
Same weights, three code paths, one hash
Where one token goes on the floor configuration: 80% of it is waiting on disk
The recurrent state is the same size at 10 tokens and at 100,000
Every token folds into the same fixed-size state
Why 69 layers are KDA: its state does not grow with context
Decay the state, read from it, write the delta, then read the updated state
A depthwise causal convolution of width 4 with the activation fused in
A sum of squares, not a mean, and applied to q and k only
The decay gate, with A indexed per head and not per channel
The delta rule: decay, read, write the difference, then read again
Nine ordered steps, and the numbering is not decoration
Norm first, then gate, then project, and that order is not interchangeable
One latent per position is cached, and k and v are rebuilt on use
The query and the token both pass through a small shared latent
Twenty-four MLA layers, expanded k and v in fp32, per position
What MLA caches per position, per layer
The released code caches expanded heads, the latent is 53x smaller
Each layer attends over the outputs of every preceding block
Blocks of twelve, so the residual stack never exceeds nine sources
Every twelve layers the running prefix is snapshotted and cleared
One layer: aggregate, attend, aggregate again, then route
The bias steers selection only, the weights come from unbiased scores
Experts run in a narrow latent, and the norm is on the aggregate
Route, run 16 experts in a 3584-wide latent, then project back up
SiTU-GLU with beta1 = 4 and beta2 = 25, so the product is bounded
The hottest experts, out of 10,010 distinct ones touched

相关项目

sammycage
sammycage/plutofilter
PlutoFilter是一个单头文件、零内存分配的图像滤镜库，快速应用各种图像效果，兼容SVG和CSS语义。
194 C
tectonic-typesetting
tectonic-typesetting/tectonic
Tectonic是一个现代化、完整的自包含TeX/LaTeX引擎，基于XeTeX和TeXLive，旨在简化文档编译过程。
4.3k C
pgvector
pgvector/pgvector
pgvector是一个用于Postgres的开源向量相似性搜索工具，可实现精确和近似最近邻搜索。支持多种向量格式和距离计算，兼具Postgres的ACID特性。
17.4k C
FelixKratz
FelixKratz/SketchyBar
SketchyBar是一个高度可定制的macOS状态栏替代品，旨在提供快速、灵活和强大的状态栏体验，支持脚本和动态配置。
9.9k C
TEN-framework
TEN-framework/ten-vad
TEN VAD是一个低延迟、高性能且轻量级的语音活动检测器，适用于各种语音处理应用。
1.4k C
tree-sitter
tree-sitter/tree-sitter-html
tree-sitter-html是用于Tree-sitter的HTML语法解析库，主要帮助开发者解析和分析HTML语言。
179 C

评论

首页 - Wiki
Copyright © 2011-2026 iteam. Current version is 2.155.2. UTC+08:00, 2026-09-12 16:17
浙ICP备14020137号-1 $访客地图$