홈 항목 library logo 글 코드베이스 상점
자세히
类库 › Daft
Eventual-Inc

Eventual-Inc/Daft

Daft是一个高性能AI与多模态数据处理引擎，支持图像、音频、视频及结构化数据。基于Python和Rust构建，提供内置AI操作如LLM推理和嵌入生成，兼容S3、Iceberg等存储，并可扩展至Ray或Kubernetes分布式集群。

5,532 478 5,532 315
在 GitHub 上查看

标签

ai-engineering ai-pipeline arrow artificial-intelligence big-data data-engineering distributed distributed-computing distributed-systems embeddings etl huggingface iceberg machine-learning multimodal parquet python ray rust

技术栈

根目录 rust

查看全部依赖 (59)

依赖

aws-lc-rs =1.15.2 aws-lc-sys =0.35.0 common-daft-config common-display common-file-formats common-hashable-float-wrapper common-logging common-metrics common-partitioning common-resource-request common-runtime common-system-info common-tracing common-version daft-ai daft-algebra daft-catalog daft-checkpoint daft-cli daft-compression daft-context daft-core daft-csv daft-dashboard daft-distributed daft-dsl daft-file daft-functions daft-functions-binary daft-functions-json daft-functions-list daft-functions-serde daft-functions-temporal daft-functions-tokenize daft-functions-uri daft-functions-utf8 daft-geo daft-hash daft-image daft-io daft-json daft-local-execution daft-local-plan daft-logical-plan daft-micropartition daft-minhash daft-parquet daft-partition-refs daft-recordbatch daft-runners daft-scan daft-session daft-shuffles daft-sql daft-stats daft-text daft-warc daft-writers log

src/common/checkpoint-config rust

查看全部依赖 (5)

依赖

common-hashable-float-wrapper common-io-config pyo3 serde uuid

src/common/daft-config rust

查看全部依赖 (4)

依赖

common-io-config common-py-serde pyo3 serde

src/common/display rust

查看全部依赖 (6)

依赖

comfy-table indexmap pyo3 serde_json terminal_size 0.4.3 textwrap 0.16.1

src/common/error rust

异步运行时

Tokio
查看全部依赖 (5)

依赖

arrow-schema pyo3 regex serde_json thiserror

src/common/file-formats rust

查看全部依赖 (4)

依赖

common-error common-py-serde pyo3 serde

src/common/hashable-float-wrapper rust

查看全部依赖 (2)

依赖

half serde

src/common/image rust

查看全部依赖 (3)

依赖

common-error common-ndarray daft-schema

src/common/io-config rust

查看全部依赖 (1)

依赖

aws-credential-types 1.2.7

src/common/logging rust

查看全部依赖 (2)

依赖

arc-swap log

src/common/macros rust

查看全部依赖 (5)

依赖

proc-macro-crate 3.4 proc-macro-error 1.0 proc-macro2 1.0 quote 1.0 syn 2.0

src/common/metrics rust

查看全部依赖 (4)

依赖

bincode common-py-serde indicatif serde

src/common/ndarray rust

查看全部依赖 (5)

依赖

NumPy common-py-serde half ndarray pyo3

src/common/partitioning rust

查看全部依赖 (2)

依赖

common-py-serde pyo3

src/common/pattern rust

查看全部依赖 (2)

依赖

regex regex-syntax 0.8

src/common/py-serde rust

查看全部依赖 (3)

依赖

bincode pyo3 serde

src/common/resource-request rust

查看全部依赖 (5)

依赖

common-error common-hashable-float-wrapper common-py-serde pyo3 serde

src/common/runtime rust

异步运行时

Tokio
查看全部依赖 (7)

依赖

common-error futures log oneshot 0.1.8 pyo3 pyo3-async-runtimes

开发依赖

rand

src/common/system-info rust

查看全部依赖 (2)

依赖

pyo3 sysinfo

src/common/tracing rust

查看全部依赖 (5)

依赖

common-runtime opentelemetry opentelemetry-otlp opentelemetry_sdk tracing

src/common/treenode rust

查看全部依赖 (1)

依赖

common-error

src/daft-ai rust

查看全部依赖 (1)

依赖

pyo3

src/daft-algebra rust

查看全部依赖 (8)

依赖

common-error common-treenode daft-core daft-dsl daft-schema half indexmap

开发依赖

rstest

src/daft-catalog rust

查看全部依赖 (12)

依赖

common-error common-pattern daft-context daft-core daft-dsl daft-logical-plan daft-micropartition daft-runners indexmap pyo3 regex sqlparser

src/daft-checkpoint rust

查看全部依赖 (18)

依赖

arrow-array arrow-schema async-trait bytes chrono common-checkpoint-config common-error common-io-config common-runtime daft-core daft-scan daft-schema futures parquet pyo3 serde serde_json snafu

src/daft-compression rust

异步运行时

Tokio
查看全部依赖 (2)

依赖

async-compression url

src/daft-context rust

异步运行时

Tokio
查看全部依赖 (15)

依赖

common-daft-config common-error common-metrics common-runtime daft-core daft-dashboard daft-logical-plan daft-micropartition daft-runners dashmap log pyo3 reqwest serde serde_json

src/daft-core rust

查看全部依赖 (4)

依赖

arrow arrow-row bincode

开发依赖

rstest

src/daft-csv rust

异步运行时

Tokio
查看全部依赖 (25)

依赖

arrow-schema async-compat async-stream common-error common-py-serde common-runtime csv csv-async 1.3.0 daft-compression daft-core daft-decoding daft-dsl daft-io daft-recordbatch daft-schema futures memchr 2.7.6 parking_lot pyo3 rayon serde smallvec snafu tokio-util

开发依赖

rstest

src/daft-dashboard rust

框架

Axum 0.8
查看全部依赖 (4)

依赖

async-stream common-metrics daft-recordbatch tower 0.5

src/daft-dashboard/frontend javascript

框架

Next.js 16.1.6 React ^19.1.1

状态管理

Zustand ^5.0.8

CSS 框架

Tailwind CSS ^4.1.14

代码规范

ESLint ^9.36.0 Prettier ^3.3.3
查看全部依赖 (31)

依赖

@radix-ui/react-collapsible ^1.1.12 @radix-ui/react-dialog ^1.1.15 @radix-ui/react-dropdown-menu ^2.1.16 @radix-ui/react-label ^2.1.7 @radix-ui/react-separator ^1.1.7 @radix-ui/react-slot ^1.2.3 @radix-ui/react-tabs ^1.1.13 @radix-ui/react-tooltip ^1.2.8 @tanstack/react-table ^8.21.3 class-variance-authority ^0.7.1 clsx ^2.1.1 framer-motion ^12.23.18 geist ^1.7.0 lucide-react ^0.548.0 motion ^12.23.18 react-dom ^19.1.1 react-icons ^5.5.0 react-resizable-panels ^3.0.6 swr ^2.3.6 tailwind-merge ^3.3.1 tailwindcss-animate ^1.0.7

开发依赖

@eslint/eslintrc ^3.3.1 @tailwindcss/postcss ^4.1.18 @types/node 25.2.3 @types/react ^19.1.13 @types/react-dom ^19.1.9 babel-plugin-react-compiler ^1.0.0 eslint-config-next 16.1.6 eslint-config-prettier ^10.1.8 postcss ^8.5.6 typescript ^5.7.3

src/daft-decoding rust

查看全部依赖 (9)

依赖

arrow-array arrow-schema atoi_simd 0.16.1 chrono csv csv-async 1.2.6 daft-schema fast-float2 0.2.3 simdutf8

src/daft-distributed rust

异步运行时

Tokio
查看全部依赖 (37)

依赖

common-checkpoint-config common-daft-config common-display common-error common-file-formats common-io-config common-metrics common-partitioning common-py-serde common-resource-request common-runtime common-treenode daft-context daft-dsl daft-functions daft-io daft-local-plan daft-logical-plan daft-micropartition daft-partition-refs daft-recordbatch daft-scan daft-schema futures itertools opentelemetry pyo3 pyo3-async-runtimes rand serde serde_json tokio-stream tokio-util tracing

开发依赖

arrow daft-core daft-local-execution

src/daft-dsl rust

异步运行时

Tokio
查看全部依赖 (22)

依赖

async-trait common-error common-hashable-float-wrapper common-macros common-metrics common-py-serde common-resource-request common-runtime common-tracing common-treenode daft-core daft-sketch derive_more indexmap itertools log opentelemetry pyo3 rand serde typetag

开发依赖

rstest

src/daft-ext-macros rust

查看全部依赖 (2)

依赖

proc-macro2 1.0 quote 1.0

src/daft-ext rust

查看全部依赖 (1)

依赖

daft-ext-macros 0.1.1

src/daft-file rust

查看全部依赖 (7)

依赖

common-error daft-core daft-dsl daft-io daft-schema mime_guess 2.0.5 pyo3

src/daft-functions-binary rust

查看全部依赖 (6)

依赖

arrow base64 common-error common-macros daft-core daft-dsl

src/daft-functions-json rust

查看全部依赖 (5)

依赖

arrow common-error daft-core daft-dsl jaq-core

src/daft-functions-list rust

查看全部依赖 (4)

依赖

arrow common-error daft-core daft-dsl

src/daft-functions-serde rust

查看全部依赖 (8)

依赖

arrow arrow-json common-error daft-core daft-dsl serde serde_json typetag

src/daft-functions-temporal rust

查看全部依赖 (9)

依赖

arrow-array chrono chrono-tz common-error daft-core daft-dsl paste 1.0.15 serde typetag

src/daft-functions-tokenize rust

查看全部依赖 (12)

依赖

arrow base64 common-error common-runtime daft-core daft-dsl daft-io rustc-hash 1.1 serde snafu tiktoken-rs typetag

src/daft-functions-uri rust

异步运行时

Tokio
查看全部依赖 (13)

依赖

arrow async-trait bytes common-error common-runtime daft-core daft-dsl daft-io futures serde typetag url uuid

src/daft-functions-utf8 rust

查看全部依赖 (15)

依赖

aho-corasick 1.1.3 arrow arrow-buffer chrono chrono-tz common-error daft-core daft-dsl heck 0.5.0 itertools num-traits regex serde typetag unicode-normalization 0.1.25

src/daft-functions rust

查看全部依赖 (14)

依赖

arrow-array arrow-buffer common-error daft-core daft-dsl daft-hash num-traits pyo3 rand serde snafu typetag uuid xxhash-rust

src/daft-geo rust

查看全部依赖 (5)

依赖

arrow-buffer daft-core daft-dsl serde typetag

src/daft-groupby rust

查看全部依赖 (6)

依赖

arrow common-error daft-core half hashbrown smallvec

src/daft-hash rust

查看全部依赖 (2)

依赖

common-error mur3

src/daft-image rust

查看全部依赖 (13)

依赖

arrow base64 common-error common-image daft-core daft-dsl daft-file daft-schema image log serde typetag

开发依赖

criterion

src/daft-io rust

查看全部依赖 (6)

依赖

async-recursion 1.0.4 async-stream async-trait aws-config 1.8.12

开发依赖

md5 0.8.0 tempfile 3.23.0

src/daft-json rust

查看全部依赖 (21)

依赖

arrow chrono common-error common-py-serde common-runtime daft-compression daft-core daft-decoding daft-dsl daft-io daft-recordbatch daft-schema futures indexmap memchr 2.7.6 memmap2 0.9.9 num-traits pyo3 rayon serde

开发依赖

rstest

src/daft-local-execution rust

查看全部依赖 (52)

依赖

arc-swap arrow-array arrow-buffer arrow-ipc arrow-schema async-trait common-checkpoint-config common-daft-config common-display common-error common-file-formats common-io-config common-logging common-metrics common-py-serde common-resource-request common-runtime common-system-info common-tracing console daft-checkpoint daft-context daft-core daft-csv daft-dsl daft-functions-list daft-functions-uri daft-groupby daft-io daft-json daft-local-plan daft-logical-plan daft-micropartition daft-parquet daft-partition-refs daft-recordbatch daft-scan daft-shuffles daft-text daft-warc daft-writers futures indexmap indicatif itertools log pin-project 1 pyo3 pyo3-async-runtimes rand smallvec snafu

src/daft-local-plan rust

查看全部依赖 (21)

依赖

arrow-array arrow-schema bincode common-checkpoint-config common-error common-io-config common-metrics common-py-serde common-resource-request common-treenode daft-core daft-dsl daft-logical-plan daft-micropartition daft-partition-refs daft-recordbatch daft-scan log pyo3 serde serde_json

src/daft-logical-plan rust

查看全部依赖 (12)

依赖

common-checkpoint-config common-daft-config common-display common-error common-file-formats common-hashable-float-wrapper common-io-config common-partitioning common-py-serde common-resource-request common-treenode daft-algebra

src/daft-micropartition rust

异步运行时

Tokio
查看全部依赖 (23)

依赖

arrow arrow-array arrow-ipc bincode common-arrow-ffi common-error common-partitioning common-runtime daft-core daft-csv daft-dsl daft-io daft-json daft-parquet daft-recordbatch daft-scan daft-stats daft-warc dashmap futures pyo3 snafu tracing

src/daft-minhash rust

查看全部依赖 (3)

依赖

arrow fastrand 2.1.0 memchr 2.7.4

src/daft-parquet rust

查看全部依赖 (16)

依赖

arrow bytes common-arrow-ffi common-error common-runtime daft-core daft-dsl daft-io daft-recordbatch daft-stats futures indexmap

开发依赖

bincode fastrand 2.1.0 parquet path_macro

src/daft-partition-refs rust

查看全部依赖 (4)

依赖

common-partitioning common-py-serde pyo3 serde

src/daft-recordbatch rust

查看全部依赖 (29)

依赖

arrow arrow-array arrow-ipc async-recursion 1.0.4 comfy-table common-arrow-ffi common-display common-error common-metrics common-py-serde common-runtime daft-core daft-dsl daft-functions-list daft-groupby daft-image fnv 1.0.7 futures hashbrown html-escape indexmap num-traits pyo3 rand rayon serde serde_json smallvec

开发依赖

typetag

src/daft-runners rust

查看全部依赖 (5)

依赖

common-error daft-logical-plan daft-micropartition log pyo3

src/daft-scan rust

异步运行时

Tokio
查看全部依赖 (33)

依赖

arrow-schema async-trait common-daft-config common-display common-error common-file-formats common-io-config common-py-serde common-runtime daft-algebra daft-compression daft-core daft-csv daft-decoding daft-dsl daft-io daft-json daft-parquet daft-recordbatch daft-schema daft-stats either 1.9 fnv 1.0.7 futures indexmap itertools pyo3 serde serde_json snafu tokio-stream url urlencoding 2.1.3

src/daft-schema rust

查看全部依赖 (15)

依赖

arrow arrow-schema common-arrow-ffi common-display common-error common-py-serde common-version derive_more educe html-escape indexmap num-derive num-traits pyo3

开发依赖

rstest

src/daft-session rust

查看全部依赖 (12)

依赖

common-error daft-ai daft-catalog daft-core daft-dsl daft-ext daft-ext-internal daft-logical-plan pyo3 uuid

开发依赖

serde typetag

src/daft-shuffles rust

查看全部依赖 (15)

依赖

arrow-array arrow-flight arrow-ipc arrow-schema async-channel 2.5.0 async-stream common-error common-runtime daft-core daft-io daft-micropartition daft-recordbatch daft-schema daft-writers futures

src/daft-sketch rust

查看全部依赖 (3)

依赖

arrow-array arrow-schema common-error

src/daft-sql rust

查看全部依赖 (21)

依赖

common-daft-config common-error common-hashable-float-wrapper common-io-config common-runtime daft-catalog daft-context daft-core daft-dsl daft-functions daft-functions-list daft-functions-temporal daft-functions-utf8 daft-logical-plan daft-micropartition daft-recordbatch daft-session itertools pyo3 sqlparser

开发依赖

rstest

src/daft-stats rust

查看全部依赖 (6)

依赖

common-error daft-core daft-dsl daft-recordbatch serde snafu

src/daft-text rust

异步运行时

Tokio
查看全部依赖 (8)

依赖

async-stream common-error daft-compression daft-core daft-io daft-recordbatch futures serde

src/daft-warc rust

查看全部依赖 (9)

依赖

arrow-array chrono common-error common-runtime daft-compression daft-core daft-dsl daft-io daft-recordbatch

src/daft-writers rust

异步运行时

Tokio
查看全部依赖 (25)

依赖

arrow-array arrow-csv arrow-ipc arrow-json arrow-schema async-trait chrono chrono-tz common-daft-config common-error common-file-formats common-runtime daft-core daft-dsl daft-io daft-logical-plan daft-micropartition daft-recordbatch hashbrown parking_lot parquet pyo3 url urlencoding 2.1.3

开发依赖

bytes

相关项目

phiresky
phiresky/sqlite-zstd
sqlite-zstd是一个为SQLite提供透明字典级行压缩的扩展，能够显著减少数据库大小，同时保持随机访问性能。
1.6k Rust
szabodanika
szabodanika/microbin
MicroBin是一个安全、可配置的文件共享和URL缩短的Web应用，便于用户自我托管，支持快速设置和使用。
3.9k Rust
tejas-raskar
tejas-raskar/noted.md
noted.md是一个命令行工具，可将手写笔记转换为干净易读的Markdown文件，支持多种图片格式和批量处理。
366 Rust
juspay
juspay/hyperswitch
Hyperswitch是一个开源支付交换平台，使用Rust编写，旨在实现快速、可靠且经济的支付解决方案。
32.5k Rust
mongodb
mongodb/kingfisher
Kingfisher 是一个快速的秘密扫描和实时验证工具，结合了哈希引擎与语言感知解析，能在生产前检测和验证秘密。
542 Rust
DioxusLabs
DioxusLabs/blitz
Blitz是一个模块化的HTML/CSS渲染引擎，旨在提供高效的网页渲染支持，适合开发者进行实验和构建应用。
2.8k Rust

评论

홈 - 위키
Copyright © 2011-2026 iteam. Current version is 2.155.2. UTC+08:00, 2026-05-30 15:15
浙ICP备14020137号-1 $방문자$