Agent Skills › vemetric/vemetric › chdb-datastore

chdb-datastore

GitHub

提供基于ClickHouse引擎的Pandas替代方案,支持DataFrame加速、多源数据连接(S3/MySQL等)及跨源Join。适用于需要高性能数据分析的场景。

.agents/skills/chdb-datastore/SKILL.md vemetric/vemetric

触发场景

用户提到DataFrame、parquet、csv或'fast pandas' 用户导入chdb.datastore模块 用户请求加速Pandas操作

安装

npx skills add vemetric/vemetric --skill chdb-datastore -g -y
更多选项

非标准路径

npx skills add https://github.com/vemetric/vemetric/tree/main/.agents/skills/chdb-datastore -g -y

不安装直接使用

npx skills use vemetric/vemetric@chdb-datastore

指定 Agent (Claude Code)

npx skills add vemetric/vemetric --skill chdb-datastore -a claude-code -g -y

安装 repo 全部 skill

npx skills add vemetric/vemetric --all -g -y

预览 repo 内 skill

npx skills add vemetric/vemetric --list

SKILL.md

Frontmatter
{
    "name": "chdb-datastore",
    "license": "Apache-2.0",
    "metadata": {
        "author": "chdb-io",
        "version": "4.1",
        "homepage": "https:\/\/clickhouse.com\/docs\/chdb"
    },
    "description": "Use when the user has tabular data (pandas DataFrame, parquet, csv, Arrow, json) and wants to filter, group, aggregate, join, or speed up slow pandas. Provides chDB DataStore — same pandas API, ClickHouse engine underneath. Also handles reading from S3, MySQL, PostgreSQL, MongoDB, ClickHouse Cloud, Iceberg, Delta Lake as DataFrames and joining across sources. TRIGGER when: user mentions DataFrame, parquet, csv, \"fast pandas\", \"speed up pandas\", or cross-source DataFrame joins; user imports `chdb.datastore` or `from datastore import DataStore`. SKIP this skill for raw SQL syntax (use chdb-sql instead), ClickHouse server administration, or non-Python DataStore API work.",
    "compatibility": "Requires Python 3.9+, macOS or Linux. pip install chdb."
}

chdb DataStore — It's Just Faster Pandas

The Key Insight

# Change this:
import pandas as pd
# To this:
import chdb.datastore as pd
# Everything else stays the same.

DataStore is a lazy, ClickHouse-backed pandas replacement. Your existing pandas code works unchanged — but operations compile to optimized SQL and execute only when results are needed (e.g., print(), len(), iteration).

pip install chdb

Decision Tree: Pick the Right Approach

1. "I have a file/database and want to analyze it with pandas"
   → DataStore.from_file() / from_mysql() / from_s3() etc.
   → See references/connectors.md

2. "I need to join data from different sources"
   → Create DataStores from each source, use .join()
   → See examples/examples.md #3-5

3. "My pandas code is too slow"
   → import chdb.datastore as pd — change one line, keep the rest

4. "I need raw SQL queries"
   → Use the chdb-sql skill instead

Connect to Any Data Source — One Pattern

from datastore import DataStore

# Local file (auto-detects .parquet, .csv, .json, .arrow, .orc, .avro, .tsv, .xml)
ds = DataStore.from_file("sales.parquet")

# Database
ds = DataStore.from_mysql(host="db:3306", database="shop", table="orders", user="root", password="pass")

# Cloud storage
ds = DataStore.from_s3("s3://bucket/data.parquet", nosign=True)

# URI shorthand — auto-detects source type
ds = DataStore.uri("mysql://root:pass@db:3306/shop/orders")

All 16+ sources and URI schemes → connectors.md

After Connecting — Full Pandas API

result = ds[ds["age"] > 25]                                          # filter
result = ds[["name", "city"]]                                        # select columns
result = ds.sort_values("revenue", ascending=False)                  # sort
result = ds.groupby("dept")["salary"].mean()                         # groupby
result = ds.assign(margin=lambda x: x["profit"] / x["revenue"])     # computed column
ds["name"].str.upper()                                               # string accessor
ds["date"].dt.year                                                   # datetime accessor
result = ds1.join(ds2, on="id")                                      # join
result = ds.head(10)                                                 # preview
print(ds.to_sql())                                                   # see generated SQL

209 DataFrame methods supported. Full API → api-reference.md

Cross-Source Join — The Killer Feature

from datastore import DataStore

customers = DataStore.from_mysql(host="db:3306", database="crm", table="customers", user="root", password="pass")
orders = DataStore.from_file("orders.parquet")

result = (orders
    .join(customers, left_on="customer_id", right_on="id")
    .groupby("country")
    .agg({"amount": "sum", "rating": "mean"})
    .sort_values("sum", ascending=False))
print(result)

More join examples → examples.md

Writing Data

source = DataStore.from_mysql(host="db:3306", database="shop", table="orders", user="root", password="pass")
target = DataStore("file", path="summary.parquet", format="Parquet")

target.insert_into("category", "total", "count").select_from(
    source.groupby("category").select("category", "sum(amount) AS total", "count() AS count")
).execute()

Troubleshooting

Problem Fix
ImportError: No module named 'chdb' pip install chdb
ImportError: cannot import 'DataStore' Use from datastore import DataStore or from chdb.datastore import DataStore
Database connection timeout Include port in host: host="db:3306" not host="db"
Join returns empty result Check key types match (both int or both string); use .to_sql() to inspect
Unexpected results Call ds.to_sql() to see the generated SQL and debug
Environment check Run python scripts/verify_install.py (from skill directory)

References

Note: This skill teaches how to use chdb DataStore. For raw SQL queries, use the chdb-sql skill. For contributing to chdb source code, see CLAUDE.md in the project root.

版本历史

  • 4e8800b 当前 2026-09-28 16:05

同 Skill 集合

.agents/skills/clickhouse-architecture-advisor/SKILL.md
.agents/skills/clickhouse-best-practices/SKILL.md
.agents/skills/clickhouse-js-node-coding/SKILL.md
.agents/skills/infra-clickhouse/SKILL.md
.agents/skills/redis-clustering/SKILL.md
.agents/skills/redis-connections/SKILL.md
.agents/skills/redis-core/SKILL.md
.agents/skills/redis-observability/SKILL.md
.agents/skills/redis-security/SKILL.md
.agents/skills/chdb-sql/SKILL.md
.agents/skills/clickhouse-js-node-troubleshooting/SKILL.md
.agents/skills/redis-search/SKILL.md

元信息

文件数
0
版本
2c917f7
Hash
93e77cfb
收录时间
2026-09-28 16:05

首页 - Wiki
Copyright © 2011-2026 iteam. Current version is 2.156.0. UTC+08:00, 2026-11-21 16:28
浙ICP备14020137号-1