类库 › LakeSoul
lakesoul-io

lakesoul-io/LakeSoul

LakeSoul是一个云原生的Lakehouse框架,支持快速数据摄取和增量数据分析,适用于BI和AI应用,能够高效处理流和批量数据。

3,177 413 3,177 16
在 GitHub 上查看

技术栈

lakesoul-common java

查看全部依赖 (11)

依赖

com.alibaba:fastjson com.github.jnr:jnr-ffi com.google.protobuf:protobuf-java com.zaxxer:HikariCP org.apache.commons:commons-lang3 org.apache.hadoop:hadoop-client-api org.postgresql:postgresql org.scala-lang:scala-compiler org.scala-lang:scala-library org.scala-lang:scala-reflect org.slf4j:slf4j-api

lakesoul-flink java

查看全部依赖 (44)

依赖

com.dmetasoul:lakesoul-common com.dmetasoul:lakesoul-io-java io.openlineage:openlineage-flink junit:junit org.apache.doris:flink-doris-connector-1.20 org.apache.flink:flink-clients org.apache.flink:flink-connector-files org.apache.flink:flink-connector-jdbc org.apache.flink:flink-connector-kafka org.apache.flink:flink-connector-mongodb org.apache.flink:flink-core org.apache.flink:flink-csv org.apache.flink:flink-formats org.apache.flink:flink-hadoop-fs org.apache.flink:flink-json org.apache.flink:flink-parquet org.apache.flink:flink-runtime org.apache.flink:flink-runtime-web org.apache.flink:flink-s3-fs-hadoop org.apache.flink:flink-sql-connector-mongodb-cdc org.apache.flink:flink-sql-connector-mysql-cdc org.apache.flink:flink-sql-connector-oracle-cdc org.apache.flink:flink-sql-connector-postgres-cdc org.apache.flink:flink-sql-connector-sqlserver-cdc org.apache.flink:flink-streaming-java org.apache.flink:flink-table org.apache.flink:flink-table-api-java-bridge org.apache.flink:flink-table-api-scala_2.12 org.apache.flink:flink-table-common org.apache.flink:flink-table-planner_2.12 org.apache.flink:flink-test-utils org.apache.flink:flink-tests org.apache.hadoop:hadoop-client-api org.apache.hadoop:hadoop-client-runtime org.apache.parquet:parquet-hadoop-bundle org.apache.spark:spark-unsafe_${scala.binary.version} org.assertj:assertj-core org.furyio:fury-core org.hamcrest:hamcrest org.mongodb:bson org.scala-lang:scala-compiler org.scala-lang:scala-library org.scala-lang:scala-reflect org.yaml:snakeyaml

lakesoul-presto java

查看全部依赖 (20)

依赖

com.alibaba:fastjson com.dmetasoul:lakesoul-common com.dmetasoul:lakesoul-io-java com.facebook.airlift:log com.facebook.presto:presto-jdbc com.facebook.presto:presto-spi com.facebook.presto:presto-tests com.fasterxml.jackson.core:jackson-annotations com.fasterxml.jackson.core:jackson-core com.fasterxml.jackson.core:jackson-databind com.fasterxml.jackson.dataformat:jackson-dataformat-yaml com.fasterxml.jackson.datatype:jackson-datatype-guava com.fasterxml.jackson.datatype:jackson-datatype-jdk8 com.fasterxml.jackson.datatype:jackson-datatype-joda com.fasterxml.jackson.datatype:jackson-datatype-jsr310 com.google.guava:guava org.antlr:antlr4-runtime org.apache.hadoop:hadoop-client-api org.apache.hadoop:hadoop-client-runtime org.apache.parquet:parquet-column

lakesoul-spark-gluten java

查看全部依赖 (14)

依赖

com.dmetasoul:lakesoul-spark org.apache.gluten:backends-velox org.apache.gluten:gluten-arrow org.apache.gluten:gluten-core org.apache.gluten:gluten-substrait org.apache.gluten:spark-sql-columnar-shims-common org.apache.gluten:spark-sql-columnar-shims-spark33 org.apache.spark:spark-catalyst_${scala.binary.version} org.apache.spark:spark-core_${scala.binary.version} org.apache.spark:spark-hive_${scala.binary.version} org.apache.spark:spark-sql_${scala.binary.version} org.apache.spark:spark-sql_2.12 org.scalatest:scalatest_${scala.binary.version} org.scalatestplus:junit-4-13_2.12

lakesoul-spark java

查看全部依赖 (28)

依赖

com.dmetasoul:lakesoul-common com.dmetasoul:lakesoul-io-java io.confluent:common-config io.confluent:common-utils io.confluent:kafka-avro-serializer io.confluent:kafka-schema-registry-client io.confluent:kafka-schema-serializer io.jhdf:jhdf io.openlineage:openlineage-spark_2.12 mysql:mysql-connector-java org.apache.avro:avro org.apache.hadoop:hadoop-aws org.apache.hadoop:hadoop-client-api org.apache.hadoop:hadoop-client-runtime org.apache.kafka:kafka-clients org.apache.spark:spark-avro_2.12 org.apache.spark:spark-catalyst_${scala.binary.version} org.apache.spark:spark-core_${scala.binary.version} org.apache.spark:spark-hive_${scala.binary.version} org.apache.spark:spark-mllib_${scala.binary.version} org.apache.spark:spark-sql-kafka-0-10_2.12 org.apache.spark:spark-sql_${scala.binary.version} org.apache.spark:spark-streaming_${scala.binary.version} org.scala-lang:scala-compiler org.scala-lang:scala-library org.scala-lang:scala-reflect org.scalatest:scalatest_${scala.binary.version} org.scalatestplus:junit-4-13_2.12

native-io/lakesoul-io-java java

查看全部依赖 (13)

依赖

com.dmetasoul:lakesoul-common com.github.jnr:jnr-ffi com.google.protobuf:protobuf-java io.netty:netty-buffer io.substrait:core org.apache.arrow:arrow-c-data org.apache.arrow:arrow-memory-core org.apache.arrow:arrow-memory-netty org.apache.arrow:arrow-vector org.apache.spark:spark-catalyst_${scala.binary.version} org.scala-lang:scala-compiler org.scala-lang:scala-library org.scala-lang:scala-reflect

根目录 java

查看全部依赖 (2)

依赖

com.codahale.metrics:metrics-core junit:junit

python/examples python

查看全部依赖 (11)

依赖

Pandas accelerate datasets evaluate lakesoul pillow scikit-learn sentence-transformers torch tqdm transformers

rust/lakesoul-console rust

异步运行时

Tokio
查看全部依赖 (10)

依赖

clap datafusion futures lakesoul-datafusion rand rootcause rustyline 16.0.0 tracing tracing-appender 0.2 tracing-subscriber

rust/lakesoul-datafusion rust

异步运行时

Tokio
查看全部依赖 (37)

依赖

adbc_core 0.21.0 adbc_driver_manager 0.21.0 arrow arrow-arith arrow-cast arrow-flight arrow-ipc async-trait bytes chrono clap datafusion datafusion-common datafusion-expr futures json 0.12 lakesoul-io lakesoul-metadata log object_store parquet prost proto rand rootcause serde serde-aux 4.6.0 serde_json tonic tpchgen tpchgen-arrow tracing tracing-subscriber url uuid

开发依赖

ctor 0.4 test-log

rust/lakesoul-flight rust

异步运行时

Tokio
查看全部依赖 (17)

依赖

arrow arrow-flight async-trait chrono datafusion futures http lakesoul-datafusion lakesoul-io lakesoul-metadata object_store tonic tower

开发依赖

arrow-array arrow-cast arrow-schema assert_cmd 2.0

rust/lakesoul-io-c rust

异步运行时

Tokio
查看全部依赖 (12)

依赖

arrow-array arrow-ipc arrow-schema bytes datafusion-common datafusion-substrait lakesoul-io prost proto rootcause tracing tracing-subscriber

rust/lakesoul-io rust

异步运行时

Tokio
查看全部依赖 (29)

依赖

arrow arrow-array arrow-buffer arrow-cast arrow-schema async-stream 0.3.6 async-trait datafusion datafusion-catalog-listing datafusion-common datafusion-common-runtime datafusion-datasource datafusion-datasource-parquet datafusion-execution datafusion-expr datafusion-physical-expr datafusion-physical-plan datafusion-session datafusion-substrait futures object_store parquet prost proto serde serde_json tokio-stream tokio-util

开发依赖

tokio-test 0.4.4

rust/lakesoul-metadata-c rust

查看全部依赖 (6)

依赖

lakesoul-metadata prost proto serde_json tracing tracing-subscriber

rust/lakesoul-metadata rust

查看全部依赖 (1)

依赖

postgres 0.19.5

rust/lakesoul-s3-proxy rust

查看全部依赖 (7)

依赖

lakesoul-metadata

开发依赖

arrow arrow-array lakesoul-datafusion lakesoul-io proto serde_json

rust/proto rust

查看全部依赖 (2)

依赖

bytes prost

website javascript

框架

React ^18.2.0
查看全部依赖 (14)

依赖

@docusaurus/core ^3.1.0 @docusaurus/module-type-aliases ^3.1.0 @docusaurus/preset-classic ^3.1.0 @mdx-js/react ^3.0.0 @react-pdf-viewer/core ^3.12.0 @react-pdf-viewer/default-layout ^3.12.0 clsx ^1.2.1 pdfjs-dist ^3.11.174 prism-react-renderer ^2.1.0 react-dom ^18.2.0 react-github-btn ^1.4.0 unist-util-visit ^5.0.0

开发依赖

@babel/plugin-syntax-dynamic-import ^7.8.3 @docusaurus/types ^3.1.0

评论

Home - Wiki
Copyright © 2011-2026 iteam. Current version is 2.155.2. UTC+08:00, 2026-05-04 17:08
浙ICP备14020137号-1 $Map of visitor$