类库
› DataflowTemplates
GoogleCloudPlatform/DataflowTemplates
Google Cloud Dataflow 提供的预置模板库,用于解决云上大数据处理任务(如数据导入/导出/备份/恢复、批量API操作),无需开发环境。基于 Apache Beam SDK,支持快速部署和定制。
标签
技术栈
ORM
SQLAlchemy
测试
pytest
网络
Requests
查看全部依赖 (461)
依赖
NumPy
Pandas
Pydantic
aiofiles
aiohappyeyeballs
aiohttp
aiosignal
annotated-types
anyio
apache-beam
asn1crypto
attrs
backports-tarfile
beartype
betterproto
cachetools
certifi
cffi
ch.qos.logback:logback-classic
ch.qos.logback:logback-core
charset-normalizer
cloud-sql-python-connector
com.amazonaws:aws-java-sdk-bom
com.amazonaws:aws-java-sdk-cloudwatch
com.amazonaws:aws-java-sdk-core
com.amazonaws:aws-java-sdk-kinesis
com.amazonaws:aws-java-sdk-kms
com.amazonaws:aws-java-sdk-s3
com.amazonaws:aws-java-sdk-sns
com.amazonaws:aws-java-sdk-sqs
com.amazonaws:aws-java-sdk-sts
com.azure:azure-messaging-eventhubs
com.codahale.metrics:metrics-core
com.datastax.astra:astra-sdk-devops
com.datastax.astra:beam-sdks-java-io-astra
com.datastax.cassandra:cassandra-driver-core
com.datastax.oss:java-driver-query-builder
com.fasterxml.jackson.dataformat:jackson-dataformat-cbor
com.fasterxml.jackson.module:jackson-module-jaxb-annotations
com.fasterxml.jackson:jackson-bom
com.github.ben-manes.caffeine:caffeine
com.github.fppt:jedis-mock
com.github.jsqlparser:jsqlparser
com.github.nosan:embedded-cassandra
com.github.spotbugs:spotbugs-annotations
com.github.stefanbirkner:system-rules
com.github.vincentrussell:json-data-generator
com.google.api-ads:google-ads
com.google.api-client:google-api-client
com.google.api.grpc:proto-google-cloud-datastream-v1
com.google.api.grpc:proto-google-cloud-firestore-admin-v1
com.google.api.grpc:proto-google-cloud-firestore-v1
com.google.api.grpc:proto-google-cloud-kms-v1
com.google.api:gax-grpc
com.google.apis:google-api-services-dataflow
com.google.apis:google-api-services-dataplex
com.google.apis:google-api-services-datastream
com.google.auto.service:auto-service
com.google.auto.service:auto-service-annotations
com.google.auto.value:auto-value
com.google.auto.value:auto-value-annotations
com.google.cloud.bigtable:bigtable-beam-import
com.google.cloud.bigtable:bigtable-hbase-beam
com.google.cloud.dataflow.cdc:cdc-common
com.google.cloud.dataflow.cdc:cdc-embedded-connector
com.google.cloud.hosted.kafka:managed-kafka-auth-login-handler
com.google.cloud.sql:mysql-socket-factory-connector-j-8
com.google.cloud.sql:postgres-socket-factory
com.google.cloud.teleport.metadata:metadata
com.google.cloud.teleport.v2:bigtable-common
com.google.cloud.teleport.v2:clickhouse-common
com.google.cloud.teleport.v2:common
com.google.cloud.teleport.v2:datastream-common
com.google.cloud.teleport.v2:datastream-to-spanner
com.google.cloud.teleport.v2:elasticsearch-common
com.google.cloud.teleport.v2:failure-injected-spanner-service
com.google.cloud.teleport.v2:failure-injection-policies
com.google.cloud.teleport.v2:jdbc-and-googlecloud
com.google.cloud.teleport.v2:jdbc-common
com.google.cloud.teleport.v2:kafka-common
com.google.cloud.teleport.v2:real-spanner-service
com.google.cloud.teleport.v2:spanner-common
com.google.cloud.teleport.v2:spanner-custom-shard
com.google.cloud.teleport.v2:spanner-migrations-sdk
com.google.cloud.teleport:core-plugin
com.google.cloud.teleport:it-cassandra
com.google.cloud.teleport:it-conditions
com.google.cloud.teleport:it-datadog
com.google.cloud.teleport:it-google-cloud-platform
com.google.cloud.teleport:it-iceberg
com.google.cloud.teleport:it-jdbc
com.google.cloud.teleport:it-splunk
com.google.cloud.teleport:plaintext-logging
com.google.cloud.teleport:structured-logging
com.google.cloud:google-cloud-aiplatform
com.google.cloud:google-cloud-bigquery
com.google.cloud:google-cloud-bigtable
com.google.cloud:google-cloud-compute
com.google.cloud:google-cloud-core
com.google.cloud:google-cloud-datacatalog
com.google.cloud:google-cloud-dataflow
com.google.cloud:google-cloud-datastore
com.google.cloud:google-cloud-datastream
com.google.cloud:google-cloud-dlp
com.google.cloud:google-cloud-firestore
com.google.cloud:google-cloud-firestore-admin
com.google.cloud:google-cloud-kms
com.google.cloud:google-cloud-logging
com.google.cloud:google-cloud-monitoring
com.google.cloud:google-cloud-nio
com.google.cloud:google-cloud-pubsub
com.google.cloud:google-cloud-secretmanager
com.google.cloud:google-cloud-spanner
com.google.cloud:google-cloud-storage
com.google.code.gson:gson
com.google.guava:guava
com.google.http-client:google-http-client
com.google.http-client:google-http-client-apache-v2
com.google.protobuf:protobuf-java
com.google.re2j:re2j
com.google.truth.extensions:truth-proto-extension
com.google.truth:truth
com.hivemq:hivemq-mqtt-client
com.microsoft.sqlserver:mssql-jdbc
com.oracle.database.jdbc:ojdbc8
com.puppycrawl.tools:checkstyle
com.squareup.okio:okio
com.zaxxer:HikariCP
commons-beanutils:commons-beanutils
commons-codec:commons-codec
commons-io:commons-io
cryptography
dev.failsafe:failsafe
distro
dnsjava:dnsjava
dnspython
docker
docstring-parser
envoy-data-plane
execnet
fastavro
fasteners
filelock
freezegun
frozenlist
fsspec
github.com/google/go-cmp
v0.7.0
github.com/hashicorp/go-version
v1.8.0
github.com/hashicorp/hc-install
v0.9.3
github.com/hashicorp/hcl/v2
v2.24.0
github.com/hashicorp/terraform-exec
v0.25.0
github.com/hashicorp/terraform-json
v0.27.2
github.com/spf13/cobra
v1.10.2
github.com/zclconf/go-cty
v1.18.0
google-api-core
google-apitools
google-auth
google-auth-httplib2
google-cloud-aiplatform
google-cloud-bigquery
google-cloud-bigquery-storage
google-cloud-bigtable
google-cloud-core
google-cloud-datastore
google-cloud-datastream
google-cloud-dlp
google-cloud-kms
google-cloud-language
google-cloud-monitoring
google-cloud-pubsub
google-cloud-pubsublite
google-cloud-recommendations-ai
google-cloud-resource-manager
google-cloud-secret-manager
google-cloud-spanner
google-cloud-storage
google-cloud-videointelligence
google-cloud-vision
google-crc32c
google-genai
google-resumable-media
googleapis-common-protos
greenlet
grpc-google-iam-v1
grpc-interceptor
grpcio
grpcio-status
grpclib
h11
h2
hf-xet
hpack
httpcore
httplib2
httpx
huggingface-hub
hyperframe
hypothesis
idna
importlib-metadata
iniconfig
io.airlift:aircompressor
io.debezium:debezium-connector-mysql
io.debezium:debezium-connector-postgres
io.debezium:debezium-connector-sqlserver
io.debezium:debezium-embedded
io.netty:netty-bom
io.opencensus:opencensus-api
jaraco-classes
jaraco-context
jaraco-functools
jeepney
jinja2
joblib
js2py
jsonpickle
jsonschema
jsonschema-specifications
junit:junit
keyring
keyrings-google-artifactregistry-auth
markupsafe
milvus-lite
mmh3
mock
more-itertools
mpmath
multidict
mysql:mysql-connector-java
networkx
nvidia-cublas-cu12
nvidia-cuda-cupti-cu12
nvidia-cuda-nvrtc-cu12
nvidia-cuda-runtime-cu12
nvidia-cudnn-cu12
nvidia-cufft-cu12
nvidia-curand-cu12
nvidia-cusolver-cu12
nvidia-cusparse-cu12
nvidia-nccl-cu12
nvidia-nvjitlink-cu12
nvidia-nvtx-cu12
oauth2client
objsize
opentelemetry-api
opentelemetry-resourcedetector-gcp
opentelemetry-sdk
opentelemetry-semantic-conventions
oracledb
org.apache.activemq:activemq-client
org.apache.avro:avro
org.apache.beam:beam-it-cassandra
org.apache.beam:beam-it-clickhouse
org.apache.beam:beam-it-common
org.apache.beam:beam-it-conditions
org.apache.beam:beam-it-elasticsearch
org.apache.beam:beam-it-jdbc
org.apache.beam:beam-it-kafka
org.apache.beam:beam-it-mongodb
org.apache.beam:beam-it-neo4j
org.apache.beam:beam-it-splunk
org.apache.beam:beam-it-testcontainers
org.apache.beam:beam-it-truthmatchers
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-sdks-java-bom
org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-extensions-avro
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-sdks-java-extensions-python
org.apache.beam:beam-sdks-java-extensions-sql
org.apache.beam:beam-sdks-java-google-cloud-platform-bom
org.apache.beam:beam-sdks-java-io-amazon-web-services2
org.apache.beam:beam-sdks-java-io-cassandra
org.apache.beam:beam-sdks-java-io-clickhouse
org.apache.beam:beam-sdks-java-io-common
org.apache.beam:beam-sdks-java-io-elasticsearch
org.apache.beam:beam-sdks-java-io-google-ads
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-io-hadoop-common
org.apache.beam:beam-sdks-java-io-hbase
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-io-jms
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-kudu
org.apache.beam:beam-sdks-java-io-mongodb
org.apache.beam:beam-sdks-java-io-mqtt
org.apache.beam:beam-sdks-java-io-neo4j
org.apache.beam:beam-sdks-java-io-parquet
org.apache.beam:beam-sdks-java-io-redis
org.apache.beam:beam-sdks-java-io-splunk
org.apache.beam:beam-sdks-java-io-synthetic
org.apache.beam:beam-sdks-java-managed
org.apache.beam:beam-sdks-java-test-utils
org.apache.cassandra:java-driver-core
org.apache.commons:commons-collections4
org.apache.commons:commons-compress
org.apache.commons:commons-configuration2
org.apache.commons:commons-csv
org.apache.commons:commons-dbcp2
org.apache.commons:commons-lang3
org.apache.commons:commons-math3
org.apache.commons:commons-pool2
org.apache.commons:commons-text
org.apache.derby:derby
org.apache.derby:derbyclient
org.apache.derby:derbynet
org.apache.hadoop:hadoop-client
org.apache.hadoop:hadoop-client-runtime
org.apache.hadoop:hadoop-common
org.apache.hadoop:hadoop-mapreduce-client-core
org.apache.hbase:hbase-common
org.apache.hbase:hbase-shaded-client
org.apache.hbase:hbase-shaded-testing-util
org.apache.iceberg:iceberg-api
org.apache.iceberg:iceberg-core
org.apache.iceberg:iceberg-data
org.apache.iceberg:iceberg-gcp
org.apache.iceberg:iceberg-parquet
org.apache.kafka:kafka-clients
org.apache.logging.log4j:log4j
org.apache.logging.log4j:log4j-api
org.apache.logging.log4j:log4j-core
org.apache.maven.plugin-testing:maven-plugin-testing-harness
org.apache.maven.plugin-tools:maven-plugin-annotations
org.apache.maven.surefire:surefire-junit47
org.apache.maven:maven-artifact
org.apache.maven:maven-compat
org.apache.maven:maven-core
org.apache.maven:maven-plugin-api
org.apache.parquet:parquet-avro
org.apache.parquet:parquet-hadoop
org.assertj:assertj-core
org.awaitility:awaitility
org.bouncycastle:bcpkix-jdk15on
org.checkerframework:checker-qual
org.codehaus.jettison:jettison
org.codehaus.mojo:extra-enforcer-rules
org.codehaus.plexus:plexus-archiver
org.conscrypt:conscrypt-openjdk-uber
org.eclipse.jetty.ee8:jetty-ee8-servlet
org.eclipse.jetty.ee8:jetty-ee8-webapp
org.eclipse.jetty.websocket:jetty-websocket-jetty-api
org.eclipse.jetty.websocket:jetty-websocket-jetty-client
org.eclipse.jetty.websocket:jetty-websocket-jetty-common
org.eclipse.jetty:jetty-alpn-client
org.eclipse.jetty:jetty-client
org.eclipse.jetty:jetty-http
org.eclipse.jetty:jetty-io
org.eclipse.jetty:jetty-security
org.eclipse.jetty:jetty-server
org.eclipse.jetty:jetty-util
org.eclipse.jetty:jetty-util-ajax
org.eclipse.jetty:jetty-xml
org.freemarker:freemarker
org.hamcrest:hamcrest
org.hamcrest:hamcrest-core
org.hamcrest:hamcrest-library
org.json:json
org.junit.jupiter:junit-jupiter-api
org.junit.jupiter:junit-jupiter-engine
org.mock-server:mockserver-client-java
org.mock-server:mockserver-junit-rule
org.mock-server:mockserver-netty
org.mockito:mockito-core
org.mockito:mockito-inline
org.mongodb:mongodb-driver-core
org.mongodb:mongodb-driver-sync
org.neo4j.driver:neo4j-java-driver
org.neo4j.importer:import-spec
org.neo4j:neo4j-cypher-dsl-schema-name-support
org.openjdk.nashorn:nashorn-core
org.postgresql:postgresql
org.reflections:reflections
org.scassandra:cql-antlr
org.scassandra:java-client
org.slf4j:slf4j-api
org.slf4j:slf4j-jdk14
org.slf4j:slf4j-nop
org.slf4j:slf4j-simple
org.springframework:spring-expression
org.syncope.identityconnectors:framework
org.syncope.identityconnectors:framework-internal
org.tensorflow:tensorflow-core-native
org.testcontainers:hivemq
org.testcontainers:mockserver
org.testcontainers:testcontainers
org.threeten:threetenbp
org.twdata.maven:mojo-executor
org.xerial.snappy:snappy-java
org.yaml:snakeyaml
orjson
overrides
packaging
parameterized
pg8000
pillow
pluggy
propcache
proto-plus
protobuf
psycopg2-binary
pyarrow
pyarrow-hotfix
pyasn1
pyasn1-modules
pycparser
pydantic-core
pygments
pyhamcrest
pyjsparser
pymilvus
pymongo
pymysql
pyparsing
pytest-timeout
pytest-xdist
python-dateutil
python-dotenv
python-tds
pytz
pyyaml
referencing
regex
requests-mock
rpds-py
rsa
safetensors
scikit-learn
scipy
scramp
secretstorage
setuptools
six
sniffio
sortedcontainers
sqlalchemy-pytds
sqlparse
sympy
tenacity
testcontainers
threadpoolctl
tokenizers
torch
torchvision
tqdm
transformers
triton
typing-extensions
typing-inspection
tzdata
tzlocal
ujson
urllib3
virtualenv-clone
websockets
wrapt
yarl
zipp
zstandard