基础设施即代码初探-开发Terraform Provider管理私有云MySQL实例
' fill='%23FFFFFF'%3E%3Crect x='249' y='126' width='1' height='1'%3E%3C/rect%3E%3C/g%3E%3C/g%3E%3C/svg%3E)
Terraform和Provider简介
Terraform和Provider简介
Provider的主要功能
1.资源管理
定义可以创建、修改和删除的资源类型。
2.数据源查询
定义只读的数据源，用于从外部系统获取信息。
3.状态同步
Provider 通过 API 调用获取资源的当前状态，与 Terraform 的状态文件保持一致。
支持自定义资源：如果你的基础设施中包含了一些自定义资源或服务（例如内部开发的私有云平台、专有 API 或者公司特定的工具），而这些资源并未被官方提供的 Terraform Providers 支持，那么开发一个自定义 Provider 就可以将这些资源纳入基础设施即代码（IaC）的管理中。 服务开放：如果你的服务未来要面向外部客户开放，一个自定义 Provider 可以作为基础设施自动化的重要组成部分，方便客户通过 Terraform 集成你的服务。 扩展现有Provider：如果现有的Provider 并未满足你的需求（例如，缺少某些资源类型的支持，或者对资源的操作不够灵活），通过自定义 Provider 可以对其进行扩展。
开发自定义Provider
开发自定义Provider
1. 环境要求
2. Provider的资源定义
Provider 定义
provider 块用来配置与具体服务的交互方式。 常见的配置项包括认证信息、API 地址、默认区域等。
provider "aws" {region = "us-east-1"access_key = "your_access_key"secret_key = "your_secret_key"}
Resource 定义(以aws s3举例)
resource "aws_s3_bucket" "example_bucket" {bucket = "my-example-bucket"acl = "private"tags = {Name = "My bucket"Environment = "Dev"}}
data "aws_s3_bucket" "existing_bucket" {bucket = "existing-bucket-name"}
2. provider结构设计
type Provider interface {// Metadata should return the metadata for the provider, such as// a type name and version data.//// Implementing the MetadataResponse.TypeName will populate the// datasource.MetadataRequest.ProviderTypeName and// resource.MetadataRequest.ProviderTypeName fields automatically.Metadata(context.Context, MetadataRequest, *MetadataResponse)// Schema should return the schema for this provider.Schema(context.Context, SchemaRequest, *SchemaResponse)// Configure is called at the beginning of the provider lifecycle, when// Terraform sends to the provider the values the user specified in the// provider configuration block. These are supplied in the// ConfigureProviderRequest argument.// Values from provider configuration are often used to initialise an// API client, which should be stored on the struct implementing the// Provider interface.Configure(context.Context, ConfigureRequest, *ConfigureResponse)// DataSources returns a slice of functions to instantiate each DataSource// implementation.//// The data source type name is determined by the DataSource implementing// the Metadata method. All data sources must have unique names.DataSources(context.Context) []func() datasource.DataSource// Resources returns a slice of functions to instantiate each Resource// implementation.//// The resource type name is determined by the Resource implementing// the Metadata method. All resources must have unique names.Resources(context.Context) []func() resource.Resource}
Metadata 方法用于提供当前 Provider 的元数据信息，例如类型名称（TypeName）和版本等。这些信息可以用于识别 Provider，或者在需要与 Terraform 核心交互时使用。Schema 方法用于定义 Provider 的配置结构。例如，用户在 Terraform 中配置 Provider 的时候，可能需要指定 API 的凭证或目标地址。这些配置信息通过此方法定义。Configure 方法用于初始化 Provider 的运行环境。通常会解析用户配置的参数（例如凭证或其他必要的初始化信息），并生成一个客户端实例或其他相关的资源。DataSources 方法返回 Provider 支持的所有数据源类型。每个数据源用于从外部系统（如 API）中读取数据并将其提供给 Terraform。Resources 方法返回 Provider 支持的所有托管资源类型。每个资源代表 Terraform 可以管理的一个实体，例如云服务中的虚拟机、数据库实例等。// Ensure the implementation satisfies the expected interfaces.var (_ provider.Provider = &ZyunDbProvider{})// New is a helper function to simplify provider server and testing implementation.func New(version string) func() provider.Provider {return func() provider.Provider {return &ZyunDbProvider{version: version,}}}// ZyunDbProvider defines the provider implementation.type ZyunDbProvider struct {// version is set to the provider version on release, "dev" when the// provider is built and ran locally, and "test" when running acceptance// testing.version string}// Metadata returns the provider type name.func (p *ZyunDbProvider) Metadata(ctx context.Context, req provider.MetadataRequest, resp *provider.MetadataResponse) {resp.TypeName = "zyundb"resp.Version = p.version}// Schema defines the provider-level schema for configuration data.func (p *ZyunDbProvider) Schema(ctx context.Context, req provider.SchemaRequest, resp *provider.SchemaResponse) {resp.Schema = schema.Schema{Attributes: map[string]schema.Attribute{"endpoint": schema.StringAttribute{Description: "The endpoint of the ZyunDB API",Required: true,},"access_key_id": schema.StringAttribute{Description: "The access key id of the ZyunDB API",Required: true,},"access_key_secret": schema.StringAttribute{Description: "The access key secret of the ZyunDB API",Required: true,Sensitive: true,},},}}// Configure prepares a ZyunDB API client for data sources and resources.func (p *ZyunDbProvider) Configure(ctx context.Context, req provider.ConfigureRequest, resp *provider.ConfigureResponse) {// Retrieve provider data from configurationvar config zyundbProviderModeldiags := req.Config.Get(ctx, &config)resp.Diagnostics.Append(diags...)if resp.Diagnostics.HasError() {return}// If practitioner provided a configuration value for any of the// attributes, it must be a known value.if config.Endpoint.IsUnknown() {resp.Diagnostics.AddAttributeError(path.Root("endpoint"),"Unknown ZyunDB API Host","The provider cannot create the ZyunDB API client as there is an unknown configuration value for the ZyunDB API host. "+"Either target apply the source of the value first, set the value statically in the configuration, or use the ZYUNDB_HOST environment variable.",)}//...........................if resp.Diagnostics.HasError() {return}// Default values to environment variables, but override// with Terraform configuration value if set.endpoint := os.Getenv("ZYUNDB_ENDPOINT")if !config.Endpoint.IsNull() {endpoint = config.Endpoint.ValueString()}// If any of the expected configurations are missing, return// errors with provider-specific guidance.if endpoint == "" {resp.Diagnostics.AddAttributeError(path.Root("endpoint"),"Missing ZyunDB API Endpoint","The provider cannot create the ZyunDB API client as there is a missing or empty value for the ZyunDB API endpoint. "+"Set the endpoint value in the configuration or use the ZYUNDB_ENDPOINT environment variable. "+"If either is already set, ensure the value is not empty.",)}//................................if resp.Diagnostics.HasError() {return}// Create a new ZyunDB client using the configuration valuesclient := client.NewZyunOpenApiClient(endpoint, accessKeyId, accessKeySecret, "v1", "https")if client == nil {resp.Diagnostics.AddError("Unable to Create ZyunDB API Client","An unexpected error occurred when creating the ZyunDB API client. "+"If the error is not clear, please contact the provider developers.",)return}// Make the ZyunDB client available during DataSource and Resource// type Configure methods.resp.DataSourceData = clientresp.ResourceData = clienttflog.Info(ctx, "Configured ZyunDB client end")}// Resources defines the resources implemented in the provider.func (p *ZyunDbProvider) Resources(ctx context.Context) []func() resource.Resource {return []func() resource.Resource{NewMysqlInstanceResource,}}// DataSources defines the data sources implemented in the provider.func (p *ZyunDbProvider) DataSources(_ context.Context) []func() datasource.DataSource {return []func() datasource.DataSource{NewMysqlInstanceDataSource,}}
3. resource结构设计
type Resource interface {// Metadata should return the full name of the resource, such as// examplecloud_thing.Metadata(context.Context, MetadataRequest, *MetadataResponse)// Schema should return the schema for this resource.Schema(context.Context, SchemaRequest, *SchemaResponse)// Create is called when the provider must create a new resource. Config// and planned state values should be read from the// CreateRequest and new state values set on the CreateResponse.Create(context.Context, CreateRequest, *CreateResponse)// Read is called when the provider must read resource values in order// to update state. Planned state values should be read from the// ReadRequest and new state values set on the ReadResponse.Read(context.Context, ReadRequest, *ReadResponse)// Update is called to update the state of the resource. Config, planned// state, and prior state values should be read from the// UpdateRequest and new state values set on the UpdateResponse.Update(context.Context, UpdateRequest, *UpdateResponse)// Delete is called when the provider must delete the resource. Config// values may be read from the DeleteRequest.//// If execution completes without error, the framework will automatically// call DeleteResponse.State.RemoveResource(), so it can be omitted// from provider logic.Delete(context.Context, DeleteRequest, *DeleteResponse)}
Metadata 方法用于提供当前资源的元数据信息，标识资源的唯一名称（资源类型名）。Schema定义了资源的所有属性及其类型。type ResourceWithConfigure interface {Resource// Configure enables provider-level data or clients to be set in the// provider-defined Resource type. It is separately executed for each// ReadResource RPC.Configure(context.Context, ConfigureRequest, *ConfigureResponse)}
{"status" : 200,"developer-message" : "","more-info" : "","errno-code" : 0,"user-message" : "","data" : {"id" : "xxx","port" : "xxx","status" : "1","ctime" : "2024-09-23 19:42:39","utime" : "2024-09-23 19:52:13","pkg_id" : "xxx","db_type" : "master-slave","is_audit" : "1","name" : "test_name","instance_type" : "EXCLUSIVE","network_id" : "xxx","subnet_id" : "xxx","idc" : [ "xxidc" ],"rs_num" : [ {"cnt" : "2","idc" : "xxidc"} ],"master" : [ {"ip" : "1.1.1.1","idc" : "xxidc","type" : "master","idc_name" : "北京A区"} ],"slave" : [ {"ip" : "2.2.2.2","idc" : "xxidc","type" : "slave","idc_name" : "北京A区"} ],"vip_data" : [ {"port" : "xxx","idc" : "xxidc","vip" : "3.3.3.3","rw_status" : "6","idc_name" : "北京A区"} ],}}
type mysqlInstanceResourceModel struct {ID types.String `tfsdk:"id"`ProjectID types.String `tfsdk:"project_id"`Port types.String `tfsdk:"port"`Name types.String `tfsdk:"name"`PkgID types.String `tfsdk:"pkg_id"`InstanceType types.String `tfsdk:"instance_type"`Mode types.String `tfsdk:"mode"`MasterIDC types.String `tfsdk:"master_idc"`RedundantIDC types.String `tfsdk:"redundant_idc"`NetworkID types.String `tfsdk:"network_id"`SubnetID types.String `tfsdk:"subnet_id"`IsAuditLog types.String `tfsdk:"is_audit_log"`Status types.String `tfsdk:"status"`CTime types.String `tfsdk:"ctime"`VipData types.List `tfsdk:"vip_data"`MasterNum types.List `tfsdk:"master_num"`SlaveNum types.List `tfsdk:"slave_num"`Timeouts timeouts.Value `tfsdk:"timeouts"`}
// Ensure the implementation satisfies the expected interfaces.var (_ resource.Resource = &mysqlInstanceResource{}_ resource.ResourceWithConfigure = &mysqlInstanceResource{})// NewMysqlInstanceResource is a helper function to simplify the provider implementation.func NewMysqlInstanceResource() resource.Resource {return &mysqlInstanceResource{}}// mysqlInstanceResource is the resource implementation.type mysqlInstanceResource struct {client *client.ZyunOpenAPI}// mysqlInstanceResourceModel maps the resource schema data.type mysqlInstanceResourceModel struct {// .........}// Metadata returns the resource type name.func (r *mysqlInstanceResource) Metadata(_ context.Context, req resource.MetadataRequest, resp *resource.MetadataResponse) {resp.TypeName = req.ProviderTypeName + "_mysql_instance"}// Schema defines the schema for the resource.func (r *mysqlInstanceResource) Schema(ctx context.Context, _ resource.SchemaRequest, resp *resource.SchemaResponse) {resp.Schema = schema.Schema{Attributes: map[string]schema.Attribute{"timeouts": timeouts.Attributes(ctx, timeouts.Opts{Create: true,}),"id": schema.StringAttribute{Description: "The id of the mysql instance",Computed: true,PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown(),},},"port": schema.StringAttribute{Description: "The port of the mysql instance",Computed: true,PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown(),stringplanmodifier.RequiresReplace(),},},"network_id": schema.StringAttribute{Description: "The network id of the mysql instance",Required: true,},"vip_data": schema.ListNestedAttribute{Description: "The vip data of the mysql instance",Computed: true,NestedObject: schema.NestedAttributeObject{Attributes: map[string]schema.Attribute{"vip": schema.StringAttribute{Description: "The vip of the vip",Computed: true,},//.....................},},},//........................},}}// Configure adds the provider configured client to the resource.func (r *mysqlInstanceResource) Configure(_ context.Context, req resource.ConfigureRequest, resp *resource.ConfigureResponse) {// Add a nil check when handling ProviderData because Terraform// sets that data after it calls the ConfigureProvider RPC.if req.ProviderData == nil {return}client, ok := req.ProviderData.(*client.ZyunOpenAPI)if !ok {resp.Diagnostics.AddError("Unexpected Data Source Configure Type",fmt.Sprintf("Expected *client.ZyunOpenAPI, got: %T. Please report this issue to the provider developers.", req.ProviderData),)return}r.client = client}// Create creates the resource and sets the initial Terraform state.func (r *mysqlInstanceResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {var plan mysqlInstanceResourceModeldiags := req.Plan.Get(ctx, &plan)resp.Diagnostics.Append(diags...)if resp.Diagnostics.HasError() {return}// 获取超时上下文createTimeout, diags := plan.Timeouts.Create(ctx, 20*time.Minute)resp.Diagnostics.Append(diags...)if resp.Diagnostics.HasError() {return}ctx, cancel := context.WithTimeout(ctx, createTimeout)defer cancel()// Generate API request body from plan// Create new mysql instanceresult, err := r.client.CreateMySQLInstance(ctx, &client.CreateMySQLInstanceParams{ProjectID: plan.ProjectID.ValueString(),//................})if err != nil {resp.Diagnostics.AddError("Error creating mysql instance","Could not create mysql instance, unexpected error: "+err.Error(),)return}instanceID := result.Detail.InsIDplan.ID = types.StringValue(instanceID)// Poll to check the instance statusticker := time.NewTicker(10 * time.Second)defer ticker.Stop()CheckLoop:for {select {case <-ctx.Done():resp.Diagnostics.AddError("Timeout waiting for MySQL instance creation",fmt.Sprintf("Instance %s creation did not complete within the timeout period", instanceID),)returncase <-ticker.C:instance, err := r.client.GetMySQLInstance(ctx, plan.ProjectID.ValueString(), instanceID)if err != nil {continue}// Status 1 means readyif instance.Status == "1" {plan.xx = xx//..........................break CheckLoop} else if instance.Status == "2" { // 2 means failedresp.Diagnostics.AddError("Error creating mysql instance","MySQL instance creation failed",)return}}}diags = resp.State.Set(ctx, plan)resp.Diagnostics.Append(diags...)}// Read refreshes the Terraform state with the latest data.func (r *mysqlInstanceResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) {// Get current state//..............// Get refreshed order value from HashiCupsinstance, err := r.client.GetMySQLInstance(ctx, state.ProjectID.ValueString(), state.ID.ValueString())if err != nil {resp.Diagnostics.AddError("Error Reading mysql instance","Could not read mysql instance ID "+state.ID.ValueString()+": "+err.Error(),)return}//................................// Set refreshed statediags = resp.State.Set(ctx, &state)resp.Diagnostics.Append(diags...)}// Update updates the resource and sets the updated Terraform state on success.func (r *mysqlInstanceResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {// 处理更新值到params................err := r.client.UpdateMySQLInstance(ctx, params)if err != nil {//............return}resp.Diagnostics.AddWarning("Asynchronous Operation","The MySQL instance update is an asynchronous operation. Use 'terraform refresh' to get the latest state after the update completes.",)diags = resp.State.Set(ctx, plan)resp.Diagnostics.Append(diags...)if resp.Diagnostics.HasError() {return}}// Delete deletes the resource and removes the Terraform state on success.func (r *mysqlInstanceResource) Delete(ctx context.Context, req resource.DeleteRequest, resp *resource.DeleteResponse) {// Retrieve values from state// ..........// Delete existing mysql instanceerr := r.client.DeleteMySQLInstance(ctx, state.ProjectID.ValueString(), state.ID.ValueString())if err != nil {resp.Diagnostics.AddError("Error Deleting mysql instance","Could not delete mysql instance, unexpected error: "+err.Error(),)return}}
unknown），此修饰器会指示 Terraform 在计划阶段使用当前状态（state）中的值作为暂时的计划值。适用于在资源生命周期中，新值可能暂时不可用，但现有值可以作为替代。例如，某些属性的值需要依赖外部计算结果（如远程 API 的响应），但这结果在计划阶段尚未可知。// 获取超时上下文createTimeout, diags := plan.Timeouts.Create(ctx, 20*time.Minute)resp.Diagnostics.Append(diags...)if resp.Diagnostics.HasError() {return}ctx, cancel := context.WithTimeout(ctx, createTimeout)defer cancel()
resp.Diagnostics.AddWarning("Asynchronous Operation","The MySQL instance update is an asynchronous operation. Use 'terraform refresh' to get the latest state after the update completes.",)
5. data结构设计
type DataSource interface {// Metadata should return the full name of the data source, such as// examplecloud_thing.Metadata(context.Context, MetadataRequest, *MetadataResponse)// Schema should return the schema for this data source.Schema(context.Context, SchemaRequest, *SchemaResponse)// Read is called when the provider must read data source values in// order to update state. Config values should be read from the// ReadRequest and new state values set on the ReadResponse.Read(context.Context, ReadRequest, *ReadResponse)}
type DataSourceWithConfigure interface {DataSource// Configure enables provider-level data or clients to be set in the// provider-defined DataSource type. It is separately executed for each// ReadDataSource RPC.Configure(context.Context, ConfigureRequest, *ConfigureResponse)}
6. 入口文件
var (// these will be set by the goreleaser configuration// to appropriate values for the compiled binary.version string = "dev"// goreleaser can pass other information to the main package, such as the specific commit// https://goreleaser.com/cookbooks/using-main.version/)func main() {var debug boolflag.BoolVar(&debug, "debug", false, "set to true to run the provider with support for debuggers like delve")flag.Parse()opts := providerserver.ServeOpts{// TODO: Update this string with the published name of your provider.// Also update the tfplugindocs generate command to either remove the// -provider-name flag or set its value to the updated provider name.Address: "local/namespace/zyundb",Debug: debug,}err := providerserver.Serve(context.Background(), provider.New(version), opts)if err != nil {log.Fatal(err.Error())}}
7.实现单元测试
func TestAccMysqlInstanceDataSource(t *testing.T) {resource.Test(t, resource.TestCase{ProtoV6ProviderFactories: testAccProtoV6ProviderFactories,Steps: []resource.TestStep{// Read testing{Config: providerConfig + `data "zyundb_mysql_instance" "all" {project_id = "xxxx"}`,Check: resource.ComposeAggregateTestCheckFunc(// Verify number of coffees returnedresource.TestCheckResourceAttrSet("data.zyundb_mysql_instance.all", "mysql_instance.#"),// Verify the first coffee to ensure all attributes are setresource.TestCheckResourceAttrSet("data.zyundb_mysql_instance.all", "mysql_instance.0.id"),resource.TestCheckResourceAttrSet("data.zyundb_mysql_instance.all", "mysql_instance.0.name"),resource.TestCheckResourceAttrSet("data.zyundb_mysql_instance.all", "mysql_instance.0.port"),),},},})}
TF_LOG=ERROR TF_ACC=1 go test -count=1 -run='TestAccMysqlInstanceDataSource' -v=== RUN TestAccMysqlInstanceDataSource--- PASS: TestAccMysqlInstanceDataSource (2.66s)PASSok terraform-provider-zyundb/internal/provider 3.574s
本地使用自己的Provider
本地使用自己的Provider
provider_installation {dev_overrides {= "/Users/xxx/terraform-providers"}# For all other providers, install them directly from their origin provider# registries as normal. If you omit this, Terraform will _only_ use# the dev_overrides block, and so no other providers will be available.direct {}}
编译
GOOS=darwin GOARCH=amd64 go build -o terraform-provider-zyundb_v1.0.0配置并测试
# Copyright (c) HashiCorp, Inc.terraform {required_providers {zyundb = {source = "local/namespace/zyundb"version = "1.0.0"}}}provider "zyundb" {endpoint = "你的endpoint"access_key_id = "你的access key"access_key_secret = "你的access key secret"}data "zyundb_mysql_instance" "all" {project_id = "你的资源组ID"}output "mysql_instance" {value = data.zyundb_mysql_instance.all}resource "zyundb_mysql_instance" "example" {name = "example"project_id = "你的资源组ID"pkg_id = "套餐ID"instance_type = "NORMAL"mode = "master-slave"master_idc = "xxidc"redundant_idc = ""network_id = "xxx"subnet_id = "xxx"is_audit_log = truetimeouts = {create = "60m"}}
terraform init 是用来初始化 Terraform 配置的，它通常会下载所需的远程提供者并初始化状态。可是，当你使用本地开发的提供者时，terraform init 并不会像往常那样从远程注册表下载提供者，因为本地提供者已经通过 dev_overrides 配置指定。因此，Terraform 不需要再运行 terraform init 来获取远程提供者。terraform init，它可能会尝试下载远程提供者，并且在你本地提供者存在的情况下，可能会引发一些错误或冲突。生成文档
生成文档
//go:build generatepackage toolsimport (_ "github.com/hashicorp/copywrite"_ "github.com/hashicorp/terraform-plugin-docs/cmd/tfplugindocs")// Generate copyright headers//go:generate go run github.com/hashicorp/copywrite headers -d .. --config ../.copywrite.hcl// Format Terraform code for use in documentation.// If you do not have Terraform installed, you can remove the formatting command, but it is suggested// to ensure the documentation is formatted properly.//go:generate terraform fmt -recursive ../examples/// Generate documentation.//go:generate go run github.com/hashicorp/terraform-plugin-docs/cmd/tfplugindocs generate --provider-dir .. -provider-name zyundb
cd tools; go generate ./….├── README.md├── docs //这部分目录下都是自动生成的文档│ ├── data-sources│ │ └── mysql_instance.md│ ├── index.md│ └── resources│ └── mysql_instance.md├── examples //这个目录下是我们写的示例tf│ ├── README.md│ ├── data-sources│ │ └── zyundb│ │ └── data-source.tf│ ├── main.tf│ ├── provider│ │ └── provider.tf│ ├── resources│ │ └── zyundb│ │ ├── import.sh│ │ └── resource.tf│ └── terraform.tfstate //执行terraform apply后生成的状态文件├── go.mod├── go.sum├── internal│ ├── client //我们自己的处理API请求的代码，可以用SDK代替│ │ ├── client.go│ │ └── mysql.go│ └── provider //实现Provider的部分│ ├── mysql_instance_data_source.go│ ├── mysql_instance_data_source_test.go│ ├── mysql_instance_resource.go│ ├── mysql_instance_resource_test.go│ ├── provider.go│ └── provider_test.go├── main.go└── tools├── go.mod├── go.sum└── tools.go推荐阅读:
如何使用whisper+ollama+ffmpeg为视频添加中文字幕
更多技术和产品文章，请关注👆 如果您对哪个产品感兴趣，欢迎留言给我们，我们会定向邀文~
360智汇云是以"汇聚数据价值，助力智能未来"为目标的企业应用开放服务平台，融合360丰富的产品、技术力量，为客户提供平台服务。
目前，智汇云提供数据库、中间件、存储、大数据、人工智能、计算、网络、视联物联与通信等多种产品服务以及一站式解决方案，助力客户降本增效，累计服务业务1000+。
智汇云致力于为各行各业的业务及应用提供强有力的产品、技术服务，帮助企业和业务实现更大的商业价值。
官网：https://zyun.360.cn 或搜索“360智汇云”
客服电话：4000052360
欢迎使用我们的产品！😊
