• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号
登陆 注册
描述文字

jinghzhu/KubernetesCRD: Example of how to create and manage Kubernetes Custom Re ...

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称（OpenSource Name）：

jinghzhu/KubernetesCRD

开源软件地址(OpenSource Url)：

https://github.com/jinghzhu/KubernetesCRD

开源编程语言(OpenSource Language)：

Go 97.2%

开源软件介绍(OpenSource Introduction)：

Kubernetes Custom Resource Definition

This repository is an example of how to create/list/update/delete Kubernetes Custom Resource Definition. If you also want to learn how to develop your own Operator, please visit:

The CRD example I write here is like the ReplicaSet. You can use my CRD with my Operators mentioned before to run following scenarios:

  1. Register CRD.
  2. The Operator can automatically reconcile current state to desired state. By saying state, it means the number of running Pods.
  3. High availability guarantee, which means if you manually delete a Pod via kubectl, the Operator will automatically reconcile it.

Environment

  1. Go: >= v1.9.0
  2. Kubernetes: >= v1.18.0
  3. Assume you have already a Kubernetes cluster and its kubeconfig file can be reached via system variable CRD_KUBECONFIG and it will create a CRD instance in namespace crd by default, which you can also modify via system variable CRD_NAMESPACE. For more details, please check package pkg/config/

Dependency Package

The CRD is mainly developed in repository apiextensions-apiserver which depends on client-go and apimachinery.

This code is based on Kubernetes v1.18.12:

  • k8s.io/client-go with version v0.18.12.
  • k8s.io/apimachinery with version v0.18.12.
  • k8s.io/apiextensions-apiserver with version v0.18.12.

Step-by-step Instructions

Define CRD Object

We need firstly create the struct of CRD. The CRD object structure has these components:

  • Metadata

    Standard Kubernetes properties like name, namespace, labels, etc.

  • Spec

    CR configuration.

    Each instance of our CR has an attached Spec, which should be defined via a struct{} to provide data format validation. In practice, this Spec is arbitrary key-value data that specifies the configuration/behavior of the CR.

  • Status

    Used by the CR controller in response to Spec updates.

// pkg/crd/jinghzhu/v1/types.go

// Jinghzhu is the CRD. Use this command to generate deepcopy for it:
type Jinghzhu struct {
	metav1.TypeMeta `json:",inline"`
	// Standard object's metadata.
	metav1.ObjectMeta `json:"metadata"`
	// Specification of the desired behavior of Jinghzhu.
	Spec JinghzhuSpec `json:"spec"`
	// Observed status of Jinghzhu.
	Status JinghzhuStatus `json:"status"`
}

// JinghzhuSpec is a desired state description of Jinghzhu.
type JinghzhuSpec struct {
	// Desired is the desired Pod number.
	Desired int `json:"desired"`
	// Current is the number of Pod currently running.
	Current int `json:"current"`
	// PodList is the name list of current Pods.
	PodList []string `json:"podList"`
}

// JinghzhuStatus describes the lifecycle status of Jinghzhu.
type JinghzhuStatus struct {
	State   string `json:"state"`
	Message string `json:"message"`
}

We need to leverage the automatically code generated script to create the deep copy methods for CRD object. The result can be found at:

  • pkg/crd/jinghzhu/v1/zz_generated.deepcopy.go
  • pkg/crd/jinghzhu/v1/apis

You can get code-generator from GitHub.

In my example, I run following command to generate that file:

$ ./k8s.io/code-generator/generate-groups.sh all github.com/jinghzhu/KubernetesCRD/pkg/crd/jinghzhu/v1/apis github.com/jinghzhu/KubernetesCRD/pkg/crd "jinghzhu:v1"

Please note that the code-generator requires annotation to work as expected. If you carefully read my code, you can find the annotations at:

  • pkg/crd/jinghzhu/v1/types.go:
// +genclient
// +genclient:noStatus
// +k8s:deepcopy-gen=true
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// +resource:path=jinghzhu

...

// +k8s:deepcopy-gen=true

...

// +k8s:deepcopy-gen=true
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// +resource:path=jinghzhu
  • pkg/crd/jinghzhu/v1/doc.go:
// +k8s:deepcopy-gen=package,register
// +k8s:defaulter-gen=TypeMeta
// +k8s:openapi-gen=true

// Package v1 is the v1 version of the API.
// +groupName=jinghzhu.io

For more details about annotations for code-generator, please read Kubernetes documents or Google it.

If you use dep to manage dependency packages, you may not directly run the script, generate-groups. It will throw the error that it can't find necessary vendor packages.

The solution is to deploy the dependency packages in $GOPATH. For more details, please view the kubernetes/code-generator#21.

Register CRD

The CRD name (CRDName string = Plural + "." + GroupName) is the combination of CR plural (Plural string = "jinghzhus") and CR group(GroupName string = "jinghzhu.io") which can be used for the reference of Kubernetes CLI or API. CR group and version also define API endpoints.

pkg/crd/jinghzhu/v1/register.go:

const (
	// Kind is normally the CamelCased singular type. The resource manifest uses this.
	Kind string = "Jinghzhu"
	// GroupVersion is the version.
	GroupVersion string = "v1"
	// Plural is the plural name used in /apis/<group>/<version>/<plural>
	Plural string = "jinghzhus"
	// Singular is used as an alias on kubectl for display.
	Singular string = "jinghzhu"
	// CRDName is the CRD name for Jinghzhu.
	CRDName string = Plural + "." + crdjinghzhu.GroupName
	// ShortName is the short alias for the CRD.
	ShortName string = "jh"
)

The method, CreateCustomResourceDefinition at pkg/crd/jinghzhu/v1/crd.go, covers the logic to register CRD into Kubernetes. Please note that Kubernetes doesn't immediately register CRD. So, it is better to add logic to wait for the creation like shown in the code.

Meanwhile, I also leverage OpenAPI v3 to perform validation check. You can find related logic at CreateCustomResourceDefinition@pkg/crd/jinghzhu/v1:

      Validation: &apiextensionsv1beta1.CustomResourceValidation{
				OpenAPIV3Schema: &apiextensionsv1beta1.JSONSchemaProps{
					Type: "object",
					Properties: map[string]apiextensionsv1beta1.JSONSchemaProps{
						"spec": {
							Type: "object",
							Properties: map[string]apiextensionsv1beta1.JSONSchemaProps{
								"desired": {Type: "integer", Format: "int"},
								"current": {Type: "integer", Format: "int"},
								"podList": {
									Type: "array",
									Items: &apiextensionsv1beta1.JSONSchemaPropsOrArray{
										Schema: &apiextensionsv1beta1.JSONSchemaProps{Type: "string"},
									},
								},
							},
							Required: []string{"desired"},
						},
					},
				},
			},

CRD Client

After creating CRD, we can access via CLI. For easily usage, we hope it can also be accessed via API. So I develop some methods to wrapper some codes for CRD Create, Update, Delete, Get, and List. You can view them at pkg/crd/jinghzhu/v1/client/client.go.

Main Logic to Use CRD

Here, I'll go through the main code (main.go) to show the main logic of everything set before for us to use CRD.

  1. Connect to Kubernetes.

    kubeConfigPath := os.Getenv("KUBECONFIG")

  // Use kubeconfig to create client config.
  clientConfig, err := clientcmd.BuildConfigFromFlags("", kubeconfigPath)
  if err != nil {
	  panic(err)
}

apiextensionsClientSet, err := apiextensionsclient.NewForConfig(clientConfig)
  if err != nil {
	  panic(err)
  }

  2. Register the CRD. Please note that it can only be accessed by CLI now as mentioned before.

    // Init a CRD kind.
  if _, err = crdjinghzhuv1.CreateCustomResourceDefinition(apiextensionsClientSet); err != nil {
	  panic(err)
  }

  3. Create the API client to help access the CRD.

    // Create a CRD client interface for Jinghzhu v1.
  crdClient, err := jinghzhuv1client.NewClient(ctx, kubeconfigPath, cfg.GetCRDNamespace())
  if err != nil {
	  panic(err)
  }

  4. Declare a CR object for test.

    instanceName := "jinghzhu-example-"
  exampleInstance := &crdjinghzhuv1.Jinghzhu{
	  ObjectMeta: metav1.ObjectMeta{
		  GenerateName: instanceName,
	  },
	  Spec: crdjinghzhuv1.JinghzhuSpec{
		  Desired: 1,
		  Current: 0,
		  PodList: make([]string, 0),
	  },
	  Status: crdjinghzhuv1.JinghzhuStatus{
		  State:   types.StatePending,
		  Message: "Created but not processed yet",
	  },
  }

  5. Use the API client created in step 3 to create new CR.

    result, err := crdClient.CreateDefault(exampleInstance)
  if err != nil && apierrors.IsAlreadyExists(err) {
	  fmt.Printf("ALREADY EXISTS: %#v\n", result)
  } else if err != nil {
	  panic(err)
  }
  crdInstanceName := result.GetName()
  fmt.Println("CREATED: " + result.String())

  // Wait until the CRD object is handled by controller and its status is changed to Processed.
  err = crdClient.WaitForInstanceProcessed(crdInstanceName)
  if err != nil {
	  panic(err)
  }
  fmt.Println("Processed " + crdInstanceName)

  // Get the list of CRs.
  exampleList, err := crdClient.List(metav1.ListOptions{})
  if err != nil {
	  panic(err)
  }
  fmt.Printf("LIST: %#v\n", exampleList)

Result

If everything goes well, you should see logs like:

$ go run cmd/crd/main.go
CRD Jinghzhu is created
CREATED:        Name = jinghzhu-example-f7wgv
        Resource Version = 2343534
        Desired = 1
        Current = 0
        PodList =
        State = Pending
        Message = Created but not processed yet

Processed jinghzhu-example-f7wgv
LIST: &v1.JinghzhuList{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ListMeta:v1.ListMeta{SelfLink:"/apis/jinghzhu.io/v1/namespaces/crd/jinghzhus", ResourceVersion:"2343539", Continue:"", RemainingItemCount:(*int64)(nil)}, Items:[]v1.Jinghzhu{v1.Jinghzhu{TypeMeta:v1.TypeMeta{Kind:"Jinghzhu", APIVersion:"jinghzhu.io/v1"}, ObjectMeta:v1.ObjectMeta{Name:"jinghzhu-example-f7wgv", GenerateName:"jinghzhu-example-", Namespace:"crd", SelfLink:"/apis/jinghzhu.io/v1/namespaces/crd/jinghzhus/jinghzhu-example-f7wgv", UID:"c9a14049-e255-45be-99f7-033a5ea87787", ResourceVersion:"2343534", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63743288791, loc:(*time.Location)(0x1c802e0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"main", Operation:"Update", APIVersion:"jinghzhu.io/v1", Time:(*v1.Time)(0xc00039d1c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00039d1a0)}}}, Spec:v1.JinghzhuSpec{Desired:1, Current:0, PodList:[]string{}}, Status:v1.JinghzhuStatus{State:"Pending", Message:"Created but not processed yet"}}}}

Now, let's check CRD.

$ kubectl get crd
NAME                    CREATED AT
jinghzhus.jinghzhu.io   2020-12-11T13:06:25Z
$ kubectl describe crd jinghzhus.jinghzhu.com
Name:         jinghzhus.jinghzhu.io
Namespace:
Labels:       <none>
Annotations:  <none>
API Version:  apiextensions.k8s.io/v1
Kind:         CustomResourceDefinition
Metadata:
  Creation Timestamp:  2020-12-11T13:06:25Z
  Generation:          1
  Managed Fields:
    API Version:  apiextensions.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:acceptedNames:
          f:kind:
          f:listKind:
          f:plural:
          f:shortNames:
          f:singular:
        f:conditions:
    Manager:      kube-apiserver
    Operation:    Update
    Time:         2020-12-11T13:06:25Z
    API Version:  apiextensions.k8s.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        f:conversion:
          .:
          f:strategy:
        f:group:
        f:names:
          f:kind:
          f:listKind:
          f:plural:
          f:shortNames:
          f:singular:
        f:preserveUnknownFields:
        f:scope:
        f:validation:
          .:
          f:openAPIV3Schema:
            .:
            f:properties:
              .:
              f:spec:
                .:
                f:properties:
                  .:
                  f:current:
                    .:
                    f:format:
                    f:type:
                  f:desired:
                    .:
                    f:format:
                    f:type:
                  f:podList:
                    .:
                    f:items:
                    f:type:
                f:required:
                f:type:
            f:type:
        f:version:
        f:versions:
      f:status:
        f:storedVersions:
    Manager:         main
    Operation:       Update
    Time:            2020-12-11T13:06:25Z
  Resource Version:  2343519
  Self Link:         /apis/apiextensions.k8s.io/v1/customresourcedefinitions/jinghzhus.jinghzhu.io
  UID:               1afe3057-4398-45f7-9200-90b879d52f6a
Spec:
  Conversion:
    Strategy:  None
  Group:       jinghzhu.io
  Names:
    Kind:       Jinghzhu
    List Kind:  JinghzhuList
    Plural:     jinghzhus
    Short Names:
      jh
    Singular:               jinghzhu
  Preserve Unknown Fields:  true
  Scope:                    Namespaced
  Versions:
    Name:  v1
    Schema:
      openAPIV3Schema:
        Properties:
          Spec:
            Properties:
              Current:
                Format:  int
                Type:    integer
              Desired:
                Format:  int
                Type:    integer
              Pod List:
                Items:
                  Type:  string
                Type:    array
            Required:
              desired
            Type:  object
        Type:      object
    Served:        true
    Storage:       true
Status:
  Accepted Names:
    Kind:       Jinghzhu
    List Kind:  JinghzhuList
    Plural:     jinghzhus
    Short Names:
      jh
    Singular:  jinghzhu
  Conditions:
    Last Transition Time:  2020-12-11T13:06:25Z
    Message:               no conflicts found
    Reason:                NoConflicts
    Status:                True
    Type:                  NamesAccepted
    Last Transition Time:  2020-12-11T13:06:25Z
    Message:               the initial names have been accepted
    Reason:                InitialNamesAccepted
    Status:                True
    Type:                  Established
  Stored Versions:
    v1
Events:  <none>
$ kubectl proxy
Starting to serve on 127.0.0.1:8001

$ curl -i 127.0.0.1:8001/apis/jinghzhu.io
HTTP/1.1 200 OK
Cache-Control: no-cache, private
Content-Length: 253
Content-Type: application/json
Date: Fri, 11 Dec 2020 13:11:34 GMT

{
  "kind": "APIGroup",
  "apiVersion": "v1",
  "name": "jinghzhu.io",
  "versions": [
    {
      "groupVersion": "jinghzhu.io/v1",
      "version": "v1"
    }
  ],
  "preferredVersion": {
    "groupVersion": "jinghzhu.io/v1",
    "version": "v1"
  }
}
$ kubectl -n crd get jh
NAME                     AGE
jinghzhu-example-f7wgv   6m20s

$  kubectl -n crd describe jh jinghzhu-example-f7wgv
Name:         jinghzhu-example-f7wgv
Namespace:    crd
Labels:       <none>
Annotations:  <none>
API Version:  jinghzhu.io/v1
Kind:         Jinghzhu
Metadata:
  Creation Timestamp:  2020-12-11T13:06:31Z
  Generate Name:       jinghzhu-example-
  Generation:          1
  Managed Fields:
    API Version:  jinghzhu.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:generateName:
      f:spec:
        .:
        f:current:
        f:desired:
        f:podList:
      f:status:
        .:
        f:message:
        f:state:
    Manager:         main
    Operation:       Update
    Time:            2020-12-11T13:06:31Z
  Resource Version:  2343534
  Self Link:         /apis/jinghzhu.io/v1/namespaces/crd/jinghzhus/jinghzhu-example-f7wgv
  UID:               c9a14049-e255-45be-99f7-033a5ea87787
Spec:
  Current:  0
  Desired:  1
  Pod List:
Status:
  Message:  Created but not processed yet
  State:    Pending
Events:     <none>

鲜花
握手
雷人
路过
鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
More+
上一篇：
jsonwebtoken/jsonwebtoken.github.io发布时间：2022-07-08
下一篇：
PKGeorgiev/Delphi-JsonToDelphiClass: Generates Delphi Classes based on JSON stri ...发布时间：2022-07-08
热门推荐
More+
热门话题
More+
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服（服务时间 9:00～18:00）

在线QQ客服
地址：深圳市南山区西丽大学城创智工业园
电邮：jeky_zhao#qq.com
移动电话：139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap