packages/eggo
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

upgrade to v0.9.4-2

Browse Source 
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
This commit is contained in:
zhangxiaoyu 2021-11-26 09:39:45 +08:00
parent d007f9a1e4
commit a7c766a98c
13 changed files with 1504 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

91
0001-correct-docs-error.patch Normal file
View File

@ -0,0 +1,91 @@
From 80b39c79d6bfa912af46fe45efec92ce34b1c0f2 Mon Sep 17 00:00:00 2001
From: zhangxiaoyu <zhangxiaoyu58@huawei.com>
Date: Fri, 15 Oct 2021 11:45:31 +0800
Subject: [PATCH 01/12] correct docs error
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
---
docs/eggops.md | 2 +-
eggops.yaml | 31 +++++++++++++++++++++++++++----
yaml/base.yaml | 2 +-
3 files changed, 29 insertions(+), 6 deletions(-)
diff --git a/docs/eggops.md b/docs/eggops.md
index 0eec9a5..075b5a5 100644
--- a/docs/eggops.md
+++ b/docs/eggops.md
@@ -239,7 +239,7 @@ metadata:
namespace: eggo-system
spec:
# 用于将package包挂载到容器中，部署集群时使用
- packagePersistentVolumeClain:
+ packagePersistentVolumeClaim:
name: nfs-pvc-example
# 暴露端口，可选项
open-ports:
diff --git a/eggops.yaml b/eggops.yaml
index 42af1e0..606f4da 100644
--- a/eggops.yaml
+++ b/eggops.yaml
@@ -560,6 +560,31 @@ spec:
type: boolean
hasCluster:
type: boolean
+ infrastructureRef:
+ description: 'ObjectReference contains enough information to let you inspect or modify the referred object. --- New uses of this type are discouraged because of difficulty describing its usage when embedded in APIs. 1. Ignored fields. It includes many fields which are not generally honored. For instance, ResourceVersion and FieldPath are both very rarely valid in actual usage. 2. Invalid usage help. It is impossible to add specific help for individual usage. In most embedded usages, there are particular restrictions like, "must refer only to types A and B" or "UID not honored" or "name must be restricted". Those cannot be well described when embedded. 3. Inconsistent validation. Because the usages are different, the validation rules are different by usage, which makes it hard for users to predict what will happen. 4. The fields are both imprecise and overly precise. Kind is not a precise mapping to a URL. This can produce ambiguity during interpretation and require a REST mapping. In most cases, the dependency is on the group,resource tuple and the version of the actual struct is irrelevant. 5. We cannot easily change it. Because this type is embedded in many locations, updates to this type will affect numerous schemas. Don''t make new APIs embed an underspecified API type they do not control. Instead of using this type, create a locally provided and used type that is well-focused on your reference. For example, ServiceReferences for admission registration: https://github.com/kubernetes/api/blob/release-1.17/admissionregistration/v1/types.go#L533 .'
+ properties:
+ apiVersion:
+ description: API version of the referent.
+ type: string
+ fieldPath:
+ description: 'If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: "spec.containers{name}" (where "name" refers to the name of the container that triggered the event) or if no container name is specified "spec.containers[2]" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object. TODO: this design is not final and this field is subject to change in the future.'
+ type: string
+ kind:
+ description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+ type: string
+ name:
+ description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
+ type: string
+ namespace:
+ description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
+ type: string
+ resourceVersion:
+ description: 'Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
+ type: string
+ uid:
+ description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
+ type: string
+ type: object
jobHistorys:
items:
properties:
@@ -1599,11 +1624,9 @@ spec:
control-plane: controller-manager
spec:
containers:
- - args:
- - --leader-elect
- command:
+ - command:
- /manager
- image: hub.oepkgs.net/haozi007/eggops:latest
+ image: hub.oepkgs.net/haozi007/eggops:v1.0.0-alpha
livenessProbe:
httpGet:
path: /healthz
diff --git a/yaml/base.yaml b/yaml/base.yaml
index a6018ed..e750db0 100644
--- a/yaml/base.yaml
+++ b/yaml/base.yaml
@@ -62,7 +62,7 @@ metadata:
namespace: eggo-system
spec:
# 用于将package包挂载到容器中，部署集群时使用
- packagePersistentVolumeClain:
+ packagePersistentVolumeClaim:
name: nfs-pvc-example
# 暴露端口
open-ports:
--
2.25.1

97
0002-disable-service-beforce-setup-service.patch Normal file
View File

@ -0,0 +1,97 @@
From 5d7a00a385e3510efdac8fc44046956417c6f7e9 Mon Sep 17 00:00:00 2001
From: zhangxiaoyu <zhangxiaoyu58@huawei.com>
Date: Tue, 2 Nov 2021 16:19:38 +0800
Subject: [PATCH 02/12] disable service beforce setup service
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
---
Makefile | 8 ++++++--
eggops.yaml | 2 +-
.../binary/cleanupcluster/cleanupcommon.go | 2 +-
.../binary/commontools/systemdservices.go | 2 ++
pkg/clusterdeployment/runtime/runtime.go | 2 +-
5 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/Makefile b/Makefile
index 9722c0c..3fac780 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,11 @@
GIT_COMMIT ?= $(if $(shell git rev-parse --short HEAD),$(shell git rev-parse --short HEAD),$(error "commit id failed"))
SOURCE_DATE_EPOCH ?= $(if $(shell date +%s),$(shell date +%s),$(error "date failed"))
VERSION := $(shell cat ./VERSION)
-ARCH := $(shell arch)
+# eggo arch amd64/arm64
+ARCH ?= amd64
+ifeq ($(shell uname -p),aarch64)
+ARCH ?= arm64
+endif
EXTRALDFLAGS :=
LDFLAGS := -X isula.org/eggo/cmd.Version=$(VERSION) \
@@ -13,7 +17,7 @@ STATIC_LDFLAGS := -extldflags=-static -linkmode=external
SAFEBUILDFLAGS := -buildmode=pie -extldflags=-ftrapv -extldflags=-zrelro -extldflags=-znow -tmpdir=/tmp/xxeggo $(LDFLAGS)
GO := go
-GO_BUILD := CGO_ENABLED=0 $(GO)
+GO_BUILD := CGO_ENABLED=0 GOARCH=$(ARCH) $(GO)
.PHONY: eggo
eggo:
diff --git a/eggops.yaml b/eggops.yaml
index 606f4da..419854a 100644
--- a/eggops.yaml
+++ b/eggops.yaml
@@ -1626,7 +1626,7 @@ spec:
containers:
- command:
- /manager
- image: hub.oepkgs.net/haozi007/eggops:v1.0.0-alpha
+ image: hub.oepkgs.net/haozi007/eggops:1.0.0-alpha
livenessProbe:
httpGet:
path: /healthz
diff --git a/pkg/clusterdeployment/binary/cleanupcluster/cleanupcommon.go b/pkg/clusterdeployment/binary/cleanupcluster/cleanupcommon.go
index 8220746..e681b63 100644
--- a/pkg/clusterdeployment/binary/cleanupcluster/cleanupcommon.go
+++ b/pkg/clusterdeployment/binary/cleanupcluster/cleanupcommon.go
@@ -43,7 +43,7 @@ func PostCleanup(r runner.Runner) {
func stopServices(r runner.Runner, services []string) error {
join := strings.Join(services, " ")
- if _, err := r.RunCommand(fmt.Sprintf("sudo -E /bin/sh -c \"systemctl stop %s\"", join)); err != nil {
+ if _, err := r.RunCommand(fmt.Sprintf("sudo -E /bin/sh -c \"systemctl stop %s && systemctl disable %s\"", join, join)); err != nil {
logrus.Errorf("stop services failed: %v", err)
return err
}
diff --git a/pkg/clusterdeployment/binary/commontools/systemdservices.go b/pkg/clusterdeployment/binary/commontools/systemdservices.go
index 790d3d4..e500aab 100644
--- a/pkg/clusterdeployment/binary/commontools/systemdservices.go
+++ b/pkg/clusterdeployment/binary/commontools/systemdservices.go
@@ -406,8 +406,10 @@ func GetSystemdServiceShell(name string, base64Data string, needRestart bool) (s
shell := `
#!/bin/bash
{{- if .content }}
+systemctl disable {{ .name }}
rm -f /usr/lib/systemd/system/{{ .name }}.service
echo {{ .content }} | base64 -d > /usr/lib/systemd/system/{{ .name }}.service
+systemctl daemon-reload
{{- end }}
which chcon
if [ $? -eq 0 ]; then
diff --git a/pkg/clusterdeployment/runtime/runtime.go b/pkg/clusterdeployment/runtime/runtime.go
index 021ce58..ca5e265 100644
--- a/pkg/clusterdeployment/runtime/runtime.go
+++ b/pkg/clusterdeployment/runtime/runtime.go
@@ -177,7 +177,7 @@ Type=notify
EnvironmentFile=-/etc/sysconfig/docker
ExecStart=/usr/bin/dockerd \
{{- range $i, $v := .registry }}
- --registry-mirrors {{ $v }} \
+ --registry-mirror {{ $v }} \
{{- end }}
{{- range $i, $v := .insecure }}
--insecure-registry {{ $v }} \
--
2.25.1

231
0003-support-disable-rollback-for-deploy-cluster.patch Normal file
View File

@ -0,0 +1,231 @@
From f57d83e82512cd423583f2e41aeb35ad348fa21c Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Wed, 3 Nov 2021 04:01:33 +0000
Subject: [PATCH 03/12] support disable rollback for deploy cluster
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
Makefile | 10 +--
cmd/deploy.go | 84 +++++++++++++++-----------
cmd/opts.go | 38 ++++++------
pkg/clusterdeployment/clusterdeploy.go | 6 +-
4 files changed, 78 insertions(+), 60 deletions(-)
diff --git a/Makefile b/Makefile
index 3fac780..4b6d7c7 100644
--- a/Makefile
+++ b/Makefile
@@ -2,9 +2,11 @@ GIT_COMMIT ?= $(if $(shell git rev-parse --short HEAD),$(shell git rev-parse --s
SOURCE_DATE_EPOCH ?= $(if $(shell date +%s),$(shell date +%s),$(error "date failed"))
VERSION := $(shell cat ./VERSION)
# eggo arch amd64/arm64
-ARCH ?= amd64
-ifeq ($(shell uname -p),aarch64)
-ARCH ?= arm64
+ifndef ARCH
+ARCH = amd64
+ifeq ($(shell uname -p), aarch64)
+ARCH = arm64
+endif
endif
EXTRALDFLAGS :=
@@ -21,7 +23,7 @@ GO_BUILD := CGO_ENABLED=0 GOARCH=$(ARCH) $(GO)
.PHONY: eggo
eggo:
- @echo "build eggo starting..."
+ @echo "build eggo of $(ARCH) starting..."
@$(GO_BUILD) build -ldflags '$(LDFLAGS) $(STATIC_LDFLAGS)' -o bin/eggo .
@echo "build eggo done!"
local:
diff --git a/cmd/deploy.go b/cmd/deploy.go
index 094fb1d..e21bcc5 100644
--- a/cmd/deploy.go
+++ b/cmd/deploy.go
@@ -25,6 +25,47 @@ import (
"isula.org/eggo/pkg/utils"
)
+func removeFailedNodes(cstatus *api.ClusterStatus, conf *DeployConfig) {
+ // if partial success, just update config of cluster, remove failed nodes
+ if cstatus.FailureCnt == 0 {
+ return
+ }
+
+ var tmp []*HostConfig
+ for _, n := range conf.Masters {
+ if success, ok := cstatus.StatusOfNodes[n.Ip]; ok && !success {
+ continue
+ }
+ tmp = append(tmp, n)
+ }
+ conf.Masters = tmp
+
+ tmp = nil
+ for _, n := range conf.Workers {
+ if success, ok := cstatus.StatusOfNodes[n.Ip]; ok && !success {
+ continue
+ }
+ tmp = append(tmp, n)
+ }
+ conf.Workers = tmp
+
+ tmp = nil
+ for _, n := range conf.Etcds {
+ if success, ok := cstatus.StatusOfNodes[n.Ip]; ok && !success {
+ continue
+ }
+ tmp = append(tmp, n)
+ }
+ conf.Etcds = tmp
+
+ if err := saveDeployConfig(conf, savedDeployConfigPath(conf.ClusterID)); err != nil {
+ fmt.Printf("Warn: failed to save config!!!\n")
+ fmt.Printf(" you can call \"eggo delete --id %s [failed nodes id]\" to remove failed node from your cluster.\n", conf.ClusterID)
+ return
+ }
+ fmt.Printf("update config of cluster: %s", conf.ClusterID)
+}
+
func deploy(conf *DeployConfig) error {
if err := saveDeployConfig(conf, savedDeployConfigPath(conf.ClusterID)); err != nil {
return fmt.Errorf("save deploy config failed: %v", err)
@@ -32,47 +73,18 @@ func deploy(conf *DeployConfig) error {
ccfg := toClusterdeploymentConfig(conf)
- cstatus, err := clusterdeployment.CreateCluster(ccfg)
+ cstatus, err := clusterdeployment.CreateCluster(ccfg, opts.deployEnableRollback)
if err != nil {
return err
}
- if cstatus.FailureCnt > 0 {
- // if partial success, just update config of cluster, remove failed nodes
- var tmp []*HostConfig
- for _, n := range conf.Masters {
- if success, ok := cstatus.StatusOfNodes[n.Ip]; ok && !success {
- continue
- }
- tmp = append(tmp, n)
- }
- conf.Masters = tmp
-
- tmp = nil
- for _, n := range conf.Workers {
- if success, ok := cstatus.StatusOfNodes[n.Ip]; ok && !success {
- continue
- }
- tmp = append(tmp, n)
- }
- conf.Workers = tmp
-
- tmp = nil
- for _, n := range conf.Etcds {
- if success, ok := cstatus.StatusOfNodes[n.Ip]; ok && !success {
- continue
- }
- tmp = append(tmp, n)
- }
- conf.Etcds = tmp
-
- err = saveDeployConfig(conf, savedDeployConfigPath(conf.ClusterID))
- if err != nil {
- fmt.Printf("")
- clusterdeployment.RemoveCluster(ccfg)
- return fmt.Errorf("update config of cluster failed: %v", err)
+ // if disable rollback, just ignore error, and wait user to cleanup
+ if opts.deployEnableRollback {
+ removeFailedNodes(&cstatus, conf)
+ } else {
+ if cstatus.FailureCnt > 0 {
+ fmt.Printf("Warn: you can call \"eggo delete --id %s [failed nodes id]\" to remove failed node from your cluster.\n", conf.ClusterID)
}
- fmt.Printf("update config of cluster: %s", conf.ClusterID)
}
fmt.Print(cstatus.Show())
diff --git a/cmd/opts.go b/cmd/opts.go
index 2b81b07..f5204f2 100644
--- a/cmd/opts.go
+++ b/cmd/opts.go
@@ -24,24 +24,25 @@ import (
)
type eggoOptions struct {
- name string
- templateConfig string
- masters []string
- nodes []string
- etcds []string
- loadbalance string
- username string
- password string
- deployConfig string
- cleanupConfig string
- cleanupClusterID string
- debug bool
- version bool
- joinType string
- joinClusterID string
- joinYaml string
- joinHost HostConfig
- delClusterID string
+ name string
+ templateConfig string
+ masters []string
+ nodes []string
+ etcds []string
+ loadbalance string
+ username string
+ password string
+ deployConfig string
+ deployEnableRollback bool
+ cleanupConfig string
+ cleanupClusterID string
+ debug bool
+ version bool
+ joinType string
+ joinClusterID string
+ joinYaml string
+ joinHost HostConfig
+ delClusterID string
}
var opts eggoOptions
@@ -64,6 +65,7 @@ func setupEggoCmdOpts(eggoCmd *cobra.Command) {
func setupDeployCmdOpts(deployCmd *cobra.Command) {
flags := deployCmd.Flags()
flags.StringVarP(&opts.deployConfig, "file", "f", defaultDeployConfigPath(), "location of cluster deploy config file, default $HOME/.eggo/deploy.yaml")
+ flags.BoolVarP(&opts.deployEnableRollback, "rollback", "", true, "rollback failed node to cleanup")
}
func setupCleanupCmdOpts(cleanupCmd *cobra.Command) {
diff --git a/pkg/clusterdeployment/clusterdeploy.go b/pkg/clusterdeployment/clusterdeploy.go
index ee57ec3..27167c4 100644
--- a/pkg/clusterdeployment/clusterdeploy.go
+++ b/pkg/clusterdeployment/clusterdeploy.go
@@ -225,7 +225,7 @@ func rollbackFailedNoeds(handler api.ClusterDeploymentAPI, nodes []*api.HostConf
}
}
-func CreateCluster(cc *api.ClusterConfig) (api.ClusterStatus, error) {
+func CreateCluster(cc *api.ClusterConfig, deployEnableRollback bool) (api.ClusterStatus, error) {
cstatus := api.ClusterStatus{
StatusOfNodes: make(map[string]bool),
}
@@ -262,7 +262,9 @@ func CreateCluster(cc *api.ClusterConfig) (api.ClusterStatus, error) {
return cstatus, err
}
// rollback failed nodes
- rollbackFailedNoeds(handler, failedNodes)
+ if deployEnableRollback {
+ rollbackFailedNoeds(handler, failedNodes)
+ }
// update status of cluster
if failedNodes != nil {
var failureIDs []string
--
2.25.1

198
0004-add-host-name-checker-add-bin-execute-permission.patch Normal file
View File

@ -0,0 +1,198 @@
From ae822ab2b0fd07d33adb7c244e78f2fcd31e1c05 Mon Sep 17 00:00:00 2001
From: zhangxiaoyu <zhangxiaoyu58@huawei.com>
Date: Wed, 3 Nov 2021 15:16:09 +0800
Subject: [PATCH 04/12] add host name checker, add bin execute permission
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
---
cmd/checker.go | 6 ++++++
config/all_online_install.config | 1 +
config/openEuler.config | 8 ++++----
docs/configuration_file_description.md | 2 +-
pkg/utils/dependency/dependency.go | 19 +++++++++++++-----
pkg/utils/dependency/install.go | 27 ++++++++++++++------------
6 files changed, 41 insertions(+), 22 deletions(-)
diff --git a/cmd/checker.go b/cmd/checker.go
index 4530f2b..9d1fda6 100644
--- a/cmd/checker.go
+++ b/cmd/checker.go
@@ -140,6 +140,12 @@ func checkHostconfig(h *HostConfig) error {
if h == nil {
return fmt.Errorf("empty hostconfig")
}
+ if h.Name == "" {
+ return fmt.Errorf("empty host name")
+ }
+ if errs := validation.IsDNS1123Subdomain(h.Name); len(errs) > 0 {
+ return fmt.Errorf("invalid host name: %v", errs)
+ }
if h.Ip == "" {
return fmt.Errorf("host: %s ip is null", h.Name)
}
diff --git a/config/all_online_install.config b/config/all_online_install.config
index dfc3ad4..c8a5eae 100644
--- a/config/all_online_install.config
+++ b/config/all_online_install.config
@@ -17,6 +17,7 @@ workers:
arch: arm64
runtime: iSulad
runtime-endpoint: unix:///var/run/isulad.sock
+pause-image: docker.io/mirrorgcrio/pause:3.2
open-ports:
worker:
- port: 111
diff --git a/config/openEuler.config b/config/openEuler.config
index 07acf9b..a69154b 100755
--- a/config/openEuler.config
+++ b/config/openEuler.config
@@ -1,17 +1,17 @@
-cluster-id: k8s-openEuler
+cluster-id: k8s-openeuler
username: root
password: "123456"
masters:
-- name: openEuler1
+- name: openeuler1
ip: 192.168.0.1
port: 22
arch: arm64
workers:
-- name: openEuler1
+- name: openeuler1
ip: 192.168.0.1
port: 22
arch: arm64
-- name: openEuler2
+- name: openeuler2
ip: 192.168.0.2
port: 22
arch: arm64
diff --git a/docs/configuration_file_description.md b/docs/configuration_file_description.md
index c169f49..4c84e5f 100644
--- a/docs/configuration_file_description.md
+++ b/docs/configuration_file_description.md
@@ -8,7 +8,7 @@ username: root // 需要部署k8s集群的机器的ssh登录
password: 123456 // 需要部署k8s集群的机器的ssh登录密码，所有机器都需要使用同一个密码
private-key-path: ~/.ssh/pri.key // ssh免密登录的密钥，可以替代password防止密码泄露
masters: // 配置master节点的列表，建议每个master节点同时作为worker节点，否则master节点可以无法直接访问pod
-- name: test0 // 该节点的名称，为k8s集群看到的该节点的名称
+- name: test0 // 该节点的名称，为k8s集群看到的该节点的名称，名字需要符合RFC 1123 subdomain规范
ip: 192.168.0.1 // 该节点的ip地址
port: 22 // ssh登录的端口
arch: arm64 // 机器架构，x86_64的填amd64
diff --git a/pkg/utils/dependency/dependency.go b/pkg/utils/dependency/dependency.go
index 3d0bf8f..9e6ac22 100644
--- a/pkg/utils/dependency/dependency.go
+++ b/pkg/utils/dependency/dependency.go
@@ -143,15 +143,23 @@ func (dd *dependencyDeb) Remove(r runner.Runner) error {
}
// install file and dir
-type dependencyFD struct {
- srcPath string
- software []*api.PackageConfig
+type dependencyFileDir struct {
+ executable bool
+ srcPath string
+ software []*api.PackageConfig
}
-func (df *dependencyFD) Install(r runner.Runner) error {
+func (df *dependencyFileDir) Install(r runner.Runner) error {
shell := `
#!/bin/bash
cd {{ .srcPath }}
+
+{{- if .executable }}
+{{- range $i, $v := .software }}
+chmod +x {{ $v.Name }}
+{{- end }}
+{{- end }}
+
{{- range $i, $v := .software }}
if [ ! -e {{ JoinPath $v.Dst $v.Name }} ]; then
mkdir -p {{ $v.Dst }} && cp -r {{ $v.Name }} {{ $v.Dst }}
@@ -161,6 +169,7 @@ fi
datastore := make(map[string]interface{})
datastore["srcPath"] = df.srcPath
datastore["software"] = df.software
+ datastore["executable"] = df.executable
shellStr, err := template.TemplateRender(shell, datastore)
if err != nil {
@@ -175,7 +184,7 @@ fi
return nil
}
-func (df *dependencyFD) Remove(r runner.Runner) error {
+func (df *dependencyFileDir) Remove(r runner.Runner) error {
var sb strings.Builder
sb.WriteString("sudo -E /bin/sh -c \"")
for _, s := range df.software {
diff --git a/pkg/utils/dependency/install.go b/pkg/utils/dependency/install.go
index 611c006..e2c0621 100644
--- a/pkg/utils/dependency/install.go
+++ b/pkg/utils/dependency/install.go
@@ -125,23 +125,26 @@ func installFD(r runner.Runner, bin, file, dir []*api.PackageConfig, hcf *api.Ho
dp := []dependency{}
if len(bin) != 0 {
- dp = append(dp, &dependencyFD{
- srcPath: path.Join(packagePath, constants.DefaultBinPath),
- software: bin,
+ dp = append(dp, &dependencyFileDir{
+ srcPath: path.Join(packagePath, constants.DefaultBinPath),
+ software: bin,
+ executable: true,
})
}
if len(file) != 0 {
- dp = append(dp, &dependencyFD{
- srcPath: path.Join(packagePath, constants.DefaultFilePath),
- software: file,
+ dp = append(dp, &dependencyFileDir{
+ srcPath: path.Join(packagePath, constants.DefaultFilePath),
+ software: file,
+ executable: false,
})
}
if len(dir) != 0 {
- dp = append(dp, &dependencyFD{
- srcPath: path.Join(packagePath, constants.DefaultDirPath),
- software: dir,
+ dp = append(dp, &dependencyFileDir{
+ srcPath: path.Join(packagePath, constants.DefaultDirPath),
+ software: dir,
+ executable: false,
})
}
@@ -233,19 +236,19 @@ func uninstallFD(r runner.Runner, bin, file, dir []*api.PackageConfig, hcf *api.
dp := []dependency{}
if len(bin) != 0 {
- dp = append(dp, &dependencyFD{
+ dp = append(dp, &dependencyFileDir{
software: bin,
})
}
if len(file) != 0 {
- dp = append(dp, &dependencyFD{
+ dp = append(dp, &dependencyFileDir{
software: file,
})
}
if len(dir) != 0 {
- dp = append(dp, &dependencyFD{
+ dp = append(dp, &dependencyFileDir{
software: dir,
})
}
--
2.25.1

105
0005-wait-task-longer-and-modify-eggops-docs.patch Normal file
View File

@ -0,0 +1,105 @@
From d1bbe9a5012b5395d4d9e988503c149f22716f0f Mon Sep 17 00:00:00 2001
From: zhangxiaoyu <zhangxiaoyu58@huawei.com>
Date: Thu, 4 Nov 2021 16:04:17 +0800
Subject: [PATCH 05/12] wait task longer and modify eggops docs
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
---
docs/eggops.md | 18 +++++++++++++++++-
.../binary/etcdcluster/etcdjoin.go | 2 +-
yaml/cluster.yaml | 1 +
yaml/machines.yaml | 2 ++
4 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/docs/eggops.md b/docs/eggops.md
index 075b5a5..7dea94c 100644
--- a/docs/eggops.md
+++ b/docs/eggops.md
@@ -82,7 +82,7 @@ $ tree /data
1 directory, 2 files
-# 安装 nfs 与 rpc 相关软件包：
+# 在nfs server机器上安装 nfs 与 rpc 相关软件包：
$ yum install nfs-utils rpcbind -y
# NFS默认的配置文件是 /etc/exports，修改配置文件
@@ -98,6 +98,9 @@ success
$ firewall-cmd --reload
success
+# 在所有的worker节点上安装nfs-utils，详情见常见问题1
+$ yum install nfs-utils -y
+
# 查看NFS分享的资源
$ showmount -e <nfs ip>
Export list for <nfs ip>:
@@ -214,6 +217,7 @@ apiVersion: eggo.isula.org/v1
kind: Machine
metadata:
name: machine1-example
+ namespace: eggo-system
labels:
masterRole: allow
workerRole: allow
@@ -376,3 +380,15 @@ kubectl delete -f cluster.yaml --wait=false
$ export KUBECONFIG=/etc/kubernetes/admin.conf
$ kubectl delete -f eggops.yaml
```
+
+### 常见问题
+
+1. pod一直ContainerCreating
+
+创建cluster时，发现cluster-example-create-job的pod一直处于ContainerCreating状态，describe该pod状态，发现报错
+```
+Mounting arguments: -t nfs -o ro 192.168.223.236:/data /var/lib/kubelet/pods/8becdc2f-a31a-4cab-8a3b-2f3e705cac92/volumes/kubernetes.io~nfs/nfs-pv-example
+Output: mount: /var/lib/kubelet/pods/8becdc2f-a31a-4cab-8a3b-2f3e705cac92/volumes/kubernetes.io~nfs/nfs-pv-example: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program.
+```
+
+原因是因为pod所在的host机器上没有nfs-utils软件包，将该软件包在pod所在的宿主机上安装即可解决此问题
diff --git a/pkg/clusterdeployment/binary/etcdcluster/etcdjoin.go b/pkg/clusterdeployment/binary/etcdcluster/etcdjoin.go
index 07b9d7b..97be436 100644
--- a/pkg/clusterdeployment/binary/etcdcluster/etcdjoin.go
+++ b/pkg/clusterdeployment/binary/etcdcluster/etcdjoin.go
@@ -48,7 +48,7 @@ func AddMember(conf *api.ClusterConfig, hostconfig *api.HostConfig) error {
return fmt.Errorf("run task on nodes failed: %v", err)
}
- if err := nodemanager.WaitNodesFinish([]string{hostconfig.Address}, time.Minute); err != nil {
+ if err := nodemanager.WaitNodesFinish([]string{hostconfig.Address}, 5*time.Minute); err != nil {
return fmt.Errorf("wait for post deploy etcds task finish failed: %v", err)
}
diff --git a/yaml/cluster.yaml b/yaml/cluster.yaml
index 6af86f3..cf883c1 100644
--- a/yaml/cluster.yaml
+++ b/yaml/cluster.yaml
@@ -18,3 +18,4 @@ spec:
name: infrastructure-example
# 启用kubelet serving证书
enableKubeletServing: true
+ eggoImageVersion: "hub.oepkgs.net/haozi007/eggo:1.0.0-alpha"
diff --git a/yaml/machines.yaml b/yaml/machines.yaml
index d9fe575..bc19743 100644
--- a/yaml/machines.yaml
+++ b/yaml/machines.yaml
@@ -4,6 +4,7 @@ apiVersion: eggo.isula.org/v1
kind: Machine
metadata:
name: machine1-example
+ namespace: eggo-system
labels:
masterRole: allow
workerRole: allow
@@ -19,6 +20,7 @@ apiVersion: eggo.isula.org/v1
kind: Machine
metadata:
name: machine2-example
+ namespace: eggo-system
labels:
masterRole: allow
workerRole: allow
--
2.25.1

47
0006-fix-eggo-deploy-bugs.patch Normal file
View File

@ -0,0 +1,47 @@
From feac66a4f25cff3a0df242fd9ef553d19d71b807 Mon Sep 17 00:00:00 2001
From: zhangxiaoyu <zhangxiaoyu58@huawei.com>
Date: Fri, 5 Nov 2021 15:40:40 +0800
Subject: [PATCH 06/12] fix eggo deploy bugs
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
---
pkg/clusterdeployment/binary/commontools/systemdservices.go | 3 ++-
pkg/clusterdeployment/runtime/runtime.go | 5 +++++
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/pkg/clusterdeployment/binary/commontools/systemdservices.go b/pkg/clusterdeployment/binary/commontools/systemdservices.go
index e500aab..0a17ee2 100644
--- a/pkg/clusterdeployment/binary/commontools/systemdservices.go
+++ b/pkg/clusterdeployment/binary/commontools/systemdservices.go
@@ -406,10 +406,11 @@ func GetSystemdServiceShell(name string, base64Data string, needRestart bool) (s
shell := `
#!/bin/bash
{{- if .content }}
-systemctl disable {{ .name }}
rm -f /usr/lib/systemd/system/{{ .name }}.service
echo {{ .content }} | base64 -d > /usr/lib/systemd/system/{{ .name }}.service
systemctl daemon-reload
+# disable success when service file exist
+systemctl disable {{ .name }}
{{- end }}
which chcon
if [ $? -eq 0 ]; then
diff --git a/pkg/clusterdeployment/runtime/runtime.go b/pkg/clusterdeployment/runtime/runtime.go
index ca5e265..edfc4a7 100644
--- a/pkg/clusterdeployment/runtime/runtime.go
+++ b/pkg/clusterdeployment/runtime/runtime.go
@@ -425,6 +425,11 @@ func (ct *DeployRuntimeTask) Run(r runner.Runner, hcg *api.HostConfig) error {
return err
}
+ if _, err := r.RunCommand("sudo -E /bin/sh -c \"rm -rf /etc/docker/daemon.json\""); err != nil {
+ logrus.Errorf("rm docker daemon.json failed: %v", err)
+ return err
+ }
+
if err := ct.runtime.PrepareRuntimeService(r, ct.workerConfig); err != nil {
logrus.Errorf("prepare container engine service failed: %v", err)
return err
--
2.25.1

27
0007-non-root-user-deploy-bug-fixed.patch Normal file
View File

@ -0,0 +1,27 @@
From 3ba4145d267d85b6c2daaa3c254b5dd44a73d6fb Mon Sep 17 00:00:00 2001
From: zhangxiaoyu <zhangxiaoyu58@huawei.com>
Date: Mon, 8 Nov 2021 10:53:43 +0800
Subject: [PATCH 07/12] non-root user deploy bug fixed
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
---
pkg/utils/runner/runner.go | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/pkg/utils/runner/runner.go b/pkg/utils/runner/runner.go
index 9e1689e..c7088df 100644
--- a/pkg/utils/runner/runner.go
+++ b/pkg/utils/runner/runner.go
@@ -179,7 +179,8 @@ func prepareUserTempDir(conn ssh.Connection, host *kkv1alpha1.HostCfg) error {
var sb strings.Builder
sb.WriteString("sudo -E /bin/sh -c \"")
sb.WriteString(fmt.Sprintf("mkdir -p %s", dir))
- sb.WriteString(fmt.Sprintf(" && chown -R %s:%s %s", host.User, host.User, dir))
+ // chown .eggo dir
+ sb.WriteString(fmt.Sprintf(" && chown -R %s:%s %s", host.User, host.User, filepath.Dir(dir)))
sb.WriteString("\"")
_, err := conn.Exec(sb.String(), host)
if err != nil {
--
2.25.1

39
0008-fix-post-join-hook-do-not-run-bug.patch Normal file
View File

@ -0,0 +1,39 @@
From 8499061627913792f96748e6aaf9cac6f9e9f025 Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Mon, 8 Nov 2021 06:50:38 +0000
Subject: [PATCH 08/12] fix post join hook do not run bug
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
pkg/clusterdeployment/clusterdeploy.go | 2 +-
pkg/utils/dependency/install.go | 1 +
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/pkg/clusterdeployment/clusterdeploy.go b/pkg/clusterdeployment/clusterdeploy.go
index 27167c4..d8941ee 100644
--- a/pkg/clusterdeployment/clusterdeploy.go
+++ b/pkg/clusterdeployment/clusterdeploy.go
@@ -191,7 +191,7 @@ func doCreateCluster(handler api.ClusterDeploymentAPI, cc *api.ClusterConfig, cs
approveServingCsr(cc, append(joinedNodes, controlPlaneNode))
// Step9: run postcreate cluster hooks
- if err = handler.PostCreateClusterHooks(joinedNodes); err != nil {
+ if err = handler.PostCreateClusterHooks(cc.Nodes); err != nil {
return nil, err
}
diff --git a/pkg/utils/dependency/install.go b/pkg/utils/dependency/install.go
index e2c0621..8687602 100644
--- a/pkg/utils/dependency/install.go
+++ b/pkg/utils/dependency/install.go
@@ -354,6 +354,7 @@ func ExecuteShell(roleInfra *api.RoleInfra, packagePath string, hcf *api.HostCon
if len(shell) == 0 {
return nil
}
+ logrus.Debugf("run %s shell %v on %v\n", string(schedule), shell, hcf.Address)
dp := &dependencyShell{
srcPath: path.Join(packagePath, constants.DefaultFilePath),
--
2.25.1

98
0009-add-spec-of-eggo-hook.patch Normal file
View File

@ -0,0 +1,98 @@
From 70113bd0d942f714abda2bb181386e7a041ee9cf Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Fri, 12 Nov 2021 09:49:35 +0800
Subject: [PATCH 09/12] add spec of eggo hook
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
docs/hooks_of_eggo.md | 63 +++++++++++++++++++++++++++++++++++++++++++
docs/manual.md | 6 +++++
2 files changed, 69 insertions(+)
create mode 100644 docs/hooks_of_eggo.md
diff --git a/docs/hooks_of_eggo.md b/docs/hooks_of_eggo.md
new file mode 100644
index 0000000..92b9a38
--- /dev/null
+++ b/docs/hooks_of_eggo.md
@@ -0,0 +1,63 @@
+# eggo支持的hooks设计
+
+为了提供更好的灵活性，eggo支持多种hooks，主要分为如下几个场景：
+
+- 集群生命周期管理的过程中针对特定角色的每个节点的hooks，支持prehook和posthook两种；
+ - 生命周期管理包括：集群创建、集群删除、节点加入、节点删除；
+ - 节点角色包括：master、worker、etcd和loadbalance；
+- 针对整个集群的hooks，集群创建和删除的过程中只会在一个master节点上执行一次，支持prehook和posthook两种；
+
+## 配置hook方式
+
+### 命令行参数方式
+
+| 参数 | 支持的命令 | 说明 |
+| ----------------------------- | --------------------------- | ------------------------------------------------------------ |
+| --cluster-prehook=[dir/file] | deploy, cleanup | 设置集群创建/删除之前执行的hooks，可以是一个脚本文件或者目录 |
+| --cluster-posthook=[dir/file] | deploy, cleanup | 设置集群创建/删除之后执行的hooks，可以是一个脚本文件或者目录 |
+| --prehook=[dir/file],role | deploy, cleanup,join,delete | 集群创建/删除，节点加入/删除之前执行的hooks，可以是脚本文件或者目录；role设置执行脚本的节点角色； |
+| --posthook=[dir/file],role | deploy, cleanup,join,delete | 集群创建/删除，节点加入/删除之后执行的hooks，可以是脚本文件或者目录；role设置执行脚本的节点角色； |
+
+说明：
+
+- 脚本目录下的所有脚本都会被执行，而子目录中的脚本不会被执行；
+- 每个脚本的超时时间为60s；
+
+### 配置文件参数方式
+
+在集群配置的addition字段中，可以设置shell类型的文件：
+
+- 支持prejoin、postjoin、precleanup和postcleanup等执行时机；
+- 而且通过在master或者worker角色下配置，设置hook执行的节点类型；
+
+示例如下：
+
+```
+ addition: // 额外的安装包或二进制文件列表
+ master:
+ - name: prejoin.sh
+ type: shell // shell脚本
+ schedule: "prejoin" // 执行时间master节点加入集群前
+ TimeOut: "30s" // 脚本执行时间，超时则被杀死，未配置默认30s
+ worker:
+ - name: postjoin.sh
+ type: shell // shell脚本
+ schedule: "postjoin" // 执行时间worker节点加入集群后
+```
+
+## hook规范
+
+eggo会在hook执行时，通过环境变量传递部分信息，用于脚本执行。环境变量如下：
+
+| key | value说明 |
+| ------------------------- | ------------------------------------------- |
+| EGGO_CLUSTER_ID | 集群ID |
+| EGGO_CLUSTER_API_ENDPOINT | 集群的API入口 |
+| EGGO_CLUSTER_CONFIG_DIR | 集群配置存放目录，默认/etc/kubernetes |
+| EGGO_NODE_IP | hook执行的节点IP |
+| EGGO_NODE_NAME | hook执行的节点name |
+| EGGO_NODE_ARCH | hook执行的节点架构 |
+| EGGO_NODE_ROLE | hook执行的节点角色 |
+| EGGO_HOOK_TYPE | hook的类型，prehook或者posthook |
+| EGGO_OPERATOR | 当前的操作，deploy，cleanup，join，delete。 |
+
diff --git a/docs/manual.md b/docs/manual.md
index a570d5c..05668ab 100644
--- a/docs/manual.md
+++ b/docs/manual.md
@@ -236,3 +236,9 @@ $ eggo -d delete --id k8s-cluster 192.168.0.5 192.168.0.6
* --id集群的id
* --type可以为master或者worker，默认worker
* 192.168.0.5 需要删除的机器的IP地址列表或者名称列表，注意第1个master节点不能删除
+
+## 规范说明
+
+### hook规范
+
+详情见[hooks spec](./hooks_of_eggo.md)
--
2.25.1

44
0010-update-roadmap-of-eggo.patch Normal file
View File

@ -0,0 +1,44 @@
From 6d7537f67aba590c1f301db3ff0ab47847921155 Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Tue, 9 Nov 2021 08:31:37 +0000
Subject: [PATCH 10/12] update roadmap of eggo
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
docs/roadmap.md => ROADMAP.md | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
rename docs/roadmap.md => ROADMAP.md (88%)
diff --git a/docs/roadmap.md b/ROADMAP.md
similarity index 88%
rename from docs/roadmap.md
rename to ROADMAP.md
index e9d2a13..ee21ca8 100644
--- a/docs/roadmap.md
+++ b/ROADMAP.md
@@ -3,8 +3,10 @@
- [x] 2021-4-1：完成整体设计文档；
- [x] 2021-6-30：完成点火组件的能力，尽快完成一键部署单集群的能力；
- [x] 2021-7-30：完成点火组件容器化；
-- [ ] 2021-10-30：完成GitOps集群管理集群的能力；
+- [x] 2021-10-30：完成GitOps集群管理集群的能力；
- [ ] 2021-12-30：完成kubeadm驱动支持；
+- [ ] 2021-11-30：支持一键部署iSulad+stratovirt；
+- [ ] 2021-12-30：支持一键部署karmada集群；
## GitOps集群管理集群
@@ -12,8 +14,8 @@
- [x] 2021-8-1：完成CRD开发；
- [x] 2021-8-15：完成创建集群的流程开发；
- [x] 2021-8-20：完成删除集群的流程开发；
-- [ ] 2021-10-25：完成节点加入的开发；
-- [ ] 2021-10-30：完成节点删除的开发；
+- [ ] 2021-12-30：完成节点加入的开发；
+- [ ] 2021-12-30：完成节点删除的开发；
## 点火组件开发计划
--
2.25.1

468
0011-run-hook-with-envs-of-cluster-info.patch Normal file
View File

@ -0,0 +1,468 @@
From f1d2766bda44a878777ea266573236271dcc65ed Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Thu, 11 Nov 2021 11:15:37 +0000
Subject: [PATCH 11/12] run hook with envs of cluster info
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
docs/hooks_of_eggo.md | 2 +-
pkg/api/tools.go | 29 ++++++++
pkg/api/types.go | 42 ++++++++++++
.../binary/infrastructure/infrastructure.go | 10 +--
pkg/utils/dependency/dependency.go | 40 ++++++++---
pkg/utils/dependency/dependency_test.go | 52 +++++++++++++++
pkg/utils/dependency/install.go | 66 +++++++++++++++----
pkg/utils/runner/runner.go | 16 ++---
8 files changed, 216 insertions(+), 41 deletions(-)
create mode 100644 pkg/utils/dependency/dependency_test.go
diff --git a/docs/hooks_of_eggo.md b/docs/hooks_of_eggo.md
index 92b9a38..b1f09cb 100644
--- a/docs/hooks_of_eggo.md
+++ b/docs/hooks_of_eggo.md
@@ -22,6 +22,7 @@
- 脚本目录下的所有脚本都会被执行，而子目录中的脚本不会被执行；
- 每个脚本的超时时间为60s；
+- role可以为master,worker,etcd或者loadbalance；
### 配置文件参数方式
@@ -60,4 +61,3 @@ eggo会在hook执行时，通过环境变量传递部分信息，用于脚本执
| EGGO_NODE_ROLE | hook执行的节点角色 |
| EGGO_HOOK_TYPE | hook的类型，prehook或者posthook |
| EGGO_OPERATOR | 当前的操作，deploy，cleanup，join，delete。 |
-
diff --git a/pkg/api/tools.go b/pkg/api/tools.go
index 861d70a..4c65dc2 100644
--- a/pkg/api/tools.go
+++ b/pkg/api/tools.go
@@ -55,6 +55,10 @@ func (p PackageSrcConfig) GetPkgDstPath() string {
return p.DstPath
}
+func (ep APIEndpoint) GetUrl() string {
+ return fmt.Sprintf("%s/%v", ep.AdvertiseAddress, ep.BindPort)
+}
+
func GetClusterHomePath(cluster string) string {
return filepath.Join(EggoHomePath, cluster)
}
@@ -191,3 +195,28 @@ func ParseScheduleType(schedule string) (ScheduleType, error) {
return SchedulePreJoin, fmt.Errorf("invalid schedule type: %s", schedule)
}
}
+
+func GetRoleString(roles uint16) []string {
+ var roleStrs []string
+ if roles&Master != 0 {
+ roleStrs = append(roleStrs, "master")
+ }
+ if roles&Worker != 0 {
+ roleStrs = append(roleStrs, "worker")
+ }
+ if roles&ETCD != 0 {
+ roleStrs = append(roleStrs, "etcd")
+ }
+ if roles&LoadBalance != 0 {
+ roleStrs = append(roleStrs, "loadbalance")
+ }
+
+ return roleStrs
+}
+
+func GetUserTempDir(user string) string {
+ if user == "root" {
+ return constants.DefaultRootCopyTempDirHome
+ }
+ return fmt.Sprintf(constants.DefaultUserCopyTempHomeFormat, user)
+}
diff --git a/pkg/api/types.go b/pkg/api/types.go
index 6a1351e..fea3f26 100644
--- a/pkg/api/types.go
+++ b/pkg/api/types.go
@@ -35,6 +35,37 @@ const (
SchedulePostCleanup ScheduleType = "postcleanup"
)
+type HookOperator string
+
+const (
+ HookOpDeploy HookOperator = "deploy"
+ HookOpCleanup HookOperator = "cleanup"
+ HookOpJoin HookOperator = "join"
+ HookOpDelete HookOperator = "delete"
+)
+
+type HookType string
+
+const (
+ PreHookType HookType = "prehook"
+ PostHookType HookType = "posthook"
+)
+
+type HookRunConfig struct {
+ ClusterID string
+ ClusterApiEndpoint string
+ ClusterConfigDir string
+
+ HookType HookType
+ Operator HookOperator
+
+ Node *HostConfig
+ Scheduler ScheduleType
+
+ HookDir string
+ Hooks []*PackageConfig
+}
+
type RoleInfra struct {
OpenPorts []*OpenPorts `json:"open-ports"`
Softwares []*PackageConfig `json:"softwares"`
@@ -201,6 +232,14 @@ type AddonConfig struct {
Filename string `json:"filename"`
}
+type ClusterHookConf struct {
+ Type HookType
+ Operator HookOperator
+ Target uint16
+ HookDir string
+ HookFiles []string
+}
+
type ClusterConfig struct {
Name string `json:"name"`
DeployDriver string `json:"deploy-driver"` // default is binary
@@ -218,6 +257,9 @@ type ClusterConfig struct {
WorkerConfig WorkerConfig `json:"workerconfig"`
RoleInfra map[uint16]*RoleInfra `json:"role-infra"`
+ // do not encode hooks, just set before use it
+ HooksConf *ClusterHookConf `json:"-"`
+
// TODO: add other configurations at here
}
diff --git a/pkg/clusterdeployment/binary/infrastructure/infrastructure.go b/pkg/clusterdeployment/binary/infrastructure/infrastructure.go
index 634e338..68faf36 100644
--- a/pkg/clusterdeployment/binary/infrastructure/infrastructure.go
+++ b/pkg/clusterdeployment/binary/infrastructure/infrastructure.go
@@ -27,7 +27,6 @@ import (
"isula.org/eggo/pkg/api"
"isula.org/eggo/pkg/clusterdeployment/binary/cleanupcluster"
- "isula.org/eggo/pkg/constants"
"isula.org/eggo/pkg/utils"
"isula.org/eggo/pkg/utils/dependency"
"isula.org/eggo/pkg/utils/nodemanager"
@@ -321,13 +320,6 @@ func (it *DestroyInfraTask) Name() string {
return "DestroyInfraTask"
}
-func getCopyDefaultDir(user string) string {
- if user == "root" {
- return constants.DefaultRootCopyTempDirHome
- }
- return fmt.Sprintf(constants.DefaultUserCopyTempHomeFormat, user)
-}
-
func (it *DestroyInfraTask) Run(r runner.Runner, hcg *api.HostConfig) error {
if hcg == nil {
return fmt.Errorf("empty host config")
@@ -348,7 +340,7 @@ func (it *DestroyInfraTask) Run(r runner.Runner, hcg *api.HostConfig) error {
logrus.Errorf("path %s not in White List and cannot remove", dstDir)
return nil
}
- copyTempDir := getCopyDefaultDir(hcg.UserName)
+ copyTempDir := api.GetUserTempDir(hcg.UserName)
if _, err := r.RunCommand(fmt.Sprintf("sudo -E /bin/sh -c \"rm -rf %s %s %s\"", dstDir, copyTempDir, it.k8sConfigDir)); err != nil {
return fmt.Errorf("rm dependency failed: %v", err)
}
diff --git a/pkg/utils/dependency/dependency.go b/pkg/utils/dependency/dependency.go
index 9e6ac22..2c5dc26 100644
--- a/pkg/utils/dependency/dependency.go
+++ b/pkg/utils/dependency/dependency.go
@@ -293,6 +293,7 @@ func (dy *dependencyYaml) Remove(r runner.Runner) error {
}
type dependencyShell struct {
+ envs []string
srcPath string
shell []*api.PackageConfig
}
@@ -305,22 +306,45 @@ func NewDependencyShell(srcPath string, shell []*api.PackageConfig) *dependencyS
}
func (ds *dependencyShell) Install(r runner.Runner) error {
- var sb strings.Builder
+ shellTemplate := `
+#!/bin/bash
+{{- range $i, $v := .Envs }}
+export {{ $v }}
+{{- end }}
- sb.WriteString("sudo -E /bin/sh -c \"")
- for _, s := range ds.shell {
- sb.WriteString(fmt.Sprintf("chmod +x %s/%s && ", ds.srcPath, s.Name))
+{{- $tout := .Timeouts }}
+{{- range $i, $v := .Shells }}
+chmod +x {{ $v }} && timeout -s SIGKILL {{index $tout $i}} {{ $v }} > /dev/null
+if [ $? -ne 0 ]; then
+ echo "run {{ $v }} failed"
+ exit 1
+fi
+{{- end }}
+exit 0
+`
+ datastore := map[string]interface{}{}
+ datastore["Envs"] = ds.envs
+ var shells []string
+ var timeouts []string
+ for _, s := range ds.shell {
+ shells = append(shells, fmt.Sprintf("%s/%s", ds.srcPath, s.Name))
timeout := s.TimeOut
if timeout == "" {
timeout = "30s"
}
- sb.WriteString(fmt.Sprintf("timeout -s SIGKILL %s %s/%s > /dev/null ; ", timeout, ds.srcPath, s.Name))
+ timeouts = append(timeouts, timeout)
}
- sb.WriteString("\"")
+ datastore["Shells"] = shells
+ datastore["Timeouts"] = timeouts
- if _, err := r.RunCommand(sb.String()); err != nil {
- return fmt.Errorf("shell execute failed: %v", err)
+ parsedShell, err := template.TemplateRender(shellTemplate, datastore)
+ if err != nil {
+ return err
+ }
+
+ if _, err := r.RunShell(parsedShell, "exechook"); err != nil {
+ return fmt.Errorf("hook execute failed: %v", err)
}
return nil
diff --git a/pkg/utils/dependency/dependency_test.go b/pkg/utils/dependency/dependency_test.go
new file mode 100644
index 0000000..58ea756
--- /dev/null
+++ b/pkg/utils/dependency/dependency_test.go
@@ -0,0 +1,52 @@
+package dependency
+
+import (
+ "testing"
+
+ "github.com/sirupsen/logrus"
+ "isula.org/eggo/pkg/api"
+)
+
+type MockRunner struct {
+}
+
+func (m *MockRunner) Copy(src, dst string) error {
+ logrus.Infof("copy %s to %s", src, dst)
+ return nil
+}
+
+func (m *MockRunner) RunCommand(cmd string) (string, error) {
+ logrus.Infof("run command: %s", cmd)
+ return "", nil
+}
+
+func (m *MockRunner) RunShell(shell string, name string) (string, error) {
+ logrus.Infof("run shell: %s, name: %s", shell, name)
+ return "", nil
+}
+
+func (m *MockRunner) Reconnect() error {
+ logrus.Infof("reconnect")
+ return nil
+}
+
+func (m *MockRunner) Close() {
+ logrus.Infof("close")
+}
+
+func TestNewDependencyShell(t *testing.T) {
+ var mr MockRunner
+
+ shell := &api.PackageConfig{
+ Name: "test.sh",
+ Type: "shell",
+ Dst: "/root",
+ Schedule: api.SchedulePreJoin,
+ TimeOut: "30s",
+ }
+
+ dp := NewDependencyShell("/tmp", []*api.PackageConfig{shell})
+ if err := dp.Install(&mr); err != nil {
+ t.Fatalf("run test failed: %v", err)
+ }
+}
diff --git a/pkg/utils/dependency/install.go b/pkg/utils/dependency/install.go
index 8687602..8cb80f6 100644
--- a/pkg/utils/dependency/install.go
+++ b/pkg/utils/dependency/install.go
@@ -349,33 +349,77 @@ func getShell(roleInfra *api.RoleInfra, schedule api.ScheduleType) []*api.Packag
return shell
}
-func ExecuteShell(roleInfra *api.RoleInfra, packagePath string, hcf *api.HostConfig, schedule api.ScheduleType) error {
- shell := getShell(roleInfra, schedule)
- if len(shell) == 0 {
+func ExecuteHooks(hookConf *api.HookRunConfig) error {
+ if hookConf == nil || len(hookConf.Hooks) == 0 {
return nil
}
- logrus.Debugf("run %s shell %v on %v\n", string(schedule), shell, hcf.Address)
- dp := &dependencyShell{
- srcPath: path.Join(packagePath, constants.DefaultFilePath),
- shell: shell,
+ var hookStr []string
+ for _, h := range hookConf.Hooks {
+ hookStr = append(hookStr, h.Name)
}
+ logrus.Debugf("run %s shell %v on %v\n", string(hookConf.Scheduler), hookStr, hookConf.Node.Address)
+
+ dp := &dependencyShell{
+ srcPath: hookConf.HookDir,
+ shell: hookConf.Hooks,
+ }
+ envs := make([]string, 9)
+ envs[0] = fmt.Sprintf("EGGO_CLUSTER_ID=%s", hookConf.ClusterID)
+ envs[1] = fmt.Sprintf("EGGO_CLUSTER_API_ENDPOINT=%s", hookConf.ClusterApiEndpoint)
+ envs[2] = fmt.Sprintf("EGGO_CLUSTER_CONFIG_DIR=%s", hookConf.ClusterConfigDir)
+ envs[3] = fmt.Sprintf("EGGO_NODE_IP=%s", hookConf.Node.Address)
+ envs[4] = fmt.Sprintf("EGGO_NODE_NAME=%s", hookConf.Node.Name)
+ envs[5] = fmt.Sprintf("EGGO_NODE_ARCH=%s", hookConf.Node.Arch)
+ envs[6] = fmt.Sprintf("EGGO_NODE_ROLE=%s", strings.Join(api.GetRoleString(hookConf.Node.Type), ","))
+ envs[7] = fmt.Sprintf("EGGO_HOOK_TYPE=%s", hookConf.HookType)
+ envs[8] = fmt.Sprintf("EGGO_OPERATOR=%s", hookConf.Operator)
+ dp.envs = envs
dependencyTask := task.NewTaskInstance(&DependencyTask{
dp: dp,
})
- if api.IsCleanupSchedule(schedule) {
+ if api.IsCleanupSchedule(hookConf.Scheduler) {
task.SetIgnoreErrorFlag(dependencyTask)
}
- if err := nodemanager.RunTaskOnNodes(dependencyTask, []string{hcf.Address}); err != nil {
- logrus.Errorf("Hook %s failed for %s: %v", string(api.SchedulePreJoin), hcf.Address, err)
+ if err := nodemanager.RunTaskOnNodes(dependencyTask, []string{hookConf.Node.Address}); err != nil {
+ logrus.Errorf("Hook %s failed for %s: %v", string(api.SchedulePreJoin), hookConf.Node.Address, err)
return err
}
return nil
}
+func executeShell(ccfg *api.ClusterConfig, role uint16, hcf *api.HostConfig, schedule api.ScheduleType) error {
+ shell := getShell(ccfg.RoleInfra[role], schedule)
+ if len(shell) == 0 {
+ return nil
+ }
+
+ htype := api.PreHookType
+ if strings.HasPrefix(string(schedule), "post") {
+ htype = api.PostHookType
+ }
+ oper := api.HookOpJoin
+ if strings.HasSuffix(string(schedule), "cleanup") {
+ oper = api.HookOpCleanup
+ }
+
+ hookConf := &api.HookRunConfig{
+ ClusterID: ccfg.Name,
+ ClusterApiEndpoint: ccfg.APIEndpoint.GetUrl(),
+ ClusterConfigDir: ccfg.ConfigDir,
+ HookType: htype,
+ Operator: oper,
+ Node: hcf,
+ HookDir: path.Join(ccfg.PackageSrc.GetPkgDstPath(), constants.DefaultFilePath),
+ Hooks: shell,
+ }
+
+ return ExecuteHooks(hookConf)
+}
+
func HookSchedule(ccfg *api.ClusterConfig, nodes []*api.HostConfig, role []uint16, schedule api.ScheduleType) error {
for _, n := range nodes {
for _, r := range role {
@@ -383,7 +427,7 @@ func HookSchedule(ccfg *api.ClusterConfig, nodes []*api.HostConfig, role []uint1
continue
}
- if err := ExecuteShell(ccfg.RoleInfra[r], ccfg.PackageSrc.GetPkgDstPath(), n, schedule); err != nil {
+ if err := executeShell(ccfg, r, n, schedule); err != nil {
if api.IsCleanupSchedule(schedule) {
logrus.Errorf("execute shell failed for %s at %s: %v", n.Address, string(schedule), err)
} else {
diff --git a/pkg/utils/runner/runner.go b/pkg/utils/runner/runner.go
index c7088df..83a81e9 100644
--- a/pkg/utils/runner/runner.go
+++ b/pkg/utils/runner/runner.go
@@ -30,7 +30,6 @@ import (
"github.com/kubesphere/kubekey/pkg/util/ssh"
"github.com/sirupsen/logrus"
"isula.org/eggo/pkg/api"
- "isula.org/eggo/pkg/constants"
)
const (
@@ -164,7 +163,7 @@ func (ssh *SSHRunner) Reconnect() error {
func clearUserTempDir(conn ssh.Connection, host *kkv1alpha1.HostCfg) {
tmpShell := "/tmp/" + RunnerShellPrefix + "*"
// scp to tmp file
- dir := getCopyDefaultDir(host.User)
+ dir := api.GetUserTempDir(host.User)
_, err := conn.Exec(fmt.Sprintf("sudo -E /bin/sh -c \"rm -rf %s; rm -rf %s\"", dir, tmpShell), host)
if err != nil {
logrus.Warnf("[%s] remove temp dir: %s failed: %v", host.Name, dir, err)
@@ -175,7 +174,7 @@ func clearUserTempDir(conn ssh.Connection, host *kkv1alpha1.HostCfg) {
func prepareUserTempDir(conn ssh.Connection, host *kkv1alpha1.HostCfg) error {
// scp to tmp file
- dir := getCopyDefaultDir(host.User)
+ dir := api.GetUserTempDir(host.User)
var sb strings.Builder
sb.WriteString("sudo -E /bin/sh -c \"")
sb.WriteString(fmt.Sprintf("mkdir -p %s", dir))
@@ -191,18 +190,11 @@ func prepareUserTempDir(conn ssh.Connection, host *kkv1alpha1.HostCfg) error {
return nil
}
-func getCopyDefaultDir(user string) string {
- if user == "root" {
- return constants.DefaultRootCopyTempDirHome + "/temp"
- }
- return fmt.Sprintf(constants.DefaultUserCopyTempHomeFormat, user) + "/temp"
-}
-
func (ssh *SSHRunner) copyFile(src, dst string) error {
if ssh.Conn == nil {
return fmt.Errorf("[%s] SSH runner is not connected", ssh.Host.Name)
}
- tempDir := getCopyDefaultDir(ssh.Host.User)
+ tempDir := api.GetUserTempDir(ssh.Host.User)
// scp to tmp file
tempCpyFile := filepath.Join(tempDir, filepath.Base(src))
err := ssh.Conn.Scp(src, tempCpyFile)
@@ -248,7 +240,7 @@ func (ssh *SSHRunner) copyDir(srcDir, dstDir string) error {
logrus.Errorf("[%s] create cert tmp tar failed: %v", ssh.Host.Name, err)
return err
}
- tmpCpyDir := getCopyDefaultDir(ssh.Host.User)
+ tmpCpyDir := api.GetUserTempDir(ssh.Host.User)
tmpPkiFile := filepath.Join(tmpCpyDir, "pkg.tar")
// scp to user home directory
err = ssh.Copy(tmpPkgFile, tmpPkiFile)
--
2.25.1

36
0012-add-deploy-problems-doc.patch Normal file
View File

@ -0,0 +1,36 @@
From 11a733cc8b9eb2c5ad58f14e75d5a88109d9f2ee Mon Sep 17 00:00:00 2001
From: zhangxiaoyu <zhangxiaoyu58@huawei.com>
Date: Thu, 25 Nov 2021 09:49:49 +0800
Subject: [PATCH 12/12] add deploy problems doc
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
---
docs/deploy_problems.md | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/docs/deploy_problems.md b/docs/deploy_problems.md
index a26418a..6f259c9 100644
--- a/docs/deploy_problems.md
+++ b/docs/deploy_problems.md
@@ -78,4 +78,16 @@ kubelet启动不了，详细报错可以参考[issue](https://gitee.com/openeule
$ wget https://curl.se/ca/cacert.pem
$ mv cacert.pem /etc/pki/ca-trust/source/anchors/
$ update-ca-trust
-```
\ No newline at end of file
+```
+
+### 证书校验失败
+
+在WaitNodeRegister阶段，报错Get "https://10.253.173.8:6443/api/v1/nodes/10.253.173.8": x509: certificate has expired or is not yet valid: current time 2021-11-24T17:12:19Z is before 2021-11-24T23:54:02Z
+
+#### 原因
+
+机器时间未同步，生成证书的节点时间晚于worker节点的时间，导致证书校验失败
+
+#### 解决方法
+
+同步机器时间
--
2.25.1

27
eggo.spec
View File

@ -1,11 +1,24 @@
Name: eggo
Version: 0.9.4
Release: 1
Release: 2
Summary: Eggo is a tool built to provide standard multi-ways for creating Kubernetes clusters.
License: Mulan PSL V2
URL: https://gitee.com/openeuler/eggo
Source0: https://gitee.com/openeuler/eggo/repository/archive/v%{version}.tar.gz
Patch0001: 0001-correct-docs-error.patch
Patch0002: 0002-disable-service-beforce-setup-service.patch
Patch0003: 0003-support-disable-rollback-for-deploy-cluster.patch
Patch0004: 0004-add-host-name-checker-add-bin-execute-permission.patch
Patch0005: 0005-wait-task-longer-and-modify-eggops-docs.patch
Patch0006: 0006-fix-eggo-deploy-bugs.patch
Patch0007: 0007-non-root-user-deploy-bug-fixed.patch
Patch0008: 0008-fix-post-join-hook-do-not-run-bug.patch
Patch0009: 0009-add-spec-of-eggo-hook.patch
Patch0010: 0010-update-roadmap-of-eggo.patch
Patch0011: 0011-run-hook-with-envs-of-cluster-info.patch
Patch0012: 0012-add-deploy-problems-doc.patch
BuildRequires: make
BuildRequires: git
BuildRequires: golang >= 1.13
@ -45,11 +58,17 @@ rm -rf src
%attr(551,root,root) %{_bindir}/eggo
%changelog
* Fri Nov 26 2021 zhangxiaoyu<zhangxiaoyu58@huawei.com> - 0.9.4-2
- Type:upgrade
- CVE:NA
- SUG:NA
- DESC:upgrade to v0.9.4-2
* Mon Sep 13 2021 zhangxiaoyu<zhangxiaoyu58@huawei.com> - 0.9.4-1
- Type:upgrade
- CVE:NA
- SUG:NA
- DESC:upgrage to v0.9.4-1
- DESC:upgrade to v0.9.4-1
* Mon Sep 13 2021 zhangxiaoyu<zhangxiaoyu58@huawei.com> - 0.9.3-4
- Type:bugfix
@ -73,13 +92,13 @@ rm -rf src
- Type:upgrade
- CVE:NA
- SUG:NA
- DESC:upgrage to v0.9.3-1
- DESC:upgrade to v0.9.3-1
* Mon Aug 02 2021 zhangxiaoyu<zhangxiaoyu58@huawei.com> - 0.9.1-1
- Type:upgrade
- CVE:NA
- SUG:NA
- DESC:upgrage to v0.9.1-1
- DESC:upgrade to v0.9.1-1
* Tue Jul 20 2021 zhangxiaoyu<zhangxiaoyu58@huawei.com> - 0.9.1-20210712.150722.gitddf3d38e
- Type:upgrade