packages/docker
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docker: sync bugfix

Browse Source 
Change-Id: I4dc92059d90415199fcd143d75cc68cfdb67c430
Signed-off-by: jingrui <jingrui@huawei.com>
This commit is contained in:
jingrui 2021-01-19 10:15:58 +08:00
parent baf4fbe362
commit af6293703d
23 changed files with 841 additions and 49 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
VERSION-openeuler
View File

@ -1 +1 @@
18.09.0.105
18.09.0.112

55
docker.spec
View File

@ -1,6 +1,6 @@
Name: docker-engine
Version: 18.09.0
Release: 111
Release: 112
Summary: The open-source application container engine
Group: Tools/Docker
@ -210,30 +210,57 @@ fi
%endif
%changelog
* Mon Jan 4 2021 yangyanchao<yangyanchao6@huawei.com> - 18.09.111
* Mon Jan 4 2021 yangyanchao<yangyanchao6@huawei.com> - 18.09.0-111
- Type:requirement
- ID:NA
- CVE:NA
- SUG:restart
- docker:components:add config files for riscv
* Mon Dec 21 2020 fengshaobao<shaobao.feng@huawei.com> - 18.09.110
* Mon Jan 18 2021 jingrui<jingrui@huawei.com> - 18.09.0-107
- Type:bugfix
- ID:NA
- CVE:NA
- SUG:restart
- DESC: append the image hostname itself as an endpoint even the registry mirror matched.
- SUG:NA
- DESC:sync bugfix include
1. fix image cleanup failed.
2. cleanup load tmp files.
3. kill residual container process.
4. resume suspend dm device.
5. dont kill containerd during dockerd starting.
6. handle exit event for restore failed container.
7. wait io with timeout when start failed.
8. support hostname mirror registry.
9. mask unused proc files.
* Fri Nov 27 2020 liuzekun<liuzekun@huawei.com> - 18.09.109
* Tue Dec 8 2020 xiadanni<xiadanni1@huawei.com> - 18.09.0-104
- Type:bugfix
- ID:NA
- SUG:NA
- DESC:runc don't deny all devices when update cgroup resource
* Thu Dec 3 2020 xiadanni<xiadanni1@huawei.com> - 18.09.0-103
- Type:bugfix
- ID:NA
- SUG:restart
- DESC:containerd fix CVE-2020-15257
* Fri Nov 27 2020 liuzekun<liuzekun@huawei.com> - 18.09.0-102
- Type:bugfix
- ID:NA
- CVE:NA
- SUG:restart
- DESC:
1.add more messages for ops when device not found
2.do not add "-w" to LDFLAG
3.add files in proc for mask
4.fix docker load files leak
5.do not sync if BYPAAS_SYNC is false
6.fix panic on single-character volumes
7.mask /proc/pin_memory
1.delete stale containerd object on start failure
2.remove redundant word item
3.delete event is not need to process
4.stat process exit file when kill process dire
5.sync cli vendor
6.fix CVE-2020-13401
7.do not add w to LDFLAGS
8.add files in proc for mask
9.fix docker load files leak
10.do not sync if BYPAAS_SYNC is false
11.fix panic on single character volumes
12.fix stats memory usage display error
13.add more messages for ops when device not found
14.mask proc pin_memory

28
gen-commit.sh
View File

@ -1,12 +1,28 @@
#!/bin/sh
#!/bin/bash
# Copyright (c) Huawei Technologies Co., Ltd. 2020. All rights reserved.
# Description: This shell script is used to generate commitID store file.
# Author: xiadanni1@huawei.com
# Create: 2020-06-09
# Copyright (c) Huawei Technologies Co., Ltd. 2018-2019. All rights reserved.
# Description: This script uses to update docker-ce component's version and release
# Author: caihaomin@huawei.com
# Create: 2018-10-25
changeID=`git log -1 | grep Change-Id | awk '{print $2}'`
if [ "${changeID}" = "" ];then
changeID=`date | sha256sum | head -c 40`
fi
echo "${changeID}" > git-commit
echo "${changeID}" > git-commit
old_version=`head -n 10 docker.spec|grep Release|awk '{print $2}'`
let new_version=$old_version+1
sed -i -e "s/^\Release: $old_version/Release: $new_version/g" ./*.spec
echo 18.09.0.$new_version > VERSION-openeuler
author=$(git config user.name)
email=$(git config user.email)
version=$(head -1 docker.spec | awk '{print $NF}')
release=$(head -10 docker.spec | grep Release | awk '{print $2}' | awk -F% '{print $1}')
new_all=$version-$release
new_changelog=$(cat << EOF
* $(LC_ALL="C" date '+%a %b %d %Y') $author<$email> - $new_all\n- Type:\n- CVE:\n- SUG:\n- DESC:\n
EOF
)
sed -i -e "/\%changelog/a$new_changelog" *.spec

2
git-commit
View File

@ -1 +1 @@
026550e43812d1fc2fcea9cc172e35a4b7f1a848
I4dc92059d90415199fcd143d75cc68cfdb67c430

24
patch/0161-docker-Delete-stale-containerd-object-on-start-f.patch → patch/0161-docker-Delete-stale-containerd-object-on-start-failure.patch
View File

@ -1,32 +1,30 @@
From 6fe2bd73981651b275e508dd2c4806b20853684b Mon Sep 17 00:00:00 2001
From: Grooooot <isula@huawei.com>
Date: Tue, 17 Mar 2020 10:34:59 +0800
Subject: [PATCH] docker: Delete stale containerd object on start failure
From 5ba30cd1dc6000ee53b34f628cbff91d7f6d7231 Mon Sep 17 00:00:00 2001
From: Brian Goff <cpuguy83@gmail.com>
Date: Wed, 12 Dec 2018 12:04:09 -0800
Subject: [PATCH] Delete stale containerd object on start failure
containerd has two objects with regard to containers.
This is a "container" object which is metadata and a "task" which is
There is a "container" object which is metadata and a "task" which is
manging the actual runtime state.
When docker starts a container, it creates both the container metadata
and the task at the same time. So when a container exists, docker deletes
When docker starts a container, it creartes both the container metadata
and the task at the same time. So when a container exits, docker deletes
both of these objects as well.
This ensures that if, on start, when we go to create the container metadata object
in containerd, if there is an error due to a name conflict taht we go
in containerd, if there is an error due to a name conflict that we go
ahead and clean that up and try again.
backport from upstream: https://github.com/moby/moby/pull/38364
Signed-off-by: Grooooot <isula@huawei.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
---
components/engine/daemon/start.go | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/components/engine/daemon/start.go b/components/engine/daemon/start.go
index 8ff636b..07bffaa 100644
index 393e00b..57a7267 100644
--- a/components/engine/daemon/start.go
+++ b/components/engine/daemon/start.go
@@ -185,9 +185,22 @@ func (daemon *Daemon) containerStart(container *container.Container, checkpoint
@@ -177,9 +177,22 @@ func (daemon *Daemon) containerStart(container *container.Container, checkpoint
return err
}

25
patch/0162-docker-remove-redundant-word-item.patch Normal file
View File

@ -0,0 +1,25 @@
From 92266f008637a02ebffa2aa2704a09701b07a405 Mon Sep 17 00:00:00 2001
From: liuzekun <liuzekun@huawei.com>
Date: Mon, 27 Apr 2020 09:43:21 +0800
Subject: [PATCH] docker: remove redundant word item
Signed-off-by: liuzekun <liuzekun@huawei.com>
---
components/cli/vendor/github.com/asaskevich/govalidator/types.go | 1 -
1 file changed, 1 deletion(-)
diff --git a/components/cli/vendor/github.com/asaskevich/govalidator/types.go b/components/cli/vendor/github.com/asaskevich/govalidator/types.go
index 4f7e9274..78be68c6 100644
--- a/components/cli/vendor/github.com/asaskevich/govalidator/types.go
+++ b/components/cli/vendor/github.com/asaskevich/govalidator/types.go
@@ -370,7 +370,6 @@ var ISO3166List = []ISO3166Entry{
{"Spain", "Espagne (l')", "ES", "ESP", "724"},
{"South Sudan", "Soudan du Sud (le)", "SS", "SSD", "728"},
{"Sudan (the)", "Soudan (le)", "SD", "SDN", "729"},
- {"Western Sahara*", "Sahara occidental (le)*", "EH", "ESH", "732"},
{"Suriname", "Suriname (le)", "SR", "SUR", "740"},
{"Svalbard and Jan Mayen", "Svalbard et l'Île Jan Mayen (le)", "SJ", "SJM", "744"},
{"Swaziland", "Swaziland (le)", "SZ", "SWZ", "748"},
--
2.19.1

0
patch/0162-docker-delete-event-is-not-need-to-process.patch → patch/0163-docker-delete-event-is-not-need-to-process.patch
View File

0
patch/0163-docker-stat-process-exit-file-when-kill-process-dire.patch → patch/0164-docker-stat-process-exit-file-when-kill-process-dire.patch
View File

69
patch/0167-docker-fix-CVE-2020-13401.patch Normal file
View File

@ -0,0 +1,69 @@
From 727ce265564d1dc3031221a84f95abad20a20f11 Mon Sep 17 00:00:00 2001
From: jingrui <jingrui@huawei.com>
Date: Thu, 11 Jun 2020 21:55:49 +0800
Subject: [PATCH] docker: fix CVE-2020-13401
Change-Id: I267bde21d88927a0beb7599651b856a2dd1371d3
Signed-off-by: jingrui <jingrui@huawei.com>
---
.../libnetwork/drivers/bridge/bridge.go | 6 ++++++
.../libnetwork/drivers/bridge/setup_device.go | 19 +++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/components/engine/vendor/github.com/docker/libnetwork/drivers/bridge/bridge.go b/components/engine/vendor/github.com/docker/libnetwork/drivers/bridge/bridge.go
index 535da3c1ad..3288ff8652 100644
--- a/components/engine/vendor/github.com/docker/libnetwork/drivers/bridge/bridge.go
+++ b/components/engine/vendor/github.com/docker/libnetwork/drivers/bridge/bridge.go
@@ -679,6 +679,12 @@ func (d *driver) createNetwork(config *networkConfiguration) (err error) {
bridgeAlreadyExists := bridgeIface.exists()
if !bridgeAlreadyExists {
bridgeSetup.queueStep(setupDevice)
+ bridgeSetup.queueStep(setupDefaultSysctl)
+ }
+
+ // For the default bridge, set expected sysctls
+ if config.DefaultBridge {
+ bridgeSetup.queueStep(setupDefaultSysctl)
}
// Even if a bridge exists try to setup IPv4.
diff --git a/components/engine/vendor/github.com/docker/libnetwork/drivers/bridge/setup_device.go b/components/engine/vendor/github.com/docker/libnetwork/drivers/bridge/setup_device.go
index a9dfd06771..9822236dfd 100644
--- a/components/engine/vendor/github.com/docker/libnetwork/drivers/bridge/setup_device.go
+++ b/components/engine/vendor/github.com/docker/libnetwork/drivers/bridge/setup_device.go
@@ -2,6 +2,9 @@ package bridge
import (
"fmt"
+ "io/ioutil"
+ "os"
+ "path/filepath"
"github.com/docker/docker/pkg/parsers/kernel"
"github.com/docker/libnetwork/netutils"
@@ -50,6 +53,22 @@ func setupDevice(config *networkConfiguration, i *bridgeInterface) error {
return err
}
+func setupDefaultSysctl(config *networkConfiguration, i *bridgeInterface) error {
+ // Disable IPv6 router advertisements originating on the bridge
+ sysPath := filepath.Join("/proc/sys/net/ipv6/conf/", config.BridgeName, "accept_ra")
+ if _, err := os.Stat(sysPath); err != nil {
+ logrus.
+ WithField("bridge", config.BridgeName).
+ WithField("syspath", sysPath).
+ Info("failed to read ipv6 net.ipv6.conf.<bridge>.accept_ra")
+ return nil
+ }
+ if err := ioutil.WriteFile(sysPath, []byte{'0', '\n'}, 0644); err != nil {
+ return fmt.Errorf("libnetwork: Unable to disable IPv6 router advertisement: %v", err)
+ }
+ return nil
+}
+
// SetupDeviceUp ups the given bridge interface.
func setupDeviceUp(config *networkConfiguration, i *bridgeInterface) error {
err := i.nlh.LinkSetUp(i.Link)
--
2.17.1

0
patch/0167-docker-add-more-messages-for-ops-when-device-not-fo.patch → patch/0167-dockerd-add-more-messages-for-ops-when-device-not-fo.patch
View File

38
patch/0173-docker-fix-stats-memory-usage-display-error.patch Normal file
View File

@ -0,0 +1,38 @@
From d26341e4c447ddbb6bd289845b7b47f0e4348c62 Mon Sep 17 00:00:00 2001
From: xiadanni1 <xiadanni1@huawei.com>
Date: Wed, 11 Nov 2020 17:35:06 +0800
Subject: [PATCH] docker:fix stats memory usage display error
fix stats memory usage display error
use total_inactive_file not cache to calculate memory usage
The new stat definition corresponds to containerd/CRI and cadvisor.
https://github.com/containerd/cri/blob/c1115d4e57f55a5f45fb3efd29d3181ce26d5c6a/pkg/server/container_stats_list_unix.go#L106-L129
https://github.com/google/cadvisor/commit/307d1b1cb320fef66fab02db749f07a459245451
Signed-off-by: xiadanni1 <xiadanni1@huawei.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
---
components/cli/cli/command/container/stats_helpers.go | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/components/cli/cli/command/container/stats_helpers.go b/components/cli/cli/command/container/stats_helpers.go
index 2300ce5..c254212 100644
--- a/components/cli/cli/command/container/stats_helpers.go
+++ b/components/cli/cli/command/container/stats_helpers.go
@@ -226,7 +226,11 @@ func calculateNetwork(network map[string]types.NetworkStats) (float64, float64)
// calculateMemUsageUnixNoCache calculate memory usage of the container.
// Page cache is intentionally excluded to avoid misinterpretation of the output.
func calculateMemUsageUnixNoCache(mem types.MemoryStats) float64 {
- return float64(mem.Usage - mem.Stats["cache"])
+ if v, isCgroup1 := mem.Stats["total_inactive_file"]; isCgroup1 && v < mem.Usage {
+ return float64(mem.Usage - v)
+ }
+
+ return float64(mem.Usage)
}
func calculateMemPercentUnixNoCache(limit float64, usedNoCache float64) float64 {
--
1.8.3.1

85
patch/0175-docker-clean-docker-load-leak-files.patch Normal file
View File

@ -0,0 +1,85 @@
From a74f1c3e4ab7c6f4a043904a8e68edf04864d98a Mon Sep 17 00:00:00 2001
From: jingrui <jingrui@huawei.com>
Date: Wed, 2 Dec 2020 17:20:50 +0800
Subject: [PATCH] docker: clean docker load leak files
Change-Id: I09b66e204f655a9fef660bb85619f5711fb5700b
Signed-off-by: jingrui <jingrui@huawei.com>
---
components/engine/daemon/daemon.go | 39 +++++++++++++++++++
.../daemon/graphdriver/devmapper/deviceset.go | 3 +-
2 files changed, 41 insertions(+), 1 deletion(-)
diff --git a/components/engine/daemon/daemon.go b/components/engine/daemon/daemon.go
index 3ff5691257..1acd355a15 100644
--- a/components/engine/daemon/daemon.go
+++ b/components/engine/daemon/daemon.go
@@ -613,11 +613,50 @@ func (daemon *Daemon) restore() error {
group.Wait()
+ daemon.cleanExit()
logrus.Info("Loading containers: done.")
return nil
}
+func (daemon *Daemon) cleanExit() {
+ mnt := filepath.Join(daemon.root, "devicemapper/mnt")
+ if dir, err := ioutil.ReadDir(mnt); err == nil {
+ for _, f := range dir {
+ fname := filepath.Join(mnt, f.Name())
+ data, err := ioutil.ReadFile(fname)
+ if err != nil {
+ continue
+ }
+ if string(data) == "exit" {
+ logrus.Infof("cleanExit remove mnt %s", fname)
+ os.Remove(fname)
+ }
+ }
+ }
+
+ tmp := filepath.Join(daemon.root, "image/devicemapper/layerdb/tmp")
+ if dir, err := ioutil.ReadDir(tmp); err == nil {
+ for _, f := range dir {
+ if strings.Contains(f.Name(), "write-set-") {
+ fname := filepath.Join(tmp, f.Name())
+ logrus.Infof("cleanExit remove layerdb %s", fname)
+ os.RemoveAll(fname)
+ }
+ }
+ }
+
+ if dir, err := ioutil.ReadDir(os.Getenv("TMPDIR")); err == nil {
+ for _, f := range dir {
+ if strings.Contains(f.Name(), "docker-import-") {
+ fname := filepath.Join(os.Getenv("TMPDIR"), f.Name())
+ logrus.Infof("cleanExit remove tmpdir %s", fname)
+ os.RemoveAll(fname)
+ }
+ }
+ }
+}
+
// RestartSwarmContainers restarts any autostart container which has a
// swarm endpoint.
func (daemon *Daemon) RestartSwarmContainers() {
diff --git a/components/engine/daemon/graphdriver/devmapper/deviceset.go b/components/engine/daemon/graphdriver/devmapper/deviceset.go
index ff90c44ce3..750f2b13f8 100644
--- a/components/engine/daemon/graphdriver/devmapper/deviceset.go
+++ b/components/engine/daemon/graphdriver/devmapper/deviceset.go
@@ -2286,8 +2286,9 @@ func (devices *DeviceSet) unmountAndDeactivateAll(dir string) {
if err := unix.Unmount(fullname, unix.MNT_DETACH); err != nil && err != unix.EINVAL {
logger.Warnf("Shutdown unmounting %s, error: %s", fullname, err)
} else if err == nil {
- logger.Debugf("Remove %s", fullname)
+ logger.Infof("cleanExit prepare %s", fullname)
os.RemoveAll(fullname)
+ ioutil.WriteFile(fullname, []byte("exit"), 0600)
}
if devInfo, err := devices.lookupDevice(name); err != nil {
--
2.17.1

0
patch/0173-docker-mask-proc-pin_memory.patch → patch/0175-docker-mask-proc-pin_memory.patch
View File

62
patch/0176-docker-kill-container-process-if-its-status-is-not-r.patch Normal file
View File

@ -0,0 +1,62 @@
From 544d24895836ec576febaf94be8affde56449fba Mon Sep 17 00:00:00 2001
From: xiadanni1 <xiadanni1@huawei.com>
Date: Fri, 27 Nov 2020 16:31:56 +0800
Subject: [PATCH] docker: kill container process if its status is not running
when start daemon
Signed-off-by: xiadanni1 <xiadanni1@huawei.com>
---
components/engine/daemon/daemon.go | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/components/engine/daemon/daemon.go b/components/engine/daemon/daemon.go
index 3ff5691..3cc2a20 100644
--- a/components/engine/daemon/daemon.go
+++ b/components/engine/daemon/daemon.go
@@ -17,8 +17,10 @@ import (
"runtime"
"strings"
"sync"
+ "syscall"
"time"
+ "golang.org/x/sys/unix"
"google.golang.org/grpc"
"github.com/containerd/containerd"
@@ -43,6 +45,7 @@ import (
"github.com/moby/buildkit/util/resolver"
"github.com/moby/buildkit/util/tracing"
"github.com/sirupsen/logrus"
+
// register graph drivers
_ "github.com/docker/docker/daemon/graphdriver/register"
"github.com/docker/docker/daemon/stats"
@@ -51,7 +54,7 @@ import (
"github.com/docker/docker/image"
"github.com/docker/docker/layer"
"github.com/docker/docker/libcontainerd"
- "github.com/docker/docker/migrate/v1"
+ v1 "github.com/docker/docker/migrate/v1"
"github.com/docker/docker/pkg/idtools"
"github.com/docker/docker/pkg/locker"
"github.com/docker/docker/pkg/plugingetter"
@@ -389,6 +392,15 @@ func (daemon *Daemon) restore() error {
}
}
+ if alive && !c.IsRunning() && pid > 1 {
+ if c.Pid == 0 {
+ c.Pid = pid
+ }
+ err := unix.Kill(pid, syscall.SIGKILL)
+ logrus.Warnf("process %v is killed as container=%s is alive but not running, err: %v", pid, c.ID, err)
+ return
+ }
+
if c.IsRunning() || c.IsPaused() {
c.RestartManager().Cancel() // manually start containers because some need to wait for swarm networking
--
1.8.3.1

82
patch/0177-resume-suspend-dm-on-start.patch Normal file
View File

@ -0,0 +1,82 @@
From 37e3e3dfb31f30b2599d05f021671f6e682f37d6 Mon Sep 17 00:00:00 2001
From: jingrui <jingrui@huawei.com>
Date: Wed, 9 Dec 2020 17:37:02 +0800
Subject: [PATCH] resume suspend dm on start
Change-Id: Ibe215c80aa62b4d4b464749cc6e995d2e0e845af
Signed-off-by: jingrui <jingrui@huawei.com>
---
components/engine/cmd/dockerd/daemon.go | 43 +++++++++++++++++++++++++
1 file changed, 43 insertions(+)
diff --git a/components/engine/cmd/dockerd/daemon.go b/components/engine/cmd/dockerd/daemon.go
index 0b3fa0e037..dbf37f3338 100644
--- a/components/engine/cmd/dockerd/daemon.go
+++ b/components/engine/cmd/dockerd/daemon.go
@@ -6,6 +6,7 @@ import (
"fmt"
"io/ioutil"
"os"
+ "os/exec"
"path/filepath"
"runtime"
"strings"
@@ -72,6 +73,45 @@ func NewDaemonCli() *DaemonCli {
return &DaemonCli{}
}
+func resumeDM() {
+ c := make(chan struct{})
+ go func() {
+ defer close(c)
+ out, err := exec.Command("dmsetup", "info", "-c", "--sort", "minor", "--noheadings", "--separator", ",", "-o", "attr,name").CombinedOutput()
+ if err != nil {
+ logrus.Errorf("resume-dm dmsetup info failed: %v", err)
+ return
+ }
+
+ args := []string{"resume"}
+ for _, line := range strings.Split(string(out), "\n") {
+ aa := strings.Split(line, ",")
+ if len(aa) != 2 || !strings.Contains(aa[0], "s") || strings.Index(aa[1], "docker-") != 0 {
+ continue
+ }
+ args = append(args, aa[1])
+ }
+ if len(args) == 1 {
+ return
+ }
+
+ logrus.Infof("resume-dm start resume suspended dm %v", args)
+ _, err = exec.Command("dmsetup", args...).CombinedOutput()
+ if err != nil {
+ logrus.Errorf("resume-dm %s failed: %v", err)
+ return
+ }
+ logrus.Infof("resume-dm finished resume suspended dm")
+ }()
+ select {
+ case <-c:
+ return
+ case <-time.After(10*time.Second):
+ logrus.Warnf("resume-dm timeout, continue anyway.")
+ return
+ }
+}
+
func cleanupLocalDB(db string) {
_, err := os.Stat(db)
if err == nil {
@@ -150,6 +190,9 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
})
system.InitLCOW(cli.Config.Experimental)
+ if cli.Config.GraphDriver == "devicemapper" {
+ resumeDM()
+ }
if err := setDefaultUmask(); err != nil {
return fmt.Errorf("Failed to set umask: %v", err)
--
2.17.1

85
patch/0178-docker-skip-kill-and-restart-containerd-during-docke.patch Normal file
View File

@ -0,0 +1,85 @@
From a56def385f835885df056d0d54372111abdc1507 Mon Sep 17 00:00:00 2001
From: xiadanni <xiadanni1@huawei.com>
Date: Sat, 19 Dec 2020 18:56:38 +0800
Subject: [PATCH] docker:skip kill and restart containerd during docker daemon
is starting
Signed-off-by: xiadanni <xiadanni1@huawei.com>
---
components/engine/cmd/dockerd/daemon.go | 4 +++-
components/engine/libcontainerd/supervisor/remote_daemon.go | 9 +++++++++
2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/components/engine/cmd/dockerd/daemon.go b/components/engine/cmd/dockerd/daemon.go
index dbf37f3..c25ee0e 100644
--- a/components/engine/cmd/dockerd/daemon.go
+++ b/components/engine/cmd/dockerd/daemon.go
@@ -10,6 +10,7 @@ import (
"path/filepath"
"runtime"
"strings"
+ "sync/atomic"
"time"
containerddefaults "github.com/containerd/containerd/defaults"
@@ -106,7 +107,7 @@ func resumeDM() {
select {
case <-c:
return
- case <-time.After(10*time.Second):
+ case <-time.After(10 * time.Second):
logrus.Warnf("resume-dm timeout, continue anyway.")
return
}
@@ -304,6 +305,7 @@ func (cli *DaemonCli) start(opts *daemonOptions) (err error) {
logrus.Info("Daemon has completed initialization")
+ atomic.StoreInt32(&supervisor.IsDockerUp, 1)
cli.d = d
routerOptions, err := newRouterOptions(cli.Config, d)
diff --git a/components/engine/libcontainerd/supervisor/remote_daemon.go b/components/engine/libcontainerd/supervisor/remote_daemon.go
index 62ea58c..19582cd 100644
--- a/components/engine/libcontainerd/supervisor/remote_daemon.go
+++ b/components/engine/libcontainerd/supervisor/remote_daemon.go
@@ -11,6 +11,7 @@ import (
"strconv"
"strings"
"sync"
+ "sync/atomic"
"time"
"github.com/BurntSushi/toml"
@@ -19,6 +20,7 @@ import (
"github.com/docker/docker/pkg/system"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
+ "golang.org/x/sys/unix"
)
const (
@@ -31,6 +33,8 @@ const (
pidFile = "containerd.pid"
)
+var IsDockerUp int32
+
type pluginConfigs struct {
Plugins map[string]interface{} `toml:"plugins"`
}
@@ -314,6 +318,11 @@ func (r *remote) monitorDaemon(ctx context.Context) {
}
if system.IsProcessAlive(r.daemonPid) {
+ if atomic.LoadInt32(&IsDockerUp) == 0 {
+ r.logger.WithField("pid", r.daemonPid).Info("dockerd is starting, skip killing containerd")
+ unix.Kill(r.daemonPid, unix.SIGCONT)
+ continue
+ }
r.logger.WithField("pid", r.daemonPid).Info("killing and restarting containerd")
r.killDaemon()
}
--
1.8.3.1

107
patch/0179-handle-exit-force.patch Normal file
View File

@ -0,0 +1,107 @@
From 66b6e3065b160bd7d480f183156acbe1cb9bf2e0 Mon Sep 17 00:00:00 2001
From: jingrui <jingrui@huawei.com>
Date: Tue, 15 Dec 2020 16:05:56 +0800
Subject: [PATCH] handle exit force
Change-Id: If08483f57b4f04d6c4961c9f588e4d599009eddc
Signed-off-by: jingrui <jingrui@huawei.com>
---
components/engine/daemon/monitor.go | 9 +++++++++
components/engine/libcontainerd/client_daemon.go | 14 ++++++++++++++
components/engine/libcontainerd/types.go | 1 +
.../plugin/executor/containerd/containerd.go | 5 +++++
4 files changed, 29 insertions(+)
diff --git a/components/engine/daemon/monitor.go b/components/engine/daemon/monitor.go
index e041bd5c69..1b577c0dae 100644
--- a/components/engine/daemon/monitor.go
+++ b/components/engine/daemon/monitor.go
@@ -26,6 +26,14 @@ func (daemon *Daemon) setStateCounter(c *container.Container) {
}
}
+func (daemon *Daemon) IsContainerRunning(id string) bool {
+ c, err := daemon.GetContainer(id)
+ if err != nil {
+ return false
+ }
+ return c.IsRunning()
+}
+
// ProcessEvent is called by libcontainerd whenever an event occurs
func (daemon *Daemon) ProcessEvent(id string, e libcontainerd.EventType, ei libcontainerd.EventInfo) error {
c, err := daemon.GetContainer(id)
@@ -51,6 +59,7 @@ func (daemon *Daemon) ProcessEvent(id string, e libcontainerd.EventType, ei libc
case libcontainerd.EventExit:
if int(ei.Pid) == c.Pid {
c.Lock()
+ logrus.Infof("handle exit event cid=%s pid=%d", c.ID, c.Pid)
_, _, err := daemon.containerd.DeleteTask(context.Background(), c.ID)
if err != nil {
logrus.WithError(err).Warnf("failed to delete container %s from containerd", c.ID)
diff --git a/components/engine/libcontainerd/client_daemon.go b/components/engine/libcontainerd/client_daemon.go
index 05c439c540..502796bd25 100755
--- a/components/engine/libcontainerd/client_daemon.go
+++ b/components/engine/libcontainerd/client_daemon.go
@@ -517,9 +517,16 @@ func (c *client) DeleteTask(ctx context.Context, containerID string) (uint32, ti
return status.ExitCode(), status.ExitTime(), nil
}
+func (c *client) deleteForce(ctx context.Context, id string) {
+ if ctr, err := c.client.LoadContainer(ctx, id); err == nil {
+ logrus.Warnf("delete containerd meta id=%s force: error=%v", id, ctr.Delete(ctx))
+ }
+}
+
func (c *client) Delete(ctx context.Context, containerID string) error {
ctr := c.getContainer(containerID)
if ctr == nil {
+ c.deleteForce(ctx, containerID)
return errors.WithStack(newNotFoundError("no such container"))
}
@@ -907,6 +914,13 @@ func (c *client) processEventStream(ctx context.Context, ns string) {
ctr = c.getContainer(ei.ContainerID)
if ctr == nil {
c.logger.WithField("container", ei.ContainerID).Warn("unknown container")
+ if et == EventExit && ei.ProcessID == ei.ContainerID && c.backend.IsContainerRunning(ei.ContainerID) {
+ c.logger.WithField("container", ei.ContainerID).Warn("handle exit event force ...")
+ c.eventQ.append(ei.ContainerID, func() {
+ c.logger.WithField("container", ei.ContainerID).Warnf("handle exit event force: error=%v",
+ c.backend.ProcessEvent(ei.ContainerID, et, ei))
+ })
+ }
continue
}
diff --git a/components/engine/libcontainerd/types.go b/components/engine/libcontainerd/types.go
index c4de5e674d..0b9df9193b 100644
--- a/components/engine/libcontainerd/types.go
+++ b/components/engine/libcontainerd/types.go
@@ -60,6 +60,7 @@ type EventInfo struct {
// Backend defines callbacks that the client of the library needs to implement.
type Backend interface {
ProcessEvent(containerID string, event EventType, ei EventInfo) error
+ IsContainerRunning(id string) bool
}
// Client provides access to containerd features.
diff --git a/components/engine/plugin/executor/containerd/containerd.go b/components/engine/plugin/executor/containerd/containerd.go
index a3401dce79..f75771fe41 100644
--- a/components/engine/plugin/executor/containerd/containerd.go
+++ b/components/engine/plugin/executor/containerd/containerd.go
@@ -141,6 +141,11 @@ func (e *Executor) ProcessEvent(id string, et libcontainerd.EventType, ei libcon
return nil
}
+func (e *Executor) IsContainerRunning(id string) bool {
+ ok, _ := e.IsRunning(id)
+ return ok
+}
+
type rio struct {
cio.IO
--
2.17.1

47
patch/0180-wait-io-with-timeout-in-task-delete.patch Normal file
View File

@ -0,0 +1,47 @@
From 0f3aa35a1c38fe7fc49cd6fb66fc47a993ad6bb8 Mon Sep 17 00:00:00 2001
From: jingrui <jingrui@huawei.com>
Date: Wed, 16 Dec 2020 18:39:00 +0800
Subject: [PATCH] wait io with timeout in task delete
Change-Id: I23ed40d69279b14a216b6ffb9988439475be5cad
Signed-off-by: jingrui <jingrui@huawei.com>
---
.../github.com/containerd/containerd/task.go | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/components/engine/vendor/github.com/containerd/containerd/task.go b/components/engine/vendor/github.com/containerd/containerd/task.go
index 6806e11620..7421432bed 100644
--- a/components/engine/vendor/github.com/containerd/containerd/task.go
+++ b/components/engine/vendor/github.com/containerd/containerd/task.go
@@ -44,6 +44,7 @@ import (
"github.com/opencontainers/image-spec/specs-go/v1"
specs "github.com/opencontainers/runtime-spec/specs-go"
"github.com/pkg/errors"
+ "github.com/sirupsen/logrus"
)
// UnknownExitStatus is returned when containerd is unable to
@@ -287,8 +288,18 @@ func (t *task) Delete(ctx context.Context, opts ...ProcessDeleteOpts) (*ExitStat
return nil, errors.Wrapf(errdefs.ErrFailedPrecondition, "task must be stopped before deletion: %s", status.Status)
}
if t.io != nil {
- t.io.Cancel()
- t.io.Wait()
+ done := make(chan struct{})
+ go func() {
+ t.io.Cancel()
+ t.io.Wait()
+ close(done)
+ }()
+ select {
+ case <-time.After(3 * time.Second):
+ logrus.Warnf("task delete wait io close timeout, some fifo io may be dropped.")
+ case <-done:
+ // ok
+ }
}
r, err := t.client.TaskService().Delete(ctx, &tasks.DeleteTaskRequest{
ContainerID: t.id,
--
2.17.1

14
patch/0174-docker-do-not-return-when-matched-registry-mirror.patch → patch/0181-docker-do-not-return-when-matched-registry-mirror.patch
View File

@ -1,18 +1,16 @@
From 56c1d6c149b18214a8d01ab3f1738cae4792109a Mon Sep 17 00:00:00 2001
From: f00231050 <shaobao.feng@huawei.com>
From 8cc3f33020152d51d38927593ba49ad3dfacf62e Mon Sep 17 00:00:00 2001
From: shaobao.feng <shaobao.feng@huawei.com>
Date: Mon, 7 Dec 2020 15:30:11 +0800
Subject: [PATCH] docker: do not return when matched registry mirror
Change-Id: I5317b91b60293e1f4c50f5a327790c5509537f9b
reason: append hostname itself to make sure the hostname itself will be tried.
---
components/engine/registry/service_v2.go | 86 +++++++++++++++-----------------
components/engine/registry/service_v2.go | 86 +++++++++++-------------
1 file changed, 41 insertions(+), 45 deletions(-)
mode change 100644 => 100755 components/engine/registry/service_v2.go
diff --git a/components/engine/registry/service_v2.go b/components/engine/registry/service_v2.go
old mode 100644
new mode 100755
index adeb10c..df66cd7
index adeb10c550..df66cd7451 100644
--- a/components/engine/registry/service_v2.go
+++ b/components/engine/registry/service_v2.go
@@ -19,8 +19,7 @@ func (s *DefaultService) lookupV2Endpoints(hostname string) (endpoints []APIEndp
@ -135,5 +133,5 @@ index adeb10c..df66cd7
if tlsConfig.InsecureSkipVerify {
endpoints = append(endpoints, APIEndpoint{
--
1.8.3.1
2.17.1

29
patch/0183-add-masked-paths-pagealloc_module-and-slaballoc_stat.patch Normal file
View File

@ -0,0 +1,29 @@
From fada5f66fcc555d706603dd3c7832e78e9955501 Mon Sep 17 00:00:00 2001
From: liuzekun <liuzekun@huawei.com>
Date: Thu, 31 Dec 2020 03:07:42 -0500
Subject: add masked paths pagealloc_module and slaballoc_statistics
Signed-off-by: liuzekun <liuzekun@huawei.com>
---
components/engine/oci/defaults.go | 2 ++
1 file changed, 2 insertions(+)
diff --git a/components/engine/oci/defaults.go b/components/engine/oci/defaults.go
index e763cb75..ff027d89 100644
--- a/components/engine/oci/defaults.go
+++ b/components/engine/oci/defaults.go
@@ -135,9 +135,11 @@ func DefaultLinuxSpec() specs.Spec {
"/proc/oom_extend",
"/proc/pagealloc_statistics",
"/proc/pagealloc_bt",
+ "/proc/pagealloc_module",
"/proc/pin_memory",
"/proc/slaballoc_bt",
"/proc/slaballoc_module",
+ "/proc/slaballoc_statistics",
"/proc/sched_debug",
"/proc/scsi",
"/proc/sig_catch",
--
2.19.1

48
patch/0184-docker-wait-io-with-timeout-when-process-Start-faile.patch Normal file
View File

@ -0,0 +1,48 @@
From ef64f4dd5d532b550bb68f60e6373e139fdf5382 Mon Sep 17 00:00:00 2001
From: xiadanni <xiadanni1@huawei.com>
Date: Fri, 15 Jan 2021 11:23:04 +0800
Subject: [PATCH] docker: wait io with timeout when process Start failed
Signed-off-by: xiadanni <xiadanni1@huawei.com>
---
.../vendor/github.com/containerd/containerd/process.go | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/components/engine/vendor/github.com/containerd/containerd/process.go b/components/engine/vendor/github.com/containerd/containerd/process.go
index ff7d838..4d0dca9 100644
--- a/components/engine/vendor/github.com/containerd/containerd/process.go
+++ b/components/engine/vendor/github.com/containerd/containerd/process.go
@@ -26,6 +26,7 @@ import (
"github.com/containerd/containerd/cio"
"github.com/containerd/containerd/errdefs"
"github.com/pkg/errors"
+ "github.com/sirupsen/logrus"
)
// Process represents a system process
@@ -111,9 +112,19 @@ func (p *process) Start(ctx context.Context) error {
ExecID: p.id,
})
if err != nil {
- p.io.Cancel()
- p.io.Wait()
- p.io.Close()
+ done := make(chan struct{})
+ go func() {
+ p.io.Cancel()
+ p.io.Wait()
+ p.io.Close()
+ close(done)
+ }()
+ select {
+ case <-time.After(30 * time.Second):
+ logrus.Warnf("process start failed with error %v, wait io close timeout, some fifo io may be dropped.", err)
+ case <-done:
+ // ok
+ }
return errdefs.FromGRPC(err)
}
p.pid = r.Pid
--
1.8.3.1

63
patch/0185-docker-delete-image-reference-when-failed-to-get-ima.patch Normal file
View File

@ -0,0 +1,63 @@
From cfc92becb2605d67a7391c43261e698d0fdd57bd Mon Sep 17 00:00:00 2001
From: xiadanni <xiadanni1@huawei.com>
Date: Fri, 15 Jan 2021 15:37:42 +0800
Subject: [PATCH] docker: delete image reference when failed to get image
configuration to avoid docker pull error
according to patch 0110-docker-Fix-can-t-pull-image-while-the-image-i.patch,
if the layers of image has been damaged, image reference should be
deleted from repositories.json to avoid docker pull failed.
however, when imageStore.Get failed, isExist flag has not been set to
false, which cause the image reference has still not been deleted, only
warning is printed.
flood warnings printed every time user restarts docker daemon, like:
Jan 15 14:09:52 localhost dockerd[3952467]:
time="2021-01-15T14:09:52.705664179+08:00" level=warning msg="Failed to
get image configration for image id
sha256:d0a015ffac5ba3b9d2a641de56b3b2ed24409b7082c7811ebac4c2f4977b0965,
error: failed to get digest
sha256:d0a015ffac5ba3b9d2a641de56b3b2ed24409b7082c7811ebac4c2f4977b0965:
open
/var/lib/docker/image/devicemapper/imagedb/content/sha256/d0a015ffac5ba3b9d2a641de56b3b2ed24409b7082c7811ebac4c2f4977b0965:
no such file or directory"
so we fix the logic, delete image reference when failed to get image
configuration.
Signed-off-by: xiadanni <xiadanni1@huawei.com>
---
components/engine/daemon/daemon.go | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/components/engine/daemon/daemon.go b/components/engine/daemon/daemon.go
index e826f6a..ed268d2 100644
--- a/components/engine/daemon/daemon.go
+++ b/components/engine/daemon/daemon.go
@@ -1097,11 +1097,10 @@ func NewDaemon(ctx context.Context, config *config.Config, pluginStore *plugin.S
return nil, err
}
- // delete reference of image not nornamlly loaded to imageStore
- var isExist bool
+ // delete reference of image not normally loaded to imageStore
for _, imageID := range rs.List() {
+ isExist := false
if img, err := imageStore.Get(image.ID(imageID)); err == nil {
- isExist = false
if chainID := img.RootFS.ChainID(); chainID != "" {
l, err := layerStores[runtime.GOOS].Get(chainID)
if err == nil {
@@ -1112,7 +1111,7 @@ func NewDaemon(ctx context.Context, config *config.Config, pluginStore *plugin.S
isExist = true
}
} else {
- logrus.Warnf("Failed to get image configration for image id %s, error: %s", imageID, err)
+ logrus.Warnf("Failed to get image configuration for image id %s, error: %s", imageID, err)
}
// If the image not exist locally, delete its reference
--
1.8.3.1

25
series.conf
View File

@ -156,20 +156,33 @@ patch/0157-docker-Support-check-manifest-and-layer-s-DiffID-inf.patch
patch/0158-docker-support-private-registry.patch
patch/0159-docker-extend-timeout-in-cli-testcases.patch
patch/0160-docker-create-a-soft-link-from-runtime-default-to-ru.patch
patch/0161-docker-Delete-stale-containerd-object-on-start-f.patch
patch/0162-docker-delete-event-is-not-need-to-process.patch
patch/0163-docker-stat-process-exit-file-when-kill-process-dire.patch
patch/0161-docker-Delete-stale-containerd-object-on-start-failure.patch
patch/0162-docker-remove-redundant-word-item.patch
patch/0163-docker-delete-event-is-not-need-to-process.patch
patch/0164-docker-stat-process-exit-file-when-kill-process-dire.patch
patch/0164-docker-use-git-commit-to-store-commit-ID.patch
patch/0165-docker-sync-cli-vendor.patch
patch/0167-docker-add-more-messages-for-ops-when-device-not-fo.patch
patch/0167-docker-fix-CVE-2020-13401.patch
patch/0167-dockerd-add-more-messages-for-ops-when-device-not-fo.patch
patch/0168-docker-do-not-add-w-to-LDFLAGS.patch
patch/0169-docker-add-files-in-proc-for-mask.patch
patch/0170-docker-fix-docker-load-files-leak.patch
patch/0171-docker-do-not-sync-if-BYPAAS_SYNC-is-false.patch
patch/0172-docker-fix-panic-on-single-character-volumes.patch
patch/0173-docker-mask-proc-pin_memory.patch
patch/0174-docker-do-not-return-when-matched-registry-mirror.patch
patch/0173-docker-fix-stats-memory-usage-display-error.patch
patch/0175-docker-mask-proc-pin_memory.patch
patch/0175-docker-clean-docker-load-leak-files.patch
patch/0176-docker-kill-container-process-if-its-status-is-not-r.patch
patch/0177-resume-suspend-dm-on-start.patch
patch/0175-docker-components-engine-vendor-add-riscv64-config.patch
patch/0176-docker-components-engine-vendor-add-new-config-file-for-riscv.patch
patch/0177-docker-components-cli-vendor-add-new-config-file-for-riscv.patch
patch/0178-docker-skip-kill-and-restart-containerd-during-docke.patch
patch/0179-handle-exit-force.patch
patch/0180-wait-io-with-timeout-in-task-delete.patch
patch/0181-docker-do-not-return-when-matched-registry-mirror.patch
patch/0183-add-masked-paths-pagealloc_module-and-slaballoc_stat.patch
patch/0184-docker-wait-io-with-timeout-when-process-Start-faile.patch
patch/0185-docker-delete-image-reference-when-failed-to-get-ima.patch
#end