packages/iSulad
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

!120 sync from openeuler

Browse Source 
From: @duguhaotian
Reviewed-by: @lifeng2221dd1
Signed-off-by: @lifeng2221dd1
This commit is contained in:
openeuler-ci-bot 2021-03-23 10:12:09 +08:00 committed by Gitee
commit ac974aa6ca
54 changed files with 3725 additions and 27 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
0001-make-thread-detach-to-avoid-resource-leak.patch
View File

@ -1,7 +1,7 @@
From 1ef7a43907ac6fc521cedd2b4744be4d102efd32 Mon Sep 17 00:00:00 2001
From: WangFengTu <wangfengtu@huawei.com>
Date: Thu, 31 Dec 2020 14:05:25 +0800
Subject: [PATCH 1/9] make thread detach to avoid resource leak
Subject: [PATCH 01/53] make thread detach to avoid resource leak
Signed-off-by: WangFengTu <wangfengtu@huawei.com>
---

2
0002-devmapper-fix-udev-wait-thread-resource-leak.patch
View File

@ -1,7 +1,7 @@
From 025416aae9f7eaaa8fe5ad52ecbbf6692505186b Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Thu, 31 Dec 2020 14:31:12 +0800
Subject: [PATCH 2/9] devmapper: fix udev wait thread resource leak
Subject: [PATCH 02/53] devmapper: fix udev wait thread resource leak
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---

2
0003-clean-code-fix-clean-code.patch
View File

@ -1,7 +1,7 @@
From 200f49ff353ee8266505316659493ffc4082c803 Mon Sep 17 00:00:00 2001
From: lifeng68 <lifeng68@huawei.com>
Date: Tue, 5 Jan 2021 18:48:20 +0800
Subject: [PATCH 3/9] clean code: fix clean code
Subject: [PATCH 03/53] clean code: fix clean code
Signed-off-by: lifeng68 <lifeng68@huawei.com>
---

2
0004-judge-isula-load-file-exists.patch
View File

@ -1,7 +1,7 @@
From c0b6c4187a3c66bef8b75a63e699df1be57d05b4 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Mon, 11 Jan 2021 18:29:26 +0800
Subject: [PATCH 4/9] judge isula load file exists
Subject: [PATCH 04/53] judge isula load file exists
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---

2
0005-modify-image_load.sh-CI-to-test-file-not-exist.patch
View File

@ -1,7 +1,7 @@
From e151821a1e092995836631b273bddc339cadffbe Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Mon, 11 Jan 2021 18:33:39 +0800
Subject: [PATCH 5/9] modify image_load.sh CI to test file not exist
Subject: [PATCH 05/53] modify image_load.sh CI to test file not exist
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---

2
0006-do-not-pause-container-when-copy.patch
View File

@ -1,7 +1,7 @@
From b69da83db290057dde5dbe34e153fb0895e456e2 Mon Sep 17 00:00:00 2001
From: WangFengTu <wangfengtu@huawei.com>
Date: Tue, 29 Dec 2020 10:16:13 +0800
Subject: [PATCH 6/9] do not pause container when copy
Subject: [PATCH 06/53] do not pause container when copy
and use libarchive to do unpack/tar instead of
execute tar command. Once not pause container,

2
0007-add-testcases-for-isula-cp.patch
View File

@ -1,7 +1,7 @@
From 085b93daf8f080f21b304058da3af404be9ac61d Mon Sep 17 00:00:00 2001
From: WangFengTu <wangfengtu@huawei.com>
Date: Fri, 8 Jan 2021 14:02:00 +0800
Subject: [PATCH 7/9] add testcases for isula cp
Subject: [PATCH 07/53] add testcases for isula cp
Signed-off-by: WangFengTu <wangfengtu@huawei.com>
---

2
0008-image_cb-rename-the-function-isula_-docker_-to-do_.patch
View File

@ -1,7 +1,7 @@
From c8d14980e145a7d400aa6c5b449a59952a422801 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Fri, 15 Jan 2021 10:34:43 +0800
Subject: [PATCH 8/9] image_cb: rename the function {isula_/docker_} to do_
Subject: [PATCH 08/53] image_cb: rename the function {isula_/docker_} to do_
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---

2
0009-fix-small-probability-of-coredump-in-CRI-streaming-s.patch
View File

@ -1,7 +1,7 @@
From 0295f347d6394294cb2c81741ece78548d4cafc6 Mon Sep 17 00:00:00 2001
From: wujing <jing.woo@outlook.com>
Date: Thu, 14 Jan 2021 10:53:07 +0800
Subject: [PATCH 9/9] fix small probability of coredump in CRI streaming
Subject: [PATCH 09/53] fix small probability of coredump in CRI streaming
services in high concurrency scenarios
Signed-off-by: wujing <wujing50@huawei.com>

2
0010-fix-ramdom-coredump-if-pull-failed.patch
View File

@ -1,7 +1,7 @@
From 7e04901403a0053f67eae6c9bb58764b529c0bd8 Mon Sep 17 00:00:00 2001
From: WangFengTu <wangfengtu@huawei.com>
Date: Tue, 19 Jan 2021 16:57:17 +0800
Subject: [PATCH 10/26] fix ramdom coredump if pull failed
Subject: [PATCH 10/53] fix ramdom coredump if pull failed
Signed-off-by: WangFengTu <wangfengtu@huawei.com>
---

2
0011-shim-optimize-io-stream.patch
View File

@ -1,7 +1,7 @@
From bbf3f17765483e2e87e96e975c1d85bb5250c8f2 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Wed, 20 Jan 2021 10:13:14 +0800
Subject: [PATCH 11/26] shim: optimize io stream
Subject: [PATCH 11/53] shim: optimize io stream
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---

2
0012-add-CI-to-test-shim-io.patch
View File

@ -1,7 +1,7 @@
From f3f4c25792721bc130aec31deea9473d5283dfc6 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Wed, 20 Jan 2021 10:13:35 +0800
Subject: [PATCH 12/26] add CI to test shim io
Subject: [PATCH 12/53] add CI to test shim io
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---

2
0013-CI-add-testcase-for-exec-without-pty.patch
View File

@ -1,7 +1,7 @@
From 0eedc0354deb5616fe7e3308547d475af01d7cc3 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Wed, 20 Jan 2021 14:50:43 +0800
Subject: [PATCH 13/26] CI: add testcase for exec without pty
Subject: [PATCH 13/53] CI: add testcase for exec without pty
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---

2
0014-adapt-for-sparse-file-when-tar-file.patch
View File

@ -1,7 +1,7 @@
From 7e9b7b16c76785c15fd1465d7985a0919848f786 Mon Sep 17 00:00:00 2001
From: WangFengTu <wangfengtu@huawei.com>
Date: Thu, 21 Jan 2021 18:44:52 +0800
Subject: [PATCH 14/26] adapt for sparse file when tar file
Subject: [PATCH 14/53] adapt for sparse file when tar file
archive_read_data_block can not process sparse file
correctly, use archive_read_data instead.

2
0015-driver-do-not-unlock-and-destroy-lock-when-clean-up.patch
View File

@ -1,7 +1,7 @@
From bba60af5e275a24ab6ae11943ce48ff71524c494 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Mon, 25 Jan 2021 11:31:54 +0800
Subject: [PATCH 15/26] driver: do not unlock and destroy lock when clean up
Subject: [PATCH 15/53] driver: do not unlock and destroy lock when clean up
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---

2
0016-driver-do-not-set-g_graphdriver-to-NULL.patch
View File

@ -1,7 +1,7 @@
From d1fbada9a7b520830d8a0c31263aadba97b2dd9d Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Mon, 25 Jan 2021 15:01:35 +0800
Subject: [PATCH 16/26] driver: do not set g_graphdriver to NULL
Subject: [PATCH 16/53] driver: do not set g_graphdriver to NULL
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---

2
0017-ignore-error-if-get-ip-failed.patch
View File

@ -1,7 +1,7 @@
From 4f2951681dbe583e80af91d808292aad8cceb599 Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Thu, 28 Jan 2021 14:04:54 +0800
Subject: [PATCH 17/26] ignore error if get ip failed
Subject: [PATCH 17/53] ignore error if get ip failed
Signed-off-by: haozi007 <liuhao27@huawei.com>
---

2
0018-GC-add-log-container-info-when-add-into-gc.patch
View File

@ -1,7 +1,7 @@
From 72e8e3163524455768986a7496ccfc5ce384fade Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Fri, 29 Jan 2021 15:33:56 +0800
Subject: [PATCH 18/26] GC: add log container info when add into gc
Subject: [PATCH 18/53] GC: add log container info when add into gc
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---

2
0019-log-use-the-same-function-to-init-log-in-export-paus.patch
View File

@ -1,7 +1,7 @@
From 171cb932bbbbfc5816ceb65223f1d5e733c79d8e Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Sat, 30 Jan 2021 10:38:11 +0800
Subject: [PATCH 19/26] log: use the same function to init log in
Subject: [PATCH 19/53] log: use the same function to init log in
export/pause/resume
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>

2
0020-init-log-config-should-before-command-parse.patch
View File

@ -1,7 +1,7 @@
From 20a6562ea0a6c50bdc6a863067eeaf7fa04909d0 Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Sat, 30 Jan 2021 14:46:13 +0800
Subject: [PATCH 20/26] init log config should before command parse
Subject: [PATCH 20/53] init log config should before command parse
Signed-off-by: haozi007 <liuhao27@huawei.com>
---

2
0021-spec-add-verify-for-device-cgroup-access-mode.patch
View File

@ -1,7 +1,7 @@
From 82d59974b5fcb0abfa2f488801e7d9ed2f93a718 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Sat, 30 Jan 2021 14:22:16 +0800
Subject: [PATCH 21/26] spec: add verify for device cgroup access mode
Subject: [PATCH 21/53] spec: add verify for device cgroup access mode
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---

2
0022-log-change-log-level-from-warn-to-error.patch
View File

@ -1,7 +1,7 @@
From 87e886b239a932f37679f12fe2920d1b36e92985 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Sat, 30 Jan 2021 16:12:29 +0800
Subject: [PATCH 22/26] log: change log level from warn to error
Subject: [PATCH 22/53] log: change log level from warn to error
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---

2
0023-Fix-create-env-path-dir-if-dir-exist.patch
View File

@ -1,7 +1,7 @@
From 2bd45202ef01260a2181270012c4781afd5cccba Mon Sep 17 00:00:00 2001
From: zhangsong234 <zhangsong34@huawei.com>
Date: Mon, 1 Feb 2021 09:25:51 +0800
Subject: [PATCH 23/26] Fix create env path dir if dir exist
Subject: [PATCH 23/53] Fix create env path dir if dir exist
---
src/daemon/modules/service/service_container.c | 4 +++-

2
0024-iSulad-calculate-memusage-with-used-total_inactive_f.patch
View File

@ -1,7 +1,7 @@
From c5aeb37655533ce84161f237ed6175153891d9e0 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Thu, 28 Jan 2021 19:30:44 +0800
Subject: [PATCH 24/26] iSulad: calculate memusage with used -
Subject: [PATCH 24/53] iSulad: calculate memusage with used -
total_inactive_file
On cgroup v1 host, the result is `mem.used - mem.["total_inactive_file"]` .

2
0025-fix-container-exit-health-check-residue-and-multiple.patch
View File

@ -1,7 +1,7 @@
From 012b3f94279b0c6d193d510aa211b977a38e7c24 Mon Sep 17 00:00:00 2001
From: wujing <jing.woo@outlook.com>
Date: Fri, 22 Jan 2021 17:13:16 +0800
Subject: [PATCH 25/26] fix container exit health check residue and multiple
Subject: [PATCH 25/53] fix container exit health check residue and multiple
health checks
Signed-off-by: wujing <wujing50@huawei.com>

2
0026-CI-supplementary-testcase-for-health-check-monitor.patch
View File

@ -1,7 +1,7 @@
From 39e9ae73804880f523d83db6c8ad5d25d8bd79ed Mon Sep 17 00:00:00 2001
From: wujing <wujing50@huawei.com>
Date: Mon, 1 Feb 2021 16:58:35 +0800
Subject: [PATCH 26/26] CI: supplementary testcase for health check monitor
Subject: [PATCH 26/53] CI: supplementary testcase for health check monitor
Signed-off-by: wujing <wujing50@huawei.com>
---

36
0027-add-container-lock-when-clean-container-resource.patch Normal file
View File

@ -0,0 +1,36 @@
From 002a546ec0ada609aebeccfc935d773968f89312 Mon Sep 17 00:00:00 2001
From: zhangxiaoyu <zhangxiaoyu58@huawei.com>
Date: Thu, 4 Feb 2021 10:43:59 +0800
Subject: [PATCH 27/53] add container lock when clean container resource
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
---
src/daemon/modules/service/service_container.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/src/daemon/modules/service/service_container.c b/src/daemon/modules/service/service_container.c
index c4b9dbd6..6551bfbf 100644
--- a/src/daemon/modules/service/service_container.c
+++ b/src/daemon/modules/service/service_container.c
@@ -968,12 +968,17 @@ int clean_container_resource(const char *id, const char *runtime, pid_t pid)
goto out;
}
+ container_lock(cont);
ret = do_clean_container(cont, pid);
if (ret != 0) {
ERROR("Runtime clean container resource failed");
ret = -1;
- goto out;
+ goto unlock;
}
+
+unlock:
+ container_unlock(cont);
+
out:
container_unref(cont);
return ret;
--
2.25.1

36
0028-sleep-some-time-before-calculate-to-make-sure-fd-clo.patch Normal file
View File

@ -0,0 +1,36 @@
From 3b8075caac328d88018d4607ee8d18138440e8b8 Mon Sep 17 00:00:00 2001
From: WangFengTu <wangfengtu@huawei.com>
Date: Thu, 4 Feb 2021 15:43:33 +0800
Subject: [PATCH 28/53] sleep some time before calculate to make sure fd closed
Signed-off-by: WangFengTu <wangfengtu@huawei.com>
---
CI/test_cases/container_cases/cp.sh | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/CI/test_cases/container_cases/cp.sh b/CI/test_cases/container_cases/cp.sh
index 67a36909..09b6ddff 100644
--- a/CI/test_cases/container_cases/cp.sh
+++ b/CI/test_cases/container_cases/cp.sh
@@ -329,6 +329,9 @@ function cp_test_t()
msg_info "${test} starting..."
local isulad_pid=$(cat /var/run/isulad.pid)
+
+ # wait some time to make sure fd closed
+ sleep 3
local fd_num1=$(ls -l /proc/$isulad_pid/fd | wc -l)
[[ $fd_num1 -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - can not get fd number" && ((ret++))
@@ -362,6 +365,8 @@ function cp_test_t()
rm -rf $cpfiles
+ # wait some time to make sure fd closed
+ sleep 3
local fd_num2=$(ls -l /proc/$isulad_pid/fd | wc -l)
[[ $fd_num2 -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - can not get fd number" && ((ret++))
--
2.25.1

41
0029-stats-fix-wrong-memory-usage-info-in-stats.patch Normal file
View File

@ -0,0 +1,41 @@
From 3660db243160d45535c1d020844b694e495b4cd7 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Mon, 8 Feb 2021 11:03:58 +0800
Subject: [PATCH 29/53] stats: fix wrong memory usage info in stats
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---
src/cmd/isula/extend/stats.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/src/cmd/isula/extend/stats.c b/src/cmd/isula/extend/stats.c
index 03544325..724cf381 100644
--- a/src/cmd/isula/extend/stats.c
+++ b/src/cmd/isula/extend/stats.c
@@ -101,10 +101,12 @@ static void stats_print(const struct isula_container_info *stats)
int len;
double cpu_percent = 0.0;
char *short_id = NULL;
+ // workingset = usage - total_inactive_file
+ uint64_t workingset = memory_get_working_set(stats);
isula_size_humanize(stats->blkio_read, iosb_read_str, sizeof(iosb_read_str));
isula_size_humanize(stats->blkio_write, iosb_write_str, sizeof(iosb_write_str));
- isula_size_humanize(stats->mem_used, mem_used_str, sizeof(mem_used_str));
+ isula_size_humanize(workingset, mem_used_str, sizeof(mem_used_str));
isula_size_humanize(stats->mem_limit, mem_limit_str, sizeof(mem_limit_str));
len = snprintf(iosb_str, sizeof(iosb_str), "%s / %s", iosb_read_str, iosb_write_str);
@@ -142,8 +144,7 @@ static void stats_print(const struct isula_container_info *stats)
if (strlen(short_id) > SHORTIDLEN) {
short_id[SHORTIDLEN] = '\0';
}
- // workingset = usage - total_inactive_file
- uint64_t workingset = memory_get_working_set(stats);
+
printf("%-16s %-10.2f %-26s %-10.2f %-26s %-10llu", short_id, cpu_percent, mem_str,
stats->mem_limit ? ((double)workingset / stats->mem_limit) * PERCENT : 0.00, iosb_str,
(unsigned long long)stats->pids_current);
--
2.25.1

36
0030-save-health-check-log-to-disk-before-unhealthy.patch Normal file
View File

@ -0,0 +1,36 @@
From cb82131f8b4b9a1d517b9e4da0d707008567a89e Mon Sep 17 00:00:00 2001
From: wujing <wujing50@huawei.com>
Date: Mon, 8 Feb 2021 11:23:24 +0800
Subject: [PATCH 30/53] save health check log to disk before unhealthy
Signed-off-by: wujing <wujing50@huawei.com>
---
src/daemon/modules/container/health_check/health_check.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/src/daemon/modules/container/health_check/health_check.c b/src/daemon/modules/container/health_check/health_check.c
index c6ccbbf2..a01679db 100644
--- a/src/daemon/modules/container/health_check/health_check.c
+++ b/src/daemon/modules/container/health_check/health_check.c
@@ -429,6 +429,10 @@ static int handle_increment_streak(container_t *cont, int retries)
ret = -1;
}
}
+ } else {
+ if (container_state_to_disk(cont)) {
+ WARN("Failed to save container \"%s\" to disk", cont->common_config->id);
+ }
}
return ret;
@@ -542,6 +546,7 @@ static int handle_probe_result(const char *container_id, const defs_health_log_e
}
// else we're starting or healthy. Stay in that state.
}
+
// note: replicate Health status changes
current = get_health_status(cont->state);
if (strcmp(old_state, current) != 0) {
--
2.25.1

179
0031-unpack-try-to-remove-and-replace-dst_path-while-unpa.patch Normal file
View File

@ -0,0 +1,179 @@
From a3ee97e18060785b22529dca0ea771e7f3e94293 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Mon, 8 Feb 2021 15:56:36 +0800
Subject: [PATCH 31/53] unpack: try to remove and replace dst_path while unpack
if dst path exits, we just want to remove and replace it.
exception: when the exited dstpath is directory and the file from the layer is also a directory.
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---
src/utils/tar/util_archive.c | 61 +++++++++++++++++++++++++-----------
1 file changed, 42 insertions(+), 19 deletions(-)
diff --git a/src/utils/tar/util_archive.c b/src/utils/tar/util_archive.c
index 1b9553c9..a7522036 100644
--- a/src/utils/tar/util_archive.c
+++ b/src/utils/tar/util_archive.c
@@ -394,6 +394,29 @@ static int rebase_hardlink(struct archive_entry *entry, const char *src_base, co
return 0;
}
+// if dst path exits, we just want to remove and replace it.
+// exception: when the exited dstpath is directory and the file from the layer is also a directory.
+static void try_to_replace_exited_dst(const char *dst_path, struct archive_entry *entry)
+{
+ struct stat s;
+ int nret;
+
+ nret = lstat(dst_path, &s);
+ if (nret < 0) {
+ return;
+ }
+
+ if (S_ISDIR(s.st_mode) && archive_entry_filetype(entry) == AE_IFDIR) {
+ return;
+ }
+
+ if (util_recursive_remove_path(dst_path) != 0) {
+ ERROR("Failed to remove path %s while unpack", dst_path);
+ }
+
+ return;
+}
+
int archive_unpack_handler(const struct io_read_wrapper *content, const struct archive_options *options)
{
int ret = 0;
@@ -499,6 +522,8 @@ int archive_unpack_handler(const struct io_read_wrapper *content, const struct a
continue;
}
+ try_to_replace_exited_dst(dst_path, entry);
+
ret = archive_write_header(ext, entry);
if (ret != ARCHIVE_OK) {
ERROR("Fail to handle tar header: %s", archive_error_string(ext));
@@ -874,8 +899,8 @@ static ssize_t stream_write_data(struct archive *a, void *client_data, const voi
return size;
}
-static int tar_all(const struct io_write_wrapper *writer, const char *tar_dir,
- const char *src_base, const char *dst_base)
+static int tar_all(const struct io_write_wrapper *writer, const char *tar_dir, const char *src_base,
+ const char *dst_base)
{
struct archive *r = NULL;
struct archive *w = NULL;
@@ -906,7 +931,7 @@ static int tar_all(const struct io_write_wrapper *writer, const char *tar_dir,
}
archive_write_set_format_pax(w);
archive_write_set_options(w, "xattrheader=SCHILY");
- ret = archive_write_open(w, (void*)writer, NULL, stream_write_data, NULL);
+ ret = archive_write_open(w, (void *)writer, NULL, stream_write_data, NULL);
if (ret != ARCHIVE_OK) {
ERROR("open archive write failed: %s", archive_error_string(w));
fprintf(stderr, "open archive write failed: %s\n", archive_error_string(w));
@@ -924,7 +949,7 @@ out:
static ssize_t fd_write(void *context, const void *data, size_t len)
{
- return util_write_nointr(*(int*)context, data, len);
+ return util_write_nointr(*(int *)context, data, len);
}
int archive_chroot_tar(char *path, char *file, char **errmsg)
@@ -989,7 +1014,7 @@ int archive_chroot_tar(char *path, char *file, char **errmsg)
goto child_out;
}
- pipe_context.context = (void*)&fd;
+ pipe_context.context = (void *)&fd;
pipe_context.write_func = fd_write;
ret = tar_all(&pipe_context, ".", ".", NULL);
@@ -1024,7 +1049,7 @@ cleanup:
static ssize_t pipe_read(void *context, void *buf, size_t len)
{
- return util_read_nointr(*(int*)context, buf, len);
+ return util_read_nointr(*(int *)context, buf, len);
}
static ssize_t archive_context_write(const void *context, const void *buf, size_t len)
@@ -1041,7 +1066,7 @@ static ssize_t archive_context_write(const void *context, const void *buf, size_
static ssize_t pipe_write(void *context, const void *data, size_t len)
{
- return util_write_nointr(*(int*)context, data, len);
+ return util_write_nointr(*(int *)context, data, len);
}
static ssize_t archive_context_read(void *context, void *buf, size_t len)
@@ -1128,9 +1153,8 @@ static int archive_context_close(void *context, char **err)
return ret;
}
-int archive_chroot_untar_stream(const struct io_read_wrapper *context, const char *chroot_dir,
- const char *untar_dir, const char *src_base, const char *dst_base,
- char **errmsg)
+int archive_chroot_untar_stream(const struct io_read_wrapper *context, const char *chroot_dir, const char *untar_dir,
+ const char *src_base, const char *dst_base, char **errmsg)
{
struct io_read_wrapper pipe_context = { 0 };
int pipe_stream[2] = { -1, -1 };
@@ -1143,10 +1167,9 @@ int archive_chroot_untar_stream(const struct io_read_wrapper *context, const cha
char *buf = NULL;
size_t buf_len = ARCHIVE_BLOCK_SIZE;
ssize_t read_len;
- struct archive_options options = {
- .whiteout_format = NONE_WHITEOUT_FORMATE,
- .src_base = src_base,
- .dst_base = dst_base
+ struct archive_options options = { .whiteout_format = NONE_WHITEOUT_FORMATE,
+ .src_base = src_base,
+ .dst_base = dst_base
};
buf = util_common_calloc_s(buf_len);
@@ -1201,7 +1224,7 @@ int archive_chroot_untar_stream(const struct io_read_wrapper *context, const cha
goto child_out;
}
- pipe_context.context = (void*)&pipe_stream[0];
+ pipe_context.context = (void *)&pipe_stream[0];
pipe_context.read = pipe_read;
ret = archive_unpack_handler(&pipe_context, &options);
@@ -1252,8 +1275,8 @@ cleanup:
return ret;
}
-int archive_chroot_tar_stream(const char *chroot_dir, const char *tar_path, const char *src_base,
- const char *dst_base, struct io_read_wrapper *reader)
+int archive_chroot_tar_stream(const char *chroot_dir, const char *tar_path, const char *src_base, const char *dst_base,
+ struct io_read_wrapper *reader)
{
struct io_write_wrapper pipe_context = { 0 };
int keepfds[] = { -1, -1, -1 };
@@ -1273,7 +1296,7 @@ int archive_chroot_tar_stream(const char *chroot_dir, const char *tar_path, cons
}
pid = fork();
- if (pid == (pid_t) - 1) {
+ if (pid == (pid_t) -1) {
ERROR("Failed to fork: %s", strerror(errno));
goto free_out;
}
@@ -1320,7 +1343,7 @@ int archive_chroot_tar_stream(const char *chroot_dir, const char *tar_path, cons
goto child_out;
}
- pipe_context.context = (void*)&pipe_stream[1];
+ pipe_context.context = (void *)&pipe_stream[1];
pipe_context.write_func = pipe_write;
ret = tar_all(&pipe_context, tar_base_name, src_base, dst_base);
--
2.25.1

93
0032-fd-leak-check-in-cp.sh-should-not-include-pull-fd-ch.patch Normal file
View File

@ -0,0 +1,93 @@
From 513530a98d627ae84b1415f93af5bc298b39ba9d Mon Sep 17 00:00:00 2001
From: WangFengTu <wangfengtu@huawei.com>
Date: Mon, 8 Feb 2021 16:38:22 +0800
Subject: [PATCH 32/53] fd leak check in cp.sh should not include pull fd check
pull may increase fd, we can check pull fd leak in registry.sh
Signed-off-by: WangFengTu <wangfengtu@huawei.com>
---
CI/test_cases/container_cases/cp.sh | 14 ++++++++------
CI/test_cases/image_cases/registry.sh | 21 +++++++++++++++++++++
2 files changed, 29 insertions(+), 6 deletions(-)
diff --git a/CI/test_cases/container_cases/cp.sh b/CI/test_cases/container_cases/cp.sh
index 09b6ddff..fef637a5 100644
--- a/CI/test_cases/container_cases/cp.sh
+++ b/CI/test_cases/container_cases/cp.sh
@@ -328,18 +328,19 @@ function cp_test_t()
msg_info "${test} starting..."
+ isula inspect ${image}
+ if [ x"$?" != x"0" ];then
+ isula pull ${image}
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to pull image: ${image}" && return ${FAILURE}
+ fi
+
local isulad_pid=$(cat /var/run/isulad.pid)
# wait some time to make sure fd closed
sleep 3
local fd_num1=$(ls -l /proc/$isulad_pid/fd | wc -l)
[[ $fd_num1 -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - can not get fd number" && ((ret++))
-
- isula inspect ${image}
- if [ x"$?" != x"0" ];then
- isula pull ${image}
- [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to pull image: ${image}" && return ${FAILURE}
- fi
+ ls -l /proc/$isulad_pid/fd
isula images | grep busybox
[[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - missing list image: ${image}" && ((ret++))
@@ -369,6 +370,7 @@ function cp_test_t()
sleep 3
local fd_num2=$(ls -l /proc/$isulad_pid/fd | wc -l)
[[ $fd_num2 -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - can not get fd number" && ((ret++))
+ ls -l /proc/$isulad_pid/fd
# make sure fd not increase after test
[[ $fd_num1 -ne $fd_num2 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - fd number not right" && ((ret++))
diff --git a/CI/test_cases/image_cases/registry.sh b/CI/test_cases/image_cases/registry.sh
index 042b1f4a..4e6adc28 100755
--- a/CI/test_cases/image_cases/registry.sh
+++ b/CI/test_cases/image_cases/registry.sh
@@ -26,6 +26,18 @@ source ../helpers.sh
function isula_pull()
{
isula rm -f `isula ps -a -q`
+
+ isula pull busybox
+ fn_check_eq "$?" "0" "isula pull busybox"
+
+ local isulad_pid=$(cat /var/run/isulad.pid)
+
+ # wait some time to make sure fd closed
+ sleep 3
+ local fd_num1=$(ls -l /proc/$isulad_pid/fd | wc -l)
+ [[ $fd_num1 -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - can not get fd number" && ((ret++))
+ ls -l /proc/$isulad_pid/fd
+
isula rmi busybox
for i in `seq 1 10`
@@ -36,6 +48,15 @@ function isula_pull()
fn_check_eq "$?" "0" "isula pull busybox"
wait
+ # wait some time to make sure fd closed
+ sleep 3
+ local fd_num2=$(ls -l /proc/$isulad_pid/fd | wc -l)
+ [[ $fd_num2 -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - can not get fd number" && ((ret++))
+ ls -l /proc/$isulad_pid/fd
+
+ # make sure fd not increase after remove and pull busybox
+ [[ $fd_num1 -ne $fd_num2 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - fd number not right" && ((ret++))
+
isula inspect busybox
fn_check_eq "$?" "0" "isula inspect busybox"
--
2.25.1

116
0033-devmapper-modify-log-msg.patch Normal file
View File

@ -0,0 +1,116 @@
From 085f9c923fc7e833bcbf93ece33dda1c0e7e0a66 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Tue, 9 Feb 2021 14:34:54 +0800
Subject: [PATCH 33/53] devmapper: modify log msg
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
.../graphdriver/devmapper/deviceset.c | 17 ++++++++---------
.../graphdriver/devmapper/metadata_store.c | 2 --
2 files changed, 8 insertions(+), 11 deletions(-)
diff --git a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
index 67b1ba9c..2bd3b9c8 100644
--- a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
+++ b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
@@ -176,7 +176,7 @@ out:
return 0;
}
-static char *metadata_dir(struct device_set *devset)
+static char *metadata_dir(const struct device_set *devset)
{
return util_path_join(devset->root, "metadata");
}
@@ -411,7 +411,7 @@ out:
return exist;
}
-static image_devmapper_device_info *load_metadata(struct device_set *devset, const char *hash)
+static image_devmapper_device_info *load_metadata(const struct device_set *devset, const char *hash)
{
image_devmapper_device_info *info = NULL;
char metadata_file[PATH_MAX] = { 0 };
@@ -420,7 +420,6 @@ static image_devmapper_device_info *load_metadata(struct device_set *devset, con
parser_error err = NULL;
if (hash == NULL) {
- ERROR("Invalid input param");
return NULL;
}
@@ -432,13 +431,13 @@ static image_devmapper_device_info *load_metadata(struct device_set *devset, con
nret = snprintf(metadata_file, sizeof(metadata_file), "%s/%s", metadata_path, util_valid_str(hash) ? hash : "base");
if (nret < 0 || (size_t)nret >= sizeof(metadata_file)) {
- ERROR("Get metadata file with hash:%s path failed", hash);
+ ERROR("Failed to snprintf metadata file path with hash:%s, path is too long", hash);
goto out;
}
info = image_devmapper_device_info_parse_file(metadata_file, NULL, &err);
if (info == NULL) {
- ERROR("load metadata file %s failed %s", metadata_file, err != NULL ? err : "");
+ SYSERROR("Load metadata file:%s failed:%s", metadata_file, err);
goto out;
}
@@ -448,7 +447,7 @@ static image_devmapper_device_info *load_metadata(struct device_set *devset, con
}
if (info->device_id > MAX_DEVICE_ID) {
- ERROR("devmapper: Ignoring Invalid DeviceId=%d", info->device_id);
+ ERROR("devmapper: device id:%d out of limits, to be ignored", info->device_id);
free_image_devmapper_device_info(info);
info = NULL;
goto out;
@@ -878,7 +877,7 @@ static int load_transaction_metadata(struct device_set *devset)
trans = image_devmapper_transaction_parse_file(fname, NULL, &err);
if (trans == NULL) {
- ERROR("devmapper: load transaction metadata file error %s", err);
+ SYSERROR("Load transaction metadata file:%s failed:%s", fname, err);
ret = -1;
goto out;
}
@@ -1061,7 +1060,7 @@ static int load_deviceset_metadata(struct device_set *devset)
deviceset_meta = image_devmapper_deviceset_metadata_parse_file(meta_file, NULL, &err);
if (deviceset_meta == NULL) {
- ERROR("devmapper: load deviceset metadata file error %s", err);
+ SYSERROR("Load deviceset metadata file:%s failed:%s", meta_file, err);
ret = -1;
goto out;
}
@@ -2781,7 +2780,7 @@ int add_device(const char *hash, const char *base_hash, struct device_set *devse
base_device_info = lookup_device(devset, util_valid_str(base_hash) ? base_hash : "base");
if (base_device_info == NULL) {
- ERROR("Lookup device %s failed", util_valid_str(base_hash) ? base_hash : "base");
+ ERROR("Lookup device %s failed, not found", util_valid_str(base_hash) ? base_hash : "base");
ret = -1;
goto free_out;
}
diff --git a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/metadata_store.c b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/metadata_store.c
index 01858748..c8d10b7a 100644
--- a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/metadata_store.c
+++ b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/metadata_store.c
@@ -34,7 +34,6 @@ void devmapper_device_info_ref_inc(devmapper_device_info_t *device_info)
static void free_devmapper_device_info_t(devmapper_device_info_t *ptr)
{
if (ptr == NULL) {
- ERROR("invalid argument");
return;
}
@@ -49,7 +48,6 @@ void devmapper_device_info_ref_dec(devmapper_device_info_t *device_info)
bool is_zero = false;
if (device_info == NULL) {
- ERROR("invalid argument");
return;
}
--
2.25.1

111
0034-name_id_index-fix-restore-fail-to-remove-name-index.patch Normal file
View File

@ -0,0 +1,111 @@
From a7f40f1e13f08f03ca369dc908a399dfc3f7fe17 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Fri, 19 Feb 2021 19:06:43 +0800
Subject: [PATCH 34/53] name_id_index: fix restore fail to remove name index
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---
src/daemon/executor/container_cb/execution_create.c | 10 ++++++++--
src/daemon/modules/container/containers_store.c | 12 ++++++++++--
src/daemon/modules/container/restore/restore.c | 4 ++--
3 files changed, 20 insertions(+), 6 deletions(-)
diff --git a/src/daemon/executor/container_cb/execution_create.c b/src/daemon/executor/container_cb/execution_create.c
index 7ad55fa1..acad7fe3 100644
--- a/src/daemon/executor/container_cb/execution_create.c
+++ b/src/daemon/executor/container_cb/execution_create.c
@@ -440,6 +440,8 @@ static char *try_generate_id()
value = container_name_index_get(id);
if (value != NULL) {
+ free(value);
+ value = NULL;
continue;
} else {
goto out;
@@ -631,10 +633,14 @@ static int maintain_container_id(const container_create_request *request, char *
EVENT("Event: {Object: %s, Type: Creating %s}", id, name);
if (!container_name_index_add(name, id)) {
- ERROR("Name %s is in use", name);
+ char *used_id = NULL;
+ used_id = container_name_index_get(name);
+ ERROR("Name %s is in use by container %s", name, used_id);
isulad_set_error_message("Conflict. The name \"%s\" is already in use by container %s. "
"You have to remove (or rename) that container to be able to reuse that name.",
- name, name);
+ name, used_id);
+ free(used_id);
+ used_id = NULL;
ret = -1;
goto out;
}
diff --git a/src/daemon/modules/container/containers_store.c b/src/daemon/modules/container/containers_store.c
index bbfbda3a..42972392 100644
--- a/src/daemon/modules/container/containers_store.c
+++ b/src/daemon/modules/container/containers_store.c
@@ -128,6 +128,7 @@ static container_t *containers_store_get_by_id(const char *id)
static container_t *containers_store_get_by_name(const char *name)
{
char *id = NULL;
+ container_t *cont = NULL;
if (name == NULL) {
ERROR("No container name supplied");
@@ -140,7 +141,10 @@ static container_t *containers_store_get_by_name(const char *name)
return NULL;
}
- return containers_store_get_by_id(id);
+ cont = containers_store_get_by_id(id);
+
+ free(id);
+ return cont;
}
/* containers store get container by prefix */
@@ -443,6 +447,7 @@ unlock_out:
char *container_name_index_get(const char *name)
{
char *id = NULL;
+ char *result = NULL;
if (name == NULL) {
return id;
@@ -451,11 +456,14 @@ char *container_name_index_get(const char *name)
ERROR("lock name index failed");
return id;
}
+
id = map_search(g_indexs->map, (void *)name);
+ result = util_strdup_s(id);
+
if (pthread_rwlock_unlock(&g_indexs->rwlock) != 0) {
ERROR("unlock name index failed");
}
- return id;
+ return result;
}
/* name index remove */
diff --git a/src/daemon/modules/container/restore/restore.c b/src/daemon/modules/container/restore/restore.c
index 13cdcd24..a7ee11a2 100644
--- a/src/daemon/modules/container/restore/restore.c
+++ b/src/daemon/modules/container/restore/restore.c
@@ -455,11 +455,11 @@ error_load:
if (remove_invalid_container(cont, runtime, rootpath, statepath, subdir[i])) {
ERROR("Failed to delete subdir:%s", subdir[i]);
}
- container_unref(cont);
if (index_flag) {
- container_name_index_remove(subdir[i]);
+ container_name_index_remove(cont->common_config->name);
}
+ container_unref(cont);
continue;
}
}
--
2.25.1

26
0035-thread-function-calls-DAEMON_CLEAR_ERRORMSG-to-preve.patch Normal file
View File

@ -0,0 +1,26 @@
From 2c08c9e9ce52afbe46753f44daec70270667f760 Mon Sep 17 00:00:00 2001
From: wujing <wujing50@huawei.com>
Date: Sat, 20 Feb 2021 11:07:59 +0800
Subject: [PATCH 35/53] thread function calls DAEMON_CLEAR_ERRORMSG to prevent
memory leak
Signed-off-by: wujing <wujing50@huawei.com>
---
src/daemon/modules/container/health_check/health_check.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/daemon/modules/container/health_check/health_check.c b/src/daemon/modules/container/health_check/health_check.c
index a01679db..dc00ae33 100644
--- a/src/daemon/modules/container/health_check/health_check.c
+++ b/src/daemon/modules/container/health_check/health_check.c
@@ -407,6 +407,7 @@ static void *stop_container_on_unhealthy(void *arg)
out:
free(container_id);
container_unref(cont);
+ DAEMON_CLEAR_ERRMSG();
return NULL;
}
--
2.25.1

359
0036-modify-resume-task-name.patch Normal file
View File

@ -0,0 +1,359 @@
From 4726f2f980f42963a753350aa5306cbe6a4cc668 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Sat, 20 Feb 2021 09:44:19 +0800
Subject: [PATCH 36/53] modify resume task name
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
.../graphdriver/devmapper/deviceset.c | 68 +++++++++++++------
.../graphdriver/devmapper/wrapper_devmapper.c | 57 +++++++++-------
.../graphdriver/devmapper/wrapper_devmapper.h | 2 +-
3 files changed, 81 insertions(+), 46 deletions(-)
diff --git a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
index 2bd3b9c8..0b0394c5 100644
--- a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
+++ b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
@@ -125,7 +125,8 @@ static int devmapper_parse_options(struct device_set *devset, const char **optio
devset->min_free_space_percent = (uint32_t)converted;
} else if (strcasecmp(dup, "dm.basesize") == 0) {
int64_t converted = 0;
- if (util_parse_byte_size_string(val, &converted) != 0) {
+ ret = util_parse_byte_size_string(val, &converted);
+ if (ret != 0) {
ERROR("Invalid size: '%s': %s", val, strerror(-ret));
isulad_set_error_message("Invalid size: '%s': %s", val, strerror(-ret));
ret = -1;
@@ -435,6 +436,11 @@ static image_devmapper_device_info *load_metadata(const struct device_set *devse
goto out;
}
+ if (!util_file_exists(metadata_file)) {
+ ERROR("No such file:%s, need not to load", metadata_file);
+ goto out;
+ }
+
info = image_devmapper_device_info_parse_file(metadata_file, NULL, &err);
if (info == NULL) {
SYSERROR("Load metadata file:%s failed:%s", metadata_file, err);
@@ -594,6 +600,7 @@ static uint64_t get_base_device_size(struct device_set *devset)
device_info = lookup_device(devset, "base");
if (device_info == NULL) {
+ ERROR("No such device:\"base\"");
return 0;
}
@@ -1688,10 +1695,10 @@ out:
(void)deactivate_device(devset, base_info);
}
- if (resume_dev && dev_resume_device(dm_name) != 0) {
- ERROR("devmapper: resume dm with name:%s failed", dm_name);
- ret = -1;
+ if (resume_dev) {
+ dev_resume_device(dm_name);
}
+
free(dm_name);
return ret;
}
@@ -2558,7 +2565,8 @@ static int determine_driver_capabilities(const char *version, struct device_set
goto out;
}
- if (util_parse_byte_size_string(tmp_str[0], &major) != 0) {
+ ret = util_parse_byte_size_string(tmp_str[0], &major);
+ if (ret != 0) {
ERROR("devmapper: invalid size: '%s': %s", tmp_str[0], strerror(-ret));
ret = -1;
goto out;
@@ -2577,7 +2585,8 @@ static int determine_driver_capabilities(const char *version, struct device_set
goto out;
}
- if (util_parse_byte_size_string(tmp_str[1], &minor) != 0) {
+ ret = util_parse_byte_size_string(tmp_str[1], &minor);
+ if (ret != 0) {
ERROR("devmapper: invalid size: '%s': %s", tmp_str[1], strerror(-ret));
ret = -1;
goto out;
@@ -2742,7 +2751,8 @@ static int parse_storage_opt(const json_map_string_string *opts, uint64_t *size)
if (strcasecmp("size", opts->keys[i]) == 0) {
int64_t converted = 0;
- if (util_parse_byte_size_string(opts->values[i], &converted) != 0) {
+ ret = util_parse_byte_size_string(opts->values[i], &converted);
+ if (ret != 0) {
ERROR("Invalid size: '%s': %s", opts->values[i], strerror(-ret));
ret = -1;
goto out;
@@ -2760,6 +2770,31 @@ out:
return ret;
}
+static int grow_device_fs(struct device_set *devset, const char *hash, uint64_t size, uint64_t base_size)
+{
+ int ret = 0;
+ devmapper_device_info_t *device_info = NULL;
+
+ if (size <= base_size) {
+ return 0;
+ } else {
+ DEBUG("devmapper: new fs size is larger than old basesize, start to grow fs");
+ device_info = lookup_device(devset, hash);
+ if (device_info == NULL) {
+ ERROR("devmapper: lookup device %s failed", hash);
+ ret = -1;
+ goto out;
+ }
+
+ if (grow_fs(devset, device_info->info) != 0) {
+ ret = -1;
+ goto out;
+ }
+ }
+out:
+ return ret;
+}
+
int add_device(const char *hash, const char *base_hash, struct device_set *devset,
const json_map_string_string *storage_opts)
{
@@ -2820,18 +2855,14 @@ int add_device(const char *hash, const char *base_hash, struct device_set *devse
goto free_out;
}
- if (size > base_device_info->info->size) {
- DEBUG("devmapper: new fs size is larger than old basesize, start to grow fs");
- device_info = lookup_device(devset, hash);
- if (device_info == NULL) {
- ERROR("devmapper: lookup device %s failed", hash);
- ret = -1;
- goto free_out;
- }
- if (grow_fs(devset, device_info->info) != 0) {
- ret = -1;
- goto free_out;
+ if (grow_device_fs(devset, hash, size, base_device_info->info->size) != 0) {
+ ERROR("Grow new deivce fs failed");
+ // Here, we need to delete device directly instead of deferred deleting, so that we can retry to add device with the same hash successfully.
+ if (do_delete_device(devset, hash, true) != 0) {
+ ERROR("devmapper: remove new snapshot device failed");
}
+ ret = -1;
+ goto free_out;
}
free_out:
@@ -2839,7 +2870,6 @@ free_out:
devmapper_device_info_ref_dec(device_info);
if (pthread_rwlock_unlock(&devset->devmapper_driver_rwlock)) {
ERROR("unlock devmapper conf failed");
- return -1;
}
return ret;
}
diff --git a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/wrapper_devmapper.c b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/wrapper_devmapper.c
index 5748ec54..38ed5615 100644
--- a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/wrapper_devmapper.c
+++ b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/wrapper_devmapper.c
@@ -45,6 +45,9 @@ char *dev_strerror(int errnum)
case ERR_TASK_SET_COOKIE:
errmsg = "Task set cookie error";
break;
+ case ERR_NIL_COOKIE:
+ errmsg = "cookie ptr can't be nil";
+ break;
case ERR_TASK_SET_ADD_NODE:
errmsg = "Task add dm node failed";
break;
@@ -443,6 +446,7 @@ free_out:
int dev_delete_device_force(const char *name)
{
int ret = 0;
+ int nret = 0;
struct dm_task *dmt = NULL;
uint32_t cookie = 0;
@@ -459,8 +463,9 @@ int dev_delete_device_force(const char *name)
goto out;
}
- if (set_cookie(dmt, &cookie, 0) != 0) {
- ERROR("devicemapper: set cookie failed");
+ nret = set_cookie(dmt, &cookie, 0);
+ if (nret != 0) {
+ ERROR("set cookie failed:%s", dev_strerror(nret));
ret = -1;
goto out;
}
@@ -468,7 +473,7 @@ int dev_delete_device_force(const char *name)
g_dm_saw_busy = false;
g_dm_saw_enxio = false;
if (dm_task_run(dmt) != 1) {
- ERROR("devicemapper: run task to delete device faild");
+ ERROR("devicemapper: task run failed");
if (g_dm_saw_busy) {
ERROR("devicemapper: delete task run err type is \"device is busy\"");
ret = ERR_BUSY;
@@ -498,6 +503,7 @@ out:
int dev_remove_device_deferred(const char *name)
{
int ret = 0;
+ int nret = 0;
struct dm_task *dmt = NULL;
uint32_t cookie = 0;
uint16_t flags = DM_UDEV_DISABLE_LIBRARY_FALLBACK;
@@ -521,15 +527,16 @@ int dev_remove_device_deferred(const char *name)
goto out;
}
- if (set_cookie(dmt, &cookie, flags) != 0) {
- ERROR("devicemapper: set cookie failed");
+ nret = set_cookie(dmt, &cookie, flags);
+ if (nret != 0) {
+ ERROR("set cookie failed:%s", dev_strerror(nret));
ret = -1;
goto out;
}
g_dm_saw_enxio = false;
if (dm_task_run(dmt) != 1) {
- ERROR("devicemapper: Error running RemoveDeviceDeferred %d", ret);
+ ERROR("devicemapper: task run failed");
if (g_dm_saw_enxio) {
ERROR("devicemapper: delete deferred task run err type is \"No such device or address\"");
ret = ERR_ENXIO;
@@ -694,7 +701,7 @@ int dev_create_device(const char *pool_fname, int device_id)
} else {
ret = -1;
}
- ERROR("devicemapper: task run failed to create device");
+ ERROR("devicemapper: task run failed");
goto cleanup;
}
@@ -754,7 +761,7 @@ int dev_delete_device(const char *pool_fname, int device_id)
DEBUG("devicemapper: device(id:%d) from pool(%s) does not exist", device_id, pool_fname);
goto cleanup;
}
- ERROR("devicemapper: Error running dev_delete_device");
+ ERROR("devicemapper: task run failed");
ret = -1;
goto cleanup;
}
@@ -786,7 +793,7 @@ int dev_suspend_device(const char *dm_name)
}
if (dm_task_run(dmt) != 1) {
- ERROR("devicemapper: Error running deviceCreate (ActivateDevice)");
+ ERROR("devicemapper: task run failed");
ret = -1;
goto out;
}
@@ -800,34 +807,32 @@ out:
// ResumeDevice is the programmatic example of "dmsetup resume".
// It un-suspends the specified device.
-int dev_resume_device(const char *dm_name)
+void dev_resume_device(const char *dm_name)
{
- int ret = 0;
+ int nret = 0;
uint32_t cookie = 0;
uint16_t flags = 0;
struct dm_task *dmt = NULL;
if (dm_name == NULL) {
ERROR("devicemapper: invalid input params to resume device");
- return -1;
+ return;
}
- dmt = task_create_named(DM_DEVICE_SUSPEND, dm_name);
+ dmt = task_create_named(DM_DEVICE_RESUME, dm_name);
if (dmt == NULL) {
- ERROR("devicemapper:create named task(DM_DEVICE_SUSPEND) failed");
- ret = -1;
+ ERROR("devicemapper: create named task(DM_DEVICE_RESUME) failed");
goto out;
}
- if (set_cookie(dmt, &cookie, flags) != 0) {
- ERROR("devicemapper: Can't set cookie %d", ret);
- ret = -1;
+ nret = set_cookie(dmt, &cookie, flags);
+ if (nret != 0) {
+ ERROR("set cookie failed:%s", dev_strerror(nret));
goto out;
}
if (dm_task_run(dmt) != 1) {
- ERROR("devicemapper: Error running deviceResume %d", ret);
- ret = -1;
+ ERROR("devicemapper: run task of DM_DEVICE_RESUME failed");
}
DEBUG("Start udev wait on resume device");
@@ -837,7 +842,6 @@ out:
if (dmt != NULL) {
dm_task_destroy(dmt);
}
- return ret;
}
int dev_active_device(const char *pool_name, const char *name, int device_id, uint64_t size)
@@ -882,14 +886,15 @@ int dev_active_device(const char *pool_name, const char *name, int device_id, ui
goto out;
}
- if (set_cookie(dmt, &cookie, flags) != 0) {
- ERROR("devicemapper: Can't set cookie");
+ nret = set_cookie(dmt, &cookie, flags);
+ if (nret != 0) {
+ ERROR("set cookie failed:%s", dev_strerror(nret));
ret = -1;
goto out;
}
if (dm_task_run(dmt) != 1) {
- ERROR("devicemapper: error running deviceCreate (ActivateDevice) %d", ret);
+ ERROR("devicemapper: task run failed");
ret = -1;
}
@@ -944,7 +949,7 @@ int dev_cancel_deferred_remove(const char *dm_name)
ret = ERR_ENXIO;
goto cleanup;
}
- ERROR("devicemapper: Error running CancelDeferredRemove");
+ ERROR("devicemapper: task run failed");
ret = -1;
goto cleanup;
}
@@ -1070,7 +1075,7 @@ int dev_create_snap_device_raw(const char *pool_name, int device_id, int base_de
ret = ERR_DEVICE_ID_EXISTS;
goto cleanup;
}
- ERROR("devicemapper: Error running deviceCreate (CreateSnapDeviceRaw)");
+ ERROR("devicemapper: task run failed");
ret = -1;
goto cleanup;
}
diff --git a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/wrapper_devmapper.h b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/wrapper_devmapper.h
index 0f45a87d..6a45db58 100644
--- a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/wrapper_devmapper.h
+++ b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/wrapper_devmapper.h
@@ -119,7 +119,7 @@ int dev_delete_device(const char *pool_fname, int device_id);
int dev_suspend_device(const char *dm_name);
-int dev_resume_device(const char *dm_name);
+void dev_resume_device(const char *dm_name);
int dev_active_device(const char *pool_name, const char *name, int device_id, uint64_t size);
--
2.25.1

41
0037-cleadcode-Remove-extra-semicolons.patch Normal file
View File

@ -0,0 +1,41 @@
From 3db92c961e93093d8520f4e46255c12e774b841b Mon Sep 17 00:00:00 2001
From: wujing <wujing50@huawei.com>
Date: Mon, 22 Feb 2021 08:04:44 +0800
Subject: [PATCH 37/53] cleadcode: Remove extra semicolons
Signed-off-by: wujing <wujing50@huawei.com>
---
src/client/connect/grpc/client_base.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/client/connect/grpc/client_base.h b/src/client/connect/grpc/client_base.h
index 496370a8..5d1e7f0c 100644
--- a/src/client/connect/grpc/client_base.h
+++ b/src/client/connect/grpc/client_base.h
@@ -153,19 +153,19 @@ protected:
virtual auto request_to_grpc(const RQ * /*rq*/, gRQ * /*grq*/) -> int
{
return 0;
- };
+ }
virtual auto response_from_grpc(gRP * /*reply*/, RP * /*response*/) -> int
{
return 0;
- };
+ }
virtual auto check_parameter(const gRQ & /*grq*/) -> int
{
return 0;
- };
+ }
virtual auto grpc_call(ClientContext * /*context*/, const gRQ & /*req*/, gRP * /*reply*/) -> Status
{
return Status::OK;
- };
+ }
auto ReadTextFile(const char *file) -> std::string
{
--
2.25.1

39
0038-restart-policy-add-support-unless-stopped-policy.patch Normal file
View File

@ -0,0 +1,39 @@
From 86f34975a4e382a2967a27c589a72c857b0c1781 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Tue, 23 Feb 2021 09:05:45 +0800
Subject: [PATCH 38/53] restart policy: add support unless-stopped policy
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---
src/daemon/modules/spec/verify.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/src/daemon/modules/spec/verify.c b/src/daemon/modules/spec/verify.c
index a3156579..cef95065 100644
--- a/src/daemon/modules/spec/verify.c
+++ b/src/daemon/modules/spec/verify.c
@@ -1864,6 +1864,11 @@ static inline bool is_restart_policy_always(const char *policy)
return strcmp(policy, "always") == 0;
}
+static inline bool is_restart_policy_unless_stopped(const char *policy)
+{
+ return strcmp(policy, "unless-stopped") == 0;
+}
+
static inline bool is_restart_policy_on_reboot(const char *policy)
{
return strcmp(policy, "on-reboot") == 0;
@@ -1881,7 +1886,8 @@ static inline bool is_restart_policy_on_failure(const char *policy)
static int verify_restart_policy_name(const host_config_restart_policy *rp, const host_config *hostconfig)
{
- if (is_restart_policy_always(rp->name) || is_restart_policy_no(rp->name) || is_restart_policy_on_reboot(rp->name)) {
+ if (is_restart_policy_always(rp->name) || is_restart_policy_no(rp->name) || is_restart_policy_on_reboot(rp->name) ||
+ is_restart_policy_unless_stopped(rp->name)) {
if (rp->maximum_retry_count != 0) {
ERROR("Maximum retry count cannot be used with restart policy '%s'", rp->name);
isulad_set_error_message("Maximum retry count cannot be used with restart policy '%s'", rp->name);
--
2.25.1

79
0039-CI-add-testcase-for-unless-stopped-restart-policy.patch Normal file
View File

@ -0,0 +1,79 @@
From 6d2ce70731b36c8e2942571dca71149c26474d25 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Tue, 23 Feb 2021 09:13:28 +0800
Subject: [PATCH 39/53] CI: add testcase for unless-stopped restart policy
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---
.../container_cases/restartpolicy.sh | 48 +++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/CI/test_cases/container_cases/restartpolicy.sh b/CI/test_cases/container_cases/restartpolicy.sh
index be0140e7..bb7ada39 100755
--- a/CI/test_cases/container_cases/restartpolicy.sh
+++ b/CI/test_cases/container_cases/restartpolicy.sh
@@ -41,6 +41,52 @@ function do_test_on_failure()
fn_check_eq "$?" "0" "rm failed"
}
+function do_test_unless_stopped()
+{
+ containername=test_rp_unless_stopped
+ isula run --name $containername -td --restart unless-stopped busybox /bin/sh -c "exit 2"
+ fn_check_eq "$?" "0" "run failed"
+
+ sleep 8
+ count=`isula inspect --format='{{json .RestartCount}}' $containername`
+ if [[ $count == "0" ]];then
+ echo "expect not 0 but get $count"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ isula stop $containername
+ testcontainer $containername exited
+
+ isula rm $containername
+ fn_check_eq "$?" "0" "rm failed"
+}
+
+function do_test_unless_stopped_kill()
+{
+ containername=test_rp_unless_stopped
+ isula run --name $containername -td --restart unless-stopped busybox /bin/sh
+ fn_check_eq "$?" "0" "run failed"
+
+ cpid=`isula inspect -f '{{json .State.Pid}}' $containername`
+ kill -9 $cpid
+ sleep 8
+ testcontainer $containername running
+
+ isula stop $containername
+ fn_check_eq "$?" "0" "stop failed"
+ testcontainer $containername exited
+
+ isula restart $containername
+ testcontainer $containername running
+
+ isula kill $containername
+ fn_check_eq "$?" "0" "stop failed"
+ testcontainer $containername exited
+
+ isula rm $containername
+ fn_check_eq "$?" "0" "rm failed"
+}
+
function do_test_always_cancel()
{
containername=test_rp_always_cancel
@@ -64,6 +110,8 @@ function do_test_t()
{
do_test_on_failure
do_test_always_cancel
+ do_test_unless_stopped
+ do_test_unless_stopped_kill
return $TC_RET_T
}
--
2.25.1

162
0040-bugfix-for-embedded-image.patch Normal file
View File

@ -0,0 +1,162 @@
From 42a961197ce8d9c7e5bde3403b444d9e93c4c855 Mon Sep 17 00:00:00 2001
From: WangFengTu <wangfengtu@huawei.com>
Date: Tue, 23 Feb 2021 09:43:05 +0800
Subject: [PATCH 40/53] bugfix for embedded image
1. do not create mtab when create container if it's embedded image
2. use mounts in config, they are embedded image's layers
Signed-off-by: WangFengTu <wangfengtu@huawei.com>
---
.../modules/service/service_container.c | 13 +++--
src/daemon/modules/spec/specs_mount.c | 57 +++++++++++++++++++
.../image/oci/oci_config_merge/CMakeLists.txt | 1 +
test/specs/specs/CMakeLists.txt | 1 +
test/specs/specs_extend/CMakeLists.txt | 1 +
5 files changed, 68 insertions(+), 5 deletions(-)
diff --git a/src/daemon/modules/service/service_container.c b/src/daemon/modules/service/service_container.c
index 6551bfbf..e1d698cd 100644
--- a/src/daemon/modules/service/service_container.c
+++ b/src/daemon/modules/service/service_container.c
@@ -731,11 +731,14 @@ static int do_start_container(container_t *cont, const char *console_fifos[], bo
goto close_exit_fd;
}
- nret = create_mtab_link(oci_spec);
- if (nret != 0) {
- ERROR("Failed to create link /etc/mtab for target /proc/mounts");
- ret = -1;
- goto close_exit_fd;
+ // embedded conainter is readonly, create mtab link will fail
+ if (strcmp(IMAGE_TYPE_EMBEDDED, cont->common_config->image_type) != 0) {
+ nret = create_mtab_link(oci_spec);
+ if (nret != 0) {
+ ERROR("Failed to create link /etc/mtab for target /proc/mounts");
+ ret = -1;
+ goto close_exit_fd;
+ }
}
if (verify_mounts(cont)) {
diff --git a/src/daemon/modules/spec/specs_mount.c b/src/daemon/modules/spec/specs_mount.c
index 04ccd415..175a0fbe 100644
--- a/src/daemon/modules/spec/specs_mount.c
+++ b/src/daemon/modules/spec/specs_mount.c
@@ -2799,6 +2799,9 @@ static int calc_mounts_len(host_config *host_spec, container_config *container_s
if (container_spec->volumes != NULL && container_spec->volumes->len != 0) {
(*len) += container_spec->volumes->len;
}
+ if (container_spec->mounts != NULL && container_spec->mounts_len != 0) {
+ (*len) += container_spec->mounts_len;
+ }
return 0;
}
@@ -2809,6 +2812,54 @@ static void add_mount(defs_mount **merged_mounts, size_t *merged_mounts_len, def
*merged_mounts_len += 1;
}
+static int add_embedded_layers(container_config *container_spec, defs_mount **merged_mounts,
+ size_t *merged_mounts_len)
+{
+ int ret = 0;
+ size_t i = 0;
+ defs_mount *mnt = NULL;
+ defs_mount *conflict = NULL;
+ mount_spec *spec = NULL;
+ char *errmsg = NULL;
+
+ for (i = 0; container_spec->mounts != 0 && i < container_spec->mounts_len; i++) {
+ ret = util_parse_mount_spec(container_spec->mounts[i], &spec, &errmsg);
+ if (ret != 0) {
+ ERROR("parse mount spec failed: %s", errmsg);
+ ret = -1;
+ goto out;
+ }
+
+ mnt = parse_mount(spec);
+ if (mnt == NULL) {
+ ERROR("parse mount failed");
+ ret = -1;
+ goto out;
+ }
+
+ // do not use duplicate mount point
+ conflict = get_conflict_mount_point(merged_mounts, *merged_mounts_len, mnt);
+ if (conflict != NULL) {
+ ERROR("Duplicate mount point: %s", conflict->destination);
+ isulad_set_error_message("Duplicate mount point: %s", conflict->destination);
+ ret = -1;
+ goto out;
+ }
+
+ add_mount(merged_mounts, merged_mounts_len, mnt);
+ mnt = NULL;
+ free_mount_spec(spec);
+ spec = NULL;
+ }
+
+out:
+ free_defs_mount(mnt);
+ free_mount_spec(spec);
+ free(errmsg);
+
+ return ret;
+}
+
static int add_mounts(host_config *host_spec, defs_mount **merged_mounts, size_t *merged_mounts_len)
{
int ret = 0;
@@ -3086,6 +3137,12 @@ static int merge_all_fs_mounts(host_config *host_spec, container_config *contain
goto out;
}
+ // add embedded layers
+ ret = add_embedded_layers(container_spec, merged_mounts, &merged_mounts_len);
+ if (ret != 0) {
+ goto out;
+ }
+
// add --mounts
ret = add_mounts(host_spec, merged_mounts, &merged_mounts_len);
if (ret != 0) {
diff --git a/test/image/oci/oci_config_merge/CMakeLists.txt b/test/image/oci/oci_config_merge/CMakeLists.txt
index 48960ff7..36dc3ead 100644
--- a/test/image/oci/oci_config_merge/CMakeLists.txt
+++ b/test/image/oci/oci_config_merge/CMakeLists.txt
@@ -18,6 +18,7 @@ add_executable(${EXE}
${CMAKE_CURRENT_SOURCE_DIR}/../../../../src/utils/cutils/util_atomic.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../../src/utils/sha256/sha256.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../../src/utils/cutils/path.c
+ ${CMAKE_CURRENT_SOURCE_DIR}/../../../../src/utils/cutils/utils_mount_spec.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../../src/daemon/modules/spec//specs_extend.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../../src/daemon/common/err_msg.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../../src/daemon/config/isulad_config.c
diff --git a/test/specs/specs/CMakeLists.txt b/test/specs/specs/CMakeLists.txt
index e0f2b5b0..e0031e08 100644
--- a/test/specs/specs/CMakeLists.txt
+++ b/test/specs/specs/CMakeLists.txt
@@ -12,6 +12,7 @@ add_executable(${EXE}
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/utils_file.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/utils_timestamp.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/util_atomic.c
+ ${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/utils_mount_spec.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/sha256/sha256.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/path.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/map/map.c
diff --git a/test/specs/specs_extend/CMakeLists.txt b/test/specs/specs_extend/CMakeLists.txt
index 7d5c7dfb..45b21ecd 100644
--- a/test/specs/specs_extend/CMakeLists.txt
+++ b/test/specs/specs_extend/CMakeLists.txt
@@ -12,6 +12,7 @@ add_executable(${EXE}
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/utils_file.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/utils_timestamp.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/util_atomic.c
+ ${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/utils_mount_spec.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/sha256/sha256.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/path.c
${CMAKE_CURRENT_SOURCE_DIR}/../../../src/utils/cutils/map/map.c
--
2.25.1

75
0041-console-client-ignore-stdin-close-event.patch Normal file
View File

@ -0,0 +1,75 @@
From f5cd35cbfb594aad4b4448dd6550eb2faf9368c9 Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Tue, 23 Feb 2021 16:27:47 +0800
Subject: [PATCH 41/53] console: client ignore stdin close event
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---
src/utils/console/console.c | 17 +++++++++++++----
src/utils/console/console.h | 1 +
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/src/utils/console/console.c b/src/utils/console/console.c
index 7fda519c..1aedd0c9 100644
--- a/src/utils/console/console.c
+++ b/src/utils/console/console.c
@@ -57,7 +57,11 @@ static int console_cb_tty_stdin_with_escape(int fd, uint32_t events, void *cbdat
r_ret = util_read_nointr(ts->stdin_reader, &c, 1);
if (r_ret <= 0) {
- ret = EPOLL_LOOP_HANDLE_CLOSE;
+ if (r_ret == 0 && ts->ignore_stdin_close) {
+ ret = EPOLL_LOOP_HANDLE_CONTINUE;
+ } else {
+ ret = EPOLL_LOOP_HANDLE_CLOSE;
+ }
goto out;
}
@@ -122,7 +126,11 @@ static int console_cb_stdio_copy(int fd, uint32_t events, void *cbdata, struct e
ret = EPOLL_LOOP_HANDLE_CONTINUE;
goto out;
} else {
- ret = EPOLL_LOOP_HANDLE_CLOSE;
+ if (r_ret == 0 && ts->ignore_stdin_close && fd == ts->stdin_reader) {
+ ret = EPOLL_LOOP_HANDLE_CONTINUE;
+ } else {
+ ret = EPOLL_LOOP_HANDLE_CLOSE;
+ }
goto out;
}
}
@@ -363,8 +371,8 @@ int console_loop_with_std_fd(int stdinfd, int stdoutfd, int stderrfd, int fifoin
int tty_exit, bool tty)
{
int ret;
- struct epoll_descr descr;
- struct tty_state ts;
+ struct epoll_descr descr = { 0 };
+ struct tty_state ts = { 0 };
ret = epoll_loop_open(&descr);
if (ret) {
@@ -378,6 +386,7 @@ int console_loop_with_std_fd(int stdinfd, int stdoutfd, int stderrfd, int fifoin
ts.stdin_reader = -1;
ts.stdout_reader = -1;
ts.stderr_reader = -1;
+ ts.ignore_stdin_close = true;
if (fifoinfd >= 0) {
ts.stdin_reader = stdinfd;
diff --git a/src/utils/console/console.h b/src/utils/console/console.h
index 406a2fe9..0dfe19d3 100644
--- a/src/utils/console/console.h
+++ b/src/utils/console/console.h
@@ -40,6 +40,7 @@ struct tty_state {
int tty_exit;
/* Flag to mark whether detected escape sequence. */
int saw_tty_exit;
+ bool ignore_stdin_close;
};
int console_fifo_name(const char *rundir, const char *subpath, const char *stdflag, char *fifo_name,
--
2.25.1

182
0042-delete-lxc-from-runc-CI-test.patch Normal file
View File

@ -0,0 +1,182 @@
From d0533ced0b9c4b721d1f7560b503070f07944e45 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Wed, 24 Feb 2021 10:19:44 +0800
Subject: [PATCH 42/53] delete lxc from runc CI test
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
.../container_cases/bigdata_stream_runc.sh | 128 ++----------------
1 file changed, 12 insertions(+), 116 deletions(-)
diff --git a/CI/test_cases/container_cases/bigdata_stream_runc.sh b/CI/test_cases/container_cases/bigdata_stream_runc.sh
index f6c2ee94..1313774e 100755
--- a/CI/test_cases/container_cases/bigdata_stream_runc.sh
+++ b/CI/test_cases/container_cases/bigdata_stream_runc.sh
@@ -55,9 +55,9 @@ function record_origin_status()
origin_isulad_cpu_usage=$(ps -o %cpu -p $(cat /var/run/isulad.pid) | sed -n '2p')
msg_info "origin isulad cpu usage: $origin_isulad_cpu_usage"
- lxc_monitor_pid=$(ps aux | grep "lxc monitor" | grep $CID | awk '{print $2}')
- origin_lxc_monitor_cpu_usage=$(ps -o %cpu -p $lxc_monitor_pid | sed -n '2p')
- msg_info "origin lxc monitor cpu usage: $origin_lxc_monitor_cpu_usage"
+ isulad_shim_pid=$(ps aux | grep "isulad-shim" | grep $CID | awk '{print $2}')
+ origin_isulad_shim_cpu_usage=$(ps -o %cpu -p $isulad_shim_pid | sed -n '2p')
+ msg_info "origin isulad shim cpu usage: $origin_isulad_shim_cpu_usage"
rm -rf /iocopy_stream_data_*
}
@@ -80,22 +80,16 @@ function check_last_status()
ps -o %cpu -p $(cat /var/run/isulad.pid)
fi
- lxc_monintor_pid=$(ps aux | grep "lxc monitor" | grep $CID | awk '{print $2}')
- last_lxc_monitor_cpu_usage=$(ps -o %cpu -p $lxc_monintor_pid | sed -n '2p')
- allowable_lxc_monitor_cpu_usage=$(echo "$origin_lxc_monitor_cpu_usage*2" | bc)
- if [[ $(echo "$allowable_lxc_monitor_cpu_usage < 1.0" | bc) -eq 1 ]]; then
- allowable_lxc_monitor_cpu_usage=1.0
+ isulad_shim_pid=$(ps aux | grep "isulad-shim" | grep $CID | awk '{print $2}')
+ last_isulad_shim_cpu_usage=$(ps -o %cpu -p $isulad_shim_pid | sed -n '2p')
+ allowable_isulad_shim_cpu_usage=$(echo "$origin_isulad_shim_cpu_usage*2" | bc)
+ if [[ $(echo "$allowable_isulad_shim_cpu_usage < 1.0" | bc) -eq 1 ]]; then
+ allowable_isulad_shim_cpu_usage=1.0
fi
- msg_info "allowable lxc_monitor cpu usage: $allowable_lxc_monitor_cpu_usage"
- if [[ $(echo "$last_lxc_monitor_cpu_usage > $allowable_lxc_monitor_cpu_usage" | bc) -eq 1 ]]; then
+ msg_info "allowable isulad_shim cpu usage: $allowable_isulad_shim_cpu_usage"
+ if [[ $(echo "$last_isulad_shim_cpu_usage > $allowable_isulad_shim_cpu_usage" | bc) -eq 1 ]]; then
msg_err "${FUNCNAME[0]}:${LINENO} - Process exception: endless loop or residual thread" && ((ret++))
- ps -o %cpu -p $lxc_monintor_pid
- fi
-
- lxc_attach_process_number=$(ps aux | grep lxc-attach | grep $CID | wc -l)
- if [[ $lxc_attach_process_number -ne 0 ]]; then
- msg_err "${FUNCNAME[0]}:${LINENO} - lxc_attach process residual" && ((ret++))
- ps aux | grep lxc-attach | grep $CID
+ ps -o %cpu -p $isulad_shim_pid
fi
client_pid=$(pidof isula)
@@ -209,100 +203,6 @@ function test_stream_with_kill_client()
return ${ret}
}
-function test_stream_with_stop_attach()
-{
- local ret=0
- local test="test_stream_with_stop_attach => (${FUNCNAME[@]})"
- msg_info "${test} starting..."
-
- nohup isula exec $CID cat test_500M > /tmp/iocopy_stream_data_500M &
- exec_pid=$!
- sleep 2
- pid=$(ps aux | grep lxc-attach | grep $CID |grep "cat test_500M" | awk '{print $2}')
- kill -19 $pid
- sleep 3
- kill -18 $pid
-
- wait $exec_pid
-
- ls -l /tmp/iocopy_stream_data_500M
- total_size=$(stat -c"%s" /tmp/iocopy_stream_data_500M)
- [[ $total_size -ne 524288000 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - stream iocopy loss data" && ((ret++))
-
- check_last_status
- [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - abnormal status" && ((ret++))
-
- msg_info "${test} finished with return ${ret}..."
- return ${ret}
-}
-
-function test_stream_with_kill_attach()
-{
- local ret=0
- local test="test_stream_with_kill_client => (${FUNCNAME[@]})"
- msg_info "${test} starting..."
-
- nohup isula exec $CID cat test_500M > /tmp/iocopy_stream_data_500M &
- sleep 3
- pid=$(ps aux | grep lxc-attach | grep $CID |grep "cat test_500M" | awk '{print $2}')
- kill -9 $pid
-
- check_last_status
- [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - abnormal status" && ((ret++))
-
- msg_info "${test} finished with return ${ret}..."
- return ${ret}
-}
-
-function test_stream_with_stop_lxc_monitor()
-{
- local ret=0
- local test="test_stream_with_stop_lxc_monitor => (${FUNCNAME[@]})"
- msg_info "${test} starting..."
-
- nohup isula exec $CID cat test_500M > /tmp/iocopy_stream_data_500M &
- exec_pid=$!
- sleep 2
- pid=$(ps aux | grep "lxc monitor" | grep $CID | awk '{print $2}')
- kill -19 $pid
- sleep 3
- kill -18 $pid
-
- wait $exec_pid
-
- ls -l /tmp/iocopy_stream_data_500M
- total_size=$(stat -c"%s" /tmp/iocopy_stream_data_500M)
- [[ $total_size -ne 524288000 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - stream iocopy loss data" && ((ret++))
-
- check_last_status
- [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - abnormal status" && ((ret++))
-
- msg_info "${test} finished with return ${ret}..."
- return ${ret}
-}
-
-function test_stream_with_kill_lxc_monitor()
-{
- local ret=0
- local test="test_stream_with_kill_lxc_monitor => (${FUNCNAME[@]})"
- msg_info "${test} starting..."
-
- nohup isula exec $CID cat test_500M > /tmp/iocopy_stream_data_500M &
- sleep 3
- pid=$(ps aux | grep "lxc monitor" | grep $CID | awk '{print $2}')
- kill -9 $pid
- sleep 3
-
- isula start $CID
- [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to start container: $CID" && ((ret++))
-
- check_last_status
- [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - abnormal status" && ((ret++))
-
- msg_info "${test} finished with return ${ret}..."
- return ${ret}
-}
-
function test_stream_with_stop_isulad()
{
local ret=0
@@ -375,7 +275,7 @@ function test_memory_leak_with_bigdata_stream()
start_isulad_with_valgrind
- CID=$(isula run -itd ${image} sh)
+ CID=$(isula run -itd --runtime runc ${image} sh)
isula exec -it $CID dd if=/dev/zero of=test_100M bs=1M count=100
[[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to create bigdata" && ((ret++))
@@ -407,10 +307,6 @@ test_cat_bigdata || ((ans++))
test_cat_bigdata_without_pty || ((ans++))
test_stream_with_stop_client || ((ans++))
test_stream_with_kill_client || ((ans++))
-test_stream_with_stop_attach || ((ans++))
-test_stream_with_kill_attach || ((ans++))
-test_stream_with_stop_lxc_monitor || ((ans++))
-test_stream_with_kill_lxc_monitor || ((ans++))
test_stream_with_stop_isulad || ((ans++))
test_stream_with_kill_isulad || ((ans++))
tear_down || ((ans++))
--
2.25.1

635
0043-add-embedded-testcases.patch Normal file
View File

@ -0,0 +1,635 @@
From f803b85eea63bbe2745678afcfcc57e5eed51c4b Mon Sep 17 00:00:00 2001
From: WangFengTu <wangfengtu@huawei.com>
Date: Mon, 22 Feb 2021 10:26:55 +0800
Subject: [PATCH 43/53] add embedded testcases
Signed-off-by: WangFengTu <wangfengtu@huawei.com>
---
CI/build.sh | 6 +-
CI/test_cases/image_cases/embedded.sh | 590 ++++++++++++++++++++++++++
2 files changed, 595 insertions(+), 1 deletion(-)
create mode 100755 CI/test_cases/image_cases/embedded.sh
diff --git a/CI/build.sh b/CI/build.sh
index d21419eb..2c535c70 100755
--- a/CI/build.sh
+++ b/CI/build.sh
@@ -37,6 +37,9 @@ BASE_IMAGE=""
devmapper_script="${TOPDIR}/CI/install_devmapper.sh"
disk=NULL
+modprobe squashfs
+losetup -D
+losetup -l
rm -rf ${TESTCASE_ASSIGN}_*
# #Run this file will generate default BASE_IMAGE and auto run isulad unit tests
@@ -370,7 +373,8 @@ if [[ "x$disk" != "xNULL" ]] && [[ "x${enable_gcov}" != "xON" ]] ; then
for index in $(seq 1 ${CONTAINER_INDEX})
do
suffix=$(ls ${CIDIR} | grep testcase_assign_ | grep -E "*[S|P]${index}$" | awk -F '_' '{print $NF}')
- cat ${CIDIR}/testcase_assign_${suffix} >> ${CIDIR}/testcase_assign_devmapper
+ # only one embedded.sh shell is allowed at the same time and embedded image will not use in devicemapper enviorment
+ cat ${CIDIR}/testcase_assign_${suffix} | grep -v embedded.sh >> ${CIDIR}/testcase_assign_devmapper
done
docker cp ${CIDIR}/testcase_assign_devmapper ${devmappercontainer}:/root
echo_success "Run container ${devmappercontainer} success"
diff --git a/CI/test_cases/image_cases/embedded.sh b/CI/test_cases/image_cases/embedded.sh
new file mode 100755
index 00000000..535077ce
--- /dev/null
+++ b/CI/test_cases/image_cases/embedded.sh
@@ -0,0 +1,590 @@
+#!/bin/bash
+#
+# attributes: isulad embedded image
+# concurrent: YES
+# spend time: 15
+
+#######################################################################
+##- @Copyright (C) Huawei Technologies., Ltd. 2020. All rights reserved.
+# - iSulad licensed under the Mulan PSL v2.
+# - You can use this software according to the terms and conditions of the Mulan PSL v2.
+# - You may obtain a copy of Mulan PSL v2 at:
+# - http://license.coscl.org.cn/MulanPSL2
+# - THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR
+# - IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR
+# - PURPOSE.
+# - See the Mulan PSL v2 for more details.
+##- @Description:CI
+##- @Author: wangfengtu
+##- @Create: 2021-02-20
+#######################################################################
+
+declare -r curr_path=$(dirname $(readlink -f "$0"))
+source ../helpers.sh
+test="embedded image test => test_embedded"
+
+free_loop=""
+embedded_basedir="$(pwd)/embedded/img"
+embedded_basedir2="$(pwd)/embedded/img2"
+embedded_basedir3="$(pwd)/embedded/img3"
+embedded_basedir4="$(pwd)/embedded/img4"
+embedded_basedir5="$(pwd)/embedded/img5"
+embedded_manifest="$embedded_basedir/test.manifest"
+embedded_manifest2="$embedded_basedir2/test.manifest"
+embedded_manifest3="$embedded_basedir3/manifest"
+embedded_manifest_template="$embedded_basedir/template.manifest"
+embedded_manifest_invalid="$embedded_basedir/invalid.manifest"
+embedded_manifest_invalid_sgn="$embedded_basedir/invalid.sgn"
+embedded_manifest_not_file="$embedded_basedir/notfile"
+embedded_manifest_not_exist="$embedded_basedir/notexist.manifest"
+embedded_manifest_sgn2="$embedded_basedir2/test.sgn"
+embedded_manifest_sgn5="$embedded_basedir5/sgn"
+embedded_app="$embedded_basedir/app.img"
+embedded_app2="$embedded_basedir2/app.img"
+embedded_platform="$embedded_basedir/platform.img"
+embedded_platform2="$embedded_basedir2/platform.img"
+embedded_rootfs0="/tmp/embedded_rootfs0"
+embedded_manifest_ori="$embedded_basedir/test.manifest.ori"
+embedded_manifest_template_ori="$embedded_basedir/template.manifest.ori"
+
+function test_load_image()
+{
+ local ret=0
+
+ isula load -i "$embedded_manifest" -t abc
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load invalid type failed" && ((ret++))
+
+ # load embedded image
+ isula load -i "$embedded_manifest" -t embedded
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load embedded image failed" && ((ret++))
+
+ # load embedded image again
+ isula load -i "$embedded_manifest" -t embedded
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load embedded image again failed" && ((ret++))
+
+ # delete embedded image
+ isula rmi test:v1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - delete embedded image failed" && ((ret++))
+
+ # load embedded image again
+ isula load -i "$embedded_manifest" -t embedded
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load embedded image again failed" && ((ret++))
+
+ # delete embedded image
+ isula rmi test:v1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - delete embedded image failed" && ((ret++))
+
+ return ${ret}
+}
+
+function test_run_image()
+{
+ local ret=0
+
+ isula run -t -n embedded_test1 nonexistentname1:v1 /bin/sh
+ [[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - run nonexistent image should failed" && ((ret++))
+
+ isula load -i "$embedded_manifest" -t embedded
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load embedded image failed" && ((ret++))
+
+ # run container based on embedded image
+ isula run --name embedded_test1 test:v1 ls /home/home/home
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - run embedded image failed" && ((ret++))
+
+ # delete container based on embedded image
+ isula rm embedded_test1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - delete container based on embedded image failed" && ((ret++))
+
+ # test image's env
+ isula run --name embedded_test1 test:v1 /bin/sh -c "echo \$c | grep \"d e\""
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test image's env failed" && ((ret++))
+
+ # delete container based on embedded image
+ isula rm embedded_test1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - delete container based on embedded image failed" && ((ret++))
+
+ # delete embedded image
+ isula rmi test:v1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - delete embedded image failed" && ((ret++))
+
+ return ${ret}
+}
+
+function test_mount()
+{
+ local ret=0
+
+ # load embedded image
+ isula load -i "$embedded_manifest" -t embedded
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load embedded imagefailed" && ((ret++))
+
+ # run --mount
+ isula run --mount type=bind,src="$embedded_basedir",dst=/usr,ro=true,bind-propagation=rprivate --name embedded_test2 test:v1 true
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - run --mount failed" && ((ret++))
+
+ testcontainer embedded_test2 exited
+
+ isula rm embedded_test2
+
+ # test invalid mode
+ isula run --mount type=bind,src="$embedded_basedir",dst=/usr,ro=invalid --name embedded_test2 test:v1 true
+ [[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - invalid mode should failed" && ((ret++))
+
+ isula rm embedded_test2
+
+ # test invalid bind propagation mode
+ isula run --mount type=bind,src="$embedded_basedir",dst=/usr,bind-propagation=invalid --name embedded_test2 test:v1 true
+ [[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - invalid bind propagation mode should failed" && ((ret++))
+
+ isula rm embedded_test2
+
+ # test source not exist
+ isula run --mount type=bind,src=abcdefg/notexist,dst=/usr --name embedded_test2 test:v1 true
+ [[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - invalid source not exist should failed" && ((ret++))
+
+ isula rm embedded_test2
+
+ # test source not a regular file
+ isula run --mount type=squashfs,src=/tmp,dst=/usr --name embedded_test2 test:v1 true
+ [[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - source not a regular file should failed" && ((ret++))
+
+ isula rm embedded_test2
+
+ # test path //tmp/test
+ mkdir -p /tmp/test_mount
+ mkdir -p /tmp/test_mount1/test
+ isula run -v /tmp/test_mount:/tmp --mount type=bind,src=/tmp/test_mount1,dst=//tmp/test_mount1,ro=true,bind-propagation=rprivate --name embedded_test2 test:v1 ls /tmp/test_mount1/test
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test path //tmp/test failed" && ((ret++))
+
+ isula rm embedded_test2
+
+ # delete embedded image
+ isula rmi test:v1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - delete embedded image failed" && ((ret++))
+
+ return ${ret}
+}
+
+function test_query_image()
+{
+ local ret=0
+
+ # load embedded image
+ isula load -i "$embedded_manifest" -t embedded
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load embedded imagefailed" && ((ret++))
+
+ # inspect embedded image
+ isula inspect test:v1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - inspect embedded image failed" && ((ret++))
+
+ # test list embedded image
+ isula images | grep test | grep v1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - list embedded image failed" && ((ret++))
+
+ # inspect nonexist item
+ isula inspect -f '{{json .abc}}' test:v1
+ [[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - inspect nonexist item should failed" && ((ret++))
+
+ # test inspect container, it should conatainer image info
+ isula run --name embedded_inspect test:v1 ls /home/home/home
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - run container for inspect failed" && ((ret++))
+
+ isula inspect -f '{{json .Image}}' embedded_inspect
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - image does not contain image info failed" && ((ret++))
+
+ # test list container based on embedded image
+ isula ps -a | grep embedded_inspect
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - ps does not contain embedded container failed" && ((ret++))
+
+ # delete container
+ isula rm embedded_inspect
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - delete container for inspect failed" && ((ret++))
+
+ # delete embedded image
+ isula rmi test:v1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - delete embedded image failed" && ((ret++))
+
+ # test inspect nonexist image
+ isula inspect test:v1
+ [[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - inspect nonexist image should failed" && ((ret++))
+
+ # test list nonexist image
+ isula images | grep test | grep v1
+ [[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - list nonexist image should failed" && ((ret++))
+
+ # test list nonexist container
+ isula ps -a | grep embedded_inspect
+ [[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - ps should not contain embedded container failed" && ((ret++))
+
+ return ${ret}
+}
+
+function test_invalid_manifest_part1()
+{
+ local ret=0
+
+ # test 'none' image name
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s/test:v1/none/g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test 'none' image name failed" && ((ret++))
+
+ # test 'none:latest' image name
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s/test:v1/none:latest/g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test 'none:latest' image name failed" && ((ret++))
+
+ # test invalid image name k~k
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s/test:v1/k~k/g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid image name k~k failed" && ((ret++))
+
+ # test invalid image name test
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s/test:v1/test/g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid image name test failed" && ((ret++))
+
+ # test invalid time
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s/Z/Zabc#$@/g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid image time failed" && ((ret++))
+
+ # test invalid layer number
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "16,36d" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid layer number failed" && ((ret++))
+
+ # test layer 0 not a device
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s#$free_loop#/home#g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test layer 0 not a device failed" && ((ret++))
+
+ # test layer(not the first layer) not a regular file
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ mkdir -p "$embedded_manifest_not_file"
+ sed -i "s#platform.img#notfile#g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test layer(not the first layer) not a regular file failed" && ((ret++))
+ rm -rf "$embedded_manifest_not_file"
+
+ # test invalid layer digest
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s/4613a9d1e9016293f53833b0ac61ea072882d468fe2fce7701ecea6f201eebbe/7a7eb18fd0a7b9ac0cdae8c9754ff846d65a4831b9ad8786d943618b497bd886/g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid layer digest failed" && ((ret++))
+
+ # test invalid layer not exist
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s/app.img/kkk/g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid layer not exist failed" && ((ret++))
+
+ # test invalid host path(not relative path)
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s#platform.img#/platform.img#g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid host path(not relative path) failed" && ((ret++))
+
+ # test invalid container path(not absolute path)
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s#/home/home#home/home#g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid container path(not absolute path) failed" && ((ret++))
+
+ # test invalid first layer(not absolute path)
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s#$free_loop#${free_loop:1}#g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid first layer(not absolute path) failed" && ((ret++))
+
+ # test invalid manifest digest
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ echo -n "sha256:36c7c17757c24fa1e86018c8009f3b98690709236f05910937d59e401d87d6c5" > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid manifest digest failed" && ((ret++))
+
+ # test invalid manifest not exist
+ isula load -i "$embedded_manifest_not_exist" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid manifest not exist failed" && ((ret++))
+
+ # test invalid manifest not a regular file
+ isula load -i /dev/zero -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid manifest not a regular file failed" && ((ret++))
+
+ # test invalid manifest empty file
+ rm -f "$embedded_manifest_invalid"
+ touch "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid manifest empty file failed" && ((ret++))
+
+ # test invalid manifest not a json file
+ rm -f "$embedded_manifest_invalid"
+ echo hello > "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid manifest not a json file failed" && ((ret++))
+
+ # test image conflict when in different path
+ rm -rf "$embedded_basedir2"
+ cp -rf "$embedded_basedir" "$embedded_basedir2"
+ isula load -i "$embedded_manifest" -t embedded
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test image conflict when in different path failed" && ((ret++))
+
+ echo -n sha256:$(sha256sum "$embedded_manifest2" | awk '{print $1}') > "$embedded_manifest_sgn2"
+ isula load -i "$embedded_manifest2" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load manifest in different path should failed" && ((ret++))
+
+ rm -rf "$embedded_basedir2"
+ isula rmi test:v1
+
+ return ${ret}
+}
+
+function test_invalid_manifest_part2()
+{
+ local ret=0
+
+ # test manifest's sgn file not exist
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ rm -f "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test manifest's sgn file not exist failed" && ((ret++))
+
+ # test content of manifest's sgn file not right
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ # note: add '\n' at the end of the sgn file
+ echo sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test content of manifest's sgn file not right failed" && ((ret++))
+
+ # test invalid schema version
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "2d" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid schema version failed" && ((ret++))
+
+ # test invalid manifest's media type
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s/embedded/invalid/g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid manifest's media type failed" && ((ret++))
+
+ # test invalid manifest's layer type
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s/squashfs/invalid/g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test invalid manifest's layer type failed" && ((ret++))
+
+ # test size negative number
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "s/823/-823/g" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test size negative number failed" && ((ret++))
+
+ # test first layer digest not empty
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "19d" "$embedded_manifest_invalid"
+ sed -i "19i\"digest\": \"a\"," "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test size negative number failed" && ((ret++))
+
+ # test first layer path in container not empty
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "21d" "$embedded_manifest_invalid"
+ sed -i "21i\"pathInContainer\": \"a\"" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 1 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test first layer path in container not empty failed" && ((ret++))
+
+ return ${ret}
+}
+
+function test_entrypoint()
+{
+ local ret=0
+
+ # load embedded image
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "12i\"/bin/ls\"," "$embedded_manifest_invalid"
+ sed -i "13i\"/home\"" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load embedded image failed" && ((ret++))
+
+ # test image's entrypoint
+ isula run --name embedded_entrypoint1 test:v1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test image's entrypoint failed" && ((ret++))
+
+ isula rm embedded_entrypoint1
+
+ # test image's entrypoint with cmds
+ isula run --name embedded_entrypoint1 test:v1 /bin
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test image's entrypoint with cmds failed" && ((ret++))
+
+ isula rm embedded_entrypoint1
+
+ # test image's entrypoint override image's entrypoint
+ isula run --entrypoint=/bin/ls --name embedded_entrypoint1 test:v1 /bin
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test image's entrypoint override image's entrypoint failed" && ((ret++))
+
+ isula rm embedded_entrypoint1
+ isula rmi test:v1
+
+ # test entrypoint with variable
+ cp -f "$embedded_manifest_template" "$embedded_manifest_invalid"
+ sed -i "12i\"/bin/sh\"," "$embedded_manifest_invalid"
+ sed -i "13i\"-c\"," "$embedded_manifest_invalid"
+ sed -i "14i\"ls /ho\${env_id}\"" "$embedded_manifest_invalid"
+ echo -n sha256:$(sha256sum "$embedded_manifest_invalid" | awk '{print $1}') > "$embedded_manifest_invalid_sgn"
+ isula load -i "$embedded_manifest_invalid" -t embedded
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test entrypoint with variable failed" && ((ret++))
+
+ isula run -e env_id=me --name embedded_entrypoint1 test:v1
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - test run embedded image with env failed" && ((ret++))
+
+ isula rm embedded_entrypoint1
+ isula rmi test:v1
+
+ return ${ret}
+}
+
+function test_symbolic()
+{
+ local ret=0
+
+ # test symbolic
+ # image layout
+ # .
+ # |__ img
+ # | |__ app.img
+ # | |__ platform.img
+ # |
+ # |__ img2
+ # | |__ app.img -> ../img/app.img
+ # | |__ platform.img -> ../img/platform.img
+ # | |__ test.manifest
+ # | |__ test.sgn -> ../img5/sgn
+ # |
+ # |__ img3
+ # | |__ manifest -> ../img2/test.manifest
+ # |
+ # |__ img4 -> img3
+ # |
+ # |__ img5
+ # |__ sgn
+ #
+ # /tmp/embedded_rootfs0 -> /dev/loopx
+
+ rm -rf "$embedded_basedir2"
+ mkdir -p "$embedded_basedir2"
+ ln -sf "$embedded_app" "$embedded_app2"
+ ln -sf "$embedded_platform" "$embedded_platform2"
+ cp -f "$embedded_manifest_template" "$embedded_manifest2"
+ sed -i "s#$free_loop#$embedded_rootfs0#g" "$embedded_manifest2"
+ ln -sf $free_loop $embedded_rootfs0
+ mkdir -p "$embedded_basedir5"
+ echo -n sha256:$(sha256sum "$embedded_manifest2" | awk '{print $1}') > "$embedded_manifest_sgn5"
+ ln -sf "$embedded_manifest_sgn5" "$embedded_manifest_sgn2"
+ mkdir -p "$embedded_basedir3"
+ ln -sf "$embedded_manifest2" "$embedded_manifest3"
+ ln -sf "$embedded_basedir3" "$embedded_basedir4"
+
+ # load embedded image
+ isula load -i "$embedded_manifest2" -t embedded
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load embedded image failed" && ((ret++))
+
+ # run container based on embedded image
+ isula run --name embedded_test_symbolic test:v1 ls /home/home/home
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - run container based on embedded image failed" && ((ret++))
+
+ isula rm embedded_test_symbolic
+ isula rmi test:v1
+
+ return ${ret}
+}
+
+function prepare_test_embedded()
+{
+ local ret=0
+
+ isula rm -f `isula ps -a -q`
+ isula rmi test:v1
+
+ free_loop=$(losetup -f)
+ losetup $free_loop $embedded_basedir/busybox.img
+
+ cp -f $embedded_manifest_ori $embedded_manifest
+ cp -f $embedded_manifest_template_ori $embedded_manifest_template
+ sed -i "s#/dev/ram0#$free_loop#g" "$embedded_manifest"
+ sed -i "s#/dev/ram0#$free_loop#g" "$embedded_manifest_template"
+ checksum=$(sha256sum $embedded_basedir/test.manifest | awk '{print $1}')
+ echo -n "sha256:$checksum" > $embedded_basedir/test.sgn
+
+ return ${ret}
+}
+
+function post_test_embedded()
+{
+ local ret=0
+
+ rm -rf "$embedded_manifest_not_file"
+ rm -rf "$embedded_basedir2"
+ rm -rf "$embedded_basedir3"
+ rm -rf "$embedded_basedir4"
+ rm -rf "$embedded_basedir5"
+
+ isula rm -f `isula ps -a -q`
+ isula rmi test:v1
+
+ umount $(mount | grep busybox.img | awk '{print $3}')
+ losetup -d $free_loop
+
+ return ${ret}
+}
+
+declare -i ans=0
+
+msg_info "${test} starting..."
+[[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - start isulad failed" && ((ret++))
+
+prepare_test_embedded || ((ans++))
+
+test_load_image || ((ans++))
+test_run_image || ((ans++))
+test_mount || ((ans++))
+test_query_image || ((ans++))
+test_invalid_manifest_part1 || ((ans++))
+test_invalid_manifest_part2 || ((ans++))
+test_entrypoint || ((ans++))
+test_symbolic || ((ans++))
+
+post_test_embedded
+
+msg_info "${test} finished with return ${ans}..."
+
+show_result ${ans} "${curr_path}/${0}"
--
2.25.1

100
0044-fix-the-error-of-ContainerStats-interface-field-valu.patch Normal file
View File

@ -0,0 +1,100 @@
From f5ebba3e76bdc39fc62b25202c9794ca2e773106 Mon Sep 17 00:00:00 2001
From: wujing <wujing50@huawei.com>
Date: Wed, 24 Feb 2021 17:41:56 +0800
Subject: [PATCH 44/53] fix the error of ContainerStats interface field value
Signed-off-by: wujing <wujing50@huawei.com>
---
.../cri/cri_container_manager_service_impl.cc | 22 ++++++++++++++-----
.../cri/cri_container_manager_service_impl.h | 5 +++--
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/daemon/entry/cri/cri_container_manager_service_impl.cc b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
index 6c6569a6..af21e78a 100644
--- a/src/daemon/entry/cri/cri_container_manager_service_impl.cc
+++ b/src/daemon/entry/cri/cri_container_manager_service_impl.cc
@@ -711,7 +711,7 @@ void ContainerManagerServiceImpl::PackContainerStatsAttributes(
}
}
-void ContainerManagerServiceImpl::SetFsUsage(const imagetool_fs_info *fs_usage,
+void ContainerManagerServiceImpl::SetFsUsage(const imagetool_fs_info *fs_usage, int64_t timestamp,
std::unique_ptr<runtime::v1alpha2::ContainerStats> &container)
{
if (fs_usage == nullptr || fs_usage->image_filesystems_len == 0 || fs_usage->image_filesystems[0] == nullptr) {
@@ -733,10 +733,18 @@ void ContainerManagerServiceImpl::SetFsUsage(const imagetool_fs_info *fs_usage,
container->mutable_writable_layer()->mutable_inodes_used()->set_value(
fs_usage->image_filesystems[0]->inodes_used->value);
}
+ container->mutable_writable_layer()->set_timestamp(timestamp);
+
+ if (fs_usage->image_filesystems[0]->fs_id != nullptr &&
+ fs_usage->image_filesystems[0]->fs_id->mountpoint != nullptr) {
+ container->mutable_writable_layer()->mutable_fs_id()->set_mountpoint(
+ fs_usage->image_filesystems[0]->fs_id->mountpoint);
+ }
}
void ContainerManagerServiceImpl::PackContainerStatsFilesystemUsage(
- const char *id, const char *image_type, std::unique_ptr<runtime::v1alpha2::ContainerStats> &container)
+ const char *id, const char *image_type, int64_t timestamp,
+ std::unique_ptr<runtime::v1alpha2::ContainerStats> &container)
{
if (id == nullptr || image_type == nullptr) {
return;
@@ -747,7 +755,7 @@ void ContainerManagerServiceImpl::PackContainerStatsFilesystemUsage(
ERROR("Failed to get container filesystem usage");
}
- SetFsUsage(fs_usage, container);
+ SetFsUsage(fs_usage, timestamp, container);
free_imagetool_fs_info(fs_usage);
}
@@ -771,21 +779,23 @@ void ContainerManagerServiceImpl::ContainerStatsToGRPC(
if (error.NotEmpty()) {
return;
}
- PackContainerStatsFilesystemUsage(response->container_stats[i]->id, response->container_stats[i]->image_type,
- container);
+ int64_t timestamp = util_get_now_time_nanos();
+ PackContainerStatsFilesystemUsage(response->container_stats[i]->id, response->container_stats[i]->image_type,
+ timestamp, container);
if (response->container_stats[i]->mem_used != 0u) {
uint64_t workingset = response->container_stats[i]->mem_used;
if (response->container_stats[i]->inactive_file_total < response->container_stats[i]->mem_used) {
workingset = response->container_stats[i]->mem_used - response->container_stats[i]->inactive_file_total;
}
container->mutable_memory()->mutable_working_set_bytes()->set_value(workingset);
+ container->mutable_memory()->set_timestamp(timestamp);
}
if (response->container_stats[i]->cpu_use_nanos != 0u) {
container->mutable_cpu()->mutable_usage_core_nano_seconds()->set_value(
response->container_stats[i]->cpu_use_nanos);
- container->mutable_cpu()->set_timestamp((int64_t)(response->container_stats[i]->cpu_system_use));
+ container->mutable_cpu()->set_timestamp(timestamp);
}
containerstats->push_back(move(container));
diff --git a/src/daemon/entry/cri/cri_container_manager_service_impl.h b/src/daemon/entry/cri/cri_container_manager_service_impl.h
index 49551469..d08d9124 100644
--- a/src/daemon/entry/cri/cri_container_manager_service_impl.h
+++ b/src/daemon/entry/cri/cri_container_manager_service_impl.h
@@ -103,9 +103,10 @@ private:
Errors &error);
void PackContainerStatsAttributes(const char *id, std::unique_ptr<runtime::v1alpha2::ContainerStats> &container,
Errors &error);
- void PackContainerStatsFilesystemUsage(const char *id, const char *image_type,
+ void PackContainerStatsFilesystemUsage(const char *id, const char *image_type, int64_t timestamp,
std::unique_ptr<runtime::v1alpha2::ContainerStats> &container);
- void SetFsUsage(const imagetool_fs_info *fs_usage, std::unique_ptr<runtime::v1alpha2::ContainerStats> &container);
+ void SetFsUsage(const imagetool_fs_info *fs_usage, int64_t timestamp,
+ std::unique_ptr<runtime::v1alpha2::ContainerStats> &container);
void ContainerStatusToGRPC(container_inspect *inspect,
std::unique_ptr<runtime::v1alpha2::ContainerStatus> &contStatus, Errors &error);
void PackContainerImageToStatus(container_inspect *inspect,
--
2.25.1

275
0045-rollback-setuped-network-if-mult-network-failed.patch Normal file
View File

@ -0,0 +1,275 @@
From e5304673ad2069b98256a942d13f959856578383 Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Tue, 23 Feb 2021 19:42:43 +0800
Subject: [PATCH 45/53] rollback setuped network if mult-network failed
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
src/daemon/entry/cri/cni_network_plugin.cc | 121 ++++++++++-----------
src/daemon/entry/cri/cni_network_plugin.h | 2 +-
2 files changed, 56 insertions(+), 67 deletions(-)
diff --git a/src/daemon/entry/cri/cni_network_plugin.cc b/src/daemon/entry/cri/cni_network_plugin.cc
index b764b2a4..de599361 100644
--- a/src/daemon/entry/cri/cni_network_plugin.cc
+++ b/src/daemon/entry/cri/cni_network_plugin.cc
@@ -382,32 +382,33 @@ void CniNetworkPlugin::Status(Errors &err)
CheckInitialized(err);
}
-
+// return: represent need rollback
bool CniNetworkPlugin::SetupMultNetworks(const std::string &ns, const std::string &defaultInterface,
const std::string &name,
const std::string &netnsPath, const std::string &podSandboxID,
const std::map<std::string, std::string> &annotations,
const std::map<std::string, std::string> &options, Errors &err)
{
- bool ret = false;
int defaultIdx = -1;
size_t len = 0;
+ struct result *preResult = nullptr;
+ CNINetwork *useDefaultNet = nullptr;
+ bool ret = true;
cri_pod_network_element **networks = CRIHelpers::GetNetworkPlaneFromPodAnno(annotations, &len, err);
if (err.NotEmpty()) {
ERROR("Couldn't get network plane from pod annotations: %s", err.GetCMessage());
err.Errorf("Couldn't get network plane from pod annotations: %s", err.GetCMessage());
- goto cleanup;
+ return false;
}
for (size_t i = 0; i < len; i++) {
if (networks[i] == nullptr || networks[i]->name == nullptr || networks[i]->interface == nullptr) {
continue;
}
- struct result *preResult = nullptr;
auto netIter = m_mutlNetworks.find(networks[i]->name);
if (netIter == m_mutlNetworks.end()) {
err.Errorf("Cannot found user defined net: %s", networks[i]->name);
- break;
+ goto cleanup;
}
if (defaultInterface == networks[i]->interface) {
defaultIdx = i;
@@ -416,32 +417,32 @@ bool CniNetworkPlugin::SetupMultNetworks(const std::string &ns, const std::strin
AddToNetwork((netIter->second).get(), name, ns, networks[i]->interface, podSandboxID, netnsPath, annotations, options,
&preResult, err);
free_result(preResult);
+ preResult = nullptr;
if (err.NotEmpty()) {
ERROR("Do setup user defined net: %s, failed: %s", networks[i]->name, err.GetCMessage());
- break;
+ goto cleanup;
}
- INFO("Setup user defained net: %s success", networks[i]->name);
+ INFO("Setup user defined net: %s success", networks[i]->name);
}
+ useDefaultNet = m_defaultNetwork.get();
// mask default network pod, if user defined net use same interface
if (defaultIdx >= 0) {
auto netIter = m_mutlNetworks.find(networks[defaultIdx]->name);
if (netIter == m_mutlNetworks.end()) {
- err.Errorf("Cannot found user defined net: %s", networks[defaultIdx]->name);
+ err.Errorf("Cannot default net: %s", networks[defaultIdx]->name);
goto cleanup;
}
-
- struct result *preResult = nullptr;
- AddToNetwork((netIter->second).get(), name, ns, networks[defaultIdx]->interface, podSandboxID, netnsPath, annotations,
- options, &preResult, err);
- free_result(preResult);
- if (err.NotEmpty()) {
- ERROR("Do setup user defined net: %s, failed: %s", networks[defaultIdx]->name, err.GetCMessage());
- goto cleanup;
- }
- INFO("Setup default net: %s success", networks[defaultIdx]->name);
- ret = true;
+ useDefaultNet = (netIter->second).get();
+ }
+ AddToNetwork(useDefaultNet, name, ns, defaultInterface, podSandboxID, netnsPath, annotations, options, &preResult, err);
+ free_result(preResult);
+ if (err.NotEmpty()) {
+ ERROR("Setup default net failed: %s", err.GetCMessage());
+ goto cleanup;
}
+ INFO("Setup default net: %s success", useDefaultNet->GetName().c_str());
+ ret = false;
cleanup:
free_cri_pod_network(networks, len);
return ret;
@@ -489,8 +490,8 @@ void CniNetworkPlugin::SetUpPod(const std::string &ns, const std::string &name,
return;
}
- struct result *preResult = nullptr;
if (m_loNetwork != nullptr) {
+ struct result *preResult = nullptr;
AddToNetwork(m_loNetwork.get(), name, ns, interfaceName, id, netnsPath, annotations, options, &preResult, err);
free_result(preResult);
preResult = nullptr;
@@ -506,38 +507,32 @@ void CniNetworkPlugin::SetUpPod(const std::string &ns, const std::string &name,
return;
}
- bool setedDefaultNet = SetupMultNetworks(ns, interfaceName, name, netnsPath, id, annotations, options, err);
- if (err.NotEmpty()) {
- goto unlock;
- }
-
- if (setedDefaultNet) {
- goto unlock;
- }
-
- AddToNetwork(m_defaultNetwork.get(), name, ns, interfaceName, id, netnsPath, annotations, options, &preResult, err);
- free_result(preResult);
- if (err.NotEmpty()) {
- ERROR("Error while adding to cni network: %s", err.GetCMessage());
+ bool needRollback = SetupMultNetworks(ns, interfaceName, name, netnsPath, id, annotations, options, err);
+ if (needRollback && err.NotEmpty()) {
+ Errors tmpErr;
+ TearDownMultNetworks(ns, interfaceName, name, netnsPath, id, annotations, tmpErr);
+ if (tmpErr.NotEmpty()) {
+ err.AppendError(tmpErr.GetMessage());
+ }
}
-unlock:
UnlockNetworkMap(err);
}
-bool CniNetworkPlugin::TearDownMultNetworks(const std::string &ns, const std::string &defaultInterface,
+void CniNetworkPlugin::TearDownMultNetworks(const std::string &ns, const std::string &defaultInterface,
const std::string &name,
const std::string &netnsPath, const std::string &podSandboxID, const std::map<std::string, std::string> &annotations,
Errors &err)
{
- bool ret = false;
int defaultIdx = -1;
size_t len = 0;
+ CNINetwork *useDefaultNet = nullptr;
+ Errors tmpErr;
cri_pod_network_element **networks = CRIHelpers::GetNetworkPlaneFromPodAnno(annotations, &len, err);
if (err.NotEmpty()) {
ERROR("Couldn't get network plane from pod annotations: %s", err.GetCMessage());
err.Errorf("Couldn't get network plane from pod annotations: %s", err.GetCMessage());
- goto cleanup;
+ return;
}
for (size_t i = 0; i < len; i++) {
@@ -553,14 +548,18 @@ bool CniNetworkPlugin::TearDownMultNetworks(const std::string &ns, const std::st
defaultIdx = i;
continue;
}
- DeleteFromNetwork((netIter->second).get(), name, ns, networks[i]->interface, podSandboxID, netnsPath, annotations, err);
- if (err.NotEmpty()) {
- ERROR("Do teardown user defined net: %s, failed: %s", networks[i]->name, err.GetCMessage());
- break;
+ DeleteFromNetwork((netIter->second).get(), name, ns, networks[i]->interface, podSandboxID, netnsPath, annotations,
+ tmpErr);
+ if (tmpErr.NotEmpty()) {
+ ERROR("Do teardown user defined net: %s, failed: %s", networks[i]->name, tmpErr.GetCMessage());
+ err.AppendError(tmpErr.GetMessage());
+ tmpErr.Clear();
+ continue;
}
INFO("Teardown user defained net: %s success", networks[i]->name);
}
+ useDefaultNet = m_defaultNetwork.get();
// mask default network pod, if user defined net use same interface
if (defaultIdx >= 0) {
auto netIter = m_mutlNetworks.find(networks[defaultIdx]->name);
@@ -568,19 +567,18 @@ bool CniNetworkPlugin::TearDownMultNetworks(const std::string &ns, const std::st
err.Errorf("Cannot found user defined net: %s", networks[defaultIdx]->name);
goto cleanup;
}
-
- DeleteFromNetwork((netIter->second).get(), name, ns, networks[defaultIdx]->interface, podSandboxID, netnsPath,
- annotations, err);
- if (err.NotEmpty()) {
- ERROR("Do teardown user defined net: %s, failed: %s", networks[defaultIdx]->name, err.GetCMessage());
- goto cleanup;
- }
- INFO("Teardown default net: %s success", networks[defaultIdx]->name);
- ret = true;
+ useDefaultNet = (netIter->second).get();
+ }
+ DeleteFromNetwork(useDefaultNet, name, ns, defaultInterface, podSandboxID, netnsPath, annotations, tmpErr);
+ if (tmpErr.NotEmpty()) {
+ ERROR("Teardown default net: %s, failed: %s", useDefaultNet->GetName().c_str(), tmpErr.GetCMessage());
+ err.AppendError(tmpErr.GetMessage());
+ goto cleanup;
}
+ INFO("Teardown default net: %s success", useDefaultNet->GetName().c_str());
+
cleanup:
free_cri_pod_network(networks, len);
- return ret;
}
void CniNetworkPlugin::TearDownPod(const std::string &ns, const std::string &name, const std::string &interfaceName,
@@ -605,21 +603,11 @@ void CniNetworkPlugin::TearDownPod(const std::string &ns, const std::string &nam
return;
}
- bool defaultNetDone = TearDownMultNetworks(ns, interfaceName, name, netnsPath, id, annotations, err);
- if (defaultNetDone) {
- goto unlock;
- }
+ TearDownMultNetworks(ns, interfaceName, name, netnsPath, id, annotations, err);
if (err.NotEmpty()) {
WARN("Teardown user defined networks failed: %s", err.GetCMessage());
}
- DeleteFromNetwork(m_defaultNetwork.get(), name, ns, interfaceName, id, netnsPath, annotations, tmpErr);
- if (tmpErr.NotEmpty()) {
- WARN("Teardown default network failed: %s", tmpErr.GetCMessage());
- err.AppendError(tmpErr.GetMessage());
- }
-
-unlock:
UnlockNetworkMap(err);
}
@@ -801,7 +789,8 @@ static void GetExtensionCNIArgs(const std::map<std::string, std::string> &annota
// get cni multinetwork extension
auto iter = annotations.find(CRIHelpers::Constants::CNI_MUTL_NET_EXTENSION_KEY);
if (iter != annotations.end()) {
- if (!CheckCNIArgValue(iter->second)) {
+ // args value must do not have ';'
+ if (iter->second.find(';') != std::string::npos) {
WARN("Ignore: invalid multinetwork cni args: %s", iter->second.c_str());
} else {
args[CRIHelpers::Constants::CNI_MUTL_NET_EXTENSION_ARGS_KEY] = iter->second;
@@ -817,9 +806,9 @@ static void GetExtensionCNIArgs(const std::map<std::string, std::string> &annota
continue;
}
auto strs = CXXUtils::Split(work.second, '=');
- iter = annotations.find(work.first);
- if (iter != annotations.end()) {
- WARN("Ignore: Same key cni args: %s", work.first.c_str());
+ iter = args.find(strs[0]);
+ if (iter != args.end()) {
+ WARN("Ignore: Same key cni args: %s", work.second.c_str());
continue;
}
args[strs[0]] = strs[1];
diff --git a/src/daemon/entry/cri/cni_network_plugin.h b/src/daemon/entry/cri/cni_network_plugin.h
index f545930f..8d51a94d 100644
--- a/src/daemon/entry/cri/cni_network_plugin.h
+++ b/src/daemon/entry/cri/cni_network_plugin.h
@@ -167,7 +167,7 @@ private:
const std::string &netnsPath, const std::string &podSandboxID, const std::map<std::string, std::string> &annotations,
const std::map<std::string, std::string> &options, Errors &err);
- bool TearDownMultNetworks(const std::string &ns, const std::string &defaultInterface, const std::string &name,
+ void TearDownMultNetworks(const std::string &ns, const std::string &defaultInterface, const std::string &name,
const std::string &netnsPath, const std::string &podSandboxID, const std::map<std::string, std::string> &annotations,
Errors &err);
--
2.25.1

242
0046-add-testcase-for-rollback-mutlnetworks.patch Normal file
View File

@ -0,0 +1,242 @@
From 389be7c170c28b24bfe762027e235f6fa986ac07 Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Wed, 24 Feb 2021 17:11:28 +0800
Subject: [PATCH 46/53] add testcase for rollback mutlnetworks
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
CI/test_cases/container_cases/cni_test.sh | 85 ++++++++++++++++++-
.../criconfigs/bins/isulad-cni | 4 +-
.../container_cases/criconfigs/bins/wrong-cni | 25 ++++++
.../criconfigs/mock_wrong.json | 9 ++
.../criconfigs/mutl_wrong_net_pod.json | 14 +++
CI/test_cases/helpers.sh | 2 +-
6 files changed, 132 insertions(+), 7 deletions(-)
create mode 100755 CI/test_cases/container_cases/criconfigs/bins/wrong-cni
create mode 100644 CI/test_cases/container_cases/criconfigs/mock_wrong.json
create mode 100644 CI/test_cases/container_cases/criconfigs/mutl_wrong_net_pod.json
diff --git a/CI/test_cases/container_cases/cni_test.sh b/CI/test_cases/container_cases/cni_test.sh
index c9e1e1ac..093178d6 100644
--- a/CI/test_cases/container_cases/cni_test.sh
+++ b/CI/test_cases/container_cases/cni_test.sh
@@ -177,6 +177,7 @@ function new_cni_config()
function check_annotation()
{
+ rm -f /etc/cni/net.d/*
cp ${data_path}/mock.json /etc/cni/net.d/bridge.json
sync;sync;
tail $ISUALD_LOG
@@ -199,19 +200,82 @@ function check_annotation()
fi
basepath=/tmp/cnilogs/
- cat ${basepath}/${sid}.env | grep CNI_MUTLINET_EXTENSION
+ cat ${basepath}/${sid}_eth0.env | grep CNI_MUTLINET_EXTENSION
if [ $? -ne 0 ];then
msg_err "lost extension for mutl network args"
TC_RET_T=$(($TC_RET_T+1))
fi
- cat ${basepath}/${sid}.env | grep "extension=first"
+ cat ${basepath}/${sid}_eth0.env | grep "extension=first"
if [ $? -ne 0 ];then
msg_err "lost extension for first cni args"
TC_RET_T=$(($TC_RET_T+1))
fi
- cat ${basepath}/${sid}.env | grep "extension=second"
+ cat ${basepath}/${sid}_eth0.env | grep "extension=second"
+ if [ $? -eq 0 ];then
+ msg_err "same extension key write to cni args"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ crictl stopp $sid
if [ $? -ne 0 ];then
- msg_err "lost extension for second cni args"
+ msg_err "stop sandbox failed"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ crictl rmp $sid
+ if [ $? -ne 0 ];then
+ msg_err "rm sandbox failed"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ return $TC_RET_T
+}
+
+function check_rollback()
+{
+ rm -f /etc/cni/net.d/*
+ cp ${data_path}/mock.json /etc/cni/net.d/bridge.json
+ sed -i "s#mock#default#g" /etc/cni/net.d/bridge.json
+ cp ${data_path}/mock.json /etc/cni/net.d/
+ cp ${data_path}/mock_wrong.json /etc/cni/net.d/
+ sync;sync;
+ tail $ISUALD_LOG
+ # wait cni updated
+ s=`date "+%s"`
+ for ((i=0;i<30;i++)); do
+ sleep 1
+ cur=`date "+%s"`
+ let "t=cur-s"
+ if [ $t -gt 6 ];then
+ break
+ fi
+ done
+ tail $ISUALD_LOG
+
+ crictl runp ${data_path}/mutl_wrong_net_pod.json
+ if [ $? -eq 0 ]; then
+ msg_err "Run sandbox success with invalid cni configs"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+ sid=`crictl pods -q | head -1`
+
+ basepath=/tmp/cnilogs/
+
+ cat ${basepath}/${sid}_eth0.env | grep "CNI_COMMAND=DEL"
+ if [ $? -ne 0 ];then
+ msg_err "do not rollback for eth0"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ cat ${basepath}/${sid}_eth1.env | grep "CNI_COMMAND=DEL"
+ if [ $? -ne 0 ];then
+ msg_err "do not rollback for eth1"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ cat ${basepath}/${sid}_eth2.env | grep "CNI_COMMAND=DEL"
+ if [ $? -ne 0 ];then
+ msg_err "do not rollback for eth2"
TC_RET_T=$(($TC_RET_T+1))
fi
@@ -227,6 +291,9 @@ function check_annotation()
TC_RET_T=$(($TC_RET_T+1))
fi
+ rm -f /etc/cni/net.d/*
+ cp ${data_path}/bridge.json /etc/cni/net.d/
+
return $TC_RET_T
}
@@ -247,6 +314,16 @@ if [ $? -ne 0 ];then
let "ret=$ret + 1"
fi
+check_annotation
+if [ $? -ne 0 ];then
+ let "ret=$ret + 1"
+fi
+
+check_rollback
+if [ $? -ne 0 ];then
+ let "ret=$ret + 1"
+fi
+
do_post
show_result $ret "cni base test"
diff --git a/CI/test_cases/container_cases/criconfigs/bins/isulad-cni b/CI/test_cases/container_cases/criconfigs/bins/isulad-cni
index e4b7e598..be48b77a 100755
--- a/CI/test_cases/container_cases/criconfigs/bins/isulad-cni
+++ b/CI/test_cases/container_cases/criconfigs/bins/isulad-cni
@@ -8,8 +8,8 @@ envpath=${basepath}/${secs}.env
env | grep CNI_CONTAINERID >/dev/null 2>&1
if [ $? -eq 0 ];then
- confpath=${basepath}/${CNI_CONTAINERID}.netconf
- envpath=${basepath}/${CNI_CONTAINERID}.env
+ confpath=${basepath}/${CNI_CONTAINERID}_${CNI_IFNAME}.netconf
+ envpath=${basepath}/${CNI_CONTAINERID}_${CNI_IFNAME}.env
fi
read -r line
diff --git a/CI/test_cases/container_cases/criconfigs/bins/wrong-cni b/CI/test_cases/container_cases/criconfigs/bins/wrong-cni
new file mode 100755
index 00000000..558c3401
--- /dev/null
+++ b/CI/test_cases/container_cases/criconfigs/bins/wrong-cni
@@ -0,0 +1,25 @@
+#!/bin/bash
+
+basepath=/tmp/cnilogs/
+mkdir -p $basepath
+secs=`date "+%s"`
+confpath=${basepath}/${secs}.netconf
+envpath=${basepath}/${secs}.env
+
+env | grep CNI_CONTAINERID >/dev/null 2>&1
+if [ $? -eq 0 ];then
+ confpath=${basepath}/${CNI_CONTAINERID}_${CNI_IFNAME}.netconf
+ envpath=${basepath}/${CNI_CONTAINERID}_${CNI_IFNAME}.env
+fi
+
+read -r line
+echo $line > ${confpath}
+
+env > ${envpath}
+
+echo "{
+ \"cniVersion\": \"0.3.1\",
+ \"dns\": {}
+}"
+
+exit 1
\ No newline at end of file
diff --git a/CI/test_cases/container_cases/criconfigs/mock_wrong.json b/CI/test_cases/container_cases/criconfigs/mock_wrong.json
new file mode 100644
index 00000000..7b2a92e4
--- /dev/null
+++ b/CI/test_cases/container_cases/criconfigs/mock_wrong.json
@@ -0,0 +1,9 @@
+{
+ "cniVersion": "0.3.1",
+ "name": "wrong",
+ "type": "wrong-cni",
+ "ipam": {
+ "type": "wrong-cni",
+ "subnet": "10.8.0.0/16"
+ }
+}
diff --git a/CI/test_cases/container_cases/criconfigs/mutl_wrong_net_pod.json b/CI/test_cases/container_cases/criconfigs/mutl_wrong_net_pod.json
new file mode 100644
index 00000000..01a6096c
--- /dev/null
+++ b/CI/test_cases/container_cases/criconfigs/mutl_wrong_net_pod.json
@@ -0,0 +1,14 @@
+{
+ "port_mappings":[{"protocol": 1, "container_port": 80, "host_port": 8080}],
+ "metadata": {
+ "name": "test",
+ "namespace": "default",
+ "attempt": 1,
+ "uid": "hdishd83djaidwnduwk28bcsb"
+ },
+ "linux": {
+ },
+ "annotations": {
+ "network.alpha.kubernetes.io/network": "[{\"name\":\"mock\",\"interface\":\"eth1\"},{\"name\":\"wrong\",\"interface\":\"eth2\"}]"
+ }
+}
diff --git a/CI/test_cases/helpers.sh b/CI/test_cases/helpers.sh
index 5a782281..fe256e8c 100755
--- a/CI/test_cases/helpers.sh
+++ b/CI/test_cases/helpers.sh
@@ -174,7 +174,7 @@ function init_cni_conf()
mkdir -p /etc/cni/net.d/
rm -rf /etc/cni/net.d/*
mkdir -p /opt/cni/bin
- cp $dtpath/bins/isulad-cni /opt/cni/bin
+ cp $dtpath/bins/* /opt/cni/bin/
cp $dtpath/good.conflist /etc/cni/net.d/
check_valgrind_log
--
2.25.1

328
0047-log-adjust-log-level-from-EVENT-to-WARN-to-reduce-lo.patch Normal file
View File

@ -0,0 +1,328 @@
From 9a605646c7e20773c52799ee4abcff20e26071de Mon Sep 17 00:00:00 2001
From: Li Feng <lifeng2221dd1@zoho.com.cn>
Date: Sat, 27 Feb 2021 14:19:24 +0800
Subject: [PATCH 47/53] log: adjust log level from EVENT to WARN to reduce log
number
Signed-off-by: Li Feng <lifeng2221dd1@zoho.com.cn>
---
src/cmd/isulad/main.c | 2 +-
.../connect/grpc/runtime_image_service.cc | 13 ++++-----
.../connect/grpc/runtime_runtime_service.cc | 28 +++++++++----------
src/daemon/executor/container_cb/list.c | 2 +-
src/daemon/modules/image/image.c | 12 ++++----
.../modules/image/oci/oci_common_operators.c | 4 +--
src/utils/cutils/utils_fs.c | 2 +-
7 files changed, 31 insertions(+), 32 deletions(-)
diff --git a/src/cmd/isulad/main.c b/src/cmd/isulad/main.c
index ce93eaa0..cb2b71a2 100644
--- a/src/cmd/isulad/main.c
+++ b/src/cmd/isulad/main.c
@@ -1515,7 +1515,7 @@ int main(int argc, char **argv)
clock_gettime(CLOCK_MONOTONIC, &t_end);
use_time = (double)(t_end.tv_sec - t_start.tv_sec) * (double)1000000000 + (double)(t_end.tv_nsec - t_start.tv_nsec);
use_time /= 1000000000;
- INFO("iSulad successfully booted in %.3f s", use_time);
+ EVENT("iSulad successfully booted in %.3f s", use_time);
#ifdef GRPC_CONNECTOR
INFO("Starting grpc server...");
#else
diff --git a/src/daemon/entry/connect/grpc/runtime_image_service.cc b/src/daemon/entry/connect/grpc/runtime_image_service.cc
index 8e740caf..23447baf 100644
--- a/src/daemon/entry/connect/grpc/runtime_image_service.cc
+++ b/src/daemon/entry/connect/grpc/runtime_image_service.cc
@@ -21,7 +21,6 @@
#include "cri_helpers.h"
#include "cri_image_manager_service_impl.h"
-
RuntimeImageServiceImpl::RuntimeImageServiceImpl()
{
std::unique_ptr<ImageManagerService> service(new ImageManagerServiceImpl);
@@ -55,7 +54,7 @@ grpc::Status RuntimeImageServiceImpl::ListImages(grpc::ServerContext *context,
std::vector<std::unique_ptr<runtime::v1alpha2::Image>> images;
Errors error;
- EVENT("Event: {Object: CRI, Type: Listing all images}");
+ WARN("Event: {Object: CRI, Type: Listing all images}");
rService->ListImages(request->filter(), &images, error);
if (!error.Empty()) {
@@ -71,7 +70,7 @@ grpc::Status RuntimeImageServiceImpl::ListImages(grpc::ServerContext *context,
*image = *(iter->get());
}
- EVENT("Event: {Object: CRI, Type: Listed all images}");
+ WARN("Event: {Object: CRI, Type: Listed all images}");
return grpc::Status::OK;
}
@@ -83,7 +82,7 @@ grpc::Status RuntimeImageServiceImpl::ImageStatus(grpc::ServerContext *context,
std::unique_ptr<runtime::v1alpha2::Image> image_info = nullptr;
Errors error;
- EVENT("Event: {Object: CRI, Type: Statusing image %s}", request->image().image().c_str());
+ WARN("Event: {Object: CRI, Type: Statusing image %s}", request->image().image().c_str());
image_info = rService->ImageStatus(request->image(), error);
if (!error.Empty() && !CRIHelpers::IsImageNotFoundError(error.GetMessage())) {
@@ -97,7 +96,7 @@ grpc::Status RuntimeImageServiceImpl::ImageStatus(grpc::ServerContext *context,
*image = *image_info;
}
- EVENT("Event: {Object: CRI, Type: Statused image %s}", request->image().image().c_str());
+ WARN("Event: {Object: CRI, Type: Statused image %s}", request->image().image().c_str());
return grpc::Status::OK;
}
@@ -109,7 +108,7 @@ grpc::Status RuntimeImageServiceImpl::ImageFsInfo(grpc::ServerContext *context,
std::vector<std::unique_ptr<runtime::v1alpha2::FilesystemUsage>> usages;
Errors error;
- EVENT("Event: {Object: CRI, Type: Statusing image fs info}");
+ WARN("Event: {Object: CRI, Type: Statusing image fs info}");
rService->ImageFsInfo(&usages, error);
if (!error.Empty()) {
@@ -126,7 +125,7 @@ grpc::Status RuntimeImageServiceImpl::ImageFsInfo(grpc::ServerContext *context,
*fs_info = *(iter->get());
}
- EVENT("Event: {Object: CRI, Type: Statused image fs info}");
+ WARN("Event: {Object: CRI, Type: Statused image fs info}");
return grpc::Status::OK;
}
diff --git a/src/daemon/entry/connect/grpc/runtime_runtime_service.cc b/src/daemon/entry/connect/grpc/runtime_runtime_service.cc
index c09153eb..c9702401 100644
--- a/src/daemon/entry/connect/grpc/runtime_runtime_service.cc
+++ b/src/daemon/entry/connect/grpc/runtime_runtime_service.cc
@@ -50,8 +50,8 @@ void RuntimeRuntimeServiceImpl::Init(Network::NetworkPluginConf mConf, isulad_da
Network::ProbeNetworkPlugins(mConf.GetPluginConfDir(), mConf.GetPluginBinDir(), &plugins);
std::shared_ptr<Network::NetworkPlugin> chosen { nullptr };
- Network::InitNetworkPlugin(&plugins, mConf.GetPluginName(), mConf.GetHairpinMode(),
- mConf.GetNonMasqueradeCIDR(), mConf.GetMTU(), &chosen, err);
+ Network::InitNetworkPlugin(&plugins, mConf.GetPluginName(), mConf.GetHairpinMode(), mConf.GetNonMasqueradeCIDR(),
+ mConf.GetMTU(), &chosen, err);
if (err.NotEmpty()) {
ERROR("Init network plugin failed: %s", err.GetCMessage());
return;
@@ -181,7 +181,7 @@ grpc::Status RuntimeRuntimeServiceImpl::ListContainers(grpc::ServerContext *cont
{
Errors error;
- EVENT("Event: {Object: CRI, Type: Listing all Container}");
+ WARN("Event: {Object: CRI, Type: Listing all Container}");
std::vector<std::unique_ptr<runtime::v1alpha2::Container>> containers;
rService->ListContainers(request->has_filter() ? &request->filter() : nullptr, &containers, error);
@@ -199,7 +199,7 @@ grpc::Status RuntimeRuntimeServiceImpl::ListContainers(grpc::ServerContext *cont
*container = *(iter->get());
}
- EVENT("Event: {Object: CRI, Type: Listed all Container}");
+ WARN("Event: {Object: CRI, Type: Listed all Container}");
return grpc::Status::OK;
}
@@ -210,7 +210,7 @@ grpc::Status RuntimeRuntimeServiceImpl::ListContainerStats(grpc::ServerContext *
{
Errors error;
- EVENT("Event: {Object: CRI, Type: Listing all Container stats}");
+ WARN("Event: {Object: CRI, Type: Listing all Container stats}");
std::vector<std::unique_ptr<runtime::v1alpha2::ContainerStats>> containers;
rService->ListContainerStats(request->has_filter() ? &request->filter() : nullptr, &containers, error);
@@ -228,7 +228,7 @@ grpc::Status RuntimeRuntimeServiceImpl::ListContainerStats(grpc::ServerContext *
*container = *(iter->get());
}
- EVENT("Event: {Object: CRI, Type: Listed all Container stats}");
+ WARN("Event: {Object: CRI, Type: Listed all Container stats}");
return grpc::Status::OK;
}
@@ -239,7 +239,7 @@ grpc::Status RuntimeRuntimeServiceImpl::ContainerStatus(grpc::ServerContext *con
{
Errors error;
- EVENT("Event: {Object: CRI, Type: Statusing Container: %s}", request->container_id().c_str());
+ WARN("Event: {Object: CRI, Type: Statusing Container: %s}", request->container_id().c_str());
std::unique_ptr<runtime::v1alpha2::ContainerStatus> contStatus =
rService->ContainerStatus(request->container_id(), error);
@@ -249,7 +249,7 @@ grpc::Status RuntimeRuntimeServiceImpl::ContainerStatus(grpc::ServerContext *con
}
*(reply->mutable_status()) = *contStatus;
- EVENT("Event: {Object: CRI, Type: Statused Container: %s}", request->container_id().c_str());
+ WARN("Event: {Object: CRI, Type: Statused Container: %s}", request->container_id().c_str());
return grpc::Status::OK;
}
@@ -339,7 +339,7 @@ grpc::Status RuntimeRuntimeServiceImpl::PodSandboxStatus(grpc::ServerContext *co
{
Errors error;
- EVENT("Event: {Object: CRI, Type: Status Pod: %s}", request->pod_sandbox_id().c_str());
+ WARN("Event: {Object: CRI, Type: Status Pod: %s}", request->pod_sandbox_id().c_str());
std::unique_ptr<runtime::v1alpha2::PodSandboxStatus> podStatus;
podStatus = rService->PodSandboxStatus(request->pod_sandbox_id(), error);
@@ -350,7 +350,7 @@ grpc::Status RuntimeRuntimeServiceImpl::PodSandboxStatus(grpc::ServerContext *co
}
*(reply->mutable_status()) = *podStatus;
- EVENT("Event: {Object: CRI, Type: Statused Pod: %s}", request->pod_sandbox_id().c_str());
+ WARN("Event: {Object: CRI, Type: Statused Pod: %s}", request->pod_sandbox_id().c_str());
return grpc::Status::OK;
}
@@ -361,7 +361,7 @@ grpc::Status RuntimeRuntimeServiceImpl::ListPodSandbox(grpc::ServerContext *cont
{
Errors error;
- EVENT("Event: {Object: CRI, Type: Listing all Pods}");
+ WARN("Event: {Object: CRI, Type: Listing all Pods}");
std::vector<std::unique_ptr<runtime::v1alpha2::PodSandbox>> pods;
rService->ListPodSandbox(request->has_filter() ? &request->filter() : nullptr, &pods, error);
@@ -378,7 +378,7 @@ grpc::Status RuntimeRuntimeServiceImpl::ListPodSandbox(grpc::ServerContext *cont
*pod = *(iter->get());
}
- EVENT("Event: {Object: CRI, Type: Listed all Pods}");
+ WARN("Event: {Object: CRI, Type: Listed all Pods}");
return grpc::Status::OK;
}
@@ -470,7 +470,7 @@ grpc::Status RuntimeRuntimeServiceImpl::Status(grpc::ServerContext *context,
{
Errors error;
- EVENT("Event: {Object: CRI, Type: Statusing daemon}");
+ WARN("Event: {Object: CRI, Type: Statusing daemon}");
std::unique_ptr<runtime::v1alpha2::RuntimeStatus> status = rService->Status(error);
if (status == nullptr || error.NotEmpty()) {
@@ -479,7 +479,7 @@ grpc::Status RuntimeRuntimeServiceImpl::Status(grpc::ServerContext *context,
}
*(reply->mutable_status()) = *status;
- EVENT("Event: {Object: CRI, Type: Statused daemon}");
+ WARN("Event: {Object: CRI, Type: Statused daemon}");
return grpc::Status::OK;
}
diff --git a/src/daemon/executor/container_cb/list.c b/src/daemon/executor/container_cb/list.c
index 34c1b956..d8f26328 100644
--- a/src/daemon/executor/container_cb/list.c
+++ b/src/daemon/executor/container_cb/list.c
@@ -503,7 +503,7 @@ static container_container *get_container_info(const char *name, const struct li
cont = containers_store_get(name);
if (cont == NULL) {
- ERROR("Container '%s' already removed", name);
+ ERROR("Container '%s' not exist", name);
return NULL;
}
cont_state = container_dup_state(cont->state);
diff --git a/src/daemon/modules/image/image.c b/src/daemon/modules/image/image.c
index 4563efea..8e663863 100644
--- a/src/daemon/modules/image/image.c
+++ b/src/daemon/modules/image/image.c
@@ -392,7 +392,7 @@ int im_get_filesystem_info(const char *image_type, im_fs_info_response **respons
goto out;
}
- EVENT("Event: {Object: get image filesystem info, Type: inspecting}");
+ WARN("Event: {Object: get image filesystem info, Type: inspecting}");
ret = q->ops->get_filesystem_info(response);
if (ret != 0) {
if (response != NULL && *response != NULL) {
@@ -402,7 +402,7 @@ int im_get_filesystem_info(const char *image_type, im_fs_info_response **respons
}
goto out;
}
- EVENT("Event: {Object: get image filesystem info, Type: inspected}");
+ WARN("Event: {Object: get image filesystem info, Type: inspected}");
out:
return ret;
@@ -442,7 +442,7 @@ int im_get_container_filesystem_usage(const char *image_type, const char *id, im
request->name_id = util_strdup_s(id);
}
- EVENT("Event: {Object: container \'%s\' filesystem info, Type: inspecting}", id != NULL ? id : "");
+ WARN("Event: {Object: container \'%s\' filesystem info, Type: inspecting}", id != NULL ? id : "");
ret = q->ops->container_fs_usage(request, &filesystemusage);
if (ret != 0) {
ERROR("Failed to get filesystem usage for container %s", id);
@@ -452,7 +452,7 @@ int im_get_container_filesystem_usage(const char *image_type, const char *id, im
*fs_usage = filesystemusage;
filesystemusage = NULL;
- EVENT("Event: {Object: container \'%s\' filesystem info, Type: inspected}", id != NULL ? id : "");
+ WARN("Event: {Object: container \'%s\' filesystem info, Type: inspected}", id != NULL ? id : "");
out:
free_im_container_fs_usage_request(request);
@@ -1597,7 +1597,7 @@ int im_inspect_image(const im_inspect_request *request, im_inspect_response **re
image_ref = util_strdup_s(request->image.image);
- EVENT("Event: {Object: %s, Type: image inspecting}", image_ref);
+ WARN("Event: {Object: %s, Type: image inspecting}", image_ref);
bim_type = bim_query(image_ref);
if (bim_type == NULL) {
@@ -1619,7 +1619,7 @@ int im_inspect_image(const im_inspect_request *request, im_inspect_response **re
goto pack_response;
}
- EVENT("Event: {Object: %s, Type: image inspected}", image_ref);
+ WARN("Event: {Object: %s, Type: image inspected}", image_ref);
pack_response:
if (g_isulad_errmsg != NULL) {
diff --git a/src/daemon/modules/image/oci/oci_common_operators.c b/src/daemon/modules/image/oci/oci_common_operators.c
index aecb63d0..845e1fde 100644
--- a/src/daemon/modules/image/oci/oci_common_operators.c
+++ b/src/daemon/modules/image/oci/oci_common_operators.c
@@ -431,7 +431,7 @@ int oci_summary_image(im_summary_request *request, im_summary_response *response
goto pack_response;
}
- EVENT("Event: {Object: %s, Type: statusing image summary}", resolved_name);
+ WARN("Event: {Object: %s, Type: statusing image summary}", resolved_name);
image_summary = storage_img_get_summary(resolved_name);
if (image_summary == NULL) {
@@ -444,7 +444,7 @@ int oci_summary_image(im_summary_request *request, im_summary_response *response
response->image_summary = image_summary;
image_summary = NULL;
- EVENT("Event: {Object: %s, Type: statused image summary}", resolved_name);
+ WARN("Event: {Object: %s, Type: statused image summary}", resolved_name);
pack_response:
free(resolved_name);
diff --git a/src/utils/cutils/utils_fs.c b/src/utils/cutils/utils_fs.c
index 788557f9..bbbf2d2d 100644
--- a/src/utils/cutils/utils_fs.c
+++ b/src/utils/cutils/utils_fs.c
@@ -515,7 +515,7 @@ int util_mount(const char *src, const char *dst, const char *mtype, const char *
if ((mntflags & MS_REMOUNT) != MS_REMOUNT) {
if (util_detect_mounted(dst)) {
- ERROR("mount dst %s had been mounted, skip mount", dst);
+ WARN("mount dst %s had been mounted, skip mount", dst);
ret = 0;
goto out;
}
--
2.25.1

249
0048-isulad-shim-fix-shim-exit-bug.patch Normal file
View File

@ -0,0 +1,249 @@
From 358e79c5e21503348eae0f1b9e56206269060ec4 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Fri, 26 Feb 2021 04:37:03 -0500
Subject: [PATCH 48/53] isulad-shim: fix shim exit bug
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
src/cmd/isulad-shim/process.c | 3 +-
src/cmd/isulad-shim/process.h | 2 +-
src/cmd/isulad-shim/terminal.c | 109 ++++++++++++++++++++-------------
src/cmd/isulad-shim/terminal.h | 2 +-
4 files changed, 70 insertions(+), 46 deletions(-)
diff --git a/src/cmd/isulad-shim/process.c b/src/cmd/isulad-shim/process.c
index 3ac739b9..25e84fbd 100644
--- a/src/cmd/isulad-shim/process.c
+++ b/src/cmd/isulad-shim/process.c
@@ -266,8 +266,7 @@ static void *do_io_copy(void *data)
fd_node_t *fn = ioc->fd_to;
for (; fn != NULL; fn = fn->next) {
if (fn->is_log) {
- shim_write_container_log_file(io_thd->terminal, ioc->id == stdid_out ? "stdout" : "stderr", buf,
- r_count);
+ shim_write_container_log_file(io_thd->terminal, ioc->id, buf, r_count);
} else {
int w_count = write_nointr_in_total(fn->fd, buf, r_count);
if (w_count < 0) {
diff --git a/src/cmd/isulad-shim/process.h b/src/cmd/isulad-shim/process.h
index c17a20b1..6e596551 100644
--- a/src/cmd/isulad-shim/process.h
+++ b/src/cmd/isulad-shim/process.h
@@ -59,7 +59,7 @@ typedef struct {
io_copy_t *ioc;
bool shutdown;
bool is_stdin;
- log_terminal *terminal;
+ log_terminal *terminal;// just used by stdout and stderr
} io_thread_t;
typedef struct process {
diff --git a/src/cmd/isulad-shim/terminal.c b/src/cmd/isulad-shim/terminal.c
index ac39539a..ed40ad5e 100644
--- a/src/cmd/isulad-shim/terminal.c
+++ b/src/cmd/isulad-shim/terminal.c
@@ -30,19 +30,11 @@
#include <unistd.h>
#include "common.h"
+#include "process.h"
-#define BUF_CACHE_SIZE (16 * 1024)
-
-static ssize_t shim_write_nointr_lock(log_terminal *terminal, const void *buf, size_t count)
-{
- ssize_t ret;
-
- (void)pthread_rwlock_wrlock(&terminal->log_terminal_rwlock);
- ret = write_nointr_in_total(terminal->fd, buf, count);
- (void)pthread_rwlock_unlock(&terminal->log_terminal_rwlock);
-
- return ret;
-}
+#define BUF_CACHE_SIZE (32 * 1024)
+#define STDOUT_STR "stdout"
+#define STDERR_STR "stderr"
static int shim_rename_old_log_file(log_terminal *terminal)
{
@@ -137,23 +129,29 @@ static int64_t get_log_file_size(int fd)
static int shim_json_data_write(log_terminal *terminal, const char *buf, int read_count)
{
- int ret;
+ int ret = 0;
+ int nret = 0;
int64_t available_space = -1;
int64_t file_size;
+ (void)pthread_rwlock_wrlock(&terminal->log_terminal_rwlock);
+
+
file_size = get_log_file_size(terminal->fd);
if (file_size < 0) {
- return SHIM_ERR;
+ ret = -1;
+ goto out;
}
available_space = terminal->log_maxsize - file_size;
if (read_count <= available_space) {
- return shim_write_nointr_lock(terminal, buf, read_count);
+ ret = write_nointr_in_total(terminal->fd, buf, read_count);
+ goto out;
}
- ret = shim_dump_log_file(terminal);
- if (ret < 0) {
- return SHIM_ERR;
+ if (shim_dump_log_file(terminal) < 0) {
+ ret = -1;
+ goto out;
}
/*
@@ -161,13 +159,18 @@ static int shim_json_data_write(log_terminal *terminal, const char *buf, int rea
* We have set the log file min size 16k, so the scenario of log_maxsize < read_count
* shouldn't happen, otherwise, discard some last bytes.
*/
- ret = shim_write_nointr_lock(terminal, buf,
+ nret = write_nointr_in_total(terminal->fd, buf,
terminal->log_maxsize < read_count ? terminal->log_maxsize : read_count);
- if (ret < 0) {
- return SHIM_ERR;
+ if (nret < 0) {
+ ret = -1;
+ goto out;
}
- return (read_count - ret);
+ ret = read_count - nret;
+
+out:
+ (void)pthread_rwlock_unlock(&terminal->log_terminal_rwlock);
+ return ret;
}
static bool util_get_time_buffer(struct timespec *timestamp, char *timebuffer, size_t maxsize)
@@ -256,10 +259,18 @@ cleanup:
return ret;
}
-void shim_write_container_log_file(log_terminal *terminal, const char *type, char *buf, int read_count)
+// BUF_CACHE_SIZE must be larger than read_count of buf readed
+static char cache_out[BUF_CACHE_SIZE] = { 0 };
+static char cache_err[BUF_CACHE_SIZE] = { 0 };
+static int size_out = 0;
+static int size_err = 0;
+
+// Just used by stdout stderr threads
+void shim_write_container_log_file(log_terminal *terminal, int type, char *buf, int read_count)
{
- static char cache[BUF_CACHE_SIZE];
- static int size = 0;
+ char *cache = NULL;
+ int *size = NULL;
+ const char *type_str = NULL;
int upto, index;
int begin = 0;
int buf_readed = 0;
@@ -268,38 +279,52 @@ void shim_write_container_log_file(log_terminal *terminal, const char *type, cha
if (terminal == NULL) {
return;
}
+ switch (type) {
+ case stdid_out:
+ type_str = STDOUT_STR;
+ cache = cache_out;
+ size = &size_out;
+ break;
+ case stdid_err:
+ type_str = STDERR_STR;
+ cache = cache_err;
+ size = &size_err;
+ break;
+ default:
+ return;
+ }
if (buf != NULL && read_count > 0) {
- if (read_count > (BUF_CACHE_SIZE - size)) {
+ if (read_count > (BUF_CACHE_SIZE - *size)) {
upto = BUF_CACHE_SIZE;
} else {
- upto = size + read_count;
+ upto = *size + read_count;
}
- if (upto > size) {
- buf_readed = upto - size;
- memcpy(cache + size, buf, buf_readed);
+ if (upto > *size) {
+ buf_readed = upto - *size;
+ memcpy(cache + *size, buf, buf_readed);
buf_left = read_count - buf_readed;
- size += buf_readed;
+ *size += buf_readed;
}
}
- if (size == 0) {
+ if (*size == 0) {
return;
}
- for (index = 0; index < size; index++) {
+ for (index = 0; index < *size; index++) {
if (cache[index] == '\n') {
- (void)shim_logger_write(terminal, type, cache + begin, index - begin + 1);
+ (void)shim_logger_write(terminal, type_str, cache + begin, index - begin + 1);
begin = index + 1;
}
}
- if (buf == NULL || (begin == 0 && size == BUF_CACHE_SIZE)) {
- if (begin < size) {
- (void)shim_logger_write(terminal, type, cache + begin, size - begin);
+ if (buf == NULL || (begin == 0 && *size == BUF_CACHE_SIZE)) {
+ if (begin < *size) {
+ (void)shim_logger_write(terminal, type_str, cache + begin, *size - begin);
begin = 0;
- size = 0;
+ *size = 0;
}
if (buf == NULL) {
return;
@@ -307,13 +332,13 @@ void shim_write_container_log_file(log_terminal *terminal, const char *type, cha
}
if (begin > 0) {
- memcpy(cache, cache + begin, size - begin);
- size -= begin;
+ memcpy(cache, cache + begin, *size - begin);
+ *size -= begin;
}
if (buf_left > 0) {
- memcpy(cache + size, buf + buf_readed, buf_left);
- size += buf_left;
+ memcpy(cache + *size, buf + buf_readed, buf_left);
+ *size += buf_left;
}
}
diff --git a/src/cmd/isulad-shim/terminal.h b/src/cmd/isulad-shim/terminal.h
index d9ed8f1a..556117a0 100644
--- a/src/cmd/isulad-shim/terminal.h
+++ b/src/cmd/isulad-shim/terminal.h
@@ -34,7 +34,7 @@ typedef struct {
pthread_rwlock_t log_terminal_rwlock;
} log_terminal;
-void shim_write_container_log_file(log_terminal *terminal, const char *type, char *buf,
+void shim_write_container_log_file(log_terminal *terminal, int type, char *buf,
int bytes_read);
int shim_create_container_log_file(log_terminal *terminal);
--
2.25.1

25
0049-remove-redundant-code.patch Normal file
View File

@ -0,0 +1,25 @@
From 72ad417b26b17b0981cd163f42bc23d98e19b4e7 Mon Sep 17 00:00:00 2001
From: wujing <wujing50@huawei.com>
Date: Mon, 1 Mar 2021 17:17:17 +0800
Subject: [PATCH 49/53] remove redundant code
Signed-off-by: wujing <wujing50@huawei.com>
---
src/daemon/modules/runtime/isula/isula_rt_ops.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/src/daemon/modules/runtime/isula/isula_rt_ops.c b/src/daemon/modules/runtime/isula/isula_rt_ops.c
index 540f1f67..ecea2b3d 100644
--- a/src/daemon/modules/runtime/isula/isula_rt_ops.c
+++ b/src/daemon/modules/runtime/isula/isula_rt_ops.c
@@ -612,7 +612,6 @@ static int status_to_exit_code(int status)
exit_code = WEXITSTATUS(status);
} else {
exit_code = -1;
- exit_code = -1;
}
if (WIFSIGNALED(status)) {
int signal;
--
2.25.1

35
0050-devicemapper-umount-when-resize2fs-command-failed.patch Normal file
View File

@ -0,0 +1,35 @@
From adde17cdd844a51fa606c74a0f241c62dbf11a27 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Fri, 5 Mar 2021 23:13:31 -0500
Subject: [PATCH 50/53] devicemapper: umount when resize2fs command failed
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
.../storage/layer_store/graphdriver/devmapper/deviceset.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
index 0b0394c5..3a271c3a 100644
--- a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
+++ b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
@@ -2158,14 +2158,15 @@ static int grow_fs(struct device_set *devset, image_devmapper_device_info *info)
if (exec_grow_fs_command("resize2fs", dev_fname) != 0) {
ERROR("Failed execute resize2fs to grow rootfs");
ret = -1;
- goto out;
+ goto clean_mount;
}
} else {
ERROR("Unsupported filesystem type %s", devset->base_device_filesystem);
ret = -1;
- goto out;
+ goto clean_mount;
}
+clean_mount:
if (umount2(FS_MOUNT_POINT, MNT_DETACH) < 0 && errno != EINVAL) {
WARN("Failed to umount directory %s:%s", FS_MOUNT_POINT, strerror(errno));
}
--
2.25.1

36
0051-ignore-to-create-mtab-when-runtime-is-kata-runtime.patch Normal file
View File

@ -0,0 +1,36 @@
From a24118b4382492e27415f25411fcaadef990b659 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Mon, 15 Mar 2021 09:49:10 -0400
Subject: [PATCH 51/53] ignore to create mtab when runtime is kata-runtime
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
src/daemon/modules/service/service_container.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/src/daemon/modules/service/service_container.c b/src/daemon/modules/service/service_container.c
index e1d698cd..d6a82587 100644
--- a/src/daemon/modules/service/service_container.c
+++ b/src/daemon/modules/service/service_container.c
@@ -60,6 +60,8 @@
#include "utils_verify.h"
#include "volume_api.h"
+#define KATA_RUNTIME "kata-runtime"
+
int set_container_to_removal(const container_t *cont)
{
int ret = 0;
@@ -732,7 +734,8 @@ static int do_start_container(container_t *cont, const char *console_fifos[], bo
}
// embedded conainter is readonly, create mtab link will fail
- if (strcmp(IMAGE_TYPE_EMBEDDED, cont->common_config->image_type) != 0) {
+ // kata-runtime container's qemu donot support to create mtab in host
+ if (strcmp(IMAGE_TYPE_EMBEDDED, cont->common_config->image_type) != 0 && strcmp(KATA_RUNTIME, cont->runtime) != 0) {
nret = create_mtab_link(oci_spec);
if (nret != 0) {
ERROR("Failed to create link /etc/mtab for target /proc/mounts");
--
2.25.1

91
0052-remove-unchecked-layer-ignore-rootfs-layer.patch Normal file
View File

@ -0,0 +1,91 @@
From 64b45885abf0c4b3563008d2be5d04b5ec8cd28d Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Thu, 18 Mar 2021 11:05:33 +0800
Subject: [PATCH 52/53] remove unchecked layer ignore rootfs layer
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
.../modules/image/oci/storage/storage.c | 42 ++++++++++++++++++-
1 file changed, 41 insertions(+), 1 deletion(-)
diff --git a/src/daemon/modules/image/oci/storage/storage.c b/src/daemon/modules/image/oci/storage/storage.c
index 0e9708f5..40fc15a8 100644
--- a/src/daemon/modules/image/oci/storage/storage.c
+++ b/src/daemon/modules/image/oci/storage/storage.c
@@ -1480,6 +1480,26 @@ out:
return ret;
}
+static bool is_rootfs_layer(const char *layer_id, const struct rootfs_list *all_rootfs)
+{
+ int j;
+
+ if (all_rootfs == NULL || layer_id == NULL) {
+ return false;
+ }
+
+ for (j = 0; j < all_rootfs->rootfs_len; j++) {
+ if (all_rootfs->rootfs[j]->layer == NULL) {
+ continue;
+ }
+ if (strcmp(layer_id, all_rootfs->rootfs[j]->layer) == 0) {
+ return true;
+ }
+ }
+
+ return false;
+}
+
static bool do_storage_integration_check(const char *path, map_t *checked_layers)
{
struct rootfs_list *all_rootfs = NULL;
@@ -1558,6 +1578,7 @@ static void delete_unchecked_layers(map_t *checked_layers)
{
struct layer_list *all_layers = NULL;
size_t i;
+ struct rootfs_list *all_rootfs = NULL;
all_layers = util_common_calloc_s(sizeof(struct layer_list));
if (all_layers == NULL) {
@@ -1570,11 +1591,29 @@ static void delete_unchecked_layers(map_t *checked_layers)
goto out;
}
+ all_rootfs = util_common_calloc_s(sizeof(struct rootfs_list));
+ if (all_rootfs == NULL) {
+ ERROR("Out of memory");
+ goto out;
+ }
+
+ if (rootfs_store_get_all_rootfs(all_rootfs) != 0) {
+ ERROR("Failed to get all container rootfs information");
+ goto out;
+ }
+
for (i = 0; i < all_layers->layers_len; i++) {
if (map_search(checked_layers, (void *)all_layers->layers[i]->id) != NULL) {
+ DEBUG("ignore checked layer: %s", all_layers->layers[i]->id);
+ continue;
+ }
+
+ if (is_rootfs_layer(all_layers->layers[i]->id, all_rootfs)) {
+ DEBUG("ignore rootfs layer: %s", all_layers->layers[i]->id);
continue;
}
- WARN("Delete unchecked layer: %s due to no related image", all_layers->layers[i]->id);
+
+ ERROR("Delete unchecked layer: %s due to no related image", all_layers->layers[i]->id);
if (layer_store_delete(all_layers->layers[i]->id) != 0) {
ERROR("Failed to delete unchecked layer %s", all_layers->layers[i]->id);
}
@@ -1582,6 +1621,7 @@ static void delete_unchecked_layers(map_t *checked_layers)
out:
free_layer_list(all_layers);
+ free_rootfs_list(all_rootfs);
}
static bool storage_integration_check()
--
2.25.1

38
0053-add-test-to-check-running-container-with-image-integ.patch Normal file
View File

@ -0,0 +1,38 @@
From 19b3a0bfd08433d39a1115f2ad9ef3eaac006514 Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Thu, 18 Mar 2021 11:25:57 +0800
Subject: [PATCH 53/53] add test to check running container with image
integration check
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
CI/test_cases/image_cases/integration_check.sh | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/CI/test_cases/image_cases/integration_check.sh b/CI/test_cases/image_cases/integration_check.sh
index 2e6f962e..fe342cc2 100755
--- a/CI/test_cases/image_cases/integration_check.sh
+++ b/CI/test_cases/image_cases/integration_check.sh
@@ -58,6 +58,9 @@ function test_image_info()
ucid=$(isula create ${uimage})
[[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - create container failed" && ((ret++))
+ isula run -tid --name checker alpine
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - run container failed" && ((ret++))
+
tmp_fname=$(echo -n "/var/run/isulad/storage" | sha256sum | awk '{print $1}')
rm -f "${ISULAD_RUN_ROOT_PATH}/storage/${tmp_fname}.json"
@@ -74,6 +77,9 @@ function test_image_info()
isula ps -a | grep ${ucid}
[[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - container: ${ucid} do not exist with valid image" && ((ret++))
+ isula exec -it checker date
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - container: checker exec failed with valid image" && ((ret++))
+
isula images | grep busybox
[[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - invalid image: ${image} exist" && ((ret++))
--
2.25.1

35
iSulad.spec
View File

@ -1,5 +1,5 @@
%global _version 2.0.8
%global _release 20210202.153251.gite082dcf3
%global _release 20210323.094917.git7e6aa593
%global is_systemd 1
Name: iSulad
@ -38,6 +38,33 @@ Patch23: 0023-Fix-create-env-path-dir-if-dir-exist.patch
Patch24: 0024-iSulad-calculate-memusage-with-used-total_inactive_f.patch
Patch25: 0025-fix-container-exit-health-check-residue-and-multiple.patch
Patch26: 0026-CI-supplementary-testcase-for-health-check-monitor.patch
Patch27: 0027-add-container-lock-when-clean-container-resource.patch
Patch28: 0028-sleep-some-time-before-calculate-to-make-sure-fd-clo.patch
Patch29: 0029-stats-fix-wrong-memory-usage-info-in-stats.patch
Patch30: 0030-save-health-check-log-to-disk-before-unhealthy.patch
Patch31: 0031-unpack-try-to-remove-and-replace-dst_path-while-unpa.patch
Patch32: 0032-fd-leak-check-in-cp.sh-should-not-include-pull-fd-ch.patch
Patch33: 0033-devmapper-modify-log-msg.patch
Patch34: 0034-name_id_index-fix-restore-fail-to-remove-name-index.patch
Patch35: 0035-thread-function-calls-DAEMON_CLEAR_ERRORMSG-to-preve.patch
Patch36: 0036-modify-resume-task-name.patch
Patch37: 0037-cleadcode-Remove-extra-semicolons.patch
Patch38: 0038-restart-policy-add-support-unless-stopped-policy.patch
Patch39: 0039-CI-add-testcase-for-unless-stopped-restart-policy.patch
Patch40: 0040-bugfix-for-embedded-image.patch
Patch41: 0041-console-client-ignore-stdin-close-event.patch
Patch42: 0042-delete-lxc-from-runc-CI-test.patch
Patch43: 0043-add-embedded-testcases.patch
Patch44: 0044-fix-the-error-of-ContainerStats-interface-field-valu.patch
Patch45: 0045-rollback-setuped-network-if-mult-network-failed.patch
Patch46: 0046-add-testcase-for-rollback-mutlnetworks.patch
Patch47: 0047-log-adjust-log-level-from-EVENT-to-WARN-to-reduce-lo.patch
Patch48: 0048-isulad-shim-fix-shim-exit-bug.patch
Patch49: 0049-remove-redundant-code.patch
Patch50: 0050-devicemapper-umount-when-resize2fs-command-failed.patch
Patch51: 0051-ignore-to-create-mtab-when-runtime-is-kata-runtime.patch
Patch52: 0052-remove-unchecked-layer-ignore-rootfs-layer.patch
Patch53: 0053-add-test-to-check-running-container-with-image-integ.patch
%ifarch x86_64 aarch64
Provides: libhttpclient.so()(64bit)
@ -240,6 +267,12 @@ fi
%endif
%changelog
* Tue Mar 23 2021 haozi007 <liuhao27@huawei.com> - 20210323.094917.git7e6aa593
- Type: sync from upstream
- ID: NA
- SUG: NA
- DESC: update from master
* Tue Feb 2 2021 lifeng <lifeng68@huawei.com> - 2.0.8-20210202.153251.gite082dcf3
- Type: sync from upstream
- ID: NA