packages/iSulad
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

!66 update from openeuler

Browse Source 
From: @gaohuatao
Reviewed-by: @lifeng2221dd1
Signed-off-by: @lifeng2221dd1
This commit is contained in:
openeuler-ci-bot 2020-11-12 19:53:05 +08:00 committed by Gitee
commit 9fb5e75d28
29 changed files with 648444 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
0001-iSulad-modify-defattr-to-755-in-spec.patch Normal file
View File

@ -0,0 +1,26 @@
From 44428ad14fda11c07dae69dc69f5f18d8172cae8 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Thu, 15 Oct 2020 09:23:19 +0800
Subject: [PATCH 01/28] iSulad: modify defattr to 755 in spec
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
iSulad.spec | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/iSulad.spec b/iSulad.spec
index 35bd125..534d616 100644
--- a/iSulad.spec
+++ b/iSulad.spec
@@ -198,7 +198,7 @@ fi
%{_includedir}/isulad/*
%attr(0755,root,root) %{_libdir}/pkgconfig
%attr(0640,root,root) %{_libdir}/pkgconfig/isulad.pc
-%defattr(0550,root,root,0750)
+%defattr(0755,root,root,0755)
%{_bindir}/*
%{_libdir}/*
%attr(0640,root,root) %{_sysconfdir}/sysconfig/iSulad
--
2.20.1

254
0002-Dockerfile-update-dockerfile-to-isulad-v2.0.6-use-mu.patch Normal file
View File

@ -0,0 +1,254 @@
From a1cf27137ef4186f4b167b2b8becde8715c11908 Mon Sep 17 00:00:00 2001
From: Les1ie <me@les1ie.com>
Date: Thu, 15 Oct 2020 18:48:27 +0800
Subject: [PATCH 02/28] Dockerfile: update dockerfile to isulad v2.0.6; use
multi-stage to decrease the size of image; remove unused packages
how to push docker image to dockerhub:
1. register a new user on dockerhub, username: sampleuser
2. create a new repo on dockerhub, we can name it as: isulad
3. build docker image on our own computer, run
`docker build -t sampleuser/isulad:latest .`
4. login docker hub, run command
`docker login` and input our username and password
5. run `docker push sampleuser/isulad:latest`
we can change `latest` to other tag if we need, eg: 2.0.6
---
Dockerfile | 134 ++++++++++++++++++++++++++++-------------------------
1 file changed, 72 insertions(+), 62 deletions(-)
diff --git a/Dockerfile b/Dockerfile
index 20e714c..94b484b 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -21,7 +21,7 @@
# -t YOUR_IMAGE_NAME -f ./Dockerfile .
-FROM centos:7.6.1810
+FROM centos:7.6.1810 AS build
MAINTAINER LiFeng <lifeng68@huawei.com>
RUN echo "nameserver 8.8.8.8" > /etc/resolv.conf && \
@@ -60,18 +60,12 @@ RUN yum clean all && yum makecache && yum install -y epel-release && yum swap -y
unzip \
tar \
wget \
- gtest \
- gtest-devel \
- gmock \
- gmock-devel \
cppcheck \
python3 \
python3-pip \
python \
python-pip \
device-mapper-devel \
- libarchive \
- libarchive-devel \
libtar \
libtar-devel \
libcurl-devel \
@@ -118,7 +112,7 @@ RUN set -x && \
cd ~ && \
git clone https://gitee.com/src-openeuler/cmake.git && \
cd cmake && \
- git checkout openEuler-20.03-LTS-tag && \
+ git checkout -b openEuler-20.03-LTS-tag openEuler-20.03-LTS-tag && \
tar -xzvf cmake-3.12.1.tar.gz && \
cd cmake-3.12.1 && \
./bootstrap && make && make install && \
@@ -131,7 +125,7 @@ RUN set -x && \
cd ~ && \
git clone https://gitee.com/src-openeuler/protobuf.git && \
cd protobuf && \
- git checkout openEuler-20.03-LTS-tag && \
+ git checkout -b openEuler-20.03-LTS-tag openEuler-20.03-LTS-tag && \
tar -xzvf protobuf-all-3.9.0.tar.gz && \
cd protobuf-3.9.0 && \
./autogen.sh && \
@@ -146,7 +140,7 @@ RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
cd ~ && \
git clone https://gitee.com/src-openeuler/c-ares.git && \
cd c-ares && \
- git checkout openEuler-20.03-LTS-tag && \
+ git checkout -b openEuler-20.03-LTS-tag openEuler-20.03-LTS-tag && \
tar -xzvf c-ares-1.15.0.tar.gz && \
cd c-ares-1.15.0 && \
autoreconf -if && \
@@ -161,7 +155,7 @@ RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
cd ~ && \
git clone https://gitee.com/src-openeuler/grpc.git && \
cd grpc && \
- git checkout openEuler-20.03-LTS-tag && \
+ git checkout -b openEuler-20.03-LTS-tag openEuler-20.03-LTS-tag && \
tar -xzvf grpc-1.22.0.tar.gz && \
cd grpc-1.22.0 && \
make -j $(nproc) && \
@@ -174,7 +168,7 @@ RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
cd ~ && \
git clone https://gitee.com/src-openeuler/libevent.git && \
cd libevent && \
- git checkout openEuler-20.03-LTS-tag && \
+ git checkout -b openEuler-20.03-LTS-tag openEuler-20.03-LTS-tag && \
tar -xzvf libevent-2.1.11-stable.tar.gz && \
cd libevent-2.1.11-stable && \
./autogen.sh && \
@@ -189,13 +183,11 @@ RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
cd ~ && \
git clone https://gitee.com/src-openeuler/libevhtp.git && \
cd libevhtp && \
- git checkout openEuler-20.03-LTS-tag && \
- tar -xzvf libevhtp-1.2.18.tar.gz && \
- cd libevhtp-1.2.18 && \
- patch -p1 -F1 -s < ../0001-decrease-numbers-of-fd-for-shared-pipe-mode.patch && \
- patch -p1 -F1 -s < ../0002-evhtp-enable-dynamic-thread-pool.patch && \
- patch -p1 -F1 -s < ../0003-close-open-ssl.-we-do-NOT-use-it-in-lcrd.patch && \
- patch -p1 -F1 -s < ../0004-Use-shared-library-instead-static-one.patch && \
+ git checkout -b openEuler-20.03-LTS-tag openEuler-20.03-LTS-tag && \
+ tar -xzvf libevhtp-1.2.16.tar.gz && \
+ cd libevhtp-1.2.16 && \
+ patch -p1 -F1 -s < ../0001-support-dynamic-threads.patch && \
+ patch -p1 -F1 -s < ../0002-close-openssl.patch && \
rm -rf build && \
mkdir build && \
cd build && \
@@ -210,7 +202,7 @@ RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
cd ~ && \
git clone https://gitee.com/src-openeuler/http-parser.git && \
cd http-parser && \
- git checkout openEuler-20.03-LTS-tag && \
+ git checkout -b openEuler-20.03-LTS-tag openEuler-20.03-LTS-tag && \
tar -xzvf http-parser-2.9.2.tar.gz && \
cd http-parser-2.9.2 && \
make -j CFLAGS="-Wno-error" && \
@@ -223,7 +215,7 @@ RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
cd ~ && \
git clone https://gitee.com/src-openeuler/libwebsockets.git && \
cd libwebsockets && \
- git checkout openEuler-20.03-LTS-tag && \
+ git checkout -b openEuler-20.03-LTS-tag openEuler-20.03-LTS-tag && \
tar -xzvf libwebsockets-2.4.2.tar.gz && \
cd libwebsockets-2.4.2 && \
patch -p1 -F1 -s < ../libwebsockets-fix-coredump.patch && \
@@ -234,56 +226,74 @@ RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
make install && \
ldconfig
-# install lxc
+# install gtest/gmock
RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
set -x && \
cd ~ && \
- git clone https://gitee.com/src-openeuler/lxc.git && \
- cd lxc && \
- ./apply-patches && \
- cd lxc-4.0.3 && \
- ./autogen.sh && \
- ./configure && \
- make -j $(nproc) && \
- make install && \
- ldconfig
-
-# install lcr
-RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
- set -x && \
- cd ~ && \
- git clone https://gitee.com/openeuler/lcr.git && \
- cd lcr && \
- mkdir build && \
+ git clone https://gitee.com/src-openeuler/gtest.git && \
+ cd gtest && \
+ git checkout -b openEuler-20.03-LTS-tag openEuler-20.03-LTS-tag && \
+ tar xf release-1.8.1.tar.gz && \
+ cd googletest-release-1.8.1 && \
+ patch -p1 -F1 -s < ../gtest-1.8.1-null-pointer.patch && \
+ patch -p1 -F1 -s < ../gtest-PR1839-Fix-Python3-support.patch && \
+ patch -p1 -F1 -s < ../gtest-1.8.1-libversion.patch && \
+ patch -p1 -F1 -s < ../gtest-1.8.1-add-missing-pkgconfig-requires.patch && \
+ mkdir -p build && \
cd build && \
- cmake ../ && \
+ cmake -DBUILD_SHARED_LIBS=ON ../ && \
make -j $(nproc) && \
make install && \
ldconfig
-# install clibcni
+# install libarchive
RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
- set -x && \
- cd ~ && \
- git clone https://gitee.com/openeuler/clibcni.git && \
- cd clibcni && \
- mkdir build && \
- cd build && \
- cmake ../ && \
- make -j $(nproc) && \
- make install && \
- ldconfig
+ set -x && \
+ cd ~ && \
+ git clone https://gitee.com/src-openeuler/libarchive.git && \
+ cd libarchive && \
+ git checkout -b openEuler-20.03-LTS-tag openEuler-20.03-LTS-tag && \
+ tar -zxvf libarchive-3.4.1.tar.gz && \
+ cd libarchive-3.4.1 && \
+ patch -p1 -F1 -s < ../libarchive-uninitialized-value.patch && \
+ cd build && \
+ cmake -DCMAKE_USE_SYSTEM_LIBRARIES=ON ../ && \
+ make -j $(nproc) && \
+ make install && \
+ ldconfig
-# install iSulad-img
RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH && \
- set -x && \
- cd ~ && \
- git clone https://gitee.com/openeuler/iSulad-img.git && \
- cd iSulad-img && \
- ./apply-patch && \
- make -j $(nproc) && \
- make install && \
- ldconfig
-
+ set -x && \
+ cd ~ && \
+ git clone https://gitee.com/openeuler/iSulad/ &&\
+ cd iSulad &&\
+ # git checkout 756c0bdc308c2845971ad9ca0c58760a84288bc0 &&\
+ git checkout v2.0.6 &&\
+ cd CI &&\
+ ./install_depends.sh &&\
+ cd .. &&\
+ mkdir build &&\
+ cd build &&\
+ cmake .. &&\
+ make -j $(nproc) && \
+ make install && \
+ ldconfig
+
+FROM centos:7.6.1810
+
+COPY --from=build /usr/local/bin /usr/local/bin
+COPY --from=build /usr/local/lib /usr/local/lib
+COPY --from=build /usr/local/lib64 /usr/local/lib64
+COPY --from=build /usr/lib64 /usr/lib64
+COPY --from=build /etc/default/isulad/ /etc/default/isulad/
+COPY --from=build /etc/isulad /etc/isulad
+
+
+
+RUN echo "/usr/lib" >> /etc/ld.so.conf && \
+ echo "/usr/local/lib" >> /etc/ld.so.conf &&\
+ ldconfig
+
+
VOLUME [ "/sys/fs/cgroup" ]
-CMD ["/usr/sbin/init"]
+CMD ["/usr/local/bin/isulad"]
\ No newline at end of file
--
2.20.1

92
0003-isulad-rt_isula_start-should-read-the-isulad-shim-pi.patch Normal file
View File

@ -0,0 +1,92 @@
From 3942bcc4c3a755d33e709046edd358f9af264d11 Mon Sep 17 00:00:00 2001
From: holyfei <yangfeiyu20092010@163.com>
Date: Wed, 21 Oct 2020 10:00:15 +0800
Subject: [PATCH 03/28] isulad: rt_isula_start should read the isulad-shim
pidinfo
reason: the ppid of init pid should be isulad-shim, read isulad-shim pidinfo and set the start time for init pidinfo
Signed-off-by: yangfeiyu <yangfeiyu2@huawei.com>
---
.../modules/runtime/isula/isula_rt_ops.c | 38 ++++++++++++++++++-
1 file changed, 36 insertions(+), 2 deletions(-)
diff --git a/src/daemon/modules/runtime/isula/isula_rt_ops.c b/src/daemon/modules/runtime/isula/isula_rt_ops.c
index 6e4512f..82d7aec 100644
--- a/src/daemon/modules/runtime/isula/isula_rt_ops.c
+++ b/src/daemon/modules/runtime/isula/isula_rt_ops.c
@@ -851,8 +851,13 @@ out:
int rt_isula_start(const char *id, const char *runtime, const rt_start_params_t *params, pid_ppid_info_t *pid_info)
{
char workdir[PATH_MAX] = { 0 };
+ char shim_pid_file_name[PATH_MAX] = { 0 };
pid_t pid = 0;
+ pid_t shim_pid = -1;
int ret = 0;
+ int splice_ret = 0;
+ proc_t *proc = NULL;
+ proc_t *p_proc = NULL;
if (id == NULL || runtime == NULL || params == NULL || pid_info == NULL) {
ERROR("nullptr arguments not allowed");
@@ -863,6 +868,12 @@ int rt_isula_start(const char *id, const char *runtime, const rt_start_params_t
return -1;
}
+ splice_ret = snprintf(shim_pid_file_name, sizeof(shim_pid_file_name), "%s/shim-pid", workdir);
+ if (splice_ret < 0 || splice_ret >= sizeof(shim_pid_file_name)) {
+ ERROR("%s: wrong shim workdir", id);
+ return -1;
+ }
+
pid = get_container_process_pid(workdir);
if (pid < 0) {
ret = -1;
@@ -870,12 +881,32 @@ int rt_isula_start(const char *id, const char *runtime, const rt_start_params_t
goto out;
}
- if (util_read_pid_ppid_info(pid, pid_info) != 0) {
+ file_read_int(shim_pid_file_name, &shim_pid);
+ if (shim_pid < 0) {
+ ret = -1;
+ ERROR("%s: failed to read isulad shim pid", id);
+ goto out;
+ }
+
+ proc = util_get_process_proc_info(pid);
+ if (proc == NULL) {
ret = -1;
- ERROR("%s: failed read pid info", id);
+ ERROR("%s: failed to read pidinfo", id);
goto out;
}
+ p_proc = util_get_process_proc_info(shim_pid);
+ if (p_proc == NULL) {
+ ret = -1;
+ ERROR("%s: failed to read isulad shim pidinfo", id);
+ goto out;
+ }
+
+ pid_info->pid = proc->pid;
+ pid_info->start_time = proc->start_time;
+ pid_info->ppid = shim_pid;
+ pid_info->pstart_time = p_proc->start_time;
+
if (runtime_call_simple(workdir, runtime, "start", NULL, 0, id) != 0) {
ERROR("call runtime start id failed");
ret = -1;
@@ -888,6 +919,9 @@ out:
shim_kill_force(workdir);
}
+ free(proc);
+ free(p_proc);
+
return ret;
}
--
2.20.1

240
0004-Realpath-add-get-realpath-for-root-and-state-dir.patch Normal file
View File

@ -0,0 +1,240 @@
From cc721f53f64e9ec7e6329beaf24b2df2a3d1ff4e Mon Sep 17 00:00:00 2001
From: lifeng68 <lifeng68@huawei.com>
Date: Wed, 21 Oct 2020 10:21:04 +0800
Subject: [PATCH 04/28] Realpath: add get realpath for root and state dir
Signed-off-by: lifeng68 <lifeng68@huawei.com>
---
src/cmd/isulad/isulad_commands.c | 20 +++++++-
.../graphdriver/devmapper/deviceset.c | 48 +++++++++----------
.../graphdriver/devmapper/driver_devmapper.c | 37 +++++++++++---
3 files changed, 70 insertions(+), 35 deletions(-)
diff --git a/src/cmd/isulad/isulad_commands.c b/src/cmd/isulad/isulad_commands.c
index 2826aae..90fc51a 100644
--- a/src/cmd/isulad/isulad_commands.c
+++ b/src/cmd/isulad/isulad_commands.c
@@ -195,6 +195,7 @@ static int check_args_graph_path(struct service_arguments *args)
{
int ret = 0;
char dstpath[PATH_MAX] = { 0 };
+ char *real_path = NULL;
ret = util_validate_absolute_path(args->json_confs->graph);
if (ret) {
@@ -207,8 +208,15 @@ static int check_args_graph_path(struct service_arguments *args)
ret = -1;
goto out;
}
+
+ if (util_realpath_in_scope("/", dstpath, &real_path) != 0) {
+ ERROR("failed to get real path");
+ ret = -1;
+ goto out;
+ }
+
free(args->json_confs->graph);
- args->json_confs->graph = util_strdup_s(dstpath);
+ args->json_confs->graph = real_path;
out:
return ret;
@@ -218,6 +226,7 @@ static int check_args_state_path(struct service_arguments *args)
{
int ret = 0;
char dstpath[PATH_MAX] = { 0 };
+ char *real_path = NULL;
ret = util_validate_absolute_path(args->json_confs->state);
if (ret != 0) {
@@ -230,8 +239,15 @@ static int check_args_state_path(struct service_arguments *args)
ret = -1;
goto out;
}
+
+ if (util_realpath_in_scope("/", dstpath, &real_path) != 0) {
+ ERROR("failed to get real path");
+ ret = -1;
+ goto out;
+ }
+
free(args->json_confs->state);
- args->json_confs->state = util_strdup_s(dstpath);
+ args->json_confs->state = real_path;
out:
return ret;
diff --git a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
index fb2f502..cadb061 100644
--- a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
+++ b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/deviceset.c
@@ -1145,9 +1145,10 @@ static int pool_has_free_space(struct device_set *devset)
ERROR("devmapper: Thin Pool has %lu free data blocks which is less than minimum required "
"%lu free data blocks. Create more free space in thin pool or use dm.min_free_space option to change behavior",
data_total - data_used, min_free_data);
- isulad_set_error_message("devmapper: Thin Pool has %lu free data blocks which is less than minimum required "
- "%lu free data blocks. Create more free space in thin pool or use dm.min_free_space option to change behavior",
- data_total - data_used, min_free_data);
+ isulad_set_error_message(
+ "devmapper: Thin Pool has %lu free data blocks which is less than minimum required "
+ "%lu free data blocks. Create more free space in thin pool or use dm.min_free_space option to change behavior",
+ data_total - data_used, min_free_data);
ret = -1;
goto out;
}
@@ -1163,10 +1164,11 @@ static int pool_has_free_space(struct device_set *devset)
"which is less than minimum required %lu free metadata blocks. "
"Create more free metadata space in thin pool or use dm.min_free_space option to change behavior",
metadata_total - metadata_used, min_free_metadata);
- isulad_set_error_message("devmapper: Thin Pool has %lu free metadata blocks "
- "which is less than minimum required %lu free metadata blocks. "
- "Create more free metadata space in thin pool or use dm.min_free_space option to change behavior",
- metadata_total - metadata_used, min_free_metadata);
+ isulad_set_error_message(
+ "devmapper: Thin Pool has %lu free metadata blocks "
+ "which is less than minimum required %lu free metadata blocks. "
+ "Create more free metadata space in thin pool or use dm.min_free_space option to change behavior",
+ metadata_total - metadata_used, min_free_metadata);
ret = -1;
goto out;
}
@@ -1652,7 +1654,8 @@ static int take_snapshot(struct device_set *devset, const char *hash, image_devm
if (dinfo.deferred_remove != 0) {
nret = cancel_deferred_removal(devset, base_info->hash);
if (nret != 0) {
- ERROR("devmapper: cancel deferred remove for device with hash:%s failed, err:%s", base_info->hash, dev_strerror(nret));
+ ERROR("devmapper: cancel deferred remove for device with hash:%s failed, err:%s", base_info->hash,
+ dev_strerror(nret));
if (nret != ERR_ENXIO) {
ERROR("devmapper: cancel device(id:%s) deferred remove failed", base_info->hash);
ret = -1;
@@ -1721,7 +1724,8 @@ static int cancel_deferred_removal_if_needed(struct device_set *devset, image_de
nret = cancel_deferred_removal(devset, info->hash);
if (nret != 0 && nret != ERR_BUSY) {
- ERROR("devmapper: cancel deferred remove for device with hash:%s failed, err:%s", info->hash, dev_strerror(nret));
+ ERROR("devmapper: cancel deferred remove for device with hash:%s failed, err:%s", info->hash,
+ dev_strerror(nret));
ret = -1;
goto out;
}
@@ -2563,7 +2567,8 @@ static int determine_driver_capabilities(const char *version, struct device_set
if (major < 4) {
ERROR("devicamapper driver version:(%ld.xxx) < 4.27.0, do not surpport deferred removal", major);
- isulad_set_error_message("devicamapper driver version:(%ld.xxx) < 4.27.0, do not surpport deferred removal", major);
+ isulad_set_error_message("devicamapper driver version:(%ld.xxx) < 4.27.0, do not surpport deferred removal",
+ major);
ret = -1;
goto out;
}
@@ -2584,7 +2589,8 @@ static int determine_driver_capabilities(const char *version, struct device_set
*/
if (minor < 27) {
ERROR("devicamapper driver version (4.%ld) < 4.27.0, , do not surpport deferred removal", minor);
- isulad_set_error_message("devicamapper driver version (4.%ld) < 4.27.0, , do not surpport deferred removal", minor);
+ isulad_set_error_message("devicamapper driver version (4.%ld) < 4.27.0, , do not surpport deferred removal",
+ minor);
ret = -1;
goto out;
}
@@ -2951,17 +2957,10 @@ int unmount_device(const char *hash, const char *mount_path, struct device_set *
goto free_out;
}
- if (util_detect_mounted(mount_path)) {
- if (umount2(mount_path, MNT_DETACH) < 0 && errno != EINVAL) {
- ERROR("Failed to umount directory %s:%s", mount_path, strerror(errno));
- ret = -1;
- goto free_out;
- }
- }
-
- if (util_path_remove(mount_path) != 0) {
- DEBUG("devmapper: doing remove on a unmounted device %s failed", mount_path);
+ if (umount2(mount_path, MNT_DETACH) < 0 && errno != EINVAL) {
+ ERROR("Failed to umount directory %s:%s", mount_path, strerror(errno));
ret = -1;
+ goto free_out;
}
if (deactivate_device(devset, device_info->info) != 0) {
@@ -3167,7 +3166,6 @@ struct status *device_set_status(struct device_set *devset)
st->sem_msg = util_strdup_s(msg);
}
-
free_out:
(void)pthread_rwlock_unlock(&devset->devmapper_driver_rwlock);
return st;
@@ -3222,10 +3220,8 @@ static int umount_deactivate_dev_all(struct device_set *devset)
continue;
}
- if (util_detect_mounted(fname)) {
- if (umount2(fname, MNT_DETACH) < 0 && errno != EINVAL) {
- ERROR("Failed to umount directory %s:%s", fname, strerror(errno));
- }
+ if (umount2(fname, MNT_DETACH) < 0 && errno != EINVAL) {
+ ERROR("Failed to umount directory %s:%s", fname, strerror(errno));
}
device_info = lookup_device(devset, entry->d_name);
diff --git a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/driver_devmapper.c b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/driver_devmapper.c
index 1043c6c..ec337a8 100644
--- a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/driver_devmapper.c
+++ b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/driver_devmapper.c
@@ -82,7 +82,36 @@ out:
static int do_create(const char *id, const char *parent, const struct graphdriver *driver,
const struct driver_create_opts *create_opts)
{
- return add_device(id, parent, driver->devset, create_opts->storage_opt);
+ int ret = 0;
+ char *mnt_parent_dir = NULL;
+ char *mnt_point_dir = NULL;
+
+ mnt_parent_dir = util_path_join(driver->home, "mnt");
+ if (mnt_parent_dir == NULL) {
+ ERROR("Failed to join devmapper mnt dir %s", id);
+ ret = -1;
+ goto out;
+ }
+
+ mnt_point_dir = util_path_join(mnt_parent_dir, id);
+ if (mnt_point_dir == NULL) {
+ ERROR("Failed to join devampper mount point dir %s", id);
+ ret = -1;
+ goto out;
+ }
+
+ if (util_mkdir_p(mnt_point_dir, DEFAULT_SECURE_DIRECTORY_MODE) != 0) {
+ ERROR("Failed to mkdir path:%s", mnt_point_dir);
+ ret = -1;
+ goto out;
+ }
+
+ ret = add_device(id, parent, driver->devset, create_opts->storage_opt);
+
+out:
+ free(mnt_parent_dir);
+ free(mnt_point_dir);
+ return ret;
}
// devmapper_create_rw creates a layer that is writable for use as a container file system
@@ -186,12 +215,6 @@ char *devmapper_mount_layer(const char *id, const struct graphdriver *driver,
goto out;
}
- if (util_mkdir_p(mnt_point_dir, DEFAULT_SECURE_DIRECTORY_MODE) != 0) {
- ERROR("Failed to mkdir path:%s", mnt_point_dir);
- ret = -1;
- goto out;
- }
-
if (mount_device(id, mnt_point_dir, mount_opts, driver->devset) != 0) {
ERROR("Mount device:%s to path:%s failed", id, mnt_parent_dir);
ret = -1;
--
2.20.1

318
0005-CI-add-testcase-for-root-and-run-dir-realpath.patch Normal file
View File

@ -0,0 +1,318 @@
From 86567b77cc367a96b80c129ad13791851768b860 Mon Sep 17 00:00:00 2001
From: lifeng68 <lifeng68@huawei.com>
Date: Wed, 21 Oct 2020 10:36:07 +0800
Subject: [PATCH 05/28] CI: add testcase for root and run dir realpath
Signed-off-by: lifeng68 <lifeng68@huawei.com>
---
.../container_cases/graph_root_test.sh | 284 ++++++++++++++++++
CI/test_cases/image_cases/image_tag.sh | 2 +
2 files changed, 286 insertions(+)
create mode 100644 CI/test_cases/container_cases/graph_root_test.sh
diff --git a/CI/test_cases/container_cases/graph_root_test.sh b/CI/test_cases/container_cases/graph_root_test.sh
new file mode 100644
index 0000000..678d176
--- /dev/null
+++ b/CI/test_cases/container_cases/graph_root_test.sh
@@ -0,0 +1,284 @@
+#!/bin/bash
+#
+# attributes: isulad root and run dir realpath test
+# concurrent: NA
+# spend time: 5
+
+#######################################################################
+##- @Copyright (C) Huawei Technologies., Ltd. 2020. All rights reserved.
+# - iSulad licensed under the Mulan PSL v2.
+# - You can use this software according to the terms and conditions of the Mulan PSL v2.
+# - You may obtain a copy of Mulan PSL v2 at:
+# - http://license.coscl.org.cn/MulanPSL2
+# - THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR
+# - IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR
+# - PURPOSE.
+# - See the Mulan PSL v2 for more details.
+##- @Description:CI
+##- @Author: lifeng
+##- @Create: 2020-09-03
+#######################################################################
+
+declare -r curr_path=$(dirname $(readlink -f "$0"))
+source ../helpers.sh
+
+function reinstall_thinpool()
+{
+ local ret=0
+
+ cat /etc/isulad/daemon.json | grep driver | grep devicemapper
+ if [[ $? -ne 0 ]]; then
+ return ${ret}
+ fi
+
+ dev_disk=`pvs | grep isulad | awk '{print$1}'`
+ rm -rf /var/lib/isulad/*
+ dmsetup remove_all
+ lvremove -f isulad/thinpool
+ lvremove -f isulad/thinpoolmeta
+ vgremove -f isulad
+ pvremove -f $dev_disk
+ mount | grep $dev_disk | grep /var/lib/isulad
+ if [ x"$?" == x"0" ]; then
+ umount /var/lib/isulad
+ fi
+ touch /etc/lvm/profile/isulad-thinpool.profile
+ cat > /etc/lvm/profile/isulad-thinpool.profile <<EOF
+activation {
+thin_pool_autoextend_threshold=80
+thin_pool_autoextend_percent=20
+}
+EOF
+ echo y | mkfs.ext4 $dev_disk
+ pvcreate -y $dev_disk
+ vgcreate isulad $dev_disk
+ echo y | lvcreate --wipesignatures y -n thinpool isulad -l 80%VG
+ echo y | lvcreate --wipesignatures y -n thinpoolmeta isulad -l 1%VG
+ lvconvert -y --zero n -c 512K --thinpool isulad/thinpool --poolmetadata isulad/thinpoolmeta
+ lvchange --metadataprofile isulad-thinpool isulad/thinpool
+ lvs -o+seg_monitor
+ return ${ret}
+}
+
+function test_run_root_dir_realpath()
+{
+ local ret=0
+ local image="busybox"
+ local test="isulad root and run dir realpath test => (${FUNCNAME[@]})"
+
+ msg_info "${test} starting..."
+
+ check_valgrind_log
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - stop isulad failed" && ((ret++))
+
+ reinstall_thinpool
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - fail to reconfig isulad-thinpool" && ((ret++))
+
+ mkdir -p /var/lib/isulad/opt/test_root
+ mkdir -p /opt/test_run
+
+ cp -f /etc/isulad/daemon.json /etc/isulad/daemon.bak
+
+ sed -i 's#"graph": "/var/lib/isulad",#"graph": "/var/lib/isulad_test",#g' /etc/isulad/daemon.json
+ sed -i 's#"state": "/var/run/isulad",#"state": "/var/run/isulad_test",#g' /etc/isulad/daemon.json
+
+ ln -s /var/lib/isulad/opt/test_root /var/lib/isulad_test
+ ln -s /opt/test_run /var/run/isulad_test
+
+ start_isulad_with_valgrind
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - start isulad failed" && ((ret++))
+
+ isula pull ${image}
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to pull image: ${image}" && ((ret++))
+
+ isula images | grep busybox
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - missing list image: ${image}" && ((ret++))
+
+ c_id=`isula run -itd --cpus 1.5 busybox sh`
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to run container with image: ${image}" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us" | grep "150000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_quota_us: ${image}" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_period_us" | grep "100000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_period_us: ${image}" && ((ret++))
+
+ isula restart -t 0 $c_id
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to restart container: $c_id" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us" | grep "150000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_quota_us: ${image}" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_period_us" | grep "100000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_period_us: ${image}" && ((ret++))
+
+ isula update --cpus 1.3 --cpu-period 20000 $c_id 2>&1 | grep "Nano CPUs and CPU Period cannot both be set"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - Nano CPUs and CPU Period cannot both be set" && ((ret++))
+
+ isula update --cpus 1.3 --cpu-quota 20000 $c_id 2>&1 | grep "Nano CPUs and CPU Quota cannot both be set"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - Nano CPUs and CPU Quota cannot both be set" && ((ret++))
+
+ isula update --cpu-period 20000 $c_id 2>&1 | grep "CPU Period cannot be updated as NanoCPUs has already been set"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - CPU Period cannot be updated as NanoCPUs has already been set" && ((ret++))
+
+ isula update --cpu-quota 20000 $c_id 2>&1 | grep "CPU Quota cannot be updated as NanoCPUs has already been set"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - CPU Quota cannot be updated as NanoCPUs has already been set" && ((ret++))
+
+ isula update --cpus 1.3 $c_id
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - Failed to update cpus" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us" | grep "130000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_quota_us: ${image}" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_period_us" | grep "100000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_period_us: ${image}" && ((ret++))
+
+ isula restart -t 0 $c_id
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to restart container: $c_id" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us" | grep "130000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_quota_us: ${image}" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_period_us" | grep "100000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_period_us: ${image}" && ((ret++))
+
+ isula rm -f $c_id
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to rm container ${c_id}" && ((ret++))
+
+ check_valgrind_log
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - stop isulad failed" && ((ret++))
+
+ cp -f /etc/isulad/daemon.bak /etc/isulad/daemon.json
+
+ rm -rf /var/lib/isulad/opt/test_root
+ rm -rf /opt/test_run
+ rm -rf /var/lib/isulad_test
+ rm -rf /var/run/isulad_test
+
+ reinstall_thinpool
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - fail to reconfig isulad-thinpool" && ((ret++))
+
+ start_isulad_with_valgrind
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - start isulad failed" && ((ret++))
+
+ msg_info "${test} finished with return ${ret}..."
+ return ${ret}
+}
+
+function test_run_root_dir_bind_realpath()
+{
+ local ret=0
+ local image="busybox"
+ local test="isulad root and run dir realpath test => (${FUNCNAME[@]})"
+
+ msg_info "${test} starting..."
+
+ check_valgrind_log
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - stop isulad failed" && ((ret++))
+
+ reinstall_thinpool
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - fail to reconfig isulad-thinpool" && ((ret++))
+
+ mkdir -p /var/lib/isulad/opt/bind_root
+ mkdir -p /opt/bind_run
+
+ cp -f /etc/isulad/daemon.json /etc/isulad/daemon.bak
+
+ sed -i 's#"graph": "/var/lib/isulad",#"graph": "/var/lib/isulad/bind/isulad_test",#g' /etc/isulad/daemon.json
+ sed -i 's#"state": "/var/run/isulad",#"state": "/var/run/isulad_test",#g' /etc/isulad/daemon.json
+
+ mkdir -p /var/lib/isulad/bind/isulad_test
+ mount --bind /var/lib/isulad/opt/bind_root /var/lib/isulad/bind/isulad_test
+
+ mkdir -p /var/run/isulad_test
+ mount --bind /opt/bind_run /var/run/isulad_test
+
+ start_isulad_with_valgrind
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - start isulad failed" && ((ret++))
+
+ isula pull ${image}
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to pull image: ${image}" && ((ret++))
+
+ isula images | grep busybox
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - missing list image: ${image}" && ((ret++))
+
+ c_id=`isula run -itd --cpus 1.5 busybox sh`
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to run container with image: ${image}" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us" | grep "150000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_quota_us: ${image}" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_period_us" | grep "100000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_period_us: ${image}" && ((ret++))
+
+ isula restart -t 0 $c_id
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to restart container: $c_id" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us" | grep "150000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_quota_us: ${image}" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_period_us" | grep "100000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_period_us: ${image}" && ((ret++))
+
+ isula update --cpus 1.3 --cpu-period 20000 $c_id 2>&1 | grep "Nano CPUs and CPU Period cannot both be set"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - Nano CPUs and CPU Period cannot both be set" && ((ret++))
+
+ isula update --cpus 1.3 --cpu-quota 20000 $c_id 2>&1 | grep "Nano CPUs and CPU Quota cannot both be set"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - Nano CPUs and CPU Quota cannot both be set" && ((ret++))
+
+ isula update --cpu-period 20000 $c_id 2>&1 | grep "CPU Period cannot be updated as NanoCPUs has already been set"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - CPU Period cannot be updated as NanoCPUs has already been set" && ((ret++))
+
+ isula update --cpu-quota 20000 $c_id 2>&1 | grep "CPU Quota cannot be updated as NanoCPUs has already been set"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - CPU Quota cannot be updated as NanoCPUs has already been set" && ((ret++))
+
+ isula update --cpus 1.3 $c_id
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - Failed to update cpus" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us" | grep "130000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_quota_us: ${image}" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_period_us" | grep "100000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_period_us: ${image}" && ((ret++))
+
+ isula restart -t 0 $c_id
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to restart container: $c_id" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us" | grep "130000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_quota_us: ${image}" && ((ret++))
+
+ isula exec -it $c_id sh -c "cat /sys/fs/cgroup/cpu/cpu.cfs_period_us" | grep "100000"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to check cfs_period_us: ${image}" && ((ret++))
+
+ isula rm -f $c_id
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to rm container ${c_id}" && ((ret++))
+
+ check_valgrind_log
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - stop isulad failed" && ((ret++))
+
+ cp -f /etc/isulad/daemon.bak /etc/isulad/daemon.json
+
+ umount /var/lib/isulad/bind/isulad_test
+ umount /var/run/isulad_test
+
+ rm -rf /var/lib/isulad/opt/bind_root
+ rm -rf /opt/bind_run
+ rm -rf /var/lib/isulad/bind/isulad_test
+ rm -rf /var/run/isulad_test
+
+ reinstall_thinpool
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - fail to reconfig isulad-thinpool" && ((ret++))
+
+ start_isulad_with_valgrind
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - start isulad failed" && ((ret++))
+
+ msg_info "${test} finished with return ${ret}..."
+ return ${ret}
+}
+
+declare -i ans=0
+
+test_run_root_dir_realpath || ((ans++))
+test_run_root_dir_bind_realpath || ((ans++))
+
+show_result ${ans} "${curr_path}/${0}"
diff --git a/CI/test_cases/image_cases/image_tag.sh b/CI/test_cases/image_cases/image_tag.sh
index ab04560..63d2687 100755
--- a/CI/test_cases/image_cases/image_tag.sh
+++ b/CI/test_cases/image_cases/image_tag.sh
@@ -32,6 +32,8 @@ function test_tag_image()
msg_info "${test} starting..."
+ isula rm -f `isula ps -aq`
+
isula pull $image_busybox
[[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to pull image: ${image_busybox}" && ((ret++))
--
2.20.1

26
0006-info-fix-typo-driverr-to-driver.patch Normal file
View File

@ -0,0 +1,26 @@
From abffe70870a2867d423dcf24f4b97791b1df16bb Mon Sep 17 00:00:00 2001
From: lifeng68 <lifeng68@huawei.com>
Date: Thu, 22 Oct 2020 11:25:35 +0800
Subject: [PATCH 06/28] info: fix typo driverr to driver
Signed-off-by: lifeng68 <lifeng68@huawei.com>
---
src/cmd/isula/information/info.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/cmd/isula/information/info.c b/src/cmd/isula/information/info.c
index 07cad9d..d6f6f7b 100644
--- a/src/cmd/isula/information/info.c
+++ b/src/cmd/isula/information/info.c
@@ -76,7 +76,7 @@ static void client_info_server(const struct isula_info_response *response)
printf("Logging Driver: %s\n", response->logging_driver);
}
if (response->cgroup_driver != NULL) {
- printf("Cgroup Driverr: %s\n", response->cgroup_driver);
+ printf("Cgroup Driver: %s\n", response->cgroup_driver);
}
if (response->huge_page_size != NULL) {
printf("Hugetlb Pagesize: %s\n", response->huge_page_size);
--
2.20.1

48
0007-create-fix-wrong-ret-code.patch Normal file
View File

@ -0,0 +1,48 @@
From ba32dabc6d6304410ed6c731c2f276c1f50a9b06 Mon Sep 17 00:00:00 2001
From: lifeng68 <lifeng68@huawei.com>
Date: Fri, 23 Oct 2020 10:05:07 +0800
Subject: [PATCH 07/28] create: fix wrong ret code
Signed-off-by: lifeng68 <lifeng68@huawei.com>
---
src/cmd/isula/base/create.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/src/cmd/isula/base/create.c b/src/cmd/isula/base/create.c
index 12903ce..96ca483 100644
--- a/src/cmd/isula/base/create.c
+++ b/src/cmd/isula/base/create.c
@@ -1249,24 +1249,24 @@ int client_create(struct client_arguments *args)
request->image = util_strdup_s(args->image_name);
container_spec = request_pack_custom_conf(args);
- if (container_spec == 0) {
- ret = -1;
+ if (container_spec == NULL) {
+ ret = EINVALIDARGS;
goto out;
}
if (generate_container_config(container_spec, &request->container_spec_json) != 0) {
- ret = -1;
+ ret = EINVALIDARGS;
goto out;
}
host_spec = request_pack_host_config(args);
- if (host_spec == 0) {
- ret = -1;
+ if (host_spec == NULL) {
+ ret = EINVALIDARGS;
goto out;
}
if (generate_hostconfig(host_spec, &request->host_spec_json) != 0) {
- ret = -1;
+ ret = EINVALIDARGS;
goto out;
}
--
2.20.1

319
0008-add-iSulad-s-build-guide-for-RISC-V.patch Normal file
View File

@ -0,0 +1,319 @@
From 2033d9ff15d01f1a2ccd62b4376eb345bd438280 Mon Sep 17 00:00:00 2001
From: shentalon <13212105191@163.com>
Date: Sat, 24 Oct 2020 14:55:22 +0000
Subject: [PATCH 08/28] add iSulad's build guide for RISC-V
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
背景说明：我今年参加了summer2020活动，选题是`在 RISC-V 架构 openEuler 平台上提供 iSulad`。之前答应老师陆续上传一些阶段性工作，因为秋招无力分身，只能在最后索性写一个相对完整的方案。下面的内容大致有：RISC-V虚拟环境的搭建、依赖安装的说明、编译安装指南及内核及内核模块编译,撰写的过程中，我参考了和各位指导老师的邮件及社区交流记录，同时还有自己在本地进行的问题记录。出于让开发者能简洁、顺畅地使用指南，里面只收录了成功编译的方法而没有加入报错的记录和尝试解决错误的过程部分，整理的可能不够详细，如果有问题欢迎[联系我](https://gitee.com/shentalon)。
---
docs/build_guide_riscv.md | 296 ++++++++++++++++++++++++++++++++++++++
1 file changed, 296 insertions(+)
create mode 100644 docs/build_guide_riscv.md
diff --git a/docs/build_guide_riscv.md b/docs/build_guide_riscv.md
new file mode 100644
index 0000000..76f97c9
--- /dev/null
+++ b/docs/build_guide_riscv.md
@@ -0,0 +1,296 @@
+
+# ISulad在RISC-V构架的openEuler的支持工作
+
+
+---
+
+## RISC-V虚拟环境的搭建
+>RISC-V的环境我们是通过在host上使用QEMU虚拟机实现的，我们要做的是使用任意一Linux发行版作为host安装QEMU虚拟机，在虚拟机中启动RISC-V的openEuler镜像，在虚拟机镜像中完成iSulad的安装。
+### 1. 安装虚拟机
+
+首先是在host上安装QEMU，打开终端，依次输入以下命令：
+```shell
+wget https://download.qemu.org/qemu-5.1.0.tar.xz
+tar xvJf qemu-5.1.0.tar.xz
+cd qemu-5.1.0
+./configure --target-list=riscv64-softmmu
+make
+make install
+```
+### 2. 启动文件准备
+安装好支持RISC-V的QEMU之后，就可以使用它来启动虚拟机的镜像，镜像的下载和安装可以参考[openEuler RISC-V 移植版的获取和运行](https://gitee.com/openeuler/RISC-V/blob/master/documents/Installing.md)，启动QEMU的虚拟机Linux环境，应该有以下几个文件：
+
+1. [oe-rv-rv64g-30G.qcow2](https://isrc.iscas.ac.cn/mirror/openeuler-sig-riscv/images/oe-rv-rv64g-30G.qcow2)
+
+2. [fw_payload_oe.elf](https://isrc.iscas.ac.cn/mirror/openeuler-sig-riscv/images/fw_payload_oe.elf)
+
+3. run_oe1_rv64.sh(可选)
+
+
+可以创建一个shell文件，内容来自[installing.md](https://gitee.com/openeuler/RISC-V/blob/master/documents/Installing.md)，如下：
+
+
+```shell
+#!/bin/sh
+
+qemu-system-riscv64 \
+ -machine virt \
+ -nographic \
+ -smp 8 \
+ -m 124G \
+ -drive file=oe-rv-base-expand.qcow2,format=qcow2,id=hd0 \
+ -object rng-random,filename=/dev/urandom,id=rng0 \
+ -device virtio-rng-device,rng=rng0 \
+ -device virtio-blk-device,drive=hd0 \
+ -netdev user,id=usernet,hostfwd=tcp::12055-:22 \
+ -device virtio-net-device,netdev=usernet \
+ -append 'root=/dev/vda1 systemd.default_timeout_start_sec=600 selinux=0 rw highres=off console=ttyS0 mem=4096M earlycon' \
+ -kernel fw_payload.elf \
+```
+
+里面是一些参数的设定，可以查看QEMU的参数说明根据本地计算机配置进行调整。
+### 3.启动虚拟机
+可以采用两种方式：
+1. 在终端直接输入shell文件中的内容
+2. 如果创建了shell文件，只需要在终端里输入 `sh run_oe1_rv64.sh`
+
+默认的登陆用户名/密码是：root/openEuler12#$
+
+## 依赖安装
+
+正式编译项目之前，要在系统上安装编译工具、代码版本控制等用途的软件包。
+这个过程会使用yum工具来对rpm软件包进行安装，如果刚刚使用`oe-rv-rv64g-30G.qcow2`，里面并没有提供yum工具，可以使用下面的命令进行yum的安装：
+
+```shell
+wget https://isrc.iscas.ac.cn/mirror/openeuler-sig-riscv/oe-RISCV-repo/noarch/yum-4.2.15-8.noarch.rpm --no-check-certificate
+rpm -ivh yum-4.2.15-8.noarch.rpm
+```
+之后，可以使用yum工具进行所需软件包的安装：
+```shell
+sudo yum --enablerepo='*' install -y automake autoconf libtool cmake make libcap libcap-devel libselinux libselinux-devel libseccomp libseccomp-devel yajl-devel git libcgroup tar python3 python3-pip device-mapper-devel libarchive libarchive-devel libcurl-devel zlib-devel glibc-headers openssl-devel gcc gcc-c++ systemd-devel systemd-libs libtar libtar-devel vim
+```
+软件包的所需依赖参考了[build_guide](https://gitee.com/openeuler/iSulad/blob/master/docs/build_guide.md),和参考文档相比，去掉了golang(iSulad转为全C开发，不再使用GO语言)，增加了vim(镜像没有文本编辑工具)。
+要修改yum源的配置，在 /etc/yum.repos.d/下打开`oe-rv.repo`文件，一般使用[Index of /oe-RISCV-repo/](https://isrc.iscas.ac.cn/mirror/openeuler-sig-riscv/oe-RISCV-repo/)为yum源的地址。
+
+## 源码编译及安装
+> 整个过程参考了[build_guide](https://gitee.com/openeuler/iSulad/blob/master/docs/build_guide.md)，编译过程中出现了一些错误，做了修改，整理成下面的编译指南。
+### 正式编译之前的准备工作及提示
+##### 1.设置 ldconfig and pkgconfig（若编译中断，再次进入系统时在源码编译之前都必须运行一次！）
+
+```javascript
+$ export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH
+$ export LD_LIBRARY_PATH=/usr/local/lib:/usr/lib:$LD_LIBRARY_PATH
+$ sudo -E echo "/usr/local/lib" >> /etc/ld.so.conf
+```
+##### 2.调整虚拟机时间为本地时间（否则编译的过程中会输出`Clock skew detected`的警告。这或许不是最好的方法，但却很简单有效，大家也可以自己找其他的方法消除警告）
+时间调整命令的格式如下： date -s 2020-09-28
+
+##### 3.建议创建一个合适的目录，如：build_isulad(后面编译的源码都将放在这个目录下面，每次`git clone`的时候，请先切换到此目录)
+##### 4.protobuf、grpc需要匹配所用系统的编译器版本，如：grpc-1.22 不支持GCC 9+。由于目前使用的虚拟机下编译器版本都一致，这一点不需要考虑
+
+### 源码编译安装protobuf（这部分的编译安装相对于参考的[build_guide](https://gitee.com/openeuler/iSulad/blob/master/docs/build_guide.md),做了比较大的调整，以满足后面的grpc能够顺利编译）
+```javascript
+$ pkg-config --cflags protobuf
+$ pkg-config --libs protobuf
+$ pkg-config --cflags --libs protobuf
+
+
+$ git clone https://gitee.com/src-openeuler/protobuf.git
+$ cd protobuf
+$ git checkout openEuler-20.03-LTS-tag
+$ tar -xzvf protobuf-all-3.9.0.tar.gz
+$ cd protobuf-3.9.0
+```
+> 此过程参考了[stack overflow](https://stackoverflow.com/questions/53586540/c-terminate-called-after-throwing-an-instance-of-stdsystem-error),如果按照[build_guide](https://gitee.com/openeuler/iSulad/blob/master/docs/build_guide.md)编译，在编译grpc时，会报` 'std::system_error'`这样的问题。
+在编译之前要对文件做一些修改,使用如下命令打开protobuf源文件下的src/google/protobuf/stubs/common.cc文件：
+```
+vi src/google/protobuf/stubs/common.cc
+```
+在这个文件中，把有关 _WIN32 的所有代码都注释掉,如下：
+```
+// updated by Aimer on linux platform
+
+//#ifdef _WIN32
+//#define WIN32_LEAN_AND_MEAN // We only need minimal includes
+//#include <windows.h>
+//#define snprintf _snprintf // see comment in strutil.cc
+//#elif defined(HAVE_PTHREAD)
+#include <pthread.h>
+//#else
+//#error "No suitable threading library available."
+//#endif
+```
+>此处参考了[protobuf 安装流程](http://blog.chinaunix.net/uid-28595538-id-5082366.html)
+``` shell
+$ sudo -E ./autogen.sh
+$ sudo -E ./configure CXXFLAGS="$(pkg-config --cflags protobuf)" LIBS="$(pkg-config --libs protobuf)"
+$ sudo -E make -j $(nproc)
+$ sudo -E make install
+$ sudo -E ldconfig
+```
+
+#### 编译成功验证
+```
+protoc --version
+```
+输出：libprotoc 3.9.0(或其他的版本号)
+#### 第二种安装方法
+由protobuf和grpc的安装的依赖关系，我们可以将它们视为一个组合，除了顺次编译外，还可以先编译grpc，再在third_party文件夹下的protobuf目录下安装protobuf，相关的编译方法网上能查到一些（可以搜protobuf+grpc编译），之前试了一下但编译成功率很低。
+
+### 源码编译安装c-cares
+
+```shell
+$ git clone https://gitee.com/src-openeuler/c-ares.git
+$ cd c-ares
+$ git checkout openEuler-20.03-LTS-tag
+$ tar -xzvf c-ares-1.15.0.tar.gz
+$ cd c-ares-1.15.0
+$ sudo -E autoreconf -if
+$ sudo -E ./configure --enable-shared --disable-dependency-tracking
+$ sudo -E make -j $(nproc)
+$ sudo -E make install
+$ sudo -E ldconfig
+```
+### 源码编译安装grpc
+
+```shell
+$ git clone https://gitee.com/src-openeuler/grpc.git
+$ cd grpc
+$ git checkout openEuler-20.03-LTS-tag
+$ tar -xzvf grpc-1.22.0.tar.gz
+$ cd grpc-1.22.0
+```
+修改源码：
+
+* 在`include/grpcpp/impl/codegen/call_op_set.h` line 90添加
+
+```shell
+ /// Default assignment operator
+ WriteOptions& operator=(const WriteOptions& other) = default;
+```
+
+* 将`src/core/lib/gpr/log_linux.cc`、`src/core/lib/gpr/log_posix.cc`、`src/core/lib/iomgr/ev_epollex_linux.cc`这几个文件中的
+ `gettid()`改为`sys_gettid()`
+
+>参考[protobuf+grpc源码编译安装过程](https://blog.csdn.net/Sindweller5530/article/details/104414856)
+
+```shell
+$ sudo -E make -j $(nproc)
+$ sudo -E make install
+$ sudo -E ldconfig
+```
+之后会遇到'cannot find -latomic'的问题,按[链接中的](https://www.cnblogs.com/mafy/p/13380332.html)处理即可:
+
+
+grpc测试用例
+```
+cd examples/cpp/helloworld/
+make //编译
+./greeter_server //服务器
+./greeter_client //客户端（重新开一个服务器连接）
+```
+### 源码编译安装http-parser
+
+```javascript
+$ git clone https://gitee.com/src-openeuler/http-parser.git
+$ cd http-parser
+$ git checkout openEuler-20.03-LTS-tag
+$ tar -xzvf http-parser-2.9.2.tar.gz
+$ cd http-parser-2.9.2
+$ sudo -E make -j CFLAGS="-Wno-error"
+$ sudo -E make CFLAGS="-Wno-error" install
+$ sudo -E ldconfig
+```
+
+
+### 源码编译安装libwebsockets
+```shell
+$ git clone https://gitee.com/src-openeuler/libwebsockets.git
+$ cd libwebsockets
+$ git checkout openEuler-20.03-LTS-tag
+$ tar -xzvf libwebsockets-2.4.2.tar.gz
+$ cd libwebsockets-2.4.2
+$ patch -p1 -F1 -s < ../libwebsockets-fix-coredump.patch
+$ mkdir build
+$ cd build
+$ sudo -E cmake -DLWS_WITH_SSL=0 -DLWS_MAX_SMP=32 -DCMAKE_BUILD_TYPE=Debug ../
+$ sudo -E make -j $(nproc)
+$ sudo -E make install
+$ sudo -E ldconfig
+```
+
+### 源码编译安装lxc
+```shell
+$ git clone https://gitee.com/src-openeuler/lxc.git
+$ cd lxc
+$ tar -zxf lxc-4.0.3.tar.gz
+$ ./apply-patches
+$ cd lxc-4.0.3
+$ sudo -E ./autogen.sh
+$ sudo -E ./configure
+$ sudo -E make -j
+$ sudo -E make install
+$ sudo -E ldconfig
+```
+在编译的过程中会遇到两个问题：
+1. 关于`__NR_signalfd`
+解决方案：[lxc的issue](https://github.com/lxc/lxc/pull/3501/files)
+2. 再次遇到'cannot find -latomic'的问题
+这次不能使用上次的方法，这次是缺少静态链接库，使用find命令搜到libatomic.a复制到/usr/lib下，编译通过。
+### 源码编译安装lcr
+```shell
+$ git clone https://gitee.com/openeuler/lcr.git
+$ cd lcr
+$ mkdir build
+$ cd build
+$ sudo -E cmake ..
+$ sudo -E make -j
+$ sudo -E make install
+$ sudo -E ldconfig
+```
+### 源码编译安装clibcni
+```shell
+$ git clone https://gitee.com/openeuler/clibcni.git
+$ cd clibcni
+$ mkdir build
+$ cd build
+$ sudo -E cmake ..
+$ sudo -E make -j
+$ sudo -E make install
+$ sudo -E ldconfig
+```
+### 源码编译安装iSulad
+```shell
+$ git clone https://gitee.com/openeuler/iSulad.git
+$ cd iSulad
+$ mkdir build
+$ cd build
+$ sudo -E cmake ..
+$ sudo -E make
+$ sudo -E make install
+$ sudo -E ldconfig
+```
+## 内核编译及内核模块的编译
+在完成上述工作之后，iSulad的启动还需要一个`overlay`的内核模块。虚拟机镜像默认没有提供，需要我们开启此模块和编译封装。
+1. 下载与当前镜像系统一致的版本的内核源码（内核版本可以使用`uname -a`命令来查看）
+```shell
+git clone https://gitee.com/openeuler/kernel.git
+git checkout 某一分支
+```
+2. 在内核源码的目录下，执行make menuconfig，在配置界面找到File systems ---> 在Overlay filesystem support前配置成[M]或[*]（单击空格键切换），之后保存并退出；
+3. 使用make Image命令，在/内核源码路径/arch/riscv/boot/ 下生成Image文件；
+4. 下载内核封装工具opensbi:
+```shell
+git clone https://gitee.com/src-openeuler/opensbi.git
+cd opensbi
+unzip v0.6.zip
+cd opensbi-0.6
+make O=build-oe/qemu-virt PLATFORM=qemu/virt FW_PAYLOAD=y FW_PAYLOAD_PATH=/生成的Image路径/Image
+```
+这一步会生成elf文件，编译结束会提示elf文件所在位置。
+5. 将elf文件拷贝至host,拷贝可以使用`scp`工具进行。将.qcow2文件、.elf文件、.sh文件放在同一路径下，修改run_oe1-rv64.sh中的kernel 参数处的elf文件名为新添加的elf文件名。
+6. 执行sh run_oe1-rv64.sh
+### 参考链接:
+* https://arkingc.github.io/2018/09/05/2018-09-05-linux-kernel/
+* https://gitee.com/src-openeuler/risc-v-kernel/blob/master/kernel.spec
+* https://gitee.com/src-openeuler/opensbi/blob/master/opensbi.spec
--
2.20.1

190
0009-add-non-root-group.patch Normal file
View File

@ -0,0 +1,190 @@
From 0c21cb71efd5f81164c67f493f6070714ff3c287 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Wed, 21 Oct 2020 09:19:45 -0400
Subject: [PATCH 09/28] add non root group
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
src/cmd/isulad/main.c | 20 ++++++++++++-----
src/common/constants.h | 2 ++
src/daemon/config/isulad_config.c | 31 +-------------------------
src/utils/cutils/utils_file.c | 36 +++++++++++++++++++++++++++++++
src/utils/cutils/utils_file.h | 2 ++
5 files changed, 56 insertions(+), 35 deletions(-)
diff --git a/src/cmd/isulad/main.c b/src/cmd/isulad/main.c
index 7a932b6..9297aad 100644
--- a/src/cmd/isulad/main.c
+++ b/src/cmd/isulad/main.c
@@ -81,20 +81,30 @@ static int create_client_run_path(const char *group)
{
int ret = 0;
const char *rundir = "/var/run/isula";
+
if (group == NULL) {
return -1;
}
- ret = util_mkdir_p(rundir, DEFAULT_SECURE_DIRECTORY_MODE);
- if (ret < 0) {
+
+ if (util_mkdir_p(rundir, ISULA_CLIENT_DIRECTORY_MODE) < 0) {
ERROR("Unable to create client run directory %s.", rundir);
- return ret;
+ ret = -1;
+ goto out;
}
- ret = chmod(rundir, DEFAULT_SECURE_DIRECTORY_MODE);
- if (ret < 0) {
+ if (chmod(rundir, ISULA_CLIENT_DIRECTORY_MODE) < 0) {
ERROR("Failed to chmod for client run path: %s", rundir);
+ ret = -1;
+ goto out;
}
+ if (util_set_file_group(rundir, group) != 0) {
+ ERROR("set group of the path: %s failed", rundir);
+ ret = -1;
+ goto out;
+ }
+
+out:
return ret;
}
diff --git a/src/common/constants.h b/src/common/constants.h
index 420ac92..52bb0a8 100644
--- a/src/common/constants.h
+++ b/src/common/constants.h
@@ -26,6 +26,8 @@ extern "C" {
#define DEFAULT_SECURE_DIRECTORY_MODE 0750
+#define ISULA_CLIENT_DIRECTORY_MODE 0770
+
#define USER_REMAP_DIRECTORY_MODE 0751
#define ROOTFS_MNT_DIRECTORY_MODE 0640
diff --git a/src/daemon/config/isulad_config.c b/src/daemon/config/isulad_config.c
index c79c6a1..4832985 100644
--- a/src/daemon/config/isulad_config.c
+++ b/src/daemon/config/isulad_config.c
@@ -13,7 +13,6 @@
* Description: provide container configure definition
******************************************************************************/
#include <unistd.h>
-#include <grp.h>
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
@@ -1157,34 +1156,6 @@ out:
return ret;
}
-/* set path group */
-static int set_path_group(const char *rpath, const char *group)
-{
- struct group *grp = NULL;
- gid_t gid;
-
- grp = getgrnam(group);
-
- if (grp != NULL) {
- gid = grp->gr_gid;
- DEBUG("Group %s found, gid: %d", group, gid);
- if (chown(rpath, -1, gid) != 0) {
- DEBUG("Failed to chown %s to gid: %d", rpath, gid);
- return -1;
- }
- } else {
- if (strcmp(group, "docker") == 0 || strcmp(group, "isula") == 0) {
- DEBUG("Warning: could not change group %s to %s", rpath, group);
- } else {
- ERROR("Group %s not found", group);
- isulad_set_error_message("Group %s not found", group);
- return -1;
- }
- }
-
- return 0;
-}
-
/* set socket group */
int set_unix_socket_group(const char *socket, const char *group)
{
@@ -1205,7 +1176,7 @@ int set_unix_socket_group(const char *socket, const char *group)
goto out;
}
INFO("set socket: %s with group: %s", socket, group);
- nret = set_path_group(rpath, group);
+ nret = util_set_file_group(rpath, group);
if (nret < 0) {
ERROR("set group of the path: %s failed", rpath);
ret = -1;
diff --git a/src/utils/cutils/utils_file.c b/src/utils/cutils/utils_file.c
index 7a965c0..92e032b 100644
--- a/src/utils/cutils/utils_file.c
+++ b/src/utils/cutils/utils_file.c
@@ -29,6 +29,8 @@
#include <dirent.h>
#include <fcntl.h>
#include <limits.h>
+#include <sys/types.h>
+#include <grp.h>
#include "constants.h"
#include "isula_libutils/log.h"
@@ -1574,3 +1576,37 @@ out:
free(line);
return ret;
}
+
+int util_set_file_group(const char *fname, const char *group)
+{
+ int ret = 0;
+ struct group *grp = NULL;
+ gid_t gid;
+
+ if (fname == NULL || group == NULL) {
+ ERROR("Invalid NULL params");
+ return -1;
+ }
+
+ grp = getgrnam(group);
+ if (grp != NULL) {
+ gid = grp->gr_gid;
+ DEBUG("Group %s found, gid: %d", group, gid);
+ if (chown(fname, -1, gid) != 0) {
+ ERROR("Failed to chown %s to gid: %d", fname, gid);
+ ret = -1;
+ goto out;
+ }
+ } else {
+ if (strcmp(group, "docker") == 0 || strcmp(group, "isula") == 0) {
+ DEBUG("Warning: could not change group %s to %s", fname, group);
+ } else {
+ ERROR("Group %s not found", group);
+ ret = -1;
+ goto out;
+ }
+ }
+
+out:
+ return ret;
+}
diff --git a/src/utils/cutils/utils_file.h b/src/utils/cutils/utils_file.h
index 1bd2d69..3aff3d6 100644
--- a/src/utils/cutils/utils_file.h
+++ b/src/utils/cutils/utils_file.h
@@ -98,6 +98,8 @@ typedef bool (*read_line_callback_t)(const char *, void *context);
int util_proc_file_line_by_line(FILE *fp, read_line_callback_t cb, void *context);
+int util_set_file_group(const char *fname, const char *group);
+
#ifdef __cplusplus
}
#endif
--
2.20.1

90
0010-add-nonroot-execute-CI.patch Normal file
View File

@ -0,0 +1,90 @@
From 79b2027b87dd1b7f110d73721b8d47941d993e0e Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Mon, 26 Oct 2020 13:57:07 +0800
Subject: [PATCH 10/28] add nonroot execute CI
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
CI/test_cases/container_cases/nonroot.sh | 70 ++++++++++++++++++++++++
1 file changed, 70 insertions(+)
create mode 100755 CI/test_cases/container_cases/nonroot.sh
diff --git a/CI/test_cases/container_cases/nonroot.sh b/CI/test_cases/container_cases/nonroot.sh
new file mode 100755
index 0000000..b123d70
--- /dev/null
+++ b/CI/test_cases/container_cases/nonroot.sh
@@ -0,0 +1,70 @@
+#!/bin/bash
+#
+# attributes: isulad inheritance start
+# concurrent: YES
+# spend time: 11
+
+#######################################################################
+##- @Copyright (C) Huawei Technologies., Ltd. 2020. All rights reserved.
+# - iSulad licensed under the Mulan PSL v2.
+# - You can use this software according to the terms and conditions of the Mulan PSL v2.
+# - You may obtain a copy of Mulan PSL v2 at:
+# - http://license.coscl.org.cn/MulanPSL2
+# - THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR
+# - IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR
+# - PURPOSE.
+# - See the Mulan PSL v2 for more details.
+##- @Description:CI
+##- @Author: gaohuatao
+##- @Create: 2020-10-19
+#######################################################################
+
+curr_path=$(dirname $(readlink -f "$0"))
+data_path=$(realpath $curr_path/../data)
+source ../helpers.sh
+group="isula"
+user="nonroot_test"
+container="test_nonroot_user"
+
+function do_test_t()
+{
+ local ret=0
+ local test="isula execute with non root => (${FUNCNAME[@]})"
+
+ msg_info "${test} starting..."
+
+ userdel $user
+ useradd -g $group $user
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - add user $user and add to group $group failed" && ((ret++))
+
+ su - $user -c "isula run -tid --name $container busybox /bin/bash"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - run container failed" && ((ret++))
+
+ su - $user -c "isula inspect $container"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - inspect container failed" && ((ret++))
+
+ su - $user -c "isula restart $container"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - restart container failed" && ((ret++))
+
+ su - $user -c "isula exec $container pwd"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - exec container failed" && ((ret++))
+
+ su - $user -c "isula stop $container"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - stop container failed" && ((ret++))
+
+ su - $user -c "isula rm $container"
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - rm container failed" && ((ret++))
+
+ userdel $user
+
+ return $TC_RET_T
+}
+
+ret=0
+
+do_test_t
+if [ $? -ne 0 ];then
+ let "ret=$ret + 1"
+fi
+
+show_result $ret "basic start"
--
2.20.1

91
0011-cni-support-extension-data-transmission.patch Normal file
View File

@ -0,0 +1,91 @@
From c24f26ead1803db98fd2cd7e4d95a34239c15221 Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Mon, 26 Oct 2020 11:13:53 +0800
Subject: [PATCH 11/28] cni: support extension data transmission
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
src/daemon/entry/cri/cni_network_plugin.cc | 15 +++++++++++++--
src/daemon/entry/cri/cri_helpers.cc | 2 ++
src/daemon/entry/cri/cri_helpers.h | 2 ++
3 files changed, 17 insertions(+), 2 deletions(-)
diff --git a/src/daemon/entry/cri/cni_network_plugin.cc b/src/daemon/entry/cri/cni_network_plugin.cc
index c1a4e1e..4676a97 100644
--- a/src/daemon/entry/cri/cni_network_plugin.cc
+++ b/src/daemon/entry/cri/cni_network_plugin.cc
@@ -581,10 +581,11 @@ void CniNetworkPlugin::DeleteFromNetwork(CNINetwork *network, const std::string
static void PrepareRuntimeConf(const std::string &podName, const std::string &podNs, const std::string &interfaceName,
const std::string &podSandboxID, const std::string &podNetnsPath,
+ const std::map<std::string, std::string> &annotations,
const std::map<std::string, std::string> &options, struct runtime_conf **cni_rc,
Errors &err)
{
- const size_t defaultLen = 5;
+ size_t defaultLen = 5;
if (cni_rc == nullptr) {
err.Errorf("Invalid arguments");
ERROR("Invalid arguments");
@@ -596,6 +597,12 @@ static void PrepareRuntimeConf(const std::string &podName, const std::string &po
if (iter != options.end()) {
podUID = iter->second;
}
+ std::string cniExtentionVal;
+ iter = annotations.find(CRIHelpers::Constants::CNI_MUTL_NET_EXTENSION_KEY);
+ if (iter != annotations.end()) {
+ cniExtentionVal = iter->second;
+ defaultLen++;
+ }
struct runtime_conf *rt = (struct runtime_conf *)util_common_calloc_s(sizeof(struct runtime_conf));
if (rt == nullptr) {
@@ -625,6 +632,10 @@ static void PrepareRuntimeConf(const std::string &podName, const std::string &po
rt->args[3][1] = util_strdup_s(podSandboxID.c_str());
rt->args[4][0] = util_strdup_s("K8S_POD_UID");
rt->args[4][1] = util_strdup_s(podUID.c_str());
+ if (defaultLen > 5) {
+ rt->args[5][0] = util_strdup_s(CRIHelpers::Constants::CNI_MUTL_NET_EXTENSION_ARGS_KEY.c_str());
+ rt->args[5][1] = util_strdup_s(cniExtentionVal.c_str());
+ }
*cni_rc = rt;
return;
@@ -639,7 +650,7 @@ void CniNetworkPlugin::BuildCNIRuntimeConf(const std::string &podName, const std
const std::map<std::string, std::string> &options,
struct runtime_conf **cni_rc, Errors &err)
{
- PrepareRuntimeConf(podName, podNs, interfaceName, podSandboxID, podNetnsPath, options, cni_rc, err);
+ PrepareRuntimeConf(podName, podNs, interfaceName, podSandboxID, podNetnsPath, annotations, options, cni_rc, err);
if (err.NotEmpty()) {
return;
}
diff --git a/src/daemon/entry/cri/cri_helpers.cc b/src/daemon/entry/cri/cri_helpers.cc
index 8aa939c..4fb9feb 100644
--- a/src/daemon/entry/cri/cri_helpers.cc
+++ b/src/daemon/entry/cri/cri_helpers.cc
@@ -51,6 +51,8 @@ const std::string Constants::CONTAINER_TYPE_ANNOTATION_SANDBOX { "sandbox" };
const std::string Constants::SANDBOX_ID_ANNOTATION_KEY { "io.kubernetes.cri.sandbox-id" };
const std::string Constants::NET_PLUGIN_EVENT_POD_CIDR_CHANGE { "pod-cidr-change" };
const std::string Constants::NET_PLUGIN_EVENT_POD_CIDR_CHANGE_DETAIL_CIDR { "pod-cidr" };
+const std::string Constants::CNI_MUTL_NET_EXTENSION_KEY { "extension.network.kubernetes.io/cni" };
+const std::string Constants::CNI_MUTL_NET_EXTENSION_ARGS_KEY { "CNI_MUTLINET_EXTENSION" };
const char *InternalLabelKeys[] = { CRIHelpers::Constants::CONTAINER_TYPE_LABEL_KEY.c_str(),
CRIHelpers::Constants::CONTAINER_LOGPATH_LABEL_KEY.c_str(),
diff --git a/src/daemon/entry/cri/cri_helpers.h b/src/daemon/entry/cri/cri_helpers.h
index 9dd3630..824d1a6 100644
--- a/src/daemon/entry/cri/cri_helpers.h
+++ b/src/daemon/entry/cri/cri_helpers.h
@@ -56,6 +56,8 @@ public:
static const std::string NET_PLUGIN_EVENT_POD_CIDR_CHANGE;
static const std::string NET_PLUGIN_EVENT_POD_CIDR_CHANGE_DETAIL_CIDR;
+ static const std::string CNI_MUTL_NET_EXTENSION_KEY;
+ static const std::string CNI_MUTL_NET_EXTENSION_ARGS_KEY;
};
auto GetDefaultSandboxImage(Errors &err) -> std::string;
--
2.20.1

386
0012-unpack-add-remove-target-file-in-handle-.wh.patch Normal file
View File

@ -0,0 +1,386 @@
From 9cbd114034321e232dfe2540216c9c8c3094e362 Mon Sep 17 00:00:00 2001
From: lifeng68 <lifeng68@huawei.com>
Date: Tue, 27 Oct 2020 16:31:37 +0800
Subject: [PATCH 12/28] unpack: add remove target file in handle .wh.
Signed-off-by: lifeng68 <lifeng68@huawei.com>
---
src/cmd/isulad/main.c | 2 +-
.../graphdriver/devmapper/driver_devmapper.c | 2 +-
src/utils/cutils/utils_file.c | 29 ++-
src/utils/cutils/utils_file.h | 3 +
src/utils/tar/util_archive.c | 172 +++++++++++++++++-
src/utils/tar/util_archive.h | 5 +-
6 files changed, 198 insertions(+), 15 deletions(-)
diff --git a/src/cmd/isulad/main.c b/src/cmd/isulad/main.c
index 9297aad..5cad285 100644
--- a/src/cmd/isulad/main.c
+++ b/src/cmd/isulad/main.c
@@ -103,7 +103,7 @@ static int create_client_run_path(const char *group)
ret = -1;
goto out;
}
-
+
out:
return ret;
}
diff --git a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/driver_devmapper.c b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/driver_devmapper.c
index ec337a8..1674c28 100644
--- a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/driver_devmapper.c
+++ b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/devmapper/driver_devmapper.c
@@ -339,7 +339,7 @@ int devmapper_apply_diff(const char *id, const struct graphdriver *driver, const
goto out;
}
- options.whiteout_format = OVERLAY_WHITEOUT_FORMATE;
+ options.whiteout_format = REMOVE_WHITEOUT_FORMATE;
if (archive_unpack(content, layer_fs, &options) != 0) {
ERROR("devmapper: failed to unpack to :%s", layer_fs);
ret = -1;
diff --git a/src/utils/cutils/utils_file.c b/src/utils/cutils/utils_file.c
index 92e032b..9f7f5fe 100644
--- a/src/utils/cutils/utils_file.c
+++ b/src/utils/cutils/utils_file.c
@@ -282,7 +282,7 @@ static int recursive_rmdir_helper(const char *dirpath, int recursive_depth, int
struct dirent *pdirent = NULL;
DIR *directory = NULL;
int failure = 0;
- char fname[MAXPATHLEN];
+ char fname[PATH_MAX];
directory = opendir(dirpath);
if (directory == NULL) {
@@ -300,8 +300,8 @@ static int recursive_rmdir_helper(const char *dirpath, int recursive_depth, int
(void)memset(fname, 0, sizeof(fname));
- pathname_len = snprintf(fname, MAXPATHLEN, "%s/%s", dirpath, pdirent->d_name);
- if (pathname_len < 0 || pathname_len >= MAXPATHLEN) {
+ pathname_len = snprintf(fname, PATH_MAX, "%s/%s", dirpath, pdirent->d_name);
+ if (pathname_len < 0 || pathname_len >= PATH_MAX) {
ERROR("Pathname too long");
failure = 1;
continue;
@@ -1142,7 +1142,7 @@ static void recursive_cal_dir_size_helper(const char *dirpath, int recursive_dep
struct dirent *pdirent = NULL;
DIR *directory = NULL;
struct stat fstat;
- char fname[MAXPATHLEN];
+ char fname[PATH_MAX];
// cal dir self node and size
nret = lstat(dirpath, &fstat);
@@ -1169,8 +1169,8 @@ static void recursive_cal_dir_size_helper(const char *dirpath, int recursive_dep
(void)memset(fname, 0, sizeof(fname));
- pathname_len = snprintf(fname, MAXPATHLEN, "%s/%s", dirpath, pdirent->d_name);
- if (pathname_len < 0 || pathname_len >= MAXPATHLEN) {
+ pathname_len = snprintf(fname, PATH_MAX, "%s/%s", dirpath, pdirent->d_name);
+ if (pathname_len < 0 || pathname_len >= PATH_MAX) {
ERROR("Pathname too long");
continue;
}
@@ -1239,7 +1239,7 @@ static void recursive_cal_dir_size__without_hardlink_helper(const char *dirpath,
int nret = 0;
struct dirent *pdirent = NULL;
DIR *directory = NULL;
- char fname[MAXPATHLEN];
+ char fname[PATH_MAX];
directory = opendir(dirpath);
if (directory == NULL) {
@@ -1257,8 +1257,8 @@ static void recursive_cal_dir_size__without_hardlink_helper(const char *dirpath,
(void)memset(fname, 0, sizeof(fname));
- pathname_len = snprintf(fname, MAXPATHLEN, "%s/%s", dirpath, pdirent->d_name);
- if (pathname_len < 0 || pathname_len >= MAXPATHLEN) {
+ pathname_len = snprintf(fname, PATH_MAX, "%s/%s", dirpath, pdirent->d_name);
+ if (pathname_len < 0 || pathname_len >= PATH_MAX) {
ERROR("Pathname too long");
continue;
}
@@ -1610,3 +1610,14 @@ int util_set_file_group(const char *fname, const char *group)
out:
return ret;
}
+
+int util_recursive_remove_path(const char *path)
+{
+ int ret = 0;
+
+ if (unlink(path) != 0 && errno != ENOENT) {
+ ret = util_recursive_rmdir(path, 0);
+ }
+
+ return ret;
+}
\ No newline at end of file
diff --git a/src/utils/cutils/utils_file.h b/src/utils/cutils/utils_file.h
index 3aff3d6..a873114 100644
--- a/src/utils/cutils/utils_file.h
+++ b/src/utils/cutils/utils_file.h
@@ -100,6 +100,9 @@ int util_proc_file_line_by_line(FILE *fp, read_line_callback_t cb, void *context
int util_set_file_group(const char *fname, const char *group);
+// try to remove the path, path is file or dir
+int util_recursive_remove_path(const char *path);
+
#ifdef __cplusplus
}
#endif
diff --git a/src/utils/tar/util_archive.c b/src/utils/tar/util_archive.c
index 0ae99be..04603a2 100644
--- a/src/utils/tar/util_archive.c
+++ b/src/utils/tar/util_archive.c
@@ -61,7 +61,7 @@ ssize_t read_content(struct archive *a, void *client_data, const void **buff)
return mydata->content->read(mydata->content->context, mydata->buff, sizeof(mydata->buff));
}
-static bool whiteout_convert_read(struct archive_entry *entry, const char *dst_path)
+static bool overlay_whiteout_convert_read(struct archive_entry *entry, const char *dst_path, map_t *unpacked_path_map)
{
bool do_write = true;
char *base = NULL;
@@ -143,6 +143,149 @@ static int copy_data(struct archive *ar, struct archive *aw)
}
}
+static int remove_files_in_opq_dir(const char *dirpath, int recursive_depth, map_t *unpacked_path_map)
+{
+ struct dirent *pdirent = NULL;
+ DIR *directory = NULL;
+ int ret = 0;
+ char fname[PATH_MAX] = { 0 };
+
+ if ((recursive_depth + 1) > MAX_PATH_DEPTH) {
+ ERROR("Reach max path depth: %s", dirpath);
+ return -1;
+ }
+
+ directory = opendir(dirpath);
+ if (directory == NULL) {
+ ERROR("Failed to open %s", dirpath);
+ return -1;
+ }
+ pdirent = readdir(directory);
+ for (; pdirent != NULL; pdirent = readdir(directory)) {
+ struct stat fstat;
+ int pathname_len;
+
+ if (!strcmp(pdirent->d_name, ".") || !strcmp(pdirent->d_name, "..")) {
+ continue;
+ }
+
+ (void)memset(fname, 0, sizeof(fname));
+
+ pathname_len = snprintf(fname, PATH_MAX, "%s/%s", dirpath, pdirent->d_name);
+ if (pathname_len < 0 || pathname_len >= PATH_MAX) {
+ ERROR("Pathname too long");
+ ret = -1;
+ continue;
+ }
+
+ // not exist in unpacked paths map, just remove the path
+ if (map_search(unpacked_path_map, (void *)fname) == NULL) {
+ if (util_recursive_remove_path(fname) != 0) {
+ ERROR("Failed to remove path %s", fname);
+ ret = -1;
+ }
+ continue;
+ }
+
+ if (lstat(fname, &fstat) != 0) {
+ ERROR("Failed to stat %s", fname);
+ ret = -1;
+ continue;
+ }
+
+ if (S_ISDIR(fstat.st_mode)) {
+ if (remove_files_in_opq_dir(fname, recursive_depth + 1, unpacked_path_map) != 0) {
+ ret = -1;
+ continue;
+ }
+ }
+ }
+
+ if (closedir(directory) != 0) {
+ ERROR("Failed to close directory %s", dirpath);
+ ret = -1;
+ }
+
+ return ret;
+}
+
+static bool remove_whiteout_convert(struct archive_entry *entry, const char *dst_path, map_t *unpacked_path_map)
+{
+ bool do_write = true;
+ char *base = NULL;
+ char *dir = NULL;
+ char *originalpath = NULL;
+
+ base = util_path_base(dst_path);
+ if (base == NULL) {
+ ERROR("Failed to get base of %s", dst_path);
+ goto out;
+ }
+
+ dir = util_path_dir(dst_path);
+ if (dir == NULL) {
+ ERROR("Failed to get dir of %s", dst_path);
+ goto out;
+ }
+
+ if (strcmp(base, WHITEOUT_OPAQUEDIR) == 0) {
+ if (remove_files_in_opq_dir(dir, 0, unpacked_path_map) != 0) {
+ SYSERROR("Failed to remove files in opq dir %s", dir);
+ goto out;
+ }
+ do_write = false;
+ goto out;
+ }
+
+ if (strncmp(base, WHITEOUT_PREFIX, strlen(WHITEOUT_PREFIX)) == 0) {
+ char *origin_base = &base[strlen(WHITEOUT_PREFIX)];
+ originalpath = util_path_join(dir, origin_base);
+ if (originalpath == NULL) {
+ ERROR("Failed to get original path of %s", dst_path);
+ goto out;
+ }
+
+ if (util_recursive_remove_path(originalpath) != 0) {
+ ERROR("Failed to delete original path %s", originalpath);
+ goto out;
+ }
+
+ do_write = false;
+ goto out;
+ }
+
+out:
+ free(base);
+ free(dir);
+ free(originalpath);
+ return do_write;
+}
+
+typedef bool (*whiteout_convert_call_back_t)(struct archive_entry *entry, const char *dst_path,
+ map_t *unpacked_path_map);
+
+struct whiteout_convert_map {
+ whiteout_format_type type;
+ whiteout_convert_call_back_t wh_cb;
+};
+
+struct whiteout_convert_map g_wh_cb_map[] = { { OVERLAY_WHITEOUT_FORMATE, overlay_whiteout_convert_read },
+ { REMOVE_WHITEOUT_FORMATE, remove_whiteout_convert }
+};
+
+static whiteout_convert_call_back_t get_whiteout_convert_cb(whiteout_format_type whiteout_type)
+{
+ size_t i = 0;
+
+ for (i = 0; i < sizeof(g_wh_cb_map) / sizeof(g_wh_cb_map[0]); i++) {
+ if (whiteout_type == g_wh_cb_map[i].type) {
+ return g_wh_cb_map[i].wh_cb;
+ }
+ }
+
+ return NULL;
+}
+
int archive_unpack_handler(const struct io_read_wrapper *content, const char *dstdir,
const struct archive_options *options)
{
@@ -153,6 +296,15 @@ int archive_unpack_handler(const struct io_read_wrapper *content, const char *ds
struct archive_entry *entry = NULL;
char *dst_path = NULL;
int flags;
+ whiteout_convert_call_back_t wh_handle_cb = NULL;
+ map_t *unpacked_path_map = NULL; // used for hanling opaque dir, marke paths had been unpacked
+
+ unpacked_path_map = map_new(MAP_STR_BOOL, MAP_DEFAULT_CMP_FUNC, MAP_DEFAULT_FREE_FUNC);
+ if (unpacked_path_map == NULL) {
+ ERROR("Out of memory");
+ ret = -1;
+ goto out;
+ }
mydata = util_common_calloc_s(sizeof(struct archive_content_data));
if (mydata == NULL) {
@@ -187,6 +339,8 @@ int archive_unpack_handler(const struct io_read_wrapper *content, const char *ds
goto out;
}
+ wh_handle_cb = get_whiteout_convert_cb(options->whiteout_format);
+
for (;;) {
free(dst_path);
dst_path = NULL;
@@ -217,28 +371,42 @@ int archive_unpack_handler(const struct io_read_wrapper *content, const char *ds
goto out;
}
- if (options->whiteout_format == OVERLAY_WHITEOUT_FORMATE && !whiteout_convert_read(entry, dst_path)) {
+ if (wh_handle_cb != NULL && !wh_handle_cb(entry, dst_path, unpacked_path_map)) {
continue;
}
ret = archive_write_header(ext, entry);
if (ret != ARCHIVE_OK) {
ERROR("Fail to handle tar header: %s", archive_error_string(ext));
+ ret = -1;
+ goto out;
} else if (archive_entry_size(entry) > 0) {
ret = copy_data(a, ext);
if (ret != ARCHIVE_OK) {
ERROR("Failed to do copy tar data: %s", archive_error_string(ext));
+ ret = -1;
+ goto out;
}
}
ret = archive_write_finish_entry(ext);
if (ret != ARCHIVE_OK) {
ERROR("Failed to freeing archive entry: %s\n", archive_error_string(ext));
+ ret = -1;
+ goto out;
+ }
+
+ bool b = true;
+ if (!map_replace(unpacked_path_map, (void *)dst_path, (void *)(&b))) {
+ ERROR("Failed to replace unpacked path map element");
+ ret = -1;
+ goto out;
}
}
ret = 0;
out:
+ map_free(unpacked_path_map);
free(dst_path);
archive_read_close(a);
archive_read_free(a);
diff --git a/src/utils/tar/util_archive.h b/src/utils/tar/util_archive.h
index 4c4e4a1..6f65daa 100644
--- a/src/utils/tar/util_archive.h
+++ b/src/utils/tar/util_archive.h
@@ -30,8 +30,9 @@ extern "C" {
#endif
typedef enum {
- NONE_WHITEOUT_FORMATE = 0,
- OVERLAY_WHITEOUT_FORMATE = 1,
+ NONE_WHITEOUT_FORMATE = 0, // handle whiteouts as normal files
+ OVERLAY_WHITEOUT_FORMATE = 1, // handle whiteouts as the way as overlay
+ REMOVE_WHITEOUT_FORMATE = 2, // handle whiteouts by removing the target files
} whiteout_format_type;
struct archive_options {
--
2.20.1

476
0013-iSulad-internal-change.patch Normal file
View File

@ -0,0 +1,476 @@
From 8eaec71c30695317bb91c881824e201c4009db67 Mon Sep 17 00:00:00 2001
From: lifeng68 <lifeng68@huawei.com>
Date: Thu, 29 Oct 2020 14:43:36 +0800
Subject: [PATCH 13/28] iSulad: internal change
Signed-off-by: lifeng68 <lifeng68@huawei.com>
---
cmake/protoc.cmake | 20 +----
src/api/services/cri/api.proto | 3 +
src/api/services/health/health.proto | 54 ------------
src/api/services/images/images.proto | 88 +++----------------
src/api/types/descriptor.proto | 49 -----------
src/client/connect/CMakeLists.txt | 4 +-
src/daemon/entry/connect/CMakeLists.txt | 4 +-
.../entry/connect/grpc/grpc_images_service.h | 8 +-
.../graphdriver/quota/project_quota.h | 56 +++++-------
9 files changed, 43 insertions(+), 243 deletions(-)
delete mode 100644 src/api/services/health/health.proto
delete mode 100644 src/api/types/descriptor.proto
diff --git a/cmake/protoc.cmake b/cmake/protoc.cmake
index 352c0db..c10d4c3 100644
--- a/cmake/protoc.cmake
+++ b/cmake/protoc.cmake
@@ -1,22 +1,11 @@
set(PROTOS_PATH ${CMAKE_CURRENT_SOURCE_DIR}/src/api/services)
-set(TYPES_PROTOS_PATH ${CMAKE_CURRENT_SOURCE_DIR}/src/api/types)
set(GRPC_OUT_PRE_PATH ${CMAKE_BINARY_DIR}/grpc)
-set(TYPES_PROTOS_OUT_PATH ${GRPC_OUT_PRE_PATH}/src/api/types)
set(CONTAINER_PROTOS_OUT_PATH ${GRPC_OUT_PRE_PATH}/src/api/services/containers)
set(IMAGE_PROTOS_OUT_PATH ${GRPC_OUT_PRE_PATH}/src/api/services/images)
set(CRI_PROTOS_OUT_PATH ${GRPC_OUT_PRE_PATH}/src/api/services/cri)
set(IMAGE_SERVICE_PROTOS_OUT_PATH ${GRPC_OUT_PRE_PATH}/src/api/image_client)
-execute_process(COMMAND mkdir -p ${TYPES_PROTOS_OUT_PATH})
-
-execute_process(COMMAND ${CMD_PROTOC} -I ${TYPES_PROTOS_PATH} --cpp_out=${TYPES_PROTOS_OUT_PATH}
- ${TYPES_PROTOS_PATH}/descriptor.proto ERROR_VARIABLE types_err)
-if (types_err)
- message("Parse types.proto failed: ")
- message(FATAL_ERROR ${types_err})
-endif()
-
if (GRPC_CONNECTOR)
message("---------------Generate GRPC proto-----------------------")
execute_process(COMMAND mkdir -p ${CONTAINER_PROTOS_OUT_PATH})
@@ -29,22 +18,19 @@ if (GRPC_CONNECTOR)
message(FATAL_ERROR ${containers_err})
endif()
- execute_process(COMMAND ${CMD_PROTOC} -I ${PROTOS_PATH}/containers --grpc_out=${CONTAINER_PROTOS_OUT_PATH}
- --plugin=protoc-gen-grpc=${CMD_GRPC_CPP_PLUGIN} ${PROTOS_PATH}/containers/container.proto ERROR_VARIABLE containers_err)
+ execute_process(COMMAND ${CMD_PROTOC} -I ${PROTOS_PATH}/containers --grpc_out=${CONTAINER_PROTOS_OUT_PATH} --plugin=protoc-gen-grpc=${CMD_GRPC_CPP_PLUGIN} ${PROTOS_PATH}/containers/container.proto ERROR_VARIABLE containers_err)
if (containers_err)
message("Parse containers.proto plugin failed: ")
message(FATAL_ERROR ${containers_err})
endif()
- execute_process(COMMAND ${CMD_PROTOC} -I ${PROTOS_PATH}/images -I ${TYPES_PROTOS_PATH}
- --cpp_out=${IMAGE_PROTOS_OUT_PATH} ${PROTOS_PATH}/images/images.proto ERROR_VARIABLE images_err)
+ execute_process(COMMAND ${CMD_PROTOC} -I ${PROTOS_PATH}/images --cpp_out=${IMAGE_PROTOS_OUT_PATH} ${PROTOS_PATH}/images/images.proto ERROR_VARIABLE images_err)
if (images_err)
message("Parse images.proto failed: ")
message(FATAL_ERROR ${images_err})
endif()
- execute_process(COMMAND ${CMD_PROTOC} -I ${PROTOS_PATH}/images -I ${TYPES_PROTOS_PATH} --grpc_out=${IMAGE_PROTOS_OUT_PATH}
- --plugin=protoc-gen-grpc=${CMD_GRPC_CPP_PLUGIN} ${PROTOS_PATH}/images/images.proto ERROR_VARIABLE images_err)
+ execute_process(COMMAND ${CMD_PROTOC} -I ${PROTOS_PATH}/images --grpc_out=${IMAGE_PROTOS_OUT_PATH} --plugin=protoc-gen-grpc=${CMD_GRPC_CPP_PLUGIN} ${PROTOS_PATH}/images/images.proto ERROR_VARIABLE images_err)
if (images_err)
message("Parse images.proto plugin failed: ")
message(FATAL_ERROR ${images_err})
diff --git a/src/api/services/cri/api.proto b/src/api/services/cri/api.proto
index 022fa51..8aba0d3 100644
--- a/src/api/services/cri/api.proto
+++ b/src/api/services/cri/api.proto
@@ -1,5 +1,8 @@
/*
Copyright 2018 The Kubernetes Authors.
+Copyright (C) Huawei Technologies., Ltd. 2019. All rights reserved.
+ modify descripe: remove unused options for example:
+ remove import "github.com/gogo/protobuf/gogoproto/gogo.proto"
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
diff --git a/src/api/services/health/health.proto b/src/api/services/health/health.proto
deleted file mode 100644
index a3d3537..0000000
--- a/src/api/services/health/health.proto
+++ /dev/null
@@ -1,54 +0,0 @@
-// #######################################################################
-// ##- @Copyright (C) Huawei Technologies., Ltd. 2019-2020. All rights reserved.
-// # - iSulad licensed under the Mulan PSL v2.
-// # - You can use this software according to the terms and conditions of the Mulan PSL v2.
-// # - You may obtain a copy of Mulan PSL v2 at:
-// # - http://license.coscl.org.cn/MulanPSL2
-// # - THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR
-// # - IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR
-// # - PURPOSE.
-// # - See the Mulan PSL v2 for more details.
-// ##- @Description: generate grpc
-// ##- @Author: tanyifeng
-// ##- @Create: 2020-01-16
-// #######################################################################
-//
-// Since some of this code is derived from grpc, their copyright
-// is retained here....
-//
-// Copyright 2015 The gRPC Authors
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-// The canonical version of this proto can be found at
-// https://github.com/grpc/grpc-proto/blob/master/grpc/health/v1/health.proto
-
-syntax = "proto3";
-option optimize_for = CODE_SIZE;
-
-message HealthCheckRequest {
- string service = 1;
-}
-
-message HealthCheckResponse {
- enum ServingStatus {
- UNKNOWN = 0;
- SERVING = 1;
- NOT_SERVING = 2;
- }
- ServingStatus status = 1;
-}
-
-service HealthService{
- rpc Check(HealthCheckRequest) returns (HealthCheckResponse);
-}
diff --git a/src/api/services/images/images.proto b/src/api/services/images/images.proto
index bdec2f8..71cce22 100644
--- a/src/api/services/images/images.proto
+++ b/src/api/services/images/images.proto
@@ -9,109 +9,43 @@
// # - PURPOSE.
// # - See the Mulan PSL v2 for more details.
// ##- @Description: generate grpc
-// ##- @Author: wujing
-// ##- @Create: 2020-01-16
+// ##- @Author: lifeng
+// ##- @Create: 2019-04-25
// #######################################################################
-
-/*
-Since some of this code is derived from containerd, their copyright
-is retained here....
-
-Copyright 2013-2016 Docker, Inc.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
syntax = "proto3";
option optimize_for = CODE_SIZE;
import "google/protobuf/timestamp.proto";
-import "descriptor.proto";
package images;
-// Images is a service that allows one to register images with containerd.
-//
-// In containerd, an image is merely the mapping of a name to a content root,
-// described by a descriptor. The behavior and state of image is purely
-// dictated by the type of the descriptor.
-//
-// From the perspective of this service, these references are mostly shallow,
-// in that the existence of the required content won't be validated until
-// required by consuming services.
-//
-// As such, this can really be considered a "metadata service".
service ImagesService {
- // List returns a list of all images known to containerd.
rpc List(ListImagesRequest) returns (ListImagesResponse);
-
- // Delete deletes the image by name.
rpc Delete(DeleteImageRequest) returns (DeleteImageResponse);
-
- // load image from archive.
rpc Load(LoadImageRequest) returns (LoadImageResponse);
-
- //inspect image
rpc Inspect(InspectImageRequest) returns (InspectImageResponse);
-
- // Login to a Docker registry
rpc Login(LoginRequest) returns (LoginResponse);
-
- // Logout from a Docker registry
rpc Logout(LogoutRequest) returns (LogoutResponse);
-
- // Add a tag to the image
rpc Tag(TagImageRequest) returns (TagImageResponse);
-
- // Import rootfs to be image
rpc Import(ImportRequest) returns (ImportResponse);
}
+message Descriptor {
+ string media_type = 1;
+ string digest = 2;
+ int64 size = 3;
+}
+
message Image {
- // Name provides a unique name for the image.
- //
- // Containerd treats this as the primary identifier.
string name = 1;
-
- // Labels provides free form labels for the image. These are runtime only
- // and do not get inherited into the package image in any way.
- //
- // Labels may be updated using the field mask.
- // The combined size of a key/value pair cannot exceed 4096 bytes.
map<string, string> labels = 2;
-
- // Target describes the content entry point of the image.
- containerd.types.Descriptor target = 3;
-
- // CreatedAt is the time the image was first created.
- google.protobuf.Timestamp created_at = 7;
-
- // UpdatedAt is the last time the image was mutated.
- google.protobuf.Timestamp updated_at = 8;
+ Descriptor target = 3;
+ google.protobuf.Timestamp created_at = 4;
+ google.protobuf.Timestamp updated_at = 5;
}
message ListImagesRequest {
- // Filters contains one or more filters using the syntax defined in the
- // containerd filter package.
- //
- // The returned result will be those that match any of the provided
- // filters. Expanded, images that match the following will be
- // returned:
- //
- // filters[0] or filters[1] or ... or filters[n-1] or filters[n]
- //
- // If filters is zero-length or nil, all items will be returned.
map<string, string> filters = 1;
}
diff --git a/src/api/types/descriptor.proto b/src/api/types/descriptor.proto
deleted file mode 100644
index 38b9d19..0000000
--- a/src/api/types/descriptor.proto
+++ /dev/null
@@ -1,49 +0,0 @@
-// #######################################################################
-// ##- @Copyright (C) Huawei Technologies., Ltd. 2019-2020. All rights reserved.
-// # - iSulad licensed under the Mulan PSL v2.
-// # - You can use this software according to the terms and conditions of the Mulan PSL v2.
-// # - You may obtain a copy of Mulan PSL v2 at:
-// # - http://license.coscl.org.cn/MulanPSL2
-// # - THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR
-// # - IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR
-// # - PURPOSE.
-// # - See the Mulan PSL v2 for more details.
-// ##- @Description: generate grpc
-// ##- @Author: wujing
-// ##- @Create: 2019-01-16
-// #######################################################################
-
-/*
-Since some of this code is derived from containerd, their copyright
-is retained here....
-
-Copyright 2013-2016 Docker, Inc.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-syntax = "proto3";
-option optimize_for = CODE_SIZE;
-
-package containerd.types;
-
-// Descriptor describes a blob in a content store.
-//
-// This descriptor can be used to reference content from an
-// oci descriptor found in a manifest.
-// See https://godoc.org/github.com/opencontainers/image-spec/specs-go/v1#Descriptor
-message Descriptor {
- string media_type = 1;
- string digest = 2;
- int64 size = 3;
-}
diff --git a/src/client/connect/CMakeLists.txt b/src/client/connect/CMakeLists.txt
index 60a3429..1097d16 100644
--- a/src/client/connect/CMakeLists.txt
+++ b/src/client/connect/CMakeLists.txt
@@ -9,16 +9,14 @@ add_subdirectory(grpc)
if (GRPC_CONNECTOR)
list(APPEND local_client_connect_srcs ${CLIENT_GRPC_SRCS})
- aux_source_directory(${CMAKE_BINARY_DIR}/grpc/src/api/types CONNECT_API_TYPES)
aux_source_directory(${CMAKE_BINARY_DIR}/grpc/src/api/services/containers CONNECT_API_CONTAINERS)
aux_source_directory(${CMAKE_BINARY_DIR}/grpc/src/api/services/images CONNECT_API_IMAGES)
aux_source_directory(${CMAKE_BINARY_DIR}/grpc/src/api/services/cri CONNECT_API_CRI)
- set(CONNECT_API ${CONNECT_API_TYPES} ${CONNECT_API_CONTAINERS} ${CONNECT_API_IMAGES} ${CONNECT_API_CRI})
+ set(CONNECT_API ${CONNECT_API_CONTAINERS} ${CONNECT_API_IMAGES} ${CONNECT_API_CRI})
list(APPEND local_client_connect_srcs ${CONNECT_API})
list(APPEND local_client_connect_incs ${CMAKE_CURRENT_SOURCE_DIR}/grpc)
list(APPEND local_client_connect_incs
- ${CMAKE_BINARY_DIR}/grpc/src/api/types
${CMAKE_BINARY_DIR}/grpc/src/api/services/containers
${CMAKE_BINARY_DIR}/grpc/src/api/services/images
${CMAKE_BINARY_DIR}/grpc/src/api/services/cri
diff --git a/src/daemon/entry/connect/CMakeLists.txt b/src/daemon/entry/connect/CMakeLists.txt
index 0220319..3b174df 100644
--- a/src/daemon/entry/connect/CMakeLists.txt
+++ b/src/daemon/entry/connect/CMakeLists.txt
@@ -15,15 +15,13 @@ endif()
if (GRPC_CONNECTOR)
# GRPC
- aux_source_directory(${CMAKE_BINARY_DIR}/grpc/src/api/types CONNECT_API_TYPES)
aux_source_directory(${CMAKE_BINARY_DIR}/grpc/src/api/services/containers CONNECT_API_CONTAINERS)
aux_source_directory(${CMAKE_BINARY_DIR}/grpc/src/api/services/images CONNECT_API_IMAGES)
aux_source_directory(${CMAKE_BINARY_DIR}/grpc/src/api/services/cri CONNECT_API_CRI)
- set(CONNECT_API ${CONNECT_API_TYPES} ${CONNECT_API_CONTAINERS} ${CONNECT_API_IMAGES} ${CONNECT_API_CRI})
+ set(CONNECT_API ${CONNECT_API_CONTAINERS} ${CONNECT_API_IMAGES} ${CONNECT_API_CRI})
list(APPEND local_server_connect_srcs ${CONNECT_API})
list(APPEND local_server_connect_incs
- ${CMAKE_BINARY_DIR}/grpc/src/api/types
${CMAKE_BINARY_DIR}/grpc/src/api/services/containers
${CMAKE_BINARY_DIR}/grpc/src/api/services/images
${CMAKE_BINARY_DIR}/grpc/src/api/services/cri
diff --git a/src/daemon/entry/connect/grpc/grpc_images_service.h b/src/daemon/entry/connect/grpc/grpc_images_service.h
index 3c498f4..921d64f 100644
--- a/src/daemon/entry/connect/grpc/grpc_images_service.h
+++ b/src/daemon/entry/connect/grpc/grpc_images_service.h
@@ -33,7 +33,6 @@ using grpc::StatusCode;
using google::protobuf::Timestamp;
using namespace images;
-using namespace containerd::types;
// Implement of images service
class ImagesServiceImpl final : public ImagesService::Service {
@@ -55,11 +54,9 @@ public:
Status Inspect(ServerContext *context, const InspectImageRequest *request, InspectImageResponse *reply) override;
- Status Login(ServerContext *context, const LoginRequest *request,
- LoginResponse *reply) override;
+ Status Login(ServerContext *context, const LoginRequest *request, LoginResponse *reply) override;
- Status Logout(ServerContext *context, const LogoutRequest *request,
- LogoutResponse *reply) override;
+ Status Logout(ServerContext *context, const LogoutRequest *request, LogoutResponse *reply) override;
private:
template <class T1, class T2>
@@ -99,4 +96,3 @@ private:
};
#endif // DAEMON_ENTRY_CONNECT_GRPC_GRPC_IMAGES_SERVICE_H
-
diff --git a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/quota/project_quota.h b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/quota/project_quota.h
index d88ac30..2aae6bc 100644
--- a/src/daemon/modules/image/oci/storage/layer_store/graphdriver/quota/project_quota.h
+++ b/src/daemon/modules/image/oci/storage/layer_store/graphdriver/quota/project_quota.h
@@ -44,46 +44,34 @@
extern "C" {
#endif
-/*
- * Check whether we have to define FS_IOC_FS[GS]ETXATTR ourselves. These
- * are a copy of the definitions moved to linux/uapi/fs.h in the 4.5 kernel,
- * so this is purely for supporting builds against old kernel headers.
- */
#if !defined FS_IOC_FSGETXATTR
+// if did not define the fsxattr, define by ourself
struct fsxattr {
- __u32 fsx_xflags; /* xflags field value (get/set) */
- __u32 fsx_extsize; /* extsize field value (get/set)*/
- __u32 fsx_nextents; /* nextents field value (get) */
- __u32 fsx_projid; /* project identifier (get/set) */
- __u32 fsx_cowextsize; /* cow extsize field value (get/set) */
- unsigned char fsx_pad[8];
+ __u32 fsx_xflags, fsx_extsize, fsx_nextents, fsx_projid, fsx_cowextsize;
+ unsigned char fsx_pad[8];
};
#endif
#ifndef FS_IOC_FSGETXATTR
-/*
- * Flags for the fsx_xflags field
- */
-#define FS_XFLAG_REALTIME 0x00000001 /* data in realtime volume */
-#define FS_XFLAG_PREALLOC 0x00000002 /* preallocated file extents */
-#define FS_XFLAG_IMMUTABLE 0x00000008 /* file cannot be modified */
-#define FS_XFLAG_APPEND 0x00000010 /* all writes append */
-#define FS_XFLAG_SYNC 0x00000020 /* all writes synchronous */
-#define FS_XFLAG_NOATIME 0x00000040 /* do not update access time */
-#define FS_XFLAG_NODUMP 0x00000080 /* do not include in backups */
-#define FS_XFLAG_RTINHERIT 0x00000100 /* create with rt bit set */
-#define FS_XFLAG_PROJINHERIT 0x00000200 /* create with parents projid */
-#define FS_XFLAG_NOSYMLINKS 0x00000400 /* disallow symlink creation */
-#define FS_XFLAG_EXTSIZE 0x00000800 /* extent size allocator hint */
-#define FS_XFLAG_EXTSZINHERIT 0x00001000 /* inherit inode extent size */
-#define FS_XFLAG_NODEFRAG 0x00002000 /* do not defragment */
-#define FS_XFLAG_FILESTREAM 0x00004000 /* use filestream allocator */
-#define FS_XFLAG_DAX 0x00008000 /* use DAX for IO */
-#define FS_XFLAG_HASATTR 0x80000000 /* no DIFLAG for this */
-
-#define FS_IOC_FSGETXATTR _IOR ('X', 31, struct fsxattr)
-#define FS_IOC_FSSETXATTR _IOW ('X', 32, struct fsxattr)
-
+// if did not define the FSGETXATTR, define by ourself
+#define FS_XFLAG_REALTIME 0x00000001
+#define FS_XFLAG_PREALLOC 0x00000002
+#define FS_XFLAG_IMMUTABLE 0x00000008
+#define FS_XFLAG_APPEND 0x00000010
+#define FS_XFLAG_SYNC 0x00000020
+#define FS_XFLAG_NOATIME 0x00000040
+#define FS_XFLAG_NODUMP 0x00000080
+#define FS_XFLAG_RTINHERIT 0x00000100
+#define FS_XFLAG_PROJINHERIT 0x00000200
+#define FS_XFLAG_NOSYMLINKS 0x00000400
+#define FS_XFLAG_EXTSIZE 0x00000800
+#define FS_XFLAG_EXTSZINHERIT 0x00001000
+#define FS_XFLAG_NODEFRAG 0x00002000
+#define FS_XFLAG_FILESTREAM 0x00004000
+#define FS_XFLAG_DAX 0x00008000
+#define FS_XFLAG_HASATTR 0x80000000
+#define FS_IOC_FSGETXATTR _IOR('X', 31, struct fsxattr)
+#define FS_IOC_FSSETXATTR _IOW('X', 32, struct fsxattr)
#endif
struct pquota_control {
--
2.20.1

26
0014-unlink-etc-dir-when-link-exists.patch Normal file
View File

@ -0,0 +1,26 @@
From c4980da3192c4ee2d8a2251c520cec6ea857523a Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Thu, 29 Oct 2020 21:29:04 +0800
Subject: [PATCH 14/28] unlink etc dir when link exists
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
src/daemon/modules/service/service_container.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/daemon/modules/service/service_container.c b/src/daemon/modules/service/service_container.c
index 4e830d3..cc4e85e 100644
--- a/src/daemon/modules/service/service_container.c
+++ b/src/daemon/modules/service/service_container.c
@@ -130,6 +130,8 @@ static int create_mtab_link(const oci_runtime_spec *oci_spec)
goto out;
}
+ (void)unlink(dir);
+
if (!util_dir_exists(dir)) {
ret = util_mkdir_p(dir, ETC_FILE_MODE);
if (ret != 0) {
--
2.20.1

182
0015-support-variable-extension-cni-args.patch Normal file
View File

@ -0,0 +1,182 @@
From 9fba9e85a4e379a94fa0418969e56587bc72f55d Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Sat, 31 Oct 2020 15:26:08 +0800
Subject: [PATCH 15/28] support variable extension cni args
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
src/daemon/entry/cri/cni_network_plugin.cc | 103 ++++++++++++++-------
src/daemon/entry/cri/cri_helpers.cc | 1 +
src/daemon/entry/cri/cri_helpers.h | 1 +
3 files changed, 72 insertions(+), 33 deletions(-)
diff --git a/src/daemon/entry/cri/cni_network_plugin.cc b/src/daemon/entry/cri/cni_network_plugin.cc
index 4676a97..f15eba3 100644
--- a/src/daemon/entry/cri/cni_network_plugin.cc
+++ b/src/daemon/entry/cri/cni_network_plugin.cc
@@ -579,68 +579,105 @@ void CniNetworkPlugin::DeleteFromNetwork(CNINetwork *network, const std::string
free(serr);
}
+static bool CheckCNIArgValue(const std::string &val)
+{
+ if (val.find(';') != std::string::npos) {
+ return false;
+ }
+ if (std::count(val.begin(), val.end(), '=') != 1) {
+ return false;
+ }
+ return true;
+}
+
+static void GetExtensionCNIArgs(const std::map<std::string, std::string> &annotations,
+ std::map<std::string, std::string> &args)
+{
+ // get cni multinetwork extension
+ auto iter = annotations.find(CRIHelpers::Constants::CNI_MUTL_NET_EXTENSION_KEY);
+ if (iter != annotations.end()) {
+ if (!CheckCNIArgValue(iter->second)) {
+ WARN("Ignore: invalid multinetwork cni args: %s", iter->second.c_str());
+ } else {
+ args[CRIHelpers::Constants::CNI_MUTL_NET_EXTENSION_ARGS_KEY] = iter->second;
+ }
+ }
+
+ for (const auto &work : annotations) {
+ if (work.first.find(CRIHelpers::Constants::CNI_ARGS_EXTENSION_PREFIX_KEY) != 0) {
+ continue;
+ }
+ if (!CheckCNIArgValue(work.second)) {
+ WARN("Ignore: invalid extension cni args: %s", work.second.c_str());
+ continue;
+ }
+ auto strs = CXXUtils::Split(work.second, '=');
+ iter = annotations.find(work.first);
+ if (iter != annotations.end()) {
+ WARN("Ignore: Same key cni args: %s", work.first.c_str());
+ continue;
+ }
+ args[strs[0]] = strs[1];
+ }
+}
+
static void PrepareRuntimeConf(const std::string &podName, const std::string &podNs, const std::string &interfaceName,
const std::string &podSandboxID, const std::string &podNetnsPath,
const std::map<std::string, std::string> &annotations,
const std::map<std::string, std::string> &options, struct runtime_conf **cni_rc,
Errors &err)
{
- size_t defaultLen = 5;
+ size_t workLen = 5;
+ std::map<std::string, std::string> cniArgs;
+
if (cni_rc == nullptr) {
err.Errorf("Invalid arguments");
ERROR("Invalid arguments");
return;
}
- auto iter = options.find("UID");
- std::string podUID;
- if (iter != options.end()) {
- podUID = iter->second;
- }
- std::string cniExtentionVal;
- iter = annotations.find(CRIHelpers::Constants::CNI_MUTL_NET_EXTENSION_KEY);
- if (iter != annotations.end()) {
- cniExtentionVal = iter->second;
- defaultLen++;
- }
-
struct runtime_conf *rt = (struct runtime_conf *)util_common_calloc_s(sizeof(struct runtime_conf));
if (rt == nullptr) {
ERROR("Out of memory");
err.SetError("Out of memory");
return;
}
-
rt->container_id = util_strdup_s(podSandboxID.c_str());
rt->netns = util_strdup_s(podNetnsPath.c_str());
rt->ifname = util_strdup_s(interfaceName.c_str());
- rt->args = (char *(*)[2])util_common_calloc_s(sizeof(char *) * 2 * defaultLen);
+ auto iter = options.find("UID");
+ std::string podUID;
+ if (iter != options.end()) {
+ podUID = iter->second;
+ }
+
+ cniArgs["K8S_POD_UID"] = podUID;
+ cniArgs["IgnoreUnknown"] = "1";
+ cniArgs["K8S_POD_NAMESPACE"] = podNs;
+ cniArgs["K8S_POD_NAME"] = podName;
+ cniArgs["K8S_POD_INFRA_CONTAINER_ID"] = podSandboxID;
+
+ GetExtensionCNIArgs(annotations, cniArgs);
+ workLen = cniArgs.size();
+
+ rt->args = (char *(*)[2])util_common_calloc_s(sizeof(char *) * 2 * workLen);
if (rt->args == nullptr) {
ERROR("Out of memory");
err.SetError("Out of memory");
- goto free_out;
+ free_runtime_conf(rt);
+ return;
}
- rt->args_len = defaultLen;
- rt->args[0][0] = util_strdup_s("IgnoreUnknown");
- rt->args[0][1] = util_strdup_s("1");
- rt->args[1][0] = util_strdup_s("K8S_POD_NAMESPACE");
- rt->args[1][1] = util_strdup_s(podNs.c_str());
- rt->args[2][0] = util_strdup_s("K8S_POD_NAME");
- rt->args[2][1] = util_strdup_s(podName.c_str());
- rt->args[3][0] = util_strdup_s("K8S_POD_INFRA_CONTAINER_ID");
- rt->args[3][1] = util_strdup_s(podSandboxID.c_str());
- rt->args[4][0] = util_strdup_s("K8S_POD_UID");
- rt->args[4][1] = util_strdup_s(podUID.c_str());
- if (defaultLen > 5) {
- rt->args[5][0] = util_strdup_s(CRIHelpers::Constants::CNI_MUTL_NET_EXTENSION_ARGS_KEY.c_str());
- rt->args[5][1] = util_strdup_s(cniExtentionVal.c_str());
+ rt->args_len = workLen;
+
+ workLen = 0;
+ for (const auto &work : cniArgs) {
+ rt->args[workLen][0] = util_strdup_s(work.first.c_str());
+ rt->args[workLen][1] = util_strdup_s(work.second.c_str());
+ workLen++;
}
*cni_rc = rt;
- return;
-free_out:
- free_runtime_conf(rt);
}
void CniNetworkPlugin::BuildCNIRuntimeConf(const std::string &podName, const std::string &podNs,
diff --git a/src/daemon/entry/cri/cri_helpers.cc b/src/daemon/entry/cri/cri_helpers.cc
index 4fb9feb..ee633b7 100644
--- a/src/daemon/entry/cri/cri_helpers.cc
+++ b/src/daemon/entry/cri/cri_helpers.cc
@@ -53,6 +53,7 @@ const std::string Constants::NET_PLUGIN_EVENT_POD_CIDR_CHANGE { "pod-cidr-change
const std::string Constants::NET_PLUGIN_EVENT_POD_CIDR_CHANGE_DETAIL_CIDR { "pod-cidr" };
const std::string Constants::CNI_MUTL_NET_EXTENSION_KEY { "extension.network.kubernetes.io/cni" };
const std::string Constants::CNI_MUTL_NET_EXTENSION_ARGS_KEY { "CNI_MUTLINET_EXTENSION" };
+const std::string Constants::CNI_ARGS_EXTENSION_PREFIX_KEY { "extension.network.kubernetes.io/cniargs/" };
const char *InternalLabelKeys[] = { CRIHelpers::Constants::CONTAINER_TYPE_LABEL_KEY.c_str(),
CRIHelpers::Constants::CONTAINER_LOGPATH_LABEL_KEY.c_str(),
diff --git a/src/daemon/entry/cri/cri_helpers.h b/src/daemon/entry/cri/cri_helpers.h
index 824d1a6..3ea9ba6 100644
--- a/src/daemon/entry/cri/cri_helpers.h
+++ b/src/daemon/entry/cri/cri_helpers.h
@@ -58,6 +58,7 @@ public:
static const std::string NET_PLUGIN_EVENT_POD_CIDR_CHANGE_DETAIL_CIDR;
static const std::string CNI_MUTL_NET_EXTENSION_KEY;
static const std::string CNI_MUTL_NET_EXTENSION_ARGS_KEY;
+ static const std::string CNI_ARGS_EXTENSION_PREFIX_KEY;
};
auto GetDefaultSandboxImage(Errors &err) -> std::string;
--
2.20.1

112
0016-CI-for-support-variable-extension-cni-args.patch Normal file
View File

@ -0,0 +1,112 @@
From c62b1266dfbc4c28b5fd4dafea8438523826549d Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Sat, 31 Oct 2020 15:48:09 +0800
Subject: [PATCH 16/28] CI for support variable extension cni args
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
CI/test_cases/container_cases/cni_test.sh | 55 +++++++++++++++++++
.../container_cases/criconfigs/mock.json | 9 +++
.../criconfigs/sandbox-config.json | 5 ++
3 files changed, 69 insertions(+)
create mode 100644 CI/test_cases/container_cases/criconfigs/mock.json
diff --git a/CI/test_cases/container_cases/cni_test.sh b/CI/test_cases/container_cases/cni_test.sh
index 115e031..8173cb5 100644
--- a/CI/test_cases/container_cases/cni_test.sh
+++ b/CI/test_cases/container_cases/cni_test.sh
@@ -154,6 +154,61 @@ function new_cni_config()
do_test_help "10\.2\."
}
+function check_annotation()
+{
+ cp ${data_path}/mock.json /etc/cni/net.d/bridge.json
+ sync;sync;
+ tail $ISUALD_LOG
+ # wait cni updated
+ s=`date "+%s"`
+ for ((i=0;i<30;i++)); do
+ sleep 1
+ cur=`date "+%s"`
+ let "t=cur-s"
+ if [ $t -gt 6 ];then
+ break
+ fi
+ done
+ tail $ISUALD_LOG
+
+ sid=`crictl runp ${data_path}/sandbox-config.json`
+ if [ $? -ne 0 ]; then
+ msg_err "Failed to run sandbox"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ basepath=/tmp/cnilogs/
+ cat ${basepath}/${sid}.env | grep CNI_MUTLINET_EXTENSION
+ if [ $? -ne 0 ];then
+ msg_err "lost extension for mutl network args"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+ cat ${basepath}/${sid}.env | grep "extension=first"
+ if [ $? -ne 0 ];then
+ msg_err "lost extension for first cni args"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+ cat ${basepath}/${sid}.env | grep "extension=second"
+ if [ $? -ne 0 ];then
+ msg_err "lost extension for second cni args"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ crictl stopp $sid
+ if [ $? -ne 0 ];then
+ msg_err "stop sandbox failed"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ crictl rmp $sid
+ if [ $? -ne 0 ];then
+ msg_err "rm sandbox failed"
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ return $TC_RET_T
+}
+
ret=0
do_pre
diff --git a/CI/test_cases/container_cases/criconfigs/mock.json b/CI/test_cases/container_cases/criconfigs/mock.json
new file mode 100644
index 0000000..85bb91e
--- /dev/null
+++ b/CI/test_cases/container_cases/criconfigs/mock.json
@@ -0,0 +1,9 @@
+{
+ "cniVersion": "0.3.1",
+ "name": "mock",
+ "type": "isulad-cni",
+ "ipam": {
+ "type": "isulad-cni",
+ "subnet": "10.3.0.0/16"
+ }
+}
diff --git a/CI/test_cases/container_cases/criconfigs/sandbox-config.json b/CI/test_cases/container_cases/criconfigs/sandbox-config.json
index c63dc0e..e9151e8 100644
--- a/CI/test_cases/container_cases/criconfigs/sandbox-config.json
+++ b/CI/test_cases/container_cases/criconfigs/sandbox-config.json
@@ -7,5 +7,10 @@
"uid": "hdishd83djaidwnduwk28bcsb"
},
"linux": {
+ },
+ "annotations": {
+ "extension.network.kubernetes.io/cni": "[multinetwork]",
+ "extension.network.kubernetes.io/cniargs/first": "extension=first",
+ "extension.network.kubernetes.io/cniargs/second": "extension=second"
}
}
--
2.20.1

26
0017-add-unlink-dir-comments.patch Normal file
View File

@ -0,0 +1,26 @@
From 45b91002a3c8fa7c4e96db8154354be1cd4296bb Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Mon, 2 Nov 2020 08:58:04 +0800
Subject: [PATCH 17/28] add unlink dir comments
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
src/daemon/modules/service/service_container.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/daemon/modules/service/service_container.c b/src/daemon/modules/service/service_container.c
index cc4e85e..7a89618 100644
--- a/src/daemon/modules/service/service_container.c
+++ b/src/daemon/modules/service/service_container.c
@@ -129,7 +129,7 @@ static int create_mtab_link(const oci_runtime_spec *oci_spec)
ret = -1;
goto out;
}
-
+ // When dir is symbol link, unlink dir to assure creating dir success following
(void)unlink(dir);
if (!util_dir_exists(dir)) {
--
2.20.1

398
0018-iSulad-add-ISULAD_TMPDIR-env-variable.patch Normal file
View File

@ -0,0 +1,398 @@
From 4509f2a6d4b5ff7b0cb1df5177bf557950c67d15 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Tue, 3 Nov 2020 11:17:13 +0800
Subject: [PATCH 18/28] iSulad: add ISULAD_TMPDIR env variable
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
src/common/constants.h | 2 +-
src/contrib/config/iSulad.sysconfig | 4 ++
src/daemon/modules/image/oci/oci_image.c | 17 ++++--
src/daemon/modules/image/oci/oci_load.c | 58 +++++++++++++++----
.../modules/image/oci/registry/registry.c | 31 ++++++++--
src/daemon/modules/image/oci/registry_type.h | 1 -
.../modules/image/oci/storage/storage.c | 32 +++++++++-
.../modules/image/oci/storage/storage.h | 3 +-
src/daemon/modules/image/oci/utils_images.c | 35 +++++++++++
src/daemon/modules/image/oci/utils_images.h | 5 +-
10 files changed, 162 insertions(+), 26 deletions(-)
diff --git a/src/common/constants.h b/src/common/constants.h
index 52bb0a8..457e242 100644
--- a/src/common/constants.h
+++ b/src/common/constants.h
@@ -44,7 +44,7 @@ extern "C" {
#define LOG_DIRECTORY_MODE 0750
-#define TEMP_DIRECTORY_MODE 0750
+#define TEMP_DIRECTORY_MODE 0700
#define CONSOLE_FIFO_DIRECTORY_MODE 0770
diff --git a/src/contrib/config/iSulad.sysconfig b/src/contrib/config/iSulad.sysconfig
index 580d6de..5d222f9 100644
--- a/src/contrib/config/iSulad.sysconfig
+++ b/src/contrib/config/iSulad.sysconfig
@@ -20,3 +20,7 @@
#SYSMONITOR_OPTIONS='-H unix:///var/run/isulad.sock'
#SYSMONITOR_OPTIONS='-H tcp://127.0.0.1:2375'
#SYSMONITOR_OPTIONS='-H tcp://127.0.0.1:2375 --tlsverify --tlscacert=/root/.iSulad/ca.pem --tlscert=/root/.iSulad/cert.pem --tlskey=/root/.iSulad/key.pem'
+
+# Location used for temporary files, such as those created by isula load and pull operations.
+# Default is /var/tmp. Can be overridden by setting the following env variable.
+# ISULAD_TMPDIR=/var/tmp
\ No newline at end of file
diff --git a/src/daemon/modules/image/oci/oci_image.c b/src/daemon/modules/image/oci/oci_image.c
index f544019..f0ba19c 100644
--- a/src/daemon/modules/image/oci/oci_image.c
+++ b/src/daemon/modules/image/oci/oci_image.c
@@ -152,13 +152,22 @@ out:
static void cleanup_image_tmpdir()
{
- if (util_recursive_rmdir(IMAGE_TMP_PATH, 0)) {
- ERROR("failed to remove directory %s", IMAGE_TMP_PATH);
+ char *image_tmp_path = NULL;
+
+ image_tmp_path = get_image_tmp_path();
+ if (image_tmp_path == NULL) {
+ ERROR("failed to get image tmp path");
+ return;
+ }
+
+ if (util_recursive_rmdir(image_tmp_path, 0)) {
+ ERROR("failed to remove directory %s", image_tmp_path);
}
- if (util_mkdir_p(IMAGE_TMP_PATH, 0600)) {
- ERROR("failed to create directory %s", IMAGE_TMP_PATH);
+ if (util_mkdir_p(image_tmp_path, TEMP_DIRECTORY_MODE)) {
+ ERROR("failed to create directory %s", image_tmp_path);
}
+ free(image_tmp_path);
return;
}
diff --git a/src/daemon/modules/image/oci/oci_load.c b/src/daemon/modules/image/oci/oci_load.c
index 5511c04..073ad55 100644
--- a/src/daemon/modules/image/oci/oci_load.c
+++ b/src/daemon/modules/image/oci/oci_load.c
@@ -44,7 +44,6 @@
#define MANIFEST_BIG_DATA_KEY "manifest"
#define OCI_SCHEMA_VERSION 2
-#define OCI_LOAD_TMP_DIR OCI_LOAD_TMP_WORK_DIR "/oci-image-load-XXXXXX"
static image_manifest_items_element **load_manifest(const char *fname, size_t *length)
{
@@ -1008,6 +1007,47 @@ out:
return res;
}
+static char *oci_load_path_create()
+{
+ int ret = 0;
+ int nret = 0;
+ char *oci_load_work_dir = NULL;
+ char tmp_dir[PATH_MAX] = { 0 };
+
+ oci_load_work_dir = storage_oci_load_work_dir();
+ if (oci_load_work_dir == NULL) {
+ ERROR("Failed to get oci load work dir");
+ isulad_try_set_error_message("Failed to get oci load work dir");
+ ret = -1;
+ goto out;
+ }
+
+ if (util_mkdir_p(oci_load_work_dir, TEMP_DIRECTORY_MODE) != 0) {
+ ERROR("Unable to create oci image load tmp work dir:%s", oci_load_work_dir);
+ isulad_try_set_error_message("Unable to create oci image load tmp work dir:%s", oci_load_work_dir);
+ ret = -1;
+ goto out;
+ }
+
+ nret = snprintf(tmp_dir, PATH_MAX, "%s/oci-image-load-XXXXXX", oci_load_work_dir);
+ if (nret < 0 || (size_t)nret >= sizeof(tmp_dir)) {
+ ERROR("Path is too long");
+ ret = -1;
+ goto out;
+ }
+
+ if (mkdtemp(tmp_dir) == NULL) {
+ ERROR("make temporary dir failed: %s", strerror(errno));
+ isulad_try_set_error_message("make temporary dir failed: %s", strerror(errno));
+ ret = -1;
+ goto out;
+ }
+
+out:
+ free(oci_load_work_dir);
+ return ret == 0 ? util_strdup_s(tmp_dir) : NULL;
+}
+
int oci_do_load(const im_load_request *request)
{
int ret = 0;
@@ -1019,23 +1059,16 @@ int oci_do_load(const im_load_request *request)
size_t manifest_len = 0;
load_image_t *im = NULL;
char *digest = NULL;
- char dstdir[] = OCI_LOAD_TMP_DIR;
+ char *dstdir = NULL;
if (request == NULL || request->file == NULL) {
ERROR("Invalid input arguments, cannot load image");
return -1;
}
- if (util_mkdir_p(OCI_LOAD_TMP_WORK_DIR, TEMP_DIRECTORY_MODE) != 0) {
- ERROR("Unable to create oci image load tmp work dir:%s", OCI_LOAD_TMP_WORK_DIR);
- isulad_try_set_error_message("Unable to create oci image load tmp work dir:%s", OCI_LOAD_TMP_WORK_DIR);
- ret = -1;
- goto out;
- }
-
- if (mkdtemp(dstdir) == NULL) {
- ERROR("make temporary direcory failed: %s", strerror(errno));
- isulad_try_set_error_message("make temporary direcory failed: %s", strerror(errno));
+ dstdir = oci_load_path_create();
+ if (dstdir == NULL) {
+ ERROR("create temporary direcory failed");
ret = -1;
goto out;
}
@@ -1132,5 +1165,6 @@ out:
if (util_recursive_rmdir(dstdir, 0)) {
WARN("failed to remove directory %s", dstdir);
}
+ free(dstdir);
return ret;
}
diff --git a/src/daemon/modules/image/oci/registry/registry.c b/src/daemon/modules/image/oci/registry/registry.c
index 2d38ea5..e4cffdc 100644
--- a/src/daemon/modules/image/oci/registry/registry.c
+++ b/src/daemon/modules/image/oci/registry/registry.c
@@ -1682,8 +1682,9 @@ static int prepare_pull_desc(pull_descriptor *desc, registry_pull_options *optio
{
int ret = 0;
int sret = 0;
- char blobpath[] = REGISTRY_TMP_DIR;
+ char blobpath[PATH_MAX] = { 0 };
char scope[PATH_MAX] = { 0 };
+ char *image_tmp_path = NULL;
if (desc == NULL || options == NULL) {
ERROR("Invalid NULL param");
@@ -1717,6 +1718,20 @@ static int prepare_pull_desc(pull_descriptor *desc, registry_pull_options *optio
update_host(desc);
+ image_tmp_path = get_image_tmp_path();
+ if (image_tmp_path == NULL) {
+ ERROR("failed to get image tmp work dir");
+ ret = -1;
+ goto out;
+ }
+
+ sret = snprintf(blobpath, PATH_MAX, "%s/registry-XXXXXX", image_tmp_path);
+ if (sret < 0 || (size_t)sret > PATH_MAX) {
+ ERROR("image tmp work path too long");
+ ret = -1;
+ goto out;
+ }
+
if (mkdtemp(blobpath) == NULL) {
ERROR("make temporary direcory failed: %s", strerror(errno));
ret = -1;
@@ -1752,7 +1767,7 @@ static int prepare_pull_desc(pull_descriptor *desc, registry_pull_options *optio
}
out:
-
+ free(image_tmp_path);
return ret;
}
@@ -1844,10 +1859,18 @@ static void cached_layers_kvfree(void *key, void *value)
int registry_init(char *auths_dir, char *certs_dir)
{
int ret = 0;
+ char *image_tmp_path = NULL;
+
+ image_tmp_path = get_image_tmp_path();
+ if (image_tmp_path == NULL) {
+ ERROR("failed to get image tmp path");
+ return -1;
+ }
- if (util_mkdir_p(IMAGE_TMP_PATH, 0600)) {
- ERROR("failed to create directory %s", IMAGE_TMP_PATH);
+ if (util_mkdir_p(image_tmp_path, TEMP_DIRECTORY_MODE)) {
+ ERROR("failed to create directory %s", image_tmp_path);
}
+ free(image_tmp_path);
auths_set_dir(auths_dir);
certs_set_dir(certs_dir);
diff --git a/src/daemon/modules/image/oci/registry_type.h b/src/daemon/modules/image/oci/registry_type.h
index 9592587..e2047cb 100644
--- a/src/daemon/modules/image/oci/registry_type.h
+++ b/src/daemon/modules/image/oci/registry_type.h
@@ -25,7 +25,6 @@
// 8 is enough for challenge, usually only one challenge is provided.
#define CHALLENGE_MAX 8
-#define REGISTRY_TMP_DIR IMAGE_TMP_PATH "registry-XXXXXX"
#define MAX_LAYER_NUM 125
#define ROOTFS_TYPE "layers"
diff --git a/src/daemon/modules/image/oci/storage/storage.c b/src/daemon/modules/image/oci/storage/storage.c
index 6e83665..f15531b 100644
--- a/src/daemon/modules/image/oci/storage/storage.c
+++ b/src/daemon/modules/image/oci/storage/storage.c
@@ -1715,10 +1715,18 @@ out:
int storage_module_init(struct storage_module_init_options *opts)
{
int ret = 0;
+ char *oci_load_work_dir = NULL;
- ret = util_recursive_rmdir(OCI_LOAD_TMP_WORK_DIR, 0);
+ oci_load_work_dir = storage_oci_load_work_dir();
+ if (oci_load_work_dir == NULL) {
+ ERROR("Get oci load work dir failed");
+ ret = -1;
+ goto out;
+ }
+
+ ret = util_recursive_rmdir(oci_load_work_dir, 0);
if (ret != 0) {
- ERROR("failed to remove dir %s", OCI_LOAD_TMP_WORK_DIR);
+ ERROR("failed to remove dir %s", oci_load_work_dir);
goto out;
}
@@ -1775,5 +1783,25 @@ int storage_module_init(struct storage_module_init_options *opts)
}
out:
+ free(oci_load_work_dir);
return ret;
}
+
+
+char *storage_oci_load_work_dir()
+{
+ char *isulad_tmp = NULL;
+ char *oci_load_work_dir = NULL;
+
+ isulad_tmp = oci_get_isulad_tmpdir();
+ if (isulad_tmp == NULL) {
+ ERROR("Failed to get isulad tmp dir");
+ goto out;
+ }
+
+ oci_load_work_dir = util_path_join(isulad_tmp, "isulad-oci-load");
+
+out:
+ free(isulad_tmp);
+ return oci_load_work_dir;
+}
\ No newline at end of file
diff --git a/src/daemon/modules/image/oci/storage/storage.h b/src/daemon/modules/image/oci/storage/storage.h
index d3c4420..b030a3a 100644
--- a/src/daemon/modules/image/oci/storage/storage.h
+++ b/src/daemon/modules/image/oci/storage/storage.h
@@ -32,7 +32,6 @@
extern "C" {
#endif
-#define OCI_LOAD_TMP_WORK_DIR "/var/tmp/isulad-oci-load"
struct layer {
char *id;
@@ -180,6 +179,8 @@ int storage_rootfs_umount(const char *container_id, bool force);
container_inspect_graph_driver *storage_get_metadata_by_container_id(const char *id);
+char *storage_oci_load_work_dir();
+
#ifdef __cplusplus
}
#endif
diff --git a/src/daemon/modules/image/oci/utils_images.c b/src/daemon/modules/image/oci/utils_images.c
index 42831cc..4bd2b1d 100644
--- a/src/daemon/modules/image/oci/utils_images.c
+++ b/src/daemon/modules/image/oci/utils_images.c
@@ -39,6 +39,7 @@
// nanos of 2038-01-19T03:14:07, the max valid linux time
#define MAX_NANOS 2147483647000000000
+#define ISULAD_DEFAULT_TMP_DIR "/var/tmp"
char *get_last_part(char **parts)
{
@@ -486,3 +487,37 @@ bool oci_valid_time(char *time)
return true;
}
+
+
+char *oci_get_isulad_tmpdir()
+{
+ char *isula_tmp = NULL;
+
+ isula_tmp = getenv("ISULAD_TMPDIR");
+ if (util_valid_str(isula_tmp) && !util_dir_exists(isula_tmp)) {
+ if (util_mkdir_p(isula_tmp, TEMP_DIRECTORY_MODE) != 0) {
+ ERROR("make dir:%s failed", isula_tmp);
+ return NULL;
+ }
+ }
+
+ return util_valid_str(isula_tmp) ? util_strdup_s(isula_tmp) : util_strdup_s(ISULAD_DEFAULT_TMP_DIR);
+}
+
+char *get_image_tmp_path()
+{
+ char *isulad_tmp = NULL;
+ char *isula_image = NULL;
+
+ isulad_tmp = oci_get_isulad_tmpdir();
+ if (isulad_tmp == NULL) {
+ ERROR("Failed to get isulad tmp dir");
+ goto out;
+ }
+
+ isula_image = util_path_join(isulad_tmp, "isula-image");
+
+out:
+ free(isulad_tmp);
+ return isula_image;
+}
diff --git a/src/daemon/modules/image/oci/utils_images.h b/src/daemon/modules/image/oci/utils_images.h
index 4ab4afc..5dedd56 100644
--- a/src/daemon/modules/image/oci/utils_images.h
+++ b/src/daemon/modules/image/oci/utils_images.h
@@ -39,7 +39,6 @@ extern "C" {
#define REPO_PREFIX_TO_STRIP "library/"
#define MAX_ID_BUF_LEN 256
-#define IMAGE_TMP_PATH "/var/tmp/isula-image/"
char *oci_get_host(const char *name);
char *oci_host_from_mirror(const char *mirror);
@@ -55,6 +54,10 @@ int add_rootfs_and_history(const layer_blob *layers, size_t layers_len, const re
docker_image_config_v2 *config);
bool oci_valid_time(char *time);
+char *oci_get_isulad_tmpdir();
+
+char *get_image_tmp_path();
+
#ifdef __cplusplus
}
#endif
--
2.20.1

642210
0019-CI-remove-test-data-from-iSulad-repo.patch Normal file
View File

File diff suppressed because it is too large Load Diff

120
0020-add-ISULAD_TMPDIR-env-CI.patch Normal file
View File

@ -0,0 +1,120 @@
From a174c586d1a6fa5dd367dc0eee72b39ffb1c6d25 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Thu, 5 Nov 2020 16:27:01 +0800
Subject: [PATCH 20/28] add ISULAD_TMPDIR env CI
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
CI/test_cases/image_cases/isulad_tmpdir.sh | 100 +++++++++++++++++++++
1 file changed, 100 insertions(+)
create mode 100644 CI/test_cases/image_cases/isulad_tmpdir.sh
diff --git a/CI/test_cases/image_cases/isulad_tmpdir.sh b/CI/test_cases/image_cases/isulad_tmpdir.sh
new file mode 100644
index 0000000..22a6ad4
--- /dev/null
+++ b/CI/test_cases/image_cases/isulad_tmpdir.sh
@@ -0,0 +1,100 @@
+#!/bin/bash
+#
+# attributes: isulad basic container hook
+# concurrent: NA
+# spend time: 4
+
+#######################################################################
+##- @Copyright (C) Huawei Technologies., Ltd. 2020. All rights reserved.
+# - iSulad licensed under the Mulan PSL v2.
+# - You can use this software according to the terms and conditions of the Mulan PSL v2.
+# - You may obtain a copy of Mulan PSL v2 at:
+# - http://license.coscl.org.cn/MulanPSL2
+# - THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR
+# - IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR
+# - PURPOSE.
+# - See the Mulan PSL v2 for more details.
+##- @Description:CI
+##- @Author: gaohuatao
+##- @Create: 2020-11-05
+#######################################################################
+
+declare -r curr_path=$(dirname $(readlink -f "$0"))
+source ../helpers.sh
+busybox_image="${curr_path}/busybox.tar"
+image_name="busybox:latest"
+
+function restart_isulad()
+{
+ check_valgrind_log
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - stop isulad failed" && ((ret++))
+
+ start_isulad_with_valgrind
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - start isulad failed" && ((ret++))
+
+}
+
+function load_pull_test()
+{
+ isula load -i $busybox_image
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - load image failed: ${busybox_image} with" && ((ret++))
+
+ isula rmi $image_name
+
+ isula pull ${image_name}
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - failed to pull image: ${image_name}" && return ${FAILURE}
+}
+
+function test_isulad_tmpdir()
+{
+ local ret=0
+ local test="ISULAD_TMPDIR env test => (${FUNCNAME[@]})"
+
+ msg_info "${test} starting..."
+ isula rm -f `isula ps -qa`
+ isula rmi `isula images | awk '{if (NR>1){print $3}}'`
+
+ # The scene of ISULAD_TMPDIR dir is not exists
+ export ISULAD_TMPDIR="/var/isula/tmp"
+ restart_isulad
+ load_pull_test
+ test -d /var/isula/tmp/isula-image
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - isula-image not exist in ISULAD_TMPDIR" && ((ret++))
+
+ # The scene of ISULAD_TMPDIR dir is symbol link that it refers to dir exists
+ rm -rf /var/isula/tmp
+ mkdir -p /var/tmpdir
+ ln -sf /var/tmpdir /var/isula/tmpdir
+ unset ISULAD_TMPDIR
+ export ISULAD_TMPDIR="/var/isula/tmpdir"
+ restart_isulad
+ load_pull_test
+ test -d /var/isula/tmpdir/isula-image
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - isula-image not exist in ISULAD_TMPDIR" && ((ret++))
+
+ # rm dest dir of symbol link
+ rm -rf /var/tmpdir
+ check_valgrind_log
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - stop isulad failed" && ((ret++))
+
+ start_isulad_with_valgrind
+ [[ $? -eq 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - start isulad failed" && ((ret++))
+
+ # default no ISULAD_TMPDIR env
+ unset ISULAD_TMPDIR
+ start_isulad_with_valgrind
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - start isulad failed" && ((ret++))
+
+ load_pull_test
+ test -d /var/tmp/isula-image
+ [[ $? -ne 0 ]] && msg_err "${FUNCNAME[0]}:${LINENO} - isula-image not exist in /var/tmp" && ((ret++))
+
+ msg_info "${test} finished with return ${ret}..."
+ return ${ret}
+}
+
+declare -i ans=0
+
+test_isulad_tmpdir || ((ans++))
+
+show_result ${ans} "${curr_path}/${0}"
--
2.20.1

41
0021-iSulad-fix-memory-leak-in-inspect-grpc-service.patch Normal file
View File

@ -0,0 +1,41 @@
From e5df207d89c8467971fd0e0529703a782bca8dfa Mon Sep 17 00:00:00 2001
From: zhangxiaoyu <zhangxiaoyu58@huawei.com>
Date: Sat, 7 Nov 2020 09:57:27 +0800
Subject: [PATCH 21/28] iSulad: fix memory leak in inspect grpc service
Signed-off-by: zhangxiaoyu <zhangxiaoyu58@huawei.com>
---
.../entry/connect/grpc/grpc_containers_service.cc | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/src/daemon/entry/connect/grpc/grpc_containers_service.cc b/src/daemon/entry/connect/grpc/grpc_containers_service.cc
index 54417c6..60b1e44 100644
--- a/src/daemon/entry/connect/grpc/grpc_containers_service.cc
+++ b/src/daemon/entry/connect/grpc/grpc_containers_service.cc
@@ -781,6 +781,11 @@ Status ContainerServiceImpl::Inspect(ServerContext *context, const InspectContai
container_inspect_request *container_req = nullptr;
container_inspect_response *container_res = nullptr;
+ Status status = GrpcServerTlsAuth::auth(context, "container_inspect");
+ if (!status.ok()) {
+ return status;
+ }
+
cb = get_service_executor();
if (cb == nullptr || cb->container.inspect == nullptr) {
return Status(StatusCode::UNIMPLEMENTED, "Unimplemented callback");
@@ -793,11 +798,6 @@ Status ContainerServiceImpl::Inspect(ServerContext *context, const InspectContai
return Status::OK;
}
- Status status = GrpcServerTlsAuth::auth(context, "container_inspect");
- if (!status.ok()) {
- return status;
- }
-
ret = cb->container.inspect(container_req, &container_res);
tret = inspect_response_to_grpc(container_res, reply);
--
2.20.1

1424
0022-clean-code-remove-unused-code-in-connect.patch Normal file
View File

File diff suppressed because it is too large Load Diff

24
0023-add-newline-character-at-end-of-iSulad.sysconfig.patch Normal file
View File

@ -0,0 +1,24 @@
From 3260ec3e2b95e93618d5650b0b9874c65521e6b2 Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Mon, 9 Nov 2020 17:14:34 +0800
Subject: [PATCH 23/28] add newline character at end of iSulad.sysconfig
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
src/contrib/config/iSulad.sysconfig | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/contrib/config/iSulad.sysconfig b/src/contrib/config/iSulad.sysconfig
index 5d222f9..43ba7cb 100644
--- a/src/contrib/config/iSulad.sysconfig
+++ b/src/contrib/config/iSulad.sysconfig
@@ -23,4 +23,4 @@
# Location used for temporary files, such as those created by isula load and pull operations.
# Default is /var/tmp. Can be overridden by setting the following env variable.
-# ISULAD_TMPDIR=/var/tmp
\ No newline at end of file
+# ISULAD_TMPDIR=/var/tmp
--
2.20.1

46
0024-clean-code-remove-unused-in-code.patch Normal file
View File

@ -0,0 +1,46 @@
From d0a258a731d2cd4ad057c02a7d91762380b2c906 Mon Sep 17 00:00:00 2001
From: lifeng68 <lifeng68@huawei.com>
Date: Mon, 9 Nov 2020 17:32:16 +0800
Subject: [PATCH 24/28] clean code: remove unused ";" in code
Signed-off-by: lifeng68 <lifeng68@huawei.com>
---
src/daemon/entry/connect/grpc/grpc_containers_service.cc | 2 --
src/daemon/executor/container_cb/execution_network.c | 1 -
2 files changed, 3 deletions(-)
diff --git a/src/daemon/entry/connect/grpc/grpc_containers_service.cc b/src/daemon/entry/connect/grpc/grpc_containers_service.cc
index 4e8f55d..123fee8 100644
--- a/src/daemon/entry/connect/grpc/grpc_containers_service.cc
+++ b/src/daemon/entry/connect/grpc/grpc_containers_service.cc
@@ -340,7 +340,6 @@ Status ContainerServiceImpl::RemoteStart(ServerContext *context,
if (sem_init(&sem, 0, 0) != 0) {
return grpc::Status(grpc::StatusCode::UNKNOWN, "Semaphore initialization failed");
- ;
}
int read_pipe_fd[2];
@@ -867,7 +866,6 @@ Status ContainerServiceImpl::AttachInit(ServerContext *context, service_executor
if (sem_init(sem_stderr, 0, 0) != 0) {
free_container_attach_request(*req);
return grpc::Status(grpc::StatusCode::UNKNOWN, "Semaphore initialization failed");
- ;
}
if ((pipe2(pipefd, O_NONBLOCK | O_CLOEXEC)) < 0) {
diff --git a/src/daemon/executor/container_cb/execution_network.c b/src/daemon/executor/container_cb/execution_network.c
index f15707e..6b6c626 100644
--- a/src/daemon/executor/container_cb/execution_network.c
+++ b/src/daemon/executor/container_cb/execution_network.c
@@ -965,7 +965,6 @@ out:
static int write_default_resolve(const char *file_path)
{
const char *default_ipv4_dns = "\nnameserver 8.8.8.8\nnameserver 8.8.4.4\n";
- ;
return util_write_file(file_path, default_ipv4_dns, strlen(default_ipv4_dns), NETWORK_MOUNT_FILE_MODE);
}
--
2.20.1

66
0025-utils-add-fdatasync-when-do-atomic-write-file.patch Normal file
View File

@ -0,0 +1,66 @@
From 8888ca136de8f864df919f6823b357238a328488 Mon Sep 17 00:00:00 2001
From: lifeng68 <lifeng68@huawei.com>
Date: Tue, 10 Nov 2020 10:08:56 +0800
Subject: [PATCH 25/28] utils: add fdatasync when do atomic write file
Signed-off-by: lifeng68 <lifeng68@huawei.com>
---
src/utils/cutils/utils_file.c | 35 ++++++++++++++++++++++++++++++++++-
1 file changed, 34 insertions(+), 1 deletion(-)
diff --git a/src/utils/cutils/utils_file.c b/src/utils/cutils/utils_file.c
index 9f7f5fe..744b425 100644
--- a/src/utils/cutils/utils_file.c
+++ b/src/utils/cutils/utils_file.c
@@ -1403,6 +1403,39 @@ out:
return result;
}
+static int do_atomic_write_file(const char *fname, const char *content, size_t content_len, mode_t mode)
+{
+ int ret = 0;
+ int dst_fd = -1;
+ ssize_t len = 0;
+
+ dst_fd = util_open(fname, O_WRONLY | O_CREAT | O_TRUNC, mode);
+ if (dst_fd < 0) {
+ ERROR("Creat file: %s, failed: %s", fname, strerror(errno));
+ ret = -1;
+ goto free_out;
+ }
+
+ len = util_write_nointr(dst_fd, content, content_len);
+ if (len < 0 || ((size_t)len) != content_len) {
+ ret = -1;
+ ERROR("Write file failed: %s", strerror(errno));
+ goto free_out;
+ }
+
+ if (fdatasync(dst_fd) != 0) {
+ ret = -1;
+ SYSERROR("Failed to sync data of file:%s", fname);
+ goto free_out;
+ }
+
+free_out:
+ if (dst_fd >= 0) {
+ close(dst_fd);
+ }
+ return ret;
+}
+
int util_atomic_write_file(const char *fname, const char *content, size_t content_len, mode_t mode)
{
int ret = 0;
@@ -1427,7 +1460,7 @@ int util_atomic_write_file(const char *fname, const char *content, size_t conten
goto free_out;
}
- ret = util_write_file(tmp_file, content, content_len, mode);
+ ret = do_atomic_write_file(tmp_file, content, content_len, mode);
if (ret != 0) {
ERROR("Failed to write content to tmp file for %s", tmp_file);
ret = -1;
--
2.20.1

975
0026-network-support-mutlnetworks.patch Normal file
View File

@ -0,0 +1,975 @@
From aa35a1a1621d911cf9b76eba232814775ea6b4d9 Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Mon, 2 Nov 2020 11:15:34 +0800
Subject: [PATCH 26/28] network: support mutlnetworks
1. support mutlnetworks
2. support dualstack for default network
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
src/api/services/cri/api.proto | 6 +
src/daemon/entry/cri/cni_network_plugin.cc | 221 +++++++++++++++++++--
src/daemon/entry/cri/cni_network_plugin.h | 14 +-
src/daemon/entry/cri/cri_helpers.cc | 4 +-
src/daemon/entry/cri/cri_helpers.h | 2 +-
src/daemon/entry/cri/cri_runtime_service.h | 13 +-
src/daemon/entry/cri/cri_sandbox.cc | 202 +++++++++----------
src/daemon/entry/cri/network_plugin.cc | 120 +++++++----
src/daemon/entry/cri/network_plugin.h | 10 +-
9 files changed, 414 insertions(+), 178 deletions(-)
diff --git a/src/api/services/cri/api.proto b/src/api/services/cri/api.proto
index 8aba0d3..67e5527 100644
--- a/src/api/services/cri/api.proto
+++ b/src/api/services/cri/api.proto
@@ -399,10 +399,16 @@ message PodSandboxStatusRequest {
bool verbose = 2;
}
+// PodIP represents an ip of a Pod
+message PodIP {
+ // an ip is a string representation of an IPV4 or an IPV6
+ string ip = 1;
+}
// PodSandboxNetworkStatus is the status of the network for a PodSandbox.
message PodSandboxNetworkStatus {
// IP address of the PodSandbox.
string ip = 1;
+ repeated PodIP additional_ips = 2;
}
// Namespace contains paths to the namespaces.
diff --git a/src/daemon/entry/cri/cni_network_plugin.cc b/src/daemon/entry/cri/cni_network_plugin.cc
index f15eba3..9cb5722 100644
--- a/src/daemon/entry/cri/cni_network_plugin.cc
+++ b/src/daemon/entry/cri/cni_network_plugin.cc
@@ -80,7 +80,7 @@ auto CNINetwork::GetPaths(Errors &err) -> char **
{
char **paths = CRIHelpers::StringVectorToCharArray(m_path);
if (paths == nullptr) {
- err.SetError("Get char ** path failed");
+ err.SetError("Get cni network paths failed");
}
return paths;
}
@@ -124,6 +124,26 @@ void CniNetworkPlugin::SetDefaultNetwork(std::unique_ptr<CNINetwork> network, st
}
}
+void CniNetworkPlugin::UpdateMutlNetworks(std::vector<std::unique_ptr<CNINetwork>> &multNets,
+ std::vector<std::string> &binDirs, Errors &err)
+{
+ if (multNets.size() == 0) {
+ return;
+ }
+ WLockNetworkMap(err);
+ if (err.NotEmpty()) {
+ return;
+ }
+
+ m_mutlNetworks.clear();
+ for (auto iter = multNets.begin(); iter != multNets.end(); ++iter) {
+ (*iter)->SetPaths(binDirs);
+ m_mutlNetworks[(*iter)->GetName()] = std::move(*iter);
+ }
+
+ UnlockNetworkMap(err);
+}
+
CniNetworkPlugin::CniNetworkPlugin(std::vector<std::string> &binDirs, const std::string &confDir,
const std::string &podCidr)
: m_confDir(confDir)
@@ -139,6 +159,7 @@ CniNetworkPlugin::~CniNetworkPlugin()
if (m_syncThread.joinable()) {
m_syncThread.join();
}
+ m_mutlNetworks.clear();
}
void CniNetworkPlugin::PlatformInit(Errors &error)
@@ -259,7 +280,9 @@ out:
void CniNetworkPlugin::GetDefaultCNINetwork(const std::string &confDir, std::vector<std::string> &binDirs, Errors &err)
{
std::vector<std::string> files;
- bool found = false;
+ std::vector<std::unique_ptr<CNINetwork>> mutlNets;
+ char *default_net_name = nullptr;
+ std::string message = { "" };
if (GetCNIConfFiles(confDir, files, err) != 0) {
goto free_out;
@@ -279,17 +302,33 @@ void CniNetworkPlugin::GetDefaultCNINetwork(const std::string &confDir, std::vec
n_list = nullptr;
continue;
}
+ DEBUG("parse cni network: %s", n_list->name);
- SetDefaultNetwork(std::unique_ptr<CNINetwork>(new (std::nothrow) CNINetwork(n_list->name, n_list)), binDirs,
- err);
- found = true;
- break;
+ if (default_net_name == nullptr) {
+ SetDefaultNetwork(std::unique_ptr<CNINetwork>(new (std::nothrow) CNINetwork(n_list->name, n_list)), binDirs, err);
+ default_net_name = util_strdup_s(n_list->name);
+ message += default_net_name;
+ continue;
+ }
+ if (strcmp(default_net_name, n_list->name) == 0) {
+ WARN("Use same name of default net: %s", default_net_name);
+ continue;
+ }
+ mutlNets.push_back(std::unique_ptr<CNINetwork>(new (std::nothrow) CNINetwork(n_list->name, n_list)));
+ message += ", " + std::string(n_list->name);
}
- if (!found) {
+ if (default_net_name == nullptr) {
err.Errorf("No valid networks found in %s", confDir.c_str());
+ goto free_out;
+ }
+ UpdateMutlNetworks(mutlNets, binDirs, err);
+ if (err.NotEmpty()) {
+ goto free_out;
}
+ INFO("Loaded cni plugins successfully, [ %s ]", message.c_str());
free_out:
+ free(default_net_name);
return;
}
@@ -350,6 +389,71 @@ void CniNetworkPlugin::Status(Errors &err)
CheckInitialized(err);
}
+
+bool CniNetworkPlugin::SetupMultNetworks(const std::string &ns, const std::string &defaultInterface,
+ const std::string &name,
+ const std::string &netnsPath, const std::string &podSandboxID,
+ const std::map<std::string, std::string> &annotations,
+ const std::map<std::string, std::string> &options, Errors &err)
+{
+ bool ret = false;
+ int defaultIdx = -1;
+ size_t len = 0;
+ cri_pod_network_element **networks = CRIHelpers::GetNetworkPlaneFromPodAnno(annotations, &len, err);
+ if (err.NotEmpty()) {
+ ERROR("Couldn't get network plane from pod annotations: %s", err.GetCMessage());
+ err.Errorf("Couldn't get network plane from pod annotations: %s", err.GetCMessage());
+ goto cleanup;
+ }
+
+ for (size_t i = 0; i < len; i++) {
+ if (networks[i] == nullptr || networks[i]->name == nullptr || networks[i]->interface == nullptr) {
+ continue;
+ }
+ struct result *preResult = nullptr;
+ auto netIter = m_mutlNetworks.find(networks[i]->name);
+ if (netIter == m_mutlNetworks.end()) {
+ err.Errorf("Cannot found user defined net: %s", networks[i]->name);
+ break;
+ }
+ if (defaultInterface == networks[i]->interface) {
+ defaultIdx = i;
+ continue;
+ }
+ AddToNetwork((netIter->second).get(), name, ns, networks[i]->interface, podSandboxID, netnsPath, annotations, options,
+ &preResult, err);
+ free_result(preResult);
+ if (err.NotEmpty()) {
+ ERROR("Do setup user defined net: %s, failed: %s", networks[i]->name, err.GetCMessage());
+ break;
+ }
+ INFO("Setup user defained net: %s success", networks[i]->name);
+ }
+
+ // mask default network pod, if user defined net use same interface
+ if (defaultIdx >= 0) {
+ auto netIter = m_mutlNetworks.find(networks[defaultIdx]->name);
+ if (netIter == m_mutlNetworks.end()) {
+ err.Errorf("Cannot found user defined net: %s", networks[defaultIdx]->name);
+ goto cleanup;
+ }
+
+ struct result *preResult = nullptr;
+ AddToNetwork((netIter->second).get(), name, ns, networks[defaultIdx]->interface, podSandboxID, netnsPath, annotations,
+ options, &preResult, err);
+ free_result(preResult);
+ if (err.NotEmpty()) {
+ ERROR("Do setup user defined net: %s, failed: %s", networks[defaultIdx]->name, err.GetCMessage());
+ goto cleanup;
+ }
+ INFO("Setup default net: %s success", networks[defaultIdx]->name);
+ ret = true;
+ }
+cleanup:
+ free_cri_pod_network(networks, len);
+ return ret;
+}
+
void CniNetworkPlugin::SetUpPod(const std::string &ns, const std::string &name, const std::string &interfaceName,
const std::string &id, const std::map<std::string, std::string> &annotations,
const std::map<std::string, std::string> &options, Errors &err)
@@ -381,17 +485,83 @@ void CniNetworkPlugin::SetUpPod(const std::string &ns, const std::string &name,
return;
}
- AddToNetwork(m_defaultNetwork.get(), name, ns, interfaceName, id, netnsPath, annotations, options, &preResult, err);
+ bool setedDefaultNet = SetupMultNetworks(ns, interfaceName, name, netnsPath, id, annotations, options, err);
+ if (err.NotEmpty()) {
+ goto unlock;
+ }
+
+ if (setedDefaultNet) {
+ goto unlock;
+ }
+ AddToNetwork(m_defaultNetwork.get(), name, ns, interfaceName, id, netnsPath, annotations, options, &preResult, err);
free_result(preResult);
- preResult = nullptr;
if (err.NotEmpty()) {
ERROR("Error while adding to cni network: %s", err.GetCMessage());
}
+unlock:
UnlockNetworkMap(err);
}
+bool CniNetworkPlugin::TearDownMultNetworks(const std::string &ns, const std::string &defaultInterface,
+ const std::string &name,
+ const std::string &netnsPath, const std::string &podSandboxID, const std::map<std::string, std::string> &annotations,
+ Errors &err)
+{
+ bool ret = false;
+ int defaultIdx = -1;
+ size_t len = 0;
+ cri_pod_network_element **networks = CRIHelpers::GetNetworkPlaneFromPodAnno(annotations, &len, err);
+ if (err.NotEmpty()) {
+ ERROR("Couldn't get network plane from pod annotations: %s", err.GetCMessage());
+ err.Errorf("Couldn't get network plane from pod annotations: %s", err.GetCMessage());
+ goto cleanup;
+ }
+
+ for (size_t i = 0; i < len; i++) {
+ if (networks[i] == nullptr || networks[i]->name == nullptr || networks[i]->interface == nullptr) {
+ continue;
+ }
+ auto netIter = m_mutlNetworks.find(networks[i]->name);
+ if (netIter == m_mutlNetworks.end()) {
+ WARN("Cannot found user defined net: %s", networks[i]->name);
+ continue;
+ }
+ if (defaultInterface == networks[i]->interface) {
+ defaultIdx = i;
+ continue;
+ }
+ DeleteFromNetwork((netIter->second).get(), name, ns, networks[i]->interface, podSandboxID, netnsPath, annotations, err);
+ if (err.NotEmpty()) {
+ ERROR("Do teardown user defined net: %s, failed: %s", networks[i]->name, err.GetCMessage());
+ break;
+ }
+ INFO("Teardown user defained net: %s success", networks[i]->name);
+ }
+
+ // mask default network pod, if user defined net use same interface
+ if (defaultIdx >= 0) {
+ auto netIter = m_mutlNetworks.find(networks[defaultIdx]->name);
+ if (netIter == m_mutlNetworks.end()) {
+ err.Errorf("Cannot found user defined net: %s", networks[defaultIdx]->name);
+ goto cleanup;
+ }
+
+ DeleteFromNetwork((netIter->second).get(), name, ns, networks[defaultIdx]->interface, podSandboxID, netnsPath,
+ annotations, err);
+ if (err.NotEmpty()) {
+ ERROR("Do teardown user defined net: %s, failed: %s", networks[defaultIdx]->name, err.GetCMessage());
+ goto cleanup;
+ }
+ INFO("Teardown default net: %s success", networks[defaultIdx]->name);
+ ret = true;
+ }
+cleanup:
+ free_cri_pod_network(networks, len);
+ return ret;
+}
+
void CniNetworkPlugin::TearDownPod(const std::string &ns, const std::string &name, const std::string &interfaceName,
const std::string &id, const std::map<std::string, std::string> &annotations,
Errors &err)
@@ -400,6 +570,7 @@ void CniNetworkPlugin::TearDownPod(const std::string &ns, const std::string &nam
if (err.NotEmpty()) {
return;
}
+ Errors tmpErr;
std::string netnsPath = m_criImpl->GetNetNS(id, err);
if (err.NotEmpty()) {
@@ -413,8 +584,21 @@ void CniNetworkPlugin::TearDownPod(const std::string &ns, const std::string &nam
return;
}
- DeleteFromNetwork(m_defaultNetwork.get(), name, ns, interfaceName, id, netnsPath, annotations, err);
+ bool defaultNetDone = TearDownMultNetworks(ns, interfaceName, name, netnsPath, id, annotations, err);
+ if (defaultNetDone) {
+ goto unlock;
+ }
+ if (err.NotEmpty()) {
+ WARN("Teardown user defined networks failed: %s", err.GetCMessage());
+ }
+
+ DeleteFromNetwork(m_defaultNetwork.get(), name, ns, interfaceName, id, netnsPath, annotations, tmpErr);
+ if (tmpErr.NotEmpty()) {
+ WARN("Teardown default network failed: %s", tmpErr.GetCMessage());
+ err.AppendError(tmpErr.GetMessage());
+ }
+unlock:
UnlockNetworkMap(err);
}
@@ -464,7 +648,7 @@ void CniNetworkPlugin::GetPodNetworkStatus(const std::string & /*ns*/, const std
PodNetworkStatus &status, Errors &err)
{
std::string netnsPath;
- std::string ip;
+ std::vector<std::string> ips;
Errors tmpErr;
if (podSandboxID.empty()) {
@@ -482,15 +666,15 @@ void CniNetworkPlugin::GetPodNetworkStatus(const std::string & /*ns*/, const std
podSandboxID.c_str());
goto out;
}
- ip = GetPodIP(m_nsenterPath, netnsPath, interfaceName, err);
+ GetPodIP(m_nsenterPath, netnsPath, interfaceName, ips, err);
if (err.NotEmpty()) {
ERROR("GetPodIP failed: %s", err.GetCMessage());
goto out;
}
- status.SetIP(ip);
+ status.SetIPs(ips);
out:
- INFO("get_pod_network_status: %s", podSandboxID.c_str());
+ INFO("Get pod: %s network status success", podSandboxID.c_str());
}
void CniNetworkPlugin::AddToNetwork(CNINetwork *snetwork, const std::string &podName, const std::string &podNamespace,
@@ -761,7 +945,8 @@ void CniNetworkPlugin::RLockNetworkMap(Errors &error)
{
int ret = pthread_rwlock_rdlock(&m_netsLock);
if (ret != 0) {
- error.Errorf("Get read lock failed: %s", strerror(ret));
+ error.Errorf("Failed to get read lock");
+ ERROR("Get read lock failed: %s", strerror(ret));
}
}
@@ -769,7 +954,8 @@ void CniNetworkPlugin::WLockNetworkMap(Errors &error)
{
int ret = pthread_rwlock_wrlock(&m_netsLock);
if (ret != 0) {
- error.Errorf("Get write lock failed: %s", strerror(ret));
+ error.Errorf("Failed to get write lock");
+ ERROR("Get write lock failed: %s", strerror(ret));
}
}
@@ -777,7 +963,8 @@ void CniNetworkPlugin::UnlockNetworkMap(Errors &error)
{
int ret = pthread_rwlock_unlock(&m_netsLock);
if (ret != 0) {
- error.Errorf("Unlock failed: %s", strerror(ret));
+ error.Errorf("Failed to unlock");
+ ERROR("Unlock failed: %s", strerror(ret));
}
}
diff --git a/src/daemon/entry/cri/cni_network_plugin.h b/src/daemon/entry/cri/cni_network_plugin.h
index 02c95fb..c59c200 100644
--- a/src/daemon/entry/cri/cni_network_plugin.h
+++ b/src/daemon/entry/cri/cni_network_plugin.h
@@ -146,6 +146,9 @@ private:
void RLockNetworkMap(Errors &error);
void WLockNetworkMap(Errors &error);
void UnlockNetworkMap(Errors &error);
+
+ void UpdateMutlNetworks(std::vector<std::unique_ptr<CNINetwork>> &multNets, std::vector<std::string> &binDirs,
+ Errors &err);
void SetDefaultNetwork(std::unique_ptr<CNINetwork> network, std::vector<std::string> &binDirs, Errors &err);
void SetPodCidr(const std::string &podCidr);
static auto GetCNIConfFiles(const std::string &pluginDir, std::vector<std::string> &vect_files, Errors &err) -> int;
@@ -155,10 +158,19 @@ private:
void ResetCNINetwork(std::map<std::string, std::unique_ptr<CNINetwork>> &newNets, Errors &err);
void UpdateDefaultNetwork();
+ bool SetupMultNetworks(const std::string &ns, const std::string &defaultInterface, const std::string &name,
+ const std::string &netnsPath, const std::string &podSandboxID, const std::map<std::string, std::string> &annotations,
+ const std::map<std::string, std::string> &options, Errors &err);
+
+ bool TearDownMultNetworks(const std::string &ns, const std::string &defaultInterface, const std::string &name,
+ const std::string &netnsPath, const std::string &podSandboxID, const std::map<std::string, std::string> &annotations,
+ Errors &err);
+
NoopNetworkPlugin m_noop;
std::unique_ptr<CNINetwork> m_loNetwork { nullptr };
-
std::unique_ptr<CNINetwork> m_defaultNetwork { nullptr };
+ std::map<std::string, std::unique_ptr<CNINetwork>> m_mutlNetworks;
+
CRIRuntimeServiceImpl *m_criImpl { nullptr };
std::string m_nsenterPath;
std::string m_confDir;
diff --git a/src/daemon/entry/cri/cri_helpers.cc b/src/daemon/entry/cri/cri_helpers.cc
index ee633b7..34d32e5 100644
--- a/src/daemon/entry/cri/cri_helpers.cc
+++ b/src/daemon/entry/cri/cri_helpers.cc
@@ -389,7 +389,7 @@ auto sha256(const char *val) -> std::string
return outputBuffer;
}
-auto GetNetworkPlaneFromPodAnno(const google::protobuf::Map<std::string, std::string> &annotations, size_t *len,
+auto GetNetworkPlaneFromPodAnno(const std::map<std::string, std::string> &annotations, size_t *len,
Errors &error) -> cri_pod_network_element **
{
auto iter = annotations.find(CRIHelpers::Constants::POD_NETWORK_ANNOTATION_KEY);
@@ -399,7 +399,7 @@ auto GetNetworkPlaneFromPodAnno(const google::protobuf::Map<std::string, std::st
parser_error err = nullptr;
result = cri_pod_network_parse_data(iter->second.c_str(), nullptr, &err, len);
if (result == nullptr) {
- error.Errorf("parse pod network json failed: %s", err);
+ error.Errorf("parse pod network json: %s failed: %s", iter->second.c_str(), err);
}
free(err);
}
diff --git a/src/daemon/entry/cri/cri_helpers.h b/src/daemon/entry/cri/cri_helpers.h
index 3ea9ba6..b9fb153 100644
--- a/src/daemon/entry/cri/cri_helpers.h
+++ b/src/daemon/entry/cri/cri_helpers.h
@@ -94,7 +94,7 @@ auto IsImageNotFoundError(const std::string &err) -> bool;
auto sha256(const char *val) -> std::string;
-auto GetNetworkPlaneFromPodAnno(const google::protobuf::Map<std::string, std::string> &annotations,
+auto GetNetworkPlaneFromPodAnno(const std::map<std::string, std::string> &annotations,
size_t *len, Errors &error) -> cri_pod_network_element **;
auto CheckpointToSandbox(const std::string &id,
diff --git a/src/daemon/entry/cri/cri_runtime_service.h b/src/daemon/entry/cri/cri_runtime_service.h
index 66837e9..1a0f601 100644
--- a/src/daemon/entry/cri/cri_runtime_service.h
+++ b/src/daemon/entry/cri/cri_runtime_service.h
@@ -210,9 +210,12 @@ private:
void ConstructPodSandboxCheckpoint(const runtime::v1alpha2::PodSandboxConfig &config,
cri::PodSandboxCheckpoint &checkpoint);
- auto GetIP(const std::string &podSandboxID, container_inspect *inspect, const std::string &networkInterface,
- Errors &error) -> std::string;
- auto GetIPFromPlugin(container_inspect *inspect, const std::string &networkInterface, Errors &error) -> std::string;
+ void GetIPs(const std::string &podSandboxID, container_inspect *inspect, const std::string &networkInterface,
+ std::vector<std::string> &ips, Errors &error);
+ void GetFormatIPsForMultNet(container_inspect *inspect, const std::string &defaultInterface,
+ const runtime::v1alpha2::PodSandboxMetadata &metadata, std::vector<std::string> &result, Errors &error);
+ auto GetIPsFromPlugin(container_inspect *inspect, const std::string &networkInterface,
+ Errors &error) -> std::vector<std::string>;
auto GetNetworkReady(const std::string &podSandboxID, Errors &error) -> bool;
void SetNetworkReady(const std::string &podSandboxID, bool ready, Errors &error);
void ClearNetworkReady(const std::string &podSandboxID);
@@ -247,9 +250,6 @@ private:
void SetupSandboxNetwork(const runtime::v1alpha2::PodSandboxConfig &config, const std::string &response_id,
const std::string &jsonCheckpoint, Errors &error);
- void SetupUserDefinedNetworkPlane(const runtime::v1alpha2::PodSandboxConfig &config, const std::string &response_id,
- container_inspect *inspect_data, std::map<std::string, std::string> &stdAnnos,
- std::map<std::string, std::string> &options, Errors &error);
void StartSandboxContainer(const std::string &response_id, Errors &error);
auto CreateSandboxContainer(const runtime::v1alpha2::PodSandboxConfig &config, const std::string &image,
std::string &jsonCheckpoint, const std::string &runtimeHandler,
@@ -273,6 +273,7 @@ private:
auto ClearCniNetwork(const std::string &realSandboxID, bool hostNetwork, const std::string &ns,
const std::string &name, std::vector<std::string> &errlist,
std::map<std::string, std::string> &stdAnnos, Errors &error) -> int;
+
auto RemoveAllContainersInSandbox(const std::string &realSandboxID, std::vector<std::string> &errors) -> int;
auto DoRemovePodSandbox(const std::string &realSandboxID, std::vector<std::string> &errors) -> int;
static void MergeSecurityContextToHostConfig(const runtime::v1alpha2::PodSandboxConfig &c, host_config *hc,
diff --git a/src/daemon/entry/cri/cri_sandbox.cc b/src/daemon/entry/cri/cri_sandbox.cc
index 6db9616..b44c86c 100644
--- a/src/daemon/entry/cri/cri_sandbox.cc
+++ b/src/daemon/entry/cri/cri_sandbox.cc
@@ -458,44 +458,6 @@ void CRIRuntimeServiceImpl::StartSandboxContainer(const std::string &response_id
free_container_start_response(start_response);
}
-void CRIRuntimeServiceImpl::SetupUserDefinedNetworkPlane(const runtime::v1alpha2::PodSandboxConfig &config,
- const std::string &response_id,
- container_inspect *inspect_data,
- std::map<std::string, std::string> &stdAnnos,
- std::map<std::string, std::string> &options, Errors &error)
-{
- google::protobuf::Map<std::string, std::string> annotations;
- CRIHelpers::ExtractAnnotations(inspect_data->config->annotations, annotations);
-
- size_t len = 0;
- cri_pod_network_element **networks = CRIHelpers::GetNetworkPlaneFromPodAnno(annotations, &len, error);
- if (error.NotEmpty()) {
- ERROR("Couldn't get network plane from pod annotations: %s", error.GetCMessage());
- error.Errorf("Couldn't get network plane from pod annotations: %s", error.GetCMessage());
- goto cleanup;
- }
- for (size_t i = 0; i < len; i++) {
- if ((networks[i] != nullptr) && (networks[i]->name != nullptr) && (networks[i]->interface != nullptr) &&
- strcmp(networks[i]->name, Network::DEFAULT_NETWORK_PLANE_NAME.c_str()) != 0) {
- INFO("SetupPod net: %s", networks[i]->name);
- m_pluginManager->SetUpPod(config.metadata().namespace_(), config.metadata().name(), networks[i]->interface,
- response_id, stdAnnos, options, error);
- if (error.Empty()) {
- continue;
- }
- Errors tmpErr;
- StopContainerHelper(response_id, tmpErr);
- if (tmpErr.NotEmpty()) {
- WARN("Failed to stop sandbox container %s for pod %s: %s", response_id.c_str(), networks[i]->name,
- tmpErr.GetCMessage());
- }
- goto cleanup;
- }
- }
-cleanup:
- free_cri_pod_network(networks, len);
-}
-
void CRIRuntimeServiceImpl::SetupSandboxNetwork(const runtime::v1alpha2::PodSandboxConfig &config,
const std::string &response_id, const std::string &jsonCheckpoint,
Errors &error)
@@ -685,45 +647,6 @@ cleanup:
return ret;
}
-auto CRIRuntimeServiceImpl::TearDownPodCniNetwork(const std::string &realSandboxID, std::vector<std::string> &errlist,
- std::map<std::string, std::string> &stdAnnos, const std::string &ns,
- const std::string &name, Errors &error) -> int
-{
- int ret = 0;
- cri_pod_network_element **networks = nullptr;
- container_inspect *inspect_data = InspectContainer(realSandboxID, error);
- if (inspect_data == nullptr) {
- return -1;
- }
-
- google::protobuf::Map<std::string, std::string> annotations;
- CRIHelpers::ExtractAnnotations(inspect_data->config->annotations, annotations);
- size_t len = 0;
-
- networks = CRIHelpers::GetNetworkPlaneFromPodAnno(annotations, &len, error);
- if (error.NotEmpty()) {
- ERROR("Couldn't get network plane from pod annotations: %s", error.GetCMessage());
- error.Errorf("Couldn't get network plane from pod annotations: %s", error.GetCMessage());
- ret = -1;
- goto cleanup;
- }
- for (size_t i = 0; i < len; i++) {
- if ((networks[i] != nullptr) && (networks[i]->name != nullptr) && (networks[i]->interface != nullptr) &&
- strcmp(networks[i]->name, Network::DEFAULT_NETWORK_PLANE_NAME.c_str()) != 0) {
- Errors tmpErr;
- m_pluginManager->TearDownPod(ns, name, networks[i]->interface, inspect_data->id, stdAnnos, tmpErr);
- if (tmpErr.NotEmpty()) {
- WARN("TearDownPod cni network failed: %s", tmpErr.GetCMessage());
- errlist.push_back(tmpErr.GetMessage());
- }
- }
- }
-cleanup:
- free_cri_pod_network(networks, len);
- free_container_inspect(inspect_data);
- return ret;
-}
-
auto CRIRuntimeServiceImpl::ClearCniNetwork(const std::string &realSandboxID, bool hostNetwork, const std::string &ns,
const std::string &name, std::vector<std::string> &errlist,
std::map<std::string, std::string> &stdAnnos, Errors &
@@ -966,8 +889,19 @@ void CRIRuntimeServiceImpl::SetSandboxStatusNetwork(container_inspect *inspect,
std::unique_ptr<runtime::v1alpha2::PodSandboxStatus> &podStatus,
Errors &error)
{
- std::string interfaceIP = GetIP(podSandboxID, inspect, Network::DEFAULT_NETWORK_INTERFACE_NAME, error);
- podStatus->mutable_network()->set_ip(interfaceIP);
+ std::vector<std::string> ips;
+ size_t i;
+
+ GetIPs(podSandboxID, inspect, Network::DEFAULT_NETWORK_INTERFACE_NAME, ips, error);
+ if (ips.size() == 0) {
+ return;
+ }
+ podStatus->mutable_network()->set_ip(ips[0]);
+
+ for (i = 1; i < ips.size(); i++) {
+ auto tPoint = podStatus->mutable_network()->add_additional_ips();
+ tPoint->set_ip(ips[i]);
+ }
}
void CRIRuntimeServiceImpl::PodSandboxStatusToGRPC(container_inspect *inspect, const std::string &podSandboxID,
@@ -1019,66 +953,126 @@ void CRIRuntimeServiceImpl::PodSandboxStatusToGRPC(container_inspect *inspect, c
}
}
-auto CRIRuntimeServiceImpl::GetIPFromPlugin(container_inspect *inspect, const std::string &networkInterface,
- Errors &error) -> std::string
+void CRIRuntimeServiceImpl::GetFormatIPsForMultNet(container_inspect *inspect, const std::string &defaultInterface,
+ const runtime::v1alpha2::PodSandboxMetadata &metadata, std::vector<std::string> &result, Errors &error)
{
- if (inspect == nullptr || inspect->id == nullptr || inspect->name == nullptr) {
- error.SetError("Empty arguments");
- return "";
+ size_t len = 0;
+ cri_pod_network_element **elems { nullptr };
+ parser_error jerr { nullptr };
+
+ if (inspect->config == nullptr || inspect->config->annotations == nullptr) {
+ return;
+ }
+
+ for (size_t i = 0; i < inspect->config->annotations->len; i++) {
+ if (strcmp(inspect->config->annotations->keys[i], CRIHelpers::Constants::POD_NETWORK_ANNOTATION_KEY.c_str()) != 0) {
+ continue;
+ }
+ elems = cri_pod_network_parse_data(inspect->config->annotations->values[i], nullptr, &jerr, &len);
+ if (elems == nullptr) {
+ ERROR("parse mutlnetwork config failed: %s", jerr);
+ error.SetError("parse mutlnetwork config failed");
+ goto out;
+ }
+ break;
+ }
+
+ for (size_t i = 0; i < len; i++) {
+ if (elems[i]->interface == nullptr || strcmp(elems[i]->interface, defaultInterface.c_str()) == 0) {
+ continue;
+ }
+ Network::PodNetworkStatus status;
+ m_pluginManager->GetPodNetworkStatus(metadata.namespace_(), metadata.name(), elems[i]->interface, inspect->id, status,
+ error);
+ if (error.NotEmpty()) {
+ goto out;
+ }
+ // add a sentry to make ips of mutlnetwork store from position 2
+ if (result.size() < 2) {
+ result.push_back("");
+ }
+
+ result.push_back(std::string(elems[i]->name) + "@" + std::string(elems[i]->interface) + "@[" + CXXUtils::StringsJoin(
+ status.GetIPs(), ", ") + "]");
+ }
+out:
+ for (size_t i = 0; i < len; i++) {
+ free_cri_pod_network_element(elems[i]);
+ elems[i] = nullptr;
}
+ free(elems);
+ free(jerr);
+}
+auto CRIRuntimeServiceImpl::GetIPsFromPlugin(container_inspect *inspect, const std::string &networkInterface,
+ Errors &error) -> std::vector<std::string>
+{
+ std::vector<std::string> ret;
runtime::v1alpha2::PodSandboxMetadata metadata;
+ std::string defaultInterface = networkInterface;
+
+ if (inspect == nullptr || inspect->id == nullptr || inspect->name == nullptr) {
+ error.SetError("Empty arguments");
+ return ret;
+ }
CRINaming::ParseSandboxName(inspect->name, metadata, error);
if (error.NotEmpty()) {
- return "";
+ return ret;
}
- std::string cid = inspect->id;
- Network::PodNetworkStatus status;
- if (networkInterface.empty()) {
- m_pluginManager->GetPodNetworkStatus(metadata.namespace_(), metadata.name(),
- Network::DEFAULT_NETWORK_INTERFACE_NAME, cid, status, error);
- } else {
- m_pluginManager->GetPodNetworkStatus(metadata.namespace_(), metadata.name(), networkInterface, cid, status,
- error);
+ if (defaultInterface.empty()) {
+ defaultInterface = Network::DEFAULT_NETWORK_INTERFACE_NAME;
}
+
+ // step 1: get ips of default network
+ Network::PodNetworkStatus status;
+ m_pluginManager->GetPodNetworkStatus(metadata.namespace_(), metadata.name(), defaultInterface, inspect->id, status,
+ error);
if (error.NotEmpty()) {
- return "";
+ return ret;
+ }
+ for (auto &iter : status.GetIPs()) {
+ ret.push_back(iter);
}
- return status.GetIP();
+ // step 2: get ips of mutl networks
+ GetFormatIPsForMultNet(inspect, defaultInterface, metadata, ret, error);
+
+ return ret;
}
-auto CRIRuntimeServiceImpl::GetIP(const std::string &podSandboxID, container_inspect *inspect,
- const std::string &networkInterface, Errors &error) -> std::string
+void CRIRuntimeServiceImpl::GetIPs(const std::string &podSandboxID, container_inspect *inspect,
+ const std::string &networkInterface, std::vector<std::string> &ips, Errors &error)
{
if (inspect == nullptr || inspect->network_settings == nullptr) {
- return "";
+ return;
}
if (SharesHostNetwork(inspect) != 0) {
// For sandboxes using host network, the shim is not responsible for reporting the IP.
- return "";
+ return;
}
bool ready = GetNetworkReady(podSandboxID, error);
if (error.Empty() && !ready) {
WARN("Network %s do not ready", podSandboxID.c_str());
- return "";
+ return;
}
error.Clear();
- auto ip = GetIPFromPlugin(inspect, networkInterface, error);
+ auto tmpIPs = GetIPsFromPlugin(inspect, networkInterface, error);
if (error.Empty()) {
- return ip;
+ for (const auto &iter : tmpIPs) {
+ ips.push_back(iter);
+ }
+ return;
}
if (inspect->network_settings->ip_address != nullptr) {
WARN("Use container inspect ip info: %s", error.GetCMessage());
error.Clear();
- return inspect->network_settings->ip_address;
+ ips.push_back(inspect->network_settings->ip_address);
}
WARN("Failed to read pod IP from plugin/docker: %s", error.GetCMessage());
- return "";
}
std::unique_ptr<runtime::v1alpha2::PodSandboxStatus>
diff --git a/src/daemon/entry/cri/network_plugin.cc b/src/daemon/entry/cri/network_plugin.cc
index 0cab31a..311ebb6 100644
--- a/src/daemon/entry/cri/network_plugin.cc
+++ b/src/daemon/entry/cri/network_plugin.cc
@@ -65,25 +65,58 @@ static void runGetIP(void *cmdArgs)
execvp(tmpArgs[0], args);
}
-static std::string GetOnePodIP(std::string nsenterPath, std::string netnsPath, std::string interfaceName,
- std::string addrType, Errors &error)
+static std::string ParseIPFromLine(const char *line, const char *stdout_str)
{
- char *stderr_str { nullptr };
- char *stdout_str { nullptr };
- char *strErr { nullptr };
- char **lines { nullptr };
+ char *cIP { nullptr };
char **fields { nullptr };
+ char *strErr { nullptr };
struct ipnet *ipnet_val {
nullptr
};
+ std::string ret;
+
+ fields = util_string_split(line, ' ');
+ if (fields == nullptr) {
+ ERROR("Out of memory");
+ goto out;
+ }
+ if (util_array_len((const char **)fields) < 4) {
+ ERROR("Unexpected address output %s ", line);
+ goto out;
+ }
+
+ if (parse_cidr(fields[3], &ipnet_val, &strErr) != 0) {
+ ERROR("CNI failed to parse ip from output %s due to %s", stdout_str, strErr);
+ goto out;
+ }
+ cIP = ip_to_string(ipnet_val->ip, ipnet_val->ip_len);
+ if (cIP == nullptr) {
+ ERROR("Out of memory");
+ goto out;
+ }
+
+ ret = cIP;
+out:
+ free(cIP);
+ free(strErr);
+ free_ipnet_type(ipnet_val);
+ util_free_array(fields);
+ return ret;
+}
+
+static void GetOnePodIP(std::string nsenterPath, std::string netnsPath, std::string interfaceName,
+ std::string addrType, std::vector<std::string> &ips, Errors &error)
+{
+ char *stderr_str { nullptr };
+ char *stdout_str { nullptr };
+ char **lines { nullptr };
char **args { nullptr };
- std::string result { "" };
- char *cIP { nullptr };
+ size_t i;
args = (char **)util_common_calloc_s(sizeof(char *) * 5);
if (args == nullptr) {
error.SetError("Out of memory");
- return result;
+ return;
}
args[0] = util_strdup_s(nsenterPath.c_str());
@@ -102,52 +135,55 @@ static std::string GetOnePodIP(std::string nsenterPath, std::string netnsPath, s
error.SetError("Out of memory");
goto free_out;
}
- if (util_array_len((const char **)lines) < 1) {
- error.Errorf("Unexpected command output %s", stdout_str);
- goto free_out;
- }
- fields = util_string_split(lines[0], ' ');
- if (fields == nullptr) {
- error.SetError("Out of memory");
- goto free_out;
- }
- if (util_array_len((const char **)fields) < 4) {
- error.Errorf("Unexpected address output %s ", lines[0]);
+ if (util_array_len((const char **)lines) == 0) {
+ error.Errorf("Unexpected command output %s", stdout_str);
goto free_out;
}
- if (parse_cidr(fields[3], &ipnet_val, &strErr) != 0) {
- error.Errorf("CNI failed to parse ip from output %s due to %s", stdout_str, strErr);
- goto free_out;
- }
- cIP = ip_to_string(ipnet_val->ip, ipnet_val->ip_len);
- if (cIP == nullptr) {
- error.SetError("Out of memory");
- goto free_out;
+ for (i = 0; i < util_array_len((const char **)lines); i++) {
+ // ip string min length must bigger than 4
+ if (lines[i] == nullptr || strlen(lines[i]) < 4) {
+ continue;
+ }
+ std::string tIP = ParseIPFromLine(lines[i], stdout_str);
+ if (tIP.empty()) {
+ error.Errorf("parse %s to ip failed", lines[i]);
+ break;
+ }
+ ips.push_back(tIP);
}
- result = cIP;
- free(cIP);
free_out:
- free_ipnet_type(ipnet_val);
free(stdout_str);
free(stderr_str);
util_free_array(args);
util_free_array(lines);
- util_free_array(fields);
- return result;
}
-std::string GetPodIP(const std::string &nsenterPath, const std::string &netnsPath, const std::string &interfaceName,
- Errors &error)
+void GetPodIP(const std::string &nsenterPath, const std::string &netnsPath, const std::string &interfaceName,
+ std::vector<std::string> &getIPs, Errors &error)
{
- std::string ip = GetOnePodIP(nsenterPath, netnsPath, interfaceName, "-4", error);
+ Errors tmpErr;
+
+ GetOnePodIP(nsenterPath, netnsPath, interfaceName, "-4", getIPs, tmpErr);
+ if (tmpErr.NotEmpty()) {
+ WARN("Get ipv4 failed: %s", tmpErr.GetCMessage());
+ }
+
+ GetOnePodIP(nsenterPath, netnsPath, interfaceName, "-6", getIPs, error);
if (error.NotEmpty()) {
- return GetOnePodIP(nsenterPath, netnsPath, interfaceName, "-6", error);
+ WARN("Get ipv6 failed: %s", tmpErr.GetCMessage());
}
- return ip;
+ if (getIPs.size() > 0) {
+ error.Clear();
+ return;
+ }
+
+ if (tmpErr.NotEmpty()) {
+ error.AppendError(tmpErr.GetMessage());
+ }
}
void InitNetworkPlugin(std::vector<std::shared_ptr<NetworkPlugin>> *plugins, std::string networkPluginName,
@@ -290,14 +326,14 @@ void PodNetworkStatus::SetAPIVersion(const std::string &version)
m_apiVersion = version;
}
-const std::string &PodNetworkStatus::GetIP() const
+const std::vector<std::string> &PodNetworkStatus::GetIPs() const
{
- return m_ip;
+ return m_ips;
}
-void PodNetworkStatus::SetIP(const std::string &ip)
+void PodNetworkStatus::SetIPs(std::vector<std::string> &ips)
{
- m_ip = ip;
+ m_ips = ips;
}
void PluginManager::Lock(const std::string &fullPodName, Errors &error)
diff --git a/src/daemon/entry/cri/network_plugin.h b/src/daemon/entry/cri/network_plugin.h
index 5a46eb8..24afd71 100644
--- a/src/daemon/entry/cri/network_plugin.h
+++ b/src/daemon/entry/cri/network_plugin.h
@@ -82,13 +82,13 @@ public:
void SetKind(const std::string &kind);
const std::string &GetAPIVersion() const;
void SetAPIVersion(const std::string &version);
- const std::string &GetIP() const;
- void SetIP(const std::string &ip);
+ const std::vector<std::string> &GetIPs() const;
+ void SetIPs(std::vector<std::string> &ips);
private:
std::string m_kind;
std::string m_apiVersion;
- std::string m_ip;
+ std::vector<std::string> m_ips;
};
class NetworkPlugin {
@@ -227,8 +227,8 @@ void InitNetworkPlugin(std::vector<std::shared_ptr<NetworkPlugin>> *plugins, std
void ProbeNetworkPlugins(const std::string &pluginDir, const std::string &binDir,
std::vector<std::shared_ptr<NetworkPlugin>> *plugins);
-std::string GetPodIP(const std::string &nsenterPath, const std::string &netnsPath, const std::string &interfaceName,
- Errors &error);
+void GetPodIP(const std::string &nsenterPath, const std::string &netnsPath, const std::string &interfaceName,
+ std::vector<std::string> &getIPs, Errors &error);
const std::string &GetInterfaceName();
} // namespace Network
--
2.20.1

159
0027-add-testcases-for-mutl-networks.patch Normal file
View File

@ -0,0 +1,159 @@
From c8eaab912c22d2a46edf308a4dd71c609d8b683c Mon Sep 17 00:00:00 2001
From: haozi007 <liuhao27@huawei.com>
Date: Tue, 10 Nov 2020 10:51:31 +0800
Subject: [PATCH 27/28] add testcases for mutl networks
Signed-off-by: haozi007 <liuhao27@huawei.com>
---
CI/install_depends.sh | 7 +++--
CI/test_cases/container_cases/cni_test.sh | 31 ++++++++++++++++---
.../container_cases/criconfigs/bridge.json | 17 +++++-----
.../criconfigs/mutlnet_pod.json | 17 ++++++++++
4 files changed, 57 insertions(+), 15 deletions(-)
create mode 100644 CI/test_cases/container_cases/criconfigs/mutlnet_pod.json
diff --git a/CI/install_depends.sh b/CI/install_depends.sh
index 61dd67d..5dd2543 100755
--- a/CI/install_depends.sh
+++ b/CI/install_depends.sh
@@ -38,10 +38,13 @@ mkdir -p ${builddir}/systemd/system
function make_crictl()
{
cd ~
- git clone -b release-1.14 https://gitee.com/duguhaotian/cri-tools.git
+ git clone https://gitee.com/duguhaotian/cri-tools.git
+ go version
cd cri-tools
+ git checkout v1.18.0
make -j $nproc
- cp ./_output/bin/crictl ${builddir}/bin/
+ echo "make cri-tools: $?"
+ cp ./_output/crictl ${builddir}/bin/
}
#install cni plugins
diff --git a/CI/test_cases/container_cases/cni_test.sh b/CI/test_cases/container_cases/cni_test.sh
index 8173cb5..c9e1e1a 100644
--- a/CI/test_cases/container_cases/cni_test.sh
+++ b/CI/test_cases/container_cases/cni_test.sh
@@ -53,7 +53,7 @@ function do_test_help()
TC_RET_T=$(($TC_RET_T+1))
fi
- sid=`crictl runp ${data_path}/sandbox-config.json`
+ sid=`crictl runp ${data_path}/$1`
if [ $? -ne 0 ]; then
msg_err "Failed to run sandbox"
TC_RET_T=$(($TC_RET_T+1))
@@ -66,7 +66,7 @@ function do_test_help()
TC_RET_T=$(($TC_RET_T+1))
fi
- cid=`crictl create $sid ${data_path}/container-config.json ${data_path}/sandbox-config.json`
+ cid=`crictl create $sid ${data_path}/container-config.json ${data_path}/$1`
if [ $? -ne 0 ];then
msg_err "create container failed"
TC_RET_T=$(($TC_RET_T+1))
@@ -96,12 +96,33 @@ function do_test_help()
nsenter -t $con_pid -n ifconfig eth0
TC_RET_T=$(($TC_RET_T+1))
fi
- nsenter -t $pod_pid -n ifconfig eth0 | grep "$1"
+ nsenter -t $pod_pid -n ifconfig eth0 | grep "$2"
if [ $? -ne 0 ];then
msg_err "expect ip: $1, get: "
nsenter -t $pod_pid -n ifconfig eth0
TC_RET_T=$(($TC_RET_T+1))
fi
+ crictl inspectp $sid | grep "$2"
+ if [ $? -ne 0 ];then
+ msg_err "inspectp: expect ip: $1, get: "
+ crictl inspectp $sid
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+
+ if [ "x$3" != "x" ];then
+ nsenter -t $pod_pid -n ifconfig eth1 | grep "$3"
+ if [ $? -ne 0 ];then
+ msg_err "expect ip: $2, get: "
+ nsenter -t $pod_pid -n ifconfig eth1
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+ crictl inspectp $sid | grep "$3"
+ if [ $? -ne 0 ];then
+ msg_err "inspectp expect ip: $2, get: "
+ crictl inspectp $sid
+ TC_RET_T=$(($TC_RET_T+1))
+ fi
+ fi
crictl stop $cid
if [ $? -ne 0 ];then
@@ -132,7 +153,7 @@ function do_test_help()
function default_cni_config()
{
- do_test_help "10\.1\."
+ do_test_help "sandbox-config.json" "10\.1\."
}
function new_cni_config()
@@ -151,7 +172,7 @@ function new_cni_config()
fi
done
tail $ISUALD_LOG
- do_test_help "10\.2\."
+ do_test_help "mutlnet_pod.json" "10\.2\." "10\.1\."
}
function check_annotation()
diff --git a/CI/test_cases/container_cases/criconfigs/bridge.json b/CI/test_cases/container_cases/criconfigs/bridge.json
index 4d19fa7..7686e26 100644
--- a/CI/test_cases/container_cases/criconfigs/bridge.json
+++ b/CI/test_cases/container_cases/criconfigs/bridge.json
@@ -1,10 +1,11 @@
{
- "cniVersion": "0.3.0",
- "name": "ok",
- "type": "bridge",
- "bridge": "cni0",
- "ipam": {
- "type": "host-local",
- "subnet": "10.2.0.0/16"
- }
+ "cniVersion": "0.3.0",
+ "name": "ok",
+ "type": "bridge",
+ "bridge": "cni0",
+ "isGateway": true,
+ "ipam": {
+ "type": "host-local",
+ "subnet": "10.2.0.0/16"
+ }
}
diff --git a/CI/test_cases/container_cases/criconfigs/mutlnet_pod.json b/CI/test_cases/container_cases/criconfigs/mutlnet_pod.json
new file mode 100644
index 0000000..f860620
--- /dev/null
+++ b/CI/test_cases/container_cases/criconfigs/mutlnet_pod.json
@@ -0,0 +1,17 @@
+{
+ "port_mappings":[{"protocol": 1, "container_port": 80, "host_port": 8080}],
+ "metadata": {
+ "name": "test",
+ "namespace": "default",
+ "attempt": 1,
+ "uid": "hdishd83djaidwnduwk28bcsb"
+ },
+ "linux": {
+ },
+ "annotations": {
+ "network.alpha.kubernetes.io/network": "[{\"name\":\"good\",\"interface\":\"eth1\"}]",
+ "extension.network.kubernetes.io/cni": "[multinetwork]",
+ "extension.network.kubernetes.io/cniargs/first": "extension=first",
+ "extension.network.kubernetes.io/cniargs/second": "extension=second"
+ }
+}
--
2.20.1

43
0028-add-filter-to-get-only-non-sandbox-containers.patch Normal file
View File

@ -0,0 +1,43 @@
From 1a01070d00c9bf9ff65308522486edcfe16ed46c Mon Sep 17 00:00:00 2001
From: gaohuatao <gaohuatao@huawei.com>
Date: Wed, 11 Nov 2020 11:10:36 +0800
Subject: [PATCH 28/28] add filter to get only non-sandbox containers
Signed-off-by: gaohuatao <gaohuatao@huawei.com>
---
src/daemon/entry/cri/cri_container.cc | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/src/daemon/entry/cri/cri_container.cc b/src/daemon/entry/cri/cri_container.cc
index c6d9599..e23e59e 100644
--- a/src/daemon/entry/cri/cri_container.cc
+++ b/src/daemon/entry/cri/cri_container.cc
@@ -788,6 +788,12 @@ void CRIRuntimeServiceImpl::ListContainersFromGRPC(const runtime::v1alpha2::Cont
error.SetError("Out of memory");
return;
}
+ // Add filter to get only non-sandbox containers
+ if (CRIHelpers::FiltersAddLabel((*request)->filters, CRIHelpers::Constants::CONTAINER_TYPE_LABEL_KEY,
+ CRIHelpers::Constants::CONTAINER_TYPE_LABEL_CONTAINER) != 0) {
+ error.SetError("Failed to add filter");
+ return;
+ }
if (filter != nullptr) {
if (!filter->id().empty()) {
@@ -811,12 +817,6 @@ void CRIRuntimeServiceImpl::ListContainersFromGRPC(const runtime::v1alpha2::Cont
}
}
- // Add some label
- if (CRIHelpers::FiltersAddLabel((*request)->filters, CRIHelpers::Constants::CONTAINER_TYPE_LABEL_KEY,
- CRIHelpers::Constants::CONTAINER_TYPE_LABEL_CONTAINER) != 0) {
- error.SetError("Failed to add filter");
- return;
- }
for (auto &iter : filter->label_selector()) {
if (CRIHelpers::FiltersAddLabel((*request)->filters, iter.first, iter.second) != 0) {
error.SetError("Failed to add filter");
--
2.20.1

37
iSulad.spec
View File

@ -1,5 +1,5 @@
%global _version 2.0.6
%global _release 20201015.154058.git4ab9f838
%global _release 20201112.193005.git8a6b73c8
%global is_systemd 1
Name: iSulad
@ -12,6 +12,35 @@ Source: https://gitee.com/openeuler/iSulad/repository/archive/v%{version}.tar
BuildRoot: {_tmppath}/iSulad-%{version}
ExclusiveArch: x86_64 aarch64
Patch6001: 0001-iSulad-modify-defattr-to-755-in-spec.patch
Patch6002: 0002-Dockerfile-update-dockerfile-to-isulad-v2.0.6-use-mu.patch
Patch6003: 0003-isulad-rt_isula_start-should-read-the-isulad-shim-pi.patch
Patch6004: 0004-Realpath-add-get-realpath-for-root-and-state-dir.patch
Patch6005: 0005-CI-add-testcase-for-root-and-run-dir-realpath.patch
Patch6006: 0006-info-fix-typo-driverr-to-driver.patch
Patch6007: 0007-create-fix-wrong-ret-code.patch
Patch6008: 0008-add-iSulad-s-build-guide-for-RISC-V.patch
Patch6009: 0009-add-non-root-group.patch
Patch6010: 0010-add-nonroot-execute-CI.patch
Patch6011: 0011-cni-support-extension-data-transmission.patch
Patch6012: 0012-unpack-add-remove-target-file-in-handle-.wh.patch
Patch6013: 0013-iSulad-internal-change.patch
Patch6014: 0014-unlink-etc-dir-when-link-exists.patch
Patch6015: 0015-support-variable-extension-cni-args.patch
Patch6016: 0016-CI-for-support-variable-extension-cni-args.patch
Patch6017: 0017-add-unlink-dir-comments.patch
Patch6018: 0018-iSulad-add-ISULAD_TMPDIR-env-variable.patch
Patch6019: 0019-CI-remove-test-data-from-iSulad-repo.patch
Patch6020: 0020-add-ISULAD_TMPDIR-env-CI.patch
Patch6021: 0021-iSulad-fix-memory-leak-in-inspect-grpc-service.patch
Patch6022: 0022-clean-code-remove-unused-code-in-connect.patch
Patch6023: 0023-add-newline-character-at-end-of-iSulad.sysconfig.patch
Patch6024: 0024-clean-code-remove-unused-in-code.patch
Patch6025: 0025-utils-add-fdatasync-when-do-atomic-write-file.patch
Patch6026: 0026-network-support-mutlnetworks.patch
Patch6027: 0027-add-testcases-for-mutl-networks.patch
Patch6028: 0028-add-filter-to-get-only-non-sandbox-containers.patch
%ifarch x86_64 aarch64
Provides: libhttpclient.so()(64bit)
Provides: libisula.so()(64bit)
@ -213,6 +242,12 @@ fi
%endif
%changelog
* Thu Nov 12 2020 gaohuatao<gaohuatao@huawei.com> - 2.0.6-20201112.193005.git8a6b73c8
- Type: update from openeuler
- ID:NA
- SUG:NA
- DESC: update from openeuler
* Wed Oct 14 2020 lifeng68<lifeng68@huawei.com> - 2.0.6-20201014.152749.gitc8a43925
- Type: upgrade to v2.0.6
- ID:NA