111 lines
141 KiB
Diff
111 lines
141 KiB
Diff
|
From 426b309efdee934f61a6fb27b278711aa5419dd5 Mon Sep 17 00:00:00 2001
|
|||
|
|
From: zhongtao <zhongtao17@huawei.com>
|
|||
|
|
Date: Mon, 13 Nov 2023 08:22:46 +0000
|
|||
|
|
Subject: [PATCH 08/14] !2233 add runc append function design doc * add runc
|
|||
|
|
append function design doc
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
.../detailed/Runtime/runc_design_append.md | 75 +++++++++++++++++++
|
|||
|
|
docs/images/runc_isula_attach_flow_chart.svg | 5 ++
|
|||
|
|
2 files changed, 80 insertions(+)
|
|||
|
|
create mode 100644 docs/design/detailed/Runtime/runc_design_append.md
|
|||
|
|
create mode 100644 docs/images/runc_isula_attach_flow_chart.svg
|
|||
|
|
|
|||
|
|
diff --git a/docs/design/detailed/Runtime/runc_design_append.md b/docs/design/detailed/Runtime/runc_design_append.md
|
|||
|
|
new file mode 100644
|
|||
|
|
index 00000000..aa17f558
|
|||
|
|
--- /dev/null
|
|||
|
|
+++ b/docs/design/detailed/Runtime/runc_design_append.md
|
|||
|
|
@@ -0,0 +1,75 @@
|
|||
|
|
+| Author | zhongtao |
|
|||
|
|
+| ------ | --------------------- |
|
|||
|
|
+| Date | 2023-10-19 |
|
|||
|
|
+| Email | zhongtao17@huawei.com |
|
|||
|
|
+
|
|||
|
|
+# 方案目标
|
|||
|
|
+
|
|||
|
|
+isulad当前默认的runtime为lcr + lxc，具有高性能的特点。但是该runtime是通过oci规范转换为lxc规范实现的，而且在lxc中进行了较多适配修改。
|
|||
|
|
+
|
|||
|
|
+随着oci 规范的日益成熟，oci规范实现的runtime能满足各种场景需求。因此iSulad实现基于oci 规范的runtime对接，具有底噪更低的优势。
|
|||
|
|
+
|
|||
|
|
+需求目标分为以下5点：
|
|||
|
|
+
|
|||
|
|
+1. 对于isula 命令行支持 oci runtime 容器的功能进行全量排查，识别缺失功能。
|
|||
|
|
+2. 补齐缺失功能，保证切换默认runtime为runc之后功能的完整性，并针对新增功能补充对应的单元测试与门禁测试。
|
|||
|
|
+3. 梳理isulad-shim依赖情况，解耦isulad-shim依赖，减少runc容器底噪，并且提高稳定性（静态编译isulad-shim，从而使得isulad-shim不依赖isulad版本，支持isulad热升级）
|
|||
|
|
+4. 重构社区门禁CI框架，支持多 runtime 测试。
|
|||
|
|
+
|
|||
|
|
+# 总体设计
|
|||
|
|
+
|
|||
|
|
+由于isulad与runc之间的交互存在gap，且将容器创建成功之后，容器进程的生命周期与isulad进程的生命周期没有必然联系，因此isulad设计了一个isulad-shim进程，用于isulad与runc的交互并将isulad与容器实例解耦。
|
|||
|
|
+
|
|||
|
|
+isulad 与 isulad-shim的关系以及整体结构请参照：[runc_design](./runc_design_zh.md) 。
|
|||
|
|
+
|
|||
|
|
+由于isula attach 涉及到attach中新建的fifo fd与容器首进程io进行数据交换，因此需要isulad与容器首进程对应的isulad-shim进行通信，完成建立io 连接并进行io copy操作。
|
|||
|
|
+
|
|||
|
|
+isula top 命令仅需要在isulad中直接调用runc二进制。
|
|||
|
|
+# 接口描述
|
|||
|
|
+
|
|||
|
|
+## isula_rt_ops模块
|
|||
|
|
+
|
|||
|
|
+```c
|
|||
|
|
+
|
|||
|
|
+int rt_isula_attach(const char *id, const char *runtime, const rt_attach_params_t *params);
|
|||
|
|
+
|
|||
|
|
+int rt_isula_listpids(const char *name, const char *runtime, const rt_listpids_params_t *params,
|
|||
|
|
+ rt_listpids_out_t *out);
|
|||
|
|
+```
|
|||
|
|
+# 详细设计
|
|||
|
|
+
|
|||
|
|
+## attach 实现流程
|
|||
|
|
+
|
|||
|
|
+### 流程图
|
|||
|
|
+
|
|||
|
|
+
|
|||
|
|
+
|
|||
|
|
+### 详细流程
|
|||
|
|
+
|
|||
|
|
+isulad端：
|
|||
|
|
+
|
|||
|
|
+1. 创建容器时（rt_isula_create），传递给isulad-shim attach socket path；
|
|||
|
|
+2. 进行attach操作时（rt_isula_attach），先根据 attach socket path与isulad-shim的socket server端建立连接，获得通信的socket fd。
|
|||
|
|
+3. 将attach 的stdin stdout与stderr fifo路径写入到socket fd中。
|
|||
|
|
+4. 从socket fd中读取isulad-shim是否成功将attach的fd加入到epoll中成功连接，若成功则直接成功返回，若失败则获取attach-log.json文件中的报错信息后报错返回。
|
|||
|
|
+
|
|||
|
|
+isulad-shim端：
|
|||
|
|
+
|
|||
|
|
+1. create isulad-shim进程时，若传递的process 中包含 attach socket path， 则创建一个本地unix socket文件，用于isulad与isulad-shim之间通信。本地unix socket文件打开获得attach_isulad_fd，将attach_isulad_fd加入到epoll需要监听的fd中。
|
|||
|
|
+2. 收到attach_isulad_fd的事件后，调用do_attach_socket_accept函数，accept到通信的conn_fd后，将conn_fd加入到epoll需要监听的fd中。
|
|||
|
|
+3. 在收到conn_fd事件后，调用attach_fd函数。attach_fd函数中从attach_isulad_fd中读出stdin stdout与stderr路径，之后在shim中打开fd。将stdin对应的fd加入epoll监听列表中，有事件时调用stdin_cb。若容器有输入与输出，则除了写入到初始isulad fd中之外，还需要写入到attach的fd list中。
|
|||
|
|
+
|
|||
|
|
+### 新增文件
|
|||
|
|
+1. /run/isulad/runc/{container_id}/attach_socket.sock 用于isulad与isulad-shim attach通信
|
|||
|
|
+2. /run/isulad/runc/{container_id}/attach-log.json 用于记录isulad-shim中attach的报错信息。目前所有的attach操作共用一个attach文件
|
|||
|
|
+
|
|||
|
|
+### 未来规划
|
|||
|
|
+1. runc容器的attach支持魔术符退出（CTRL + Q），退出后不影响容器首进程的运行，也不影响其他并发的attach操作。
|
|||
|
|
+2. 由于支持魔术符退出，在GRPC版本中，可设置了ISULAD_INFO_DETACH错误码，用于标识用户输入魔术符退出，而rest版本中由于实现差异，无法识别exit退出与魔术符退出的差异，因此在魔术符退出时，会存在一条INFO级别的报错信息：`INFO("Wait container stopped status timeout.");`
|
|||
|
|
+
|
|||
|
|
+## top 实现流程
|
|||
|
|
+
|
|||
|
|
+isulad端：
|
|||
|
|
+
|
|||
|
|
+1. 直接调用runc二进制ps容器：runtime_call_simple(), 设置选项" --format json"；
|
|||
|
|
+2. 直接解析调用返回的stdout中的容器进程pid，将其写入到rt_listpids_out_t结构体中；
|
|||
|
|
\ No newline at end of file
|
|||
|
|
diff --git a/docs/images/runc_isula_attach_flow_chart.svg b/docs/images/runc_isula_attach_flow_chart.svg
|
|||
|
|
new file mode 100644
|
|||
|
|
index 00000000..e57a4ec0
|
|||
|
|
--- /dev/null
|
|||
|
|
+++ b/docs/images/runc_isula_attach_flow_chart.svg
|
|||
|
|
@@ -0,0 +1,5 @@
|
|||
|
|
+<svg version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1339.8517879909941 1669.333441840278" width="1339.8517879909941" height="1669.333441840278" filter="invert(93%) hue-rotate(180deg)">
|
|||
|
|
+ <!-- svg-source:excalidraw -->
|
|||
|
|
+
|
|||
|
|
+ <defs><style> @font-face {font-family: "Virgil";src: url("data:application/font-woff;charset=utf-8;base64,d09GMk9UVE8AAO9AAAkAAAABO1AAAO73AAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAADYTJHQZgAIluATYCJAOQWAQGBY9lByBbpzpxQ1QZOitEWvl1ExHg7NqqxsS6RlCwSkciyuhqMfv//89KOsZwUAeGmJXV32EeijNbRUcYuVUTmWNvG5FWRBfkAW4YWVueXU6tq9d0pZufMcACcUzc7AmXTyBwYmw0WojBScItqLq853lRPMu/i21F/1VrveHFeyj+LdHE3AfsFiWacmNotmSIhpJesVGSFIFWJDgsMxNmRIsQQzGF4rvOX33MPtY+LeOF490dgR0G/l0zmGbP8ku9beGBOf2+EcKFm8Md59ESM5GZKYtFok/F14jmMBPlbsKEWTmjIcDMVCgM0RQv1VD4IZddn+x6g2Ff91p6BriTOHYqRgBPP/Z8O3Pn/WXBXUTBKoZoSbZg6N3qwPP/937uvc+bD5huHSziTSgio5mQNpAsr235cGq9P8/r5vc1H/KatrSSHj8fGwzDhavuia24cMyBvutkjYQtKFtwTcS1wVWcODY7KMSFOM69OS/+4fm59f7vRf5FsY0aMXLQQgsGrRwggmTYYCJVEmVBm4V5Z0ThidFYPeyrtRKEoeme3dvtgT6iOeJnlC8MsAKKT0UoFD469oV+l7DD8pn6lvkVc++0kSEfkCM0Vauqh+RnHNszHtYIqREKELpb3S0eAhMvIXmB7+tkb9/wpNO+5fOjmT/aU+6YYwu6wVlib1xrbGxcwNgduqmiNNOESimjUWmogISEAOOOe086XjvNccGlpjQnb8gjOY3Gyr1sLlmavf8vKSIJwtUoBiEBfY1jpaqrq1ET/z2/7f+/4/hmD/yU4cZxDvyseBGJ7/W9rVwMSlAkJUsk6hzqQCsqJVbd9NbifX/4r6pJ4LDfsD5vl0a3Nl1TSqk09SUxpviFT9KCpj3jFmScMz4t9f8VWXkadd4bSR35QqxNH33Ai5BewBI5hQAbQsZFe2nQZ84A787uLKHXduwYMHUawjI9wuP+7R+jZdW/WpL3hdof92mkcoac8YBwj3gJB7CB3aayqzgrsSIikYrAZYK2u3l6AJfgdm+OybOvB8+JTFAi4hU026JOz7fHCxQujUCxVVutY8Wgje+dK4CA+afpvl+9Nyecky85rLBnngvwxHEpBa9ATSI+MxnxjMcbadI6LaWIPXHlOeusTrrDBkAFCAzMc6UTOFEFKTY1Nm4GuAPasdAah4+DcVmNg/pfU5Nao+e7QykouG0KYR4ewkKQVmtP1n91ike6Kvt60/WySWeBpeOKr7QCg0hoKAwgPPOvqrl+UPI78CqVirROp/RhKm2YMy0C6CJQdAHdBNBOBOmKyGuC5AZe6X1LKQUflBN+2M7DB2VHoBs/KOcRvCZQfn6i0lsdM5Ypw5otoyCn0UlftixbMiaTk2XKy5Q1Y17G2zJOB7nKUm0/PkgM7J3K/X5azIxlS8eShkmeIYrIIuvN47j+8Xk5xmbzq/fd0AYlB/aifvZu/3or31eH2eCosZNM23gKUvMyo25K7fx8x2ZRqLM4/xxLbOST4x/7aOuym35y8olHctP4b8V/TX/YCP2p/1mult99+Gj0u488j59Q7/DTaWN4EkPx7itzRBhgghZ7XPAiQJAIcTLMtFCJKku1WqHHWlvsst9xLnKDYR4xwicQZ7tG+knSE5NnLF69LLqrR0X89vtHVudoGtiw59gDkletTr9hov04IcNsWAhCFGnUMYNrWMA6cqigBQoCbHTgQRhxDOMAjuIkzuIyzuAybuEuHmMEbxC773VEWIigqlrWqT4NbHCjmtiM5nd+y1vXlq5ob9d1c/f3eC93qA/7pk6NEihFn0Rmmnk61thin7O8wilOc4krXGeGedbYJEaSKhN6GOUQD/IKz/ACr/IW7/ERn/Il3/ITf/B1//9y0DjoHH5x+M2hIRCCxhEpTKlFmUZcYipGDGMmFmBNbIBtYYfYXewxq9h9hvn6/vr3thQeZ+KUArHScmqorznNakXbKqkpSqIM+SLlVlB9GtJ+HdUZXdZZXdGQ7uuZ3uizfnlBkASHEFk57pwrbnvKU17wunOuGjNrxbYj2+1zj/u92yM+4fO+5gu+7jt+6Od+40+uMZH/QTH0f6P/F2WjGegMdD6aieagxWg5WoU2oG0oH5WhWgfrEB2GI+UoOOqOGcc5+Rat6/9k/8+mUCCeJiIC5sZapHx9QEX2IMLTPuIbQjZoyoY9XgDCc1CSUcEfzRVFJcPJ4YfarTTXnEEp9Fll4bmCyF6huXD0c2NkyBrUclqFDS0fCcldmnBxq5ZzUdjQ0hBhra3noZkdrY2QIXdtwDXbzuJpombNK3hcpdld/aaHaGW4yOQ0Uq2KuSpqqHgkIlXMSxG5hZWvipLO5Yja+dGe2Ec0Xl7hJVeLriry+vKRHMnapHSSWh3Na/r0xosnKF8vBiOnHHm02DmrmKh7JKa/6Kx4A1EGB5L5gPZja3EC36Px8gVmpEMfpVo7eKal3dCDQk3OQWrtEfywxsP7kXKdc+xllnS10CSorh6UNHRFJNThKpI3Uj12NefH5EBdrYWSRruZkfa+i7mL+aR7b+irdu+/Svdr602Xpz3AJtUl38juYpi/pk0HYWKZtRHy//WcNGHEROmgcepGC9vOyzGRY1+qfJxX8Ka4G2gxbNKvLqrqBjSlQRMYHTrDDGsYBjAs71XrIXW0LYetKxhHqGHoNTTSyi1NsvSW9pYPZAY4gkxWtrU0z9vbRMU08fOJQxc+Syy+gtk9G54gA+xj+zrZX5Y0rjZrdBmGy68X/sBnzCF5AdMtKp4mqPbyQez0lZGifaq47Tf7SIZExoiy9kkO2JjFGs8VnQMdK9nMR0/nKEUbGADCEcjocG6DUGjz+CoAhQFwBBIDEwuFxo4bN+++CUBhAAhHIDEwsVBobBxhCwAAwIcDAomBiYVCY+OISUAQBH0SBIEgCIIgCIKgcvQdCMIRSAy0MCC+M6Y5BcVI/gkYkMn8sRZzoj6jjymGesj9+I2a22/ocz8NNPA2C+fgZtOvbe7ZbLKKuW1NchzK6mJ1Pf/1IF9PGNzuWk+fWr5AJLsGbXx6WqXtz7Sw5sK/2TRigdQEnKWCSwHOJDBxwFWcy/MuPpBExwGTydS++B38ygHYhoswSGJKao0cwRU+cxa87AMKDql645JDKYO+5m4QVzcJ3y7stzuYTaZTHAbNHXHpPfKmuHugSHKiUk/ZPC/TXrC0xqt/dsGQ57sgXd3hs/6nDbLRY/DknG/yxVziZVz+ldwy0Iy6cNfv9o1UVPeO7SRI/Eej3M5tWU/0ydP7mv8W+vb7QPcr3ytNdXF9bro22Y7bj7Zf+/5e7C/73n67/xrB1Iy9URj6uDPOjzvj1fgeZsnDRCrKMR3LsR2VaAcbarhhDXcEIhLDcSROx6U4E5fjZgzHoxiJt3PJ9GZwpV151l4c+73B+fGXExniT6JM1AELXkAM6XgMP5YTZEIWvN+8x96yP86/HmkfeR30BDeCq6FruBPGwn1qRw26oBG6Ql8N9qqpBupSDaqaqtbOeqAPtE9H9a6ua4yag2bZtBiT0RqTSZtKYzeKbdmu2Fors26rtyFbtJX2mzudd3CO+7zBVdzBKraziKWsZzf7OcUPuIzfccG8beSYgWdMypiWsTBjeUZ2xraMHRl1GW0ZwgxVhjlDyrAzchn1jJlUWRX48WpiZWxuhR6qmw57kBDolGzsgvY89rhb7XvJzwQfY4+20SX4OtWreKpuSFaXlECpiOVamcv4JA4/x8nKhM2qKCxtMjn4oyulp5+2+DzPxYtYzPSoPf9Yvlvd4vcuYU3142xsRkmAfbwfK6ODt6039/4JpW8VEqlHIZIunBL+6JmzfHyE3o3mF33fTPtyd+kt8NrwT5yY3tXcECqX0RZRjsN416y3bBO8IjDJcPLMjeX3UFc4USotsE
|
|||
|
|
+ <rect x="0" y="0" width="1339.8517879909941" height="1669.333441840278" fill="#ffffff"/><g stroke-linecap="round" transform="translate(174.05543178982202 183.77775743272582) rotate(0 107.5 36.5)"><path d="M18.25 0 M18.25 0 C53 -1.5, 90.87 -2.3, 196.75 0 M18.25 0 C62.45 1.19, 105.88 0.54, 196.75 0 M196.75 0 C207.71 0.99, 215.09 4.48, 215 18.25 M196.75 0 C208.37 -1.99, 217.23 8, 215 18.25 M215 18.25 C216.3 28.51, 213.15 40.3, 215 54.75 M215 18.25 C213.85 27.93, 213.77 35.5, 215 54.75 M215 54.75 C214.62 68.28, 207.35 74.34, 196.75 73 M215 54.75 C215.37 68.77, 208.59 72.07, 196.75 73 M196.75 73 C141.79 73.81, 90.16 72.82, 18.25 73 M196.75 73 C155.72 72.13, 114.23 72.98, 18.25 73 M18.25 73 C5.95 71.03, -1.38 66.71, 0 54.75 M18.25 73 C5.89 71.93, 1.51 66.94, 0 54.75 M0 54.75 C-0.8 40.57, -2.19 28.17, 0 18.25 M0 54.75 C-0.05 47.77, 0.3 39.75, 0 18.25 M0 18.25 C-0.39 5.34, 7.37 0.73, 18.25 0 M0 18.25 C1.35 7.7, 6.44 2.04, 18.25 0" stroke="#1e1e1e" stroke-width="1" fill="none"/></g><g transform="translate(199.90550655788843 207.77775743272582) rotate(0 81.6499252319336 12.5)"><text x="81.6499252319336" y="0" font-family="Virgil, Segoe UI Emoji" font-size="20px" fill="#1e1e1e" text-anchor="middle" style="white-space: pre;" direction="ltr" dominant-baseline="text-before-edge">rt_isula_create</text></g><g stroke-linecap="round"><g transform="translate(390.055431789822 220.95750009135367) rotate(0 220.39825439453125 -3.209137645198979)"><path d="M-0.01 -1.17 C73.54 -1.86, 367.34 -4.59, 440.8 -5.38 M-1.48 0.84 C71.97 -0.07, 366.29 -5.49, 439.86 -6.96" stroke="#1e1e1e" stroke-width="1" fill="none"/></g><g transform="translate(390.055431789822 220.95750009135367) rotate(0 220.39825439453125 -3.209137645198979)"><path d="M409.92 3.85 C420.83 0.99, 427.08 -1.06, 439.48 -7.35 M412.46 4.54 C419.31 1.78, 427.28 -2.89, 439.76 -6.39" stroke="#1e1e1e" stroke-width="1" fill="none"/></g><g transform="translate(390.055431789822 220.95750009135367) rotate(0 220.39825439453125 -3.209137645198979)"><path d="M409.54 -16.67 C420.34 -14.3, 426.69 -11.12, 439.48 -7.35 M412.07 -15.98 C419.16 -13.34, 427.23 -12.62, 439.76 -6.39" stroke="#1e1e1e" stroke-width="1" fill="none"/></g></g><mask/><g transform="translate(577.1851009792751 179.99996948242188) rotate(0 98.7699203491211 12.5)"><text x="0" y="0" font-family="Virgil, Segoe UI Emoji" font-size="20px" fill="#1e1e1e" text-anchor="start" style="white-space: pre;" direction="ltr" dominant-baseline="text-before-edge">attach socket path</text></g><g stroke-linecap="round" transform="translate(835.3518795437283 169.83329010009766) rotate(0 147 42.5)"><path d="M21.25 0 M21.25 0 C82.49 -1.78, 141.93 -0.47, 272.75 0 M21.25 0 C92.49 2.14, 163.07 2.39, 272.75 0 M272.75 0 C285.28 1.47, 292.6 8.31, 294 21.25 M272.75 0 C284.94 2.16, 292.99 5.01, 294 21.25 M294 21.25 C294.43 29.9, 291.88 41.25, 294 63.75 M294 21.25 C293.92 35.61, 293.91 51.62, 294 63.75 M294 63.75 C295.59 78.77, 287.85 83.3, 272.75 85 M294 63.75 C295.57 79.39, 286.48 82.82, 272.75 85 M272.75 85 C202.88 87.1, 130.72 85.83, 21.25 85 M272.75 85 C175.41 86.87, 77.57 86.3, 21.25 85 M21.25 85 C8.79 85, -0.61 79.28, 0 63.75 M21.25 85 C7.17 84.43, -0.79 79.56, 0 63.75 M0 63.75 C1.7 46.22, 1.62 31.99, 0 21.25 M0 63.75 C0.57 51.27, -0.75 39.44, 0 21.25 M0 21.25 C1.47 8.86, 6.33 1.07, 21.25 0 M0 21.25 C-2.18 4.83, 5.08 1.8, 21.25 0" stroke="#1e1e1e" stroke-width="1" fill="none"/></g><g transform="translate(842.4419979519314 187.33329010009766) rotate(0 139.90988159179688 25)"><text x="139.90988159179688" y="0" font-family="Virgil, Segoe UI Emoji" font-size="20px" fill="#1e1e1e" text-anchor="middle" style="white-space: pre;" direction="ltr" dominant-baseline="text-before-edge">create local unix socket for</text><text x="139.90988159179688" y="25" font-family="Virgil, Segoe UI Emoji" font-size="20px" fill="#1e1e1e" text-anchor="middle" style="white-space: pre;" direction="ltr" dominant-baseline="text-before-edge">attach: attach_socket_fd</text></g><g stroke-linecap="round" transform="translate(10 10) rotate(0 659.9258939954971 210.0000152587891)"><path d="M32 0
|
|||
|
|
\ No newline at end of file
|
|||
|
|
--
|
|||
|
|
2.42.0
|
|||
|
|
|