CNB 文档

简体中文

English

简体中文

English

外观

系统级 Runner 接入

复制页面

2130 字约 7 分钟

概述

系统管理员可在 管理后台 添加机器作为 云原生构建/云原生开发 的 Runner，供用户使用。

CNB 支持 Linux、Mac 和 Windows 构建机。

两种托管模式

具体使用方式见管理端配置和Runner 配置。

系统托管（默认）

由 CNB 平台调度管理的机器，需与平台双向互通，通过 ip:port 方式添加。

自托管

由用户自行管理的机器，仅需能访问 CNB 平台即可，通过唯一标识方式添加。

接入主体

  • 系统管理员：在 管理端 / 系统管理 / 云原生构建 / Runner 管理 中接入，作为全平台共享的 Runner 资源。
  • 根组织管理员（可选）：当系统管理员开启「允许添加自定义 Runner」开关后， 根组织管理员可在 组织设置 / 构建节点 自助接入 Mac/Windows 构建机，仅供本组织使用。 详见下文 根组织自定义 Runner。

系统环境要求

Linux 构建机

  • TencentOS 3.2 / TencentOS 3.1 / CentOS 8 及衍生版
  • 内核版本推荐 5.4+
  • Docker >= 26.1.5
  • 8087、8088 端口未被占用，且允许本机监听

Mac 构建机

  • Git >= 2.43.0
  • 8087、8088 端口未被占用，且允许本机监听

由于 Mac 不支持 OverlayFS，秒级克隆会降级为普通克隆。

Windows 构建机

  • 建议 Windows 10 及以上
  • Git >= 2.43.0
  • 8087、8088 端口未被占用，且允许本机监听

由于 Windows 不支持 OverlayFS，秒级克隆会降级为普通克隆。

配置过程

1. 配置 Linux 构建机 （标准方案）

初始化 RUNNER 脚本

a-sysctl.sh

b-raid0.sh

c-daemon-json.sh

d-install-docker.sh

e-install-runner-config.sh

f-install-runner.sh

g-install-binfmt.sh

h-install-docker-plugin.sh

i-install-clean-cgroup.sh

a-sysctl.sh
#!/bin/bash
# =============================================================================
# Script:     a-sysctl.sh
# Description: 安装依赖并配置内核参数（转发、inotify、IPv6 等）
#              Install dependencies and configure kernel parameters (forwarding,
#              inotify, IPv6, etc.)
# Note:       自动适配 dnf / yum / apt-get 包管理器
#             Automatically adapts to dnf / yum / apt-get package managers
#             配置方式为幂等：已存在的参数会更新值，不存在的会追加
#             Idempotent configuration: existing parameters are updated, new
#             ones are appended
# =============================================================================

set -eo pipefail

# Install dependencies based on package manager / 根据包管理器安装依赖
if command -v dnf &>/dev/null; then
    dnf -y install lvm2 libcgroup-tools
elif command -v yum &>/dev/null; then
    yum -y install lvm2 libcgroup-tools
elif command -v apt-get &>/dev/null; then
    apt-get -y install lvm2 cgroup-tools
else
    echo "Unsupported package manager / 不支持的包管理器" >&2
    exit 1
fi

# -----------------------------------------------------------------------------
# Configure sysctl parameter: update if exists, append if not
# 配置 sysctl 参数：已存在则更新值，不存在则追加
# Usage / 用法: configure_sysctl "parameter" "value"
# -----------------------------------------------------------------------------
configure_sysctl() {
    local param="$1"
    local value="$2"
    local conf_file="/etc/sysctl.conf"

    if grep -q "^${param}\s*=" "$conf_file" 2>/dev/null; then
        sed -i "s|^${param}\s*=.*|${param} = ${value}|" "$conf_file"
    else
        echo "${param} = ${value}" >> "$conf_file"
    fi
}

# -----------------------------------------------------------------------------
# Configure kernel parameters / 配置内核参数
# -----------------------------------------------------------------------------
# Container runtime optimizations / 容器运行时优化
configure_sysctl "net.ipv4.ip_forward" "1"
configure_sysctl "fs.inotify.max_user_instances" "10000"
configure_sysctl "fs.inotify.max_user_watches" "1600000"
configure_sysctl "net.core.netdev_max_backlog" "65536"
configure_sysctl "kernel.keys.maxkeys" "20000"
configure_sysctl "kernel.keys.maxbytes" "500000"

# Apply configuration / 应用配置
sysctl -p
b-raid0.sh
#!/bin/bash
# =============================================================================
# Script:     b-raid0.sh
# Description: 检测数据盘并配置挂载：单盘直接格式化挂载，多盘创建 RAID0
#              Detect data disks and configure mounting: single disk is directly
#              formatted and mounted, multiple disks create RAID0
# Note:       系统盘通过 df / 自动识别并排除
#             System disk is automatically identified and excluded via df /
#             数据盘统一挂载到 /data，使用 XFS 文件系统（ftype=1）
#             Data disks are uniformly mounted to /data using XFS filesystem
#             (ftype=1)
#             多盘时自动处理已有的 /dev/md0 或 /dev/md127
#             Automatically handles existing /dev/md0 or /dev/md127 for
#             multiple disks
#             需要提前安装 mdadm 工具
#             Requires mdadm to be installed beforehand
# =============================================================================

set -eo pipefail

# -----------------------------------------------------------------------------
# Constants / 常量
# -----------------------------------------------------------------------------
MOUNT_POINT="/data"
RAID_DEVICE="/dev/md0"
FS_TYPE="xfs"
MOUNT_OPTS="defaults,pquota"

# 检测系统盘并排除，获取数据盘列表 / Detect system disk and exclude, get data disk list
SYSTEM_DISK=$(echo "/dev/$(lsblk -no pkname $(findmnt -no SOURCE /))")
DATA_DISKS=$(lsblk -d -o NAME,TYPE | awk '/disk/ {print "/dev/"$1}' | grep -v "$SYSTEM_DISK" | grep -v "/dev/md")
DATA_DISK_COUNT=$(echo "$DATA_DISKS" | grep -c . || true)

# -----------------------------------------------------------------------------
# format_and_mount: 格式化设备并挂载到 /data，同时更新 fstab
#   Format device and mount to /data, also update fstab
#   $1 - 设备路径，如 /dev/sdb 或 /dev/md0 / Device path, e.g. /dev/sdb or /dev/md0
# -----------------------------------------------------------------------------
format_and_mount() {
    local device="$1"

    if ! mountpoint -q "$MOUNT_POINT"; then
        echo "mount $device to $MOUNT_POINT"
        mkfs.xfs -f -n ftype=1 "$device"
        mkdir -p "$MOUNT_POINT"
        mount -o "$MOUNT_OPTS" "$device" "$MOUNT_POINT"
    else
        echo "$MOUNT_POINT is already mounted"
    fi

    # 更新 fstab：移除该设备旧的 UUID 记录，追加新记录
    # Update fstab: remove old UUID records for the device, append new record
    local uuid
    uuid=$(blkid -s UUID -o value "$device")
    grep -v "$uuid" /etc/fstab > /etc/fstab.tmp || true
    echo "UUID=$uuid  $MOUNT_POINT  $FS_TYPE  $MOUNT_OPTS  0  0" >> /etc/fstab.tmp
    mv /etc/fstab.tmp /etc/fstab
}

# -----------------------------------------------------------------------------
# 单盘模式：直接格式化数据盘并挂载 / Single disk mode: directly format and mount
# -----------------------------------------------------------------------------
if [[ "$DATA_DISK_COUNT" -eq 1 ]]; then
    echo "数据盘数量等于 1 / Data disk count is 1"
    format_and_mount "$DATA_DISKS"
    df -Th

# -----------------------------------------------------------------------------
# RAID0 (multiple disks) / RAID0（多盘模式）
# -----------------------------------------------------------------------------
elif [[ "$DATA_DISK_COUNT" -gt 1 ]]; then
    echo "数据盘数量大于 1 / Data disk count is greater than 1"

    if fdisk -l 2>/dev/null | grep -q '/dev/md0'; then
        echo "/dev/md0 已存在 / /dev/md0 already exists"

    elif fdisk -l 2>/dev/null | grep -q '/dev/md127'; then
        echo "发现 /dev/md127，迁移为 /dev/md0 / Found /dev/md127, migrating to /dev/md0"
        mdadm --stop /dev/md127
        mdadm --assemble "$RAID_DEVICE" --name=0 --update=name $DATA_DISKS
        mdadm --detail --scan > /etc/mdadm.conf
    else
        echo "/dev/md0 不存在，进行初始化 / /dev/md0 does not exist, initializing"
        yes | mdadm -C -a yes "$RAID_DEVICE" -l 0 -n "$DATA_DISK_COUNT" $DATA_DISKS
        mdadm -D "$RAID_DEVICE"
        mkfs.xfs -f -n ftype=1 "$RAID_DEVICE"
        mdadm --detail --scan > /etc/mdadm.conf
    fi

    mkdir -p "$MOUNT_POINT"
    format_and_mount "$RAID_DEVICE"
    cat /etc/fstab
    df -Th

# -----------------------------------------------------------------------------
# No data disks / 无数据盘
# -----------------------------------------------------------------------------
else
    echo "数据盘小于1，不进行任何磁盘操作 / Data disk count less than 1, no disk operations performed"
fi
c-daemon-json.sh
#!/bin/bash
# =============================================================================
# Script:     c-daemon-json.sh
# Description: 配置 Docker daemon.json（数据目录、镜像加速、buildkit 等）
#              Configure Docker daemon.json (data directory, registry mirror,
#              buildkit, etc.)
# Note:       Docker 数据目录设置为 /data/docker，需要先完成磁盘挂载
#             Docker data directory is set to /data/docker, disk mounting must
#             be completed first
#             镜像加速使用腾讯云内网地址，仅适用于腾讯云 CVM
#             Registry mirror uses Tencent Cloud internal address, only
#             applicable to Tencent Cloud CVM
# =============================================================================

set -eo pipefail
mkdir -p /etc/docker


cat>/etc/docker/daemon.json<<EOF
{
  "default-address-pools": [
    {
      "base": "172.17.0.0/12",
      "size": 24
    }
  ],
  "data-root": "/data/docker",
  "registry-mirrors": ["https://mirror.ccs.tencentyun.com"],
  "features": {
    "buildkit": true
  }
}
EOF



#cat> /etc/docker/daemon.json <<EOF
#{
#  "debug": true,
#  "data-root": "/data/docker",
#  "registry-mirrors": ["https://mirror.ccs.tencentyun.com"],
#  "features": {
#    "buildkit": true
#  },
#  "metrics-addr": "$(ip route get 1 | awk '{print $7; exit}'):9323",
#  "icc": false,
#  "bip": "172.17.0.1/16"
#}
#EOF


cat /etc/docker/daemon.json
d-install-docker.sh
#!/bin/bash
# =============================================================================
# Script:     d-install-docker.sh
# Description: 下载并安装 Docker 二进制文件，配置 systemd 服务（docker、containerd）
#              Download and install Docker binaries, configure systemd services
#              (docker, containerd)
# Note:       自动从腾讯镜像下载 Docker tgz 包，无需预先下载到 files/ 目录
#             Automatically downloads Docker tgz from Tencent mirror, no need to
#             pre-download to files/ directory
#             Docker 数据目录为 /data/docker，需要先完成磁盘挂载
#             Docker data directory is /data/docker, disk mounting must be
#             completed first
#             安装完成后会自动启用并启动 docker、docker.socket、containerd
#             After installation, docker, docker.socket, and containerd are
#             automatically enabled and started
# =============================================================================

set -eo pipefail

# -----------------------------------------------------------------------------
# Constants / 常量
# -----------------------------------------------------------------------------
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
TARGET_DOCKER_VERSION="27.5.1"
SERVICE_DIR="/usr/lib/systemd/system"

ARCH="$(arch)"

# 下载目录 / Download directory
DOWNLOAD_DIR="${SCRIPT_DIR}/files"
mkdir -p "$DOWNLOAD_DIR"

# Docker 二进制包下载地址 / Docker binary package download URL
DOCKER_TGZ_URL="https://mirrors.cloud.tencent.com/docker-ce/linux/static/stable/${ARCH}/docker-${TARGET_DOCKER_VERSION}.tgz"
DOCKER_TGZ="${DOWNLOAD_DIR}/docker-${TARGET_DOCKER_VERSION}-${ARCH}.tgz"

# -----------------------------------------------------------------------------
# Download Docker binaries / 下载 Docker 二进制文件
# -----------------------------------------------------------------------------
echo "Downloading Docker ${TARGET_DOCKER_VERSION} for ${ARCH}..."
wget --show-progress -O "${DOCKER_TGZ}" "${DOCKER_TGZ_URL}"

# -----------------------------------------------------------------------------
# Stop existing Docker services / 停止现有 Docker 服务
# -----------------------------------------------------------------------------
systemctl stop docker docker.socket containerd 2>/dev/null || true

# -----------------------------------------------------------------------------
# Install Docker binaries / 安装 Docker 二进制文件
# -----------------------------------------------------------------------------
tar -xvf "${DOCKER_TGZ}" -C /root/
cp /root/docker/* /usr/bin/

# -----------------------------------------------------------------------------
# Write docker.service / 写入 docker.service
# -----------------------------------------------------------------------------
cat > "${SERVICE_DIR}/docker.service" << 'EOF'
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target docker.socket firewalld.service containerd.service time-set.target
Wants=network-online.target containerd.service
Requires=docker.socket

[Service]
Type=notify
EnvironmentFile=-/etc/sysconfig/docker
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock $OPTIONS
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutStartSec=0
RestartSec=2
Restart=always

# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3

# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
TasksMax=infinity

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process
OOMScoreAdjust=-500

[Install]
WantedBy=multi-user.target
EOF

# -----------------------------------------------------------------------------
# Write docker.socket / 写入 docker.socket
# -----------------------------------------------------------------------------
cat > "${SERVICE_DIR}/docker.socket" << 'EOF'
[Unit]
Description=Docker Socket for the API

[Socket]
ListenStream=/run/docker.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker

[Install]
WantedBy=sockets.target
EOF

# -----------------------------------------------------------------------------
# Write containerd.service / 写入 containerd.service
# -----------------------------------------------------------------------------
cat > "${SERVICE_DIR}/containerd.service" << 'EOF'
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target dbus.service

[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/bin/containerd
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
TasksMax=infinity
OOMScoreAdjust=-999

[Install]
WantedBy=multi-user.target
EOF

# -----------------------------------------------------------------------------
# Enable and start services / 启用并启动服务
# -----------------------------------------------------------------------------
systemctl daemon-reload
systemctl enable docker docker.socket containerd
systemctl start docker docker.socket containerd


echo "Docker installed ($(arch)):"
echo "docker ps:"
docker ps
e-install-runner-config.sh
#!/bin/bash
# 证书文件列表 / Certificate file list
CERTS=("orange-ca.pem" "orange-runner.crt" "orange-runner.key")
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
CERTS_DIR="${SCRIPT_DIR}/files"

# 检查并复制证书文件 / Check and copy certificate files
for cert in "${CERTS[@]}"; do
    if [[ ! -f "${CERTS_DIR}/${cert}" ]]; then
        echo "错误: 证书文件不存在 / Error: Certificate file not found ${CERTS_DIR}/${cert}"
        exit 1
    fi
    cp "${CERTS_DIR}/${cert}" /etc/orange-ci/
done

mkdir -p /etc/orange-ci
cat>/etc/orange-ci/runner-config.json<<EOF
{
    "cpus": 64,
    "defaultCpus": 8,
    "memory": 128,
    "useCgroupParent": true,
    "quotaSize": {
        "volume": "512G",
        "container": "256G",
        "workspace": "512G"
    },
    "xfsPath": "/data/orange-ci",
    "maxWorkspacePerGit": 30,
    "volumes": [
        "/var/lib/lxcfs/proc/cpuinfo:/proc/cpuinfo",
        "/var/lib/lxcfs/proc/diskstats:/proc/diskstats",
        "/var/lib/lxcfs/proc/loadavg:/proc/loadavg",
        "/var/lib/lxcfs/proc/meminfo:/proc/meminfo",
        "/var/lib/lxcfs/proc/slabinfo:/proc/slabinfo",
        "/var/lib/lxcfs/proc/stat:/proc/stat",
        "/var/lib/lxcfs/proc/swaps:/proc/swaps",
        "/var/lib/lxcfs/proc/uptime:/proc/uptime",
        "/var/lib/lxcfs/sys/devices/system/cpu:/sys/devices/system/cpu"
    ],
    "dockerArgsForDind": [
        "-v /var/lib/lxcfs:/var/lib/lxcfs"
    ],
    "networkIsolation": true
}
EOF


echo "/etc/orange-ci:"
ls -lh /etc/orange-ci
f-install-runner.sh
#!/bin/bash
# =============================================================================
# Script:     e-install-runner.sh
# Description: 使用 docker-compose 部署 Orange CI Runner 和 lxcfs 容器
#              Deploy Orange CI Runner and lxcfs containers using docker-compose
# Note:       需要 Docker 和 docker-compose 插件已安装
#             Requires Docker and docker-compose plugin to be installed
#             compose 文件生成到 /opt/orange/docker-compose.yml
#             Compose file is generated at /opt/orange/docker-compose.yml
#             可通过环境变量 RUNNER_VERSION 指定 Runner 版本（默认 latest）
#             Runner version can be specified via RUNNER_VERSION env var
#             (default: latest)
# =============================================================================

set -eo pipefail

# -----------------------------------------------------------------------------
# Constants / 常量
# -----------------------------------------------------------------------------
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
COMPOSE_FILE="/opt/orange/docker-compose.yml"
COMPOSE_DIR="$(dirname "$COMPOSE_FILE")"

RUNNER_VERSION="${RUNNER_VERSION:-latest}"

# -----------------------------------------------------------------------------
# Write docker-compose.yml / 写入 docker-compose.yml
# -----------------------------------------------------------------------------
mkdir -p "$COMPOSE_DIR"
cat > "$COMPOSE_FILE" << EOF
services:
  orange-runner:
    image: cnbcool/orange-runner:${RUNNER_VERSION}
    restart: always
    network_mode: host
    privileged: true
    container_name: orange-runner
    volumes:
      - /dev:/dev
      - /data/orange-ci:/data/orange-ci:rshared
      - /etc/orange-ci:/etc/orange-ci
      - /etc/docker:/etc/docker:ro
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
      - /usr/libexec/docker/cli-plugins:/usr/libexec/docker/cli-plugins
      - /var/log/orange-runner:/data/orange-runner/
      - /sys/fs/cgroup:/sys/fs/cgroup
    cgroup: host
    pid: host

  orange-lxcfs:
    image: orangeci/lxcfs:latest
    pull_policy: always
    restart: always
    container_name: orange-lxcfs
    network_mode: none
    privileged: true
    cgroup: host
    pid: host
    volumes:
      - /var/lib/lxcfs:/var/lib/lxcfs:rshared
EOF

# -----------------------------------------------------------------------------
# Pull images and start services / 拉取镜像并启动服务
# -----------------------------------------------------------------------------
docker compose -f "$COMPOSE_FILE" pull
docker compose -f "$COMPOSE_FILE" up -d
g-install-binfmt.sh
#!/bin/bash
# =============================================================================
# Script:     f-install-binfmt.sh
# Description: 安装 binfmt systemd 服务，实现多架构容器构建支持
#              Install binfmt systemd service for multi-architecture container
#              build support
# Note:       依赖 Docker 服务已启动，会在服务启动时自动注册 binfmt
#             Requires Docker service to be running; automatically registers
#             binfmt on service startup
#             包含交叉验证：安装全部架构后反向验证异架构是否正常
#             Includes cross-validation: after installing all architectures,
#             verifies that foreign architectures work correctly
# =============================================================================

set -eo pipefail

# -----------------------------------------------------------------------------
# Write binfmt.service / 写入 binfmt.service
# -----------------------------------------------------------------------------
cat > /usr/lib/systemd/system/binfmt.service << 'EOF'
[Unit]
Description=binfmt
Requires=proc-sys-fs-binfmt_misc.mount docker.service
After=proc-sys-fs-binfmt_misc.mount docker.service

[Service]
Type=oneshot
Restart=on-failure
RestartSec=10s
# 主服务 / Main service
ExecStart=/bin/bash -c 'docker run --privileged --rm tonistiigi/binfmt --install all && docker run --rm --platform $(arch | sed -e 's/x86_64/arm64/' -e 's/aarch64/amd64/') alpine arch'
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable binfmt.service
systemctl start binfmt.service
journalctl --no-pager -n35  -u binfmt
h-install-docker-plugin.sh
#!/bin/bash
# =============================================================================
# Script:     h-install-docker-plugin.sh
# Description: 从 Docker 官方镜像中提取 docker-buildx 和 docker-compose 到 Docker CLI 插件目录
#              Extract docker-buildx and docker-compose from the official Docker
#              image into the Docker CLI plugins directory
# Note:       使用 Docker 镜像中自带的插件文件，无需预先下载
#             Uses plugin files bundled in the Docker image, no need to
#             pre-download
#             插件安装路径: /usr/libexec/docker/cli-plugins/
#             Plugin installation path: /usr/libexec/docker/cli-plugins/
#             自动根据当前架构选择对应的二进制文件
#             Automatically selects the corresponding binary based on the
#             current architecture
# =============================================================================

set -eo pipefail
TARGET_DOCKER_VERSION="27.5.1"

PLUGIN_DST_DIR="/usr/libexec/docker/cli-plugins"
PLUGIN_SRC_DIR="/usr/local/libexec/docker/cli-plugins"
DOCKER_IMAGE="docker:${TARGET_DOCKER_VERSION}-dind"

# -----------------------------------------------------------------------------
# Ensure the plugin directory exists / 确保插件目录存在
# -----------------------------------------------------------------------------
mkdir -p "${PLUGIN_DST_DIR}"

# -----------------------------------------------------------------------------
# Pull the Docker image / 拉取 Docker 镜像
# -----------------------------------------------------------------------------
echo "Pulling Docker image: ${DOCKER_IMAGE}..."
docker pull "${DOCKER_IMAGE}"

# -----------------------------------------------------------------------------
# Create a container from the image and copy plugins
# 从镜像创建容器并复制插件
# -----------------------------------------------------------------------------
CONTAINER_ID=$(docker create "${DOCKER_IMAGE}")

cleanup() {
    echo "Removing temporary container... / 移除临时容器..."
    docker rm "${CONTAINER_ID}" >/dev/null 2>&1
}
trap cleanup EXIT

echo "Copying docker-buildx from image... / 从镜像中复制 docker-buildx..."
docker cp "${CONTAINER_ID}:${PLUGIN_SRC_DIR}/." "${PLUGIN_DST_DIR}/"


# Ensure plugins are executable / 确保插件可执行
chmod +x "${PLUGIN_DST_DIR}/docker-buildx"
chmod +x "${PLUGIN_DST_DIR}/docker-compose"

echo "Docker CLI plugins installed / Docker CLI 插件已安装:"
ls -lh "${PLUGIN_DST_DIR}/"
echo ""
echo "Verification / 验证:"
docker buildx version
docker compose version
i-install-clean-cgroup.sh
#!/bin/bash
# =============================================================================
# Script:     g-install-clean-cgroup.sh
# Description: 安装 cgroup 清理定时任务，定期清理残留 cgroup、orphan 挂载和 ifb 网卡
#              Install cgroup cleanup scheduled task, periodically clean up
#              residual cgroups, orphan mounts, and ifb network interfaces
# Note:       每 10 分钟自动清理一次（通过 systemd timer 实现）
#             Automatically cleans up every 10 minutes (via systemd timer)
#             同时清理 containerd 中已被 Docker 回收但仍残留的容器
#             Also cleans up containers in containerd that have been reclaimed
#             by Docker but still have residual entries
#             同时支持 cgroup v1 和 cgroup v2
#             Supports both cgroup v1 and cgroup v2
# =============================================================================

set -eo pipefail

# -----------------------------------------------------------------------------
# Write clean-cgroup.sh / 写入 clean-cgroup.sh
# -----------------------------------------------------------------------------
cat > /usr/bin/clean-cgroup.sh << 'CLEANCGROUP_SCRIPT'
#!/bin/bash
if grep -q cgroup2 /proc/mounts ; then
  # clean expired cgroup / 清理过期 cgroup
  # links 2 说明只有两个链接  一个是. 一个是父目录中的自身 即没有子目录的..
  # links 2 means only two links: one is '.' and the other is itself in the
  # parent directory, i.e. no subdirectories
  find /sys/fs/cgroup/ -maxdepth 1 -type d -links 2 -mmin +10 -name 'cnb.*.slice' -delete

  # clean orphan slice / 清理孤立 slice
  for slice in $(systemctl list-units 'cnb*' |grep slice |awk '{print $1}'); do
    if [ ! -d "/sys/fs/cgroup/$slice" ]; then
      systemctl stop "$slice"
    fi
  done

  # clean orphan mount / 清理孤立挂载
  mounts=$(mount | grep cnb- | awk '{print $3}')
  for mount in $mounts; do
    slice=$(echo "$mount" | grep -oE 'cnb-\w+-\w+-\w+' | sed 's/-/./g').slice
    if [ ! -d "/sys/fs/cgroup/$slice" ] && [ -d "$mount" ]; then
      umount "$mount"
    fi
  done
else
  find /sys/fs/cgroup/ -maxdepth 2 -type d -links 2 -mmin +10 -name 'cnb*' -delete

  # clean orphan mount / 清理孤立挂载
  mounts=$(mount | grep cnb- | awk '{print $3}')
  for mount in $mounts; do
    pipeline_id=$(echo "$mount" | grep -oE 'cnb-\w+-\w+-\w+')
    if [ ! -d "/sys/fs/cgroup/memory/$pipeline_id" ] && [ -d "$mount" ]; then
      umount "$mount"
    fi
  done
fi

# clean ifb / 清理 ifb 网卡
veths=$(ls /sys/class/net | grep '\-ifb' | sed 's/-ifb//')
for veth in $veths; do
  # 如果源网卡不存在 ifb则删除 / Delete ifb if source network interface does not exist
  if [ ! -e "/sys/class/net/$veth" ]; then
    ip link delete "${veth}-ifb"
  fi
done

ctr_containers=$(ctr -n moby c ls -q)
for container in $ctr_containers; do
  if ! docker container inspect "$container" &> /dev/null; then
    ctr -n moby c rm "$container"
  fi
done
CLEANCGROUP_SCRIPT
chmod +x /usr/bin/clean-cgroup.sh

# -----------------------------------------------------------------------------
# Write clean-cgroup.service / 写入 clean-cgroup.service
# -----------------------------------------------------------------------------
cat > /usr/lib/systemd/system/clean-cgroup.service << 'EOF'
[Unit]
Description=clean cgroup and mount for cnb

[Service]
ExecStart=/usr/bin/clean-cgroup.sh
EOF

# -----------------------------------------------------------------------------
# Write clean-cgroup.timer / 写入 clean-cgroup.timer
# -----------------------------------------------------------------------------
cat > /usr/lib/systemd/system/clean-cgroup.timer << 'EOF'
[Unit]
Description=clean cgroup for cnb every 10min

[Timer]
OnUnitInactiveSec=10m
OnBootSec=10m
OnActiveSec=0

[Install]
WantedBy=multi-user.target
EOF

# -----------------------------------------------------------------------------
# Enable timer / 启用定时器
# -----------------------------------------------------------------------------
systemctl daemon-reload
systemctl enable --now clean-cgroup.timer

2. 配置自托管构建机

2.1 管理后台创建自托管 RUNNER

注

配置的标签以 cnb 开头。

创建自托管 RUNNER
创建自托管 RUNNER
#### ==2.2=={.info} 下载证书放置指定目录 > [!IMPORTANT] > 生产环境请务必执行下列操作，否则会有非法的入侵请求，有严重的安全风险。

① 在管理后台下载证书。

② 将证书压缩包上传到构建机配置目录，并解压。

  • Linux: /etc/orange-ci
  • Mac: $HOME/orange-ci/etc/orange-ci
  • Windows: C:/etc/orange-ci

③ 安装脚本会根据指定目录下是否有 orange-ci.pem、orange-runner.crt、orange-runner.key，来决定是否开启 TLS 校验。

2.2 接入自托管 RUNNER

注

① 构建节点需能访问当前 CNB 站点域名。

② 创建 Runner 后，点击行尾的「连接指引」；首次安装时在弹窗内点击「获取一次性安装码」。

③ Windows 请注意脚本和配置文件编码，避免隐藏后缀或 BOM 头导致启动失败。

Linux
  1. 一键接入脚本
curl -fsSL https://docs.cnbprod.alpha.qq.com/system-runner/scripts/linux.sh \
  | CONNECTOR_ENDPOINT=wss://connector.cnbprod.alpha.qq.com \
    SELF_HOSTED_ID=xxxx-xxxx-xxxx ENROLLMENT_TOKEN=REPLACE_WITH_TOKEN_FROM_CONNECTION_GUIDE bash
  1. 配置和存储目录
默认目录和路径：
① 配置证书目录: `/etc/orange-ci`
② RUNNER 配置文件: `/etc/orange-ci/runner-config.json`
③ 数据存储目录: `/data/orange-ci`

如何修改:
① 修改配置目录: 配置环境变量 `ORANGE_RUNNER_CONFIG_DIR`
② 修改数据目录: 配置环境变量 `ORANGE_RUNNER_DATA_DIR`。
  1. 其他常用命令
# 安装服务
orange-runner install 
# 启动服务
orange-runner start
# 停止服务
orange-runner stop
# 卸载 service
orange-runner uninstall
# 删除全局命令
rm /usr/local/bin/orange-runner
# 查询 service 运行状态
orange-runner status
# 查看日志
tail -f /data/orange-runner/logs/app.log
Mac
  1. 一键接入脚本
curl -fsSL https://docs.cnbprod.alpha.qq.com/system-runner/scripts/mac.sh \
  | sudo env CONNECTOR_ENDPOINT=wss://connector.cnbprod.alpha.qq.com \
    SELF_HOSTED_ID=xxxx-xxxx-xxxx bash
  1. 配置和存储目录
默认目录和路径：
① 配置目录: `$HOME/orange-ci/etc/orange-ci`
② RUNNER 配置文件: `$HOME/orange-ci/etc/orange-ci/runner-config.json`
③ 数据存储目录: `$HOME/orange-ci/data/orange-ci`

如何修改:
① 修改配置目录: 配置环境变量 `ORANGE_RUNNER_CONFIG_DIR`
② 修改数据目录: 配置环境变量 `ORANGE_RUNNER_DATA_DIR`。
  1. 其他常用命令
# 安装服务
orange-runner install 
# 启动服务
orange-runner start
# 停止服务
orange-runner stop
# 卸载 service
orange-runner uninstall
# 删除全局命令
rm /usr/local/bin/orange-runner
# 查询 service 运行状态
orange-runner status
# 查看日志
tail -f $HOME/orange-ci/data/orange-runner/logs/app.log
Windows
  1. 一键接入脚本
Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Confirm:$False -Force
invoke-webrequest https://docs.cnbprod.alpha.qq.com/system-runner/scripts/windows.ps1 \
  -outfile windows.ps1
$env:CONNECTOR_ENDPOINT='wss://connector.cnbprod.alpha.qq.com'
$env:SELF_HOSTED_ID='xxxx-xxxx-xxxx'
$env:ENROLLMENT_TOKEN='REPLACE_WITH_TOKEN_FROM_CONNECTION_GUIDE'
./windows.ps1
  1. 配置和存储目录
默认目录和路径：
① 配置证书目录: `C:/etc/orange-ci`
② RUNNER 配置文件: `C:/etc/orange-ci/runner-config.json`
③ 数据存储目录: `C:/data/orange-ci`

如何修改:
① 修改配置目录: 配置环境变量 `ORANGE_RUNNER_CONFIG_DIR`
② 修改数据目录: 配置环境变量 `ORANGE_RUNNER_DATA_DIR`。
  1. 常用命令
# cd 到 orange-runner.exe 所在目录
cd $HOME\Downloads\orange-runner
# 启动服务
.\orange-runner.exe start
# 停止服务
.\orange-runner.exe stop
# 卸载 service
.\orange-runner.exe uninstall
# 删除 exe 文件
Remove-Item -Path .\orange-runner.exe
# 查询 service 运行状态
.\orange-runner.exe status
# 查看日志
Get-Content 'C:\data\orange-runner\logs\app.log' -Wait -Tail 10

根组织自定义 Runner

除了由系统管理员统一接入构建节点，平台还允许根组织管理员自助接入 Mac/Windows 构建机，作为该组织专属的构建资源。

1. 系统管理员开启开关

进入 管理端 / 系统管理 / 云原生构建 / 自定义 Runner，打开 允许添加自定义 Runner 开关。

开启后，所有根组织的「组织设置」中将显示 构建节点 入口，根组织管理员可自助新增 Runner。

2. 根组织管理员接入构建机

  1. 进入 根组织 / 组织设置 / 构建节点，点击 + 新增 Runner，填写名称、标签等信息。
  2. 创建后节点初始为「离线」状态，点击行尾的 连接指引；在弹窗中点击 「获取一次性安装码」，再复制生成的完整脚本。安装码 1 小时内有效，无需手动替换 SELF_HOSTED_ID 或拼接 token。
  3. 在目标 Mac/Windows 构建机上执行对应平台的脚本，节点状态变为「在线」即接入成功。

如果连接指引提示 secret 已过期或本地 state.json 丢失，点击“重置 Runner 并获取安装码”， 然后使用新命令重新安装；不要复用旧命令或旧 token。

重置后的生效时间

服务端对 Runner 凭证有最多 5 分钟的鉴权缓存。重置、清空机器指纹等操作在缓存过期后完全生效， 原机器在此期间可能仍能连接，新机器也可能短暂被拒。属正常现象，等待后重试即可，无需重复操作。

脚本内容、配置目录与常用命令与上文 接入自托管 RUNNER 一致。

3. 在 .cnb.yml 中调度

参见用户文档 构建节点 - 根组织自定义 Runner。

限制

  • 仅支持 Mac 与 Windows 构建机，Linux 自托管节点仍由系统管理员统一接入。
  • 自定义构建机需要能够访问 CNB 站点 （用于注册节点、上报心跳、拉取流水线任务及上传日志/制品）； 若机器位于内网，请确保已开通对应出网策略。
  • 自定义 Runner 默认不开启 Docker：runner.cpus 配置无效，且无法使用基于镜像的构建环境。
  • 仅限云原生构建使用，不能用于云原生开发（远程开发）任务。
  • 在使用范围限制、证书放置等高级能力方面，与系统级自托管 Runner 保持一致。

常见问题

Linux 构建机

  1. docker buildx 多平台构建 arm 镜像时，出现报错： Error while loading /usr/sbin/dpkg-split: No such file or directory

解决办法：

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes --credential yes
  1. 宿主机 git clone 正常，但流水线容器内 clone 代码时端口超时。

原因：需开启 IPv4 转发，cat /proc/sys/net/ipv4/ip_forward 应为 1。如未开启，执行以下命令编辑：

自托管构建机

  1. 什么情况下适合使用自托管 Runner？
  • 需要 Windows 或 Mac 构建环境
  • CNB 提供的构建机无法访问目标网络，而自托管机器可以
  1. 自托管 Runner 能做什么？不能做什么？
  • 只能在裸机上执行命令
  • 只能运行 CI 任务
  • 不能运行远程开发任务
  1. 能否限制自托管 Runner 被哪些仓库和组织使用，防止机器被滥用？
{
  "allowUseThisRunner": {
    "groups": [],
    "projects": []
  }
}

在 runner-config.json 中添加以下配置，使用全等匹配，不符合预期时可查看 Runner 日志。

© 2026 cnb.cool