阿里云云效工作流设置单元测试覆盖率门禁(go&python)
·
步骤
以go项目为例,python与之对应即可。
-
创建单元测试节点

-
设置脚本与环境变量
在这里可设置用于单元测试覆盖率的运行shell脚本和shell脚本所需的环境变量。

单元测试shell脚本
go单元测试覆盖率脚本(全量)
#!/usr/bin/env bash
set -eo pipefail
mkdir -p golang-report
go test -v -json -cover -coverprofile=cover.out ./test/cover/... > ./golang-report/report.jsonl
# 计算总覆盖率并判断是否 >=80%
if [ -f cover.out ]; then
coverage=$(go tool cover -func=cover.out | awk '/total/ {gsub(/%/,""); print $3}')
echo "Total coverage: ${coverage}%"
if (( $(echo "$coverage < 80" | bc -l) )); then
echo "ERROR: Coverage is below 80%. Pipeline will fail."
exit 1
fi
else
echo "No coverage data generated."
exit 1
fi
云效默认的构建环境(尤其是 Linux 容器环境)中未预装 bc(Basic Calculator)工具,使用下面的脚本代替
#!/usr/bin/env bash
set -eo pipefail
mkdir -p golang-report
go test -v -json -cover -coverprofile=cover.out ./test/cover/... > ./golang-report/report.jsonl
if [ -f cover.out ]; then
coverage=$(go tool cover -func=cover.out | awk '/total/ {gsub(/%/,""); print $3}')
echo "Total coverage: ${coverage}%"
# 使用 awk 进行浮点比较,避免依赖 bc
if awk -v cov="$coverage" 'BEGIN { exit (cov >= 80) ? 0 : 1 }'; then
echo "Coverage meets threshold (>=80%)."
else
echo "ERROR: Coverage is below 80%. Pipeline will fail."
exit 1
fi
else
echo "No coverage data generated."
exit 1
fi
单元测试覆盖率增量脚本
gotest+git+covercheck
原理:go test 生成 cover.out → covercheck 用 git diff 只统计本次新增/修改行的命中率,并与 -min 80 比较 → 脚本再算一遍并打印当前增量覆盖率。
工具:go test/go tool cover(测与全量参考)、git(找变更)、covercheck(增量门禁)、tee(落日志)。
#!/usr/bin/env bash
# =============================================================================
# 云效 Flow:Go 增量覆盖率门禁(工具版,最短)
# 工具:covercheck(只卡变更行覆盖率)
#
# 用法:
# export DIFF_BASE=origin/master
# export COVERAGE_THRESHOLD=80
# bash scripts/yunxiao_coverage_gate_go_tool.sh
#
# 与 yunxiao_coverage_gate.sh(纯 awk 版)互不影响。
# =============================================================================
set -euo pipefail
THRESHOLD="${COVERAGE_THRESHOLD:-80}"
DIFF_BASE="${DIFF_BASE:-origin/feature/dqp-260801-2}"
OUT_DIR="${REPORT_DIR:-./test/report}"
COVER_OUT="${COVER_OUT:-${OUT_DIR}/cover.out}"
# 多个目录之间用空格隔开
COVER_PKGS="${COVER_PKGS:-./test/cover/...}"
TEST_PKGS="${TEST_PKGS:-./test/cover/...}"
mkdir -p "${OUT_DIR}" ./golang-report
echo "[yunxiao] go=$(go version) threshold=${THRESHOLD}% base=${DIFF_BASE}"
# 1) 安装门禁工具(必须装 cmd 子目录,模块根不是 main package)
go install github.com/dr-dobermann/covercheck/cmd/covercheck@latest
export PATH="$(go env GOPATH)/bin:${PATH}"
command -v covercheck >/dev/null 2>&1 || {
echo "ERROR: covercheck 安装失败,未找到可执行文件"
exit 2
}
# 2) 确保基线分支可解析(浅克隆时)
if ! git rev-parse --verify "${DIFF_BASE}" >/dev/null 2>&1; then
BRANCH="${DIFF_BASE#origin/}"
git fetch --no-tags --depth=2000 origin \
"+refs/heads/${BRANCH}:refs/remotes/origin/${BRANCH}" || true
DIFF_BASE="origin/${BRANCH}"
fi
# 3) 跑单测(-json 供云效 UnitTestReport 解析)
# 云效配置:报告类型=Go Test JSON,报告路径=golang-report/report.jsonl
TEST_JSONL=./golang-report/report.jsonl
IFS=',' read -r -a TEST_DIRS <<< "${TEST_PKGS}"
IFS=',' read -r -a COVER_DIRS <<< "${COVER_PKGS}"
echo "单元测试目录: ${TEST_DIRS[@]}"
echo "业务代码目录: ${COVER_DIRS[@]}"
set +e
# shellcheck disable=SC2086
go test -json -count=1 -timeout=15m -covermode=atomic \
-coverpkg="${COVER_DIRS[@]}" \
-coverprofile="${COVER_OUT}" "${TEST_DIRS[@]}" \
> "${TEST_JSONL}" 2>./golang-report/go_test.err.log
TEST_EXIT=$?
set -e
if [[ -s ./golang-report/go_test.err.log ]]; then
echo "ERROR: go test 失败 (exit=${TEST_EXIT}),详情见 ./golang-report/go_test.err.log,如下:"
cat ./golang-report/go_test.err.log
echo "ERROR: go test 失败 (exit=${TEST_EXIT}),详情见 ./golang-report/go_test.err.log, 以上"
fi
if [[ "${TEST_EXIT}" -ne 0 ]]; then
echo "ERROR: go test 失败 (exit=${TEST_EXIT}),部署已阻断。"
echo " 详情见 ${TEST_JSONL}"
exit "${TEST_EXIT}"
fi
echo "[yunxiao] 单元测试 JSON 报告: ${TEST_JSONL} (云效 UnitTestReport 填这个路径)"
echo "[yunxiao] repo total (info): $(go tool cover -func="${COVER_OUT}" | awk '/^total:/{print $3}')"
# 4) 增量覆盖率门禁(日志与单测报告分开,避免覆盖 jsonl)
# - covercheck 低于 -min 时 exit 1,阻断后续部署
REPORT_LOG=./golang-report/diff_coverage_report.log
set +e
covercheck -min "${THRESHOLD}" -base "${DIFF_BASE}" -profiles "${COVER_OUT}" \
-exclude-paths '_test\.go$,(^|/)vendor/' \
2>&1 | tee "${REPORT_LOG}"
GATE_EXIT=${PIPESTATUS[0]}
set -e
# 计算并输出当前增量覆盖率(变更可执行行)
MODULE="$(awk '/^module /{print $2; exit}' go.mod)"
DIFF_TMP="${OUT_DIR}/_patch.diff"
CHANGED_TMP="${OUT_DIR}/_changed_lines.txt"
git diff -U0 "${DIFF_BASE}...HEAD" -- "*.go" > "${DIFF_TMP}" 2>/dev/null \
|| git diff -U0 "${DIFF_BASE}" HEAD -- "*.go" > "${DIFF_TMP}" 2>/dev/null \
|| true
awk '
/^diff --git / {
file=$4; sub(/^b\//,"",file); skip=0
if (file ~ /_test\.go$/) skip=1
if (file ~ /(^|\/)vendor\//) skip=1
next
}
skip { next }
/^@@ / {
if (match($0, /\+[0-9]+(,[0-9]+)?/)) {
chunk=substr($0,RSTART+1,RLENGTH-1)
split(chunk,a,/,/)
start=a[1]+0
count=(a[2]==""?1:a[2]+0)
if (count<=0) next
for (n=0;n<count;n++) print file "\t" (start+n)
}
}
' "${DIFF_TMP}" > "${CHANGED_TMP}"
PATCH_COV="$(
awk -v module="${MODULE}" '
BEGIN { FS="[\t ]+" }
FNR==NR {
if (NF<2) next
f=$1; gsub(/\\/,"/",f); sub(/^\.\//,"",f)
changed[f,$2+0]=1
next
}
/^mode:/ { next }
{
loc=$1; cpos=0
for (i=length(loc);i>0;i--) if (substr(loc,i,1)==":"){cpos=i;break}
if (cpos==0) next
cfile=substr(loc,1,cpos-1)
gsub(/\\/,"/",cfile)
pref=module "/"
if (index(cfile,pref)==1) cfile=substr(cfile,length(pref)+1)
range=substr(loc,cpos+1); cnt=$3+0
split(range,r,/[,.]/); sl=r[1]+0; el=r[3]+0
for (l=sl;l<=el;l++) {
if (!((cfile,l) in changed)) continue
key=cfile SUBSEP l
coverable[key]=1
if (cnt>0) covered[key]=1
}
}
END {
total=0; hit=0
for (k in coverable){ total++; if (k in covered) hit++ }
if (total==0) print "N/A"
else printf "%.2f", hit*100.0/total
}
' "${CHANGED_TMP}" "${COVER_OUT}"
)"
echo "[yunxiao] 当前增量单元测试覆盖率: ${PATCH_COV}% (要求 >= ${THRESHOLD}%)"
if [[ "${GATE_EXIT}" -ne 0 ]]; then
echo "ERROR: 当前增量代码单元测试覆盖率(${PATCH_COV}%)低于要求的 ${THRESHOLD}%,部署已阻断。"
echo " 详情见 ${REPORT_LOG}"
exit "${GATE_EXIT}"
fi
echo "OK: 当前增量覆盖率 ${PATCH_COV}% >= ${THRESHOLD}%,允许继续部署。"
说明
- 在进行增量单元测试时必须找一个基准分支做对比,如shell脚本中的DIFF_BASE,在工作流中可以以环境变量形式设置其值。
- 必须指定需要测试单元测试模块的路径(包含业务代码和测试代码),如:脚本中的TEST_PKGS,在工作流中可以以环境变量形式设置其值。
- 必须指定需要测试业务模块的路径(包含业务代码和测试代码),如:脚本中的COVER_PKGS,在工作流中可以以环境变量形式设置其值。
- 必须指定需要达到的单元测试代码覆盖率的标准,如:脚本中的THRESHOLD,在工作流中可以以环境变量形式设置其值。
Python
原理与go的类似,只是使用的单元测试工具因语言不同而已。
整体流程:环境准备 → pip 安装依赖 → 处理 git 浅克隆 → 执行 pytest 单元测试生成覆盖率 xml → diff‑cover 做增量覆盖率校验 → awk 提取覆盖率数值 → 判断阈值阻断 / 放行流水线

#!/usr/bin/env bash
# =============================================================================
# 云效 Flow:Python 增量覆盖率门禁(工具版)
# 工具:pytest-cov + diff-cover(只卡变更行覆盖率 >= 阈值)
#
# 对齐 yunxiao_coverage_gate_go_tool.sh:
# - 安装工具 → 跑单测 → 增量门禁 → 打印当前增量覆盖率 → 不达标阻断
#
# 用法:
# export DIFF_BASE=origin/master
# export COVERAGE_THRESHOLD=80
# export COV_SOURCE=src
# export TEST_PATHS=tests
# bash scripts/yunxiao_coverage_gate_python_tool.sh
#
# 与 yunxiao_coverage_gate_python.sh(纯 awk 版)互不影响。
# =============================================================================
set -euo pipefail
THRESHOLD="${COVERAGE_THRESHOLD:-80}"
DIFF_BASE="${DIFF_BASE:-origin/feature/dqp-work-test}"
OUT_DIR="${REPORT_DIR:-./test/report}"
COVER_XML="${COVER_XML:-${OUT_DIR}/coverage.xml}"
# ./python-report/diff_coverage_report.log
REPORT_LOG="${REPORT_LOG:-${OUT_DIR}/diff_coverage_report.log}"
# 多个目录之间用,隔开
COV_SOURCE="${COV_SOURCE:-./test/cover}"
# tests
TEST_PATHS="${TEST_PATHS:-./test/cover}"
PYTEST_ARGS="${PYTEST_ARGS:--q}"
if [[ -z "${PYTHON_BIN:-}" ]]; then
if command -v python3 >/dev/null 2>&1; then
PYTHON_BIN=python3
else
PYTHON_BIN=python
fi
fi
mkdir -p "${OUT_DIR}" ./python-report
IFS=',' read -r -a TEST_DIRS <<< "${TEST_PATHS}"
echo "[yunxiao] python=$(${PYTHON_BIN} --version 2>&1) threshold=${THRESHOLD}% base=${DIFF_BASE}"
echo "[yunxiao] cov_source=${COV_SOURCE} test_paths=${TEST_DIRS[@]}"
# ---------- 1) 安装门禁工具 ----------
# 说明:不要用 pip -q。静默安装时云效日志会长时间无输出,看起来像“卡住”。
# 云效环境访问 PyPI 可能较慢,默认走阿里云镜像(可用 PIP_INDEX_URL 覆盖)。
PIP_INDEX_URL="${PIP_INDEX_URL:-https://mirrors.aliyun.com/pypi/simple/}"
PIP_TRUSTED_HOST="${PIP_TRUSTED_HOST:-mirrors.aliyun.com}"
echo "[yunxiao] pip installing pytest/pytest-cov/coverage/diff-cover/pytest-html ..."
echo "[yunxiao] pip index: ${PIP_INDEX_URL}"
${PYTHON_BIN} -m pip install \
--disable-pip-version-check \
-i "${PIP_INDEX_URL}" \
--trusted-host "${PIP_TRUSTED_HOST}" \
pytest pytest-cov coverage diff-cover pytest-html
echo "[yunxiao] pip install done."
if ! ${PYTHON_BIN} -c 'import diff_cover' >/dev/null 2>&1; then
echo "ERROR: diff-cover 安装失败"
exit 2
fi
# 使用 python -m 调用,避免云效 PATH 找不到 scripts 目录
DIFF_COVER=(${PYTHON_BIN} -m diff_cover.diff_cover_tool)
# ---------- 2) 确保基线分支可解析(浅克隆时) ----------
if ! git rev-parse --verify "${DIFF_BASE}" >/dev/null 2>&1; then
BRANCH="${DIFF_BASE#origin/}"
git fetch --no-tags --depth=200 origin \
"+refs/heads/${BRANCH}:refs/remotes/origin/${BRANCH}" || true
DIFF_BASE="origin/${BRANCH}"
fi
echo "[yunxiao] using DIFF_BASE=${DIFF_BASE}"
# ---------- 3) 组装 --cov ----------
COV_ARGS=()
IFS=',' read -r -a _ARR <<< "${COV_SOURCE}"
for s in "${_ARR[@]}"; do
s="$(echo "${s}" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')"
[[ -n "${s}" ]] && COV_ARGS+=(--cov="${s}")
done
[[ ${#COV_ARGS[@]} -eq 0 ]] && COV_ARGS=(--cov=.)
# ---------- 4) 跑单元测试 ----------
# - coverage html → ${OUT_DIR}/htmlcov/index.html (覆盖率浏览,不是云效 UnitTestReport)
# - pytest-html → ${OUT_DIR}/pytest_report.html (云效 UnitTestReport / Python-Pytest 用这个)
PYTEST_HTML="${OUT_DIR}/pytest_report.html"
set +e
# shellcheck disable=SC2086
${PYTHON_BIN} -m pytest "${TEST_DIRS[@]}" \
"${COV_ARGS[@]}" \
--cov-report="term" \
--cov-report="xml:${COVER_XML}" \
--cov-report="html:${OUT_DIR}/htmlcov" \
--html="${PYTEST_HTML}" \
--self-contained-html \
${PYTEST_ARGS} \
2>&1 | tee ./python-report/pytest.log
TEST_EXIT=${PIPESTATUS[0]}
set -e
if [[ "${TEST_EXIT}" -ne 0 ]]; then
echo "ERROR: pytest 失败 (exit=${TEST_EXIT}),部署已阻断。"
exit "${TEST_EXIT}"
fi
if [[ ! -f "${COVER_XML}" ]]; then
echo "ERROR: 未生成覆盖率文件 ${COVER_XML}"
exit 2
fi
echo "[yunxiao] 云效 UnitTestReport(reportPath) 请填: ${PYTEST_HTML}"
echo "[yunxiao] 覆盖率 HTML(仅浏览): ${OUT_DIR}/htmlcov/index.html"
# ---------- 5) diff-cover:只卡本次变更行覆盖率 ----------
# --fail-under:低于阈值 exit 1
# --compare-branch:相对基线做 diff
# 注意:--exclude 后跟多个模式时不要写 --exclude=xxx,否则只有第一个生效,其余会被当成非法参数
set +e
"${DIFF_COVER[@]}" "${COVER_XML}" \
--compare-branch="${DIFF_BASE}" \
--fail-under="${THRESHOLD}" \
--exclude "*/tests/*" "*/test_*.py" "*_test.py" "*/conftest.py" \
2>&1 | tee "${REPORT_LOG}"
GATE_EXIT=${PIPESTATUS[0]}
set -e
# ---------- 6) 解析并输出当前增量覆盖率 ----------
# diff-cover 新旧格式兼容:
# 新:Coverage: 100%(Total: 58 lines 是行数,不是百分比)
# 旧:Total: 85% / Diff Coverage: 85%
PATCH_COV="$(
awk '
{
# 优先匹配新版汇总行 Coverage: 100%
if (match($0, /^[[:space:]]*Coverage:[[:space:]]*[0-9]+(\.[0-9]+)?%/)) {
s = substr($0, RSTART, RLENGTH)
if (match(s, /[0-9]+(\.[0-9]+)?/)) last = substr(s, RSTART, RLENGTH)
# 兼容旧版 Total: 85% / Diff Coverage: 85%(排除 Total: 58 lines)
} else if (match($0, /(Total|Diff Coverage)[: ]+[0-9]+(\.[0-9]+)?%/)) {
s = substr($0, RSTART, RLENGTH)
if (match(s, /[0-9]+(\.[0-9]+)?/)) last = substr(s, RSTART, RLENGTH)
} else if (match($0, /[0-9]+(\.[0-9]+)?%.*covered/)) {
s = substr($0, RSTART, RLENGTH)
if (match(s, /[0-9]+(\.[0-9]+)?/)) last = substr(s, RSTART, RLENGTH)
}
}
END { if (last != "") print last; else print "N/A" }
' "${REPORT_LOG}"
)"
echo "[yunxiao] 当前增量单元测试覆盖率: ${PATCH_COV}% (要求 >= ${THRESHOLD}%)"
if [[ "${GATE_EXIT}" -ne 0 ]]; then
echo "ERROR: 当前增量代码单元测试覆盖率(${PATCH_COV}%)低于要求的 ${THRESHOLD}%,部署已阻断。"
echo " 详情见 ${REPORT_LOG}"
exit "${GATE_EXIT}"
fi
echo "OK: 当前增量覆盖率 ${PATCH_COV}% >= ${THRESHOLD}%,允许继续部署。"
说明
可通过环境变量传入:
DIFF_BASE:对比的基线分支,比如origin/master
COVERAGE_THRESHOLD:增量覆盖率阈值,默认 80
COV_SOURCE:要统计覆盖率业务源码目录,支持英文逗号,分割多目录
TEST_PATHS:pytest 测试用例目录,支持英文逗号,分割多目录
官网参考
https://help.aliyun.com/zh/yunxiao/user-guide/scenario-example-developing-sonarqube-scanning-steps-with-redlines
更多推荐
所有评论(0)