Make a release one final commit with make release-ready and make release
Last week a release needed a commit for every phone check and CI repair: the release commit came first, then fixes, then "Record" commits, and CI went red after phone checks because it only ran on GitHub. make release-ready runs CI's checks before pushing: in the Test job's Ubuntu 24.04 with its apt packages (.github/local-ci/Containerfile), on HEAD's committed content with the locked packages, through an overlay of this checkout so nothing reaches the working tree. It covers Secret Scan, package URLs, the large-file guard, make test, make visual-test and make package-compat-test, and records a stamp for that content. make release then, with the development Moto connected, refuses uncommitted app files and package overrides that are not clean on the lock, syncs ./update_version.sh output, requires the release-ready stamp for this content, builds the debug APK for the phone's ABI, runs the phone check and makes the one "Release Inner Breeze X.Y.Z" commit with the receipt. After a late fix, running it again moves the unpushed release commit above the fix (commits are replayed with merge-tree; only refs move) instead of adding another. It never pushes or tags.
This commit is contained in:
1 parent
c538bda159
commit
1388441cf5
7 files changed
+650
-11
No files matched your search
@@ -0,0 +1,20 @@
|
||||
# The Test job's machine for `make release-ready`: the runner's Ubuntu, the
|
||||
# packages ci.yml installs from .github/apt/test-packages.txt, and Node 22.
|
||||
# Build context: .github/apt (scripts/release-ready.sh builds and tags it).
|
||||
FROM docker.io/library/ubuntu:24.04
|
||||
|
||||
ARG NODE_VERSION=22.12.0
|
||||
|
||||
COPY test-packages.txt /tmp/test-packages.txt
|
||||
|
||||
# The hosted runner also ships git, Git LFS, curl, cmake, zip, jq, OpenSSL's
|
||||
# headers and the Android SDK (scripts/release-ready.sh mounts this machine's).
|
||||
RUN apt-get update \
|
||||
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
|
||||
git git-lfs python3 curl xz-utils make cmake zip unzip jq file libssl-dev \
|
||||
$(grep -v '^#' /tmp/test-packages.txt | tr '\n' ' ') \
|
||||
&& rm -rf /var/lib/apt/lists/* /tmp/test-packages.txt
|
||||
|
||||
RUN curl -fsSL "https://nodejs.org/dist/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-x64.tar.xz" \
|
||||
| tar -xJ -C /usr/local --strip-components=1 \
|
||||
&& node --version
|
||||
@@ -13,11 +13,14 @@
|
||||
The canonical macros are `APP_VERSION_STRING`, `APP_VERSION_MAJOR`,
|
||||
`APP_VERSION_MINOR`, and `APP_VERSION_PATCH`; do not create aliases.
|
||||
Only the GitHub Actions release workflow creates or pushes release tags.
|
||||
- A new version publishes only with a passing phone check of its exact
|
||||
sources: build the APK from the committed release, run
|
||||
`make phone-check APK=...` with the development Moto connected through the
|
||||
ThinkPad, and commit `releases/phone-checks/VERSION.json`. Changes to app
|
||||
sources after the check require a new one.
|
||||
- A new version publishes only with a passing phone check of its exact app
|
||||
content (`app_tree` over `APP_PATHS` in `scripts/phone-check.py`). Release
|
||||
with `make release-ready` (CI's checks in its Ubuntu container), then
|
||||
`make release` with the development Moto connected: it checks the APK and
|
||||
makes the one "Release Inner Breeze X.Y.Z" commit. After a late fix, run
|
||||
both again; the unpushed release commit moves above the fix, so there are
|
||||
no "Record" commits. Tests, docs and tooling may change after the check,
|
||||
app paths may not. See [docs/cells.md](docs/cells.md#verification).
|
||||
- Packages declare the oldest host interface they run on (`host_api` per app
|
||||
in `apps/versions.json`); the host runs `oldest_host_api` to `host_api`.
|
||||
Raise `host_api` when the host interface changes and `oldest_host_api` only
|
||||
|
||||
@@ -3104,6 +3104,22 @@ phone-check:
|
||||
@test -n "$(APK)" || { echo "Set APK=path/to/inbe.apk"; exit 1; }
|
||||
python3 scripts/phone-check.py "$(APK)" --record
|
||||
|
||||
# The release routine (docs/cells.md, "Verification"): run CI's checks here, then
|
||||
# check on the development Moto and make the one release commit. Never pushes.
|
||||
# make release [APK=path/to/inbe.apk]
|
||||
.PHONY: release-ready release release-tooling-test
|
||||
release-ready:
|
||||
sh scripts/release-ready.sh
|
||||
|
||||
release:
|
||||
python3 scripts/release.py $(if $(APK),--apk "$(APK)",)
|
||||
|
||||
release-tooling-test:
|
||||
python3 tests/release_test.py
|
||||
python3 tests/file_size_test.py
|
||||
|
||||
test: release-tooling-test
|
||||
|
||||
test: package-release-test
|
||||
|
||||
.PHONY: app-preferences-sync-test
|
||||
|
||||
+32
-6
@@ -582,12 +582,38 @@ to native host capabilities still require a wrapper release.
|
||||
|
||||
## Verification
|
||||
|
||||
A new app version is published only with a passing phone check of its
|
||||
exact sources. `make phone-check APK=...` installs that APK on the development
|
||||
Moto, runs the Wim Hof session for a minute while it reads SurfaceFlinger's
|
||||
frame times, checks resuming from the background and records
|
||||
`releases/phone-checks/VERSION.json`; the Release workflow refuses a new tag
|
||||
without it or when app sources changed after it.
|
||||
A new app version is published only with a passing phone check of its exact
|
||||
app content. The check installs the APK on the development Moto, runs the
|
||||
Wim Hof session for a minute while it reads SurfaceFlinger's frame times,
|
||||
checks resuming from the background and records
|
||||
`releases/phone-checks/VERSION.json` with `app_tree`, a hash of every app
|
||||
path (`APP_PATHS` in `scripts/phone-check.py`: sources, build rules, assets,
|
||||
locales, version files and the design files packages are built from, minus
|
||||
the `TOOLING` that only checks or publishes). The Release workflow refuses a
|
||||
new tag without it or when the pushed app content differs. Tests, docs and
|
||||
tooling may change after the check; any app change needs a new one.
|
||||
|
||||
A release is one commit, made last:
|
||||
|
||||
1. Add the numeric `CHANGELOG.md` entry and run `./update_version.sh` while
|
||||
the release takes shape, as before.
|
||||
2. `make release-ready` runs CI's checks on HEAD in the Test job's Ubuntu
|
||||
24.04 container with the locked packages: Secret Scan, package URLs, the
|
||||
large-file guard, `make test`, `make visual-test` and
|
||||
`make package-compat-test`. It does not run the browser add-on smoke
|
||||
tests, shard 0's Daochi sync test or the platform builds. Outputs stay in
|
||||
`build/release-ready/`, so later runs are incremental.
|
||||
3. With the development Moto connected (here or through the ThinkPad),
|
||||
`make release` builds the debug APK for its ABI (or takes `APK=...`),
|
||||
checks it and commits the receipt and any version files as
|
||||
"Release Inner Breeze X.Y.Z". It refuses uncommitted app files, package
|
||||
overrides that are not clean on their `ziran.lock` commit, and content
|
||||
that `make release-ready` did not pass.
|
||||
4. A late fix is an ordinary commit followed by steps 2 and 3 again; the
|
||||
unpushed release commit moves above the fix and takes the new receipt.
|
||||
Pushed history is never rewritten.
|
||||
5. Push master only with the owner's say-so; the workflow tags and
|
||||
publishes.
|
||||
|
||||
Harmony Projects shows this wrapper release, every cell version and its Moto
|
||||
screenshots. Harmony's `make inbe-release-verify` checks the latest published
|
||||
|
||||
Executable
+130
@@ -0,0 +1,130 @@
|
||||
#!/bin/sh
|
||||
# Run CI's checks here before a release, the way CI runs them, so a release
|
||||
# cannot go red after its phone check. `make release` requires a pass for the
|
||||
# exact content it commits.
|
||||
#
|
||||
# The container is the Test job's machine: Ubuntu 24.04 with
|
||||
# .github/apt/test-packages.txt and Node 22 (.github/local-ci/Containerfile).
|
||||
# It sees this checkout through an overlay and resets it to HEAD there, so it
|
||||
# tests the committed content with the locked packages (no ziran.local.toml),
|
||||
# without untracked files or secrets; nothing it writes reaches the checkout.
|
||||
# Build outputs stay in build/release-ready/ for faster later runs.
|
||||
#
|
||||
# Covered: ci.yml Secret Scan (publishers, versions, secrets), Package URLs,
|
||||
# the large-file guard, Test (make test: all three shards, then
|
||||
# make visual-test) and release.yml's make package-compat-test.
|
||||
# Not covered: Build Browser Add-ons (Chrome and Firefox web smoke), shard 0's
|
||||
# Daochi sync server test, and the release platform builds.
|
||||
#
|
||||
# sh scripts/release-ready.sh every covered job; writes the stamp
|
||||
# sh scripts/release-ready.sh TARGET... only these make targets; no stamp
|
||||
set -eu
|
||||
|
||||
root=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
|
||||
cd "$root"
|
||||
|
||||
output=build/release-ready
|
||||
image_tag=$(cat .github/local-ci/Containerfile .github/apt/test-packages.txt | sha256sum | cut -c1-12)
|
||||
image=localhost/inbe-ci:$image_tag
|
||||
commit=$(git rev-parse HEAD)
|
||||
|
||||
content_of() {
|
||||
python3 -c 'import importlib.util, sys
|
||||
spec = importlib.util.spec_from_file_location("phone_check", "scripts/phone-check.py")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
print(module.release_content(module.ROOT, sys.argv[1]))' "$1"
|
||||
}
|
||||
|
||||
# Runs inside the container, from this variable rather than the script file,
|
||||
# because resetting the overlay to HEAD may rewrite this script.
|
||||
inside='set -eu
|
||||
filesystem=$(awk '\''$5 == "/src" { for (i = 7; i <= NF; i++) if ($i == "-") { print $(i + 1); exit } }'\'' /proc/self/mountinfo)
|
||||
if [ "$filesystem" != overlay ]; then
|
||||
echo "release-ready: /src is not an overlay; refusing to reset it" >&2
|
||||
exit 2
|
||||
fi
|
||||
git config --global --add safe.directory /src
|
||||
git lfs install --skip-repo >/dev/null
|
||||
git -c core.hooksPath=/dev/null reset -q --hard "$RELEASE_READY_COMMIT"
|
||||
git clean -ffdxq -e /build
|
||||
failed=""
|
||||
job() {
|
||||
name=$1
|
||||
shift
|
||||
printf "\n==> %s\n" "$name"
|
||||
if ! "$@"; then
|
||||
failed="$failed
|
||||
$name"
|
||||
fi
|
||||
}
|
||||
package_urls() {
|
||||
test ! -f .gitmodules || { echo "Dependencies are Ziran packages, not submodules"; return 1; }
|
||||
bad=$(grep -nE "^[[:space:]]*git[[:space:]]*=" ziran.toml \
|
||||
| grep -vE "^[0-9]+:[[:space:]]*git[[:space:]]*=[[:space:]]*\"https://" || true)
|
||||
test -z "$bad" || { echo "Non-HTTPS package URL(s):$bad"; return 1; }
|
||||
}
|
||||
printf "\n==> Fetch Ziran packages (locked)\n"
|
||||
sh scripts/packages.sh --locked
|
||||
if [ "$#" -gt 0 ]; then
|
||||
job "make $*" make -k "$@"
|
||||
else
|
||||
job "Secret Scan: app publisher" python3 scripts/check-package-publishers.py
|
||||
job "Secret Scan: version consistency" sh -c "python3 scripts/check-version.py && python3 tests/version_test.py"
|
||||
job "Secret Scan: tracked files and history" sh -c "python3 scripts/check-secrets.py --working-tree && python3 scripts/check-secrets.py --history"
|
||||
job "Package URLs are public HTTPS" package_urls
|
||||
job "Files over 1 MB" python3 scripts/check-file-size.py --since origin/master
|
||||
job "Test: make test" make -k test
|
||||
job "Test: make visual-test" make visual-test
|
||||
job "Test: make package-compat-test" make package-compat-test
|
||||
fi
|
||||
if [ -n "$failed" ]; then
|
||||
printf "\nrelease-ready: FAILED:%s\n" "$failed"
|
||||
exit 1
|
||||
fi
|
||||
printf "\nrelease-ready: every covered CI job passed\n"'
|
||||
|
||||
mkdir -p "$output/build" "$output/home"
|
||||
# One run at a time: runs share build/release-ready/.
|
||||
exec 9>"$output/lock"
|
||||
if ! flock -n 9; then
|
||||
echo "release-ready: another run is using $output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! podman image exists "$image"; then
|
||||
echo "release-ready: building $image"
|
||||
podman build -t "$image" -f .github/local-ci/Containerfile .github/apt
|
||||
# Replace the images of older package lists; one still in use stays.
|
||||
podman images --format '{{.Repository}}:{{.Tag}}' localhost/inbe-ci \
|
||||
| grep -v ":$image_tag\$" | xargs -r podman rmi >/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
rm -f "$output/stamp.json"
|
||||
content=$(content_of "$commit")
|
||||
started=$(date -u +%Y-%m-%dT%H:%M:%SZ)
|
||||
echo "release-ready: testing $(git log -1 --format='%h %s' "$commit")"
|
||||
|
||||
# Hosted runners have the Android SDK and NDK; the JNI layout tests use it.
|
||||
android_sdk=${ANDROID_SDK_ROOT:-${ANDROID_HOME:-$HOME/Android/Sdk}}
|
||||
android_mount=""
|
||||
if [ -d "$android_sdk/ndk" ]; then
|
||||
android_mount="-v $android_sdk:/opt/android-sdk:ro -e ANDROID_HOME=/opt/android-sdk -e ANDROID_SDK_ROOT=/opt/android-sdk"
|
||||
fi
|
||||
|
||||
status=0
|
||||
# shellcheck disable=SC2086
|
||||
podman run --rm --pull=never --userns=keep-id \
|
||||
-v "$root:/src:O" \
|
||||
-v "$root/$output/build:/src/build" \
|
||||
-v "$root/$output/home:/home/ubuntu" \
|
||||
$android_mount \
|
||||
-e HOME=/home/ubuntu -e CI=true -e RELEASE_READY_COMMIT="$commit" \
|
||||
-w /src "$image" sh -c "$inside" release-ready "$@" || status=$?
|
||||
|
||||
if [ "$status" -ne 0 ] || [ "$#" -gt 0 ]; then
|
||||
exit "$status"
|
||||
fi
|
||||
printf '{\n "passed": true,\n "commit": "%s",\n "content": "%s",\n "started": "%s",\n "finished": "%s"\n}\n' \
|
||||
"$commit" "$content" "$started" "$(date -u +%Y-%m-%dT%H:%M:%SZ)" > "$output/stamp.json"
|
||||
echo "release-ready: recorded $output/stamp.json for $(git log -1 --format=%h "$commit")"
|
||||
Executable
+315
@@ -0,0 +1,315 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check Inner Breeze on the development Moto and make its one release commit.
|
||||
|
||||
make release build the debug APK for the Moto, check it, commit
|
||||
make release APK=PATH check an APK already built from these sources
|
||||
|
||||
1. The version is the newest numeric CHANGELOG.md entry, not yet tagged.
|
||||
2. App files, CHANGELOG.md, the store notes and the receipts have no
|
||||
uncommitted changes (another session's work must be committed first), and
|
||||
every ziran.local.toml override sits clean on its locked commit, so the
|
||||
phone runs what CI builds.
|
||||
3. An unpushed "Release Inner Breeze X.Y.Z" commit with later commits on top
|
||||
moves to the top (only refs move; the working tree and index stay as is),
|
||||
so the release remains one commit however many late fixes it needed.
|
||||
4. ./update_version.sh syncs the version files into the release commit.
|
||||
5. `make release-ready` must have passed for exactly this content.
|
||||
6. The APK is built (unless given) and checked on the Moto; a passing check
|
||||
is recorded with the app's content identity (scripts/phone-check.py).
|
||||
7. The receipt joins the release commit, and the release gate checks HEAD.
|
||||
|
||||
Nothing is pushed or tagged. Pushing master starts the Release workflow, so it
|
||||
waits for the owner's say-so.
|
||||
"""
|
||||
import argparse
|
||||
import fnmatch
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
READY_STAMP = ROOT / "build/release-ready/stamp.json"
|
||||
SUBJECT = "Release Inner Breeze {version}"
|
||||
# What ./update_version.sh writes; only these may join the release commit.
|
||||
VERSION_FILES = ("droid/app/build.gradle", "src/core/version.h", "src/core/version.zi", "windows/inbe.rc",
|
||||
"packaging/*", "fastlane/metadata/android/en-US/changelogs/*.txt", "apps/*/CHANGELOG.md",
|
||||
"apps/versions.json", "src/cells/versions.zi", "src/cells/host_api.zi")
|
||||
|
||||
|
||||
def load(name, path):
|
||||
spec = importlib.util.spec_from_file_location(name, ROOT / path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
phone_check = load("phone_check", "scripts/phone-check.py")
|
||||
gate = load("check_phone_check", "scripts/check-phone-check.py")
|
||||
|
||||
|
||||
class Refused(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def git(*argv, env=None):
|
||||
return phone_check.git(ROOT, *argv, env=env)
|
||||
|
||||
|
||||
def step(text):
|
||||
print(f"\n==> {text}", flush=True)
|
||||
|
||||
|
||||
def changelog_version():
|
||||
found = re.search(r"^## \[(\d+\.\d+\.\d+)\]", (ROOT / "CHANGELOG.md").read_text(), re.M)
|
||||
if not found:
|
||||
raise Refused("CHANGELOG.md has no numeric release entry")
|
||||
return found[1]
|
||||
|
||||
|
||||
def require_new_version(version):
|
||||
tag = f"refs/tags/v{version}"
|
||||
if subprocess.run(["git", "-C", ROOT, "rev-parse", "-q", "--verify", tag],
|
||||
capture_output=True).returncode == 0:
|
||||
raise Refused(f"v{version} is already tagged; add the next version to CHANGELOG.md")
|
||||
remote = subprocess.run(["git", "-C", ROOT, "ls-remote", "--tags", "origin", tag],
|
||||
capture_output=True, text=True, timeout=60)
|
||||
if remote.returncode == 0 and remote.stdout.strip():
|
||||
raise Refused(f"v{version} is already tagged on GitHub; add the next version to CHANGELOG.md")
|
||||
|
||||
|
||||
def release_inputs_changed():
|
||||
changed = []
|
||||
for _, path in phone_check.status(ROOT):
|
||||
if path.startswith(phone_check.RECEIPTS + "/"):
|
||||
continue
|
||||
if phone_check.is_app_path(path) or path == "CHANGELOG.md" or path.startswith("fastlane/"):
|
||||
changed.append(path)
|
||||
return changed
|
||||
|
||||
|
||||
def overrides():
|
||||
path = ROOT / "ziran.local.toml"
|
||||
if not path.is_file():
|
||||
return {}
|
||||
found = {}
|
||||
section = ""
|
||||
for line in path.read_text().splitlines():
|
||||
line = line.split("#", 1)[0].strip()
|
||||
if line.startswith("["):
|
||||
section = line
|
||||
elif section == "[overrides]" and "=" in line:
|
||||
name, value = (part.strip() for part in line.split("=", 1))
|
||||
found[name.lower()] = (ROOT / value.strip('"')).resolve()
|
||||
return found
|
||||
|
||||
|
||||
def require_locked_overrides():
|
||||
locked = {package["name"].lower(): package["commit"]
|
||||
for package in json.loads((ROOT / "ziran.lock").read_text())["packages"]}
|
||||
problems = []
|
||||
for name, path in overrides().items():
|
||||
if name not in locked:
|
||||
continue
|
||||
head = phone_check.git(path, "rev-parse", "HEAD").strip()
|
||||
if head != locked[name]:
|
||||
problems.append(f"{name} ({path}) is at {head[:10]}, the lock pins {locked[name][:10]}")
|
||||
continue
|
||||
dirty = phone_check.status(path)
|
||||
if dirty:
|
||||
problems.append(f"{name} ({path}) has uncommitted changes: "
|
||||
+ ", ".join(entry for _, entry in dirty[:5]))
|
||||
if problems:
|
||||
raise Refused("the phone would not run what CI builds:\n " + "\n ".join(problems)
|
||||
+ "\nCommit and pin the package (ziran update NAME), or wait for its owner.")
|
||||
|
||||
|
||||
def unpushed_release(version):
|
||||
subject = SUBJECT.format(version=version)
|
||||
log = git("log", "--format=%H%x00%P%x00%s", "origin/master..HEAD")
|
||||
found = []
|
||||
for line in log.splitlines():
|
||||
commit, parents, text = line.split("\0")
|
||||
if text == subject:
|
||||
if len(parents.split()) != 1:
|
||||
raise Refused(f"the release commit {commit[:10]} is a merge")
|
||||
found.append(commit)
|
||||
if len(found) > 1:
|
||||
raise Refused("several unpushed release commits: " + ", ".join(c[:10] for c in found))
|
||||
return found[0] if found else ""
|
||||
|
||||
|
||||
def require_unpushed(commit):
|
||||
remote = subprocess.run(["git", "-C", ROOT, "ls-remote", "origin", "refs/heads/master"],
|
||||
capture_output=True, text=True, timeout=60)
|
||||
if remote.returncode != 0 or not remote.stdout.strip():
|
||||
raise Refused("cannot read GitHub's master to confirm the release commit is unpushed")
|
||||
pushed = remote.stdout.split()[0]
|
||||
known = subprocess.run(["git", "-C", ROOT, "cat-file", "-e", pushed + "^{commit}"],
|
||||
capture_output=True).returncode == 0
|
||||
if not known:
|
||||
raise Refused(f"GitHub's master {pushed[:10]} is not here; fetch it before releasing")
|
||||
if subprocess.run(["git", "-C", ROOT, "merge-base", "--is-ancestor", commit, pushed],
|
||||
capture_output=True).returncode == 0:
|
||||
raise Refused(f"the release commit {commit[:10]} is already pushed")
|
||||
|
||||
|
||||
def replay(commit, parent):
|
||||
"""Commit `commit`'s change again on `parent`, without a working tree."""
|
||||
merged = subprocess.run(["git", "-C", ROOT, "merge-tree", "--write-tree", "--merge-base",
|
||||
commit + "^", parent, commit], capture_output=True, text=True)
|
||||
if merged.returncode != 0:
|
||||
raise Refused(f"{commit[:10]} changes the release commit's files, so the release commit "
|
||||
"cannot move to the top; nothing was changed")
|
||||
tree = merged.stdout.split()[0]
|
||||
name, email, date = git("log", "-1", "--format=%an%x00%ae%x00%ad", "--date=raw", commit).strip().split("\0")
|
||||
message = git("log", "-1", "--format=%B", commit)
|
||||
environment = dict(os.environ, GIT_AUTHOR_NAME=name, GIT_AUTHOR_EMAIL=email, GIT_AUTHOR_DATE=date)
|
||||
return subprocess.run(["git", "-C", ROOT, "commit-tree", tree, "-p", parent, "-F", "-"], input=message,
|
||||
check=True, capture_output=True, text=True, env=environment).stdout.strip()
|
||||
|
||||
|
||||
def move_to_top(release, version):
|
||||
"""Replay the commits after the unpushed release commit onto its parent,
|
||||
then commit the release commit's files again on top. Only refs move; the
|
||||
working tree and the index keep every file as it is."""
|
||||
require_unpushed(release)
|
||||
head = git("rev-parse", "HEAD").strip()
|
||||
files = git("diff-tree", "--no-commit-id", "--name-only", "-r", "--no-renames", release).split()
|
||||
later = git("rev-list", "--reverse", "--parents", f"{release}..{head}").splitlines()
|
||||
parent = release + "^"
|
||||
for line in later:
|
||||
commit, *parents = line.split()
|
||||
if len(parents) != 1:
|
||||
raise Refused(f"{commit[:10]} is a merge; nothing was changed")
|
||||
parent = replay(commit, git("rev-parse", parent).strip())
|
||||
subprocess.run(["git", "-C", ROOT, "update-ref", "-m", "release: move the release commit to the top",
|
||||
"refs/heads/master", parent, head], check=True)
|
||||
commit_files(files, version, amend=False)
|
||||
print(f"Moved the release commit to the top ({len(files)} files).")
|
||||
|
||||
|
||||
def commit_files(files, version, amend, details=""):
|
||||
message = SUBJECT.format(version=version) + ("\n\n" + details if details else "")
|
||||
argv = ["git", "-C", ROOT, "commit", "-q", "--only", "-m", message]
|
||||
if amend:
|
||||
argv.insert(4, "--amend")
|
||||
subprocess.run([*argv, "--", *files], check=True)
|
||||
|
||||
|
||||
def release_at_head(version):
|
||||
subject = git("log", "-1", "--format=%s").strip()
|
||||
return subject == SUBJECT.format(version=version) and unpushed_release(version) != ""
|
||||
|
||||
|
||||
def require_release_ready():
|
||||
try:
|
||||
stamp = json.loads(READY_STAMP.read_text())
|
||||
except (OSError, ValueError):
|
||||
raise Refused("make release-ready has not passed here; run it first") from None
|
||||
content = phone_check.release_content(ROOT, "HEAD")
|
||||
if stamp.get("passed") is not True or stamp.get("content") != content:
|
||||
raise Refused("make release-ready passed for other content "
|
||||
f"({stamp.get('commit', '?')[:10]}); run it again for HEAD")
|
||||
|
||||
|
||||
def build_apk(phone):
|
||||
abi = phone.shell("getprop", "ro.product.cpu.abi").strip()
|
||||
if not re.fullmatch(r"[a-z0-9_-]+", abi):
|
||||
raise Refused(f"the phone reported no usable ABI: {abi!r}")
|
||||
started = time.time()
|
||||
environment = dict(os.environ)
|
||||
environment["ORG_GRADLE_PROJECT_app.onlyAbi"] = abi
|
||||
for name in ("DISPLAY", "WAYLAND_DISPLAY", "XAUTHORITY", "DBUS_SESSION_BUS_ADDRESS"):
|
||||
environment.pop(name, None)
|
||||
subprocess.run(["make", "-C", ROOT, "android-debug"], check=True, env=environment)
|
||||
apk = ROOT / f"droid/app/build/outputs/apk/debug/app-{abi}-debug.apk"
|
||||
if not apk.is_file() or apk.stat().st_mtime < started:
|
||||
raise Refused(f"the build did not produce {apk.relative_to(ROOT)}")
|
||||
return apk
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--apk", type=Path, help="an APK built from the current sources")
|
||||
parser.add_argument("--host", default=os.environ.get("INBE_PHONE_HOST", "auto"))
|
||||
parser.add_argument("--serial", default=os.environ.get("INBE_PHONE_SERIAL", "ZE2223BQZT"))
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
if git("symbolic-ref", "--short", "HEAD").strip() != "master":
|
||||
raise Refused("releases are made on master")
|
||||
version = changelog_version()
|
||||
step(f"Inner Breeze {version}")
|
||||
require_new_version(version)
|
||||
changed = release_inputs_changed()
|
||||
if changed:
|
||||
raise Refused("uncommitted app or release files; commit them or wait for their owner:\n "
|
||||
+ "\n ".join(changed))
|
||||
require_locked_overrides()
|
||||
|
||||
release = unpushed_release(version)
|
||||
if release and release != git("rev-parse", "HEAD").strip():
|
||||
step("Moving the unpushed release commit above the later commits")
|
||||
move_to_top(release, version)
|
||||
|
||||
step("Synchronizing version files (./update_version.sh)")
|
||||
subprocess.run([ROOT / "update_version.sh"], check=True, cwd=ROOT)
|
||||
synced = release_inputs_changed()
|
||||
stray = [path for path in synced
|
||||
if not any(fnmatch.fnmatchcase(path, pattern) for pattern in VERSION_FILES)]
|
||||
if stray:
|
||||
raise Refused("files changed beside the version files (another session?); nothing was "
|
||||
"committed:\n " + "\n ".join(stray))
|
||||
if synced:
|
||||
commit_files(synced, version, amend=release_at_head(version))
|
||||
print("Version files: " + ", ".join(synced))
|
||||
|
||||
step("Requiring make release-ready for this content")
|
||||
require_release_ready()
|
||||
checked_app = phone_check.app_tree(ROOT, "HEAD")
|
||||
if phone_check.app_tree(ROOT) != checked_app:
|
||||
raise Refused("app files changed on disk; commit them or wait for their owner")
|
||||
|
||||
phone = phone_check.Phone(phone_check.resolve_host(args.host, args.serial), args.serial)
|
||||
if phone.adb("get-state").decode().strip() != "device":
|
||||
raise Refused("the development Moto is not connected and authorized")
|
||||
if args.apk:
|
||||
apk = args.apk.resolve(strict=True)
|
||||
else:
|
||||
step("Building the debug APK for the Moto")
|
||||
apk = build_apk(phone)
|
||||
if phone_check.app_tree(ROOT) != checked_app:
|
||||
raise Refused("app files changed during the build; run make release again")
|
||||
|
||||
step(f"Phone check of {apk.name} on the Moto")
|
||||
subprocess.run([sys.executable, ROOT / "scripts/phone-check.py", apk, "--record",
|
||||
"--host", phone.host, "--serial", args.serial], check=True)
|
||||
receipt = gate.validate(ROOT, version)
|
||||
if receipt["app_tree"] != checked_app:
|
||||
raise Refused("the recorded check is for other app content")
|
||||
|
||||
step("Release commit")
|
||||
practice = receipt["practice"]
|
||||
details = (f"Phone check on the {receipt['device']['model']}: {practice['fps']} fps, "
|
||||
f"{practice['late_percent']}% late frames, app content {checked_app[:12]}.")
|
||||
commit_files([f"{phone_check.RECEIPTS}/{version}.json"], version,
|
||||
amend=release_at_head(version), details=details)
|
||||
gate.validate(ROOT, version, "HEAD")
|
||||
print(git("log", "-1", "--format=%h %s").strip())
|
||||
print("Not pushed. Pushing master starts the Release workflow, so push only with the owner's "
|
||||
"say-so: git push origin master")
|
||||
return 0
|
||||
except Refused as error:
|
||||
print(f"release: {error}", file=sys.stderr)
|
||||
return 1
|
||||
except (subprocess.CalledProcessError, ValueError, OSError) as error:
|
||||
print(f"release: stopped: {error}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,129 @@
|
||||
"""make release keeps one unpushed release commit and checks what CI will build."""
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
spec = importlib.util.spec_from_file_location("release", ROOT / "scripts/release.py")
|
||||
release = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(release)
|
||||
|
||||
|
||||
def git(root, *argv):
|
||||
return subprocess.run(["git", "-C", str(root), *argv], check=True, capture_output=True,
|
||||
text=True).stdout
|
||||
|
||||
|
||||
class ReleaseTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
temporary = tempfile.TemporaryDirectory(prefix="inbe-release-")
|
||||
self.addCleanup(temporary.cleanup)
|
||||
base = Path(temporary.name)
|
||||
self.origin = base / "origin.git"
|
||||
self.root = base / "inbe"
|
||||
git(base, "init", "-q", "--bare", "-b", "master", str(self.origin))
|
||||
git(base, "init", "-q", "-b", "master", str(self.root))
|
||||
for key, value in (("user.email", "[email protected]"), ("user.name", "Test")):
|
||||
git(self.root, "config", key, value)
|
||||
git(self.root, "remote", "add", "origin", str(self.origin))
|
||||
self.commit({"src/main.zi": "one\n", "src/core/version.zi": "2.9.9\n", "docs/notes.md": "notes\n"},
|
||||
"Start")
|
||||
git(self.root, "push", "-q", "origin", "master")
|
||||
git(self.root, "fetch", "-q", "origin")
|
||||
self.previous_root = release.ROOT
|
||||
release.ROOT = self.root
|
||||
self.addCleanup(setattr, release, "ROOT", self.previous_root)
|
||||
|
||||
def commit(self, files, message):
|
||||
for name, text in files.items():
|
||||
path = self.root / name
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(text)
|
||||
git(self.root, "add", name)
|
||||
git(self.root, "commit", "-q", "-m", message)
|
||||
return git(self.root, "rev-parse", "HEAD").strip()
|
||||
|
||||
def subjects(self):
|
||||
return git(self.root, "log", "--format=%s").splitlines()
|
||||
|
||||
def release_commit(self, receipt="{}\n"):
|
||||
return self.commit({"src/core/version.zi": "3.0.0\n",
|
||||
"releases/phone-checks/3.0.0.json": receipt}, "Release Inner Breeze 3.0.0")
|
||||
|
||||
def test_a_buried_release_commit_moves_to_the_top(self):
|
||||
old = self.release_commit()
|
||||
self.commit({"src/fix.zi": "late fix\n"}, "Fix a late bug")
|
||||
(self.root / "docs/notes.md").write_text("another session's work\n")
|
||||
tree = git(self.root, "rev-parse", "HEAD^{tree}")
|
||||
self.assertEqual(release.unpushed_release("3.0.0"), old)
|
||||
self.assertFalse(release.release_at_head("3.0.0"))
|
||||
|
||||
release.move_to_top(old, "3.0.0")
|
||||
|
||||
self.assertEqual(self.subjects(), ["Release Inner Breeze 3.0.0", "Fix a late bug", "Start"])
|
||||
self.assertEqual(git(self.root, "rev-parse", "HEAD^{tree}"), tree)
|
||||
self.assertEqual(git(self.root, "show", "--format=", "--name-only", "HEAD").split(),
|
||||
["releases/phone-checks/3.0.0.json", "src/core/version.zi"])
|
||||
self.assertEqual(git(self.root, "show", "HEAD~1:src/core/version.zi"), "2.9.9\n")
|
||||
self.assertEqual(git(self.root, "status", "--porcelain"), " M docs/notes.md\n")
|
||||
self.assertTrue(release.release_at_head("3.0.0"))
|
||||
|
||||
def test_a_pushed_release_commit_never_moves(self):
|
||||
old = self.release_commit()
|
||||
git(self.root, "push", "-q", "origin", "master")
|
||||
self.commit({"src/fix.zi": "late fix\n"}, "Fix a late bug")
|
||||
head = git(self.root, "rev-parse", "HEAD")
|
||||
with self.assertRaisesRegex(release.Refused, "already pushed"):
|
||||
release.move_to_top(old, "3.0.0")
|
||||
self.assertEqual(git(self.root, "rev-parse", "HEAD"), head)
|
||||
|
||||
def test_later_changes_to_its_files_keep_the_history(self):
|
||||
old = self.release_commit()
|
||||
self.commit({"src/core/version.zi": "3.0.1\n"}, "Change the version again")
|
||||
head = git(self.root, "rev-parse", "HEAD")
|
||||
with self.assertRaisesRegex(release.Refused, "cannot move"):
|
||||
release.move_to_top(old, "3.0.0")
|
||||
self.assertEqual(git(self.root, "rev-parse", "HEAD"), head)
|
||||
|
||||
def test_one_release_commit_at_most(self):
|
||||
self.release_commit()
|
||||
self.release_commit('{"passed": true}\n')
|
||||
with self.assertRaisesRegex(release.Refused, "several"):
|
||||
release.unpushed_release("3.0.0")
|
||||
|
||||
def test_the_release_commit_takes_only_version_files(self):
|
||||
files = ["droid/app/build.gradle", "src/core/version.zi", "packaging/snap/snap/snapcraft.yaml",
|
||||
"fastlane/metadata/android/en-US/changelogs/173.txt", "apps/diary/CHANGELOG.md",
|
||||
"apps/versions.json", "src/cells/versions.zi"]
|
||||
stray = ["src/screens/diary_screen.zi", "locales/pt.txt", "apps/diary/diary.zi"]
|
||||
allowed = [path for path in files + stray
|
||||
if any(release.fnmatch.fnmatchcase(path, pattern) for pattern in release.VERSION_FILES)]
|
||||
self.assertEqual(allowed, files)
|
||||
|
||||
def test_package_overrides_must_sit_clean_on_the_lock(self):
|
||||
package = self.root.parent / "kryon"
|
||||
git(self.root.parent, "init", "-q", str(package))
|
||||
for key, value in (("user.email", "[email protected]"), ("user.name", "Test")):
|
||||
git(package, "config", key, value)
|
||||
(package / "ui.zi").write_text("one\n")
|
||||
git(package, "add", "ui.zi")
|
||||
git(package, "commit", "-q", "-m", "one")
|
||||
locked = git(package, "rev-parse", "HEAD").strip()
|
||||
(self.root / "ziran.lock").write_text(json.dumps(
|
||||
{"format": 1, "packages": [{"name": "kryon", "commit": locked}]}))
|
||||
(self.root / "ziran.local.toml").write_text('[overrides]\nKryon = "../kryon"\n')
|
||||
release.require_locked_overrides()
|
||||
|
||||
(package / "ui.zi").write_text("work in progress\n")
|
||||
with self.assertRaisesRegex(release.Refused, "uncommitted changes"):
|
||||
release.require_locked_overrides()
|
||||
git(package, "commit", "-q", "-am", "two")
|
||||
with self.assertRaisesRegex(release.Refused, "the lock pins"):
|
||||
release.require_locked_overrides()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user