Check packages across app versions and release only after a phone check
make package-compat-test runs every package the Waozi node serves in this build, and unpacks the last released Linux app to run each new package that declares an interface it has. A failed host call inside a package fails the check, since the screen would otherwise silently lose that part. The Release test job runs both directions, and the package workflow runs new packages in the last release before publishing them. make phone-check APK=... installs the exact APK on the development Moto, starts the Wim Hof session through the app's start-practice intent, reads SurfaceFlinger's frame times for a minute, checks resuming from the background and records releases/phone-checks/VERSION.json for the commit it checked. The Release workflow creates a new version tag only with a passing receipt whose sources match the release, so a laggy build cannot publish.
This commit is contained in:
1 parent
2b06f0ba72
commit
c2202c4270
9 files changed
+658
-1
No files matched your search
@@ -40,7 +40,9 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: ./.github/actions/ziran-packages
|
||||
- name: Install signing and test dependencies
|
||||
run: sudo apt-get update && sudo apt-get install -y python3-cryptography python3-fonttools
|
||||
run: |
|
||||
sudo apt-get update && sudo apt-get install -y python3-cryptography python3-fonttools
|
||||
.github/scripts/install-apt-packages.sh .github/apt/test-packages.txt
|
||||
- name: Build independent packages and check signatures
|
||||
run: |
|
||||
make package-version-test
|
||||
@@ -49,6 +51,8 @@ jobs:
|
||||
make package-upload-test
|
||||
make cells build-laws
|
||||
python3 tests/package_release_test.py
|
||||
- name: Run new packages in the last released app
|
||||
run: python3 tests/package_compat_test.py --skip-forward
|
||||
- name: Configure publisher
|
||||
id: publisher
|
||||
env:
|
||||
|
||||
@@ -100,6 +100,9 @@ jobs:
|
||||
- name: Require Visual Tests
|
||||
run: make visual-test
|
||||
|
||||
- name: Run Packages Across App Versions
|
||||
run: make package-compat-test
|
||||
|
||||
- name: Save Visual Test Evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
@@ -896,6 +899,16 @@ jobs:
|
||||
done < <(scripts/list-expected-release-assets.sh "$VERSION")
|
||||
exit "$missing"
|
||||
|
||||
- name: Require a Passing Phone Check for a New Version
|
||||
run: |
|
||||
set -euo pipefail
|
||||
VERSION=$(python3 scripts/check-version.py --print-version)
|
||||
if git rev-parse "v$VERSION" >/dev/null 2>&1; then
|
||||
echo "v$VERSION is already tagged; no new version to check"
|
||||
exit 0
|
||||
fi
|
||||
python3 scripts/check-phone-check.py "$VERSION"
|
||||
|
||||
- name: Create and Push Tag If Version Is New
|
||||
id: tag_version
|
||||
env:
|
||||
|
||||
@@ -13,6 +13,16 @@
|
||||
The canonical macros are `APP_VERSION_STRING`, `APP_VERSION_MAJOR`,
|
||||
`APP_VERSION_MINOR`, and `APP_VERSION_PATCH`; do not create aliases.
|
||||
Only the GitHub Actions release workflow creates or pushes release tags.
|
||||
- A new version publishes only with a passing phone check of its exact
|
||||
sources: build the APK from the committed release, run
|
||||
`make phone-check APK=...` with the development Moto connected through the
|
||||
ThinkPad, and commit `releases/phone-checks/VERSION.json`. Changes to app
|
||||
sources after the check require a new one.
|
||||
- Packages declare the oldest host interface they run on (`host_api` per app
|
||||
in `apps/versions.json`); the host runs `oldest_host_api` to `host_api`.
|
||||
Raise `host_api` when the host interface changes and `oldest_host_api` only
|
||||
when it can no longer run older packages. `make package-compat-test` runs
|
||||
the node's packages in this build and new packages in the last release.
|
||||
- Translate changed strings in every affected locale; no English placeholders.
|
||||
- `make locale-translated-test` fails on any locale string that still equals
|
||||
its English text. Translate it; add it to `tests/locale_same_as_english.txt`
|
||||
|
||||
@@ -3093,6 +3093,25 @@ test: app-click-block-zi-test
|
||||
package-release-test: cells $(ZIRAN_BUILD_DIR)/libziran.a
|
||||
python3 tests/package_release_test.py
|
||||
|
||||
# This build runs the packages the node serves, and the last released app runs
|
||||
# each new package that declares a host interface it has.
|
||||
.PHONY: package-compat-test
|
||||
package-compat-test: $(TARGET) $(CELL_BUNDLES) $(INBE_BUNDLE)
|
||||
python3 tests/package_compat_test.py $(abspath $(TARGET))
|
||||
|
||||
.PHONY: phone-check-gate-test
|
||||
phone-check-gate-test:
|
||||
python3 tests/phone_check_gate_test.py
|
||||
|
||||
test: phone-check-gate-test
|
||||
|
||||
# Check an exact APK on the development Moto and record the release receipt:
|
||||
# make phone-check APK=path/to/inbe.apk
|
||||
.PHONY: phone-check
|
||||
phone-check:
|
||||
@test -n "$(APK)" || { echo "Set APK=path/to/inbe.apk"; exit 1; }
|
||||
python3 scripts/phone-check.py "$(APK)" --record
|
||||
|
||||
test: package-release-test
|
||||
|
||||
.PHONY: app-preferences-sync-test
|
||||
|
||||
@@ -187,6 +187,19 @@ Packaged resource and translation changes also trigger only the package workflow
|
||||
`make package-version-test` checks every module's numeric changelog, ledger,
|
||||
sequence and bundled status before publishing.
|
||||
|
||||
Hosts and packages agree on interfaces, not exact versions. Every package
|
||||
declares the oldest host interface it needs (`host_api` in its ledger entry),
|
||||
and the host runs packages from `oldest_host_api` to its own `host_api`.
|
||||
Records between them match fields by name, so a field one side lacks keeps its
|
||||
zero value and an unknown field is ignored. Update checks send the host's
|
||||
interfaces and a stable device bucket; the node answers with the newest
|
||||
release that fits, so apps that keep an older host still receive the packages
|
||||
built for it. `package-release.py --rollout PERCENT --rollout-hours HOURS`
|
||||
stages changed releases to that share of devices first; the node releases them
|
||||
to everyone when the time runs out. `make package-compat-test` runs the
|
||||
packages the node serves in the current build, and runs each new package that
|
||||
declares an older interface in the last released app.
|
||||
|
||||
## Publishing configuration
|
||||
|
||||
The `Cell packages` GitHub Actions workflow builds and verifies package releases
|
||||
@@ -296,6 +309,13 @@ to native host capabilities still require a wrapper release.
|
||||
|
||||
## Verification
|
||||
|
||||
A new app version is published only with a passing phone check of its
|
||||
exact sources. `make phone-check APK=...` installs that APK on the development
|
||||
Moto, runs the Wim Hof session for a minute while it reads SurfaceFlinger's
|
||||
frame times, checks resuming from the background and records
|
||||
`releases/phone-checks/VERSION.json`; the Release workflow refuses a new tag
|
||||
without it or when app sources changed after it.
|
||||
|
||||
Harmony Projects shows this wrapper release, every cell version and its Moto
|
||||
screenshots. Harmony's `make inbe-release-verify` checks the latest published
|
||||
APK on the development Moto and writes private evidence under
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Allow a new release only with a passing phone check of its exact sources.
|
||||
|
||||
scripts/phone-check.py --record writes releases/phone-checks/VERSION.json for
|
||||
the commit it checked. The released commit may differ from that commit only in
|
||||
paths that do not change the app, such as the receipt itself.
|
||||
"""
|
||||
import argparse
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
spec = importlib.util.spec_from_file_location("phone_check", ROOT / "scripts/phone-check.py")
|
||||
phone_check = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(phone_check)
|
||||
|
||||
|
||||
def git(root, *argv):
|
||||
return subprocess.run(["git", "-C", str(root), *argv], check=True, capture_output=True,
|
||||
text=True).stdout
|
||||
|
||||
|
||||
def changes_app(path):
|
||||
return any(path == source or path.startswith(source + "/") for source in phone_check.SOURCE_PATHS)
|
||||
|
||||
|
||||
def validate(root, version):
|
||||
path = root / "releases/phone-checks" / f"{version}.json"
|
||||
if not path.is_file():
|
||||
raise ValueError(f"{version} has no phone check; run scripts/phone-check.py --record on the "
|
||||
"development Moto and commit the receipt")
|
||||
receipt = json.loads(path.read_text())
|
||||
if receipt.get("schema") != 1 or receipt.get("version") != version:
|
||||
raise ValueError("the phone check is for a different version")
|
||||
if receipt.get("apk", {}).get("version_name") != version:
|
||||
raise ValueError("the checked APK has a different version")
|
||||
commit = receipt.get("source_commit", "")
|
||||
if not re.fullmatch(r"[0-9a-f]{40}", commit):
|
||||
raise ValueError("the phone check has no full source commit")
|
||||
timing = receipt.get("practice", {})
|
||||
if receipt.get("passed") is not True or not phone_check.smooth(timing) or timing.get("seconds", 0) < 55:
|
||||
raise ValueError(f"the phone check did not pass: {timing}")
|
||||
if subprocess.run(["git", "-C", str(root), "merge-base", "--is-ancestor", commit, "HEAD"],
|
||||
capture_output=True).returncode != 0:
|
||||
raise ValueError(f"the checked commit {commit} is not part of this release")
|
||||
changed = [name for name in git(root, "diff", "--name-only", commit, "HEAD").splitlines()
|
||||
if changes_app(name)]
|
||||
if changed:
|
||||
raise ValueError("the app changed after its phone check: " + ", ".join(changed[:10]))
|
||||
return receipt
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("version")
|
||||
parser.add_argument("--root", type=Path, default=ROOT)
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
receipt = validate(args.root, args.version)
|
||||
except (OSError, ValueError, subprocess.CalledProcessError) as error:
|
||||
print(f"Phone check required: {error}", file=sys.stderr)
|
||||
return 1
|
||||
timing = receipt["practice"]
|
||||
print(f"{args.version} passed its phone check on {receipt['device']['model']}: {timing['fps']} fps, "
|
||||
f"{timing['late_percent']}% late frames")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,249 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check an exact Inner Breeze APK on the development Moto before release.
|
||||
|
||||
The APK is installed over the existing app, launched, and the Wim Hof session
|
||||
is started through the app's own start-practice intent. SurfaceFlinger's frame
|
||||
timestamps for the app's window are collected for the whole session, then the
|
||||
app is sent to the background and resumed. With --record the result is written
|
||||
to releases/phone-checks/VERSION.json for the release workflow, which publishes
|
||||
a new version only with a passing check of its exact source commit.
|
||||
"""
|
||||
import argparse
|
||||
from datetime import datetime, timezone
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
ACTIVITY = "xyz.waozi.inbe.MainActivity"
|
||||
START_PRACTICE = "xyz.waozi.inbe.action.START_PRACTICE"
|
||||
WIM_HOF = 0
|
||||
# Paths whose changes alter the APK. The release gate allows only other
|
||||
# changes between the checked commit and the released one.
|
||||
SOURCE_PATHS = ("src", "droid", "apps", "assets", "locales", "themes", "fonts",
|
||||
"Makefile", "ziran.toml", "ziran.lock", "CHANGELOG.md", "scripts", "packaging")
|
||||
# The breathing circle animates for the whole measured window, so the app
|
||||
# must keep the display's rate with only occasional late frames.
|
||||
MINIMUM_FPS = 55.0
|
||||
MAXIMUM_LATE_PERCENT = 3.0
|
||||
MAXIMUM_FRAME_MS = 250.0
|
||||
PENDING = 9223372036854775807
|
||||
|
||||
|
||||
def run(argv, *, timeout=60, **kwargs):
|
||||
return subprocess.run(list(map(str, argv)), check=True, capture_output=True,
|
||||
timeout=timeout, **kwargs).stdout
|
||||
|
||||
|
||||
class Phone:
|
||||
def __init__(self, host, serial):
|
||||
if not re.fullmatch(r"[A-Za-z0-9_.@-]+", host) or host.startswith("-"):
|
||||
raise ValueError("invalid ADB host")
|
||||
if not re.fullmatch(r"[A-Za-z0-9]+", serial):
|
||||
raise ValueError("invalid phone serial")
|
||||
self.host = host
|
||||
self.serial = serial
|
||||
|
||||
def remote(self, *argv, timeout=60, **kwargs):
|
||||
if self.host == "local":
|
||||
return run(argv, timeout=timeout, **kwargs)
|
||||
return run(["ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8", self.host,
|
||||
shlex.join(list(map(str, argv)))], timeout=timeout, **kwargs)
|
||||
|
||||
def adb(self, *argv, **kwargs):
|
||||
return self.remote("adb", "-s", self.serial, *argv, **kwargs)
|
||||
|
||||
def shell(self, *argv, **kwargs):
|
||||
return self.adb("shell", *argv, **kwargs).decode(errors="replace")
|
||||
|
||||
def focused(self, package):
|
||||
windows = self.shell("dumpsys", "window", "windows")
|
||||
line = next((line for line in windows.splitlines() if "mCurrentFocus=" in line), "")
|
||||
if f"{package}/{ACTIVITY}" not in line:
|
||||
raise ValueError("Inner Breeze does not own the foreground window: " + line.strip())
|
||||
|
||||
def screenshot(self, package, path):
|
||||
from PIL import Image, ImageStat
|
||||
|
||||
self.focused(package)
|
||||
data = self.adb("exec-out", "screencap", "-p")
|
||||
with Image.open(io.BytesIO(data)) as image:
|
||||
body = image.convert("RGB").crop((0, image.height // 10, image.width, image.height * 9 // 10))
|
||||
if max(ImageStat.Stat(body).stddev) < 2:
|
||||
raise ValueError(f"Inner Breeze rendered a blank screen ({path.name})")
|
||||
path.write_bytes(data)
|
||||
|
||||
def start(self, package, *extra):
|
||||
output = self.shell("am", "start", "-W", "-n", f"{package}/{ACTIVITY}", *extra)
|
||||
if "Status: ok" not in output and "Warning: Activity not started" not in output:
|
||||
raise ValueError("Inner Breeze did not start: " + output.strip())
|
||||
|
||||
def pid(self, package):
|
||||
return self.shell("pidof", package).strip()
|
||||
|
||||
|
||||
def apk_identity(apk):
|
||||
tools = sorted((Path(os.environ.get("ANDROID_SDK_ROOT") or os.environ.get("ANDROID_HOME") or
|
||||
Path.home() / "Android/Sdk") / "build-tools").glob("*/aapt2"))
|
||||
if not tools:
|
||||
raise ValueError("aapt2 not found; set ANDROID_SDK_ROOT")
|
||||
badging = run([tools[-1], "dump", "badging", apk]).decode()
|
||||
match = re.search(r"package: name='([^']+)' versionCode='(\d+)' versionName='([^']+)'", badging)
|
||||
if not match:
|
||||
raise ValueError("the APK has no package identity")
|
||||
return dict(package=match[1], version_code=int(match[2]), version_name=match[3])
|
||||
|
||||
|
||||
def source_state():
|
||||
commit = run(["git", "-C", ROOT, "rev-parse", "HEAD"]).decode().strip()
|
||||
dirty = run(["git", "-C", ROOT, "status", "--porcelain", "--", *SOURCE_PATHS]).decode().strip()
|
||||
version = run([sys.executable, ROOT / "scripts/check-version.py", "--print-version"]).decode().strip()
|
||||
return commit, dirty, version
|
||||
|
||||
|
||||
def frame_timing(phone, package, seconds):
|
||||
"""Collect every presented frame of the app's window for the session."""
|
||||
layers = [line.strip() for line in phone.shell("dumpsys", "SurfaceFlinger", "--list").splitlines()
|
||||
if line.strip().startswith(f"{package}/{ACTIVITY}")]
|
||||
if not layers:
|
||||
raise ValueError("Inner Breeze has no window layer")
|
||||
presented = {}
|
||||
period = 0
|
||||
deadline = time.monotonic() + seconds
|
||||
for layer in layers:
|
||||
phone.shell("dumpsys", "SurfaceFlinger", "--latency-clear", shlex.quote(layer))
|
||||
while time.monotonic() < deadline:
|
||||
for layer in layers:
|
||||
lines = phone.shell("dumpsys", "SurfaceFlinger", "--latency", shlex.quote(layer)).split("\n")
|
||||
if lines and lines[0].strip().isdigit():
|
||||
period = int(lines[0])
|
||||
for line in lines[1:]:
|
||||
fields = line.split()
|
||||
if len(fields) == 3 and fields[1].isdigit():
|
||||
actual = int(fields[1])
|
||||
if 0 < actual < PENDING:
|
||||
presented.setdefault(layer, set()).add(actual)
|
||||
time.sleep(0.5)
|
||||
if not presented or period <= 0:
|
||||
raise ValueError("SurfaceFlinger reported no frames for Inner Breeze")
|
||||
frames = sorted(max(presented.values(), key=len))
|
||||
if len(frames) < 2:
|
||||
raise ValueError("Inner Breeze presented fewer than two frames")
|
||||
gaps = [(right - left) / 1e6 for left, right in zip(frames, frames[1:])]
|
||||
period_ms = period / 1e6
|
||||
late = sum(1 for gap in gaps if gap > period_ms * 1.5)
|
||||
ordered = sorted(gaps)
|
||||
return dict(seconds=round((frames[-1] - frames[0]) / 1e9, 2), frames=len(frames),
|
||||
refresh_hz=round(1000 / period_ms, 1),
|
||||
fps=round((len(frames) - 1) / ((frames[-1] - frames[0]) / 1e9), 1),
|
||||
late_percent=round(100 * late / len(gaps), 2),
|
||||
p95_frame_ms=round(ordered[int(len(ordered) * 0.95) - 1], 1),
|
||||
worst_frame_ms=round(ordered[-1], 1))
|
||||
|
||||
|
||||
def smooth(timing):
|
||||
return (timing["fps"] >= MINIMUM_FPS and timing["late_percent"] <= MAXIMUM_LATE_PERCENT and
|
||||
timing["worst_frame_ms"] <= MAXIMUM_FRAME_MS)
|
||||
|
||||
|
||||
def check(phone, apk, seconds, evidence):
|
||||
identity = apk_identity(apk)
|
||||
package = identity["package"]
|
||||
if phone.adb("get-state").decode().strip() != "device":
|
||||
raise ValueError("the phone is not connected and authorized")
|
||||
transfer = ""
|
||||
try:
|
||||
if phone.host != "local":
|
||||
transfer = phone.remote("mktemp", "-d", "/tmp/inbe-phone-check-XXXXXXXX").decode().strip()
|
||||
if not re.fullmatch(r"/tmp/inbe-phone-check-[A-Za-z0-9]+", transfer):
|
||||
raise ValueError("unsafe transfer folder on the ADB host")
|
||||
remote = transfer + "/inbe.apk"
|
||||
phone.remote("sh", "-c", "umask 077; cat > " + shlex.quote(remote), input=apk.read_bytes(),
|
||||
timeout=300)
|
||||
else:
|
||||
remote = apk
|
||||
if "Success" not in phone.adb("install", "-r", remote, timeout=300).decode():
|
||||
raise ValueError("the phone rejected the APK; its existing app and data were kept")
|
||||
finally:
|
||||
if transfer:
|
||||
phone.remote("rm", "-rf", transfer)
|
||||
phone.shell("input", "keyevent", "KEYCODE_WAKEUP")
|
||||
phone.shell("wm", "dismiss-keyguard")
|
||||
phone.shell("am", "force-stop", package)
|
||||
phone.start(package)
|
||||
time.sleep(4)
|
||||
pid = phone.pid(package)
|
||||
if not pid:
|
||||
raise ValueError("Inner Breeze did not stay running after launch")
|
||||
phone.screenshot(package, evidence / "launch.png")
|
||||
phone.start(package, "-a", START_PRACTICE, "--ei", "practice_id", str(WIM_HOF))
|
||||
time.sleep(3)
|
||||
phone.screenshot(package, evidence / "practice-start.png")
|
||||
timing = frame_timing(phone, package, seconds)
|
||||
phone.screenshot(package, evidence / "practice-end.png")
|
||||
if phone.pid(package) != pid:
|
||||
raise ValueError("Inner Breeze restarted during the practice")
|
||||
phone.shell("input", "keyevent", "KEYCODE_HOME")
|
||||
time.sleep(2)
|
||||
phone.start(package)
|
||||
time.sleep(2)
|
||||
if phone.pid(package) != pid:
|
||||
raise ValueError("Inner Breeze restarted when it returned from the background")
|
||||
phone.screenshot(package, evidence / "resume.png")
|
||||
model = phone.shell("getprop", "ro.product.model").strip()
|
||||
android = phone.shell("getprop", "ro.build.version.release").strip()
|
||||
return identity, timing, dict(model=model, android=android)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("apk", type=Path)
|
||||
parser.add_argument("--host", default=os.environ.get("INBE_PHONE_HOST", "[email protected]"),
|
||||
help="SSH host that runs adb, or 'local'")
|
||||
parser.add_argument("--serial", default=os.environ.get("INBE_PHONE_SERIAL", "ZE2223BQZT"))
|
||||
parser.add_argument("--seconds", type=int, default=60)
|
||||
parser.add_argument("--record", action="store_true",
|
||||
help="write releases/phone-checks/VERSION.json for the release gate")
|
||||
args = parser.parse_args()
|
||||
for name in ("DISPLAY", "WAYLAND_DISPLAY", "XAUTHORITY", "DBUS_SESSION_BUS_ADDRESS"):
|
||||
os.environ.pop(name, None)
|
||||
apk = args.apk.resolve(strict=True)
|
||||
commit, dirty, version = source_state()
|
||||
if args.record and (dirty or args.seconds < 60):
|
||||
parser.error("a recorded check needs a full minute and committed sources:\n" + dirty)
|
||||
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
|
||||
evidence = ROOT / "backup/phone-checks" / f"{version}-{stamp}"
|
||||
evidence.mkdir(parents=True, exist_ok=True)
|
||||
identity, timing, device = check(Phone(args.host, args.serial), apk, args.seconds, evidence)
|
||||
if identity["version_name"] != version:
|
||||
raise SystemExit(f"the APK is {identity['version_name']}, but the source is {version}")
|
||||
passed = smooth(timing)
|
||||
receipt = dict(schema=1, version=version, source_commit=commit,
|
||||
apk=dict(sha256=hashlib.sha256(apk.read_bytes()).hexdigest(), **identity),
|
||||
device=device, practice=dict(name="Wim Hof", **timing),
|
||||
limits=dict(minimum_fps=MINIMUM_FPS, maximum_late_percent=MAXIMUM_LATE_PERCENT,
|
||||
maximum_frame_ms=MAXIMUM_FRAME_MS),
|
||||
checks=["installed over the existing app", "launch", "Wim Hof frame timing",
|
||||
"background and resume"],
|
||||
passed=passed, checked_at=datetime.now(timezone.utc).isoformat(timespec="seconds"))
|
||||
(evidence / "receipt.json").write_text(json.dumps(receipt, indent=2) + "\n")
|
||||
if args.record:
|
||||
target = ROOT / "releases/phone-checks" / f"{version}.json"
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(json.dumps(receipt, indent=2) + "\n")
|
||||
print(f"Recorded {target.relative_to(ROOT)}; commit it with the release.")
|
||||
print(f"{device['model']} (Android {device['android']}): {timing['fps']} fps over {timing['seconds']} s, "
|
||||
f"{timing['late_percent']}% late frames, worst {timing['worst_frame_ms']} ms -> "
|
||||
f"{'smooth' if passed else 'NOT smooth'}; evidence in {evidence.relative_to(ROOT)}")
|
||||
return 0 if passed else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,176 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run app packages across Inner Breeze versions before they reach phones.
|
||||
|
||||
Forward: this build runs every package the Waozi node serves to an app with its
|
||||
host interface. Phones still hold those packages when they install this app.
|
||||
|
||||
Reverse: the last released app runs each package built here that declares a
|
||||
host interface the released app accepts, so a package update that claims to
|
||||
support older apps cannot break them.
|
||||
|
||||
Each package is loaded with --bundle, which validates it and fails the run when
|
||||
the host rejects it, and its app is opened and rendered once.
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
from PIL import Image, ImageStat
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
REPOSITORY = "waozixyz/inbe"
|
||||
# (ledger name, app ID, local bundle, --feature name)
|
||||
PACKAGES = (
|
||||
("inbe", "inbe", "build/inbe.zib", None),
|
||||
("lists", "inbe.lists", "build/cells/lists.zib", "lists"),
|
||||
("habits", "inbe.habits", "build/cells/habits.zib", "habits"),
|
||||
("practices", "inbe.practices", "build/cells/practices.zib", "practices"),
|
||||
("diary", "inbe.diary", "build/cells/diary.zib", "diary"),
|
||||
("lumi", "inbe.lumi", "build/cells/lumi.zib", "lumi"),
|
||||
("leaderboard", "inbe.leaderboard", "build/cells/leaderboard.zib", None),
|
||||
)
|
||||
SCRUBBED = ("DISPLAY", "WAYLAND_DISPLAY", "XAUTHORITY", "DBUS_SESSION_BUS_ADDRESS",
|
||||
"SESSION_MANAGER", "GDK_DISPLAY")
|
||||
|
||||
|
||||
def fetch(url, limit):
|
||||
request = urllib.request.Request(url, headers={"User-Agent": "Inbe-Compatibility-Test/1.0"})
|
||||
with urllib.request.urlopen(request, timeout=60) as response:
|
||||
data = response.read(limit + 1)
|
||||
if len(data) > limit:
|
||||
raise ValueError(f"{url} exceeds {limit} bytes")
|
||||
return data
|
||||
|
||||
|
||||
def host_range(ledger):
|
||||
"""Apps released before version ranges accept only their own interface."""
|
||||
return ledger.get("oldest_host_api", ledger["host_api"]), ledger["host_api"]
|
||||
|
||||
|
||||
def declared_host(ledger, name):
|
||||
return ledger["apps"][name].get("host_api", ledger["host_api"])
|
||||
|
||||
|
||||
def run_app(binary, bundle, feature, output, label):
|
||||
"""Open one package and render its app once on a private display."""
|
||||
environment = {key: value for key, value in os.environ.items() if key not in SCRUBBED}
|
||||
environment.update(YUE_DESKTOP_RECOVERY="0", SDL_AUDIODRIVER="dummy", APP_NO_TRAY="1")
|
||||
image = output / (label + ".png")
|
||||
log = output / (label + ".log")
|
||||
image.unlink(missing_ok=True)
|
||||
command = ["xvfb-run", "-a", "-n", "361", "-s", "-screen 0 1280x900x24", str(binary),
|
||||
"--bundle", str(bundle), *(["--feature", feature] if feature else []),
|
||||
"--screenshot", str(image), "--screenshot-width", "540", "--screenshot-height", "960"]
|
||||
with log.open("w") as stream:
|
||||
result = subprocess.run(command, env=environment, stdout=stream, stderr=subprocess.STDOUT,
|
||||
timeout=180)
|
||||
text = log.read_text(errors="replace")
|
||||
if result.returncode != 0 or not image.exists():
|
||||
raise AssertionError(f"{label}: the app rejected the package (exit {result.returncode})\n{text[-2000:]}")
|
||||
if "cell launch rejected" in text:
|
||||
raise AssertionError(f"{label}: the package loaded but its app did not open\n{text[-2000:]}")
|
||||
# A host call whose records do not fit fails inside the package and the
|
||||
# screen silently loses that part, so any failed call fails the check.
|
||||
failures = sorted({line for line in text.splitlines() if "portable execution failed" in line})
|
||||
if failures:
|
||||
raise AssertionError(f"{label}: package calls failed: " + "; ".join(failures))
|
||||
with Image.open(image) as picture:
|
||||
if max(ImageStat.Stat(picture.convert("RGB")).stddev) < 2:
|
||||
raise AssertionError(f"{label}: the app rendered a blank screen")
|
||||
|
||||
|
||||
def forward(binary, origin, ledger, output):
|
||||
oldest, current = host_range(ledger)
|
||||
checked = 0
|
||||
for name, app_id, _, feature in PACKAGES:
|
||||
route = f"{origin}/api/v2/packages/{app_id}"
|
||||
try:
|
||||
metadata = json.loads(fetch(f"{route}/latest?host_api={current}&module_api={ledger['module_api']}", 65536))
|
||||
except urllib.error.HTTPError as error:
|
||||
if error.code == 404:
|
||||
print(f"forward {app_id}: the node has no release this build can run")
|
||||
continue
|
||||
raise
|
||||
if not oldest <= metadata["host_api"] <= current or metadata["module_api"] != ledger["module_api"]:
|
||||
print(f"forward {app_id}: skipped {metadata['version']}, built for host "
|
||||
f"{metadata['host_api']} outside {oldest}..{current}")
|
||||
continue
|
||||
artifact = metadata["artifacts"][0]
|
||||
data = fetch(f"{route}/{artifact['sha256']}.zib", artifact["size"])
|
||||
if len(data) != artifact["size"] or hashlib.sha256(data).hexdigest() != artifact["sha256"]:
|
||||
raise AssertionError(f"forward {app_id}: the node served bytes that differ from its release")
|
||||
bundle = output / f"published-{name}.zib"
|
||||
bundle.write_bytes(data)
|
||||
run_app(binary, bundle, feature, output, f"forward-{name}")
|
||||
print(f"forward {app_id} {metadata['version']} (host {metadata['host_api']}): runs in this build")
|
||||
checked += 1
|
||||
return checked
|
||||
|
||||
|
||||
def released_app(tag, output):
|
||||
"""Unpack the release's Linux AppImage without FUSE."""
|
||||
image = output / "released.AppImage"
|
||||
image.write_bytes(fetch(f"https://github.com/{REPOSITORY}/releases/download/{tag}/"
|
||||
"inbe-linux-x86_64.AppImage", 256 * 1024 * 1024))
|
||||
image.chmod(image.stat().st_mode | stat.S_IXUSR)
|
||||
environment = {key: value for key, value in os.environ.items() if key not in SCRUBBED}
|
||||
subprocess.run([str(image), "--appimage-extract"], cwd=output, env=environment, check=True,
|
||||
stdout=subprocess.DEVNULL, timeout=300)
|
||||
return output / "squashfs-root/AppRun"
|
||||
|
||||
|
||||
def reverse(tag, ledger, output):
|
||||
released = json.loads(fetch(f"https://raw.githubusercontent.com/{REPOSITORY}/{tag}/apps/versions.json", 65536))
|
||||
oldest, current = host_range(released)
|
||||
candidates = [package for package in PACKAGES
|
||||
if oldest <= declared_host(ledger, package[0]) <= current and
|
||||
ledger["module_api"] == released["module_api"]]
|
||||
if not candidates:
|
||||
print(f"reverse {tag}: no package here declares host {oldest}..{current}; nothing to check")
|
||||
return 0
|
||||
binary = released_app(tag, output)
|
||||
for name, app_id, path, feature in candidates:
|
||||
bundle = ROOT / path
|
||||
if not bundle.exists():
|
||||
raise AssertionError(f"reverse {app_id}: {path} is missing; build the packages first")
|
||||
run_app(binary, bundle, feature, output, f"reverse-{name}")
|
||||
print(f"reverse {app_id} (declares host {declared_host(ledger, name)}): runs in {tag}")
|
||||
return len(candidates)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("binary", type=Path, nargs="?", help="this build's app; needed unless --skip-forward")
|
||||
parser.add_argument("--origin", default="https://api.waozi.xyz")
|
||||
parser.add_argument("--release", help="Release tag to check new packages against (default: the latest)")
|
||||
parser.add_argument("--skip-forward", action="store_true")
|
||||
parser.add_argument("--skip-reverse", action="store_true")
|
||||
args = parser.parse_args()
|
||||
ledger = json.loads((ROOT / "apps/versions.json").read_text())
|
||||
output = ROOT / "build/package-compat-test"
|
||||
output.mkdir(parents=True, exist_ok=True)
|
||||
checked = 0
|
||||
if not args.skip_forward:
|
||||
if args.binary is None:
|
||||
parser.error("the forward check needs this build's app binary")
|
||||
checked += forward(args.binary.resolve(strict=True), args.origin.rstrip("/"), ledger, output)
|
||||
if not args.skip_reverse:
|
||||
tag = args.release
|
||||
if tag is None:
|
||||
latest = json.loads(fetch(f"https://api.github.com/repos/{REPOSITORY}/releases/latest", 1 << 20))
|
||||
tag = latest["tag_name"]
|
||||
with tempfile.TemporaryDirectory(prefix="inbe-released-", dir=output) as directory:
|
||||
checked += reverse(tag, ledger, Path(directory))
|
||||
print(f"Package compatibility: {checked} package runs passed")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,92 @@
|
||||
"""The release gate accepts only a passing phone check of the released sources."""
|
||||
import copy
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
spec = importlib.util.spec_from_file_location("check_phone_check", ROOT / "scripts/check-phone-check.py")
|
||||
gate = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(gate)
|
||||
|
||||
RECEIPT = {
|
||||
"schema": 1, "version": "3.0.0", "source_commit": "",
|
||||
"apk": {"sha256": "0" * 64, "package": "xyz.waozi.inbe.debug", "version_code": 300, "version_name": "3.0.0"},
|
||||
"device": {"model": "moto e6 play", "android": "9"},
|
||||
"practice": {"name": "Wim Hof", "seconds": 60.0, "frames": 3580, "refresh_hz": 60.0, "fps": 59.6,
|
||||
"late_percent": 0.4, "p95_frame_ms": 16.9, "worst_frame_ms": 41.2},
|
||||
"passed": True,
|
||||
}
|
||||
|
||||
|
||||
class PhoneCheckGateTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
temporary = tempfile.TemporaryDirectory(prefix="inbe-phone-gate-")
|
||||
self.addCleanup(temporary.cleanup)
|
||||
self.root = Path(temporary.name)
|
||||
self.git("init", "-q")
|
||||
self.git("config", "user.email", "[email protected]")
|
||||
self.git("config", "user.name", "Test")
|
||||
self.commit({"src/main.zi": "one\n"})
|
||||
self.checked = self.git("rev-parse", "HEAD").strip()
|
||||
|
||||
def git(self, *argv):
|
||||
return subprocess.run(["git", "-C", str(self.root), *argv], check=True, capture_output=True,
|
||||
text=True).stdout
|
||||
|
||||
def commit(self, files):
|
||||
for name, text in files.items():
|
||||
path = self.root / name
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(text)
|
||||
self.git("add", "-A")
|
||||
self.git("commit", "-q", "-m", "change")
|
||||
|
||||
def record(self, change=None):
|
||||
receipt = copy.deepcopy(RECEIPT)
|
||||
receipt["source_commit"] = self.checked
|
||||
if change:
|
||||
change(receipt)
|
||||
self.commit({"releases/phone-checks/3.0.0.json": json.dumps(receipt)})
|
||||
|
||||
def test_passing_check_of_the_released_sources(self):
|
||||
self.record()
|
||||
self.commit({"docs/notes.md": "later documentation\n", "tests/extra.py": "\n"})
|
||||
self.assertEqual(gate.validate(self.root, "3.0.0")["version"], "3.0.0")
|
||||
|
||||
def test_missing_check(self):
|
||||
with self.assertRaisesRegex(ValueError, "no phone check"):
|
||||
gate.validate(self.root, "3.0.0")
|
||||
|
||||
def test_app_changed_after_the_check(self):
|
||||
self.record()
|
||||
for name in ("src/main.zi", "ziran.lock", "droid/app/build.gradle", "CHANGELOG.md"):
|
||||
with self.subTest(path=name):
|
||||
self.commit({name: "changed " + name + "\n"})
|
||||
with self.assertRaisesRegex(ValueError, "changed after its phone check"):
|
||||
gate.validate(self.root, "3.0.0")
|
||||
|
||||
def test_failed_or_mismatched_checks(self):
|
||||
cases = {
|
||||
"slow": lambda value: value["practice"].update(fps=41.0),
|
||||
"late frames": lambda value: value["practice"].update(late_percent=9.0),
|
||||
"long stall": lambda value: value["practice"].update(worst_frame_ms=600.0),
|
||||
"short run": lambda value: value["practice"].update(seconds=20.0),
|
||||
"marked failed": lambda value: value.update(passed=False),
|
||||
"other version": lambda value: value.update(version="2.9.9"),
|
||||
"other APK": lambda value: value["apk"].update(version_name="2.9.9"),
|
||||
"short commit": lambda value: value.update(source_commit=value["source_commit"][:12]),
|
||||
"unknown commit": lambda value: value.update(source_commit="1" * 40),
|
||||
}
|
||||
for name, change in cases.items():
|
||||
with self.subTest(case=name):
|
||||
self.record(change)
|
||||
with self.assertRaises(ValueError):
|
||||
gate.validate(self.root, "3.0.0")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user