Add the new-model skill with real-turn and remote-install scripts

Adopting GPT-6.2-Sol took about 1.5 hours, mostly avoidable: builds
restarted after a raw cargo run without ZIRAN reran the Ziran build
scripts, two logins turned out to be at their usage limit only after the
build, Omega's 8 s DNS lookups slowed every turn, and Omega and Delta
were installed by hand.

skills/new-model lists the steps in the order that avoids those waits.
scripts/model-turns.py runs a plain answer, a shell call and a patch edit
at each effort and accepts a turn only on its real effect.
scripts/install-xcoder-remote.sh installs the locally installed build on
Omega or Delta (through Omega), renaming each file into place.
This commit is contained in:
waozi committed 2026-11-17 17:27:22 -03:00
1 parent 095c9cfa21
commit 1c11446a78
4 files changed
+188

No files matched your search

+1
View File
@@ -20,6 +20,7 @@ The changelog can be found on the [releases page](https://github.com/openai/code
- Parallel tool cells no longer fail at random with "Text file busy": starting a freshly compiled cell retries while another thread's fork still holds the file.
- The upstream Codex version check runs on its own thread, so a slow network never delays Xcoder exiting; the offline local tests turn it off.
- GPT-6.2-Sol (`gpt-6.2-sol`) is in the bundled model table, first, so it is the default when no model is configured even if the live catalog cannot be fetched. The entry is the provider's own (500,000-token context, efforts low to ultra, freeform `apply_patch`, code-mode-only tools, the `ultrafast` service tier, audio input). Before, a session whose catalog request timed out (the request has 5 s, including DNS) started on GPT-6.1-Sol, or ran GPT-6.2-Sol with fallback metadata and the warning "Model metadata for `gpt-6.2-sol` not found".
- The `new-model` skill (`skills/new-model/SKILL.md`) lists the steps for adopting a new model, with the checks that save time: logins with quota and DNS speed first, builds only through `make`, real turns started as soon as the binary exists. `scripts/model-turns.py` runs the 12 real turns (a plain answer, a shell call and a patch edit at each effort). `scripts/install-xcoder-remote.sh omega|delta` installs the locally installed build on Omega or Delta, renaming each file into place.
## 0.1.0 — 2026-11-14
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Install the xcode build installed on this machine (~/.local/lib/xcoder/bin, written by
# `make install`) on another device, replacing each file atomically so running sessions
# keep their old executable. No copies of replaced binaries are kept.
#
# Usage: scripts/install-xcoder-remote.sh omega|delta
# omega: ssh omega
# delta: through Omega (Alpha has no host key for Delta), [email protected]
set -euo pipefail
target=${1:?usage: install-xcoder-remote.sh omega|delta}
src="$HOME/.local/lib/xcoder/bin"
files=(codex codex-code-mode-host codex.xcoder.json)
shas=()
for f in "${files[@]}"; do shas+=("$(sha256sum "$src/$f" | cut -d' ' -f1)"); done
# Runs on the target: verify the staged files, keep each file's mode, rename into place,
# and make `xcode` a link to `codex` when it is still a separate copy.
read -r -d '' finish <<'EOF' || true
set -eu
d="$HOME/.local/lib/xcoder/bin"; cd "$d"
i=0
for f in codex codex-code-mode-host codex.xcoder.json; do
i=$((i + 1)); want=$(eval echo "\${$i}")
[ "$(sha256sum "$f.xcoder-new" | cut -d' ' -f1)" = "$want" ] || { echo "checksum mismatch: $f" >&2; exit 1; }
[ -e "$f" ] && chmod --reference="$f" "$f.xcoder-new"
done
mv -f codex-code-mode-host.xcoder-new codex-code-mode-host
mv -f codex.xcoder-new codex
mv -f codex.xcoder.json.xcoder-new codex.xcoder.json
if [ -f xcode ] && [ ! -L xcode ]; then ln -s codex xcode.xcoder-new; mv -fT xcode.xcoder-new xcode; fi
echo "$(hostname): $(sha256sum codex | cut -c1-16) $("$d/xcode" --version)"
EOF
case "$target" in
omega)
for f in "${files[@]}"; do scp -q "$src/$f" "omega:.local/lib/xcoder/bin/$f.xcoder-new"; done
ssh -o BatchMode=yes omega bash -s "${shas[@]}" <<<"$finish"
;;
delta)
# Relay through Omega's installed copy, which must already match this machine's.
remote=$(ssh -o BatchMode=yes omega "sha256sum ~/.local/lib/xcoder/bin/codex | cut -d' ' -f1")
[ "$remote" = "${shas[0]}" ] || { echo "install on omega first" >&2; exit 1; }
ssh -o BatchMode=yes omega "cd ~/.local/lib/xcoder/bin && for f in ${files[*]}; do scp -q -o BatchMode=yes \$f [email protected]:.local/lib/xcoder/bin/\$f.xcoder-new; done"
ssh -o BatchMode=yes omega "ssh -o BatchMode=yes [email protected] bash -s ${shas[*]}" <<<"$finish"
;;
*) echo "unknown target: $target" >&2; exit 2 ;;
esac
+120
View File
@@ -0,0 +1,120 @@
#!/usr/bin/env python3
"""Run short real turns on a model: a plain answer, one shell call and one patch edit per effort.
Usage: model-turns.py BINARY WORKDIR [--model M] [--efforts low,medium,...] [--codex-home DIR]
Each turn runs in its own folder under WORKDIR with the sandbox bypassed (agent shells
cannot start bubblewrap). A turn is ACCEPTED only when it exits 0, has no error event or
error item (a refusal such as {"detail":"Bad Request"} or "Model metadata ... not found"
fails it) and has the expected effect. Catalog refresh timeouts are counted separately:
they mean the host's DNS or network is slow, not that the model failed.
Prints one line per turn; never prints credentials (only event types and short messages).
"""
import argparse
import json
import os
import shutil
import subprocess
import sys
import time
from pathlib import Path
p = argparse.ArgumentParser()
p.add_argument('binary')
p.add_argument('workdir')
p.add_argument('--model')
p.add_argument('--efforts', default='low,medium,high,xhigh')
p.add_argument('--codex-home')
p.add_argument('--timeout', type=int, default=420)
p.add_argument('--only', default='plain,shell,patch')
args = p.parse_args()
env = dict(os.environ)
if args.codex_home:
env['CODEX_HOME'] = args.codex_home
for name in [k for k in env if k.startswith('RUST_LOG')]:
del env[name]
def run_turn(cwd, effort, prompt):
cmd = [args.binary, 'exec', '--json', '--skip-git-repo-check', '--ephemeral',
'--dangerously-bypass-approvals-and-sandbox', '-C', str(cwd)]
if args.model:
cmd += ['-m', args.model]
if effort:
cmd += ['-c', f'model_reasoning_effort="{effort}"']
cmd.append(prompt)
start = time.time()
try:
proc = subprocess.run(cmd, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
text=True, timeout=args.timeout, stdin=subprocess.DEVNULL)
except subprocess.TimeoutExpired:
return dict(rc='timeout', secs=round(time.time() - start), events=[], model=None, text='', errors=['timeout'])
events, errors, model, text = [], [], None, ''
for line in proc.stdout.splitlines():
try:
ev = json.loads(line)
except ValueError:
continue
kind = ev.get('type', '')
item = ev.get('item') or {}
if kind.startswith('item.') and item:
events.append(f"{kind}:{item.get('type')}")
if item.get('type') == 'agent_message' and kind == 'item.completed':
text = (item.get('text') or '')[:160]
if item.get('type') == 'error' and kind == 'item.completed':
errors.append('error item: ' + (item.get('message') or json.dumps(item))[:300])
else:
events.append(kind)
if kind in ('error', 'turn.failed'):
errors.append(json.dumps(ev)[:400])
if kind == 'thread.started' or 'model' in ev:
model = ev.get('model') or model
catalog_timeouts = 0
for line in proc.stderr.splitlines():
low = line.lower()
if 'failed to refresh available models: request timed out' in low:
catalog_timeouts += 1
elif 'error' in low or 'bad request' in low or 'unsupported' in low or 'model' in low:
if 'token' not in low and 'bearer' not in low:
errors.append(line[:300])
return dict(rc=proc.returncode, secs=round(time.time() - start), events=events, model=model,
text=text, errors=errors[:6], catalog_timeouts=catalog_timeouts)
root = Path(args.workdir)
root.mkdir(parents=True, exist_ok=True)
results = []
for effort in [e for e in args.efforts.split(',') if e]:
for kind in args.only.split(','):
cwd = root / f'{effort}-{kind}'
shutil.rmtree(cwd, ignore_errors=True)
cwd.mkdir(parents=True)
if kind == 'plain':
r = run_turn(cwd, effort, 'Reply with exactly the word PONG and nothing else. Do not run any tools.')
ok = r['rc'] == 0 and 'PONG' in r['text'] and not r['errors']
detail = f"reply={r['text']!r}"
elif kind == 'shell':
token = f'sol62-{effort}-{int(time.time())}'
(cwd / 'seed.txt').write_text(token + '\n')
r = run_turn(cwd, effort, 'Run exactly one shell command, `cat seed.txt`, and then reply with only the text it printed.')
ran = any(e.endswith(':command_execution') for e in r['events'])
ok = r['rc'] == 0 and ran and token in r['text'] and not r['errors']
detail = f"command_execution={ran} reply={r['text']!r}"
else:
(cwd / 'notes.txt').write_text('first line\ncolor: alpha\nlast line\n')
r = run_turn(cwd, effort, 'Use the apply_patch tool (not a shell command) to change "color: alpha" to "color: omega" in notes.txt. Make no other change. Then reply DONE.')
content = (cwd / 'notes.txt').read_text()
patched = any(e.endswith(':file_change') for e in r['events'])
file_ok = content == 'first line\ncolor: omega\nlast line\n'
ok = r['rc'] == 0 and patched and file_ok and not r['errors']
detail = f"file_change={patched} file_ok={file_ok}"
status = 'ACCEPTED' if ok else 'FAILED'
line = f"{effort:6} {kind:5} {status} rc={r['rc']} {r['secs']}s {detail}"
if r.get('catalog_timeouts'):
line += f" (catalog refresh timeouts: {r['catalog_timeouts']})"
if not ok:
line += f" events={sorted(set(r['events']))} errors={r['errors']}"
print(line, flush=True)
results.append(ok)
print(f"summary: {sum(results)}/{len(results)} accepted", flush=True)
sys.exit(0 if all(results) else 1)
+19
View File
@@ -0,0 +1,19 @@
---
name: new-model
description: Make a new provider model (for example the next Sol) work in xcode and the default, prove it with real turns, and install it on Alpha, Omega and Delta in one pass.
---
Work on Alpha in `~/Projects/waozixyz/xcoder` (it holds the latest master). Never print tokens or account details. Logins: Alpha `~/.codex`; Omega `~/.codex` (codex-1) and `~/.config/harmony/codex/accounts/account-d369c369a25044208f83` (codex-2); Delta has its own copies of both. Never copy credential files between machines. Harmony's own default and tier mapping belong to the Harmony task on Omega; send it the model ID as soon as you have it.
1. **First 2 minutes, before reading code:** for every login run `CODEX_HOME=<home> xcoder exec --skip-git-repo-check --ephemeral "Reply PONG"` (full path `~/.local/lib/xcoder/bin/xcode` over SSH). A usage-limit error rules that login out for turns; pick the turn login now. On each host time `getent ahosts chatgpt.com`: over 1 s means broken DNS (Omega 2026-11-17: its IPv4 Quad9 servers timed out, 8 s per lookup). xcode's catalog request has 5 s, DNS included, and silently falls back to the bundled table.
2. **Live list:** `CODEX_HOME=<home> xcoder debug models > live.json`; print only slug, priority, visibility, context, efforts. If it equals `xcoder debug models --bundled`, the fetch failed (step 1). Stop before changing defaults if no login lists the model.
3. **What is new:** diff the new entry against the previous Sol in `live.json`. Check each `experimental_supported_tools` name with `rg` in `codex-rs/core/src/tools/spec_plan.rs` (unknown names are ignored, not errors). Look upstream only for new `ModelInfo` fields: `gh api 'repos/openai/codex/commits?since=<date>'`. Port nothing else.
4. **Bundled entry:** insert the live entry into `codex-rs/models-manager/models.json` in the bundled key order (the keys of the existing Sol entry), with a priority below every other entry, leaving the other entries untouched (`json.dumps(indent=2, ensure_ascii=False)` round-trips the file exactly). `xcoder debug models --bundled` must list it first. `failed_catalog_fetch_defaults_to_newest_bundled_sol` checks that the newest bundled Sol is the default.
5. **Build:** only through `make` (it exports `ZIRAN`; a raw `cargo` without `ZIRAN=ziran` reruns every Ziran build script and the next `make` rebuilds the workspace). Run it detached so a session restart does not kill it: `systemd-run --user --unit=xcoder-build --slice=harmony-agents.slice --working-directory=$PWD --setenv=PATH="$PATH" --setenv=HOME="$HOME" bash -c 'exec build-queue -- make -j2 all > build/build.log 2>&1'`. Alpha has 15 GiB: use 2–3 jobs. If the memory guard stops it, finished crates are kept; rerun with fewer jobs (the final `codex-cli` link alone: `-j1`).
6. **As soon as `build/bin/xcoder` exists, start the real turns in the background:** copy `build/bin/xcoder` and `build/bin/codex-code-mode-host` to `/tmp/xcode-<model>-build/` on the turn login's host, then `python3 -I scripts/model-turns.py /tmp/xcode-<model>-build/xcoder /tmp/xcode-<model>-turns --model <slug> --codex-home <home>`. All 12 lines (plain, shell, patch × low, medium, high, xhigh) must say ACCEPTED. About 15 s per turn on good DNS, 70–90 s on Omega's broken DNS.
7. **Tests, meanwhile, one at a time:** models-manager: `systemd-run ... --setenv=ZIRAN=ziran --setenv=PATH=$PWD/build/toolchain/bin:$PATH --setenv=CARGO_HOME=$PWD/build/cargo-home --setenv=CARGO_TARGET_DIR=$PWD/codex-rs/target bash -c 'exec build-queue -- cargo test --manifest-path codex-rs/Cargo.toml --profile local -p codex-models-manager --lib > build/mm.log 2>&1'`. Then the certified suite from an unrestricted host shell (agent shells have NoNewPrivs, so bubblewrap fails): `ssh omega`, `ssh -F ~/.ssh/alpha.conf alpha`, and there `systemd-run` with `PATH=$HOME/.local/bin:...`, `XCODER_ZIRAN_TOOLCHAIN=$(ziran pkg path ziran)` and `build-queue -- make -j2 test`. Passing prints "Local suite passed; installation receipt saved for the tested binaries"; `backup/xcoder-validation.json` must hold the binary's sha256.
8. **Commit and push:** a line under Unreleased in `CHANGELOG.md`; commit only your files from a private `GIT_INDEX_FILE` (`git read-tree HEAD`, `git add`, commit, then `git reset -q -- <files>` in the real index) as `Waozi <[email protected]>` alone, no co-author lines; `git push origin master`. No version bump, tag or release.
9. **Install:** Alpha `make install`; then `scripts/install-xcoder-remote.sh omega` and `scripts/install-xcoder-remote.sh delta` (Delta goes through Omega's installed copy; its link is slow, about 10 minutes). Files are renamed into place, so running sessions keep working until they restart.
10. **Config pins:** `grep -n '^model *=' <home>/config.toml` for every login on every host. A pin to an older model keeps plain `xcode` on it. Remove the line (Harmony never reads it) under an exclusive `flock` on `config.toml.harmony-config.lock`, writing the file atomically.
11. **Verify each host:** `CODEX_HOME=<home> ~/.local/lib/xcoder/bin/xcode exec --skip-git-repo-check --ephemeral "Reply with exactly the word PONG"` must print `model: <slug>` and, on a login with quota, PONG. A usage limit or "access token could not be refreshed" is an account problem for the owner, not a model failure; report it.
12. **Report:** model ID and properties, the turn table, tests with their results, the commit link, installs per host with sha256, account and DNS problems, and that running sessions get the model when they restart.