Files
waozi 2cafe84dc8
Alpha check / check (push) Successful in 10m8s
Integrate verified ZX modules and compiler improvements
Add typed separately compiled ZX libraries with independent instances and compatible state replacement. Tighten generated-name diagnostics, remove the fixed loader graph limit, and resolve platform conditions without eager enum-import cycles.

Resolve known function calls, bound specialization work, improve loop inlining, and copy borrowed Wasm records safely across overlapping pointers. Retain measured native vector growth and allocated-state verification improvements with startup phase profiling. Include focused regression coverage, synchronized bootstrap and site documentation, Lean research plans, and raw Alpha comparison receipts with rejected prototypes clearly identified.

Local validation: all 273 retained tests passed across the full run (272/273) and the audio rerun after refreshing stale disposable CMake compiler paths (1/1), with no implementation drift. Linux, Darwin and SIMD bootstrap seeds match. Current-tree Omega installation verified all 11 binary identities and ran Hello World through the installed compiler.

Alpha matched measurements support smaller Wasm modules; the native vector workload improves 4.62%, while printing is 1.41% slower. Separate application, controller and Moto integration remains outside this compiler milestone.
2026-11-19 00:01:23 -03:00
..

Runtime and compiler benchmarks

Every benchmark runs through one entry point, from the repository root:

build/bin/ziran run --target=c bench/bench.zi -- list
build/bin/ziran run --target=c bench/bench.zi -- NAME [DRIVER ARGUMENTS]
build/bin/ziran run --target=c bench/bench.zi -- --local NAME --validate ...

bench/bench.zi names every driver, gives each the same environment (no display or desktop-session variables, no ZIRAN_PROFILE, and ZIRAN_LEARN=off so the learning corpus never records during a benchmark) and refuses to measure a host benchmark anywhere but the physical host named by ZIRAN_BENCH_HOST, alpha by default. --local is for measurement-free modes. A new driver is registered in bench/bench.zi; tests/bench_entry.sh fails while one is not. The per-benchmark commands below are the driver arguments; prefix each with the entry point instead of running the driver directly.

Run benchmark builds and measurements on Alpha's physical host, as requested by the owner on 2026-11-11, from the canonical Ziran repository there. Reject virtualized execution. If Alpha is unavailable, wait rather than running the benchmark on the current desktop. Record the host in every report and collect both sides of a comparison on the same host; retained desktop results keep their original provenance.

Standard benchmarks and the host gate

bench/standard/README.md describes the standard language benchmarks (wasm-tier --standard, --standard-full for the Benchmarks Game sizes) and standard-compile. Anything the site's kernel or standard tables publish is measured with --host-gate: each measured job runs on one logical CPU of the quietest core, inherited by the child so no wrapper process adds time, with Go at GOMAXPROCS=1. A poller on the other cores reads that CPU's scaling_cur_freq every 10 ms and the sibling's busy time. A job is kept only if its 10th to 90th percentile clock is within 3% of its median, the median reaches 95% of the clock calibrated at start, and the hyperthread sibling is busy at most 15% of the job; otherwise it runs again (up to eight attempts), and rejected attempts stay in the receipt with their observations. The site generators refuse receipts without it. Alpha's powersave/HWP clock otherwise moves between 0.8 and 4.3 GHz and run time follows it (results/records-native-frequency-alpha-2026-11-16.json). Compile-time builds are not pinned.

Timed compiler/codec kernels are an explicit benchmark command, outside make check: build/bin/ziran run --target=c bench/bench.zi -- compiler-kernels on Alpha. Routine tests still validate benchmark statistics and command interfaces.

JSON and UTF-8 workloads

On Alpha, run build/bin/ziran run --target=c bench/bench.zi -- json-text --samples 11 to build and measure the six Ziran programs in bench/json_text/. Each timed batch verifies its result. The website retains generated artifacts, raw samples and compiler diagnostics for unsupported targets in site/generated/json-text/. The qualified Alpha snapshot is retained in bench/results/json-text/alpha-matrix.json. Use build/bin/ziran run --target=c bench/bench.zi -- --local json-text --check to validate that snapshot without timing it.

Portable ZX execution

build/bin/ziran run --target=c bench/bench.zi -- zx-execution

This Ziran script measures every unchanged portable bundle in the published target matrix, verifies output and checksums, and records three repetitions of nine warm batches. Symmetric 1.5×IQR filtering excludes anomalies from each median while retaining the raw samples and excluded indices. Source, library and bundle hashes detect drift. Retained before/after measurements and the bottleneck analysis are in results/zx-loops/README.md.

Library workloads live upstream. The website's generic library harness accepts Kryon's manifest without adding widget behavior to Ziran:

build/bin/ziran run --target=c bench/bench.zi -- library-targets \
  --project ../../kryonlabs/kryon \
  --manifest ../../kryonlabs/kryon/tests/widget_benchmarks.json --samples 9

Use the actual Kryon checkout path. The published archive contains all checked modules; the source fixtures, generated C/Go/ZX and raw samples are downloadable. These desktop cases measure public widget behavior and recorded paint commands, excluding pixel rasterization and OS I/O.

ZX assembly comparison receipts

assembly_collect.zi validates an exact campaign plan and, only after fresh Alpha physical-host attestation, executes explicitly selected modes into the strict raw schema consumed by assembly_compare.zi. Local plan validation is measurement-free:

build/bin/ziran run --target=c bench/bench.zi -- --local assembly-collect --validate PLAN.json

Run PLAN.json OUTPUT.json instead only in the canonical checkout on Alpha's physical host. The collector hashes the workload and complete declared runtime closure before and after collection, enforces private cache/process boundaries, checks every sample's stdout oracle, and atomically emits raw samples. It supports explicit old VM, lowered portable, direct-AOT and control selectors. The current default ZX runtime is not native-only proof, so old-c-tinycc-native remains missing; no new-native ZX selector exists. Direct AOT is never relabeled as that engine. The exact plan schema, phase units, invocation, and remaining gates are in ../ASSEMBLY/performance-report.md.

ZX runtime comparison 2 driver

zx_compare.zi measures the portable engine, cold native ZX, warm native ZX, the zx2native AOT ELF control and the C control over identical workloads in one receipt. Every trial verifies the oracle checksum, the reported backend and, for native cohorts, the native evidence flag. The receipt records the machine, virtualization and load averages before and after the run, hashes of the driver, helper, control and library sources, the fixture artifact hashes, and the copied M0 TinyCC summary:

build/bin/ziran run --target=c bench/bench.zi -- zx-compare --engine --native-cold --native-warm --aot \
  --c-control --m0-receipts bench/results --samples 9

Every prepare step (fixture zi2zx, the zx-host helper build, the C control build, the per-kind AOT source generation and builds) runs through the queue and is recorded with its command line, exit code and wall time. The receipt is written atomically after every trial, and the integrity block re-hashes every input before the driver reports success. Without --out, the receipt is bench/results/zx-compare-<host>-<date>.json. --alpha additionally enforces the Alpha physical-host attestation gate for receipts that claim it.

Fresh compiler rebuild

build/bin/ziran run --target=c bench/bench.zi -- compiler-build

This times make -B -jN all at the normal build/ paths: all native Ziran compiler tools and libziran.a rebuild, including generated compiler modules. --jobs N selects concurrency; the default uses the queue's allocation when available, otherwise the Makefile's usual half-CPU limit of 2–16 jobs.

Results, raw build output and sample history are retained under the tracked bench/results/compiler-build/ path. The wall time excludes queue waiting, tests, installation and website builds. The report records CPU time, source diff fingerprint, revision, dirty state and compilation count. Filesystem caches remain warm; make -B forces all build targets to rebuild without deleting other build outputs.

Add --profile to record every make shell recipe's monotonic start, finish and elapsed time in recipes.json, with the 20 slowest in slowest-recipes.json. Profiling keeps compiler optimization flags unchanged, but adds process instrumentation; use ordinary runs for timing claims.

The temporary bootstrap seed defaults to -O0: it only emits the compiler modules' C before the final tools are built with the normal CFLAGS=-O2. BOOTSTRAP_CFLAGS=-O2 restores the former seed settings for a comparison.

Complete native builds

build/bin/ziran run --target=c bench/bench.zi -- native-builds \
  --out build/benchmarks/native-builds

Equivalent modular Ziran, C and Go programs produce an independently computed 64-bit recurrence. Every build runs a small checksum oracle; runtime samples run about twenty million module calls. The default matrix uses 1, 16 and 64 modules, five build samples and nine runtime samples after warmups.

fresh removes fixture build outputs and changes comments in every application source to miss Go application caches while preserving standard-library/toolchain caches. unchanged preserves inputs and outputs. leaf-edit changes one leaf body; shared-edit changes an imported constant used by all modules. Each edit uses a new value. C uses Make dependency files; Go uses -p=4/GOMAXPROCS=4; Ziran uses the ordinary native --exe driver. The Go cache is private to the benchmark and its first toolchain warmup is recorded separately.

This measures whole build commands rather than adding separate phase times. Source/command/tool hashes, raw samples, CPU time, artifact size and peak child RSS are retained. Peak RSS is not the sum of concurrent compiler memory. C's dynamic system libraries and Go's normal linked runtime have different packaging costs. Results describe these kernels on one shared host with warm filesystem caches, not a general language ranking or cold-machine build claim. Reuse the stable output after archiving needed receipts and checking that its outputs are inactive.

Add --before-bin-dir DIR to include a preserved ziran launcher and zi2c compiler as ziran-before. Both Ziran variants use the same source files and flags, separate output directories, and shuffled build/run order alongside C and Go. The metadata records generated-C and executable byte identity without requiring it for comparisons that intentionally change generated code.

Opt-in compiler profiles for native --exe builds include native.toolchain (compiler identity and cache preparation), native.objects (cache validation, compiler workers and dependency recording), and native.link (linking and atomic executable replacement). Counters record compilation attempts, reused objects and event/poll completion paths. Collect profiles separately from timings; profiler counters add work and the profile environment is part of the native object-cache identity. native.cache.lookup and native.cache.record separate validation from dependency recording within the object stage.

native_process_spawn and native_process_fork count the tool-launch paths, including compiler probes. Native builds prefer posix_spawnp with the same capture-pipe actions and inherited output lock. A failed spawn uses the existing fork/exec path, which also preserves executable shell scripts without a shebang. These counters identify the path taken; they do not imply a build speed improvement on every host or workload.

The native_probe_pipe and native_probe_compile counters distinguish GCC's piped empty-input probe from the ordinary compile probe retained for Clang and other eligible drivers. These probes run even when all objects are reusable; examine native.toolchain separately from object validation and linking. Successful piped probes also enable piped object compilation. The native_object_pipe counter records object attempts with that flag, including an explicit -pipe in CFLAGS; reused objects do not increment it. Compilers whose probe is unsupported or unsuccessful retain ordinary compilation. This uses GCC's documented pipe between compilation stages, while retaining separate translation units and the existing worker limit.

For a complete build of the formatter application itself:

build/bin/ziran run --target=c bench/bench.zi -- formatter-builds \
  --before-bin-dir PATH/TO/PRESERVED/bin \
  --out build/benchmarks/native-programs/formatter

This compares fresh and unchanged source/saved-IR builds with the same flags. Each executable formats an unformatted fixture containing a Unicode comment and a record, and checks the canonical result against the existing formatter. Before/after generated C and executable bytes must match. Profiles are collected separately from timings; the current checker prepares the saved IR for both compiler versions. The metadata retains all source/tool hashes and profiles. A large change on many small modules need not improve this application by the same amount; report both results and retain the warm-cache/shared-host limitations.

For a large shared header graph, use bench/native_header_graphs.py with --before-bin-dir DIR --out build/benchmarks/native-header-graphs and optional --repetitions N. It builds source and saved IR with 512 leaf records and 32 consumers. A foreign signature retains the complete record ABI; C static assertions check its size and final field offset. Every build runs two dynamic input oracles, and the final generated C and executable bytes must match. Dependency metadata must contain at least 512 inputs per consumer. Fresh and unchanged samples share warm machine caches and identical source inputs. This compares two compiler versions, without a Go or C language baseline.

Both harnesses save raw samples, tool/source hashes and actual CPU affinity. For a before/after comparison that reaches late record types within the former 512-module limit, add --groups 4 --read-last to the header-graph benchmark. --groups and --width control the shared record graph; defaults retain the original 512-leaf first-field workload. Static ABI assertions and runtime oracles follow the selected dimensions and field.

bench/reachable_types.py --before-lib DIR/libziran.a --out build/benchmarks/reachable-types measures two qualified lookups per round in wide, deep and shared import graphs. Graph creation and process startup are outside its internal CPU/wall timers. Cold samples invalidate entry-scope answers; cached samples retain them. Every sample records correctness. The previous implementation fails positive lookups beyond 512 modules, so those rows are explicitly labeled as incorrect and excluded from equivalent work comparisons. Peak process RSS includes graph construction and storage; it does not isolate traversal scratch memory. Symbol verification and both profilers are removed from timed environments. Set --before-limit 0 when the selected baseline already resolves the larger qualified graphs; the default 512 describes the earlier bounded walker.

bench/type_exports.py --before-lib DIR/libziran.a --out build/benchmarks/type-exports compares owned type names and public re-export graphs. Each round resolves two names, with entry caches invalidated for cold samples and retained for cached samples. Incorrect baseline depth-limited answers are retained separately from equivalent-work comparisons. Timing binaries have no allocator wrappers. Separate binaries wrap malloc, calloc and realloc to count calls and cumulative requested bytes during lookup; those figures describe allocator traffic, not live or peak RAM. Input-graph construction and startup are outside the internal CPU and wall timers. Every non-local graph case includes a back edge to its entry module; report these as cyclic graphs rather than generalizing their speedups to acyclic imports.

bench/native_probe.py --before-bin-dir DIR --out build/benchmarks/native-probe measures the five-object cache regression fixture with stable messages, alternating within-line version interleaving, and changed GMP versions. It reuses the regression's native compiler shim and serves saved real diagnostics plus controlled version records. Real object compilation and linking are timed, with runtime oracles and native byte identity checks. The ordinary probe subprocess cost and real-world frequency of interleaving are outside this experiment; pair it with ordinary build controls before drawing a general compilation-speed conclusion.

Entry-scope symbol resolution profiles report type_lookup_hit, type_lookup_miss, function_lookup_hit and function_lookup_miss. Misses include uncached ambiguous or absent results; repeated results count as hits until declarations or imports change. Set ZIRAN_VERIFY_SYMBOL_LOOKUPS=1 for correctness checks against fresh searches, and leave it unset when measuring performance. Import-free scopes with at most four procedures scan directly without cache or graph-visit storage.

Paired checking and body eligibility

Preserve ziran, zi2zir, zi2c and zi2zx together before changing the compiler, then run:

build/bin/ziran run --target=c bench/bench.zi -- body-eligibility \
  --before-bin-dir build/body-eligibility/before \
  --out build/benchmarks/body-eligibility

This reuses compiler_scaling.py's module and generic generators. Defaults cover 8, 32 and 128 modules with 16 and 64 fields, plus local and imported generic specializations. Both compiler versions must produce identical saved IR, emitted C and bundles. Identical C is compiled once per source/saved form; native and both portable runtimes must match independent arithmetic oracles. Nine shuffled pairs measure whole checking, C emission and bundle construction commands. Validation runs and a separately profiled warmup precede timings; profiles retain body-expression and record-field counters without adding profiling overhead to timing samples. CPU time and peak child RSS are retained.

Use --project NAME ROOT ENTRY for an offline package check with its existing lock and local overrides. Keep both compiler variants under the overridden Ziran root so the package launcher uses each variant's sibling tools. Project sources and manifests are hashed and checked for changes. These workloads measure compiler work on one shared host, not full native build times or general Go/C performance parity.

Repeated portable execution

Before changing the runtime, preserve zi2zir, zi2zx and libziran.a together in a stable baseline directory. Then run:

build/bin/ziran run --target=c bench/bench.zi -- vm-execution \
  --before-dir build/vm-execution/before --out build/benchmarks/vm-execution

This suite exercises integer arithmetic, mixed signed/unsigned comparisons, floating-point arithmetic, record calls over an array, fixed-array returns, population count, immutable string-byte scans, and the standard-library JSON parser. The array case passes a 4,096-element value through two functions and checks untouched zero elements as well as both ends. The JSON case scans nested records, reads signed fractional numbers, and decodes raw UTF-8 and escaped strings, including surrogate pairs. Its checksum comes from independently decoded Python JSON values and their UTF-8 bytes. The string case weights UTF-8 bytes at two traversal strides, with an independent Python checksum and an empty input among the documents. Each uses a separate Python result calculation. Source and saved IR must create identical bundles, and both runtime versions must recreate the source bundle before timing. Every first and repeated run must match the oracle. The default uses ten thousand numeric iterations, one JSON corpus pass, and nine repeated executions per instance, within the runner's million-statement limit. Results separate checked loading, instance opening, first execution, repeated execution and closing through vm_startup.py; its measurement and memory limitations below apply. These are synthetic VM comparisons on the recorded host, not native C/Go or whole-application performance claims. Fixture manifests retain the generator hashes and expected results alongside the existing raw timing receipts.

To calibrate these same workloads against generated native C and Go:

build/bin/ziran run --target=c bench/bench.zi -- vm-native \
  --vm-results build/benchmarks/vm-execution \
  --out build/benchmarks/vm-native

The harness regenerates each workload and its independent oracle, requires identical native output from source and saved IR, and verifies both forms. C defines ZIRAN_NO_ASM; Go uses CGO_ENABLED=0 and GOMAXPROCS=1, retaining ordinary garbage collection. Both controls therefore execute the checked body. Native timers cover a batch of 128 calls by default, including the call and checksum loop; per-call medians are compared with the recorded VM steady medians. Increase --runs when short batches quantize CPU usage to zero; --runs 4096 gives longer CPU samples for this suite. Every sample checks the complete batch checksum. Child peak RSS is measured separately from the driver process. Keep the raw samples, generated sources, commands, build observations and source/tool/artifact hashes.

These are matched generated backends, rather than independent handwritten implementations. Native batches and repeated VM instances differ in scheduling and memory management. Their ratios expose remaining interpreter overhead on the measured workloads; they do not establish whole-application C/Go parity.

Android device execution

Prepare matched Ziran source/saved-IR C and Go programs, handwritten C/Go controls, and a portable engine fixture for one Android ABI:

build/bin/ziran run --target=c bench/bench.zi -- android-execution prepare \
  --out build/benchmarks/android-device/execution \
  --ndk "$ANDROID_NDK_HOME" --abi armeabi-v7a

Run the measurement stage on the host that owns the device's ADB connection, passing its exact serial:

build/bin/ziran run --target=c bench/bench.zi -- android-execution run \
  --out build/benchmarks/android-device/execution --serial DEVICE_SERIAL

The script verifies transferred hashes, checks every checksum against an independent Python calculation, shuffles native trials, discards process warmups, and records internal kernel time and peak process RSS. Go uses one logical processor. The VM runs the small input and separates checked loading, instance preparation and repeated execution. A fresh directory under /data/local/tmp holds each attempt's files and is removed afterward; the result file records failures and cleanup status and protects existing results.

These are single-workload device measurements. Clocks, temperature and background activity remain uncontrolled, short CPU samples can round to zero, and peak RSS includes startup/runtime memory. ARMv7 uses the checked body for this fixture's x86_64/AArch64 assembly variants. Cross-build observations do not establish Go-like compilation speed on Android.

To compare the same seven portable workloads across saved VM libraries:

build/bin/ziran run --target=c bench/bench.zi -- vm-android prepare \
  --vm-results build/benchmarks/vm-execution \
  --out build/benchmarks/vm-json/android \
  --ndk "$ANDROID_NDK_HOME" --abi armeabi-v7a \
  --library before PATH/TO/BASELINE/libziran.a \
  --library after build/bundle-runtime/android-armeabi-v7a/libziran.a
build/bin/ziran run --target=c bench/bench.zi -- vm-android run \
  --out build/benchmarks/vm-json/android --serial DEVICE_SERIAL --cpu 0

Libraries must match the selected ABI and current VM API. Add another --library LABEL PATH to retain an earlier reference alongside a recent baseline. Preparation verifies fixture sources and expected results against the independent generator and records library, runner and bundle hashes. The device stage runs every variant consecutively for each case/repetition, with shuffled groups and rotated variant orders that balance positions across repetitions. It discards process warmups. It saves each sample immediately, refuses to overwrite results, checks transferred files, and verifies removal of its own remote directory. --cpu pins only the test child; omit it to retain normal scheduling. Record affinity with the results: neither mode controls device frequency, temperature or background activity, and a result on one core or ABI does not establish gains on another.

CPU-selected assembly

build/bin/ziran run --target=c bench/bench.zi -- cpu-features \
  --out build/benchmarks/cpu-features

This dependent xorshift/popcount workload supplies x86_64 popcnt and AArch64 simd assembly loops with an optimized portable Ziran population count. The harness compares CPU selection, forced Ziran fallback, handwritten C with CPU selection, and handwritten C fallback. Each input is checked against a Python integer/bit-count oracle, including negative and zero iteration counts. The small case also runs in the VM. Source and saved IR must emit identical C and bundles. The metadata records whether the host has the requested feature; native runs can legitimately select their portable implementation.

Use --assembly-form block for a local block around the complete loop, or --assembly-form operation-block for a local population-count block called in every iteration. The default remains function. Use separate stable outputs for each form. CPU-feature blocks pay selection and helper-call costs at each block execution; comparing both granularities shows when those costs outweigh a specialized instruction. A whole-loop block amortizes them.

The default large input is twenty million iterations to reduce startup's share of runtime. Loop assembly and optimized C can have different scheduling, so these numbers do not isolate instruction cost or establish a language ranking. Results retain commands, source/tool/artifact hashes, warmups, shuffled runtime order, CPU time, peak RSS and raw samples. Timings include process startup and the GNU/BusyBox time launcher when available. They measure one host with warm caches and no CPU isolation; cross-compilation is not hardware performance evidence. Reuse the stable output directory after preserving required receipts and removing inactive outputs.

Native Go assembly bridges

build/bin/ziran run --target=c bench/bench.zi -- go-assembly \
  --before-bin-dir build/go-assembly/before/bin \
  --out build/go-assembly/benchmark

Preserve the prior compiler binaries before rebuilding the modified emitter. This harness measures the same dependent xorshift/popcount workload as cpu_features.py, using function and block assembly around the whole loop, then a local assembly block per operation. It compares the prior emitter, current default Go, current CGO_ENABLED=0 Go, native cgo with -tags=ziran_asm, and generated native/forced-portable C. Source and saved IR must produce identical Go companions. A runtime field records whether the Go native dispatcher was installed; every result is checked with the independent integer oracle.

The native bridge is opt-in because per-call cgo overhead can outweigh the instruction savings. Compare all three placements before choosing a native build. Kernel timers exclude startup; CPU time covers the kernel and peak child RSS includes the runtime. Keep raw shuffled samples and warmups, commands, tool/artifact/source hashes and individual build observations. These warm-cache, shared-host synthetic measurements do not establish whole application speed, Go compilation parity or universal memory improvements. Reuse the stable output only after preserving needed receipts and confirming that its binaries and build outputs are inactive.

Bundle startup and real-package checks

Host embedding has a separate capability-scaling benchmark:

build/bin/ziran run --target=c bench/bench.zi -- host-capabilities \
  --before-dir build/host-capabilities/before --out build/benchmarks/host-capabilities

Preserve zi2zx and libziran.a before changing the runtime. The default uses 1, 16, 64 and 256 host capabilities, nine paired process samples and nine fresh instances after discarded process/instance warmups. Each instance executes 32 rounds of every capability. Callbacks check their names, contexts, arguments and order; results must match a separate Python integer recurrence. Both compilers must generate identical bundle bytes. Receipts separate checked load, capability enumeration, binding/initialization, execution and close CPU/wall times. Optional glibc allocator in-use deltas are affected by caching and are not total live memory or RSS. Results cover these synthetic fixtures on one shared host.

For the public embedding API, compare one checked BundleOpen followed by independent BundleInstantiate/run/close operations:

build/bin/ziran run --target=c bench/bench.zi -- bundle-instances \
  --before-dir build/bundle-runtime/before --out build/benchmarks/bundle-instances

The preserved directory contains zi2zx and libziran.a. Scalar, 256-member enum and 64-module fixtures use independent Python integer oracles. Both compilers must produce identical bundle bytes. Nine interleaved process samples per version follow a process warmup; each process measures 49 fresh instances after an instance warmup. The receipt separates load, instantiation, execution and close CPU/wall times, and first-instance costs. Small phases include clock overhead; whole-process wall time includes startup. These synthetic results on one shared host do not establish general execution speed or memory savings. The raw-IR runner below always verifies a caller-owned program on instance open.

bench/enum_values.py generates enum-heavy fixtures with independent integer sum oracles, verifies source/saved bundle identity, and runs the comparison below. For example:

build/bin/ziran run --target=c bench/bench.zi -- enum-values \
  --before-dir build/compiler-check/before --out build/benchmarks/enum-values

Use --project NAME ROOT ENTRY to include real package checks. Compiler profiles count enum_value_parse and enum_values_reuse. The runtime reuses immutable evaluated bodies keyed by body text and backing type, including invalid bodies; each lookup still requires the complete enum to be valid. Snapshots live until process exit and add retained metadata, so fewer evaluations do not by themselves prove a memory reduction. Reuse the stable output after preserving receipts and checking that old outputs are inactive.

bench/vm_startup.py compares two preserved compiler/runtime builds. Before changing the compiler, keep its zi2zir, zi2zx, optional ziran launcher, and libziran.a in a stable baseline directory. Do not replace that baseline while a comparison is running. The current tools default to build/bin, with the current library in build/libziran.a.

For example, using fixtures generated by compiler_scaling.py with its default 16 fields:

build/bin/ziran run --target=c bench/bench.zi -- vm-startup \
  --before-dir build/vm-performance/before \
  --out build/vm-performance/comparison \
  --case modules128 \
    build/benchmarks/compiler-scaling/modules-128/source/program.zi \
    build/benchmarks/compiler-scaling/modules-128/source/program.zx 130048

Each --case names a source file, its bundle, and an independently derived integer result. Both bundlers must recreate exactly the supplied bundle before measurement. Every first and repeated VM execution must return that result. The C runner measures checked bundle loading, instance opening, first execution, repeated execution in the same instance, and closing separately. Loading retains structural, canonical, semantic, and law validation; opening retains VM verification and global initialization. Execution has a million-statement limit. Cases must return the same expected result across repeated instance runs.

Add --project NAME /absolute/project/root relative/entry.zi to compare complete package checks through each build's sibling ziran launcher. These checks use --project --offline, preserve the project's lock and local overrides, and record compiler profiles separately from timing samples. The project sources and revision must remain stable during measurement. Package cases can also run without VM cases. They measure checking, not complete native application builds.

Results include source/tool/library/runner hashes, commands and working directories, warmups, shuffled before/after order, raw timings, CPU time, peak RSS, and live VM value bytes. The latter excludes IR and metadata caches; peak RSS is process memory, not live allocated bytes. GNU/BusyBox time avoids attributing Python's inherited peak memory to the measured process when available. Whole-process timing includes that launcher. These are warm-cache, single-host comparisons without CPU isolation; they do not establish cold-build speed, a language ranking, or performance on other hardware. Preserve needed receipts and remove inactive outputs before reusing the stable output path.

Assembly and portable fallback

build/bin/ziran run --target=c bench/bench.zi -- assembly \
  --out build/benchmarks/assembly

The byte-swap recurrence compares source and saved IR with native assembly, forced Ziran fallback, handwritten C and Go, and the .zx fallback. Every sample is checked against a separate Python oracle; emitted source and bundles must agree between source and saved IR. On x86_64 the harness also checks for a bswap instruction in the native binary. The fixture includes an AArch64 variant, but measurements prove only the recorded host's behavior.

Raw samples include wall time, user/system CPU time, process peak RSS, discarded warmups, commands, and output. Metadata contains tool versions, compiler and fixture hashes, and limitations. When GNU time or BusyBox time is available, a small launcher avoids counting Python's inherited high-water mark as the native program's memory. Metadata identifies the fallback to raw wait4 otherwise. RSS is not live allocated bytes, and wall/CPU times include that launcher. Runtime order is shuffled. Go build timings use a warm cache after the warmup. Compile phases are separate; these are not end-to-end build measurements or a language ranking. Reuse the stable output path after preserving needed receipts and removing inactive outputs.

For compiler scaling, module graphs, generic specialization, and source/saved IR validation, run:

build/bin/ziran run --target=c bench/bench.zi -- compiler-scaling \
  --out build/benchmarks/compiler-scaling --sizes 8 32 128

The harness validates every C and portable result against an independent checksum and records checking, saved IR, C emission, native compilation, and execution separately. --project /absolute/path/to/entry.zi adds a real package check using its package map. Results include source and binary hashes, raw samples, a discarded warmup, tool versions, cache caveats, and compiler profiles. Choose a new output directory for every run.

Set ZIRAN_PROFILE=/absolute/path/to/profile.jsonl for any compiler command to append a process profile without changing ordinary output or JSON diagnostics. Profiles contain inclusive parse/load/check/emission/IR-write times, record field parsing and snapshot-reuse counts, peak RSS, and allocation-request counts for compiler workspace allocations through AllocateOrExit. These allocations are a subset, not total allocations or currently live bytes. Nested phase times overlap and must not be summed into a claimed build duration; the harness measures whole commands separately.

Profiles also count kept_text_pointer, kept_text_compare and kept_text_lookup. A small per-thread cache recognizes canonical immutable pointers directly. Other input addresses are hints: their current bytes, including the terminator, must match the immutable saved copy before reuse. The cache has 128 entries (3 KiB per thread on 64-bit hosts). Saved bundles and generated source retain the same contents. Use tests/kept_text.sh to check mutable buffers, substring identity, truncation, table growth and concurrent reuse.

The October 3 field-snapshot comparison used module/generic sizes 8 and 32, three measured repetitions, and checks of the real Kryon and Terminal packages (162 validated measured samples per run). Kryon's field-parser calls fell from 2,056,209 to 2,883 and Terminal's from 2,747,749 to 3,839. Whole-command check medians were 515/537 ms for Kryon and 813/849 ms for Terminal (before/after), so this run establishes elimination of repeated parsing, not a wall-clock speedup. The machine was shared and both metadata records identify dirty working trees; each records the actual tool hashes. Local raw samples are under build/benchmarks/compiler-fields-before-fixed and build/benchmarks/compiler-fields-after. Repeat on a stable machine before using these timings as a regression threshold.

Cross-target smoke benchmark

From the repository root, build the current toolchain and run:

env -u DISPLAY -u WAYLAND_DISPLAY make -j4 all
build/bin/ziran run --target=c bench/bench.zi -- smoke

The script creates a new directory under build/benchmarks/ with metadata.json, samples.jsonl, summary.json, and benchmark.md. Pass --out PATH to select a new output directory. --compile-repetitions and --run-repetitions adjust the measured sample counts; each case also has one discarded warmup. The run requires GCC, G++, Go and the Ziran tools. Rust, Java and Node comparisons are included when their toolchains are present, and absences are recorded.

One integer recurrence is built from source and saved IR to C, Go, and portable .zx. The harness checks that generated source and bundles match, and validates every runtime result against an independent modular-series oracle before reporting times. It separately measures Ziran check/emission, downstream native compilation, and whole-process execution. Runtime cases are shuffled with a fixed seed between sample rounds. The portable engine runs only the small input because its instruction budget excludes the large one.

These numbers are a smoke baseline, not a language ranking. They include process startup; Go build measurements use a warm build cache and are labeled accordingly. JIT languages are launched fresh rather than timed at warmed steady state. The output records compiler revisions, source and tool hashes, tool versions, commands, fixture hashes, raw timings, and limitations so that a comparison can be checked against the exact experiment. A broader suite still needs generics, larger module graphs, more compiler-scaling cases, and real applications.

For a second workload that grows a Vec(s32) and then scans it, run:

build/bin/ziran run --target=c bench/bench.zi -- vector-growth

This measures source and saved-IR checks and emissions to C, Go, and portable .zx, downstream native builds, and whole-process execution. Handwritten C, C++, Go, Rust, Java, JavaScript, and Python versions provide algorithm-level comparisons when their toolchains are installed; unavailable tools are recorded explicitly. The checksum uses a closed-form arithmetic oracle, and every runtime sample must match it. The portable engine runs the small input only. Raw commands, samples, source and tool hashes, and caveats are written alongside the report. These workloads remain a smoke and collection baseline; they do not cover text, records, generics, larger module graphs, or real applications.

For a record-value and call workload over a fixed [32]Point table, run:

build/bin/ziran run --target=c bench/bench.zi -- record-calls

Each round updates all records through a checked Mix(Point, Point, s32, s32) call and accumulates a signed 64-bit checksum with wrapping s32 field arithmetic. The harness measures source and saved IR through C, Go, and portable .zx, downstream native builds, and whole-process execution. Handwritten C, C++, Go, Rust, Java, JavaScript, and Python versions participate when installed. Every sample must match an independent Python oracle; source and saved-IR generated output and bundles are checked for equality. The VM runs the small input only because the large input exceeds its instruction budget.

For a focused before/after comparison of two validated Go binaries, run bench/vector_growth_ab.py --before PATH --after PATH --out NEW.json. It shuffles execution order, checks every checksum, and records raw samples and binary hashes.

For a UTF-8 scanning workload over aé中🙂, run:

build/bin/ziran run --target=c bench/bench.zi -- text-scan

The text has 10 bytes and 4 Unicode scalar values; every runtime sample must return 148921 * rounds. The harness compares Ziran source and saved IR to C, Go, and .zx, plus handwritten C, C++, Go, Rust, Java, JavaScript, and Python when installed. It validates generated-source and bundle equality and records unavailable tools rather than estimating them.