From fb71aca4be05bd7fb4d62a68cebca8dbee209a81 Mon Sep 17 00:00:00 2001 From: sergio Date: Tue, 16 Jun 2026 21:59:38 -0400 Subject: [PATCH] selfhost-verify: swap rust hammer-built 1.91.1 en /toolchain (SWAP_RUST, auto-consistencia) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pieza rust del auto-alojamiento "variante b" (la última del frente). A DIFERENCIA de make/busybox/bwrap/coreutils/linux-headers (herramientas que orquestan/copian ⇒ bytes idénticos ⇒ of_tree 9adefb82 por construcción), **rustc EMITE los binarios del 4/4** (arje-zero, hammerd) ⇒ un rustc distinto diverge of_tree. Criterio elegido: AUTO- CONSISTENCIA (REF recomputado con hammer-rust = nuevo EXPECT_REF; la VM reproduce ESE), no igualdad byte-a-byte con el baseline Alpine. - swap-rust-into-toolchain.sh: overlay aditivo y reversible de .scratch/rust-1.91.1-prefix sobre /toolchain. El rustlib x86_64-unknown-linux-musl de hammer y sus .so con hash propio NO chocan con el x86_64-alpine-linux-musl de Alpine ⇒ sólo se reemplazan /usr/bin/{rustc, cargo} (Alpine guardado en *.alpine; --restore deshace). Los 4/4 compilan NATIVO (target x86_64-linux-musl == SANDBOX_NATIVE_TARGET, sin --target) ⇒ hammer-rustc emite para su triple nativo x86_64-unknown-linux-musl. - selfhost-verify.sh: SWAP_RUST=1 aplica el swap en $TOOLCHAIN antes del build de stage1 (así el REF host se computa con hammer-rust) y restaura al salir (trap). Sirve TANTO al REF host COMO al toolchain in-VM (--toolchain $TOOLCHAIN). EXPECT_REF pasa a RUST_EXPECT_REF (no el 9adefb82 de Alpine). Vars: RUST_PREFIX, RUST_EXPECT_REF. VALIDADO: el swap aplica/restaura limpio; hammer-rust 1.91.1 (host x86_64-unknown-linux-musl) compila+corre un crate NATIVO (sin --target) en el devfs — el camino exacto de los 4/4. NEXT: computar el nuevo RUST_EXPECT_REF (bootstrap stage1+stage2 con SWAP_RUST) y verify in-VM. Co-Authored-By: Claude Opus 4.8 --- scripts/rust-frontier/README.md | 10 +++ .../rust-frontier/swap-rust-into-toolchain.sh | 80 +++++++++++++++++++ scripts/selfhost-verify.sh | 21 +++++ 3 files changed, 111 insertions(+) create mode 100755 scripts/rust-frontier/swap-rust-into-toolchain.sh diff --git a/scripts/rust-frontier/README.md b/scripts/rust-frontier/README.md index 324ba5a4..e058ff67 100644 --- a/scripts/rust-frontier/README.md +++ b/scripts/rust-frontier/README.md @@ -36,6 +36,16 @@ These scripts + patch are the working artifacts. The mrustc tree itself lives in OpenSSL/curl/libgit2, so this makes bootstrap build cargo with the `all-static` feature set (vendored openssl/libgit2 + static curl/libz, compiled from the vendored C sources via gcc). Without it the cargo tool fails on `openssl-sys`. +- **`swap-rust-into-toolchain.sh`** — overlay the hammer-built rust 1.91.1 + (`.scratch/rust-1.91.1-prefix`) onto a toolchain dir (default `.dev-fs/alpine`) + so `hammer build` / `bootstrap stage1` compile the 4/4 rust inputs (arje-zero, + hammerd) with hammer-rust. Additive + reversible: hammer's `x86_64-unknown-linux-musl` + rustlib and hash-suffixed `.so`s don't collide with Alpine's + `x86_64-alpine-linux-musl`, so only `/usr/bin/{rustc,cargo}` are replaced (Alpine + backed up to `*.alpine`). `--restore` undoes it. Wired into `selfhost-verify.sh` + as `SWAP_RUST=1` (applies before the stage1 build, restores on exit via trap). + **This swap changes `of_tree`** (rustc emits the binaries), so the criterion is + auto-consistency against a NEW `RUST_EXPECT_REF`, not the Alpine `9adefb82`. - **`run-xpy.sh`** — run rust's `x.py` inside the devfs sandbox, binding the source at `/src`, the install prefix at `/out`, and the mrustc tree at `/mrustc` (so the stage0 + `llvm-config` paths in `bootstrap.toml` resolve). Set the diff --git a/scripts/rust-frontier/swap-rust-into-toolchain.sh b/scripts/rust-frontier/swap-rust-into-toolchain.sh new file mode 100755 index 00000000..d09a79ad --- /dev/null +++ b/scripts/rust-frontier/swap-rust-into-toolchain.sh @@ -0,0 +1,80 @@ +#!/usr/bin/env bash +# swap-rust-into-toolchain.sh — overlay the hammer-built rust 1.91.1 onto a toolchain +# dir (default .dev-fs/alpine), so `hammer build` / `bootstrap stage1` compile the +# 4/4 rust inputs (arje-zero, hammerd) with hammer-rust instead of Alpine's rustc. +# +# This is the rust tool-swap of the selfhost-verify "variante b". Unlike make/bwrap/… +# (tools that orchestrate/copy → identical bytes), **rustc emits the 4/4 binaries**, so +# a hammer-built rustc produces a DIFFERENT of_tree than Alpine's 9adefb82. Success +# criterion is therefore AUTO-CONSISTENCY: stage1' built with hammer-rust == stage1'' +# rebuilt with hammer-rust (a NEW EXPECT_REF), not byte-identity with the Alpine baseline. +# +# The overlay is mostly ADDITIVE and fully reversible: +# - hammer's rustc/cargo use host triple x86_64-unknown-linux-musl and hash-suffixed +# .so names that DON'T collide with Alpine's (x86_64-alpine-linux-musl / other +# hashes), so the new rustlib triple + librustc_driver just sit alongside Alpine's. +# - only /usr/bin/{rustc,cargo} are REPLACED; the originals are backed up to +# usr/bin/.alpine so `--restore` can put them back. +# The 4/4 rust recipes build NATIVE (target x86_64-linux-musl == SANDBOX_NATIVE_TARGET +# ⇒ no --target, see hammer-build/src/lib.rs:269), so hammer-rustc compiles them for +# its OWN native triple x86_64-unknown-linux-musl. +# +# Usage: +# swap-rust-into-toolchain.sh [--prefix DIR] [--toolchain DIR] # apply the swap +# swap-rust-into-toolchain.sh --restore [--toolchain DIR] # undo it +set -euo pipefail +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +PREFIX="$ROOT/.scratch/rust-1.91.1-prefix" +TOOLCHAIN="$ROOT/.dev-fs/alpine" +RESTORE=0 +while [ $# -gt 0 ]; do + case "$1" in + --prefix) PREFIX="$2"; shift 2 ;; + --toolchain) TOOLCHAIN="$2"; shift 2 ;; + --restore) RESTORE=1; shift ;; + *) echo "uso: $0 [--prefix DIR] [--toolchain DIR] [--restore]" >&2; exit 2 ;; + esac +done + +say() { printf '\033[1;36m==> %s\033[0m\n' "$*"; } +die() { printf '\033[1;31mERROR: %s\033[0m\n' "$*" >&2; exit 1; } + +[ -d "$TOOLCHAIN/usr/bin" ] || die "no veo el toolchain en $TOOLCHAIN" + +if [ "$RESTORE" = 1 ]; then + say "restaurar el rust de Alpine en $TOOLCHAIN" + for b in rustc cargo; do + if [ -f "$TOOLCHAIN/usr/bin/$b.alpine" ]; then + mv -f "$TOOLCHAIN/usr/bin/$b.alpine" "$TOOLCHAIN/usr/bin/$b" + echo " restaurado usr/bin/$b" + else + echo " (sin backup de usr/bin/$b — ¿ya restaurado?)" + fi + done + echo " (los .so y el rustlib x86_64-unknown-linux-musl añadidos quedan; son inertes — el rustc de Alpine no los usa)" + exit 0 +fi + +[ -x "$PREFIX/bin/rustc" ] || die "no veo el prefix hammer-rust en $PREFIX (corré el climb primero)" + +say "swap: overlay hammer-rust 1.91.1 ($PREFIX) → $TOOLCHAIN" + +# 1) librustc_driver de hammer (hash único, no choca con el de Alpine). +cp -a "$PREFIX"/lib/librustc_driver-*.so "$TOOLCHAIN/usr/lib/" +echo " + usr/lib/$(basename "$PREFIX"/lib/librustc_driver-*.so)" + +# 2) sysroot de hammer para su triple nativo (dir nuevo, no choca con x86_64-alpine-linux-musl). +rm -rf "$TOOLCHAIN/usr/lib/rustlib/x86_64-unknown-linux-musl" +cp -a "$PREFIX/lib/rustlib/x86_64-unknown-linux-musl" "$TOOLCHAIN/usr/lib/rustlib/" +echo " + usr/lib/rustlib/x86_64-unknown-linux-musl ($(du -sh "$PREFIX/lib/rustlib/x86_64-unknown-linux-musl" | cut -f1))" + +# 3) reemplazar rustc y cargo (backup de los de Alpine). +for b in rustc cargo; do + [ -f "$TOOLCHAIN/usr/bin/$b.alpine" ] || cp -a "$TOOLCHAIN/usr/bin/$b" "$TOOLCHAIN/usr/bin/$b.alpine" + cp -a "$PREFIX/bin/$b" "$TOOLCHAIN/usr/bin/$b" + echo " ~ usr/bin/$b (Alpine guardado en usr/bin/$b.alpine)" +done + +say "listo. verificá con: bwrap --bind $TOOLCHAIN / ... /usr/bin/rustc --version" +echo " esperado: rustc 1.91.1 (ed61e7d7e …) host x86_64-unknown-linux-musl" +echo " deshacer: $0 --restore --toolchain $TOOLCHAIN" diff --git a/scripts/selfhost-verify.sh b/scripts/selfhost-verify.sh index 6bf24f91..0e64072a 100755 --- a/scripts/selfhost-verify.sh +++ b/scripts/selfhost-verify.sh @@ -68,6 +68,27 @@ SEED_HASH="$(ls -d "$STORE"/*-seed-zig 2>/dev/null | head -1 | sed -E 's#.*/([0- [[ "$SEED_HASH" == b3:* && ${#SEED_HASH} -eq 67 ]] || die "no encuentro la semilla sellada en $STORE" say "semilla: $SEED_HASH" +# 0c) Auto-alojamiento del COMPILADOR (variante b, pieza rust). A DIFERENCIA de make/busybox/bwrap/… +# (herramientas que ORQUESTAN o COPIAN ⇒ bytes idénticos), **rustc EMITE los binarios del 4/4** +# (arje-zero, hammerd): un rustc distinto ⇒ of_tree DIVERGE del baseline Alpine 9adefb82. Por eso +# el criterio no es igualdad con Alpine sino AUTO-CONSISTENCIA: el REF se RECOMPUTA con hammer-rust +# (nuevo EXPECT_REF) y la VM debe reproducir ESE. El swap entra en $TOOLCHAIN, que sirve TANTO al +# build host del REF (este script, abajo) COMO al toolchain in-VM del builder (--toolchain); se +# restaura al salir (trap). hammer-rust se construye con scripts/rust-frontier (climb mrustc→1.91.1). +# SWAP_RUST=1 → overlay .scratch/rust-1.91.1-prefix sobre /toolchain/usr/{bin,lib}. +# RUST_PREFIX=DIR → prefix hammer-rust alternativo (default .scratch/rust-1.91.1-prefix). +# RUST_EXPECT_REF=… → REF conocido-bueno de hammer-rust (cross-check; vacío ⇒ sólo lo imprime). +if [[ "${SWAP_RUST:-0}" == 1 ]]; then + RUST_PREFIX="${RUST_PREFIX:-.scratch/rust-1.91.1-prefix}" + [[ -x "$RUST_PREFIX/bin/rustc" ]] || die "SWAP_RUST=1 pero no veo el prefix hammer-rust en $RUST_PREFIX (corré el climb: scripts/rust-frontier/README.md)" + say "variante b (compilador) — swapear hammer-rust 1.91.1 en $TOOLCHAIN (criterio: auto-consistencia)" + scripts/rust-frontier/swap-rust-into-toolchain.sh --prefix "$RUST_PREFIX" --toolchain "$TOOLCHAIN" + trap 'scripts/rust-frontier/swap-rust-into-toolchain.sh --restore --toolchain "$TOOLCHAIN" >/dev/null 2>&1 || true' EXIT + # Con hammer-rust el REF NO es el 9adefb82 de Alpine; comparamos contra RUST_EXPECT_REF (si se + # conoce de una corrida previa), no contra el baseline Alpine. + EXPECT_REF="${RUST_EXPECT_REF:-}" +fi + # 1) Store baseline: stage1 (idempotente; rebuildea lo que falte con -mcpu=baseline ya en fuente). say "stage1 — construir y sellar el rootfs baseline" "$HAMMER" --store "$STORE" bootstrap stage1 --seed-hash "$SEED_HASH" --recipes recipes