takana/scripts/rust-frontier

rust-frontier — self-hosting the builder's rust toolchain

The last frontier of selfhost-verify variante (b): replace the Alpine rustc/cargo 1.91.1 in /toolchain with a hammer-built rust, via the purist mrustc chain (no prebuilt rustc binary). Path:

mrustc → rustc 1.90.0 (builds bundled LLVM) → 1.91.0 → 1.91.1

These scripts + patch are the working artifacts. The mrustc tree itself lives in .scratch/mrustc (gitignored — it's a large third-party clone, upstream master be69c74); only our deltas are tracked here.

Files

• run_rustc.patch — patch against mrustc be69c74 for run_rustc/Makefile and run_rustc/rustc_proxy.sh. Apply from the mrustc root: git -C .scratch/mrustc apply /path/to/run_rustc.patch
• build-stage0-musl.sh — rebuild the mrustc stage0 rustc+cargo with a musl host triple (output-1.90.0-x86_64-unknown-linux-musl/). Run inside the Alpine devfs sandbox (gcc, MRUSTC_TARGET_VER=1.90). Reuses the already-built LLVM and bin/mrustc.
• run-runrustc.sh — run a run_rustc make target inside the devfs sandbox, pointed at the musl-host stage0.
• bootstrap.toml — rust's own bootstrap config for the climb hop (1.90.0 → 1.91.0): build/host/target = x86_64-unknown-linux-musl, stage0 rustc/cargo = the run_rustc musl-host 1.90.0 prefix, external LLVM 20.1.8 reused via llvm-config (clears bootstrap's >=19 gate, no LLVM rebuild), crt-static = false (dynamic link against the devfs system musl, like Alpine and the run_rustc toolchain), rpath = true. Copy it into the rust source root.
• run-xpy.sh — run rust's x.py inside the devfs sandbox, binding the source at /src, the install prefix at /out, and the mrustc tree at /mrustc (so the stage0 + llvm-config paths in bootstrap.toml resolve). Usage: run-xpy.sh <SRCDIR> <PREFIXDIR> -- build --stage 2 compiler/rustc library/std.

The runners hardcode /home/sergio/hammer and assume the prepared devfs at .dev-fs/alpine (256-TLS-key musl loader + g++; see the bootstrap-devfs notes).

Why the patches exist (the snags)

The lab is musl-only (Alpine devfs); the 4/4 inputs (arje-zero/hammerd) are built --target x86_64-unknown-linux-musl. Three coupled problems had to be fixed to bootstrap a usable rust there:

1. stage-1 std built for gnu (mmap64/open64 undefined at link). minicargo was invoked without --target, so it used mrustc's default gnu triple — and gnu std references glibc's LFS64 symbols that musl 1.2.5 doesn't export. Fix: pass --target $(RUSTC_TARGET) to minicargo (Makefile:162) and to the stage-1 smoke-test (Makefile:172). A correct musl std uses the base libc symbols (open/mmap/…), matching Alpine's reference std.

2. self-contained CRT absent. musl's default link is -static-pie against a lib/rustlib/<t>/lib/self-contained/ CRT dir that the mrustc sysroot never populates (also needs static -lunwind). Fix: -C target-feature=-crt-static -C link-self-contained=no so the Alpine gcc supplies crt1/crti/crtn + crtbeginS/crtendS and links dynamically against the system musl (RUSTFLAGS in Makefile:111, plus both hello_world smoke-test rules: the stage-1 one at Makefile:172 and the final fully-bootstrapped one at Makefile:262 — the latter only surfaces once stage-2/3 actually completes, i.e. after snag #3 is fixed).

3. host-triple = gnu on a musl system (the deep one). cargo compiles build-scripts and proc-macros for the host; mrustc's stage0 rustc has its host triple baked as x86_64-unknown-linux-gnu (not runtime-overridable). A proxy --target musl inject fixes build-script bins, but proc-macros are dlopen'd and must match the host triple structurally — injection makes them target artifacts that rustc rejects. Aliasing musl rlibs as gnu also fails: rustc reads the triple from rlib metadata (E0461), not the path.

   The only fix is a stage0 rustc whose host triple is musl — that's what build-stage0-musl.sh produces (CFG_COMPILER_HOST_TRIPLE via RUSTC_TARGET, --target via MRUSTC_TARGET). Then host==target==musl: cargo doesn't cross-compile, proc-macros load natively, and rustc_proxy.sh only needs the CRT flags (no --target inject).

   Gotcha: do not pass OUTDIR_SUF on make's command line — a command-line var can't be reassigned by the makefile, so the -musl suffix append is skipped and the build contaminates the gnu output dir. Let it default.

4. final cargo needs vendored OpenSSL. The fully-bootstrapped prefix/bin/cargo rule (Makefile:235) builds cargo with default features, so openssl-sys looks for a system OpenSSL via pkg-config and fails (the hermetic musl devfs has none, and no openssl.pc). Fix: add --features vendored-openssl to that cargo build so openssl-src compiles OpenSSL from source — same approach the stage0 cargo build uses. Real cargo (unlike minicargo) handles the OUT_DIR fine, so snag #4 doesn't recur here. Only surfaces after the final hello_world (snag #2/262) passes.

5. minicargo OUT_DIR host/target split (only when building the musl stage0's cargo). With RUSTC_TARGET set, minicargo runs each cargo build-script under cargo-build/host/build_X/, but the target crate's include! / include_bytes!(concat!(env!("OUT_DIR"), "/..")) resolves OUT_DIR to cargo-build/build_X/ (no host/, empty) → mrustc aborts with signal 6. Hit libsqlite3-sys (bindgen.rs) and the cargo binary (man.tgz). The gnu build never split (single triple → single dir). Since host==target the script outputs are bit-identical, so build-stage0-musl.sh symlinks the target dir to the host dir (symlink_outdirs). Late sys-crates (cargo, curl, openssl, libgit2-sys, libssh2-sys) run their scripts near the end, so it's applied in a retry loop around the incremental cargo build.

Workflow

# 1. apply the run_rustc patch into the mrustc clone
git -C .scratch/mrustc apply scripts/rust-frontier/run_rustc.patch

# 2. rebuild stage0 rustc+cargo with a musl host triple (~hours, reuses LLVM)
.scratch/build-stage0-musl.sh            # -> output-1.90.0-x86_64-unknown-linux-musl/

# 3. build the optimised rustc + cargo via run_rustc (stage 1/2/3)
.scratch/run-runrustc.sh all

# 4. (next) climb 1.90.0 → 1.91.0 → 1.91.1, then swap rust into /toolchain

Full diagnostic history is in the selfhost-verify-rust-frontier memory.