Testing and CI

Purpose and Scope

The Rust repository has many test suites because it is not a single application: it is the source tree for the compiler, standard libraries, documentation tooling, editor tooling, and alternative code generation backends. This page explains the test organization represented by the repository documentation for LLVM codegen tests, codegen-unit partitioning tests, crash tests, Cranelift backend testing, and rust-analyzer testing. The goal is to help contributors choose the right test style before they add a regression test, update expected output, or validate a change against local compiler artifacts.

The word "test" has different meanings across this tree. Some tests compare generated LLVM text with FileCheck directives, some inspect compiler-internal mono item partitioning, some intentionally reproduce compiler crashes, and rust-analyzer uses snapshot-style assertions against IDE analysis output. These suites are complementary rather than interchangeable. A compiler backend change normally needs output-sensitive tests, an incremental compilation or partitioning change may need CGU assertions, a diagnostic-quality bug may need compiletest coverage, and an editor-analysis change often needs an expect-test update. Sources: tests/codegen-llvm/README.md, tests/codegen-units/partitioning/README.md, tests/crashes/README.md, src/tools/rust-analyzer/docs/book/src/contributing/testing.md

CI is not described in these snippets as one monolithic pipeline, but the documented suites reveal the CI contract: tests should be deterministic, machine-readable where possible, and precise enough to catch regressions in the subsystem they target. The official Rust Reference test summary reinforces that Rust documentation and implementation are connected by traceable tests, but the repository-level evidence here is more operational: it tells contributors what input format, expected-output mechanism, and update workflow each area expects. That is the practical information needed when preparing a pull request.

Relevant Source Files

• tests/codegen-llvm/README.md — documents LLVM FileCheck-based codegen tests and the revision-prefix extension used to test target- or flag-dependent output.
• tests/codegen-units/partitioning/README.md — explains codegen-unit partitioning tests, their use of -Zprint-mono-items, and the scope limits of the suite.
• tests/crashes/README.md — points contributors to crash-test documentation in the rustc-dev-guide and shows how to serve that guide locally from this repository.
• compiler/rustc_codegen_cranelift/docs/rustc_testing.md — describes how to build and test the Cranelift backend against local changes in the main Rust repository.
• src/tools/rust-analyzer/docs/book/src/contributing/testing.md — explains rust-analyzer snapshot testing, helper functions such as check_no_mismatches, check_types, and check_infer, and expectation updates with UPDATE_EXPECT=1.

Test Suite Map

LLVM codegen tests are output-oriented. Files under tests/codegen-llvm use the LLVM FileCheck framework, so they are best suited for asserting that a compiler change produces or does not produce particular LLVM output. The README calls out an important Rust-specific extension: revisions can become custom FileCheck prefixes. That lets one test file express multiple compiler invocations, for example different targets or different compiler flags, and then attach expected or forbidden output to a specific revision prefix. Sources: tests/codegen-llvm/README.md

This revision model matters because backend behavior often varies without changing the high-level Rust program. A single source file can exercise a default case and a flag-specific case while keeping the semantic fixture in one place. The documented pattern combines // revisions: aaa bbb, revision-scoped compile flags such as // [bbb] compile-flags: --flags-for-bbb, and FileCheck directives such as aaa-SAME, aaa-NOT, bbb-NOT, and bbb-SAME. Contributors should use this shape when the distinction under test is compiler configuration rather than a different Rust example.

Codegen-unit partitioning tests have a narrower purpose. The README explicitly says the suite tests that codegen unit partitioning works as intended, not whether the partitioning is good. That distinction is important for reviewer expectations: these tests validate the partitioning algorithm's observable decisions, while benchmark suites evaluate quality. The suite uses -Zprint-mono-items, which prints a machine-readable summary of collected MonoItems, the CGUs they were assigned to, and their linkage in each CGU. Sources: tests/codegen-units/partitioning/README.md

The partitioning suite is shaped by the implementation it observes. The documented current algorithm groups MonoItems by the module in which they are defined and then merges small CGUs, so the tests include many inline modules because that is the way to make partitioning visible. The README also notes a historical bias toward incremental builds with -Copt-level=0, because the suite was added during incremental compilation work and because CGUs are the unit of incrementality for the codegen backend. That context helps contributors avoid overgeneralizing test results.

Crash tests are documented differently because the local README is a pointer to the rustc-dev-guide's compiletest documentation. The file under tests/crashes tells contributors where the canonical crash-test guidance lives and gives a local documentation workflow. If you need to work offline or confirm the current checked-out guide content, build and serve the guide from the repository with the documented command. Sources: tests/crashes/README.md

mdbook serve --open src/doc/rustc-dev-guide

That command serves the guide locally and makes the crash-test section available through the compiletest page. The important operational lesson is that crash tests are part of the compiler test infrastructure rather than ad hoc reproducer files. When a change fixes an internal compiler error or another compiler crash, contributors should use the crash-test guidance to encode the reproducer in the form expected by compiletest, so the regression remains visible to local test runs and CI.

Backend-Specific Testing: Cranelift

The Cranelift backend documentation covers a workflow for testing backend changes that depend on local rustc changes. This is useful when modifying rustc_codegen_cranelift as part of broader compiler work, such as implementing a new compiler intrinsic. The workflow begins from a Rust checkout, uses python x.py setup with the compiler option, builds a stage2 compiler and required tools, links the resulting toolchain with rustup, and then runs the Cranelift build system through that linked stage2 compiler. Sources: compiler/rustc_codegen_cranelift/docs/rustc_testing.md

The sequence is deliberately explicit because backend testing can accidentally use the wrong compiler. The documented build command is python x.py build --stage=2 compiler library/std src/tools/rustdoc src/tools/rustfmt, with Cargo as an optional additional tool. The instructions then copy Cargo from a nightly toolchain into the stage2 bin directory, link the stage2 toolchain with rustup toolchain link stage2 ./build/host/stage2/, and require each ./y.sh command to be prefixed with rustup run stage2. This ensures cg_clif sees the local rustc changes.

cd $RustCheckoutDir
python x.py setup
python x.py build --stage=2 compiler library/std src/tools/rustdoc src/tools/rustfmt
rustup toolchain link stage2 ./build/host/stage2/
rustup run stage2 ./y.sh prepare
rustup run stage2 ./y.sh build
rustup run stage2 ./y.sh test

After that build, the documentation says the resulting cargo-clif can compile other Rust programs, for example by opening another crate and running a build with the distributed Cranelift cargo wrapper. This is a system-level test pattern rather than a single fixture: the goal is to validate that a locally changed compiler and backend can build real crates together. The same document also notes that rust-analyzer.rustc.source can point to the Rust workspace so rust-analyzer understands local compiler changes.

rust-analyzer Snapshot Testing

rust-analyzer uses snapshot tests: a test provides input text, usually Rust code, and compares feature output against expected output. The documentation names expect-test and a custom testing framework as the basis for this style. This is a different testing model from LLVM FileCheck, but the shared theme is explicit expected output. Snapshot tests make IDE behavior reviewable because inference results, diagnostics, assists, hover data, or other feature outputs can be stored as text and updated intentionally. Sources: src/tools/rust-analyzer/docs/book/src/contributing/testing.md

The rust-analyzer guide explains type inference tests as a concrete example. Tests in crates/hir-ty/src/tests can use helpers such as check_no_mismatches() to assert that rust-analyzer's own type analysis does not report mismatches. The guide emphasizes that these mismatches are determined by rust-analyzer, not by rustc, because rust-analyzer's analysis is what the tests are exercising. That distinction matters when a test misses a compiler error or reports a bogus one: the snapshot is validating language-server behavior, not delegating correctness to the compiler.

More precise tests can use annotations embedded in Rust source strings. The guide defines $0 as a position marker, $0...$0 as a range marker, and caret labels such as ^...^ in comments as labels for the line above. The check_types() helper uses those labels to assert inferred types for ranges. This convention is common in editor tests because it lets a compact Rust snippet encode both source input and the editor position or selected range relevant to the feature being tested.

For larger inference output, the guide shows check_infer() with an expect! block. The expected output lists byte ranges, source snippets, and inferred types. The major maintenance feature is automatic expectation updating: set UPDATE_EXPECT=1 and rerun the test to rewrite the expected output. Some editors, including VS Code according to the guide, expose an Update Expect action next to normal test controls. Contributors should still review generated updates carefully, because an updated snapshot records changed behavior whether the change is intended or accidental.

UPDATE_EXPECT=1 cargo test -p hir-ty

Execution Flow for Contributors

Start by identifying the subsystem whose behavior changed. If the change affects emitted LLVM or backend text, prefer a tests/codegen-llvm test with FileCheck directives and revisions when one source program should cover multiple compiler configurations. If the change affects CGU assignment or linkage under partitioning, use the partitioning suite and assert the -Zprint-mono-items output rather than trying to infer behavior from performance. If the change fixes a compiler crash, follow the compiletest crash-test guide linked from tests/crashes. Sources: tests/codegen-llvm/README.md, tests/codegen-units/partitioning/README.md, tests/crashes/README.md

For backend work involving Cranelift and local compiler edits, first make sure the backend is built against the intended stage2 rustc. The Cranelift instructions are designed to avoid stale-toolchain mistakes by linking the local build as a rustup toolchain and using rustup run stage2 for backend commands. Only after that setup should you interpret ./y.sh test or downstream cargo-clif build results as evidence about your compiler changes. Sources: compiler/rustc_codegen_cranelift/docs/rustc_testing.md

For rust-analyzer work, choose the helper that matches the feature under test. Use a narrow helper such as check_no_mismatches() when the assertion is simply that analysis accepts the code, use check_types() when annotated ranges should have specific inferred types, and use check_infer() plus expect! when the full textual inference output is the contract. Update snapshots with UPDATE_EXPECT=1 only after deciding that the new behavior is correct. Sources: src/tools/rust-analyzer/docs/book/src/contributing/testing.md

CI and Maintenance Signals

The common CI signal across these files is reproducibility. FileCheck tests define expected and forbidden backend output. Partitioning tests consume a machine-readable mono-item listing. Crash tests live under compiletest guidance so a reproducer becomes a stable regression test. Cranelift testing links an explicit local stage2 toolchain to prevent accidental use of an unrelated compiler. rust-analyzer snapshots make analysis output visible in source control. These mechanisms give reviewers a concrete artifact to inspect instead of relying on manual reproduction.

A second signal is scope control. The partitioning README explicitly says it does not judge whether partitioning is good, only whether it works as intended. That statement is a model for test selection elsewhere: a test should be clear about what it proves. A FileCheck test proves text-level backend output under configured revisions; a rust-analyzer snapshot proves a feature's textual output for a given input; a crash test proves a reproducer no longer crashes. Performance, usability, and broad compatibility may require different suites or benchmarks.

When adding or updating tests, keep expected output close to the behavior being validated. Use revision-specific prefixes instead of duplicating LLVM tests for flag variants. Use inline modules in partitioning tests when module boundaries are what make CGU behavior visible. Use local rustc-dev-guide documentation for crash-test mechanics. Use rust-analyzer annotations to make cursor positions, ranges, and labels explicit in the source snippet itself. These conventions reduce reviewer effort and make future failures easier to diagnose.

Next Steps

If you are new to Rust compiler testing, start by reading the README for the directory where you plan to add a test, then run the smallest local command that exercises that directory or tool. For emitted-code regressions, learn the FileCheck prefix style before writing expectations. For partitioning regressions, inspect the MONO_ITEM format and decide which module structure exposes the behavior. For crash regressions, open the compiletest crash-test guide through the local mdbook command. For rust-analyzer changes, run the relevant test with and without UPDATE_EXPECT=1 so you understand both the failure and the generated update.

Related pages that provide broader context are building-rust-from-source for the bootstrap workflow, codegen-backends for backend architecture, rust-analyzer-overview for language-server structure, and compiler-architecture for how compiler subsystems fit together.