Skip to content

Add a way for the test suite to test the output of a stable rustc #123404

Closed
Closed
Add a way for the test suite to test the output of a stable rustc#123404
Labels
A-test-infraArea: test infrastructure (may span bootstrap/compiletest/more)A-testsuiteArea: The testsuite used to check the correctness of rustcC-feature-requestCategory: A feature request, i.e: not implemented / a PR.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

Description

@estebank
estebank
opened on Apr 3, 2024

Right now ui tests only target the output of the currently built rustc. As far as I can tell, there's no -Z flag to tell rustc "act as if you were a stable release". This means that when we customize the diagnostic output based on release channel, we cannot test what stable users will actually see. An example of this is this (as of now unmerged) change that on stable refers to unsized locals and unsized fn arguments with the same note telling the user that's not allowed, while on nightly it tells the user which feature flag is gating that being allowed, without the note stating it's not allowed. That kind of behavior is replicated in multiple diagnostics involving nightly features already. It's the same "problem" we'd have with editions if we didn't already have a stable flag to specify them.

Activity

  1. estebank
    added
    A-testsuiteArea: The testsuite used to check the correctness of rustc
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
    C-feature-requestCategory: A feature request, i.e: not implemented / a PR.
    on Apr 3, 2024
  2. rustbot
    added
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Apr 3, 2024
  3. onur-ozkan
    removed
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Apr 3, 2024

  4. matthiaskrgr commented on Apr 3, 2024

    @matthiaskrgr
    matthiaskrgr
    on Apr 3, 2024
    Member

    Another usecase would be to use something like
    RUSTC_STABLE=1 rustc ice.rs to check if an ICE would happen on $todays nightly in several months after promotion to beta/stable has happened already. 🙂

  5. Mark-Simulacrum commented on Apr 3, 2024

    @Mark-Simulacrum
    Mark-Simulacrum
    on Apr 3, 2024
    Member

    RUSTC_BOOTSTRAP=0 feels like a reasonable mechanism/extension to me.

  6. estebank commented on Apr 4, 2024

    @estebank
    estebank
    on Apr 4, 2024
    ContributorAuthor

    I'd be concerned with people having set RUSTC_BOOTSTRAP=0 globally in their machines and suddenly having issues with nightly, but at the same time "setting that flag has no stability guarantees".

  7. jieyouxu
    self-assigned this
    on Nov 13, 2024
  8. jieyouxu
    linked a pull request that will close this issueMake rustc consider itself a stable compiler when `RUSTC_BOOTSTRAP=-1` #132993on Nov 13, 2024

  9. jieyouxu commented on Nov 13, 2024

    @jieyouxu
    jieyouxu
    on Nov 13, 2024
    Member

    FWIW I went with RUSTC_BOOTSTRAP=-1 to mean "force-stable" in #132993 instead of RUSTC_BOOTSTRAP=0 which can seem like "not bootstrap" which is not quite the same I feel like, and probably confusing.

    Implementation steps

    UI tests can use something like

    //@ rustc-env:RUSTC_BOOTSTRAP=-1
//@ only-nightly

    to fool convince the unstable compiler under test to think it's very stable. We can of course also add a //@ pretend-stable-rustc but yeah.

  10. bjorn3 commented on Nov 13, 2024

    @bjorn3
    bjorn3
    on Nov 13, 2024
    Member

    There is -Zallow-features= to disable support for all #![feature], but this doesn't affect commandline flags and doesn't work on stable either.

  11. jieyouxu
    removed a link to a pull requestMake rustc consider itself a stable compiler when `RUSTC_BOOTSTRAP=-1` #132993on Nov 13, 2024
  12. jhpratt
    added a commit that references this issue on Nov 18, 2024

    Rollup merge of rust-lang#132993 - jieyouxu:i_am_very_stable, r=cheny…

    c68fef9
  13. rust-timer
    added a commit that references this issue on Nov 18, 2024

    Unrolled build for rust-lang#132993

    d8f0381
  14. jieyouxu
    added
    A-test-infraArea: test infrastructure (may span bootstrap/compiletest/more)
    on Nov 18, 2024

  15. jieyouxu commented on Nov 18, 2024

    @jieyouxu
    jieyouxu
    on Nov 18, 2024
    Member

    Update: #132993 has now merged to support RUSTC_BOOTSTRAP=-1 to mean "pretend I am a stable compiler".

  16. jieyouxu
    removed their assignment
    on Nov 18, 2024

  17. jieyouxu commented on Jan 2, 2025

    @jieyouxu
    jieyouxu
    on Jan 2, 2025
    Member

    For future reference, while you can force a given rustc (well, later than #132993 of course) to pretend to be stable for all intents and purposes, there will still be an important caveat:

    -> real stable rustc    <- actually stable
|
-> beta rustc
|
-> nightly rustc
|
-> `master` rustc       <- pretend to be stable

    the master or nightly or beta rustc will almost certainly contain changes that are not present in the actual stable rustc (unavoidable), which is something important to keep in mind.

    That being said, I think we can consider this feature request to be sufficiently satisfied via RUSTC_BOOTSTRAP=-1.

  18. jieyouxu
    closed this as completedon Jan 2, 2025

  19. Kobzol commented on Oct 2, 2026

    @Kobzol
    Kobzol
    on Oct 2, 2026
    Member

    This didn't actually enable testing stable output in UI tests, because compiletest unconditionally passes certain -Z flags, and that breaks if you truly make rustc act like stable. We needed -2 all along (#163664), which acts as stable, but allows unstable compiler flags 😆

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-test-infraArea: test infrastructure (may span bootstrap/compiletest/more)A-testsuiteArea: The testsuite used to check the correctness of rustcC-feature-requestCategory: A feature request, i.e: not implemented / a PR.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions