Get rid of LLM disclosure checkboxes by jyn514 · Pull Request #160785 · rust-lang/rust

jyn514 · 2026-08-09T01:17:44Z

These had a bunch of issues:

They're extra work every time someone opens a PR
They didn't show up at all if people used gh pr create
They had "bad vibes" -- reviewers often don't want to think about LLMs, and adding a checkbox makes them very "in-your-face" for the reviewer.
Every PR now shows "1 of 2 tasks", which is useless noise.
The hovered PR description is now useless.

Replace them with an HTML comment that says "remember to disclose if you used an LLM". This seems ok and low-noise for now.
If we find that people are ignoring the comment,
we could edit the triagebot welcome message to include a reminder and a link to the policy (cc @Kobzol, i believe you'd planned to do this already).

I did not use an LLM to create a change in this PR.
I used an LLM to create a change in this PR, and I have explained below how it was used.

These had a bunch of issues: - They're extra work every time someone opens a PR - They didn't show up at all if people used `gh pr create` - They had "bad vibes" -- reviewers often don't want to think about LLMs, and adding a checkbox makes them very "in-your-face" for the reviewer. - Every PR now shows "1 of 2 tasks", which is useless noise. - The hovered PR description is now useless. Replace them with an HTML comment that says "remember to disclose if you used an LLM". This seems ok and low-noise for now. If we find that people are ignoring the comment, we could edit the triagebot welcome message to include a reminder and a link to the policy.

rustbot · 2026-08-09T01:17:49Z

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Fallback group: @Mark-Simulacrum, @jieyouxu
@Mark-Simulacrum, @jieyouxu expanded to Mark-Simulacrum, jieyouxu

asquared31415 · 2026-08-09T02:10:36Z

-If you do not check a box, a reviewer may ask you whether an LLM was involved.
+If you used an LLM to generate code, please disclose that according to our [guidelines][disclosure guidelines].
 LLM contributions are not banned, but are held to a higher standard of review and correctness.
+If you do not want your disclosure to be part of the permanent git history, add `<!-- homu-ignore:start` before it.


Is the tooling smart enough to not parse the homu-ignore when it's within a comment?

View changes since the review

Not for that reason.
This one will be ignored because there's no matching end:
https://github.com/rust-lang/bors/blob/a4b1de2d268c4b54c0830efa4f315b14e6a2335f/src/bors/mod.rs#L341

This is probably important to fix then, either in bors or in the template. I could see this interacting poorly with other ignore comments. I believe that if you had a situation of:

start (from this template)

real content

start (manual)

ignored content

end

then all of the real content would be ignored.

We could fix the implementation to ignore escaped comment blocks, but that seems like it might complicate the bors code a bit.

Or we could change the PR template to link to documentation for the ignore block?

theemathas · 2026-08-09T02:28:03Z

A potential problem is: If a PR doesn't mention LLMs, we're left guessing whether it's a human who has seen the instructions, or if it's an LLM that hasn't seen the instructions.

jyn514 · 2026-08-09T03:04:07Z

yes. that’s why the penalty is “a reviewer might ask you whether your PR is LLM-generated”, not “close the PR on sight”.

joshtriplett · 2026-08-09T03:30:43Z

-If you do not check a box, a reviewer may ask you whether an LLM was involved.
+If you used an LLM to generate code, please disclose that according to our [guidelines][disclosure guidelines].
 LLM contributions are not banned, but are held to a higher standard of review and correctness.
+If you do not want your disclosure to be part of the permanent git history, add `<!-- homu-ignore:start` before it.


Could we just drop this line instead, please? The disclosure is useful information.

View changes since the review

people should have control over what they put in a permanent record.

I think requiring the disclosure to be in the permanent record is useful.

https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/rust-forge/pull/1040#discussion_r3382826548

i don't think github has any kind of moderation controls on commits themselves, even though it lets people comment.

We can disable comments on commits entirely:
https://github.blog/changelog/2026-03-25-disable-comments-on-individual-commits/

I think this s useful technical information that's worth preserving. But 🤷 I also won't spend a lot of effort arguing about this now so fine to let it go.

nnethercote · 2026-08-09T06:05:37Z

 Please read our [LLM policy] before opening a PR,
-and check one of the boxes above to indicate whether you've used an LLM.
-If you do not check a box, a reviewer may ask you whether an LLM was involved.
+If you used an LLM to generate code, please disclose that according to our [guidelines][disclosure guidelines].


s/code/any part of this PR, including the PR description/ ?

Point being that the prohibition is on code, comments, docs, and the PR description.

View changes since the review

Also, the disclosure guidelines don't mention or link to the LLM policy. They briefly mention requiring a mentor, but the general tone makes it sound like LLM-generated contributions are by default accepted.

Oh, I see now the policy is linked to from the parent page (https://rustc-dev-guide.rust-lang.org/llm-guidance.html). But if someone doesn't navigate back to that parent page, the overall tenor of the discussion seems misleading.

let’s please not turn this into a review of the dev guide by proxy. open an issue or PR there if you want to suggest changes.

s/code/any part of this PR, including the PR description/ ?

👍 i’ll write something up

fereidani · 2026-08-09T11:14:13Z

Apologies in advance if this conversation is intended only for Rust team members,
But I'll keep the boxes for now. rustbot(or any other bot) can edit the PR message automatically remove those boxes after the PR is open and convert them to a tag like llm-used, so reviewers willing to review those PRs can be selected.

If none of the boxes are checked, the PR must be rejected or delayed-reviewer-selection by the bot with a respectful message mentioning guidelines and PR requirements.

I will add another section to the second box:

I used an LLM to create a change in this PR. I have reviewed the LLM-generated code and I fully understand what it does and what it changes. I have explained below how the LLM was used.

If someone sends a PR that is LLM-generated, they must understand the code. Otherwise they must open an issue instead. I think reviewers are capable enough to use an LLM themselves and review the code.

If the boxes are redundant work for regular contributors, make them optional for non-first-time contributors. Having a gate in the current situation is useful.

jyn514 · 2026-08-09T18:26:55Z

But I'll keep the boxes for now. rustbot(or any other bot) can edit the PR message automatically remove those boxes after the PR is open and convert them to a tag like llm-used, so reviewers willing to review those PRs can be selected.

this seems like a neat idea, but i won't have time to work on it myself.

Mark-Simulacrum · 2026-08-09T18:28:43Z

Happy to defer merging this to you whenever (and we can keep iterating obviously).

jyn514 · 2026-08-09T19:17:02Z

thanks :) i think this is a good start and i would rather iterate in follow-ups instead of delaying.

Mark-Simulacrum · 2026-08-09T19:31:45Z

@bors delegate+

rust-bors · 2026-08-09T19:31:49Z

✌️ @jyn514, you can now approve this pull request!

If @Mark-Simulacrum told you to "r=me" after making some further change, then please make that change and post @bors r=Mark-Simulacrum.

View changes since this delegation.

jyn514 · 2026-08-09T19:43:12Z

@bors r=Mark-Simulacrum rollup=always

rust-bors · 2026-08-09T19:43:15Z

📌 Commit 91f0930 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

rust-bors · 2026-08-09T19:54:22Z

⌛ Testing commit 91f0930 with merge 1efebfc...

Workflow: https://github.com/rust-lang/rust/actions/runs/31332889040

@Kobzol

Get rid of LLM disclosure checkboxes These had a bunch of issues: - They're extra work every time someone opens a PR - They didn't show up at all if people used `gh pr create` - They had "bad vibes" -- reviewers often don't want to think about LLMs, and adding a checkbox makes them very "in-your-face" for the reviewer. - Every PR now shows "1 of 2 tasks", which is useless noise. - The hovered PR description is now useless. Replace them with an HTML comment that says "remember to disclose if you used an LLM". This seems ok and low-noise for now. If we find that people are ignoring the comment, we could edit the triagebot welcome message to include a reminder and a link to the policy (cc @Kobzol, i believe you'd planned to do this already).

jhpratt · 2026-08-10T01:32:26Z

One runner was never picked up for some reason.

@bors yield

rust-bors · 2026-08-10T01:32:29Z

Auto build was cancelled. Cancelled workflows:

https://github.com/rust-lang/rust/actions/runs/31332889040

The next pull request likely to be tested is #160823.

Rollup of 11 pull requests Successful merges: - #160675 (bootstrap: Remove `PATH_REMAP` from command-line selector handling ) - #160807 (Update rustc crate rkyv to 0.8.18) - #159690 (Clarify `--remap-path-scope` impact on `rustc` metadata) - #160560 (Add nightly-only support for Cargo unremap trim-paths files in `rust-gdb`) - #160608 (normalization rework: clean up projection_ty_core) - #160785 (Get rid of LLM disclosure checkboxes) - #160804 (Change .expect message on net/parser to follow precondition style) - #160805 (`extern "custom"`: add tests) - #160816 (Mark const ptr methods and free functions as inline(always) to match *mut) - #160820 (Stabilize fs_set_times) - #160826 (Rename parse_delimited_token_tree in cfg_select)

@Kobzol

Rollup merge of #160785 - jyn514:no-checkboxes, r=Mark-Simulacrum Get rid of LLM disclosure checkboxes These had a bunch of issues: - They're extra work every time someone opens a PR - They didn't show up at all if people used `gh pr create` - They had "bad vibes" -- reviewers often don't want to think about LLMs, and adding a checkbox makes them very "in-your-face" for the reviewer. - Every PR now shows "1 of 2 tasks", which is useless noise. - The hovered PR description is now useless. Replace them with an HTML comment that says "remember to disclose if you used an LLM". This seems ok and low-noise for now. If we find that people are ignoring the comment, we could edit the triagebot welcome message to include a reminder and a link to the policy (cc @Kobzol, i believe you'd planned to do this already).

steffahn · 2026-08-10T16:58:40Z

Why did this get rid of the  and  from #126501 ? It removes the self-documenting example of how exactly to remove those markers, and also won't this reintroduce the problem of these comments being included in the git messages?

Edit: I've opened #160872 to fix this

…ieyouxu Add back homu-ignore markers around the PR template Besides reinstantiating the `homu-ignore` markers from rust-lang#126501, this PR also adds better clarification how the homu-ignore markers actually work; I've then also moved that section further down as it seems less important than the other parts of the PR template message. (We shouldn't forget that this is a *general PR template message*, not a LLM-policy-specific message.) The disclosure markers appear to have been accidentally removed together with the checkboxes in rust-lang#160785. In the future, if support for ignoring HTML comments in places such as markdown code blocks was added, the new description about how ``/`` can be used could be further simplified[^1], but I'd like to address the immediate issues first before considering any more involved improvements (there already *exist* [currently ~~6~~ 8] new PRs now that contain the whole template comment without `homu-ignore` markers). [^1]: actually it may always stay non-ideal given that HTML comments don't nest 🫠

Rollup merge of #160872 - steffahn:put_back_homu-ignore, r=jieyouxu Add back homu-ignore markers around the PR template Besides reinstantiating the `homu-ignore` markers from #126501, this PR also adds better clarification how the homu-ignore markers actually work; I've then also moved that section further down as it seems less important than the other parts of the PR template message. (We shouldn't forget that this is a *general PR template message*, not a LLM-policy-specific message.) The disclosure markers appear to have been accidentally removed together with the checkboxes in #160785. In the future, if support for ignoring HTML comments in places such as markdown code blocks was added, the new description about how ``/`` can be used could be further simplified[^1], but I'd like to address the immediate issues first before considering any more involved improvements (there already *exist* [currently ~~6~~ 8] new PRs now that contain the whole template comment without `homu-ignore` markers). [^1]: actually it may always stay non-ideal given that HTML comments don't nest 🫠

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 9, 2026

rustbot assigned Mark-Simulacrum Aug 9, 2026

asquared31415 reviewed Aug 9, 2026

View reviewed changes

joshtriplett reviewed Aug 9, 2026

View reviewed changes

nnethercote reviewed Aug 9, 2026

View reviewed changes

generalize from 'code' to 'any part of this PR'

91f0930

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 9, 2026

jhpratt mentioned this pull request Aug 10, 2026

Rollup of 11 pull requests #160830

Merged

rust-bors Bot merged commit e50f8fe into rust-lang:main Aug 10, 2026
13 of 14 checks passed

rustbot added this to the 1.99.0 milestone Aug 10, 2026

jyn514 deleted the no-checkboxes branch August 10, 2026 16:56

steffahn mentioned this pull request Aug 10, 2026

Add back homu-ignore markers around the PR template #160872

Merged

Uh oh!

Uh oh!

Conversation

jyn514 commented Aug 9, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Aug 9, 2026

Uh oh!

asquared31415 Aug 9, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

teor2345 Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

theemathas commented Aug 9, 2026

Uh oh!

jyn514 commented Aug 9, 2026

Uh oh!

joshtriplett Aug 9, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Aug 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nnethercote Aug 9, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fereidani commented Aug 9, 2026

Uh oh!

jyn514 commented Aug 9, 2026

Uh oh!

Mark-Simulacrum commented Aug 9, 2026

Uh oh!

jyn514 commented Aug 9, 2026

Uh oh!

Mark-Simulacrum commented Aug 9, 2026

Uh oh!

rust-bors Bot commented Aug 9, 2026

Uh oh!

jyn514 commented Aug 9, 2026

Uh oh!

rust-bors Bot commented Aug 9, 2026

Uh oh!

rust-bors Bot commented Aug 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jhpratt commented Aug 10, 2026

Uh oh!

rust-bors Bot commented Aug 10, 2026

Uh oh!

Uh oh!

steffahn commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

jyn514 commented Aug 9, 2026 •

edited by rustbot

Loading

asquared31415 Aug 9, 2026 •

edited by rustbot

Loading

teor2345 Aug 10, 2026 •

edited

Loading

joshtriplett Aug 9, 2026 •

edited by rustbot

Loading

RalfJung Aug 9, 2026 •

edited

Loading

nnethercote Aug 9, 2026 •

edited by rustbot

Loading

rust-bors Bot commented Aug 9, 2026 •

edited

Loading

steffahn commented Aug 10, 2026 •

edited

Loading