Skip to content

Explore AI powered lints #1378

Open
Open

Description

@benalleng

Rather than have manual review catch this every time, this is the sort of probabilistic lint we may be able to lean on LLMs for. I'm proposing an issue. I think we can add this one and then use the robot to check as we see repeated slips. I think it could hold us to this type of checklist with much less effort than a human reviewer following this checklist rust-bitcoin/rust-bitcoin#843.

To pull on this thread further I think we could also try to observe some common catches either from our personal mistakes or something that the LLM finds frequently to hard code allowing the LLM to focus on smaller surface areas.

Activity

  1. DanGould commented on Mar 3, 2026

    @DanGould
    Member

    Take a casual look at my thinking on this for a first blush https://claude.ai/share/c7733f88-cbf9-4e13-bdc3-ab2d39a15fa9

    And see how I used coderabit as a poor man's (it's free for OSS) custom api endpoint that removes all the actual coderabbit stuff but just prompts it to review this one lint DanGould#31 (review). I think a real, thorough setup would need multiple sub agents to at least chunk the massive API checklist for guidelines rust gives us, but I'm increasingly convinced this isn't only possible but also valuable

  2. DanGould commented on Mar 4, 2026

    @DanGould
    Member

    Every AI lint needs to have extremely strict acceptance criteria. So in the case of monitoring the public API, we need to make sure we tell the AIs what the public API actually is. If we can do that generating rustdoc, we can definitely define it accurately for AIs #1373 (comment)

  3. Naomi-Gift commented on Jun 16, 2026

    @Naomi-Gift

    @DanGould Can I be assigned to this issue? I'd love to contribute.

  4. benalleng commented on Jun 22, 2026

    @benalleng
    CollaboratorAuthor

    Well the non-deterministic nature seems to rear its ugly head every once in a while... #1665 (comment)

    What I think happened, twice now in the last 16 hours, since I got a codex update, the agent has been making its own goals in the background. The worst case is that it just runs out of tokens attempting some self-defined goals, but I can see it being more useful... this is probably a glitch in the goal system that keeps self-prompting until goal status is achieved or blocked, for example, if it's pursuing a goal and you tell it to stop without using /goal clear, it enters a self prompt standby loop until you actually clear the goal or say "continue". Non-deterministic at the core, though

    Valuable lesson though

  5. Naomi-Gift commented on Jun 22, 2026

    @Naomi-Gift

    @benalleng Can i work on this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions