Skip to content

Shared rules implementation - #757

Merged
lbarcziova merged 3 commits into
packit:mainfrom
lbarcziova:shared-rules-implementation
Sep 3, 2026
Merged

Shared rules implementation#757
lbarcziova merged 3 commits into
packit:mainfrom
lbarcziova:shared-rules-implementation

Conversation

@lbarcziova

@lbarcziova lbarcziova commented Aug 17, 2026

Copy link
Copy Markdown
Member

No description provided.

TomasTomecek
TomasTomecek previously approved these changes Aug 18, 2026

@TomasTomecek TomasTomecek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nicely done

async def _fetch_registry(self) -> dict[str, list[str]]:
if (
self._registry_fetched_at
and (time.monotonic() - self._registry_fetched_at) < REGISTRY_TTL_SECONDS

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this definitely makes a lot of sense, we don't need to fetch the same file multiple times per minute

I do wonder though if we could just check if the file changed using the respective HTTP header

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean just for the periodic recheck? I didn't consider it since the yaml should be rather small, but if this needs to scale up later, we can definitely add it, good point

@lbarcziova
lbarcziova force-pushed the shared-rules-implementation branch from 45b8cb7 to 3c697da Compare September 1, 2026 11:16
@lbarcziova
lbarcziova marked this pull request as ready for review September 1, 2026 11:17
@lbarcziova
lbarcziova force-pushed the shared-rules-implementation branch from 8cd28d7 to c72e6e8 Compare September 1, 2026 13:30
@lbarcziova

Copy link
Copy Markdown
Member Author

/agentic_review

@lbarcziova
lbarcziova force-pushed the shared-rules-implementation branch from c72e6e8 to 38b0488 Compare September 1, 2026 13:41
@lbarcziova

Copy link
Copy Markdown
Member Author

/agentic_review

@lbarcziova
lbarcziova force-pushed the shared-rules-implementation branch 2 times, most recently from 974fa91 to 1b79fbb Compare September 2, 2026 12:49
@nforro

nforro commented Sep 2, 2026

Copy link
Copy Markdown
Member

/agentic_review

@nforro

nforro commented Sep 2, 2026

Copy link
Copy Markdown
Member

@lbarcziova It seems Qodo doesn't like you.

nforro
nforro previously approved these changes Sep 2, 2026

@nforro nforro left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qodo-for-packit

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (1) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. shared-rules dependency undocumented 📘 Rule violation § Compliance
Description
Agents now rely on the GitLab-hosted shared-rules registry, but THREAT_MODEL.md was not updated to
describe this dependency, its rule data, or its role. The existing threat model discusses agent
access to privileged tools and GitLab only in other contexts.
Code

ymir/tools/privileged/shared_rules.py[R66-68]

+        project_path = quote(f"{RULES_NAMESPACE}/{SHARED_RULES_REPO}", safe="")
+        file_path = quote(REGISTRY_FILE, safe="")
+        url = f"{GITLAB_API_URL}/projects/{project_path}/repository/files/{file_path}/raw?ref=main"
Relevance

●●● Strong

Recent repository patterns accept documenting newly introduced operational dependencies and
configuration requirements.

PR-#627
PR-#725

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
PR Compliance ID 3951 requires every new agent service dependency to be documented with its
identity, purpose or data, and system role. The PR exposes get_shared_rules to agents and fetches
its registry through the GitLab API, while the threat model's privileged-tool description does not
identify or describe this registry dependency.

Rule 3951: Document new agent service dependencies in THREAT_MODEL.md
ymir/agents/triage_agent.py[450-453]
ymir/tools/privileged/shared_rules.py[66-76]
THREAT_MODEL.md[46-54]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new GitLab-hosted shared-rules registry dependency is not documented in `THREAT_MODEL.md`.

## Issue Context
Agents now invoke `get_shared_rules`, which retrieves package-to-rule mappings from a central GitLab repository. Document the service identifier, its purpose and data, and its downstream relationship to agents through the privileged MCP gateway.

## Fix Focus Areas
- THREAT_MODEL.md[46-54]
- ymir/tools/privileged/shared_rules.py[66-76]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Rule precedence is unspecified 🐞 Bug ≡ Correctness
Description
The testing analyst receives shared and package-specific rules in one prompt, but is never told that
package-specific rules must win. Conflicting rules can therefore cause issue verification to follow
the shared instruction and produce the wrong testing decision.
Code

ymir/agents/issue_verification_agent.py[R150-152]

+    shared_rules = await _fetch_shared_rules(gateway_tools, package)
+    if shared_rules:
+        maintainer_rules = shared_rules + "\n\n--- Package-specific rules ---\n" + maintainer_rules
Relevance

●●● Strong

Precedence is explicit PR intent and analogous prompt guidance consistently requires
package-specific rules to win.

PR-#674
PR-#655

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The new code concatenates both rule levels but only labels their sections. The analyst template
inserts that combined value without any precedence instruction, while the other updated prompts
explicitly state that package-specific rules take precedence.

ymir/agents/issue_verification_agent.py[150-156]
ymir/agents/prompts/issue_verification/common.j2[8-10]
ymir/agents/prompts/backport/instructions.j2[24-27]
ymir/agents/prompts/triage/prompt.j2[75-79]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Issue verification combines shared and package-specific rules without telling the testing analyst how to resolve conflicts. Explicitly state that package-specific rules take precedence over shared rules.

## Issue Context
Other agent prompts introduced by this PR consistently define package-specific rules as authoritative when rules conflict, while the testing analyst prompt only exposes a single `MAINTAINER_RULES` block.

## Fix Focus Areas
- ymir/agents/issue_verification_agent.py[150-152]
- ymir/agents/prompts/issue_verification/common.j2[8-10]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Rule fetches are unbounded 🐞 Bug ☼ Reliability
Description
_fetch_shared_rules awaits one maintainer-rules request at a time for every matching registry
entry, with no limit on match count. Because each request has a 120-second tool timeout, an
oversized or erroneous registry mapping can delay issue verification for many minutes before
analysis starts.
Code

ymir/agents/issue_verification_agent.py[R100-103]

+    for name in rule_names:
+        try:
+            content = await run_tool(
+                "get_maintainer_rules",
Relevance

●●● Strong

Reliability precedents favor bounding or isolating potentially unbounded asynchronous work and
preventing workflow stalls.

PR-#675
PR-#540

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The registry tool returns every matching rule-set name without a cap. Issue verification then awaits
each corresponding fetch sequentially, and the called tool declares a 120-second timeout; all
retrieval finishes before the analyst is run.

ymir/tools/privileged/shared_rules.py[120-126]
ymir/agents/issue_verification_agent.py[99-113]
ymir/tools/privileged/maintainer_rules.py[27-30]
ymir/agents/issue_verification_agent.py[150-162]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Shared rule documents are fetched serially and without a maximum count, allowing registry growth or configuration errors to create an effectively unbounded delay. Bound the number of applicable rule sets and fetch accepted entries concurrently with controlled concurrency and an overall timeout.

## Issue Context
`get_shared_rules` returns all matching registry keys, and each follow-up `get_maintainer_rules` invocation can run for up to 120 seconds. Preserve per-rule failure isolation and deterministic output ordering.

## Fix Focus Areas
- ymir/agents/issue_verification_agent.py[99-113]
- ymir/tools/privileged/shared_rules.py[120-126]
- ymir/tools/privileged/maintainer_rules.py[27-29]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
✅ Compliance rules (platform): 8 rules

Grey Divider

Tip of the day
💡 Did you know, you can turn on the rule miner and Qodo learns your standards from review history

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread ymir/tools/privileged/shared_rules.py
Comment thread ymir/agents/issue_verification_agent.py Outdated
Comment thread ymir/agents/issue_verification_agent.py Outdated
Implements a new MCP tool to fetch shared rules from a central registry,
allowing common guidelines to be defined once per ecosystem (Python, Perl,
etc.) and automatically applied to groups of packages.

Core changes:
- New SharedRulesTool: Looks up which shared rule sets apply to a package
  by querying registry.yaml in rules/shared-rules repo
- Returns JSON list of ecosystem names; agents fetch actual AGENTS.md
  content via existing get_maintainer_rules tool
- TTL-based caching (1 hour) at tool instance level for registry.yaml
- Centralize GITLAB_API_URL and RULES_NAMESPACE constants (currently
  pointing at ymir-rules-test for testing)
- Register tool in MCP gateway alongside existing MaintainerRulesTool
- Add PyYAML>=5.1 dependency for registry parsing
- Full test coverage (8 test cases) for lookup and caching behavior

Design rationale:
- Tool returns only the list, not content (keeps it simple)
- Agent orchestrates fetching (maintains control over precedence logic)
- Per-package rules always take precedence over shared rules
- Registry cached to avoid repeated GitLab API calls
- Transient errors don't cache; permanent errors do (404, bad YAML)

Assisted-by: Claude Opus 4.6
Lets shared rules be disabled without a code change if the registry
or a rule set turns out to cause problems in production. Wired into
both OpenShift (configmap + mcp-gateway deployment) and local dev
(compose.yaml + make start/start-detached) so the flag behaves the
same in both environments.

Assisted-by: Claude Sonnet 5
Fixes three issues from PR review: state that package-specific rules
take precedence over shared rules in the testing analyst prompt (the
one integration point that omitted it), fetch shared rule sets
concurrently instead of serially to avoid unbounded latency from
per-rule 120s tool timeouts, and document the shared-rules registry
as a new untrusted-content entry point in THREAT_MODEL.md given its
cross-package blast radius.

Assisted-by: Claude Sonnet 5
@lbarcziova
lbarcziova merged commit 7bc1767 into packit:main Sep 3, 2026
13 checks passed
@lbarcziova
lbarcziova deleted the shared-rules-implementation branch September 3, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants