Skip to content

feat(datasource-pylon): Pylon datasource for the Ruby agent (EXT-4) - #369

Merged
christophebrun-forest merged 42 commits into
mainfrom
feat/datasource-pylon
Sep 1, 2026
Merged

feat(datasource-pylon): Pylon datasource for the Ruby agent (EXT-4)#369
christophebrun-forest merged 42 commits into
mainfrom
feat/datasource-pylon

Conversation

@christophebrun-forest

@christophebrun-forest christophebrun-forest commented Aug 24, 2026

Copy link
Copy Markdown
Member

Integration branch for EXT-4forest_admin_datasource_pylon, feature parity with the Zendesk datasource.

The ten stories of the epic are Done and merged, main is merged in, and the branch touches exactly three files outside the new package.

What is in the branch

A new forest_admin_datasource_pylon package: 44 files under lib, 24 spec files, 786 examples, 100.0% line coverage (1686/1686) and 94.5% branch coverage. Ten story PRs, all reviewed and merged into this branch:

PR Story Content
#341 EXT-5 foundation — gem, configuration, resilient client
#347 EXT-6 PylonIssue read-only, cursor pagination
#351 EXT-7 filters, search, count, PK short-circuit hardening
#354 EXT-8 all collections + relations
#358 EXT-9 messages / conversation thread
#359 EXT-10 custom fields introspection
#362 EXT-11 write operations (CRUD)
#366 EXT-12 action plugins: close issue, create issue with notification
#373 EXT-13 story 9 — proactive per-endpoint throttling on top of the 429 retry
#375 EXT-17 bound the boot-time custom-fields introspection

That is every sub-issue of EXT-4 and every PR ever opened against this branch. Re-checked against
Linear on 2026-08-27: EXT-4 has exactly these ten children, all Done, and no eleventh — so the
table above is the whole epic. EXT-4 itself is the only thing left In Progress, this PR being what
closes it.

Three follow-ups were filed out of the review rather than fixed here. They are not sub-issues of
EXT-4 — each is repo-wide, not Pylon's — and they are listed so the table is the full Linear picture:

Ticket Scope Why not here
EXT-19 .releaserc.js chains all fourteen successCmd pushes with ; Fixing only the mambu/pylon boundary the diff touches would leave twelve packages masking failures
EXT-20 No datasource gemspec declares the Forest gems as runtime deps Declaring them on Pylon alone makes it the only gem with a different contract
EXT-21 A non-positive page limit reads as "no limit" in one place, "empty" in another Read path of every collection; the two readings have to be settled together

Shape of the package: client.rb + client/writes.rb over the Pylon API with a retry_policy, a rate_limiter / rate_limits / throttle trio metering each endpoint in its own sliding window, a base_collection specialised into cursor_collection / fetch_all_collection, one directory per collection (account, contact, issue, team, user) each with its own api_filters / schema_definition / serializer, a query/ layer translating condition trees into Pylon filters, pagination/cursor_walker, schema/custom_fields_introspector, and two action plugins.

Review pass on the whole branch

65190b9f merges main; the two conflicts were both the #368 sort-decorator cherry-pick, resolved in favour of main, so those files have left this diff. Fifteen commits then close out the review of the assembled branch — these are direct commits, so they appear in no story PR above:

Commit Kind Content
b9cb7d15 fix refuse a Number filter cast to Infinity / NaN; deduplicate action issue ids; stop collect_pages on a cursor already followed
a93474b1 fix MAX_ID_LOOKUPS bounds a page rather than a selection
24951097 fix refuse two plugin configurations that misbehave silently
cc015514 fix keep no column for a projection naming only relations
373f94de refactor drop the countable: flag nothing opts into
7033ba1d fix stop the custom-field introspection at its first failure
bd55e8ba docs the package README, last open item of EXT-13
2a1b4d6d fix serve the whole record for an empty projection
67cd786a fix refuse an internal note the form never offered
1fd03cbd fix keep a select with no option left read-only
96a0575a docs record what bounds CloseIssue and what does not
0a2d4e4d chore align version.rb with the other packages
d75bab2c fix intersect the id conditions of an and, on both the read and the write path
524cc95e fix let a refused selection reach the operator of a close action
b6bd085b docs correct which scope bounds CloseIssue

Defects fixed

  • b9cb7d15 — a Number filter the agent cast to Infinity or NaN reached to_i, which raises, and then the JSON encoder, which raises too: a 500 either way, on a value the operator typed. Now refused with a 400 naming the field. Same commit: an action reading issue ids off a column deduplicates them, and collect_pages stops on a cursor it has already followed rather than only on one that did not move.
  • a93474b1MAX_ID_LOOKUPS bounded a selection where it should bound a page: the ids were truncated to the first 20 before the window was applied, so an offset at or past 20 answered an empty page from records that exist. The window is now taken off the ids before any of them is read, which also spends one request per record the caller asked to see. A lookup carrying a residual condition cannot be paged that way and is refused past the cap rather than answered with a fraction of itself.
  • 24951097 — a plugin scope that is neither a symbol nor a string raised NoMethodError instead of the error naming it; duplicate email-template titles, and a template titled like the "pick none" sentinel, are now refused at registration instead of silently sending the wrong message.
  • cc015514project served the whole record when the projection named no column of the collection, so a row asked for as account:name came back carrying every native column too. Relevant now that main has landed serve only the columns of collections the caller may read. It also changed what an empty projection returns, which is not that case at all; 2a1b4d6d separates the two.
  • 7033ba1d — EXT-17 bounded each boot-time introspection per request, not the three of them together, so a Pylon that hangs could add about a minute to a Rails boot rather than the twenty seconds intended. The first failure now stands for the object types after it.

Also

  • 373f94de — drops the countable: flag: no collection ever passed it, Pylon having no count endpoint and no total.
  • bd55e8ba — the package README, last open item of EXT-13.

Macroscope: 21 review threads across seven passes, every one answered in thread. The seventh pass landed one finding, on base_collection.rb:107, and it was real: fixed in d75bab2c, described below. Two of the last four findings were real and are fixed (67cd786a, 1fd03cbd); one is a false positive (nil.to_i is 0, so the guard already re-raises) and one is not reachable through the agent stack, which rewrites the date operators upstream — both answered with the reasoning. The normalize_templates thread was Macroscope reading a stale index; the method has been there since 24951097.

Recorded on Linear rather than fixed here: SnoozeIssue (EXT-12) and aggregate / Count (EXT-7) are in the epic scope but were never delivered; both are commented on their story with what exists instead. EXT-4's rate-limit figures were low by a factor of 4 to 15 and have been corrected against rate_limits.rb. The three findings declined as repo-wide now have tickets of their own, so they outlive these threads: EXT-19 (.releaserc.js chaining with ;), EXT-20 (no datasource gemspec declares the Forest gems) and EXT-21 (a non-positive page limit).

Behaviour outside the new package

Three files, each additively, to register the gem: .github/workflows/build.yml (test matrix, coverage files:), .releaserc.js (prepareCmd, successCmd, git assets) and .rubocop.yml (MFA opt-out, the version.rb string cops, and the per-file metric excludes the package actually trips).

Two entries in build.yml, not three: #380 replaced the lint job's per-package matrix with a single root bundle exec rubocop, which is where the merge conflict was and what 219d8bcc resolves. That PR also rewrote step 5 of CLAUDE.md to say the lint job needs no entry, so the guidance and the workflow agree again -- the four points of CLAUDE.md are covered against its current text.

Verification

Re-run locally on cbaa5619, Ruby 4.0.2:

  • forest_admin_datasource_pylon: 827 examples, 0 failures, line coverage 100.0% (1755/1755).
  • The packages the cross-package fixes touch, and their neighbours: forest_admin_agent 1180, forest_admin_datasource_toolkit 486, forest_admin_datasource_customizer 703, forest_admin_datasource_active_record 203 — all 0 failures. RuboCop over the whole repo: 911 files, 0 offense.
  • forest_admin_datasource_mongoid could not be run locally (its gems are not installed on this machine); CI covers it.

.releaserc.js was loaded through node to confirm both prepareCmd and successCmd are valid concatenations, and both YAML files were parsed. forest_admin_datasource_pylon is unclaimed on RubyGems, so the first publish creates it.

Three fixes from the seventh pass

Local re-run on b6bd085b, Ruby 4.0.2: 794 examples, 0 failures, line coverage 100.0% (1695/1695), branch 94.3%, RuboCop 71 files / 0 offense.

  • d75bab2c — an and names the records all of its conditions name, so two id leaves name their intersection. Both the lookup short-circuit and the write-resolution bound kept only the first and left the second to be applied later, which put the cap around the wider set: a selection of 25 ids narrowed to 3 was refused over the 25 it named, after reading all 25 one request each. Worth being exact about the failure — the refusal is an explicit UnsupportedOperatorError, never a short page, so this over-refused rather than under-delivered. extract_id_lookup now partitions the id leaves out of the branch and intersects them, leaving the residual, ensure_residual_appliable! and guard_nil_comparisons untouched; disjoint sets reduce to the empty lookup with no request at all. writes.rb#filtered_ids moves with it, that count being what refuse_unresolvable_selection is worded around — its own comment had been excusing the same shape. Narrow on the read side: only PylonIssue reaches the and branch, cursor collections short-circuiting on a bare leaf alone.
  • 524cc95eIssueTargets.resolve_issue_ids rescued StandardError and answered [], which the action reports as "No Pylon issue selected." That swallowed the datasource's own refusals: a bulk CloseIssue on PylonIssue, with a scope or segment set and more than MAX_ID_LOOKUPS records selected, raises inside list and the operator was told nothing was selected about a selection they can see they made. A Pylon outage read the same way. A ValidationError now travels, the agent surfacing its message; everything else still degrades, so a renamed column is still "no issue selected" rather than a stack trace in the panel.
  • b6bd085b — the README's CloseIssue caveat had the scope question backwards. It said a scope on PylonIssue does not bound what the action closes, on the grounds that the state is written through the client. The ids are resolved first, through the collection the action sits on, and Actions#get_record_selection intersects the operator's scope into the filter that reads them — so mounted on PylonIssue, the scope bounds exactly what it closes. What the caveat describes is the host-collection form, where the ids come off a column and that column is the authority. As written it told an operator a scope bought them nothing when it bought them the whole selection.

Human review, and main merged in

Three commits since the seventh pass. a08ab420 answers the four findings @matthv raised on the assembled branch, 219d8bcc merges main, e1865da7 closes the last open Macroscope thread of the previous pass.

Commit Kind Content
a08ab420 fix refuse an edit reduced to nothing; bound what one 429 costs; follow the cursor of an endpoint documented unpaginated; pin the residual/page-window order
219d8bcc chore merge main, resolving the build.yml conflict
e1865da7 fix keep the user-info of a base url out of an inspect

The four findings

  • update reported a success on a patch it dropped entirely. writable_attributes drops read-only and unknown keys, and a patch reduced to nothing hit return if attributes.empty?: the route then re-reads the record and hands back the values it already had, so the operator watched their change revert with no reason given -- the exact answer the module's preamble says it refuses. Refused now, naming the keys. Deliberately narrow: a dropped key alongside a real edit is still dropped and the edit performed, and a patch naming no field at all (a form whose every field is a relation) still asks for nothing. The refusal also stays settled before the ids are, so a wide selection with an unwritable patch is refused for its patch and spends no request.
  • One 429 episode could hold a thread for ~3 minutes. faraday-retry sleeps the raw Retry-After whenever it fits under max_interval, so max_retries: 3 against a per-minute quota meant up to 180s of blocking sleep, plus the Throttle waits inside the retry -- on a request the Forest server had long since timed out, and multiplied by the 20 sequential calls one list view may spend. DEFAULT_MAX_INTERVAL is 12 rather than 65, which is the figure that keeps the sleeps of a whole request under a minute (3 x 12 plus 4 x RateLimiter::DEFAULT_MAX_WAIT = 56s); past it a 429 surfaces as an error instead of being waited out. A spec computes that bound rather than asserting the constant, so it cannot drift silently. The trade is stated in the code and in the README, with the snippet to choose the other way.
  • Client#fetch_all ignored the response's pagination block. FetchAllCollection builds its exactness guarantee -- including the package's only exact aggregate -- on "the records in hand are all of them", so the day Pylon paginates GET /users or GET /teams its first page would silently become everything. The cursor is followed now, the way /messages already was, and refused rather than truncated past MAX_COLLECTED_PAGES: a count answered short and presented as exact is the one answer this datasource refuses everywhere else. Zero cost today -- no pagination block is advertised, so the walk leaves after the first page and spends the single request it always did.
  • Missing test: residual combined with a page window. Every residual spec used a filter without a page, so the ordering in records_by_id -- residual first, page_window second -- was pinned by nothing. The added example was checked against the regression rather than only against green: flipping the implementation to slice the ids first makes it fail with expected [i3], got [i2].

The inspect hardening, which the previous description had left as a follow-up and which the eighth Macroscope pass then raised as its own finding: base_url is operator-supplied, and an egress proxy fronting Pylon is spelled https://user:pass@proxy.internal. Both places that printed the url printed that user-info in clear. Configuration#redacted_url is now what the two inspects print -- the finding named Client#inspect alone, but Configuration#inspect printed @base_url the same way. Cut with a regexp rather than parsed: nothing validates that the base url parses, and an inspect raising on the way to a Rails error page would replace the page with its own failure.

Open items before merge

  • CI is green. Run 33509392217 on e17c4328, and 33498265681 on the merge commit 219d8bcc -- the latter being what validates the resolved workflow, the lint job now being the single root RuboCop run of ci(build): halve CI wall clock by parallelizing jobs and caching gems #380 while pylon keeps its two test jobs and its coverage entry. The run for cbaa5619 is in flight at the time of writing. Earlier: 33158331718 on 3e38d094, 33084670532 on b6bd085b, both successful on Ruby 3.4 and 4.0.
  • The eighth Macroscope pass is answered. Thirteen findings, verified one by one; seven led to a change (e17c4328, cbaa5619), six are answered in thread. What it caught, and what it got wrong:
    • Its best catch was on this branch's own new code. collect_pages treated an empty page advertising a cursor, and a cursor answered twice, as the end of the collection -- so refuse_past_cap had closed the exactness hole for the page cap alone. Only next_cursor being nil is Pylon saying there is nothing more; every other stop is now a truncation the caller reads as a warning or as a refusal.
    • Four more real, and fixed: an integer custom field past 2**53 read through a Float (parsed as an integer now, in base 10 explicitly -- Integer("012") is 10, so the suggested patch would have renumbered every zero-padded value); a timezone checked stripped and stored unstripped, falling back to UTC on a padded identifier; CloseIssue letting two variants configured with the same name overwrite each other in the collection's action hash; and an :action_name given as a Symbol reaching the schema, where get_action_slug calls strip on it and GeneratorCollection sorts it against a String -- schema generation, so the whole agent, not the action.
    • Two bugs outside the package, both found by verifying a finding rather than in it. Charts#compute_value indexed result[0]['value'] while Aggregation#apply answers an empty selection with no row, so a filtered value chart matching nothing was a 500 where the figure is zero -- reachable from every collection that aggregates in memory. And Aggregation compared Max with <, so every Max chart over an in-memory aggregation answered with the minimum, on every datasource; no spec covered Max at all. Both fixed, with specs.
    • Its one Critical does not hold. messages is not a relation: it is a single column whose column_type is [MESSAGE_THREAD_SCHEMA], a nested object shape the agent projects whole, and every field the finding names (is_private, author_email, both author ids) is a declared member of that column's own type -- covered by the operator's permission on the column. No agent path produces the messages: sub-projection it assumes. The comparison with RelationEmbedder is what misleads: that one re-projects because it embeds records of another collection, carrying their own permissions.
    • One High is not reproducible as described. update_summary_in_place guards with if value, so false is skipped and no TrueClass/FalseClass comparison ever happens. It answers wrongly rather than raising -- a Boolean Min can never report false -- which is filed with the Sum/Avg-over-a-String finding as EXT-23, both being the same method applying an operation the column type cannot take.
    • Two accurate limitations, filed rather than rushed: a nested and carrying an id is refused instead of resolved, on the read and the write path (EXT-22). It fails closed, and ConditionTreeFactory.group flattens one level, so every shape the agent builds keeps its id at the top level.
    • One already answered in the code: the Date/Dateonly operators the validator rejects are described in the comment above TIME_OPS, with the toolkit contradiction tracked as PRD-989. And one left as it is: Time/DateTime filter bounds losing their fractional seconds is unreachable through the agent, Transforms::Times already handing over second-truncated strings from the same iso8601 call one layer up.
  • The 44 qlty threads are resolved without a change, deliberately. 26 function-parameters (count = 4), 16 function-complexity, one file-complexity on base_collection.rb and one identical-code on the scaffold Gemfiles -- which are identical because CLAUDE.md asks for them to be mirrored. The repo carries no .qlty config, qlty check reports No blocking issues, and the three raised on this pass's own code (collect_pages, refuse_truncated_walk, log_truncated_walk) are all the 4-parameter threshold, where each parameter is a value the message or the walk names. Resolved so that an open thread on this PR means feedback still to answer, rather than advisory noise.
  • Deliberately not fixed here, each with the reasoning recorded in thread:
    • CloseIssue's batch is uncapped: one request per selected record, so a wide bulk selection is a long sequential run the request may time out on, leaving what it already closed closed. Stated in the README (96a0575a, reworded in b6bd085b); putting a cap on a bulk action is a product call, not a defect fix. Worth knowing that Writes::MAX_WRITE_REQUESTS refuses exactly this shape at 20 requests on the collection itself, so the datasource currently answers the same question two ways.
    • NIL_UNSAFE_OPERATORS and residual_leaf_appliable? are derived from two different sets, so the guard is narrower than what the residual admits and only the upstream OperatorsEquivalence rewrite closes the gap. Not reachable today; deriving the guard from the equivalence touches the filtering path of every collection.
    • Configuration#validate! checks neither that base_url parses nor that it is https, so a typo surfaces as a raw URI error at the first request and an http:// base url would carry the bearer token in clear on the wire. Refusing http would break local mock servers, so this wants a warning rather than a refusal, in its own change. (What an inspect prints is no longer part of this item: e1865da7 closes that half.)
  • A human review.

🤖 Generated with Claude Code

Note

Add ForestAdminDatasourcePylon datasource with API client, collections, and plugins

  • Introduces the ForestAdminDatasourcePylon gem, exposing Issue, Account, Contact, User, and Team collections backed by the Pylon API.
  • Adds an HTTP client with retry policies, per-endpoint rate limiting, and Pagination::CursorWalker to handle cursor-based pagination.
  • Implements read and write operations for all collections, including custom field introspection and relation embedding.
  • Adds CloseIssue and CreateIssueWithNotification plugins for issue management actions.
  • Fixes ConditionTreeBranch#match in condition_tree_branch.rb to preserve nested branch structure during matching.
  • Risk: The ConditionTreeBranch#match fix changes how nested And/Or branches are evaluated; verify existing toolkit consumers depending on flattened leaf evaluation are unaffected.

Changes since #369 opened

  • Added credential redaction to URL display in ForestAdminDatasourcePylon package [e1865da]
  • Fixed Max aggregation returning minimum value instead of maximum and value charts crashing on empty result sets [e17c432]
  • Added explicit truncation detection and handling for paginated cursor walks in Pylon datasource client [e17c432]
  • Prevented silent action name collisions when registering CloseIssue plugin with multiple scopes [e17c432]
  • Preserved full integer precision and base-10 interpretation in numeric field coercion [e17c432]
  • Stripped surrounding whitespace from timezone identifiers in filter value initialization [e17c432]
  • Modified ForestAdminDatasourcePylon::Plugins::CloseIssue.name_for to coerce action names to strings and added documentation comments [cbaa561]
  • Added test coverage for string coercion and collision detection of action names in ForestAdminDatasourcePylon::Plugins::CloseIssue [cbaa561]

Macroscope summarized 219d8bc.

christophebrun-forest and others added 11 commits August 7, 2026 18:23
…341)

* feat(pylon): scaffold datasource gem with config and resilient client

Story 1 of the Pylon datasource (EXT-5).

Adds the forest_admin_datasource_pylon gem skeleton: Zeitwerk autoloading,
typed error hierarchy with an APIError carrying HTTP status and parsed body,
configurable logger, Configuration with api_key validation, and a Faraday
client authenticating with a Bearer token plus a GET /me health check.

The Faraday middleware order is deliberate and differs from the Mambu
Payments gem: raise_error sits outside the JSON parser so errors carry an
already-parsed body, and retry sits innermost so it can observe raw statuses.
Behind raise_error the retry middleware never sees a 429 and retry_statuses
silently does nothing. Non-idempotent verbs are only retried on 429, where
Pylon rejected the request before processing it.

Wires the package into the CI lint, test and coverage jobs. The
semantic-release publish pipeline is intentionally left untouched until
Story 9, so an incomplete gem is never pushed to RubyGems.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…agination (#347)

* feat(pylon): add PylonIssue read-only collection with cursor pagination

Registers the first Pylon collection: issues in list + record-detail mode,
backed by POST /issues/search and GET /issues/{id}.

Forest asks for an offset/limit window while Pylon only hands out the next
page of a cursor, so CursorWalker walks pages until the window is covered
then slices. The walk is capped (20 pages / 5000 records, with a truncation
warning) because /issues/search allows 20 requests per minute and an
uncapped deep-offset walk would spend an agent's whole budget on one list
view. It also stops defensively on an empty page or a cursor that does not
advance.

The schema follows the live API rather than the ticket: Pylon has no
priority field, first_response_time/resolution_time are RFC3339 timestamps
and not durations, and /issues/search exposes no sort parameter, so no
column is sortable and translate_sort is not ported from Zendesk. Nested
account/requester/assignee/team objects are flattened into id columns until
the related collections exist.

Search and Count default to disabled in BaseCollection, the inverse of the
Zendesk template, since both land with the condition-tree translator. Until
then a condition the collection cannot honour is dropped with a warning
naming what was discarded, so an unfiltered result set does not read as a
filtered one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#351)

* feat(datasource): translate Forest filters for Pylon search

PylonIssue honoured no filter but the primary-key short-circuit, so a
segment or a UI filter returned unfiltered rows that looked filtered.

- Query::ConditionTreeTranslator maps a condition tree onto the structured
  JSON filter of POST /issues/search: native OR through subfilters, a
  depth-3 guard, and timezone-aware date bounds. Anything the API cannot
  express raises instead of being dropped.
- Issue::ApiFilters transcribes Pylon's per-field allow-list and is the
  single source of truth: define_schema derives every filter_operators
  from it, so the schema can no longer advertise a filter the API refuses.
  Declaring the bare comparisons on date columns lets the toolkit rewrite
  Today / Previous* into a pair of bounds, so time_range is never emitted.
- filter.search is forwarded as search_text (enable_search).
- extract_id_lookup also pulls the id leaf out of a top-level AND and
  returns the leftover conditions, applied in memory. Forest sends
  AND(id equal X, scope) on a record detail as soon as a scope is set, and
  id is not a Pylon filter field.
- translate_sort / timezone_for ported from the Zendesk base collection.
  /issues/search has no sort parameter, so PylonIssue's allow-list is empty
  and a requested order is reported rather than silently swallowed.

Count stays disabled: Pylon exposes neither a count endpoint nor a total,
so counting means walking pages against a 20 req/min budget on every list
view. Moved to the hardening story that owns throttling.

176 examples, 0 failures, 100% line coverage, rubocop clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gem was registered in build.yml and .rubocop.yml but not in
.releaserc.js, so semantic-release never bumped its version, never built
it and never pushed it, while CI stayed green: the omission has no
failing check to surface it. Added to the three places a package has to
appear -- the prepareCmd sed, the successCmd build and push, and the
git assets, so the bumped version.rb travels with the release commit.

query/filter_value.rb requires active_support/core_ext/time/zones and
calls Time.use_zone, which only the monorepo Gemfile was providing. The
zendesk and mambu datasources both declare activesupport >= 6.1; without
it, installing the gem on its own gives a datasource that raises on the
first date filter it translates.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(pylon): client endpoints for accounts, contacts, users and teams

Adds search/list/fetch methods for the four new resources, routed
through shared private helpers; search_issues and fetch_issue now
delegate to the same helpers with unchanged behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(pylon): embed the issue conversation thread

Ports the Zendesk comments embedder to Pylon: the messages of an issue are
read through GET /issues/{id}/messages and embedded as a structured array
column on PylonIssue.

The thread is asked for without a limit, so Pylon answers with the whole
conversation in one request; a page would have handed back the oldest
messages and cut the recent ones off. Authors are flattened from the payload
Pylon already nests in each message, so no author lookup is spent at all.

The fan-out is bounded like the primary-key lookups of this collection: the
endpoint allows 20 requests per minute and a thread costs one request per
row, so rows past MAX_MESSAGE_EMBEDS are left at nil -- unknown, never the
empty list, which would read as "this issue has no message". A thread that
cannot be read degrades the same way instead of failing the page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: bound every job and harden the unixODBC install

The workflow declared no timeout anywhere, so a job that hangs pins the
run for GitHub's 6 hour default. A hung `apt-get` on the two Snowflake
lint legs did exactly that: `test` needs `lint`, and `coverage` needs
`test`, so one stuck leg of the matrix froze the whole pipeline and no
test ever ran.

Lint, test and coverage now carry a timeout, and the apt step that hangs
carries its own, tighter one, plus retries on the mirror and a
noninteractive frontend. A failing leg is now visible in minutes and can
be re-run.

Deploy is left unbounded on purpose: cutting semantic-release off in the
middle of pushing thirteen gems is worse than waiting for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit c5df079)
* feat(pylon): client endpoint for custom fields

GET /custom-fields takes a mandatory object_type, so the definitions of
each collection are read by their own call. The walk carries the
parameter on every page: dropped on the second request, it would answer
for another object type or with a 400.

Degrades to an empty list, like the message thread: this is read while
the agent boots, and a token missing the permission has to cost the
operator the custom columns rather than the whole datasource.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(pylon): write operations (CRUD)

Open the datasource to writes: a `writes` mixin on the client, one method
per Pylon endpoint, and create/update/delete on every collection, through
a mechanism shared by the three collection bases.

What may be written is `is_read_only` on the column, the single source of
truth the payload builder reads, the way `api_filters` already is for
filtering. Pylon's own `is_read_only` is now honoured on a custom field,
and a value is written back through the list of `{slug, value}` entries
the API takes, `values` for a multiselect and the option slug for a
select.

The verbs Pylon exposes no endpoint for -- no POST or DELETE on a user,
no DELETE on a team -- refuse with a message rather than the contract's
NotImplementedError, which the agent answers as an unexpected 500. So do
the fields it only accepts in one direction: `body_html` on a create,
`state` on an update, and the like, dropped when they ask for nothing and
refused when the operator really changed them.

A filter-driven update or delete resolves its ids exactly or refuses:
an id filter is answered without a request, anything else goes through
the collection's own list so the scope applies, and a selection wider
than one pass of writes is refused rather than written halfway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…gins (EXT-12) (#366)

* feat(pylon): close and create-with-notification action plugins

Two plugins at parity with the Ruby Zendesk ones, built on the write
primitives of the previous commit.

CloseIssue moves the selected issues to a state, `closed` unless told
otherwise -- a custom status slug is taken as readily, Pylon accepting
one wherever it accepts a standard state. One variant per scope rather
than the four Zendesk builds, there being a single terminal state here.
Each id is written under its own rescue and named in the message, so a
batch that only half applied never reads as a plain success.

CreateIssueWithNotification opens an issue and delivers its first
message. Pylon says the delivery outright where Zendesk infers it from a
public comment: `destination_metadata.destination` names the channel, and
no metadata at all is what leaves the issue internal, which is what the
"send as internal note" checkbox writes. The form carries no Type field,
`POST /issues` taking none, and the priority it does carry is never read
back -- no Pylon read returns one.

Both find the issues to act on through one option: `issue_id_field` names
a column of the host collection, and its absence falls back to the
primary keys, which is what an action registered on PylonIssue acts on.

Snooze is left out until its endpoint is confirmed against a live
organization; IssueTargets and the messages module are already shared.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…comments

Review follow-up on the previous commit, no behaviour change.

Building the fields hash with to_h removes both the reassignment of a
key while the hash is being iterated and the now useless dup of it, and
the comments are cut back to the two non-obvious points: the key? read
and the copy before the write.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@linear-code

linear-code Bot commented Aug 24, 2026

Copy link
Copy Markdown

EXT-4

Comment thread .releaserc.js
Brings the 29 commits main gained since 1.36.2, the merge base.

Three files were touched on both sides and conflicted, all three because
main registered forest_admin_datasource_graphql_hasura where this branch
registered forest_admin_datasource_pylon. Every resolution is additive,
both packages kept:

- .releaserc.js: prepareCmd, successCmd and the git assets list
- .github/workflows/build.yml: lint matrix, test matrix, coverage files
- .rubocop.yml: the Gemspec/RequireMFA exclusion

main also put forest_admin_rails back in the test matrix and moved the
coverage job to Ruby 4.0; both come over untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@qltysh

qltysh Bot commented Aug 25, 2026

Copy link
Copy Markdown

Qlty


Coverage Impact

⬆️ Merging this pull request will increase total coverage on main by 0.5%.

Modified Files with Diff Coverage (46)

RatingFile% DiffUncovered Line #s
Coverage rating: A Coverage rating: A
...est_admin_agent/lib/forest_admin_agent/routes/charts/charts.rb100.0%
New file Coverage rating: A
...forest_admin_datasource_pylon/collections/cursor_collection.rb100.0%
New file Coverage rating: A
...pylon/lib/forest_admin_datasource_pylon/collections/contact.rb100.0%
New file Coverage rating: A
...orest_admin_datasource_pylon/collections/contact/serializer.rb100.0%
New file Coverage rating: A
..._pylon/lib/forest_admin_datasource_pylon/query/filter_value.rb100.0%
New file Coverage rating: A
...tasource_pylon/lib/forest_admin_datasource_pylon/datasource.rb100.0%
New file Coverage rating: A
...pylon/lib/forest_admin_datasource_pylon/query/operator_maps.rb100.0%
New file Coverage rating: A
..._pylon/lib/forest_admin_datasource_pylon/collections/writes.rb100.0%
New file Coverage rating: A
...st_admin_datasource_pylon/lib/forest_admin_datasource_pylon.rb100.0%
New file Coverage rating: A
...b/forest_admin_datasource_pylon/collections/base_collection.rb100.0%
New file Coverage rating: A
...ce_pylon/lib/forest_admin_datasource_pylon/collections/team.rb100.0%
New file Coverage rating: A
...ce_pylon/lib/forest_admin_datasource_pylon/collections/user.rb100.0%
New file Coverage rating: A
...rest_admin_datasource_pylon/collections/contact/api_filters.rb100.0%
New file Coverage rating: A
...est_admin_datasource_pylon/collections/fetch_all_collection.rb100.0%
New file Coverage rating: A
..._admin_datasource_pylon/collections/issue/messages_embedder.rb100.0%
New file Coverage rating: A
...e_pylon/plugins/create_issue_with_notification/form_builder.rb100.0%
New file Coverage rating: A
...lon/lib/forest_admin_datasource_pylon/plugins/issue_targets.rb100.0%
New file Coverage rating: A
...pylon/lib/forest_admin_datasource_pylon/plugins/close_issue.rb100.0%
New file Coverage rating: A
...dmin_datasource_pylon/collections/account/schema_definition.rb100.0%
New file Coverage rating: A
...t_admin_datasource_pylon/collections/issue/id_lookup_reader.rb100.0%
New file Coverage rating: A
...source_pylon/plugins/create_issue_with_notification/payload.rb100.0%
New file Coverage rating: A
...rest_admin_datasource_pylon/collections/account/api_filters.rb100.0%
New file Coverage rating: A
.../forest_admin_datasource_pylon/plugins/close_issue/messages.rb100.0%
New file Coverage rating: A
...min_datasource_pylon/plugins/create_issue_with_notification.rb100.0%
New file Coverage rating: A
...asource_pylon/lib/forest_admin_datasource_pylon/issue_enums.rb100.0%
New file Coverage rating: A
...n_datasource_pylon/lib/forest_admin_datasource_pylon/client.rb100.0%
New file Coverage rating: A
...ource_pylon/lib/forest_admin_datasource_pylon/configuration.rb100.0%
New file Coverage rating: A
...forest_admin_datasource_pylon/collections/relation_embedder.rb100.0%
New file Coverage rating: A
.../forest_admin_datasource_pylon/collections/issue/serializer.rb100.0%
New file Coverage rating: A
..._admin_datasource_pylon/collections/issue/schema_definition.rb100.0%
New file Coverage rating: A
...est_admin_datasource_pylon/collections/record_serialization.rb100.0%
New file Coverage rating: A
...orest_admin_datasource_pylon/collections/account/serializer.rb100.0%
New file Coverage rating: A
...forest_admin_datasource_pylon/collections/issue/api_filters.rb100.0%
New file Coverage rating: A
...e_pylon/lib/forest_admin_datasource_pylon/collections/issue.rb100.0%
New file Coverage rating: A
...pylon/lib/forest_admin_datasource_pylon/collections/account.rb100.0%
New file Coverage rating: A
.../lib/forest_admin_datasource_pylon/pagination/cursor_walker.rb100.0%
New file Coverage rating: A
...dmin_datasource_pylon/collections/contact/schema_definition.rb100.0%
New file Coverage rating: A
...rest_admin_datasource_pylon/query/condition_tree_translator.rb100.0%
New file Coverage rating: A
...ource_pylon/lib/forest_admin_datasource_pylon/client/writes.rb100.0%
Coverage rating: A Coverage rating: A
...components/query/condition_tree/nodes/condition_tree_branch.rb100.0%
Coverage rating: A Coverage rating: A
...orest_admin_datasource_toolkit/components/query/aggregation.rb100.0%
New file Coverage rating: A
...asource_pylon/lib/forest_admin_datasource_pylon/rate_limits.rb100.0%
New file Coverage rating: A
...source_pylon/lib/forest_admin_datasource_pylon/rate_limiter.rb100.0%
New file Coverage rating: A
...st_admin_datasource_pylon/schema/custom_fields_introspector.rb100.0%
New file Coverage rating: A
...source_pylon/lib/forest_admin_datasource_pylon/retry_policy.rb100.0%
New file Coverage rating: A
...datasource_pylon/lib/forest_admin_datasource_pylon/throttle.rb100.0%
Total100.0%
🚦 See full report on Qlty Cloud »

🛟 Help
  • Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.

  • Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.

  • File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)

    • Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

* feat(datasource): proactive throttling for the pylon datasource (EXT-13)

Space requests out inside each endpoint's documented budget instead of
discovering the quota as a 429.

RateLimits holds the per-endpoint figures read off the Pylon API reference,
one window per endpoint since Pylon meters per endpoint rather than per
token. RateLimiter keeps a sliding window per bucket, reserves the slot a
request will take under a mutex and waits outside it, so concurrent callers
spread over distinct slots rather than waking onto the same one. Throttle
is the Faraday middleware, placed inside `retry` so a replay waits for a
slot like a first attempt.

The wait is bounded (5s): past that the window is saturated by more than
this agent's own traffic, so the request goes out and the 429 retry stays
the backstop. Queueing behind a full window would trade a retry the client
already handles for a request the operator watches spin.

Also corrects the rate-limit figures the comments and one operator-facing
message asserted. They were 4x to 15x below what Pylon documents: the
search endpoints grant 120 requests a minute rather than 20, the record
reads 300 rather than 60, and no endpoint this gem touches sits at the 10
a minute the story assumed. The caps built on those figures are unchanged
and re-justified on what actually bounds them - sequential round-trips and
unbounded payloads, not the quota.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…on (EXT-17) (#375)

* refactor(pylon): extract the faraday connection builder

Everything but the timeouts is the same on every connection the client
needs, so the builder takes those and holds the rest -- middleware order
and the shared limiter included -- in one place.

No behaviour change: the single connection is now its memoized call.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both conflicts are the sort decorator cherry-pick 837988d, whose
content landed on main as a04900d (#368). main's side is taken on
each, so the two files leave this diff entirely.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
christophebrun-forest and others added 3 commits August 26, 2026 17:06
A Number filter the agent cast to Infinity or NaN reached `to_i`, which
raises, and then the JSON encoder, which raises too: a 500 either way, on
a value the operator typed. It is refused with a 400 naming the field.

An action reading its issue ids off a column deduplicates them: that
column is not a key, so two selected records could name the same issue,
which was then written twice and counted twice in what the action
reported back.

`collect_pages` stops on a cursor it has already followed, not only on
one that did not move: a cursor cycling over three pages walked them
again until the page cap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
MAX_ID_LOOKUPS bounded a selection rather than a page: the ids were
truncated to the first 20 before the window was applied, so an offset
at or past 20 answered an empty page from records that exist, and a
wider selection lost its tail with nothing but a server-side log to say
so.

The window is now taken off the ids before any of them is read, which
also spends one request per record the caller asked to see rather than
one per record in the selection. A residual condition takes that away,
which records the window holds being known only once they are all read,
so past the cap that one selection is refused rather than answered with
a fraction of itself.

Reading by id moves to its own mixin, the collection having reached the
class-length cap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A scope that is neither a symbol nor a string raised NoMethodError from
`to_sym`, hiding the unknown-scope error that names what was passed.

An email template title is both the enum value and the key its content
is looked up by, so it has to name one template: a duplicate made the
first unreachable and sent the other one's content under its name, and a
template titled like the sentinel of the "pick none" option could never
be picked at all. Both are configuration, so both are refused at
registration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
christophebrun-forest and others added 2 commits August 26, 2026 17:11
`project` served the whole record when the projection named no column of
the collection, so a row asked for as `account:name` alone came back
carrying every native column too. The relation is embedded onto the row
afterwards; what `project` owns is the columns, and a projection naming
none asks for none. Only a nil projection still means the record as it
is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pylon exposes no count endpoint and no total, so no collection ever
passed `countable: true`: the cursor-backed ones refuse `aggregate`
outright rather than count the pages they walked, and the two holding
their whole dataset are not advertised as countable either. The flag was
carried by the constructor and exercised by one spec testing the flag
itself.

Recorded on EXT-7, whose scope still lists Count.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
christophebrun-forest and others added 2 commits August 26, 2026 17:16
EXT-17 bounded each boot-time introspection per request, not the three
of them together: a Pylon that hangs cost the full boot bound once per
object type, so a Rails boot could wait around a minute where the
ticket promised twenty seconds.

The first failure now stands for the rest, which it does in practice: a
Pylon that is down, or a token missing the permission, fails the two
that follow the same way. `fetch_custom_fields` degrades to nil rather
than to an empty list so the introspector can tell that apart from an
organization that simply defined no custom field, which says nothing
about the next object type.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Last open item of EXT-13. Covers configuration, the five collections and
what each one can be written through, the action plugins, and the two
things an operator needs to know before deploying: what Pylon cannot do
and how this refuses it rather than answering something that looks
right, and how the per-endpoint throttling behaves under saturation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
christophebrun-forest and others added 6 commits August 27, 2026 16:52
The caveat said a scope on PylonIssue does not bound what the action
closes, on the grounds that the state is written through the client. The
ids are resolved first, through the collection the action sits on, and the
agent intersects the operator's scope into the filter that reads them: so
mounted on PylonIssue, the scope bounds exactly what it closes. What the
caveat describes is the host-collection form, where the ids come off a
column and that column is the authority.

Also names the discrepancy the uncapped batch has with the collections
themselves, which refuse a filter-driven write past MAX_WRITE_REQUESTS.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pylon has no join, so a related record is read through its own endpoint,
which takes no field list: the foreign collection hands back everything it
holds and the embedder nested all of it. A row asked for as `account:name`
came back carrying every column of the account.

The agent redacts a projection per collection since #365, dropping the paths
through relations the caller may not read, so the columns served that way
were also the ones their permissions had taken out. Nothing downstream cuts
them back: RelationCollectionDecorator returns early when the projection is
unchanged, which it is here, no relation being emulated.

The nested record is now cut to the fields the projection named, the way
`project` cuts the row it sits on. This is the relation half of cc01551,
which made the same argument for the native columns.

The primary key is unioned back in rather than assumed: the route's
projection carries it, but the nested record is what the serializer reads an
included resource's id off, and a caller inside the agent may project
without it. By hand rather than through `Projection#with_pks`, which also
walks the relations of the projection it is given and dereferences their
schema with no nil guard -- a path reaching through a relation the
collection does not declare would raise there, where it used to be ignored.

The sub-projections are rebuilt rather than read off the argument: `list` is
also called with a plain array of field names, and a relation read from one
of those would go unprojected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`api_key` is a bearer token behind an attr_reader, and nothing prints a
Configuration on purpose: what reaches an `inspect` is a Rails error page,
or a `logger.debug` of something holding one.

Masking the Configuration is one path of three. The token also rides in the
headers of the client's two memoized Faraday connections, and
`Faraday::Connection#inspect` prints those in clear, so a Client leaks it
whatever the Configuration does about its own. Every collection reaches the
same connections through the `@datasource` the toolkit's Collection keeps.

Configuration, Client and Datasource each mask their own; together they cut
every path from an object this package hands out to the credential. Each
keeps what made it worth printing -- the base url, the collection names --
and the Datasource one also spares the recursive dump the default walks
into, a datasource and its collections pointing at each other.

The spec asserts the whole graph rather than the Configuration alone, so a
new holder cannot reopen a path quietly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`extract_data` hands an envelope carrying no `data` straight back, which is
what a read of an unwrapped body wants. `Array()` then split that envelope
into [key, value] pairs, and the collection serialized them into rows
holding nothing: a page that looks answered and is empty, on every endpoint
read as a list.

The write path already refuses exactly this shape, and for the reason that
applies here too -- `extract_written` says the collection would serialize
the envelope into a record with no id. The read half now says the same, one
extractor per shape, naming the endpoint that broke the contract.

An absent or null `data` stays the empty answer it is: Pylon spells "no
record" that way, and a search matching nothing is not a broken contract.

The two best-effort callers keep degrading rather than raising: a malformed
thread costs the conversation column, and a malformed custom-fields
response boots the datasource on its native schema.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The walk compared the next cursor to the previous one alone, so it caught a
cursor that did not move and nothing wider. A cycle spanning two pages ran
until MAX_PAGES or MAX_RECORDS cut it short, and handed the same records
back several times over as if they were distinct rows.

`Client#collect_pages` already keeps the whole set, that being the
hardening b9cb7d1 landed on it; the walk now stops on the same terms. The
loop moved into a private `collect` to keep `walk` at what it does -- bound
the window and slice it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PylonUser and PylonTeam read their endpoint in one response, so the records
in hand are every record Pylon holds and a count over them is the figure a
server-side count would have given. `FetchAllCollection#aggregate` already
answered exactly for that reason, but the collections never called
`enable_count`, so the route short-circuited and the panel showed no total
it could have served on a request the list makes anyway.

They are the exception the refusal was written around: every cursor-backed
collection stays uncountable, counting the pages a walk collected being a
fraction of a collection presented as the whole of it.

Nothing else reads the flag -- Count and CountRelated are its only two
readers in the agent, and no OneToMany points at either collection, so no
related count is opened by this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
christophebrun-forest and others added 2 commits August 28, 2026 11:10
`ConditionTreeBranch#match` recursed through `every_leaf` / `some_leaf`,
which walk all the way down to the leaves. The `all?` of an `And` was
therefore applied to the leaves of an `Or` nested inside it, and the
aggregator of that inner branch was simply lost: `And(a, Or(b, c))` was
evaluated as `And(a, b, c)`.

Two ways that surfaced. A scope combined with an "any of" filter -- the
shape every route builds, `ConditionTreeFactory.intersect` flattening
same-aggregator children only -- silently dropped the rows matching one
side of the union. And a nil-unsafe comparison paired with a presence
check, the guard a datasource wraps `less_than` / `greater_than` in,
stopped guarding: read as `any?`, the presence check answered false on
the null and the comparison was evaluated anyway, raising NoMethodError.

Recursing through `match` fixes both. For a branch holding leaves only
the two forms are equivalent -- a leaf yields itself to either walker --
so nothing changes there, and an empty branch keeps answering what it
did: `Or []` false, `And []` true.

Only three call sites read this outside the toolkit: the validation
decorator, and the two in-memory passes of the Pylon datasource.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The plugin writes through the client rather than through the collection,
being registered on the host collection rather than on PylonIssue, so
none of the caps `Writes` defends applied to it: `apply_state` looped
over every id it was handed, one PATCH apiece, sequentially.

On a bulk action over a wide selection that is a run the request times
out on, leaving part of the selection moved and reporting which part to
nobody, `apply_state` only naming what landed once the loop returns.
Selected on PylonIssue itself the ids are bounded by the cursor walk, at
5000; on a business collection carrying issue ids they are bounded by
nothing at all.

The batch is now capped at `MAX_TARGETS`, read off the very constant a
filter-driven `update` or `delete` already spends, so the two move
together. Past it the run is refused before its first write, with a
message naming the count and the cap. The cap counts the issues named
rather than the records selected: a column of issue ids is not a key, so
a hundred host records naming ten issues stays a batch of ten.

The README documented the fan-out as a known limitation; it now
documents the bound.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
matthv

This comment was marked as outdated.

@matthv

matthv commented Sep 1, 2026

Copy link
Copy Markdown
Member

Review of the assembled branch at 3e38d094 — the important points (full detail available on request):

  1. update reports success on a patch it dropped entirelycollections/writes.rb:53-54. writable_attributes silently drops unknown and read-only fields; a patch reduced to nothing hits return if attributes.empty? and the agent reports a successful edit that never landed — the exact failure the module's own preamble says it refuses. A non-empty patch with zero writable attributes should raise UnsupportedWriteError naming the dropped fields.

  2. One Pylon 429 episode can hold a server thread for ~3 minutesretry_policy.rb:52. faraday-retry sleeps the raw Retry-After whenever it is ≤ max_interval (65), so max_retries: 3 × Retry-After: 60 is up to 180s of blocking sleep, plus up to 4 × 5s of Throttle waits (the throttle sits inside the retry); f.options.timeout covers only the adapter. A few concurrent list views park most of a Puma pool during a 429 episode. Worth a wall-clock budget per request, or max_retries: 1 at this interval.

  3. Client#fetch_all ignores the response's pagination blockclient.rb:157-159. FetchAllCollection builds its exactness guarantee — including the package's only exact aggregate — on "the records in hand are all of them". If Pylon starts paginating GET /users/GET /teams (the client already follows cursors defensively on /messages for this very eventuality), the first page silently becomes "everything": counts and filters turn into fractions presented as exact, with no log and no error. One if pagination['has_next_page'] — follow with collect_pages, or raise — buys the tripwire.

  4. Missing test: residual + page window combinedspec/.../collections/issue_spec.rb:604. Every residual test uses a filter without a page, so the ordering in records_by_id — apply the residual first, slice page_window second — is pinned by nothing; a regression slicing first would answer pages with silently missing records and pass the suite. One example locks it: And(id IN [i1,i2,i3], state = new) where only i2/i3 match, page(offset: 1, limit: 1) expecting [i3].

Also: the latest Macroscope finding is not addressedclient.rb:134 (2026-08-28, High: Client#inspect prints @configuration.url unredacted, so a base URL carrying user-info leaks credentials into logs/error pages). No reply in the thread and the code at 3e38d094 is unchanged.

Four findings from the review of the assembled branch, each with the spec
that pins it.

- update() dropped every read-only and unknown key of a patch and then
  returned, so a patch reduced to nothing was reported as a successful
  edit -- the route re-reads the record and hands back the values it
  already had, showing the operator their change reverting with no reason
  given. Refused now, naming the keys. A dropped key alongside a real edit
  is still dropped, and a patch naming no field at all still asks for
  nothing.

- RetryPolicy honoured a Retry-After of up to a full minute on each of
  three retries, parking the calling thread for three minutes on a request
  the Forest server had long since timed out, and multiplied by the 20
  sequential calls one list view may spend. DEFAULT_MAX_INTERVAL is 12
  rather than 65, the figure that keeps the sleeps of a whole request under
  a minute; past it a 429 surfaces as an error instead of being waited out.

- Client#fetch_all ignored the pagination block, so the day Pylon
  paginates GET /users or GET /teams its first page would silently become
  "everything" and FetchAllCollection would keep calling its filter, sort
  and count exact over a fraction of the collection. The cursor is
  followed now, and refused rather than truncated past the page cap.

- A residual combined with a page window was pinned by nothing: every
  residual spec used a filter without a page, so a regression slicing the
  ids before applying the residual would have answered pages with records
  silently missing and passed the suite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One conflict, in .github/workflows/build.yml, resolved in favour of main.

#380 replaced the lint job's per-package matrix with a single job running one
root `bundle exec rubocop`, so the third place this branch registered the
package no longer exists. Its two other entries auto-merged and are kept: the
test matrix and the "Send coverage" files list, 13 packages each.

Nothing is lost by dropping the lint entry -- the root .rubocop.yml has no
Include, so one rubocop run already walks packages/**, which is what main's
CLAUDE.md now says to rely on. That same CLAUDE.md edit fixes the step-5
instruction this branch had followed, so the guidance and the workflow agree
again.

Verified on the merged tree: toolkit 485, customizer 703, pylon 815, agent
1179, active_record 203 examples, 0 failures; rubocop 911 files, 0 offense.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`base_url` is operator-supplied, and an egress proxy fronting Pylon is
spelled `https://user:pass@proxy.internal`: that user-info is a credential,
and both places that print the url printed it in clear -- a Rails error
page or a `logger.debug` of a Client or a Configuration.

Configuration#redacted_url is now what the two inspects print. The finding
named Client#inspect alone; Configuration#inspect printed `@base_url` the
same way, so masking only the one it named would have left the other path
open.

Cut with a regexp rather than through URI: nothing validates that the base
url parses, and an inspect raising on the way to a Rails error page would
replace the page with its own failure. A spec pins that, and one pins a url
carrying no user-info being printed as it is -- placing the deployment is
what makes an inspect worth printing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The eighth review pass, verified one by one. Six of its twelve findings led
to a change; the six others are answered in thread.

- `collect_pages` treated two of its four stops as the end of the
  collection. An empty page advertising a cursor, and a cursor answered
  twice, both left records behind and `fetch_all` still called the result
  exact -- the same hole the previous commit had closed for the page cap
  alone. Only `next_cursor` being nil is Pylon saying there is nothing
  more; every other stop is now a truncation, which the caller reads as a
  warning or as a refusal.

- `coerce_number` reached an integer through a Float, so a custom field
  holding more than 2**53 was displayed and compared with its last digits
  rewritten. Parsed as an integer first, in base 10 explicitly: Ruby reads
  `Integer("012")` as octal, and the suggested patch would have renumbered
  every zero-padded value Pylon holds.

- `FilterValue` checked the timezone stripped and then stored it unstripped,
  so a padded identifier passed the blank guard, failed the zone lookup and
  fell back to UTC -- a day boundary off by the offset.

- `CloseIssue` registered one action per scope through `add_action`, which
  keys them by name: two variants configured with the same name left the
  second overwriting the first, so the panel offered one action scoped for
  the wrong selection and nothing said the other had gone. Refused at
  registration, like the plugin's other silent misconfigurations.

Two outside the package, both found by verifying a finding rather than in
it:

- `Charts#compute_value` indexed `result[0]['value']` while
  `Aggregation#apply` answers an empty selection with no row at all, so a
  filtered value chart matching nothing was a 500 where the figure is zero.
  Reachable from every collection that aggregates in memory, `PylonUser`
  and `PylonTeam` being the first to do so exactly.

- `Aggregation` compared `Max` with `<`, so it kept the smallest value it
  saw: every Max chart over an in-memory aggregation answered with the
  minimum. No spec covered Max at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… read

`:action_name` taken as a Symbol was registered as one, and a Symbol name
breaks the agent rather than the action: `GeneratorAction.get_action_slug`
calls `strip` on it, and `GeneratorCollection` sorts the names of a
collection's actions, which raises `comparison of Symbol with String
failed` as soon as one sits beside a String. Schema generation, so the
whole agent, for a plugin option typed the way the `:scopes` option
already accepts.

Coerced through `to_s`, like `normalize_scopes` does -- which also makes
`refuse_colliding_names` read `:Resolve` and `'Resolve'` as the one name
they are, where it had just let them register as two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@christophebrun-forest
christophebrun-forest merged commit cf20168 into main Sep 1, 2026
33 checks passed
@christophebrun-forest
christophebrun-forest deleted the feat/datasource-pylon branch September 1, 2026 13:50
forest-bot added a commit that referenced this pull request Sep 1, 2026
# [1.41.0](v1.40.1...v1.41.0) (2026-09-01)

### Features

* **datasource-pylon:** Pylon datasource for the Ruby agent (EXT-4) ([#369](#369)) ([cf20168](cf20168)), closes [#380](#380)
@forest-bot

Copy link
Copy Markdown
Member

🎉 This PR is included in version 1.41.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants