Skip to content

Latest commit

 

History

History
339 lines (245 loc) · 13.6 KB

multiple-collections.md

File metadata and controls

339 lines (245 loc) · 13.6 KB

More than one collection

Status: built, 2026-07-28. One token may now own several collections. The reasoning that led here is kept below the line; how to undo it is at the end and is the reason this page exists.

Before this, a Telegraph token owned exactly one index and a second collection meant a second identity. That still works, and is still the stronger separation — separate tokens mean a leak of one touches nothing else.


What was built

The model is primary plus curated extras, the option argued for below.

Primary One per token. /site enumerates the whole account into it, exactly as before. An index written before this change has no collection field and is therefore primary — which is what makes the change reversible.
Curated Any number. Entries arrive only through /link, /repo and the extension. /site refreshes the title, masthead, footer and byline but never fills the list.

The one format change

Extras carry a field on the marker line already used for byline=:

Index generated by Telepatch. byline=linked collection=extra

That is the whole data change. Nothing else about an index differs.

Commands

Command
/collections Lists every live collection, one message each so any of them can be replied to
/newsite <name> Starts a curated collection, empty
/retire Reply to a collection to take it out of service

/about, /footer, /byline, /link and /repo are unchanged when replying to a collection's message. Invoked bare with more than one collection they now refuse and ask rather than guessing — /about and /footer overwrite prose, so a wrong guess destroys writing.

/site invoked bare still rebuilds the primary. That is deliberate: it is the one unambiguous answer, and it keeps the old behaviour for anyone who never asks for a second collection.

Writing for a collection

/post replying to a collection's message writes for that collection: the page is created as usual and then added to that index, with its reading time and opening paragraph, because a curated collection is never enumerated and the entry would otherwise never appear. Replying to the identity message writes for the primary, as before.

The path travels on the ForceReply carrier, which is a hidden link with no length limit — the same reason /revise can carry a page path while a button cannot.

Filing

The primary is the inbox. A collection is a folder. Filing moves a page.

A page listed in any live curated collection is left out of the primary when /site rebuilds it. One rule, no exceptions — not for hand-added entries, not for nav pages.

It needs no marker and no extra request, because membership is already written down in the collection's own list, and scan_pages has fetched every one of those lists on its way past.

Three things follow, and all three are deliberate:

  • /site says what it left out — "3 filed in collections, left out". A page quietly missing from a rebuild run for some other reason is how trust in this goes.
  • A retired collection claims nothing. Otherwise retiring one would strand its pages outside every index at once.
  • Only telegra.ph pages can be filed. A GitHub or outward entry was never in the account listing to begin with.

/unfile is the way back, and was built in the same change rather than after it. Without it a stray /link would hide a page from the primary for good, short of editing Telegraph by hand. Reply to a Published message and it needs no argument at all — that message carries both the collection and the page. It also removes GitHub and outward entries, which nothing else could do, and is aliased /unlink for that reason.

What this costs: cross-references. A collection can no longer point at an article as a mention while leaving it in the primary. Linking means moving. Keeping both would need a per-entry flag in the collection's list — possible, since that line is already parsed — but one rule learned once is worth more than the refinement.

The trap that had to be closed first

Five separate commands rewrite an index, and every one of them rebuilt the marker line from scratch. Left alone, editing the footer of a collection would have silently demoted it to a primary. They all now go through kept_marker(), which preserves what the caller did not come to change.

Anything added to that line in future must be preserved there too.


What already works

find_site() — the function behind /about, /footer, /byline, /link and /repo — already prefers a path carried in the message and only falls back to guessing:

reply carries site=<path>   →  use it
no carrier                  →  pick_master()

So replying to a particular Site message already edits a particular index. The per-collection editing path needs no redesign. This is the carrier design paying off: because the path travels in the message rather than in the bot's memory, the code was never allowed to assume a singular current index, and most of it does not.

The one broken case is the fallback, where nothing carries a path. pick_master() returns the newest page bearing the marker. With several indexes that is no longer an answer, it is a coin toss — and for /about and /footer, which overwrite prose, a wrong guess destroys writing.


What genuinely breaks

Not a plumbing problem. /site enumerates every page on the account and lists them all, skipping only pages that carry an index marker.

Put two collections on one token, run /site on each, and both list every article on the account. Two identical collections. Nothing in an article says which collection it belongs to, so nothing can tell them apart.

Multiple indexes per token therefore forces a decision about what an index is:

Model Consequence
Keep auto-enumeration One collection per token forever. Today's design.
Tag each article with a collection Needs a marker inside every post and a filter in /site. Pollutes the visible category line unless a reserved prefix is invented.
Indexes become curated only /site stops enumerating; entries arrive only through /link, /repo and an explicit add. Loses "publish and it appears," which is most of why the bot is pleasant.
Primary plus curated extras One auto-enumerating index per token, and additional hand-built ones alongside it.

Leaning: primary plus curated extras. It keeps publishing effortless for the collection that matters most, and treats the others as what they actually are in practice — curated sets, not dumps of everything. It is also the only option that changes no existing behaviour for anyone who never asks for a second collection.


Command impact

Unaffected. Account-scoped rather than index-scoped:

/new · /manage · /views · /privacy · /telepatch · /howto · /revise

Need a "which one?" answer when invoked bare. Replying to a Site message already works; it is the bare invocation that currently picks the newest index silently:

/about · /footer · /byline · /link · /repo

Change meaningfully:

  • /site — no longer the index. Bare invocation has to list collections rather than assume one, and needs a way to distinguish "make a new one" from "rebuild an existing one."
  • /pages — should say which collection each page appears in, or it stops being useful for deciding what to curate.
  • /post — unchanged if a primary index keeps auto-enumerating. Its Published message could grow Add to… buttons.

New commands:

  • /collections — list them with site links; set the default
  • create — either /newsite or /site new <title>
  • /retire — now genuinely necessary. Telegraph cannot delete a page, so with several indexes there has to be a way to take one out of service.

Two hard constraints

callback_data is full

Telegram caps it at 64 bytes. It currently holds <action>:<token> — one character, a colon, and a 60-character token. 62 bytes. Two to spare. A collection path cannot travel alongside the token in a button.

The workaround already exists in the codebase: buttons carry action and token as now, and the message they are attached to carries the path in a hidden text_link, which has no length limit. Both are read together. It is how /revise already carries a page path.

This is the third time the same limit has produced the same answer — move the data into a carrier with no ceiling, and keep the constrained channel for the minimum that must be there. The deep link, the buttons, and now collection paths.

The pinned message becomes plural

The whole interaction model is reply to the pinned message. One identity, one anchor. Several collections means several Site messages to reply to, and replying to the wrong one silently edits the wrong site.

A /collections message with one button per collection is the fix, but it means the anchor concept itself becomes plural. That is the part users would actually feel, and it is a bigger change than any single command.


An accident of foresight

RETIRED_MARK — "Retired Telepatch index." — is defined in bot.py, and pick_master() already refuses to rewrite a page carrying it.

Nothing has ever written it. It is an escape hatch: add that line by hand in the Telegraph editor and the page stops being the master.

/retire is the feature it was waiting for. Worth noticing that the reason it is needed — Telegraph has no deletePage — is the same constraint that made /site rewrite in place rather than create, which is exactly what makes multiple collections awkward in the first place. One missing verb, three consequences.


The extension

extension/ supports both shapes already, and they are complementary rather than alternatives:

Several tokens Works today. Paste each; they stack in the dropdown. Hard isolation — one token leaking does not touch the others.
Several indexes per token discover() already keeps every marked page rather than stopping at the first.

Whichever way the bot goes, the extension needs no structural change. Two small things would want doing:

  • Honour the retired marker. The extension matches a paragraph starting with Index generated by Telepatch. and does not check for RETIRED_MARK. A hand-retired page would still appear as a live collection. A real divergence between the two front-ends, small and only reachable by retiring something manually.
  • Order the dropdown once there are more than a handful.

Rollback

The feature was built to be removable. Two things have to happen, and the order matters.

Why the order matters

Reverted code does not know what collection=extra means. Its pick_master() takes the newest page carrying Index generated by Telepatch. — which, if a curated collection is newer than the primary, is the wrong page. /site would then rebuild a curated collection as though it were the account index, replacing a hand-made list with everything.

Retiring first prevents that. RETIRED_MARK is understood by the old code as well as the new, so a retired page is invisible to both.

Verified, using the same functions the bot uses:

with an extra still live : extra-1     ← wrong page
after /retire on extras  : primary-1   ← correct
extras listed as articles: False       ← never, in either state

The procedure

  1. Retire every curated collection. /collections, then reply to each curated one with /retire. /retire refuses to touch a primary, so this cannot go too far.
  2. Revert the code. git revert the commits.

Nothing else is needed. No page has to be edited by hand, and no data is left in a state the old code misreads.

Filing adds no step. It is a filter, not a format: nothing is written to Telegraph to record that a page is filed, so removing the filter puts every filed page back in the primary on the next rebuild. Retiring the collections in step 1 has the same effect even before the code goes — a retired collection claims nothing.

Verified with the bot's own functions:

claimed by a live collection : Filed-One, Filed-Two
primary lists                : Stranded, Loose page
without the filter           : Filed one, Filed two, Stranded, Loose page

What cannot be undone

Telegraph has no deletePage. Every collection ever created still exists and its link still works for anyone holding it. Retiring removes it from /collections, from the extension, and from anything that writes — it does not remove it from the web. That is a property of Telegraph, not of this feature, and it is the same reason /site rewrites in place rather than creating.

If a page was retired by hand

Adding Retired Telepatch index. in the Telegraph editor also works, and is understood by the bot, the site and the extension. To bring a collection back, edit its marker line to read:

Index generated by Telepatch. byline=linked collection=extra

Drop collection=extra and it becomes a primary — which, if a primary already exists, gives the account two. Only the newest would be found.


Open questions

  1. Is a collection a view of an account's pages, or a thing pages belong to? Answered: a thing pages belong to. See Filing above.
  2. If articles get tagged with a collection, where does the tag live so that a page written by hand in the Telegraph editor still works?
  3. Should /new stay the recommended way to separate concerns? Separate tokens give real isolation that separate indexes on one token do not.
  4. What happens to a collection when its identity is revoked?