Skip to content

lore: sort emails on a total order - #115

Open
vladwing wants to merge 1 commit into
facebookexperimental:mainfrom
vladwing:lore-sort-total-order
Open

vladwing wants to merge 1 commit into
facebookexperimental:mainfrom
vladwing:lore-sort-total-order

Conversation

@vladwing

Copy link
Copy Markdown
Contributor

semcode lore -b nvme_tcp_queue --limit 0 aborts with "user-provided comparison function does not correctly implement a total order". Any result set that mixes Date headers that parse with ones that do not can trigger this panic.

sort_emails_by_date() compares parsed dates when both Date headers parse and raw strings otherwise, which is not transitive. With A = "Tue, 01 Jan 2030 00:00:00 +0000", B = "U-not-a-date" and C = "Wed, 01 Jan 2020 00:00:00 +0000", it says A < B and B < C by string but C < A by date. Rust's sort panics when it detects such a comparator.

Sort on a key instead: emails whose date parses come first in time order, the rest follow by raw header, and ties break on the raw header and then on message_id. Same-second emails now come out in the same order on every run, whatever order the query returned them in. The function's signature and its five callers are unchanged. The key is computed once per email, so each Date header is parsed once rather than on every comparison.

The newest-first sort in dig was already a total order; its same-instant ties now break on message_id too.

Both new tests fail on the old comparator. One sorts the A/B/C triple from all six input orders. The other sorts 2,000 emails, a third of them with unparseable dates and most sharing a timestamp, from two fixed permutations, and checks the order and that both permutations give the same result.

`semcode lore -b nvme_tcp_queue --limit 0` aborts with "user-provided
comparison function does not correctly implement a total order". Any
result set that mixes Date headers that parse with ones that do not can
trigger this panic.

`sort_emails_by_date()` compares parsed dates when both Date headers
parse and raw strings otherwise, which is not transitive. With
A = "Tue, 01 Jan 2030 00:00:00 +0000", B = "U-not-a-date" and
C = "Wed, 01 Jan 2020 00:00:00 +0000", it says A < B and B < C by
string but C < A by date. Rust's sort panics when it detects such a
comparator.

Sort on a key instead: emails whose date parses come first in time
order, the rest follow by raw header, and ties break on the raw header
and then on `message_id`. Same-second emails now come out in the same
order on every run, whatever order the query returned them in. The
function's signature and its five callers are unchanged. The key is
computed once per email, so each Date header is parsed once rather than
on every comparison.

The newest-first sort in `dig` was already a total order; its
same-instant ties now break on `message_id` too.

Both new tests fail on the old comparator. One sorts the A/B/C triple
from all six input orders. The other sorts 2,000 emails, a third of them
with unparseable dates and most sharing a timestamp, from two fixed
permutations, and checks the order and that both permutations give the
same result.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 11, 2026
@vladwing vladwing closed this Nov 11, 2026
@vladwing vladwing reopened this Nov 11, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant