Skip to content

Latest commit

 

History

History
159 lines (121 loc) · 6.35 KB

File metadata and controls

159 lines (121 loc) · 6.35 KB

Where this style and Harper disagree

Two kinds of difference: rules this style does not port at all, and findings it makes that Harper does not. The second kind is listed in full, because it is the one a reader notices.

📌 The pin

Harper v2.7.0, commit dc1182c (HARPER_REF in the Makefile)
Vale v3.17.0
Corpus corpus.txt, 667,584 bytes, built by cmd/corpusgen from that checkout
Measured on 2026-08-04, Apple M1, macOS 15.7.4
make measure
harper findings : 3099
we reproduced   : 2972
we missed       : 127
we invented     : 3
accepted        : 2

recall     95.9%
precision  99.8%
F1         97.8%

The comparison is -ported, so it covers only the rules this style claims to implement; without that flag it reports the whole distance to Harper, spelling included. Rerunning against a different Harper changes every number here.

Findings are matched by overlap rather than equality. Two tools rarely agree on the exact span — Harper flags GB where a rule here flags 16GB — and demanding the same span would count that one finding as both a miss and a false positive.

🔍 The five findings Harper does not make

Three are defects. The other two are findings this style is right to make and keeps, recorded in accepted.tsv. Each was re-run on its own sentence, alone in a file, to separate the ones that survive from the ones the corpus manufactures.

Right, where Harper is silent (2)

rule text suggests
ItsPossessive FYI, USL is changing it's tact internally about MLS II teams. it'sits
ItsPossessive FYI, USL is changing it's tack internally about MLS II teams. it'sits

changing its tack is correct English and changing it's tack is not. Harper's tagger declines to call tact or tack a noun there, so its own rule never fires. Counting these against the port is an artifact of measuring against Harper rather than against English.

Simply wrong (3)

rule text suggests why it is wrong
ItsPossessive They doubt it's Tesla this year. it'sits it is Tesla; the contraction is right
Everyday Our everyday make up stays in this drawer. everydayevery day the adjective, modifying make up
ThereIsAgreementPluralBe if there were bias, the average size of the classes verb/noun number bias is not countable here

Three in 2,977 findings, 0.10%. All three are plausible suggestions in a context that happens not to want them.

🧱 Why these five and not fewer

Four of the five come down to Harper's tagger and Vale's disagreeing about one word:

I doubt it's compression this year.    Harper flags
They doubt it's compression.           Harper is silent

He is changing it's direction.         Harper flags
He is changing it's tact.              Harper is silent

The structure is identical in each pair. Harper's tagger calls doubt a verb after I and a noun after They; it declines to call tact or Tesla a noun where Vale's does. No pattern reaches that, because the rules already agree — the taggers do not.

The fifth is the same disagreement about a different property. there were bias is a number mismatch by Vale's reading, which has bias as a singular count noun; Harper reads it as already plural and says nothing.

⚖️ Why the rules are not dropped

Three rules produce all five. Removing them would take precision to 100%:

rule reproduces invents net
ItsPossessive 36 1 +35
Everyday 20 1 +19
ThereIsAgreementPluralBe 52 1 +51

That costs 108 of Harper's findings to remove 3 disagreements — thirty-six correct findings for each one — and two of the five are cases where this style catches an error Harper misses. A rule is only free to drop when it is net zero or worse, and these three are not.

The ThereIsAgreementPluralBe row is the family's whole count: Harper reports all of this under ThereIsAgreement, so there is nothing finer to divide between the six rules covering it here.

Excluding the individual words instead (tact, Tesla, doubt, bias) would reach zero at almost no cost to recall, and is worse than either: it's tact is exactly the error ItsPossessive exists to catch.

🚧 What is not ported

Vale's tagger is less certain about broken sentences. Vale tags parts of speech with a statistical model that reads context; Harper leans on its dictionary. The model is the more accurate of the two on ordinary prose — but a grammar rule only ever runs on text that is already wrong, which is exactly where context stops helping:

aware   JJ   She was aware of the risk.               <- ordinary prose
aware   NN   They were all ready aware of the risk.   <- what the rule sees

Shipping Harper's dictionary helps, but only so far. model: is a lexicon laid over the model, not a replacement: where it lists one tag it decides, and where it lists several the model's answer stands. Harper gives better three readings, so the model decides, and it reads the following word:

better than the other   ->  better/JJR
better then the other   ->  better/NN

A rule written against JJR to catch then/than confusion is therefore defeated by the confusion itself. Rules triggered by a misspelling name their words outright instead of asking for a tag; Harper/ThenThan.yml, WereWhere.yml and SimplePastToPastParticiple.yml all do, and say so.

Some of Harper's rules count evidence instead of matching patterns. ConvenientStore weighs how many nearby words suggest a shop against how many suggest a database. Vale's checks match; they do not score. Not ported.

Some of Harper's rules are not ones Harper runs. Nine of its 823 linters ship disabled, and two more — HopeYoure and TakeSerious — exist as Rust with passing tests but never join its lint group, so Harper cannot report them however it is configured. cmd/rustgen skips them, and cmd/offcheck rechecks the list on every make verify.

One Harper rule can become several Vale rules. A sequence matches a fixed number of tokens, so a Harper rule accepting phrases of several lengths becomes a numbered family. They keep the original name, so disabling Harper.Everyday still disables all of it.

None of these produces a false correction, which is the property worth keeping.