Skip to content

feat(imap): APPEND with an explicit INTERNALDATE and from raw bytes - #292

Open
hobleyd wants to merge 2 commits into
Enough-Software:mainfrom
hobleyd:feat/imap-append-internaldate-raw-bytes
Open

hobleyd wants to merge 2 commits into
Enough-Software:mainfrom
hobleyd:feat/imap-append-internaldate-raw-bytes

Conversation

@hobleyd

@hobleyd hobleyd commented Sep 22, 2026

Copy link
Copy Markdown

Two additions to APPEND, both needed when copying a fetched message into another mailbox or account (for example, migrating an account).

internalDate on appendMessage / appendMessageText

APPEND takes an optional date-time argument (RFC 3501 section 6.3.11) that the server stores as the message's INTERNALDATE. Without it the server uses the time of the append, and since most servers sort and display by INTERNALDATE every copied message lands at "now". The new optional parameter is formatted as "05-Jan-2026 10:00:00 +0000" (always UTC) and placed between the flags and the literal size.

appendMessageBytes

appendMessageText computes {n} over utf8.encode(text) and the socket's IOSink.write sends UTF-8, which agree for a message rendered as a String. A message fetched with BODY[] is not that: it is CHAR8 in whatever transfer encoding it was sent with. Turning it into a String to go through appendMessageText re-encodes every byte above 0x7F as two bytes on the wire, corrupting the message and making the declared literal length disagree with what is sent, which desyncs the connection.

appendMessageBytes(Uint8List) computes {n} over the bytes and sends the same bytes via a new Command.withRawContinuation, which onContinuationResponse writes with writeData instead of writeText. Existing text-based continuation commands are untouched.

The APPEND command-line construction shared by both paths is factored into _buildAppendCommandText.

Tests

Three new tests in imap_client_test.dart: internalDate produces the RFC 3501 date-time; a non-UTC DateTime is sent in UTC; appendMessageBytes sends a byte outside 7-bit ASCII unchanged with a matching {n}. This branch is stacked on the mailbox-parentheses PR (it shares the MockImapServer.requests recorder introduced there), so its first commit appears here until that PR merges; the APPEND change is the second commit.

No CHANGELOG entry added; happy to add one under a heading of your choice.

Two independent failures for a mailbox such as INBOX.Financial.Audit(s):

ListParser, for an extended LIST response (LIST-STATUS / LIST-EXTENDED
return options), found the extended-data block with indexOf('(') which
matched the '(' inside the quoted mailbox name. The name was truncated
(taking the closing quote with it), so the later quote-strip ate the
final character and the mailbox surfaced as 'Aud' at a path that did not
exist on the server. It now looks for the first '(' outside a
double-quoted string.

_encodeMailboxPath only quoted a path containing a space. RFC 3501
section 9 lists '(' ')' '{' among the atom-specials, so an unquoted
`COPY 1:3 INBOX.Audit(s)` (or SELECT, MOVE, CREATE, ...) is rejected by
servers with "BAD Invalid characters in atom". Those characters now
force quoting as a space does.

MockImapServer records what the client sends so tests can assert on the
command text actually written.
Two additions to APPEND, both needed when copying a fetched message into
another mailbox or account:

appendMessage/appendMessageText gain an optional `internalDate`, sent as
the command's optional date-time argument (RFC 3501 section 6.3.11) so
the server records the original date as INTERNALDATE rather than the
time of the append. Most servers sort and display by INTERNALDATE, so
without it every copied message lands at "now".

appendMessageBytes appends already-encoded MIME source as bytes.
appendMessageText computes `{n}` over utf8.encode(text) and the socket
writes UTF-8, which agrees for a message rendered as a String. A message
fetched with BODY[] is not that: it is CHAR8 in whatever transfer
encoding it was sent with, and turning it into a String to go through
appendMessageText re-encodes every byte above 0x7F as two bytes on the
wire, corrupting the message and desyncing the declared literal length.
Command.withRawContinuation carries the literal as bytes to writeData so
the declared and transmitted lengths stay identical.

The shared command-line construction is factored into
_buildAppendCommandText. MockImapServer records the last request so
tests can assert on the command text actually sent.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants