diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index ae847d4..e4963eb 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -135,9 +135,10 @@ jobs: # easyvista_python_client.__version__ -- and the git tag is a third. PyPI # takes whatever pyproject says, so a tag that disagrees publishes a # release nobody can find by version, and a __version__ that disagrees - # misreports at runtime. Both are unfixable after upload. Tags in this - # repo are v-prefixed (see the CHANGELOG compare links), so the leading v - # is stripped before comparing. + # misreports at runtime. Both are unfixable after upload. The repo's only + # existing tag, 0.1.0, is UNPREFIXED; v-prefixing starts at v0.2.0. The + # leading v is therefore stripped before comparing, so both forms + # validate. - name: Validate release tag matches package version if: github.event_name == 'release' shell: bash diff --git a/CHANGELOG.md b/CHANGELOG.md index 049457d..7142dd8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,8 +9,91 @@ a deprecation policy will follow the 1.0 release. ## [Unreleased] +### Removed + +- **BREAKING**: `RequestUpdate.status_id`. There is no flat status update on this + API and this field never worked. Sent alone the PUT is rejected 590/2013; sent + beside any other field the PUT returns **200, applies the other field, and + drops the status in silence** — measured on one ticket, title updated, + `STATUS_ID` unchanged. A write that reports success and stores nothing is worse + than one that fails, so the field is gone; `extra="forbid"` now makes + `RequestUpdate(status_id=...)` raise at construction. Use `set_status`. + +### Added + +- `EasyvistaClient.set_status` / `AsyncEasyvistaClient.set_status` set a ticket's + status, addressed by `STATUS_GUID`. This sends the documented + `{"closed": {"status_GUID": ...}}` body — the same request `close_ticket` + sends, under a name that matches what it does. Despite the wire name, the + envelope is **not** limited to closing: handed each of six different status + GUIDs in turn, a fresh ticket landed on exactly the status requested every + time, non-terminal ones included. Status GUIDs are per-instance configuration + and are not portable between deployments; read one off any ticket already in + that status (the nested `STATUS` object carries `STATUS_GUID`). + +### Fixed + +- `PostRequest`'s docstring claimed a ticket "needs at minimum `catalog_code` + plus `title`". That was wrong in a way that cost real debugging time. **Send + the whole documented create body** — `catalog_code`, `origin`, `title`, + `description`, `department_id`, `urgency_id`, `impact_id`. The full body is + accepted everywhere tried; the same body minus those ids is accepted on some + catalogs and rejected on others with the *identical* remaining bytes. The + rejection's message is a bare **SQL parser error** naming no field + (`=(1,35) expected token:( * + - . IDENTIFIER CASE NOT JOIN ...`), which reads + like a server-side defect and is not one — it is what an under-specified create + looks like here. Every id in that body was verified to persist by reading it + back under an explicit projection (these columns are absent from the default + projection, like `TITLE`, so an unprojected read shows `None` regardless). +- Documented that **a rejected create may still have created the ticket**: 12 + attempts returned 3 `RFC_NUMBER`s and afterwards all 12 tickets existed. A 590 + therefore means *possibly created*, never *not created* — retrying duplicates, + and the caller never learns the id. The `external_reference` marker does + survive the failed insert and is searchable, which is the only way to reconcile + such an orphan. +- `integration_tests/test_live_smoke.py` leaked one ticket per live run. Its + `test_missing_mandatory_field_raises_validation_error` asserted that a create + with a catalog but no title is rejected "(no ticket created), so this stays + read-only-safe by construction" — both halves false: `title` is not the + mandatory field (the full documented body with no title creates fine), and the + rejection does create a row. Replaced by + `test_an_underspecified_create_body_raises_validation_error`, which omits the + ids that really are required and reconciles the leftover ticket by its marker + in a `finally`. Two tests added beside it: one pinning that the documented body + lands every id, one pinning that `set_status` reaches a **non-terminal** + status. + +## [0.2.0] - 2026-08-18 + ### Added +- `EasyvistaClient.stream_document` / `AsyncEasyvistaClient.stream_document` + yield an attachment's bytes in chunks (64 KiB by default, `chunk_size=` to + change it) instead of returning them whole. Motivation: a consumer mirroring + attachments had no choice but to buffer, because `download_document` + materialises the whole file before it returns anything. What this removes is + that download buffer and only that — one attachment's worth of memory, so a + 32 MB file is held a chunk at a time instead of whole. The upload leg is + unaffected: `add_document` takes `content: bytes` and base64-encodes it, so a + mirror that re-uploads still materialises that payload in full (see below for + why no streaming upload is possible). Accepts exactly what `download_document` + accepts and resolves the URL identically, so the same-origin refusal, the + `follow_redirects` behaviour and the error mapping (a 403 is still + `EasyvistaAuthError`, a 590 is still not retried) are the same on both paths. + **Only the download direction streams, and that is the API's constraint:** + EasyVista takes an attachment as base64 inside a JSON body, so `add_document` + must materialise the whole payload before it can send anything — no streaming + upload is possible, and the asymmetry is not an oversight here. + **A mid-stream failure is not retried.** Opening the download is retried under + the usual policy, and the first chunk is fetched inside that retried unit so a + failure fetching it is still safe to restart; from that chunk onwards the + request is committed and a transport error raises `EasyvistaConnectionError` + rather than starting over, because starting over would re-deliver bytes the + caller already holds. Nothing resumes a partly consumed stream, so a caller + that must survive a mid-stream failure decides for itself whether to discard + what it collected and ask again; `download_document` retries the whole fetch + and stays the simpler choice for a file small enough to buffer. + - Python 3.13 and 3.14 are now tested and declared supported (classifiers, and the CI/release matrices, now span 3.10--3.14). No code changed: the suite passes unmodified on both, with the same statement count and coverage as on @@ -31,6 +114,26 @@ a deprecation policy will follow the 1.0 release. code snippets against the real public API, so a rename fails CI. - Public `filters.py`: `ev_equals_filter`, `ev_in_filter`, `escape_ev_value`, and `is_safe_ev_value` for building EasyVista `search` expressions safely. +- `ev_since_filter` / `ev_between_filter` — the interval grammar + (`FIELD:(a;b)`) that is the only server-side range filter this API honours. + EasyVista has no comparison operator (`>=`, `BETWEEN`, `[a TO b]`…): either + rendering is silently dropped or, if it keeps `FIELD:"value"` syntax while + embedding the operator in the value, raises HTTP 590 as a type mismatch. + Neither ever narrows a result, which is why these builders exist. +- `ev_contains_filter` / `ev_starts_with_filter` — `~` with an explicit + wildcard (`*` or `%`; both work identically). A bare value under `~` + degenerates to exact match, which these builders avoid by construction. + A value containing any of `*`, `%`, `_` or `[` raises `ValueError`: all four + are metacharacters to `~` (`_` matches any single character, `[` opens a + character class — measured live: replacing one character of an RFC that + matched 1 row with `_`, or with `[0-9]`, matched 9), and no escape for them + exists (`\_` is compared literally). Refusing beats silently matching records + the caller did not ask for, which matters because `_` is pervasive in + EasyVista codes: `ev_contains_filter("ASSET_TAG", "LAPTOP_01")` would + otherwise also match `LAPTOP-01` and `LAPTOP001`, with HTTP 200 and no hint. +- `parse_ev_datetime` / `format_ev_datetime` (new `timestamps.py` module) — + parse an EasyVista timestamp to an aware `datetime` and render one back to + the literal the search grammar and the wire format both accept. - `Request` now declares fields that were previously reachable only as untyped `extra="allow"` data — each verified present on live single-ticket GETs: `title`, `request_id`, `external_reference`, `sd_catalog_id`, `urgency_id`, @@ -39,15 +142,20 @@ a deprecation policy will follow the 1.0 release. and `last_update`. - `RequestUpdate.title` — a ticket's title can now be changed after creation (`PUT /requests/{rfc}`), not only set at create time. +- `RequestUpdate` now also carries `impact_id`, `owner_id` and + `external_reference` (capped at 50 characters — bisected live: 50 is + accepted, 51 is rejected). `severity_id`, a writable priority field, and + `urgency_id` are deliberately still absent; see the `O-590-PARTIAL` note. - `EasyvistaClient.download_document` / `AsyncEasyvistaClient.download_document` fetch an attachment's bytes. An absolute download URL is followed only when its scheme and host match the configured `server`: every request carries the instance's Bearer token, so a URL naming another host is refused rather than followed. - `Request` now declares the official time-limit fields as typed attributes: - `creation_date_ut`, `max_resolution_date_ut`, `expected_date_ut`, - `end_date_ut`, `sla_id` and `time_used_to_solve_request`. As with the existing - timestamps, they are verified *returned* and no datetime parsing is claimed. + `creation_date_ut`, `max_resolution_date_ut`, `expected_date_ut` and + `end_date_ut` (timezone-aware `datetime`, parsed the same way as the other + timestamps below), plus `sla_id` (int) and `time_used_to_solve_request` (a + string on every ticket checked, never an int, so no int branch is declared). The instance-specific `E_GTR_*` / `E_GTI_*` family stays undeclared and reachable through `classify_fields().custom`. - `EasyvistaClient.get_action` / `AsyncEasyvistaClient.get_action` fetch a single @@ -62,43 +170,95 @@ a deprecation policy will follow the 1.0 release. tail is an RFC number rather than an id. A created action's id is therefore not recoverable from its create response at all; diff `list_actions` across the create to identify it (verified live). +- `list_actions(fields=...)` — project timestamps and author onto the list and + read a whole **page** of action metadata in one request instead of one item + fetch per action. Three silent footguns come with it: `"*"` is not a wildcard + (it reduces to `ACTION_ID` alone), a dotted path (`DESCRIPTION.HREF`) is + silently dropped, and `list_actions` returns **one page and does not + paginate** — a ticket with more actions than `config.default_max_rows` is + truncated with no error, and the call discards the envelope's total so the + caller cannot detect it. That is not a corner case: a freshly created ticket + already carries about twelve actions, most of them workflow-generated. The + same cap therefore truncates `get_ticket_context`'s action log and + `TicketContext.to_markdown()`'s rendering of it. `list_actions` now sends + `config.default_max_rows` explicitly, the way every sibling search does, so + the cap is the client's and can be raised; real pagination is a follow-up. +- `Action` now declares its timestamps (`created_at`/`CREATION_DATE_UT`, + `updated_at`/`LAST_UPDATE`), author (`done_by_id`) and workflow context + (`action_type_id`, `group_id`, `request_id`, `action_number`, `stage_id`, + `workflow_id`, `parent_action_id`) — verified live 2026-08-17. Availability + on the LIST endpoint is not uniform across these; pass `fields=` to project + the ones a default list row omits. Note the naming diverges from the two + models already shipped: `Action.created_at`/`updated_at` alias the same wire + columns that `Request` and `Employee` expose as + `creation_date_ut`/`last_update`. The wire aliases are identical on all three, + so code spanning record types should reach the value through + `classify_fields()` / `.reference()` rather than a shared attribute name — + `getattr(record, "last_update")` raises `AttributeError` on an `Action`. +- `update_action` and `delete_document`, with `ActionUpdate`. `PUT + actions/{id}` edits an action's note (verified live by re-reading it + afterwards, not by trusting HTTP 200); an action can be edited but not + deleted (`DELETE actions/{id}` is refused with HTTP 403). `DELETE + requests/{rfc}/documents/{document_id}` removes an attachment — the + top-level `DELETE documents/{id}` returns HTTP 403. - `get_ticket_context(..., resolve_action_bodies=True)` resolves each action's note text. Pass `False` to skip it — it costs two extra requests per action. -### Removed - -- **Breaking:** `PostRequest.catalog_guid` and `Request.catalog_guid` are gone. - `CATALOG_GUID` is absent from every sampled live ticket (0/25 single-ticket - GETs), from the documented create body, and from the vendor field inventory — - it could never populate. `PostRequest(catalog_guid=...)` previously validated - and was sent to the API; it now raises (`extra="forbid"`) instead of being - silently accepted. Use `catalog_code` to name a catalog on create. - -### Fixed - -- `find_departments` and `list_actions` interpolated caller values into a `search` expression - unescaped. Because `,` is an EasyVista combinator, a crafted value could silently widen the - result set (verified live: a department lookup returned 2 records instead of 1). Both now - validate the value. -- `TicketContext.to_markdown` rendered every action with an empty body. It read - the text from `Action.comment`, but `COMMENT` is a distinct field that never - carries it; the note supplied as `PostAction.description` comes back through - the action's `DESCRIPTION` Memo, which is reachable only via an item-level - `GET actions/{id}`. Verified against a live instance. -- **Every mapped exception's message no longer interpolates the raw HTTP response - body.** For a body this client does not recognize (an nginx or WAF HTML page, a - plain-text 503, any unmodelled shape), the message previously ended with that - body's literal text — which then surfaced verbatim wherever the exception was - rendered (`str(exc)`, a traceback, a test runner's failure summary), regardless - of what the body actually contained. The message now reports only the byte - count. **Added:** `EasyvistaError.body` (`bytes | None`) carries the raw - response body, so the content dropped from the message is not lost — it is the - only way left to retrieve an unrecognized body. `.status_code`, `.ev_code` and - `.ev_message` are unaffected: a *recognized* EasyVista error body (one with a - parseable `error`/`error_code` shape) reads exactly as it did before. - ### Changed +- **BREAKING:** Read-model timestamps are now timezone-aware `datetime` + instead of `str`: `Request.submit_date_ut`, `creation_date_ut`, + `max_resolution_date_ut`, `expected_date_ut`, `end_date_ut`, `last_update`, + and `Employee.last_update`. EasyVista returns ISO 8601 with an explicit UTC + offset and millisecond precision (verified live 2026-08-17), so parsing is + no longer left to callers. An unset date (`""` on the wire) is `None`. Write + models are **unchanged** — the accepted write format for a date is still + unverified. Migration: drop your own parsing; to rebuild a search literal + use `format_ev_datetime(value)`, or pass the `datetime` straight to + `ev_since_filter`. One more consequence, easy to miss: a record dump is no + longer directly JSON-serialisable. `model_dump()` and `classify_fields()` + now yield `datetime` objects for these columns, so + `json.dumps(ticket.classify_fields().official)` raises + `TypeError: Object of type datetime is not JSON serializable` where it used to + work — pass `model_dump(mode="json")` on any path that caches, exports or logs + a record as JSON. `classify_fields()` takes **no arguments**, so `mode="json"` + cannot be applied to it: render its values with `format_ev_datetime` before + serialising, or re-key a JSON-mode dump by the bucket's keys + (`dumped = ticket.model_dump(mode="json", by_alias=True)`, then + `{k: dumped[k] for k in ticket.classify_fields().official}`). + **Scope note — the `0.1.0` boundary is ambiguous, read both.** Relative to + the `## [0.1.0] - 2026-07-15` release **commit** (`6df6a75`), only + `Employee.last_update` is a pre-existing field — the six `Request` fields + above were themselves first declared later, during this 0.2.0 cycle + (see `Added`), so under that reading only one field is retyped out + from under a shipped release. But the `0.1.0` **git tag** currently resolves + to a later commit (`3216a33`, 2026-08-04, 117 commits after the release + commit), at which all six `Request` fields and `Employee.last_update` were + already declared as `str | None`. Anyone who installed or pinned against the + `0.1.0` tag therefore sees **all seven** fields change type, not one — check + which commit your `0.1.0` actually resolves to before assuming the narrower + case. +- **Documentation correction:** the `search` operator `~` was documented as + exact-match-only, identical to `:`. Measured live, `~` **is** a pattern + operator — it needs an explicit wildcard (`*` or `%`, both work identically) + to act as one: `~"*260817*"` matched 33 rows and `~"I26081*"` matched 32, + while `:"I26081*"` matched 0, because `:` never expands a wildcard. Without + one, `~` degenerates to exact match, which is exactly what the earlier + tests observed and over-generalised from. Examples implying substring + matching with a bare value (`ASSET_TAG~LAPTOP`) were wrong and have been + replaced with `ev_contains_filter("ASSET_TAG", "LAPTOP")` → + `ASSET_TAG~"*LAPTOP*"`. `*` and `%` are not the only metacharacters either: + under `~`, `_` matches any single character and `[` opens a character class + (both measured live). The unverified `!~` / `!` / `is_null` / + `is_not_null` operators are still not documented as fact. +- **Documentation correction:** the README's and user guide's tutorial examples filtered with + `ev_equals_filter("STATUS_EN", "Open")`. `STATUS_EN` is a sub-key of the nested `STATUS` + object, not a top-level column, so EasyVista silently ignored the condition and every example + returned *all* tickets, not just open ones. This was a documentation defect, not a library bug + — the library does not special-case field names, so nothing in the shipped code was broken. + Replaced with `ev_equals_filter("STATUS_ID", 3)` throughout, and the user guide now documents + which returned fields are actually searchable and the third (HTTP 590 type-mismatch) search + outcome. - `AsyncEasyvistaClient.get_ticket_context` and `get_department_context` now issue their independent sub-requests **concurrently** instead of one after another. The async client previously awaited every call in sequence, so it was no faster than the synchronous one @@ -125,22 +285,12 @@ a deprecation policy will follow the 1.0 release. - **Documentation of observed behaviour, not a code change:** a `description` supplied to `PostRequest` at create time is not readable back through either the `DESCRIPTION` or the `COMMENT` Memo on the verified instance. `RequestUpdate.description` writes the ticket's - `COMMENT` Memo, not `DESCRIPTION` — verified live (0/15 sampled tickets, portal-created - included, have a non-empty `DESCRIPTION`; 15/15 have a non-empty `COMMENT`). Read the body + `COMMENT` Memo, not `DESCRIPTION` — verified live by re-reading the memo after a write, not + by trusting HTTP 200. Nothing is claimed here about how often `DESCRIPTION` is populated on + an instance: an earlier reading of that (`0/15` sampled tickets) is explicitly withdrawn by + the DESCRIPTION-sampling correction under `Fixed` below. Read the body text back with `TicketContext.comment` (or `resolve_memo("requests/{rfc}/comment")` directly), not `Request.description`. Both fields stay as they are; nothing was renamed. -- **Documentation correction:** the `search` operator `~` was documented as "contains". It is - **exact match**, identical to `:` — verified against a live instance. Examples implying - substring matching (`ASSET_TAG~LAPTOP`) were wrong and have been replaced. The unverified - `!~` / `!` / `is_null` / `is_not_null` operators are no longer documented as fact. -- **Documentation correction:** the README's and user guide's tutorial examples filtered with - `ev_equals_filter("STATUS_EN", "Open")`. `STATUS_EN` is a sub-key of the nested `STATUS` - object, not a top-level column, so EasyVista silently ignored the condition and every example - returned *all* tickets, not just open ones. This was a documentation defect, not a library bug - — the library does not special-case field names, so nothing in the shipped code was broken. - Replaced with `ev_equals_filter("STATUS_ID", 3)` throughout, and the user guide now documents - which returned fields are actually searchable and the third (HTTP 590 type-mismatch) search - outcome. - `Request.status_id`, along with the model's other numeric identity/classification fields, now uses an `OptionalInt` type that tolerates the API's `""` for an absent numeric; `status_id` previously raised a validation error on that value. @@ -164,6 +314,117 @@ a deprecation policy will follow the 1.0 release. action bodies, rather than after. The same requests are issued and the result is identical; only their order on the wire changed. +### Removed + +- **Breaking:** `PostRequest.catalog_guid` and `Request.catalog_guid` are gone. + `CATALOG_GUID` is absent from every sampled live ticket (0/25 single-ticket + GETs), from the documented create body, and from the vendor field inventory — + it could never populate. `PostRequest(catalog_guid=...)` previously validated + and was sent to the API; it now raises (`extra="forbid"`) instead of being + silently accepted. Use `catalog_code` to name a catalog on create. + +### Fixed + +- `ev_since_filter` / `ev_between_filter` accepted a **timestamp string with no + UTC offset** and passed it to the wire. EasyVista accepts such a literal and + reads it in a different zone, which moves the bound and **silently skips + records** — measured live 2026-08-18, the same wall-clock text with and without + its offset enumerated 13 rows and 11 rows against one instance. Both builders + now refuse a time that carries no offset (or `Z`), matching the guard + `format_ev_datetime` already applied to a naive `datetime`; a bare date is + still accepted, having no time to misplace. Found by probing, not by review: + the datetime path was guarded and the string path was not, for the identical + hazard. +- `ev_since_filter` / `ev_between_filter` now **normalise** a timestamp string + bound instead of passing it through. The offset gate above made an offset + mandatory, and the obvious way to comply with a stored + `"2026-08-17T20:26:40"` watermark is to append `+02:00` — but measured live + 2026-08-18, `LAST_UPDATE:(2025-11-28T16:14:41+01:00;)` is **HTTP 590**, as are + minute precision, `seconds+00:00` and a space instead of `T` (which is what + `str(aware_datetime)` produces). Only a bare date and + millisecond-precision-with-offset (or `Z`) are honoured. An admitted string + bound is therefore re-rendered through + `format_ev_datetime(parse_ev_datetime(text))`, so the string and datetime + paths now emit byte-identical bounds and both emit a rendering the wire + accepts; a bare date is still passed through unchanged. Lowercase `z` is now + accepted too — `parse_ev_datetime` already accepted it on the read path, so + refusing it here rejected a value this package itself produces. The rendered + bound is validated as well, so a `datetime` in a zone whose UTC offset is not + a whole number of minutes (every pre-1900 `zoneinfo` entry) raises locally + instead of emitting `+05:53:20`. +- **Documented, not changed:** the interval's lower bound is **inclusive** and + milliseconds are honoured (verified live on three independent boundaries), so + a watermark set to `max(t.last_update)` re-reads that boundary record on the + next sweep. And an offset-pagination sweep over a change window must be sorted + **descending** (`sort="LAST_UPDATE DESC"`) and de-duplicated: the rows the + filter selects are by construction the rows that are changing, so a ticket + touched between two pages moves within the set being paged and can slip past + the read cursor. Descending, the row that slips is the re-touched one, whose + stamp is now *above* the watermark, so the next sweep picks it up — the miss is + deferred. Ascending, the row that slips is a neighbour whose stamp did *not* + change, so it falls *below* the watermark and is lost. A caller who cannot + tolerate even a deferred miss must page `search_tickets` with keyset + pagination (advance the window to the last row's stamp instead of an offset), + which `iter_tickets` cannot express. The sweep examples in `ev_since_filter`, + the user guide and the search-syntax skill all carry the sort and the + de-duplication. **Undocumented until now:** because descending yields the + newest row first, the watermark reaches its final value on page 1 of any + given sweep, so a sweep that is interrupted or capped with `max_records` + still ends up holding the newest stamp — advancing the watermark from it + permanently excludes every row the incomplete sweep never read. The four + sites above now say so: advance the watermark only after a sweep runs to + completion, and checkpoint a mid-sweep caller with keyset pagination instead. +- `Request`/`Action`/`Employee` timestamp columns now **raise** on a malformed + value instead of falling through to pydantic's own datetime parser, which is + far more permissive than EasyVista's format and invented plausible-looking + instants: `"20260817"` became `1970-08-23T12:00:17Z` (56 years off) and + `1755434441610` — what an epoch-millis format change would look like — became + a wholly credible `2025-08-17T12:40:41.610Z`. Absorbing a format change is the + opposite of what the guard exists for, and the docstring already promised a + raise. The `""` unset sentinel still becomes `None`, unchanged. +- **Documentation correction:** `RequestUpdate`'s docstring claimed `DESCRIPTION` + is empty on every ticket of the verified instance. It is not. A pooled 77-row + sample across four orderings found `COMMENT` populated on 57 rows, + `DESCRIPTION` on 27 and *both* on 24, with the proportions flipping by slice + (measured 2026-08-18). The earlier 0/15 reading was a sampling artifact drawn + from probe-authored tickets. The load-bearing claim is unchanged and still + verified: `RequestUpdate.description` writes the `COMMENT` memo. What is + withdrawn is the generalisation about `DESCRIPTION` being universally empty — + which also means an instance's body memo cannot be auto-detected by sampling. +- `find_departments` and `list_actions` interpolated caller values into a `search` expression + unescaped. Because `,` is an EasyVista combinator, a crafted value could silently widen the + result set (verified live: a department lookup returned 2 records instead of 1). Both now + validate the value. +- `TicketContext.to_markdown` rendered every action with an empty body. It read + the text from `Action.comment`, but `COMMENT` is a distinct field that never + carries it; the note supplied as `PostAction.description` comes back through + the action's `DESCRIPTION` Memo, which is reachable only via an item-level + `GET actions/{id}`. Verified against a live instance. +- **Every mapped exception's message no longer interpolates the raw HTTP response + body.** For a body this client does not recognize (an nginx or WAF HTML page, a + plain-text 503, any unmodelled shape), the message previously ended with that + body's literal text — which then surfaced verbatim wherever the exception was + rendered (`str(exc)`, a traceback, a test runner's failure summary), regardless + of what the body actually contained. The message now reports only the byte + count. **Added:** `EasyvistaError.body` (`bytes | None`) carries the raw + response body, so the content dropped from the message is not lost — it is the + only way left to retrieve an unrecognized body. `.status_code`, `.ev_code` and + `.ev_message` are unaffected: a *recognized* EasyVista error body (one with a + parseable `error`/`error_code` shape) reads exactly as it did before. +- `RECENT_TICKETS_SORT` used a colon-separated token (`RFC_NUMBER:DESC`) that + EasyVista silently ignores, so `get_department_context(recent_tickets=...)` + returned tickets in the API's default order rather than newest-first. The + descending token must be space-separated (`RFC_NUMBER DESC`) — verified live + 2026-08-17 by `integration_tests/test_live_change_window.py`. Closes O-DIR-1. + +### Notes + +- Open item **O-590-PARTIAL**: `PUT requests/{rfc}` with `URGENCY_ID` returned + HTTP 590 (code 2013) while nevertheless changing the stored value. A rejected + update may therefore have partially applied — re-read before retrying. Needs a + focused live probe (set each id from `GET /urgencies` in turn and re-read) + before `urgency_id` can be added to `RequestUpdate`. + ## [0.1.0] - 2026-07-15 Initial public release. @@ -188,5 +449,6 @@ Initial public release. status/error code, with non-retryable validation errors (HTTP 590, code 2013). - `py.typed` marker — the package ships inline type information. -[Unreleased]: https://github.com/baraline/easyvista_python_client/compare/v0.1.0...HEAD -[0.1.0]: https://github.com/baraline/easyvista_python_client/releases/tag/v0.1.0 +[Unreleased]: https://github.com/baraline/easyvista_python_client/compare/v0.2.0...HEAD +[0.2.0]: https://github.com/baraline/easyvista_python_client/compare/0.1.0...v0.2.0 +[0.1.0]: https://github.com/baraline/easyvista_python_client/releases/tag/0.1.0 diff --git a/README.md b/README.md index cda7682..5442678 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ [![CI](https://github.com/baraline/easyvista_python_client/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/baraline/easyvista_python_client/actions/workflows/ci.yml) [![Coverage](https://codecov.io/gh/baraline/easyvista_python_client/branch/main/graph/badge.svg)](https://codecov.io/gh/baraline/easyvista_python_client) -[![License](https://img.shields.io/github/license/baraline/easyvista_python_client)](LICENSE) +[![License](https://img.shields.io/github/license/baraline/easyvista_python_client)](https://github.com/baraline/easyvista_python_client/blob/main/LICENSE) [![Python](https://img.shields.io/badge/python-3.10%2B-blue)](https://github.com/baraline/easyvista_python_client) [![Docs](https://readthedocs.org/projects/easyvista-python-client/badge/?version=latest)](https://easyvista-python-client.readthedocs.io/en/latest/) @@ -10,7 +10,8 @@ Typed Python client for the EasyVista Service Manager REST API. Sync + async, Pydantic models, Bearer or Basic auth. -While the package is preparing for 1.0, alot of potential breaking change might happen between versions. A deprecation policy will be put in place once 1.0 is out and the package have been stabilized. +While the package is preparing for 1.0, breaking changes may land between +minor versions; a deprecation policy will follow the 1.0 release. ## Documentation @@ -78,6 +79,7 @@ from easyvista_python_client import ( EasyvistaClient, EasyvistaConfig, PostAsset, + ev_contains_filter, ev_equals_filter, ) @@ -86,6 +88,13 @@ with EasyvistaClient(EasyvistaConfig.from_env()) as client: tag_filter = ev_equals_filter("ASSET_TAG", "LAPTOP-001") found = client.search_assets(search=tag_filter, max_rows=50) + # `~` needs an explicit wildcard to mean "contains" -- a bare value is exact + # match, identical to `:`. ev_contains_filter adds the wildcard for you, and + # raises ValueError if the value itself carries one of * % _ [ (all four are + # metacharacters to `~`, with no escape). For an exact match on a tag like + # "LAPTOP_01", use ev_equals_filter: `:` does not expand a wildcard. + partial = client.search_assets(search=ev_contains_filter("ASSET_TAG", "LAPTOP")) + # attach a file to a ticket (uploaded as base64 inside the JSON body) pdf = Path("report.pdf") client.add_document("I240101_0001", filename=pdf.name, content=pdf.read_bytes()) @@ -112,7 +121,7 @@ then call `EasyvistaConfig.from_env()`. `skills/` holds Agent Skills for driving this client from an AI agent — one per domain (client setup, search syntax, tickets, actions, documents, assets, directory, reporting and context). Each is a directory with a `SKILL.md` -following the Agent Skills specification; see [skills/README.md](skills/README.md) +following the Agent Skills specification; see [skills/README.md](https://github.com/baraline/easyvista_python_client/blob/main/skills/README.md) for the index. They are source-tree material: present in the git repository and the source @@ -120,11 +129,11 @@ distribution, absent from the installed wheel. ## Contributing -See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and quality checks. +See [CONTRIBUTING.md](https://github.com/baraline/easyvista_python_client/blob/main/CONTRIBUTING.md) for development setup and quality checks. ## License -MIT — see [LICENSE](LICENSE). +MIT — see [LICENSE](https://github.com/baraline/easyvista_python_client/blob/main/LICENSE). ## Sponsoring diff --git a/docs/api_reference.rst b/docs/api_reference.rst index 1ae3600..5fdf1d7 100644 --- a/docs/api_reference.rst +++ b/docs/api_reference.rst @@ -26,6 +26,8 @@ Models .. autoclass:: easyvista_python_client.models.action.PostAction +.. autoclass:: easyvista_python_client.models.action.ActionUpdate + .. autoclass:: easyvista_python_client.models.asset.Asset .. autoclass:: easyvista_python_client.models.asset.PostAsset @@ -61,17 +63,35 @@ Filters ------- Build ``search`` expressions with these rather than f-strings: EasyVista ignores a filter it cannot -parse and returns every record, and ``,`` combines conditions — so an unescaped value fails silently -or widens the result rather than raising. +parse and returns every record, ``,`` combines conditions so an unescaped value can silently widen +the result, and there is no comparison operator — a range must be expressed as an interval. .. autofunction:: easyvista_python_client.filters.ev_equals_filter .. autofunction:: easyvista_python_client.filters.ev_in_filter +.. autofunction:: easyvista_python_client.filters.ev_contains_filter + +.. autofunction:: easyvista_python_client.filters.ev_starts_with_filter + +.. autofunction:: easyvista_python_client.filters.ev_since_filter + +.. autofunction:: easyvista_python_client.filters.ev_between_filter + .. autofunction:: easyvista_python_client.filters.escape_ev_value .. autofunction:: easyvista_python_client.filters.is_safe_ev_value +Timestamps +---------- + +EasyVista's timestamp format, parsed and rendered in one place — see +:ref:`timestamps` for how the read models use these. + +.. autofunction:: easyvista_python_client.timestamps.parse_ev_datetime + +.. autofunction:: easyvista_python_client.timestamps.format_ev_datetime + References ---------- diff --git a/docs/publishing.rst b/docs/publishing.rst index eb394b3..b128089 100644 --- a/docs/publishing.rst +++ b/docs/publishing.rst @@ -12,15 +12,26 @@ the workflow's OIDC identity. Cutting a release ----------------- -#. Bump the version in **both** places -- ``pyproject.toml`` (``project.version``) and - ``easyvista_python_client.__version__``. The release workflow refuses to build if they - disagree, or if they disagree with the tag. +#. Bump the version in **all four** places, or CI goes red on an otherwise correct + bump: + + * ``pyproject.toml`` (``project.version``); + * ``easyvista_python_client.__version__``; + * the hardcoded literal in ``easyvista_python_client/testing/test_public_api.py`` + (asserted by the unit suite); + * every ``skills/*/SKILL.md`` ``metadata.version`` (asserted by + ``scripts/tests/test_skills_contract.py``). + + The release workflow additionally refuses to build if the first two disagree with + each other or with the tag. #. Move the ``CHANGELOG.md`` ``[Unreleased]`` entries under the new version and update the compare links at the bottom of the file. #. Merge to ``main`` and let CI go green. #. Publish a GitHub release whose tag is the version, ``v``-prefixed -- ``v0.2.0`` for version ``0.2.0``. (The workflow strips a leading ``v`` before comparing, - so an unprefixed tag also passes; the repository's existing tags are prefixed.) + so an unprefixed tag also passes. The only tag that exists today, ``0.1.0``, is + **unprefixed** -- ``v``-prefixing starts at ``v0.2.0``, which is why the + ``CHANGELOG.md`` link for ``0.1.0`` points at the bare tag.) The workflow then runs the test matrix (3.10--3.14) and the quality gates -- Ruff, mypy, the generated-``_sync``-tree check, the hand-written-twin lint and a warnings-as-errors diff --git a/docs/user_guide.rst b/docs/user_guide.rst index 6d3ec8c..3478c84 100644 --- a/docs/user_guide.rst +++ b/docs/user_guide.rst @@ -208,12 +208,20 @@ Assets .. code-block:: python - from easyvista_python_client import PostAsset, ev_equals_filter + from easyvista_python_client import PostAsset, ev_contains_filter, ev_equals_filter asset = client.create_asset(PostAsset(catalog_id=3153, asset_tag="LAPTOP-001")) one = client.get_asset(str(asset.asset_id)) found = client.search_assets(search=ev_equals_filter("ASSET_TAG", "LAPTOP-001"), max_rows=50) + # A bare '~' is exact match, identical to ':' -- substring search needs an + # explicit wildcard, which ev_contains_filter adds for you: ASSET_TAG~"*LAPTOP*" + # The value itself must carry none of * % _ [ -- all four are metacharacters + # to '~', and ev_contains_filter raises ValueError rather than widening the + # match silently. So "LAPTOP_01" raises; use ev_equals_filter for an exact + # match on a tag containing '_'. See "Searching and pagination" below. + laptops = client.search_assets(search=ev_contains_filter("ASSET_TAG", "LAPTOP"), max_rows=50) + Documents --------- @@ -237,6 +245,63 @@ Attach a file to a ticket (uploaded as base64 inside the JSON body) and list a t another host is refused rather than followed. Multipart upload is still not implemented; uploads go as base64 inside the JSON body. +Streaming a large attachment +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:meth:`~easyvista_python_client.EasyvistaClient.stream_document` yields the same bytes in +chunks instead of returning them in one object, so a large attachment never has to exist +in memory whole. It accepts exactly what ``download_document`` accepts and resolves the +URL the same way, including the same refusal of a URL outside the configured instance. + +.. code-block:: python + + from pathlib import Path + + with Path("downloaded.pdf").open("wb") as sink: + for chunk in client.stream_document(attachments[0], chunk_size=1024 * 1024): + sink.write(chunk) + +The name is ``stream_`` rather than ``iter_`` because every ``iter_*`` method on the +client iterates *records*; this one iterates the bytes of a single document. + +The async client streams with ``async for`` — like the ``iter_*`` methods and unlike +every other method on it, ``stream_document`` is not awaited: + +.. code-block:: python + + with Path("downloaded.pdf").open("wb") as sink: + async for chunk in client.stream_document(attachments[0]): + sink.write(chunk) + +.. note:: + + **Stopping early on the async surface needs an explicit close.** If you + ``break`` out of the ``async for`` — sniffing a magic number, hashing the + first block, aborting on a size check — the response stays checked out of the + connection pool until the event loop's async-generator finalizer runs, which + is a garbage-collection cycle away (measured). Use + ``contextlib.aclosing(client.stream_document(doc))``, or call ``aclose()`` + yourself, so the connection is released at the ``break``. The synchronous + surface releases it immediately by refcounting and needs nothing. + +.. note:: + + **Only the download streams.** There is no streaming upload, and it is not an + oversight: EasyVista takes an attachment as base64 inside a JSON body, so + ``add_document`` has to materialise the whole payload before it can send anything. + The asymmetry belongs to the API, not to this client. + +.. warning:: + + **A mid-stream failure is not retried.** Opening the download is retried under the + usual policy, but from the first chunk onwards the request is committed: a transport + failure raises :class:`~easyvista_python_client.EasyvistaConnectionError` rather than + starting over, because starting over would hand you bytes you already have. Nothing + resumes a partly consumed stream, so if you must survive a mid-stream failure, decide + for yourself whether to discard what you collected and stream the document again. + ``download_document`` retries the whole fetch and is the simpler choice when the file + is small enough to buffer. + Exporting a ticket to Markdown ------------------------------ @@ -288,9 +353,29 @@ Searching and pagination The verified search grammar is: -- ``FIELD:"value"`` — exact match. ``~`` is a synonym: despite its appearance it is **exact match**, - not "contains" — identical to ``:``. No substring operator has been identified; ``%`` inside a - value is a literal character, not a wildcard. +- ``FIELD:"value"`` — exact match. +- ``~`` — a **pattern operator**, not a synonym for ``:``. It only acts as one with an *explicit* + wildcard in the value: ``*`` and ``%`` both expand (verified live 2026-08-17 — ``~"I26081*"`` + matched 32 rows, ``~"*260817*"`` matched 33, ``~"*0001"`` matched 432, and ``~"%"`` + reproduced the same count as the ``*`` equivalent, so ``%`` is a wildcard too). Given a + **bare** value with no wildcard, ``~`` degenerates to exact match — identical to ``:`` — which is + why this package once documented it as exact-match-only; that conclusion held only for the + wildcard-free inputs it was tested with. ``:`` never expands a wildcard even when one is present + in the value: ``:"I26081*"`` matched **0** rows on the same data. Build the pattern with + :func:`~easyvista_python_client.ev_contains_filter` (``FIELD~"*value*"``) or + :func:`~easyvista_python_client.ev_starts_with_filter` (``FIELD~"value*"``) rather than by hand. +- ``*`` and ``%`` are **not** the only metacharacters under ``~``. ``_`` matches any **single** + character and ``[`` opens a character class — measured live 2026-08-18: replacing one character + of an RFC that matched 1 row with ``_``, or with ``[0-9]``, matched 9, while ``[x]`` still matched 1. There is **no escape**: ``\_`` matched 0 rows, i.e. the backslash + is compared literally. Both builders above therefore raise ``ValueError`` for a value containing + any of ``* % _ [`` rather than silently matching records you did not ask for. This bites on + ordinary input: ``_`` is pervasive in EasyVista codes, and + ``ev_contains_filter("ASSET_TAG", "LAPTOP_01")`` raises for that reason — unhandled, it would + also have matched ``LAPTOP-01`` and ``LAPTOP001`` with HTTP 200 and no hint. For an **exact** + match on such a value use :func:`~easyvista_python_client.ev_equals_filter`, since ``:`` does not + expand a wildcard; to pattern-match *around* one, filter server-side on a wider condition and + compare exactly in Python. - ``,`` — combines conditions: **OR** when every condition names the same field, **AND** across different fields. ``;`` is *not* a combinator. @@ -305,6 +390,20 @@ The verified search grammar is: ``ev_equals_filter("STATUS_ID", "Open")`` sends a status *name* to an integer column and fails loudly. That is the friendlier failure; the silent ones above are the dangerous ones. + There is **no comparison operator** (``>=``, ``BETWEEN``, ``[a TO b]``…), and writing one has + *two* different fates depending on its exact shape, not one: + + - drop the ``FIELD:`` colon entirely (e.g. ``LAST_UPDATE>="2026-01-01"``) and the expression is + structurally unparseable, so it takes the **silent-drop** path above — the whole table comes + back; + - keep ``FIELD:"value"`` syntax but embed the operator *inside* the quoted value + (``LAST_UPDATE:">=2026-01-01"`` or ``LAST_UPDATE:"[2026-01-01 TO *]"``) and the quoted text must + still parse as the column's type — a date, here — so it instead trips the **type-mismatch** + fate and raises ``EasyvistaValidationError`` (HTTP 590). + + Either way, no comparison operator ever narrows the result — see :ref:`change-window-filtering` + for the interval grammar that does. + Build filters with the helpers, not f-strings: .. code-block:: python @@ -379,6 +478,177 @@ The async client paginates with ``async for``: ): print(ticket.rfc_number) +.. _change-window-filtering: + +Filtering by a change window +----------------------------- + +EasyVista has **no** comparison operator. ``LAST_UPDATE >= x`` in any spelling is +either structurally unparseable (silently dropped, every record comes back) or, +if it keeps ``FIELD:"value"`` syntax while embedding the operator inside the +quoted value, a type mismatch that raises HTTP 590 — see the warning above. A +range is instead an interval in the *value position*: + +.. code-block:: python + + from easyvista_python_client import ev_since_filter + + search = ev_since_filter("LAST_UPDATE", watermark) # LAST_UPDATE:(...;) + if search is not None: + seen = set() + for ticket in client.iter_tickets(search=search, sort="LAST_UPDATE DESC"): + if ticket.rfc_number in seen: + continue + seen.add(ticket.rfc_number) + ... + +``watermark`` may be a :class:`datetime.datetime` (preferred) or a timestamp +string. Pass a ``datetime`` and the bound cannot be malformed; ``Request`` +timestamps are already aware datetimes (see :ref:`timestamps`), so a value read +from one ticket can be fed straight back in. A string naming a time is +re-rendered into the one form the wire honours (millisecond precision with an +offset), so a stored watermark string and the ``datetime`` it came from produce +byte-identical bounds. + +The bound is **inclusive**, and milliseconds are honoured (verified live on +three independent boundaries). A watermark set to ``max(t.last_update)`` +therefore re-reads that boundary record on the next sweep — hence the +de-duplication above. + +.. warning:: + + **Sort the sweep descending, and de-duplicate.** ``iter_tickets`` walks the + result set by *offset*, and the rows a change window selects are by + construction the rows that are changing, so a ticket touched between page N + and page N+1 moves *within the very set being paged*. An unsorted sweep can + drop such a row with no way to tell — and so can either sort direction. What + differs is where the dropped row's own timestamp lands relative to the + watermark this sweep records: + + - **Descending** (``LAST_UPDATE DESC``): the re-touched row jumps to the head, + behind the read cursor, so this sweep misses it — but its ``LAST_UPDATE`` is + now *above* the watermark, so the next sweep selects it again. The miss is + **deferred and self-healing**. + - **Ascending** (bare ``LAST_UPDATE``, or ``LAST_UPDATE ASC``): the re-touched + row moves to the tail and everything behind it shifts one place head-ward, + so the row that crosses the cursor is one whose own stamp did **not** + change. It falls *below* the new watermark and no later sweep selects it. + The miss is **permanent**. + + Both tokens are honoured (measured live); descending is chosen for the reason + above, not for availability. De-duplicate by ``rfc_number``: the duplicates + are the deferred rows arriving on a later sweep, plus the inclusive-boundary + re-read described above. + + **A sweep that does not run to completion is a separate trap.** Descending + yields the newest row first, so the watermark reaches its *final* value on + page 1. A sweep that is interrupted, or capped with ``max_records`` (as the + pagination examples above do), still ends up holding the newest stamp — + advance the watermark from that and the next window's ``(newest;)`` bound + permanently excludes every row the incomplete sweep never read. Only advance + the watermark after a sweep runs to completion. + + Descending is the safe direction, not a guarantee. If even a deferred miss is + unacceptable, page + :meth:`~easyvista_python_client.EasyvistaClient.search_tickets` yourself with + **keyset** pagination: sort ascending and, after each page, advance the + *window* — ``ev_since_filter("LAST_UPDATE", max(stamps on the page))`` read + again at ``offset=0`` — instead of incrementing an offset. With no offset there + is no cursor for a row to shift past. ``iter_tickets`` cannot express this, + because it owns its own offset. + + An earlier release of this guide recommended sorting *ascending* here, on the + reasoning that it turns a permanent miss into a duplicate. That was wrong: the + row an ascending sweep drops is not the re-touched one. + + The sort token must stay space-separated. ``LAST_UPDATE:DESC``, + ``-LAST_UPDATE`` and ``DESC(LAST_UPDATE)`` are each **silently ignored** + (measured live) and degrade to the server's default order with no error, so a + sweep written with one of those forms is an unsorted sweep that looks sorted. + +.. warning:: + + **A bound that names a time must carry its UTC offset.** EasyVista accepts an + offset-less literal and reads it in a different zone: measured live, the same + wall-clock text with and without its offset returned 13 rows and 11 rows + against one instance — the offset-less form moves the bound *later* and skips + records, with no error of any kind. Both builders therefore refuse a naive + time, whether it arrives as a ``datetime`` or as a string. A bare date + (``"2026-01-31"``) stays accepted: day granularity has no time to misplace. + +Use :func:`~easyvista_python_client.ev_between_filter` for a closed interval. +Both refuse a bound that is not a timestamp: the bound is interpolated +*unquoted*, so a ``;`` or ``)`` inside it would silently change the query. + +.. code-block:: python + + from datetime import datetime, timezone + from easyvista_python_client import ev_between_filter + + window = ev_between_filter( + "LAST_UPDATE", + datetime(2026, 1, 1, tzinfo=timezone.utc), + datetime(2026, 2, 1, tzinfo=timezone.utc), + ) + recent = client.search_tickets(search=window, max_rows=100) + +.. _timestamps: + +Timestamps +~~~~~~~~~~ + +``Request``'s timestamp fields (``submit_date_ut``, ``creation_date_ut``, +``max_resolution_date_ut``, ``expected_date_ut``, ``end_date_ut``, +``last_update``) and ``Employee.last_update`` are timezone-aware +:class:`datetime.datetime`, parsed from EasyVista's ISO-8601-with-offset wire +format (``2026-08-17T15:40:41.610+02:00``, millisecond precision — verified +live 2026-08-17). An unset date is ``None``. The ``_UT`` suffix is a naming +convention, **not** a promise of UTC normalization: these columns carry the +same local offset as ``LAST_UPDATE``. + +Only the *read* path is parsed. The accepted *write* format is still +unverified, so no write model accepts a ``datetime`` — set a date-typed field +with a raw request if you need to. That includes ``custom_fields``: a +``datetime`` placed there is not serialisable and fails inside the HTTP layer +with a bare ``TypeError``, so render it yourself first. + +Only the *declared* columns are parsed. An instance-specific date column reached +through ``classify_fields().custom`` or plain ``extra="allow"`` attribute access +is still the raw wire string, so within one record dump +``official["CREATION_DATE_UT"]`` is a ``datetime`` while +``official["EXPECTED_START_DATE_UT"]`` is a ``str`` — comparing the two raises +``TypeError``. Pass the undeclared one through +:func:`~easyvista_python_client.parse_ev_datetime` before comparing them. + +.. note:: + + **A record dump is no longer directly JSON-serialisable.** ``model_dump()`` + and ``classify_fields()`` yield ``datetime`` objects for the columns above, so + both ``json.dumps(ticket.model_dump(by_alias=True))`` and + ``json.dumps(ticket.classify_fields().official)`` raise + ``TypeError: Object of type datetime is not JSON serializable``. For a dump, + pass ``model_dump(mode="json")``. ``classify_fields()`` takes **no arguments**, + so there is nowhere to put that keyword: render the ``datetime`` values with + :func:`~easyvista_python_client.format_ev_datetime` before serialising the + bucket, or classify the JSON-mode dump yourself — the buckets are keyed by + wire column name, so ``{k: dumped[k] for k in ticket.classify_fields().official}`` + over ``dumped = ticket.model_dump(mode="json", by_alias=True)`` gives the same + split with serialisable values. + +Use :func:`~easyvista_python_client.format_ev_datetime` to render a +``datetime`` back into the literal EasyVista's grammar accepts (e.g. as an +interval bound above), and :func:`~easyvista_python_client.parse_ev_datetime` +to parse a raw string yourself. + +.. code-block:: python + + from easyvista_python_client import format_ev_datetime, parse_ev_datetime + + ticket = client.get_ticket(ticket.rfc_number) + watermark = ticket.last_update # already an aware datetime + literal = format_ev_datetime(watermark) # "2026-08-17T15:40:41.610+02:00" + assert parse_ev_datetime(literal) == watermark + Counting and statistics ----------------------- diff --git a/easyvista_python_client/__init__.py b/easyvista_python_client/__init__.py index eb98465..1b58b9e 100644 --- a/easyvista_python_client/__init__.py +++ b/easyvista_python_client/__init__.py @@ -18,11 +18,15 @@ from .field_model import FieldClassification from .filters import ( escape_ev_value, + ev_between_filter, + ev_contains_filter, ev_equals_filter, ev_in_filter, + ev_since_filter, + ev_starts_with_filter, is_safe_ev_value, ) -from .models.action import Action, PostAction +from .models.action import Action, ActionUpdate, PostAction from .models.asset import Asset, PostAsset from .models.department import Department, DepartmentUpdate, PostDepartment from .models.document import Document @@ -31,11 +35,13 @@ from .pagination import SearchResult from .references import Reference from .reporting import TicketStatistics, aggregate_tickets +from .timestamps import format_ev_datetime, parse_ev_datetime -__version__ = "0.1.0" +__version__ = "0.2.0" __all__ = [ "Action", + "ActionUpdate", "Asset", "AsyncEasyvistaClient", "Department", @@ -68,7 +74,13 @@ "__version__", "aggregate_tickets", "escape_ev_value", + "ev_between_filter", + "ev_contains_filter", "ev_equals_filter", "ev_in_filter", + "ev_since_filter", + "ev_starts_with_filter", + "format_ev_datetime", "is_safe_ev_value", + "parse_ev_datetime", ] diff --git a/easyvista_python_client/_async/_transport.py b/easyvista_python_client/_async/_transport.py index 407708c..a15a96a 100644 --- a/easyvista_python_client/_async/_transport.py +++ b/easyvista_python_client/_async/_transport.py @@ -16,6 +16,7 @@ from __future__ import annotations import json +from collections.abc import AsyncGenerator, AsyncIterator from typing import Any, NoReturn from urllib.parse import urlsplit @@ -39,6 +40,17 @@ EasyvistaValidationError, ) +#: Default chunk size, in bytes, for :meth:`Transport.stream_bytes`. +#: +#: 64 KiB is the ceiling this default is chosen to set: a caller streams an +#: attachment precisely so the whole file never sits in memory, and the chunk +#: size is what one step of that costs. Large enough that a 32 MB attachment is +#: ~512 iterations rather than tens of thousands, small enough that the resident +#: peak stays negligible beside the file. Deliberately not a config field -- +#: nobody has asked for an instance-wide value, and the one caller who cares +#: about a specific payload can pass ``chunk_size`` per call. +DEFAULT_STREAM_CHUNK_SIZE = 64 * 1024 + class BaseTransport: """Pure transport logic, independent of how a request is executed (no I/O).""" @@ -59,8 +71,16 @@ def resolve_url(self, path_or_url: str) -> str: That check is load-bearing, not decoration. Every request this transport makes carries the instance's Bearer token, so following an absolute URL taken out of a response body (an attachment's ``DDL_HREF``, say) would - hand that credential to whatever host the body named. The API is trusted - to describe its own instance, not to redirect us off it. + hand that credential to whatever host the body named. + + What is guaranteed is exactly that and no more: a foreign URL in a + response **body** is refused. An HTTP **redirect** off the instance is + still *followed* -- both download paths run with + ``follow_redirects=True``, which signed-location hops depend on -- and it + merely loses the credential (verified: no ``authorization`` header on the + foreign request, for Bearer and for Basic; a same-host redirect keeps + it). So streamed or downloaded bytes are not proof of instance origin, + and a caller must not treat them as such. """ parsed = urlsplit(path_or_url) if not parsed.scheme and not parsed.netloc: @@ -264,3 +284,111 @@ async def get_bytes(self, path_or_url: str) -> bytes: self._raise_for_response(exc.response) except httpx.TransportError as exc: raise EasyvistaConnectionError(f"connection failed: {exc}") from exc + + async def _open_stream( + self, path_or_url: str, chunk_size: int + ) -> tuple[httpx.Response, AsyncIterator[bytes], list[bytes]]: + """Open a streaming GET and take its first chunk, as one retryable unit. + + Returns the still-open response, its chunk iterator, and the first chunk + wrapped in a list -- empty for an empty body, which is how "the body is + over" is distinguished from "there is a chunk" without a sentinel. + + Taking the first chunk *here* rather than in :meth:`stream_bytes` is the + whole point of this helper: everything inside it can be retried safely + because nothing it produces has reached the caller yet, so restarting + the request cannot deliver a byte twice. See :meth:`stream_bytes` for + the policy that rests on it. + + Two details are forced by streaming. The response must be closed on + every failure path, because an unread streaming response holds its + connection open. And :meth:`BaseTransport._raise_for_response` reads + ``.content``, which on a streaming response raises until the body has + actually been read -- hence the read before each raise, which is what + makes the error mapping identical to :meth:`get_bytes`. + """ + response = await self._client.send( + self._client.build_request("GET", self.resolve_url(path_or_url)), + stream=True, + follow_redirects=True, + ) + try: + if self.is_retryable_status(response.status_code): + await response.aread() + raise _RetryableResponse(response) + if not response.is_success: + await response.aread() + self._raise_for_response(response) + chunks = response.aiter_bytes(chunk_size) + first: list[bytes] = [] + async for chunk in chunks: + first.append(chunk) + break + except BaseException: + await response.aclose() + raise + return response, chunks, first + + async def stream_bytes( + self, path_or_url: str, *, chunk_size: int = DEFAULT_STREAM_CHUNK_SIZE + ) -> AsyncGenerator[bytes, None]: + """GET raw bytes (an attachment) in chunks, never as one object. + + The streaming twin of :meth:`get_bytes`, and deliberately identical to + it everywhere it can be: the same URL resolution through + :meth:`BaseTransport.resolve_url` (so a URL outside the configured + instance is refused here too), the same ``follow_redirects=True`` for + the signed-location hop, the same attempt count and backoff, and the + same error mapping -- a 403 on an attachment still raises + :class:`EasyvistaAuthError`, and a 590 is still not retried. What + differs is that the body is handed over in ``chunk_size`` pieces as it + arrives, so a large attachment never has to exist in memory whole. + + **Retrying stops as soon as a byte reaches the caller.** A retryable + status or a transport error while opening the download is retried like + any other request, and the first chunk is fetched inside that retried + unit so that a failure fetching it is still safe to restart. From that + chunk onwards the request is committed: a transport failure raises + :class:`EasyvistaConnectionError` instead of starting over, because + starting over would re-deliver bytes the caller already has. Nothing + resumes a partly consumed stream -- a caller that must survive a + mid-stream failure has to decide for itself whether to discard what it + collected and ask again, and this method will not make that choice by + silently duplicating data. + + No request is made until iteration begins. This is a generator, so a + refused URL -- and a non-positive ``chunk_size`` -- raises on the first + step rather than at the call. + """ + if chunk_size <= 0: + # Guarded here rather than left to httpx, which raises from inside + # its own ByteChunker: `chunk_size=0` surfaces as + # "ValueError: range() arg 3 must not be zero" and a negative one as + # "IndexError: list index out of range" -- both several frames below + # this client, so a caller computing a size (`total // n`, a config + # value that defaulted to 0) reads it as a library bug rather than + # bad input. + raise ValueError(f"chunk_size must be positive, got {chunk_size}") + retryer = AsyncRetrying( + stop=stop_after_attempt(self.config.max_retries + 1), + wait=wait_exponential(multiplier=0.5, max=10), + retry=retry_if_exception_type((_RetryableResponse, httpx.TransportError)), + reraise=True, + ) + opened: tuple[httpx.Response, AsyncIterator[bytes], list[bytes]] + try: + opened = await retryer(self._open_stream, path_or_url, chunk_size) + except _RetryableResponse as exc: + self._raise_for_response(exc.response) + except httpx.TransportError as exc: + raise EasyvistaConnectionError(f"connection failed: {exc}") from exc + response, chunks, first = opened + try: + for chunk in first: + yield chunk + async for chunk in chunks: + yield chunk + except httpx.TransportError as exc: + raise EasyvistaConnectionError(f"connection failed: {exc}") from exc + finally: + await response.aclose() diff --git a/easyvista_python_client/_async/client.py b/easyvista_python_client/_async/client.py index 14db9d6..e701941 100644 --- a/easyvista_python_client/_async/client.py +++ b/easyvista_python_client/_async/client.py @@ -10,11 +10,14 @@ from __future__ import annotations -from collections.abc import AsyncIterator, Sequence +from collections.abc import AsyncIterator, Iterable, Sequence from datetime import datetime from easyvista_python_client._async._concurrency import Semaphore, settle -from easyvista_python_client._async._transport import Transport +from easyvista_python_client._async._transport import ( + DEFAULT_STREAM_CHUNK_SIZE, + Transport, +) from easyvista_python_client._transport import RequestSpec from easyvista_python_client.config import EasyvistaConfig from easyvista_python_client.context import TicketContext @@ -27,7 +30,7 @@ from easyvista_python_client.exceptions import EasyvistaAuthError, EasyvistaNotFound from easyvista_python_client.field_model import parse_memo from easyvista_python_client.filters import ev_equals_filter, is_safe_ev_value -from easyvista_python_client.models.action import Action, PostAction +from easyvista_python_client.models.action import Action, ActionUpdate, PostAction from easyvista_python_client.models.asset import Asset, PostAsset from easyvista_python_client.models.department import ( Department, @@ -141,6 +144,21 @@ async def iter_tickets( Pages of ``page_size`` (default ``config.default_max_rows``) until the server reports no further page (``@next``) or ``max_records`` is reached. + + ``sort`` is forwarded to the wire and its token must be + **space-separated** — ``"LAST_UPDATE"`` or ``"LAST_UPDATE DESC"``. + ``"LAST_UPDATE:DESC"``, ``"-LAST_UPDATE"`` and ``"DESC(LAST_UPDATE)"`` + are each **silently ignored** (measured live): the server returns its + default order with no error, so an unsorted result looks sorted. This is + not validated locally, so the token is the caller's to get right. + + Sorting is not cosmetic when the filter selects rows that are changing: + an unsorted offset sweep over a change window can skip a record + permanently, and the two sort DIRECTIONS do not fail the same way -- + descending defers such a miss to the next sweep, ascending loses it. See + :func:`~easyvista_python_client.ev_since_filter`, which rules on the + direction and names the keyset alternative for a caller who cannot + tolerate even a deferred miss. """ if page_size is None: page_size = self.config.default_max_rows @@ -214,9 +232,40 @@ async def ticket_statistics( ) async def update_ticket(self, rfc_number: str, update: RequestUpdate) -> Request: + """Update a ticket's writable fields. + + Cannot set a status: there is no flat status update on this API. See + :meth:`set_status`, and :class:`RequestUpdate` for the measurements. + """ spec, parse = requests_res.build_update_ticket(rfc_number, update) return parse(await self._transport.send(spec)) + async def set_status( + self, rfc_number: str, *, status_guid: str, comment: str | None = None + ) -> Request: + """Set a ticket's status, addressed by ``STATUS_GUID``. + + This is the API's only working status write, and it reaches **every** + status rather than only terminal ones: given six different status GUIDs + in turn, a fresh ticket landed on exactly the status requested every + time, non-terminal ones included. + + It sends the documented ``{"closed": {"status_GUID": ...}}`` body -- the + same request :meth:`close_ticket` sends, under a name that matches what + it does, because "close" is what the wire calls it and not what it is + limited to. + + Note the addressing. A ``STATUS_GUID`` is not a ``STATUS_ID``; the two + are different columns, and only the GUID works here. Read a status's GUID + off any ticket in that status (the nested ``STATUS`` object carries + ``STATUS_GUID``) -- they are stable per instance but are **not** + portable between instances. + """ + spec, parse = requests_res.build_set_status( + rfc_number, status_guid=status_guid, comment=comment + ) + return parse(await self._transport.send(spec)) + async def close_ticket( self, rfc_number: str, @@ -246,8 +295,43 @@ async def create_action(self, rfc_number: str, action: PostAction) -> Action: spec, parse = actions_res.build_create_action(rfc_number, action) return parse(await self._transport.send(spec)) - async def list_actions(self, rfc_number: str) -> list[Action]: - spec, parse = actions_res.build_list_actions(rfc_number) + async def list_actions( + self, rfc_number: str, *, fields: Iterable[str] | str | None = None + ) -> list[Action]: + """List a ticket's actions. + + The default projection is slim: it carries ``ACTION_ID``, + ``ACTION_LABEL_FR``, ``ACTION_NUMBER``, ``DONE_BY_ID`` and + ``EXPECTED_START_DATE_UT`` but **no** ``CREATION_DATE_UT`` or + ``LAST_UPDATE``. Pass ``fields`` to project them onto the list and read + a page of actions' timestamps and authors in one request rather than one + item fetch each:: + + actions = client.list_actions( + rfc, + fields=["ACTION_ID", "ACTION_TYPE_ID", "CREATION_DATE_UT", + "LAST_UPDATE", "DONE_BY_ID"], + ) + + **Returns at most ONE page and does not paginate.** The cap is + ``config.default_max_rows``; a ticket with more actions than that is + **truncated with no error**, and this method discards the envelope's + total, so a caller cannot detect the truncation from the result. This is + not hypothetical: a freshly created ticket already carries about twelve + actions, most of them workflow-generated. Raise + ``EasyvistaConfig.default_max_rows`` if a ticket's whole log matters. + + The note text is never projectable — ``DESCRIPTION`` and ``COMMENT`` + are Memo sub-resources and come back as HREF objects under every + projection, so a body still costs one :meth:`resolve_memo` per action. + + ``fields`` has two more silent footguns: ``"*"`` is not a wildcard — + it silently reduces to ``ACTION_ID`` alone — and a dotted path such as + ``DESCRIPTION.HREF`` is silently dropped. + """ + spec, parse = actions_res.build_list_actions( + rfc_number, fields=fields, max_rows=self.config.default_max_rows + ) return parse(await self._transport.send(spec)) async def get_action(self, action_id: str | int) -> Action: @@ -259,6 +343,23 @@ async def get_action(self, action_id: str | int) -> Action: spec, parse = actions_res.build_get_action(action_id) return parse(await self._transport.send(spec)) + async def update_action(self, action_id: str | int, update: ActionUpdate) -> Action: + """Edit an existing action's note text. + + Live-verified 2026-08-17 by re-reading the memo afterwards, not by the + status code. Note that an action can be edited but **not deleted** — + ``DELETE actions/{id}`` is refused with HTTP 403 — so there is + deliberately no ``delete_action``. + + The returned :class:`Action` is the API's own echo and is **not + verified**: the PUT's response body has never been captured, and if it + answers empty or href-only the parser yields an ``Action`` whose fields + are all ``None``. Re-read with :meth:`get_action` rather than reading + fields off the return value. + """ + spec, parse = actions_res.build_update_action(action_id, update) + return parse(await self._transport.send(spec)) + async def _resolve_action_body(self, action: Action) -> Action: """Return ``action`` with its note text resolved onto ``description``. @@ -348,6 +449,17 @@ async def list_documents(self, rfc_number: str) -> list[Document]: spec, parse = documents_res.build_list_documents(rfc_number) return parse(await self._transport.send(spec)) + async def delete_document(self, rfc_number: str, document_id: str) -> None: + """Remove an attachment from a ticket. + + ``document_id`` is the ``DOCUMENT_ID`` from :meth:`list_documents`. + Live-verified 2026-08-17 by re-listing the ticket's documents + afterwards. Returns nothing: the API answers with an empty body. + """ + await self._transport.send( + documents_res.build_delete_document(rfc_number, document_id) + ) + async def download_document(self, document: Document | str) -> bytes: """Fetch an attachment's bytes. @@ -359,6 +471,74 @@ async def download_document(self, document: Document | str) -> bytes: """ return await self._transport.get_bytes(documents_res.download_href(document)) + async def stream_document( + self, document: Document | str, *, chunk_size: int = DEFAULT_STREAM_CHUNK_SIZE + ) -> AsyncIterator[bytes]: + """Fetch an attachment's bytes in chunks, without holding the file whole. + + Accepts exactly what :meth:`download_document` accepts -- a + :class:`Document` from :meth:`list_documents` or a raw href/path -- and + resolves it identically, refusing a URL outside the configured instance + for the same reason. Use this when the bytes are on their way somewhere + else in pieces (a file on disk, a hash, another API) and + :meth:`download_document` when a single ``bytes`` object is what you + wanted anyway. + + Called ``stream_`` rather than ``iter_`` on purpose: every ``iter_*`` + method on this client iterates *records*, and this iterates the bytes of + one document. + + The opposite direction cannot stream at all. :meth:`add_document` sends + base64 inside a JSON body, so an upload has to materialise the whole + payload however it is called; the asymmetry is the API's, not an + oversight here. + + Retrying covers opening the download only. Once the first chunk has been + handed over the request is committed, and a transport failure raises + :class:`~easyvista_python_client.exceptions.EasyvistaConnectionError` + rather than starting again -- starting again would re-deliver bytes the + caller already has. A partly consumed stream is never resumed, so + deciding what to do with a mid-stream failure is the caller's. See + :meth:`~easyvista_python_client._async._transport.Transport.stream_bytes`. + + Nothing is requested until iteration begins: this is a generator, so + :class:`ValueError` for a record carrying no download URL and + :class:`EasyvistaError` for one pointing off the instance both surface + on the first step rather than at the call. + + **Stopping early:** on the async surface a bare ``break`` leaves the + response checked out of the connection pool until the event loop's + async-generator finalizer runs, which is a garbage-collection cycle away + (measured) -- so a caller that reads only a prefix of many attachments + under a bounded ``max_connections`` can stall on connections it appears + to have released. Close the generator instead (``aclose()``, or + ``contextlib.aclosing``). On the sync surface refcounting releases it at + the ``break`` and nothing is needed. + """ + stream = self._transport.stream_bytes( + documents_res.download_href(document), chunk_size=chunk_size + ) + # Close the inner generator in a `finally` rather than leaving it to be + # collected. `stream`'s own `finally` is what releases the response and + # returns its connection to the pool, and unwinding *this* generator + # does not reach it on its own -- the loop below simply exits. + # + # What this buys, stated precisely: one deferral instead of two. Closing + # this generator -- explicitly, or by an exception propagating out of it + # -- now releases the response at once. A bare `break` still defers on + # the async surface, because unwinding this generator is itself left to + # the event loop's async-generator finalizer; what the `finally` removes + # is the *second* wait, for `stream` to become garbage in its own right. + # On the sync surface refcounting closes it promptly either way. + # `contextlib.closing`/`aclosing` would say this in one line, but those + # two names differ by more than a token so the codegen cannot generate + # the pair; `stream.close()`/`stream.aclose()` it can. + try: + async for chunk in stream: + yield chunk + finally: + await stream.aclose() + # --- departments ---------------------------------------------------------- async def get_department(self, department_id: str | int) -> Department: spec, parse = departments_res.build_get_department(department_id) @@ -571,6 +751,13 @@ async def get_ticket_context( It costs two extra requests per action; pass ``False`` to skip it when you only need the action list. + **The action log is capped at one page.** It comes from + :meth:`list_actions`, which returns at most ``config.default_max_rows`` + actions and does not paginate, so on a busy ticket + :attr:`TicketContext.actions` — and therefore + :meth:`TicketContext.to_markdown`'s rendered log — is silently truncated + with no error. Raise ``default_max_rows`` if completeness matters. + On the async surface the independent requests (the two memos plus the actions and documents lists) are issued concurrently, in up to three waves; on the sync surface they run one after another in source order, @@ -672,9 +859,14 @@ async def get_department_context( 403/404 degrades it to ``[]`` / ``None`` / ``0`` (same pattern as :meth:`get_ticket_context`). The flags trim the heavier related calls. Tickets and assets filter on ``DEPARTMENT_ID:""``. ``recent_tickets`` - ordering is best-effort: it relies on the server honoring - ``RECENT_TICKETS_SORT`` (open item O-DIR-1) and silently degrades to the - API's default order otherwise. + is ordered by **descending ``RFC_NUMBER``** (``RECENT_TICKETS_SORT``), + which is newest-first only where RFC numbers are issued monotonically: + it is a varchar, so the sort orders by the request-type prefix letter + before the date. The token must stay + space-separated: a colon form is silently ignored and degrades to the + API's default order with no error (measured live 2026-08-17). Ordering + therefore depends on the server honouring that token, which the live + suite asserts rather than this method. On the async surface the seven independent branches are issued concurrently, costing two waves instead of eight serial steps; on the diff --git a/easyvista_python_client/_async/tests/test_client.py b/easyvista_python_client/_async/tests/test_client.py index 5615403..5071415 100644 --- a/easyvista_python_client/_async/tests/test_client.py +++ b/easyvista_python_client/_async/tests/test_client.py @@ -11,6 +11,7 @@ """ import json +from collections.abc import AsyncIterator import httpx import pytest @@ -19,8 +20,8 @@ from easyvista_python_client._async import client as client_module from easyvista_python_client._async.client import AsyncEasyvistaClient from easyvista_python_client.directory import DepartmentContext -from easyvista_python_client.exceptions import EasyvistaError -from easyvista_python_client.models.action import PostAction +from easyvista_python_client.exceptions import EasyvistaAuthError, EasyvistaError +from easyvista_python_client.models.action import ActionUpdate, PostAction from easyvista_python_client.models.asset import PostAsset from easyvista_python_client.models.department import ( Department, @@ -110,7 +111,7 @@ async def test_update_and_close_ticket(config): return_value=httpx.Response(200, json={"records": [{"RFC_NUMBER": "I1"}]}) ) async with AsyncEasyvistaClient(config) as client: - await client.update_ticket("I1", RequestUpdate(status_id=4)) + await client.update_ticket("I1", RequestUpdate(impact_id=4)) await client.close_ticket("I1", comment="resolved") @@ -142,6 +143,38 @@ async def test_create_and_list_actions(config): assert listed[0].action_id == 5 +@respx.mock +async def test_list_actions_forwards_a_fields_projection(config): + """The client must pass fields= through, not just accept it. + + EV-R3: fields= is what turns comment metadata into one request per ticket + instead of one request per action. + """ + route = respx.get(f"{ROOT}/actions").mock( + return_value=httpx.Response(200, json={"records": []}) + ) + async with AsyncEasyvistaClient(config) as client: + await client.list_actions("I240101_0001", fields=["ACTION_ID", "LAST_UPDATE"]) + assert route.calls.last.request.url.params["fields"] == "ACTION_ID,LAST_UPDATE" + + +@respx.mock +async def test_list_actions_sends_the_configured_row_cap(config): + """``list_actions`` returns one page, so the cap must be the client's own. + + Without this the request carried no ``max_rows`` at all and the truncation + point was the server's unstated default -- invisible to the caller and not + raisable by configuration. + """ + route = respx.get(f"{ROOT}/actions").mock( + return_value=httpx.Response(200, json={"records": []}) + ) + async with AsyncEasyvistaClient(config) as client: + expected = str(client.config.default_max_rows) + await client.list_actions("I240101_0001") + assert route.calls.last.request.url.params["max_rows"] == expected + + @respx.mock async def test_get_action_fetches_the_item_level_record(config): respx.get(f"{ROOT}/actions/52990").mock( @@ -159,6 +192,18 @@ async def test_get_action_fetches_the_item_level_record(config): assert action.description == {"HREF": f"{ROOT}/actions/52990/description"} +@respx.mock +async def test_update_action_sends_a_put_to_the_top_level_path(config): + """The nested requests/{rfc}/actions/{id} form returns 403 (verified live).""" + route = respx.put(f"{ROOT}/actions/57483").mock( + return_value=httpx.Response(200, json={"ACTION_ID": 57483}) + ) + async with AsyncEasyvistaClient(config) as client: + action = await client.update_action(57483, ActionUpdate(description="edited")) + assert json.loads(route.calls.last.request.content) == {"description": "edited"} + assert action.action_id == 57483 + + @respx.mock async def test_from_env_constructs_working_client(monkeypatch): monkeypatch.setenv("EASYVISTA_URL", "https://ev.test") @@ -257,6 +302,18 @@ async def test_add_and_list_documents(config): assert body["documents"][0]["filename"] == "a.txt" +@respx.mock +async def test_delete_document_sends_a_delete_to_the_nested_path(config): + """Returns None: the API answers a delete with an empty body.""" + route = respx.delete(f"{ROOT}/requests/I240101_0001/documents/12345_abcdef").mock( + return_value=httpx.Response(200) + ) + async with AsyncEasyvistaClient(config) as client: + result = await client.delete_document("I240101_0001", "12345_abcdef") + assert route.call_count == 1 + assert result is None + + @respx.mock async def test_download_document_fetches_the_ddl_href(config): route = respx.get("https://ev.test/dl/7").mock( @@ -287,6 +344,130 @@ async def test_download_document_refuses_a_foreign_download_url(config): await client.download_document(doc) +@respx.mock +async def test_stream_document_chunks_reassemble_to_the_download(config): + # 3076 bytes at chunk_size=512: six full chunks and a 4-byte tail. Sized off + # the boundary on purpose -- an exact multiple never exercises a short final + # chunk, and reassembly passes either way. + body = bytes(range(256)) * 12 + b"tail" + respx.get("https://ev.test/dl/7").mock( + return_value=httpx.Response(200, content=body) + ) + doc = Document.model_validate( + {"DOCUMENT": "big.bin", "DDL_HREF": "https://ev.test/dl/7"} + ) + chunks = [] + async with AsyncEasyvistaClient(config) as client: + async for chunk in client.stream_document(doc, chunk_size=512): + chunks.append(chunk) + assert b"".join(chunks) == body + assert len(chunks) == 7, "the body arrived in one piece instead of streaming" + assert len(chunks[-1]) == 4, "the short final chunk was padded or dropped" + + +class _ClosableStream(httpx.AsyncByteStream): + """A response body that records when the transport closed it. + + ``closed`` flips in ``aclose()``/``close()``, which httpx calls when the + response is + released -- which is what ``stream_bytes`` does in its own ``finally``. So + the flag answers "has the connection gone back to the pool yet". + """ + + def __init__(self, body: bytes) -> None: + self._body = body + self.closed = False + + async def __aiter__(self) -> AsyncIterator[bytes]: + yield self._body + + async def aclose(self) -> None: + self.closed = True + + +@respx.mock +async def test_stream_document_closes_the_inner_stream_when_stopped_early(config): + """Stopping early releases the connection there and then, not eventually. + + ``stream_document`` hands out chunks from an inner ``stream_bytes`` + generator, and only *that* generator's ``finally`` closes the response and + returns its connection to the httpx pool. Closing the outer generator + unwinds its loop with ``GeneratorExit`` and does not close the inner one, so + without the explicit close in ``stream_document`` the release waits for the + inner generator to be collected -- and this asserts with no ``gc`` round and + no scheduling hop in between. Until the release happens the connection stays + checked out, which a caller with a small ``max_connections`` feels. + + Closing explicitly is also the only moment both client trees share: a caller + that just stops iterating leaves the outer generator to be collected, and + when that happens is a property of the runtime, not of this method. + """ + stream = _ClosableStream(b"0123456789abcdef") + respx.get("https://ev.test/dl/7").mock( + return_value=httpx.Response(200, stream=stream) + ) + doc = Document.model_validate({"DDL_HREF": "https://ev.test/dl/7"}) + async with AsyncEasyvistaClient(config) as client: + chunks = client.stream_document(doc, chunk_size=8) + assert await chunks.__anext__() == b"01234567" + assert not stream.closed, "closed while the caller was still reading" + await chunks.aclose() + assert stream.closed, "the abandoned stream still holds its connection" + + +@respx.mock +async def test_stream_document_accepts_a_relative_path_like_download_document(config): + """Same accepted inputs as download_document, resolved the same way.""" + respx.get(f"{ROOT}/documents/7/content").mock( + return_value=httpx.Response(200, content=b"bytes") + ) + path = "documents/7/content" + async with AsyncEasyvistaClient(config) as client: + streamed = [chunk async for chunk in client.stream_document(path)] + downloaded = await client.download_document(path) + assert b"".join(streamed) == downloaded == b"bytes" + + +@respx.mock +async def test_stream_document_and_download_document_agree_on_a_403(config): + """One error mapping, not two: the streaming path must not soften a failure. + + Asserted as an equality between the paths rather than against a hardcoded + type, so the two cannot drift apart without this failing -- which is the + actual risk, since a streaming response needs its body read before the + mapping can look at it at all. + """ + respx.get("https://ev.test/dl/7").mock( + return_value=httpx.Response(403, json={"error": "forbidden"}) + ) + doc = Document.model_validate({"DDL_HREF": "https://ev.test/dl/7"}) + async with AsyncEasyvistaClient(config) as client: + with pytest.raises(EasyvistaError) as streamed: + [chunk async for chunk in client.stream_document(doc)] + with pytest.raises(EasyvistaError) as downloaded: + await client.download_document(doc) + assert type(streamed.value) is type(downloaded.value) is EasyvistaAuthError + assert streamed.value.status_code == downloaded.value.status_code == 403 + assert streamed.value.ev_message == downloaded.value.ev_message == "forbidden" + + +async def test_stream_document_refuses_a_foreign_download_url(config): + # The same-origin guard covers the streaming path too: it is a property of + # the download, not of one method. Nothing is requested until iteration + # begins, so the refusal lands on the first step. + doc = Document.model_validate({"DDL_HREF": "https://attacker.test/dl/7"}) + async with AsyncEasyvistaClient(config) as client: + with pytest.raises(EasyvistaError, match="outside the configured instance"): + [chunk async for chunk in client.stream_document(doc)] + + +async def test_stream_document_needs_a_download_url(config): + doc = Document.model_validate({"DOCUMENT": "orphan.txt"}) + async with AsyncEasyvistaClient(config) as client: + with pytest.raises(ValueError, match="no download URL"): + [chunk async for chunk in client.stream_document(doc)] + + # --- pagination -------------------------------------------------------------- diff --git a/easyvista_python_client/_async/tests/test_transport.py b/easyvista_python_client/_async/tests/test_transport.py index 9ca4204..3af68ac 100644 --- a/easyvista_python_client/_async/tests/test_transport.py +++ b/easyvista_python_client/_async/tests/test_transport.py @@ -7,6 +7,8 @@ only one surface. """ +from collections.abc import AsyncIterator + import httpx import pytest import respx @@ -434,3 +436,293 @@ async def test_get_bytes_keeps_the_bearer_token_on_a_same_host_redirect(): content = await transport.get_bytes("https://ev.test/download/42") assert content == b"blob" assert signed.calls.last.request.headers["Authorization"] == "Bearer tok" + + +# --- stream_bytes ------------------------------------------------------------ +# +# The streaming download. Its contract is "identical to get_bytes except that +# the body arrives in pieces", so most of these are the get_bytes assertions +# above re-made against the chunked path -- that duplication is the point, since +# the two implementations share no code past `resolve_url`. The one claim with +# no get_bytes counterpart is the retry boundary: retrying stops once a byte has +# reached the caller, because restarting would deliver it twice. + + +class _StreamThatFailsMidBody(httpx.AsyncByteStream): + """A response body that delivers ``prefix`` and then drops the connection. + + ``respx`` can fail a request before a response exists, which is what the + ``side_effect=httpx.ConnectError`` mocks elsewhere in this module do. It has + no way to fail one *after* the status line, and that is exactly the case the + retry boundary is about -- so the failure is injected into the body stream + itself, which httpx surfaces as a real ``TransportError`` while iterating. + """ + + def __init__(self, prefix: bytes) -> None: + self._prefix = prefix + + async def __aiter__(self) -> AsyncIterator[bytes]: + yield self._prefix + raise httpx.ReadError("connection dropped mid-body") + + +class _StreamThatFailsBeforeTheFirstByte(httpx.AsyncByteStream): + """A response body that drops the connection without yielding anything. + + The sibling of :class:`_StreamThatFailsMidBody`, for the *other* side of the + retry boundary: the status line arrived, so this is past the point respx can + fail a request, but no byte has reached the caller yet, so restarting is + still safe and must happen. + """ + + async def __aiter__(self) -> AsyncIterator[bytes]: + raise httpx.ReadError("dropped before the first byte") + yield b"" # unreachable; the yield is what makes this a generator function + + +async def _collect(chunks: AsyncIterator[bytes]) -> list[bytes]: + """Every chunk a stream yields, kept separate rather than joined.""" + return [chunk async for chunk in chunks] + + +@respx.mock +async def test_stream_bytes_reassembles_to_the_whole_body(): + # 10244 bytes at chunk_size=1024: deliberately NOT a multiple of it, so the + # last chunk is a short one. A body sized to an exact multiple never + # exercises the ragged tail, and the reassembly assertion below would + # pass either way. + body = bytes(range(256)) * 40 + b"tail" + respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(200, content=body) + ) + async with Transport(_cfg()) as transport: + chunks = await _collect( + transport.stream_bytes("documents/1/content", chunk_size=1024) + ) + assert b"".join(chunks) == body + # More than one chunk, and each bounded: proves the body is delivered + # progressively rather than read whole and handed over in a single piece. + assert len(chunks) == 11 + assert max(len(chunk) for chunk in chunks) <= 1024 + assert len(chunks[-1]) == 4, "the short final chunk was padded or dropped" + + +@respx.mock +async def test_stream_bytes_chunks_at_the_documented_default_size(): + """The default chunk size is 64 KiB, and this is what says so. + + ``DEFAULT_STREAM_CHUNK_SIZE`` is quoted as "64 KiB" in the CHANGELOG, in the + document-workflow skill and in the constant's own comment (whose "a 32 MB + attachment is ~512 iterations" arithmetic only holds at that value). Every + other chunk-counting test passes ``chunk_size`` explicitly, so without this + one the constant could change to anything and leave all three false with a + green suite. 160 KiB of body -> three chunks, the last a short one. + """ + body = bytes(range(256)) * 640 # 163840 bytes == 2.5 * 64 KiB + respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(200, content=body) + ) + async with Transport(_cfg()) as transport: + chunks = await _collect(transport.stream_bytes("documents/1/content")) + assert b"".join(chunks) == body + assert [len(chunk) for chunk in chunks] == [65536, 65536, 32768] + + +@pytest.mark.parametrize("bad", [0, -8]) +async def test_stream_bytes_refuses_a_non_positive_chunk_size(bad): + """The newest public parameter must fail as bad input, not as a library bug. + + Left unguarded, httpx raises from inside its own ``ByteChunker`` several + frames below this client: ``chunk_size=0`` surfaces as "range() arg 3 must + not be zero" and a negative one as "list index out of range". A caller + computing a chunk size reads either as our bug. No request is made, so this + needs no mock -- and because ``stream_bytes`` is a generator, the raise lands + on the first iteration step, matching the deferred ``ValueError`` for a + record with no download URL. + """ + async with Transport(_cfg()) as transport: + with pytest.raises(ValueError, match="chunk_size must be positive"): + await _collect( + transport.stream_bytes("documents/1/content", chunk_size=bad) + ) + + +@respx.mock +async def test_stream_bytes_yields_nothing_for_an_empty_body(): + respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(200, content=b"") + ) + async with Transport(_cfg()) as transport: + assert await _collect(transport.stream_bytes("documents/1/content")) == [] + + +@respx.mock +async def test_stream_bytes_sends_the_bearer_header(): + route = respx.get("https://ev.test/download/42").mock( + return_value=httpx.Response(200, content=b"ok") + ) + async with Transport(_cfg()) as transport: + await _collect(transport.stream_bytes("https://ev.test/download/42")) + assert route.calls.last.request.headers["Authorization"] == "Bearer tok" + + +@respx.mock +async def test_stream_bytes_maps_403_to_auth_error(): + # The status is on an unread streaming response, whose `.content` raises + # until the body is read -- so the error mapping cannot simply be reused, it + # has to read the body first. This asserts the mapped type AND that the + # parsed EasyVista fields survived that detour. + respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(403, json={"error": "forbidden", "code": "9"}) + ) + async with Transport(_cfg()) as transport: + with pytest.raises(EasyvistaAuthError) as ei: + await _collect(transport.stream_bytes("documents/1/content")) + assert ei.value.status_code == 403 + assert ei.value.ev_message == "forbidden" + assert ei.value.ev_code == "9" + + +@respx.mock +async def test_stream_bytes_does_not_retry_a_590(): + route = respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(590, json={"error": "rejected"}) + ) + async with Transport(_cfg(max_retries=3)) as transport: + with pytest.raises(EasyvistaValidationError): + await _collect(transport.stream_bytes("documents/1/content")) + assert route.call_count == 1 + + +@respx.mock +async def test_stream_bytes_retries_a_retryable_status_on_the_open(): + route = respx.get(f"{ROOT}/documents/1/content").mock( + side_effect=[httpx.Response(503), httpx.Response(200, content=b"bytes")] + ) + async with Transport(_cfg(max_retries=2)) as transport: + chunks = await _collect(transport.stream_bytes("documents/1/content")) + assert b"".join(chunks) == b"bytes" + assert route.call_count == 2 + + +@respx.mock +async def test_stream_bytes_exhausts_retries_raises_server_error(): + route = respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(503) + ) + async with Transport(_cfg(max_retries=1)) as transport: + with pytest.raises(EasyvistaServerError): + await _collect(transport.stream_bytes("documents/1/content")) + assert route.call_count == 2 + + +@respx.mock +async def test_stream_bytes_transport_error_on_the_open_raises_connection_error(): + respx.get(f"{ROOT}/documents/1/content").mock( + side_effect=httpx.ConnectError("boom") + ) + async with Transport(_cfg()) as transport: + with pytest.raises(EasyvistaConnectionError): + await _collect(transport.stream_bytes("documents/1/content")) + + +@respx.mock +async def test_stream_bytes_retries_a_failure_fetching_the_first_chunk(): + """The first chunk is fetched INSIDE the retried unit. This is what pins it. + + ``_open_stream`` takes the first chunk itself, so a body that dies before + yielding a byte is still a safe restart -- nothing has reached the caller, so + replaying the request cannot deliver anything twice. Move that fetch out of + the retried unit (open there, iterate the whole body here) and the failure + below escapes as ``EasyvistaConnectionError`` on the first attempt instead, + with ``call_count == 1``. Every other ``stream_bytes`` test passes under both + arrangements, including the mid-body one just after this: the two differ only + on a first-chunk failure, which is only this test. + """ + route = respx.get(f"{ROOT}/documents/1/content").mock( + side_effect=[ + httpx.Response(200, stream=_StreamThatFailsBeforeTheFirstByte()), + httpx.Response(200, content=b"0123456789abcdef"), + ] + ) + async with Transport(_cfg(max_retries=2)) as transport: + chunks = await _collect( + transport.stream_bytes("documents/1/content", chunk_size=8) + ) + assert b"".join(chunks) == b"0123456789abcdef" + assert route.call_count == 2, "a pre-first-byte failure was not retried" + + +@respx.mock +async def test_stream_bytes_does_not_retry_after_a_chunk_has_been_yielded(): + """A mid-body failure is the caller's to handle, never silently restarted. + + Retrying here would hand the caller the opening bytes a second time, so the + request is committed the moment a chunk is delivered. The delivered prefix + stays visible -- the caller keeps what it already collected -- and the + failure arrives as a mapped ``EasyvistaConnectionError``, not as a raw httpx + error. A "helpful" change making this resumable fails on the call count. + """ + route = respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response( + 200, stream=_StreamThatFailsMidBody(b"0123456789abcdef") + ) + ) + collected: list[bytes] = [] + async with Transport(_cfg(max_retries=3)) as transport: + with pytest.raises(EasyvistaConnectionError): + async for chunk in transport.stream_bytes( + "documents/1/content", chunk_size=8 + ): + collected.append(chunk) + assert b"".join(collected) == b"0123456789abcdef" + assert route.call_count == 1, "a mid-stream failure was retried" + + +async def test_stream_bytes_rejects_a_foreign_origin(): + # The same-origin guard is a security property of the download path, not of + # one method on it: every request carries the instance Bearer token. Nothing + # is requested until iteration starts, so the refusal surfaces there. + async with Transport(_cfg()) as transport: + with pytest.raises(EasyvistaError, match="outside the configured instance"): + await _collect(transport.stream_bytes("https://attacker.test/download/42")) + + +@respx.mock +async def test_stream_bytes_drops_the_bearer_token_on_a_cross_host_redirect(): + # `follow_redirects=True` is as deliberate here as on get_bytes (a download + # URL commonly redirects to a signed location), and so is the reason it is + # safe: httpx strips Authorization when a redirect leaves the origin. Pinned + # on this path too, because the streaming send() call passes the flag itself + # rather than inheriting anything from the non-streaming one. + respx.get("https://ev.test/download/42").mock( + return_value=httpx.Response( + 302, headers={"Location": "https://cdn.attacker.test/blob/42"} + ) + ) + foreign = respx.get("https://cdn.attacker.test/blob/42").mock( + return_value=httpx.Response(200, content=b"blob") + ) + async with Transport(_cfg()) as transport: + chunks = await _collect(transport.stream_bytes("https://ev.test/download/42")) + assert b"".join(chunks) == b"blob" + leaked = "authorization" in foreign.calls.last.request.headers + assert not leaked, "the instance token followed a redirect off the instance" + + +@respx.mock +async def test_stream_bytes_keeps_the_bearer_token_on_a_same_host_redirect(): + # Control for the test above, exactly as on get_bytes: without it, a path + # that never sent Authorization at all would look like a pass. + respx.get("https://ev.test/download/42").mock( + return_value=httpx.Response( + 302, headers={"Location": "https://ev.test/download/42/signed"} + ) + ) + signed = respx.get("https://ev.test/download/42/signed").mock( + return_value=httpx.Response(200, content=b"blob") + ) + async with Transport(_cfg()) as transport: + chunks = await _collect(transport.stream_bytes("https://ev.test/download/42")) + assert b"".join(chunks) == b"blob" + assert signed.calls.last.request.headers["Authorization"] == "Bearer tok" diff --git a/easyvista_python_client/_fields.py b/easyvista_python_client/_fields.py index 12a1d9e..ce178ba 100644 --- a/easyvista_python_client/_fields.py +++ b/easyvista_python_client/_fields.py @@ -1,19 +1,49 @@ """Shared helpers for extracting human labels from EasyVista field objects. EasyVista nests label+href objects (``STATUS``, ``DEPARTMENT``, ``CATALOG_REQUEST``, -``URGENCY``, ``IMPACT``). Both the Markdown renderer (:mod:`context`) and the -statistics aggregator (:mod:`reporting`) need to pick the human label and never an -href, so the logic lives here once. +``URGENCY``, ``IMPACT``). The Markdown renderer (:mod:`context`) needs to pick the +human label and never an href, so the logic lives here once. (:mod:`reporting` +aggregates the same nested objects but does **not** route through this module -- +it goes through :func:`~easyvista_python_client.references.resolve_reference`.) + +Its consumer reads a ``model_dump(by_alias=True)`` dict, so a timestamp column +(``LAST_UPDATE``, ``CREATION_DATE_UT``, …) arrives here as a ``datetime`` since +the 2026-08-17 read-path retype +(:class:`~easyvista_python_client.models.common.OptionalDateTime`), not a +``str``. :func:`_text` renders it rather than discarding it, which is why this +leaf now imports :mod:`~easyvista_python_client.timestamps`. """ from __future__ import annotations +from datetime import datetime from typing import Any +from .timestamps import format_ev_datetime + def _text(value: Any) -> str: - """First stripped string form of ``value``; ``""`` if it is not a string.""" - return value.strip() if isinstance(value, str) else "" + """First stripped string form of ``value``; ``""`` if it doesn't render as text. + + A ``datetime`` renders as EasyVista's own wire format (:func:`format_ev_datetime`) + so the extracted text is byte-identical to EasyVista's own + millisecond-precision-with-offset rendering, and to what ``ev_since_filter`` + accepts. Not necessarily byte-identical to the *input* bytes: the fraction is + always 3 digits, so a source string with a different precision round-trips to + 3 digits here. ``format_ev_datetime`` raises on a *naive* + datetime, which should not occur for a value that came through + ``OptionalDateTime`` (it always normalizes to aware) -- but this is an + extractor, which must never raise, so a naive value still falls back to + plain ``.isoformat()`` instead. + """ + if isinstance(value, str): + return value.strip() + if isinstance(value, datetime): + try: + return format_ev_datetime(value) + except ValueError: + return value.isoformat() + return "" def _label(obj: Any, keys: tuple[str, ...]) -> str: diff --git a/easyvista_python_client/_sync/_transport.py b/easyvista_python_client/_sync/_transport.py index f4495a1..82cf4a1 100644 --- a/easyvista_python_client/_sync/_transport.py +++ b/easyvista_python_client/_sync/_transport.py @@ -16,6 +16,7 @@ from __future__ import annotations import json +from collections.abc import Generator, Iterator from typing import Any, NoReturn from urllib.parse import urlsplit @@ -39,6 +40,17 @@ EasyvistaValidationError, ) +#: Default chunk size, in bytes, for :meth:`Transport.stream_bytes`. +#: +#: 64 KiB is the ceiling this default is chosen to set: a caller streams an +#: attachment precisely so the whole file never sits in memory, and the chunk +#: size is what one step of that costs. Large enough that a 32 MB attachment is +#: ~512 iterations rather than tens of thousands, small enough that the resident +#: peak stays negligible beside the file. Deliberately not a config field -- +#: nobody has asked for an instance-wide value, and the one caller who cares +#: about a specific payload can pass ``chunk_size`` per call. +DEFAULT_STREAM_CHUNK_SIZE = 64 * 1024 + class BaseTransport: """Pure transport logic, independent of how a request is executed (no I/O).""" @@ -59,8 +71,16 @@ def resolve_url(self, path_or_url: str) -> str: That check is load-bearing, not decoration. Every request this transport makes carries the instance's Bearer token, so following an absolute URL taken out of a response body (an attachment's ``DDL_HREF``, say) would - hand that credential to whatever host the body named. The API is trusted - to describe its own instance, not to redirect us off it. + hand that credential to whatever host the body named. + + What is guaranteed is exactly that and no more: a foreign URL in a + response **body** is refused. An HTTP **redirect** off the instance is + still *followed* -- both download paths run with + ``follow_redirects=True``, which signed-location hops depend on -- and it + merely loses the credential (verified: no ``authorization`` header on the + foreign request, for Bearer and for Basic; a same-host redirect keeps + it). So streamed or downloaded bytes are not proof of instance origin, + and a caller must not treat them as such. """ parsed = urlsplit(path_or_url) if not parsed.scheme and not parsed.netloc: @@ -264,3 +284,111 @@ def get_bytes(self, path_or_url: str) -> bytes: self._raise_for_response(exc.response) except httpx.TransportError as exc: raise EasyvistaConnectionError(f"connection failed: {exc}") from exc + + def _open_stream( + self, path_or_url: str, chunk_size: int + ) -> tuple[httpx.Response, Iterator[bytes], list[bytes]]: + """Open a streaming GET and take its first chunk, as one retryable unit. + + Returns the still-open response, its chunk iterator, and the first chunk + wrapped in a list -- empty for an empty body, which is how "the body is + over" is distinguished from "there is a chunk" without a sentinel. + + Taking the first chunk *here* rather than in :meth:`stream_bytes` is the + whole point of this helper: everything inside it can be retried safely + because nothing it produces has reached the caller yet, so restarting + the request cannot deliver a byte twice. See :meth:`stream_bytes` for + the policy that rests on it. + + Two details are forced by streaming. The response must be closed on + every failure path, because an unread streaming response holds its + connection open. And :meth:`BaseTransport._raise_for_response` reads + ``.content``, which on a streaming response raises until the body has + actually been read -- hence the read before each raise, which is what + makes the error mapping identical to :meth:`get_bytes`. + """ + response = self._client.send( + self._client.build_request("GET", self.resolve_url(path_or_url)), + stream=True, + follow_redirects=True, + ) + try: + if self.is_retryable_status(response.status_code): + response.read() + raise _RetryableResponse(response) + if not response.is_success: + response.read() + self._raise_for_response(response) + chunks = response.iter_bytes(chunk_size) + first: list[bytes] = [] + for chunk in chunks: + first.append(chunk) + break + except BaseException: + response.close() + raise + return response, chunks, first + + def stream_bytes( + self, path_or_url: str, *, chunk_size: int = DEFAULT_STREAM_CHUNK_SIZE + ) -> Generator[bytes, None]: + """GET raw bytes (an attachment) in chunks, never as one object. + + The streaming twin of :meth:`get_bytes`, and deliberately identical to + it everywhere it can be: the same URL resolution through + :meth:`BaseTransport.resolve_url` (so a URL outside the configured + instance is refused here too), the same ``follow_redirects=True`` for + the signed-location hop, the same attempt count and backoff, and the + same error mapping -- a 403 on an attachment still raises + :class:`EasyvistaAuthError`, and a 590 is still not retried. What + differs is that the body is handed over in ``chunk_size`` pieces as it + arrives, so a large attachment never has to exist in memory whole. + + **Retrying stops as soon as a byte reaches the caller.** A retryable + status or a transport error while opening the download is retried like + any other request, and the first chunk is fetched inside that retried + unit so that a failure fetching it is still safe to restart. From that + chunk onwards the request is committed: a transport failure raises + :class:`EasyvistaConnectionError` instead of starting over, because + starting over would re-deliver bytes the caller already has. Nothing + resumes a partly consumed stream -- a caller that must survive a + mid-stream failure has to decide for itself whether to discard what it + collected and ask again, and this method will not make that choice by + silently duplicating data. + + No request is made until iteration begins. This is a generator, so a + refused URL -- and a non-positive ``chunk_size`` -- raises on the first + step rather than at the call. + """ + if chunk_size <= 0: + # Guarded here rather than left to httpx, which raises from inside + # its own ByteChunker: `chunk_size=0` surfaces as + # "ValueError: range() arg 3 must not be zero" and a negative one as + # "IndexError: list index out of range" -- both several frames below + # this client, so a caller computing a size (`total // n`, a config + # value that defaulted to 0) reads it as a library bug rather than + # bad input. + raise ValueError(f"chunk_size must be positive, got {chunk_size}") + retryer = Retrying( + stop=stop_after_attempt(self.config.max_retries + 1), + wait=wait_exponential(multiplier=0.5, max=10), + retry=retry_if_exception_type((_RetryableResponse, httpx.TransportError)), + reraise=True, + ) + opened: tuple[httpx.Response, Iterator[bytes], list[bytes]] + try: + opened = retryer(self._open_stream, path_or_url, chunk_size) + except _RetryableResponse as exc: + self._raise_for_response(exc.response) + except httpx.TransportError as exc: + raise EasyvistaConnectionError(f"connection failed: {exc}") from exc + response, chunks, first = opened + try: + for chunk in first: + yield chunk + for chunk in chunks: + yield chunk + except httpx.TransportError as exc: + raise EasyvistaConnectionError(f"connection failed: {exc}") from exc + finally: + response.close() diff --git a/easyvista_python_client/_sync/client.py b/easyvista_python_client/_sync/client.py index 737b273..5913f89 100644 --- a/easyvista_python_client/_sync/client.py +++ b/easyvista_python_client/_sync/client.py @@ -10,11 +10,14 @@ from __future__ import annotations -from collections.abc import Iterator, Sequence +from collections.abc import Iterator, Iterable, Sequence from datetime import datetime from easyvista_python_client._sync._concurrency import Semaphore, settle -from easyvista_python_client._sync._transport import Transport +from easyvista_python_client._sync._transport import ( + DEFAULT_STREAM_CHUNK_SIZE, + Transport, +) from easyvista_python_client._transport import RequestSpec from easyvista_python_client.config import EasyvistaConfig from easyvista_python_client.context import TicketContext @@ -27,7 +30,7 @@ from easyvista_python_client.exceptions import EasyvistaAuthError, EasyvistaNotFound from easyvista_python_client.field_model import parse_memo from easyvista_python_client.filters import ev_equals_filter, is_safe_ev_value -from easyvista_python_client.models.action import Action, PostAction +from easyvista_python_client.models.action import Action, ActionUpdate, PostAction from easyvista_python_client.models.asset import Asset, PostAsset from easyvista_python_client.models.department import ( Department, @@ -141,6 +144,21 @@ def iter_tickets( Pages of ``page_size`` (default ``config.default_max_rows``) until the server reports no further page (``@next``) or ``max_records`` is reached. + + ``sort`` is forwarded to the wire and its token must be + **space-separated** — ``"LAST_UPDATE"`` or ``"LAST_UPDATE DESC"``. + ``"LAST_UPDATE:DESC"``, ``"-LAST_UPDATE"`` and ``"DESC(LAST_UPDATE)"`` + are each **silently ignored** (measured live): the server returns its + default order with no error, so an unsorted result looks sorted. This is + not validated locally, so the token is the caller's to get right. + + Sorting is not cosmetic when the filter selects rows that are changing: + an unsorted offset sweep over a change window can skip a record + permanently, and the two sort DIRECTIONS do not fail the same way -- + descending defers such a miss to the next sweep, ascending loses it. See + :func:`~easyvista_python_client.ev_since_filter`, which rules on the + direction and names the keyset alternative for a caller who cannot + tolerate even a deferred miss. """ if page_size is None: page_size = self.config.default_max_rows @@ -214,9 +232,40 @@ def ticket_statistics( ) def update_ticket(self, rfc_number: str, update: RequestUpdate) -> Request: + """Update a ticket's writable fields. + + Cannot set a status: there is no flat status update on this API. See + :meth:`set_status`, and :class:`RequestUpdate` for the measurements. + """ spec, parse = requests_res.build_update_ticket(rfc_number, update) return parse(self._transport.send(spec)) + def set_status( + self, rfc_number: str, *, status_guid: str, comment: str | None = None + ) -> Request: + """Set a ticket's status, addressed by ``STATUS_GUID``. + + This is the API's only working status write, and it reaches **every** + status rather than only terminal ones: given six different status GUIDs + in turn, a fresh ticket landed on exactly the status requested every + time, non-terminal ones included. + + It sends the documented ``{"closed": {"status_GUID": ...}}`` body -- the + same request :meth:`close_ticket` sends, under a name that matches what + it does, because "close" is what the wire calls it and not what it is + limited to. + + Note the addressing. A ``STATUS_GUID`` is not a ``STATUS_ID``; the two + are different columns, and only the GUID works here. Read a status's GUID + off any ticket in that status (the nested ``STATUS`` object carries + ``STATUS_GUID``) -- they are stable per instance but are **not** + portable between instances. + """ + spec, parse = requests_res.build_set_status( + rfc_number, status_guid=status_guid, comment=comment + ) + return parse(self._transport.send(spec)) + def close_ticket( self, rfc_number: str, @@ -246,8 +295,43 @@ def create_action(self, rfc_number: str, action: PostAction) -> Action: spec, parse = actions_res.build_create_action(rfc_number, action) return parse(self._transport.send(spec)) - def list_actions(self, rfc_number: str) -> list[Action]: - spec, parse = actions_res.build_list_actions(rfc_number) + def list_actions( + self, rfc_number: str, *, fields: Iterable[str] | str | None = None + ) -> list[Action]: + """List a ticket's actions. + + The default projection is slim: it carries ``ACTION_ID``, + ``ACTION_LABEL_FR``, ``ACTION_NUMBER``, ``DONE_BY_ID`` and + ``EXPECTED_START_DATE_UT`` but **no** ``CREATION_DATE_UT`` or + ``LAST_UPDATE``. Pass ``fields`` to project them onto the list and read + a page of actions' timestamps and authors in one request rather than one + item fetch each:: + + actions = client.list_actions( + rfc, + fields=["ACTION_ID", "ACTION_TYPE_ID", "CREATION_DATE_UT", + "LAST_UPDATE", "DONE_BY_ID"], + ) + + **Returns at most ONE page and does not paginate.** The cap is + ``config.default_max_rows``; a ticket with more actions than that is + **truncated with no error**, and this method discards the envelope's + total, so a caller cannot detect the truncation from the result. This is + not hypothetical: a freshly created ticket already carries about twelve + actions, most of them workflow-generated. Raise + ``EasyvistaConfig.default_max_rows`` if a ticket's whole log matters. + + The note text is never projectable — ``DESCRIPTION`` and ``COMMENT`` + are Memo sub-resources and come back as HREF objects under every + projection, so a body still costs one :meth:`resolve_memo` per action. + + ``fields`` has two more silent footguns: ``"*"`` is not a wildcard — + it silently reduces to ``ACTION_ID`` alone — and a dotted path such as + ``DESCRIPTION.HREF`` is silently dropped. + """ + spec, parse = actions_res.build_list_actions( + rfc_number, fields=fields, max_rows=self.config.default_max_rows + ) return parse(self._transport.send(spec)) def get_action(self, action_id: str | int) -> Action: @@ -259,6 +343,23 @@ def get_action(self, action_id: str | int) -> Action: spec, parse = actions_res.build_get_action(action_id) return parse(self._transport.send(spec)) + def update_action(self, action_id: str | int, update: ActionUpdate) -> Action: + """Edit an existing action's note text. + + Live-verified 2026-08-17 by re-reading the memo afterwards, not by the + status code. Note that an action can be edited but **not deleted** — + ``DELETE actions/{id}`` is refused with HTTP 403 — so there is + deliberately no ``delete_action``. + + The returned :class:`Action` is the API's own echo and is **not + verified**: the PUT's response body has never been captured, and if it + answers empty or href-only the parser yields an ``Action`` whose fields + are all ``None``. Re-read with :meth:`get_action` rather than reading + fields off the return value. + """ + spec, parse = actions_res.build_update_action(action_id, update) + return parse(self._transport.send(spec)) + def _resolve_action_body(self, action: Action) -> Action: """Return ``action`` with its note text resolved onto ``description``. @@ -348,6 +449,17 @@ def list_documents(self, rfc_number: str) -> list[Document]: spec, parse = documents_res.build_list_documents(rfc_number) return parse(self._transport.send(spec)) + def delete_document(self, rfc_number: str, document_id: str) -> None: + """Remove an attachment from a ticket. + + ``document_id`` is the ``DOCUMENT_ID`` from :meth:`list_documents`. + Live-verified 2026-08-17 by re-listing the ticket's documents + afterwards. Returns nothing: the API answers with an empty body. + """ + self._transport.send( + documents_res.build_delete_document(rfc_number, document_id) + ) + def download_document(self, document: Document | str) -> bytes: """Fetch an attachment's bytes. @@ -359,6 +471,74 @@ def download_document(self, document: Document | str) -> bytes: """ return self._transport.get_bytes(documents_res.download_href(document)) + def stream_document( + self, document: Document | str, *, chunk_size: int = DEFAULT_STREAM_CHUNK_SIZE + ) -> Iterator[bytes]: + """Fetch an attachment's bytes in chunks, without holding the file whole. + + Accepts exactly what :meth:`download_document` accepts -- a + :class:`Document` from :meth:`list_documents` or a raw href/path -- and + resolves it identically, refusing a URL outside the configured instance + for the same reason. Use this when the bytes are on their way somewhere + else in pieces (a file on disk, a hash, another API) and + :meth:`download_document` when a single ``bytes`` object is what you + wanted anyway. + + Called ``stream_`` rather than ``iter_`` on purpose: every ``iter_*`` + method on this client iterates *records*, and this iterates the bytes of + one document. + + The opposite direction cannot stream at all. :meth:`add_document` sends + base64 inside a JSON body, so an upload has to materialise the whole + payload however it is called; the asymmetry is the API's, not an + oversight here. + + Retrying covers opening the download only. Once the first chunk has been + handed over the request is committed, and a transport failure raises + :class:`~easyvista_python_client.exceptions.EasyvistaConnectionError` + rather than starting again -- starting again would re-deliver bytes the + caller already has. A partly consumed stream is never resumed, so + deciding what to do with a mid-stream failure is the caller's. See + :meth:`~easyvista_python_client._sync._transport.Transport.stream_bytes`. + + Nothing is requested until iteration begins: this is a generator, so + :class:`ValueError` for a record carrying no download URL and + :class:`EasyvistaError` for one pointing off the instance both surface + on the first step rather than at the call. + + **Stopping early:** on the async surface a bare ``break`` leaves the + response checked out of the connection pool until the event loop's + async-generator finalizer runs, which is a garbage-collection cycle away + (measured) -- so a caller that reads only a prefix of many attachments + under a bounded ``max_connections`` can stall on connections it appears + to have released. Close the generator instead (``aclose()``, or + ``contextlib.aclosing``). On the sync surface refcounting releases it at + the ``break`` and nothing is needed. + """ + stream = self._transport.stream_bytes( + documents_res.download_href(document), chunk_size=chunk_size + ) + # Close the inner generator in a `finally` rather than leaving it to be + # collected. `stream`'s own `finally` is what releases the response and + # returns its connection to the pool, and unwinding *this* generator + # does not reach it on its own -- the loop below simply exits. + # + # What this buys, stated precisely: one deferral instead of two. Closing + # this generator -- explicitly, or by an exception propagating out of it + # -- now releases the response at once. A bare `break` still defers on + # the async surface, because unwinding this generator is itself left to + # the event loop's async-generator finalizer; what the `finally` removes + # is the *second* wait, for `stream` to become garbage in its own right. + # On the sync surface refcounting closes it promptly either way. + # `contextlib.closing`/`aclosing` would say this in one line, but those + # two names differ by more than a token so the codegen cannot generate + # the pair; `stream.close()`/`stream.aclose()` it can. + try: + for chunk in stream: + yield chunk + finally: + stream.close() + # --- departments ---------------------------------------------------------- def get_department(self, department_id: str | int) -> Department: spec, parse = departments_res.build_get_department(department_id) @@ -571,6 +751,13 @@ def get_ticket_context( It costs two extra requests per action; pass ``False`` to skip it when you only need the action list. + **The action log is capped at one page.** It comes from + :meth:`list_actions`, which returns at most ``config.default_max_rows`` + actions and does not paginate, so on a busy ticket + :attr:`TicketContext.actions` — and therefore + :meth:`TicketContext.to_markdown`'s rendered log — is silently truncated + with no error. Raise ``default_max_rows`` if completeness matters. + On the async surface the independent requests (the two memos plus the actions and documents lists) are issued concurrently, in up to three waves; on the sync surface they run one after another in source order, @@ -672,9 +859,14 @@ def get_department_context( 403/404 degrades it to ``[]`` / ``None`` / ``0`` (same pattern as :meth:`get_ticket_context`). The flags trim the heavier related calls. Tickets and assets filter on ``DEPARTMENT_ID:""``. ``recent_tickets`` - ordering is best-effort: it relies on the server honoring - ``RECENT_TICKETS_SORT`` (open item O-DIR-1) and silently degrades to the - API's default order otherwise. + is ordered by **descending ``RFC_NUMBER``** (``RECENT_TICKETS_SORT``), + which is newest-first only where RFC numbers are issued monotonically: + it is a varchar, so the sort orders by the request-type prefix letter + before the date. The token must stay + space-separated: a colon form is silently ignored and degrades to the + API's default order with no error (measured live 2026-08-17). Ordering + therefore depends on the server honouring that token, which the live + suite asserts rather than this method. On the async surface the seven independent branches are issued concurrently, costing two waves instead of eight serial steps; on the diff --git a/easyvista_python_client/_sync/tests/test_client.py b/easyvista_python_client/_sync/tests/test_client.py index a7d0296..bedea37 100644 --- a/easyvista_python_client/_sync/tests/test_client.py +++ b/easyvista_python_client/_sync/tests/test_client.py @@ -11,6 +11,7 @@ """ import json +from collections.abc import Iterator import httpx import pytest @@ -19,8 +20,8 @@ from easyvista_python_client._sync import client as client_module from easyvista_python_client._sync.client import EasyvistaClient from easyvista_python_client.directory import DepartmentContext -from easyvista_python_client.exceptions import EasyvistaError -from easyvista_python_client.models.action import PostAction +from easyvista_python_client.exceptions import EasyvistaAuthError, EasyvistaError +from easyvista_python_client.models.action import ActionUpdate, PostAction from easyvista_python_client.models.asset import PostAsset from easyvista_python_client.models.department import ( Department, @@ -110,7 +111,7 @@ def test_update_and_close_ticket(config): return_value=httpx.Response(200, json={"records": [{"RFC_NUMBER": "I1"}]}) ) with EasyvistaClient(config) as client: - client.update_ticket("I1", RequestUpdate(status_id=4)) + client.update_ticket("I1", RequestUpdate(impact_id=4)) client.close_ticket("I1", comment="resolved") @@ -142,6 +143,38 @@ def test_create_and_list_actions(config): assert listed[0].action_id == 5 +@respx.mock +def test_list_actions_forwards_a_fields_projection(config): + """The client must pass fields= through, not just accept it. + + EV-R3: fields= is what turns comment metadata into one request per ticket + instead of one request per action. + """ + route = respx.get(f"{ROOT}/actions").mock( + return_value=httpx.Response(200, json={"records": []}) + ) + with EasyvistaClient(config) as client: + client.list_actions("I240101_0001", fields=["ACTION_ID", "LAST_UPDATE"]) + assert route.calls.last.request.url.params["fields"] == "ACTION_ID,LAST_UPDATE" + + +@respx.mock +def test_list_actions_sends_the_configured_row_cap(config): + """``list_actions`` returns one page, so the cap must be the client's own. + + Without this the request carried no ``max_rows`` at all and the truncation + point was the server's unstated default -- invisible to the caller and not + raisable by configuration. + """ + route = respx.get(f"{ROOT}/actions").mock( + return_value=httpx.Response(200, json={"records": []}) + ) + with EasyvistaClient(config) as client: + expected = str(client.config.default_max_rows) + client.list_actions("I240101_0001") + assert route.calls.last.request.url.params["max_rows"] == expected + + @respx.mock def test_get_action_fetches_the_item_level_record(config): respx.get(f"{ROOT}/actions/52990").mock( @@ -159,6 +192,18 @@ def test_get_action_fetches_the_item_level_record(config): assert action.description == {"HREF": f"{ROOT}/actions/52990/description"} +@respx.mock +def test_update_action_sends_a_put_to_the_top_level_path(config): + """The nested requests/{rfc}/actions/{id} form returns 403 (verified live).""" + route = respx.put(f"{ROOT}/actions/57483").mock( + return_value=httpx.Response(200, json={"ACTION_ID": 57483}) + ) + with EasyvistaClient(config) as client: + action = client.update_action(57483, ActionUpdate(description="edited")) + assert json.loads(route.calls.last.request.content) == {"description": "edited"} + assert action.action_id == 57483 + + @respx.mock def test_from_env_constructs_working_client(monkeypatch): monkeypatch.setenv("EASYVISTA_URL", "https://ev.test") @@ -257,6 +302,18 @@ def test_add_and_list_documents(config): assert body["documents"][0]["filename"] == "a.txt" +@respx.mock +def test_delete_document_sends_a_delete_to_the_nested_path(config): + """Returns None: the API answers a delete with an empty body.""" + route = respx.delete(f"{ROOT}/requests/I240101_0001/documents/12345_abcdef").mock( + return_value=httpx.Response(200) + ) + with EasyvistaClient(config) as client: + result = client.delete_document("I240101_0001", "12345_abcdef") + assert route.call_count == 1 + assert result is None + + @respx.mock def test_download_document_fetches_the_ddl_href(config): route = respx.get("https://ev.test/dl/7").mock( @@ -287,6 +344,130 @@ def test_download_document_refuses_a_foreign_download_url(config): client.download_document(doc) +@respx.mock +def test_stream_document_chunks_reassemble_to_the_download(config): + # 3076 bytes at chunk_size=512: six full chunks and a 4-byte tail. Sized off + # the boundary on purpose -- an exact multiple never exercises a short final + # chunk, and reassembly passes either way. + body = bytes(range(256)) * 12 + b"tail" + respx.get("https://ev.test/dl/7").mock( + return_value=httpx.Response(200, content=body) + ) + doc = Document.model_validate( + {"DOCUMENT": "big.bin", "DDL_HREF": "https://ev.test/dl/7"} + ) + chunks = [] + with EasyvistaClient(config) as client: + for chunk in client.stream_document(doc, chunk_size=512): + chunks.append(chunk) + assert b"".join(chunks) == body + assert len(chunks) == 7, "the body arrived in one piece instead of streaming" + assert len(chunks[-1]) == 4, "the short final chunk was padded or dropped" + + +class _ClosableStream(httpx.SyncByteStream): + """A response body that records when the transport closed it. + + ``closed`` flips in ``aclose()``/``close()``, which httpx calls when the + response is + released -- which is what ``stream_bytes`` does in its own ``finally``. So + the flag answers "has the connection gone back to the pool yet". + """ + + def __init__(self, body: bytes) -> None: + self._body = body + self.closed = False + + def __iter__(self) -> Iterator[bytes]: + yield self._body + + def close(self) -> None: + self.closed = True + + +@respx.mock +def test_stream_document_closes_the_inner_stream_when_stopped_early(config): + """Stopping early releases the connection there and then, not eventually. + + ``stream_document`` hands out chunks from an inner ``stream_bytes`` + generator, and only *that* generator's ``finally`` closes the response and + returns its connection to the httpx pool. Closing the outer generator + unwinds its loop with ``GeneratorExit`` and does not close the inner one, so + without the explicit close in ``stream_document`` the release waits for the + inner generator to be collected -- and this asserts with no ``gc`` round and + no scheduling hop in between. Until the release happens the connection stays + checked out, which a caller with a small ``max_connections`` feels. + + Closing explicitly is also the only moment both client trees share: a caller + that just stops iterating leaves the outer generator to be collected, and + when that happens is a property of the runtime, not of this method. + """ + stream = _ClosableStream(b"0123456789abcdef") + respx.get("https://ev.test/dl/7").mock( + return_value=httpx.Response(200, stream=stream) + ) + doc = Document.model_validate({"DDL_HREF": "https://ev.test/dl/7"}) + with EasyvistaClient(config) as client: + chunks = client.stream_document(doc, chunk_size=8) + assert chunks.__next__() == b"01234567" + assert not stream.closed, "closed while the caller was still reading" + chunks.close() + assert stream.closed, "the abandoned stream still holds its connection" + + +@respx.mock +def test_stream_document_accepts_a_relative_path_like_download_document(config): + """Same accepted inputs as download_document, resolved the same way.""" + respx.get(f"{ROOT}/documents/7/content").mock( + return_value=httpx.Response(200, content=b"bytes") + ) + path = "documents/7/content" + with EasyvistaClient(config) as client: + streamed = [chunk for chunk in client.stream_document(path)] + downloaded = client.download_document(path) + assert b"".join(streamed) == downloaded == b"bytes" + + +@respx.mock +def test_stream_document_and_download_document_agree_on_a_403(config): + """One error mapping, not two: the streaming path must not soften a failure. + + Asserted as an equality between the paths rather than against a hardcoded + type, so the two cannot drift apart without this failing -- which is the + actual risk, since a streaming response needs its body read before the + mapping can look at it at all. + """ + respx.get("https://ev.test/dl/7").mock( + return_value=httpx.Response(403, json={"error": "forbidden"}) + ) + doc = Document.model_validate({"DDL_HREF": "https://ev.test/dl/7"}) + with EasyvistaClient(config) as client: + with pytest.raises(EasyvistaError) as streamed: + [chunk for chunk in client.stream_document(doc)] + with pytest.raises(EasyvistaError) as downloaded: + client.download_document(doc) + assert type(streamed.value) is type(downloaded.value) is EasyvistaAuthError + assert streamed.value.status_code == downloaded.value.status_code == 403 + assert streamed.value.ev_message == downloaded.value.ev_message == "forbidden" + + +def test_stream_document_refuses_a_foreign_download_url(config): + # The same-origin guard covers the streaming path too: it is a property of + # the download, not of one method. Nothing is requested until iteration + # begins, so the refusal lands on the first step. + doc = Document.model_validate({"DDL_HREF": "https://attacker.test/dl/7"}) + with EasyvistaClient(config) as client: + with pytest.raises(EasyvistaError, match="outside the configured instance"): + [chunk for chunk in client.stream_document(doc)] + + +def test_stream_document_needs_a_download_url(config): + doc = Document.model_validate({"DOCUMENT": "orphan.txt"}) + with EasyvistaClient(config) as client: + with pytest.raises(ValueError, match="no download URL"): + [chunk for chunk in client.stream_document(doc)] + + # --- pagination -------------------------------------------------------------- diff --git a/easyvista_python_client/_sync/tests/test_transport.py b/easyvista_python_client/_sync/tests/test_transport.py index 5dc5ec4..ae9304a 100644 --- a/easyvista_python_client/_sync/tests/test_transport.py +++ b/easyvista_python_client/_sync/tests/test_transport.py @@ -7,6 +7,8 @@ only one surface. """ +from collections.abc import Iterator + import httpx import pytest import respx @@ -434,3 +436,293 @@ def test_get_bytes_keeps_the_bearer_token_on_a_same_host_redirect(): content = transport.get_bytes("https://ev.test/download/42") assert content == b"blob" assert signed.calls.last.request.headers["Authorization"] == "Bearer tok" + + +# --- stream_bytes ------------------------------------------------------------ +# +# The streaming download. Its contract is "identical to get_bytes except that +# the body arrives in pieces", so most of these are the get_bytes assertions +# above re-made against the chunked path -- that duplication is the point, since +# the two implementations share no code past `resolve_url`. The one claim with +# no get_bytes counterpart is the retry boundary: retrying stops once a byte has +# reached the caller, because restarting would deliver it twice. + + +class _StreamThatFailsMidBody(httpx.SyncByteStream): + """A response body that delivers ``prefix`` and then drops the connection. + + ``respx`` can fail a request before a response exists, which is what the + ``side_effect=httpx.ConnectError`` mocks elsewhere in this module do. It has + no way to fail one *after* the status line, and that is exactly the case the + retry boundary is about -- so the failure is injected into the body stream + itself, which httpx surfaces as a real ``TransportError`` while iterating. + """ + + def __init__(self, prefix: bytes) -> None: + self._prefix = prefix + + def __iter__(self) -> Iterator[bytes]: + yield self._prefix + raise httpx.ReadError("connection dropped mid-body") + + +class _StreamThatFailsBeforeTheFirstByte(httpx.SyncByteStream): + """A response body that drops the connection without yielding anything. + + The sibling of :class:`_StreamThatFailsMidBody`, for the *other* side of the + retry boundary: the status line arrived, so this is past the point respx can + fail a request, but no byte has reached the caller yet, so restarting is + still safe and must happen. + """ + + def __iter__(self) -> Iterator[bytes]: + raise httpx.ReadError("dropped before the first byte") + yield b"" # unreachable; the yield is what makes this a generator function + + +def _collect(chunks: Iterator[bytes]) -> list[bytes]: + """Every chunk a stream yields, kept separate rather than joined.""" + return [chunk for chunk in chunks] + + +@respx.mock +def test_stream_bytes_reassembles_to_the_whole_body(): + # 10244 bytes at chunk_size=1024: deliberately NOT a multiple of it, so the + # last chunk is a short one. A body sized to an exact multiple never + # exercises the ragged tail, and the reassembly assertion below would + # pass either way. + body = bytes(range(256)) * 40 + b"tail" + respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(200, content=body) + ) + with Transport(_cfg()) as transport: + chunks = _collect( + transport.stream_bytes("documents/1/content", chunk_size=1024) + ) + assert b"".join(chunks) == body + # More than one chunk, and each bounded: proves the body is delivered + # progressively rather than read whole and handed over in a single piece. + assert len(chunks) == 11 + assert max(len(chunk) for chunk in chunks) <= 1024 + assert len(chunks[-1]) == 4, "the short final chunk was padded or dropped" + + +@respx.mock +def test_stream_bytes_chunks_at_the_documented_default_size(): + """The default chunk size is 64 KiB, and this is what says so. + + ``DEFAULT_STREAM_CHUNK_SIZE`` is quoted as "64 KiB" in the CHANGELOG, in the + document-workflow skill and in the constant's own comment (whose "a 32 MB + attachment is ~512 iterations" arithmetic only holds at that value). Every + other chunk-counting test passes ``chunk_size`` explicitly, so without this + one the constant could change to anything and leave all three false with a + green suite. 160 KiB of body -> three chunks, the last a short one. + """ + body = bytes(range(256)) * 640 # 163840 bytes == 2.5 * 64 KiB + respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(200, content=body) + ) + with Transport(_cfg()) as transport: + chunks = _collect(transport.stream_bytes("documents/1/content")) + assert b"".join(chunks) == body + assert [len(chunk) for chunk in chunks] == [65536, 65536, 32768] + + +@pytest.mark.parametrize("bad", [0, -8]) +def test_stream_bytes_refuses_a_non_positive_chunk_size(bad): + """The newest public parameter must fail as bad input, not as a library bug. + + Left unguarded, httpx raises from inside its own ``ByteChunker`` several + frames below this client: ``chunk_size=0`` surfaces as "range() arg 3 must + not be zero" and a negative one as "list index out of range". A caller + computing a chunk size reads either as our bug. No request is made, so this + needs no mock -- and because ``stream_bytes`` is a generator, the raise lands + on the first iteration step, matching the deferred ``ValueError`` for a + record with no download URL. + """ + with Transport(_cfg()) as transport: + with pytest.raises(ValueError, match="chunk_size must be positive"): + _collect( + transport.stream_bytes("documents/1/content", chunk_size=bad) + ) + + +@respx.mock +def test_stream_bytes_yields_nothing_for_an_empty_body(): + respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(200, content=b"") + ) + with Transport(_cfg()) as transport: + assert _collect(transport.stream_bytes("documents/1/content")) == [] + + +@respx.mock +def test_stream_bytes_sends_the_bearer_header(): + route = respx.get("https://ev.test/download/42").mock( + return_value=httpx.Response(200, content=b"ok") + ) + with Transport(_cfg()) as transport: + _collect(transport.stream_bytes("https://ev.test/download/42")) + assert route.calls.last.request.headers["Authorization"] == "Bearer tok" + + +@respx.mock +def test_stream_bytes_maps_403_to_auth_error(): + # The status is on an unread streaming response, whose `.content` raises + # until the body is read -- so the error mapping cannot simply be reused, it + # has to read the body first. This asserts the mapped type AND that the + # parsed EasyVista fields survived that detour. + respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(403, json={"error": "forbidden", "code": "9"}) + ) + with Transport(_cfg()) as transport: + with pytest.raises(EasyvistaAuthError) as ei: + _collect(transport.stream_bytes("documents/1/content")) + assert ei.value.status_code == 403 + assert ei.value.ev_message == "forbidden" + assert ei.value.ev_code == "9" + + +@respx.mock +def test_stream_bytes_does_not_retry_a_590(): + route = respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(590, json={"error": "rejected"}) + ) + with Transport(_cfg(max_retries=3)) as transport: + with pytest.raises(EasyvistaValidationError): + _collect(transport.stream_bytes("documents/1/content")) + assert route.call_count == 1 + + +@respx.mock +def test_stream_bytes_retries_a_retryable_status_on_the_open(): + route = respx.get(f"{ROOT}/documents/1/content").mock( + side_effect=[httpx.Response(503), httpx.Response(200, content=b"bytes")] + ) + with Transport(_cfg(max_retries=2)) as transport: + chunks = _collect(transport.stream_bytes("documents/1/content")) + assert b"".join(chunks) == b"bytes" + assert route.call_count == 2 + + +@respx.mock +def test_stream_bytes_exhausts_retries_raises_server_error(): + route = respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response(503) + ) + with Transport(_cfg(max_retries=1)) as transport: + with pytest.raises(EasyvistaServerError): + _collect(transport.stream_bytes("documents/1/content")) + assert route.call_count == 2 + + +@respx.mock +def test_stream_bytes_transport_error_on_the_open_raises_connection_error(): + respx.get(f"{ROOT}/documents/1/content").mock( + side_effect=httpx.ConnectError("boom") + ) + with Transport(_cfg()) as transport: + with pytest.raises(EasyvistaConnectionError): + _collect(transport.stream_bytes("documents/1/content")) + + +@respx.mock +def test_stream_bytes_retries_a_failure_fetching_the_first_chunk(): + """The first chunk is fetched INSIDE the retried unit. This is what pins it. + + ``_open_stream`` takes the first chunk itself, so a body that dies before + yielding a byte is still a safe restart -- nothing has reached the caller, so + replaying the request cannot deliver anything twice. Move that fetch out of + the retried unit (open there, iterate the whole body here) and the failure + below escapes as ``EasyvistaConnectionError`` on the first attempt instead, + with ``call_count == 1``. Every other ``stream_bytes`` test passes under both + arrangements, including the mid-body one just after this: the two differ only + on a first-chunk failure, which is only this test. + """ + route = respx.get(f"{ROOT}/documents/1/content").mock( + side_effect=[ + httpx.Response(200, stream=_StreamThatFailsBeforeTheFirstByte()), + httpx.Response(200, content=b"0123456789abcdef"), + ] + ) + with Transport(_cfg(max_retries=2)) as transport: + chunks = _collect( + transport.stream_bytes("documents/1/content", chunk_size=8) + ) + assert b"".join(chunks) == b"0123456789abcdef" + assert route.call_count == 2, "a pre-first-byte failure was not retried" + + +@respx.mock +def test_stream_bytes_does_not_retry_after_a_chunk_has_been_yielded(): + """A mid-body failure is the caller's to handle, never silently restarted. + + Retrying here would hand the caller the opening bytes a second time, so the + request is committed the moment a chunk is delivered. The delivered prefix + stays visible -- the caller keeps what it already collected -- and the + failure arrives as a mapped ``EasyvistaConnectionError``, not as a raw httpx + error. A "helpful" change making this resumable fails on the call count. + """ + route = respx.get(f"{ROOT}/documents/1/content").mock( + return_value=httpx.Response( + 200, stream=_StreamThatFailsMidBody(b"0123456789abcdef") + ) + ) + collected: list[bytes] = [] + with Transport(_cfg(max_retries=3)) as transport: + with pytest.raises(EasyvistaConnectionError): + for chunk in transport.stream_bytes( + "documents/1/content", chunk_size=8 + ): + collected.append(chunk) + assert b"".join(collected) == b"0123456789abcdef" + assert route.call_count == 1, "a mid-stream failure was retried" + + +def test_stream_bytes_rejects_a_foreign_origin(): + # The same-origin guard is a security property of the download path, not of + # one method on it: every request carries the instance Bearer token. Nothing + # is requested until iteration starts, so the refusal surfaces there. + with Transport(_cfg()) as transport: + with pytest.raises(EasyvistaError, match="outside the configured instance"): + _collect(transport.stream_bytes("https://attacker.test/download/42")) + + +@respx.mock +def test_stream_bytes_drops_the_bearer_token_on_a_cross_host_redirect(): + # `follow_redirects=True` is as deliberate here as on get_bytes (a download + # URL commonly redirects to a signed location), and so is the reason it is + # safe: httpx strips Authorization when a redirect leaves the origin. Pinned + # on this path too, because the streaming send() call passes the flag itself + # rather than inheriting anything from the non-streaming one. + respx.get("https://ev.test/download/42").mock( + return_value=httpx.Response( + 302, headers={"Location": "https://cdn.attacker.test/blob/42"} + ) + ) + foreign = respx.get("https://cdn.attacker.test/blob/42").mock( + return_value=httpx.Response(200, content=b"blob") + ) + with Transport(_cfg()) as transport: + chunks = _collect(transport.stream_bytes("https://ev.test/download/42")) + assert b"".join(chunks) == b"blob" + leaked = "authorization" in foreign.calls.last.request.headers + assert not leaked, "the instance token followed a redirect off the instance" + + +@respx.mock +def test_stream_bytes_keeps_the_bearer_token_on_a_same_host_redirect(): + # Control for the test above, exactly as on get_bytes: without it, a path + # that never sent Authorization at all would look like a pass. + respx.get("https://ev.test/download/42").mock( + return_value=httpx.Response( + 302, headers={"Location": "https://ev.test/download/42/signed"} + ) + ) + signed = respx.get("https://ev.test/download/42/signed").mock( + return_value=httpx.Response(200, content=b"blob") + ) + with Transport(_cfg()) as transport: + chunks = _collect(transport.stream_bytes("https://ev.test/download/42")) + assert b"".join(chunks) == b"blob" + assert signed.calls.last.request.headers["Authorization"] == "Bearer tok" diff --git a/easyvista_python_client/directory.py b/easyvista_python_client/directory.py index 93d9080..fc7ee82 100644 --- a/easyvista_python_client/directory.py +++ b/easyvista_python_client/directory.py @@ -10,10 +10,24 @@ from .models.request import Request from .reporting import TicketStatistics -# O-DIR-1: the descending-sort token for "most recent" is not yet live-confirmed. -# EasyVista ignores an unknown ``sort`` param (falls back to default order) rather -# than erroring, so this is safe; adjust once confirmed against the live instance. -RECENT_TICKETS_SORT = "RFC_NUMBER:DESC" +# O-DIR-1: the descending-sort token must be SPACE-separated. Measured live +# 2026-08-17 on a date column: `FIELD DESC` and `FIELD desc` genuinely reorder, +# while `FIELD:DESC`, `-FIELD` and `DESC(FIELD)` are silently ignored — they +# return the API's default order, byte-identical to an unsorted page, with no +# error. This constant previously used the ignored colon form, so +# `recent_tickets` was never actually sorted. The rule is syntactic rather than +# field-specific, so it is applied to RFC_NUMBER here by inference; +# integration_tests/test_live_change_window.py pins this exact token live. +# +# What is measured is the DESCENDING-ness, not recency. RFC_NUMBER is a varchar +# (`I240101_0001`), so a descending string sort orders by the request-type +# prefix FIRST and the date second: on an instance that issues more than one +# prefix letter, every `R...` ticket outranks every `I...` ticket regardless of +# date. Hence the docstrings say "descending RFC_NUMBER" rather than +# "newest-first". Switching to a date column (`CREATION_DATE_UT DESC`) would +# make recency literal and is a candidate follow-up; it is a behaviour change +# and wants its own live check first. +RECENT_TICKETS_SORT = "RFC_NUMBER DESC" @dataclass diff --git a/easyvista_python_client/filters.py b/easyvista_python_client/filters.py index 511e2fb..c811bb9 100644 --- a/easyvista_python_client/filters.py +++ b/easyvista_python_client/filters.py @@ -1,6 +1,6 @@ """Safe builders for EasyVista ``search`` expressions. -EasyVista's search grammar has two traps a caller cannot see, both verified +EasyVista's search grammar has three traps a caller cannot see, all verified against a live instance: 1. An expression it cannot parse is **silently ignored** and every record is @@ -8,8 +8,12 @@ 2. ``,`` is a live combinator (OR within one field, AND across fields), so an unescaped value that closes its quote can append conditions and silently widen the result set. +3. A comparison operator does not exist. A change window is an *interval in the + value position* — ``FIELD:(a;b)`` — and its bound is UNQUOTED, so + ``ev_since_filter``/``ev_between_filter`` validate the bound's shape rather + than escaping it. -These builders exist so neither can happen. Filters return ``None`` for blank +These builders exist so none of them can happen. Filters return ``None`` for blank input so callers compose without conditionals:: search = ev_equals_filter("DEPARTMENT_CODE", code) @@ -23,7 +27,11 @@ from __future__ import annotations +import re from collections.abc import Iterable +from datetime import datetime + +from .timestamps import format_ev_datetime, parse_ev_datetime # A double quote terminates the quoted value, letting a caller reach the ',' # combinator; no escape for it is known (verified live). ',' itself is NOT @@ -74,9 +82,351 @@ def ev_in_filter(field: str, values: Iterable[str | int | None]) -> str | None: return ",".join(parts) +# An interval bound is rendered UNQUOTED inside `(...)`, so the quote-based +# defence the other builders rely on does not apply here: a ';' would append a +# second bound and a ')' would close the interval early. Validating the shape is +# therefore the guard, not escaping. +# +# This regex is the ADMISSION gate — which strings a caller may hand in — and it +# is deliberately WIDER than the set the wire honours, because +# :func:`_interval_bound` re-renders every admitted time through +# :func:`~easyvista_python_client.format_ev_datetime` before emitting it. +# Measured live 2026-08-18, a *time* bound is honoured only at millisecond +# precision with an explicit offset (or ``Z``); second precision with an offset +# (``2025-11-28T16:14:41+01:00``), minute precision, and a space instead of +# ``T`` are all HTTP 590. A bare date is honoured as written. Admitting the +# wider set and normalising is what lets a caller pass a stored watermark string +# at all instead of having to pre-render it in exactly one shape. +# +# This is a SHAPE gate only, not a validity gate: `[0-9]` (not `\d`, which is +# Unicode-aware) keeps non-ASCII digits out, and `fullmatch` anchors both ends +# unconditionally rather than relying on a trailing `$` — which would also +# match just before a newline — so the guard does not silently depend on the +# `.strip()` above having already removed one. Calendar/time validity (e.g. +# ``9999-99-99``, ``25:61:61``) is not this regex's job; :func:`_interval_bound` +# checks that separately via :func:`~easyvista_python_client.parse_ev_datetime`. +# ``z`` is accepted lowercase because ``parse_ev_datetime`` accepts it on the +# read path; refusing it here would reject a value this package itself produced. +_TIMESTAMP_RE = re.compile( + r"[0-9]{4}-[0-9]{2}-[0-9]{2}" # YYYY-MM-DD + r"(?:[T ][0-9]{2}:[0-9]{2}:[0-9]{2}" # optional T HH:MM:SS + r"(?:\.[0-9]{1,6})?" # optional fractional seconds + r"(?:[Zz]|[+-][0-9]{2}:[0-9]{2})?)?" # optional offset +) + +# A bare calendar date, which is passed through UNCHANGED rather than +# normalised: it has day granularity, the API honours it as written, and +# re-rendering it would invent a midnight instant in some zone. +_DATE_ONLY_RE = re.compile(r"[0-9]{4}-[0-9]{2}-[0-9]{2}") + +# A datetime carrying NO ``Z`` and no ``+-HH:MM``. The wire accepts these, and +# reads them in another zone -- measured live 2026-08-18, the same wall-clock +# text with and without its offset enumerated 13 rows and 11 rows against one +# instance, the offset-less form moving the bound *later* and skipping records +# with no error. A bare date deliberately does NOT match: it has day +# granularity and no time to misplace, and round 1 measured it honoured. +_OFFSETLESS_TIME_RE = re.compile( + r"[0-9]{4}-[0-9]{2}-[0-9]{2}[T ][0-9]{2}:[0-9]{2}:[0-9]{2}(?:\.[0-9]{1,6})?" +) + +# A time whose UTC offset carries SECONDS (``+05:53:20``), which +# :data:`_TIMESTAMP_RE` refuses. Matched separately only to DIAGNOSE it: the +# value is perfectly good ISO 8601 -- it is what ``isoformat()`` produces for +# any pre-1900 ``zoneinfo`` instant -- so the generic "not an EasyVista +# timestamp ... pass a datetime to be certain" message would be wrong twice +# over, since parsing it back to a datetime and passing that raises for the +# same underlying reason (see :func:`_render_interval_bound`). +_SUB_MINUTE_OFFSET_RE = re.compile( + r"[0-9]{4}-[0-9]{2}-[0-9]{2}[T ][0-9]{2}:[0-9]{2}:[0-9]{2}" + r"(?:\.[0-9]{1,6})?[+-][0-9]{2}:[0-9]{2}:[0-9]{2}" +) + + +def _render_interval_bound(moment: datetime) -> str: + """Render one time bound in the single form measured live as honoured. + + :func:`~easyvista_python_client.format_ev_datetime` emits millisecond + precision with an explicit offset, which is the ONLY time rendering the + interval grammar accepts: second precision with an offset, minute precision + and a space separator instead of ``T`` are each HTTP 590 (measured live + 2026-08-18). + + The rendering is re-checked against :data:`_TIMESTAMP_RE` rather than + trusted, because a zone whose UTC offset is not a whole number of minutes -- + every pre-1900 ``zoneinfo`` entry has one, e.g. ``Asia/Kolkata`` at + ``+05:53:20`` -- renders as ``+05:53:20``, which the string path refuses and + which the wire has no reason to honour either. Without this check the + datetime path could emit a bound the string path would reject, which is the + asymmetry this function exists to remove. + """ + rendered = format_ev_datetime(moment) + if not _TIMESTAMP_RE.fullmatch(rendered): + raise ValueError( + f"{moment!r} renders as {rendered!r}, which EasyVista's interval " + "grammar cannot express: its UTC offset is not a whole number of " + "minutes (historical zoneinfo zones carry such offsets). Convert " + "the datetime to UTC, or to a zone with a whole-minute offset." + ) + return rendered + + +def _interval_bound(value: str | datetime | None) -> str: + """Render one interval bound, or ``""`` for an open end. + + Raises ``ValueError`` for anything that is not a timestamp, because the + bound is interpolated unquoted (see :data:`_TIMESTAMP_RE`). The regex is a + shape gate; :func:`~easyvista_python_client.parse_ev_datetime` is the + validity gate behind it, so a well-shaped but impossible timestamp (e.g. + ``9999-99-99`` or ``25:61:61``) is also refused rather than reaching the + wire, where a dropped condition returns the whole table rather than an + error. + + A third gate refuses a **time without a UTC offset**, even though the wire + accepts one. An offset-less literal is read in another zone, which moves the + bound and skips records silently -- the one failure mode a watermark must + never have. :func:`~easyvista_python_client.format_ev_datetime` already + refuses a naive ``datetime`` on this reasoning; this keeps the string path + consistent with it. A bare date stays legal. + + Past those gates an admitted **time** is NORMALISED rather than passed + through: it is re-rendered by :func:`_render_interval_bound`, so the string + and datetime paths emit byte-identical bounds and both emit the one + rendering measured live as honoured. This is not cosmetic -- + ``"2025-11-28T16:14:41+01:00"``, the most natural way to comply with the + offset gate, is HTTP 590 on the wire as written and becomes + ``2025-11-28T16:14:41.000+01:00`` here. Sub-millisecond precision is + truncated to milliseconds, EasyVista's own precision -- truncated **down**, + never rounded (``.133999`` becomes ``.133``). That direction is not + symmetric between the two ends of an interval: on the **inclusive lower** + bound it widens the window, so the worst case is re-reading a record, while + on an **upper** bound it moves the bound up to 999 microseconds *earlier* + and NARROWS the window, excluding anything stamped inside the truncated + remainder. A caller who needs an exact upper bound should hand in a value + already at millisecond precision. A bare **date** is passed through + unchanged (see :data:`_DATE_ONLY_RE`). + """ + if value is None: + return "" + if isinstance(value, datetime): + return _render_interval_bound(value) + text = str(value).strip() + if not text: + return "" + parsed = parse_ev_datetime(text) if _TIMESTAMP_RE.fullmatch(text) else None + if parsed is None: + if _SUB_MINUTE_OFFSET_RE.fullmatch(text): + raise ValueError( + f"{value!r} carries a UTC offset that is not a whole number of " + "minutes, which EasyVista's interval grammar cannot express -- " + "the value itself is valid ISO 8601 (every pre-1900 zoneinfo " + "zone renders like this). Convert it to UTC, or to a zone with " + "a whole-minute offset. Parsing it back to a datetime and " + "passing that does NOT help: the datetime path refuses the same " + "offset for the same reason." + ) + raise ValueError( + f"{value!r} is not an EasyVista timestamp. An interval bound is " + "interpolated unquoted, so only a date or an ISO-8601 timestamp is " + "accepted; pass a datetime to be certain." + ) + if _OFFSETLESS_TIME_RE.fullmatch(text): + raise ValueError( + f"{value!r} names a time with no UTC offset, which EasyVista reads " + "in another zone: that silently moves the bound and skips records " + "with no error at all. Append the offset (or 'Z'), or pass an aware " + "datetime and let format_ev_datetime render it. A date alone is " + "accepted -- it has no time to misplace." + ) + if _DATE_ONLY_RE.fullmatch(text): + return text + return _render_interval_bound(parsed) + + +def ev_since_filter(field: str, start: str | datetime | None) -> str | None: + """Build an open-ended lower bound: ``FIELD:(start;)``. + + This is the change-window filter. EasyVista has **no** comparison operator — + ``>=``, ``>``, ``BETWEEN``, ``[a TO b]`` and ``a..b`` are all silently + dropped, which returns the whole table (256 live trials, zero honoured). + A range is instead an *interval in the value position*, and the open-ended + form is exactly a watermark:: + + search = ev_since_filter("LAST_UPDATE", watermark) + if search is not None: + seen = set() + for ticket in client.iter_tickets( + search=search, sort="LAST_UPDATE DESC" + ): + if ticket.rfc_number in seen: + continue + seen.add(ticket.rfc_number) + ... + + **The sort is load-bearing, and its DIRECTION decides whether a dropped row + ever comes back.** ``iter_tickets`` walks the result set by offset, and the + rows this filter selects are by construction the rows that are changing, so + a row touched between page N and page N+1 moves *within the very set being + paged*. Either direction can drop a row from the current sweep; what differs + is where the dropped row's own timestamp ends up relative to the watermark + this sweep will record: + + * **Descending** (``LAST_UPDATE DESC``) — the re-touched row jumps to the + *head*, behind the read cursor, so this sweep misses it. But its + ``LAST_UPDATE`` is now *above* the watermark, so the next sweep selects it + again: the miss is **deferred and self-healing**. + * **Ascending** (bare ``LAST_UPDATE``, or ``LAST_UPDATE ASC``) — the + re-touched row moves to the tail, and every row between its old place and + the tail shifts one position head-ward. The row that crosses the cursor is + therefore one whose own stamp did **not** change; it falls *below* the new + watermark, so no later sweep selects it either. The miss is **permanent**. + + So sweep **descending** and de-duplicate by ``rfc_number`` as above. Both + directions are honoured tokens (measured live); the direction is chosen for + this reason, not for availability. An earlier docstring in this package + recommended ascending, on the reasoning that it turns a permanent miss into a + duplicate — that was wrong: the row an ascending sweep drops is not the + re-touched one. + + **A sweep that does not run to completion is a separate trap under this + sort.** Because ``DESC`` yields the newest row first, the watermark reaches + its *final* value on page 1. A sweep interrupted partway through, or capped + with ``max_records``, still ends up holding the newest stamp — so advancing + the watermark from it makes the next window's ``(newest;)`` bound + permanently exclude every row the incomplete sweep never reached. Advance the + watermark only after a sweep runs to completion. + + Descending is the safe direction, not a guarantee: ``iter_tickets`` owns its + offset. A caller who cannot tolerate even a deferred miss should page + :meth:`~easyvista_python_client.EasyvistaClient.search_tickets` directly + with **keyset** pagination: sort ascending and, after each page, advance the + *window* — ``ev_since_filter(field, max(stamps on the page))`` read again at + ``offset=0`` — instead of incrementing an offset. With no offset there is no + cursor for a row to shift past. ``iter_tickets`` cannot express this. + + The lower bound is **INCLUSIVE** and milliseconds are honoured (verified + live on three independent boundaries), so a watermark taken as + ``max(t.last_update)`` re-reads that boundary record on the next sweep — + another duplicate the same de-duplication absorbs. + + ``start`` may be a ``datetime`` (the preferred input) or a timestamp + string; a string naming a time is re-rendered to the one form the wire + honours (see :func:`_interval_bound`). Blank or ``None`` returns ``None``, + matching the other builders so callers compose without conditionals. + """ + bound = _interval_bound(start) + if not bound: + return None + return f"{field}:({bound};)" + + +def ev_between_filter( + field: str, start: str | datetime | None, end: str | datetime | None +) -> str | None: + """Build a closed interval: ``FIELD:(start;end)``. + + Either bound may be omitted for a half-open interval. With both omitted the + result is ``None`` rather than ``FIELD:(;)``, which would match everything. + + Note ``,`` is **not** the separator — ``FIELD:(a,b)`` raises HTTP 590 live. + + Both bounds are normalised exactly as :func:`ev_since_filter`'s is (see + :func:`_interval_bound`): a bare date passes through, and a value naming a + time is re-rendered at millisecond precision with an explicit offset. The + sub-millisecond truncation that involves runs **downward**, which is the + safe direction for the lower bound but not for the upper one — + ``"...41.133999Z"`` as ``end`` becomes ``"...41.133Z"``, up to 999 + microseconds early. Pass an ``end`` already at millisecond precision when + the exact instant matters. (EasyVista itself returns millisecond precision, + so a bound read back from a record cannot fall inside that remainder. The + *upper* bound's inclusivity is unmeasured — only the lower bound's was + verified live.) + """ + low, high = _interval_bound(start), _interval_bound(end) + if not low and not high: + return None + return f"{field}:({low};{high})" + + +# Every character that is a metacharacter to `~`, measured live 2026-08-18 +# against one instance: +# * % multi-character wildcards, interchangeable +# _ SINGLE-character wildcard -- replacing one character of an RFC that +# matched 1 row turned it into 9 +# [ opens a character class -- `[0-9]` in the same position also gave 9, +# and `[x]` gave 1, so the class is genuinely evaluated +# There is no escape: `\_` returned 0 rows, i.e. the backslash is compared +# literally. So these builders refuse rather than silently changing which +# records match -- and `_` is not exotic in EasyVista, it is pervasive in asset +# tags, catalog codes and `e_*` column values. +_PATTERN_METACHARS = ("*", "%", "_", "[") + + +def _wildcard_filter(field: str, value: str | None, pattern: str) -> str | None: + """Shared body for the ``~`` pattern builders. + + ``pattern`` is a format string over ``{v}`` placing the wildcards. + """ + if value is None: + return None + text = str(value).strip() + if not text: + # A blank value would render `FIELD~"**"`, which matches every row — + # the silent-widening failure these builders exist to prevent. + return None + if any(char in text for char in _PATTERN_METACHARS): + raise ValueError( + f"{value!r} contains a pattern metacharacter (one of * % _ [). " + "These builders add the wildcards themselves; a metacharacter " + "inside the value would change which records match rather than " + "being compared literally, and EasyVista provides no escape for it " + "-- a backslash is taken literally (verified live). For an EXACT " + "match on a value containing one, use ev_equals_filter: ':' does " + "not expand a wildcard. To pattern-match around one, filter " + "server-side on a wider condition and compare exactly in Python." + ) + return f'{field}~"{pattern.format(v=escape_ev_value(text))}"' + + +def ev_contains_filter(field: str, value: str | None) -> str | None: + """Build a substring match: ``FIELD~"*value*"``. + + ``~`` **is** a pattern operator, and it needs an explicit wildcard — verified + live: ``RFC_NUMBER~"*260817*"`` matched 33 rows while + ``RFC_NUMBER:"I26081*"`` matched 0, because ``:`` never expands wildcards. + Without a wildcard, ``~`` degenerates to exact match, which is why this + package previously documented it as "exact-match, not contains" — that + conclusion held only for the inputs it was tested with. + + A value containing ``*``, ``%``, ``_`` or ``[`` raises ``ValueError``: all + four are metacharacters to ``~`` (``_`` matches any single character, ``[`` + opens a character class), and no escape for them exists. Refusing beats + silently matching records the caller did not ask for — + ``ev_contains_filter("ASSET_TAG", "LAPTOP_01")`` would otherwise also match + ``LAPTOP-01`` and ``LAPTOP001`` with HTTP 200 and no hint. For an **exact** + match on such a value use :func:`ev_equals_filter`, whose ``:`` does not + expand a wildcard; only pattern-matching *around* a literal metacharacter is + impossible, and that needs a wider server-side condition plus an exact + comparison in Python. + """ + return _wildcard_filter(field, value, "*{v}*") + + +def ev_starts_with_filter(field: str, value: str | None) -> str | None: + """Build a prefix match: ``FIELD~"value*"`` (verified live: 32 rows). + + Refuses the same four metacharacters in ``value`` as + :func:`ev_contains_filter`, for the same reason and with the same exit. + """ + return _wildcard_filter(field, value, "{v}*") + + __all__ = [ "escape_ev_value", + "ev_between_filter", + "ev_contains_filter", "ev_equals_filter", "ev_in_filter", + "ev_since_filter", + "ev_starts_with_filter", "is_safe_ev_value", ] diff --git a/easyvista_python_client/models/action.py b/easyvista_python_client/models/action.py index 142c2a3..b0e16a4 100644 --- a/easyvista_python_client/models/action.py +++ b/easyvista_python_client/models/action.py @@ -6,7 +6,7 @@ from pydantic import Field, model_validator -from .common import EasyvistaModel, EasyvistaWriteModel, OptionalInt +from .common import EasyvistaModel, EasyvistaWriteModel, OptionalDateTime, OptionalInt class Action(EasyvistaModel): @@ -18,6 +18,34 @@ class Action(EasyvistaModel): caller supplied as ``PostAction.description`` comes back through ``DESCRIPTION`` — **not** ``COMMENT``, and not on the list endpoint at all (verified live). ``extra="allow"`` preserves everything else. + + Item-level reads additionally carry timestamps (``CREATION_DATE_UT``, + ``LAST_UPDATE``), the author (``DONE_BY_ID`` plus a nested ``DONE_BY`` + employee object) and workflow context (``STAGE_ID``, ``WORKFLOW_ID``) — + verified live 2026-08-17. Their availability on the LIST endpoint is not + uniform: ``CREATION_DATE_UT``, ``LAST_UPDATE``, ``GROUP_ID``, ``STAGE_ID``, + ``WORKFLOW_ID`` and ``PARENT_ACTION_ID`` are genuinely absent from the + default list projection; ``DONE_BY_ID`` and ``ACTION_NUMBER`` are already + present there as top-level scalars; ``ACTION_TYPE_ID`` and ``REQUEST_ID`` + are present on a list row too, but only *nested* (inside ``ACTION_TYPE`` / + ``REQUEST``) — since the declared fields alias the top-level key, + ``action_type_id``/``request_id`` read ``None`` off a default list row + even though the API did return the data. Pass ``fields=`` to + ``list_actions`` to get these top-level for a whole PAGE of actions in one + request instead of an item fetch per action — ``list_actions`` returns one + page and does not paginate, so it is a page's worth, not a ticket's. + + Naming: this model calls its two timestamps ``created_at``/``updated_at`` + where :class:`~easyvista_python_client.models.request.Request` and + :class:`~easyvista_python_client.models.employee.Employee` mirror the wire + and call the identical columns ``creation_date_ut``/``last_update``. The + divergence is only in the Python surface; the aliases + (``CREATION_DATE_UT``/``LAST_UPDATE``) are the same on all three. Code that + spans record types should therefore reach for the wire name via + :meth:`~easyvista_python_client.models.common.EasyvistaModel.classify_fields` + or ``.reference()`` rather than a shared attribute name, because + ``getattr(record, "last_update")`` raises ``AttributeError`` on an + ``Action``. """ action_id: OptionalInt = Field(default=None, alias="ACTION_ID") @@ -28,6 +56,39 @@ class Action(EasyvistaModel): # The live API returns ACTION_TYPE as a nested object (id/name/...), not a # bare string, so accept either (same polymorphism as Request.description). action_type: str | dict[str, Any] | None = Field(default=None, alias="ACTION_TYPE") + # --- item-level fields (EV-R1, verified live 2026-08-17) ------------------ + # All ten are present on ``GET actions/{id}``. On the LIST endpoint: + # CREATION_DATE_UT, LAST_UPDATE, GROUP_ID, STAGE_ID, WORKFLOW_ID and + # PARENT_ACTION_ID are genuinely ABSENT from the default projection (the + # default list row carries only ACTION_ID, ACTION_LABEL_FR, ACTION_NUMBER, + # DONE_BY_ID and EXPECTED_START_DATE_UT) -- use a ``fields=`` projection + # (see ``list_actions(fields=...)``) or an item fetch to get them. + # ACTION_TYPE_ID and REQUEST_ID are NOT list-absent -- the default list row + # already returns them, nested inside ACTION_TYPE / REQUEST respectively -- + # but because these fields alias the top-level key, they still read + # ``None`` off a default list row; a ``fields=`` projection or the item GET + # returns them top-level instead. + # + # Named ``created_at``/``updated_at`` rather than mirroring the API's + # ``CREATION_DATE_UT``/``LAST_UPDATE`` because these are the two timestamps + # a caller reaches for; the aliases keep the wire names authoritative. + created_at: OptionalDateTime = Field(default=None, alias="CREATION_DATE_UT") + updated_at: OptionalDateTime = Field(default=None, alias="LAST_UPDATE") + done_by_id: OptionalInt = Field(default=None, alias="DONE_BY_ID") + action_type_id: OptionalInt = Field(default=None, alias="ACTION_TYPE_ID") + group_id: OptionalInt = Field(default=None, alias="GROUP_ID") + request_id: OptionalInt = Field(default=None, alias="REQUEST_ID") + action_number: OptionalInt = Field(default=None, alias="ACTION_NUMBER") + # Workflow context. A freshly created ticket auto-spawns ~12 actions from the + # catalog's workflow -- on one live ticket, only ONE of the twelve was + # human-authored; the rest were the workflow's own generated steps. Those + # generated actions carry these fields, an EMPTY ``DONE_BY_ID``, and also a + # ``STATUS_ID_ON_CREATE`` (deliberately not declared here) -- together how a + # caller tells a generated step from a human note. Filter on + # ``action_type_id`` — the comment-like type ids are per-deployment config. + stage_id: OptionalInt = Field(default=None, alias="STAGE_ID") + workflow_id: OptionalInt = Field(default=None, alias="WORKFLOW_ID") + parent_action_id: OptionalInt = Field(default=None, alias="PARENT_ACTION_ID") @model_validator(mode="after") def _derive_action_id_from_href(self) -> Action: @@ -63,3 +124,22 @@ class PostAction(EasyvistaWriteModel): group_id: int | None = None group_name: str | None = None description: str | None = None + + +class ActionUpdate(EasyvistaWriteModel): + """Payload for editing an existing action's note. + + ``PUT actions/{id}`` is live-verified (2026-08-17): writing the action's + ``DESCRIPTION`` memo really changed it, confirmed by re-reading it rather + than by trusting HTTP 200. The **nested** + ``PUT requests/{rfc}/actions/{id}`` returns 403, as does + ``DELETE actions/{id}`` — an action can be edited but not deleted. + + ``description`` is the note text. On the verified instance an action's text + lives in the ``DESCRIPTION`` memo and ``COMMENT`` is empty, mirroring how + ``PostAction.description`` round-trips; ``comment`` is offered for a + deployment configured the other way round, and is **not** live-verified. + """ + + description: str | None = None + comment: str | None = None diff --git a/easyvista_python_client/models/common.py b/easyvista_python_client/models/common.py index b0529b8..93a1f6d 100644 --- a/easyvista_python_client/models/common.py +++ b/easyvista_python_client/models/common.py @@ -2,12 +2,14 @@ from __future__ import annotations +from datetime import datetime from typing import Annotated, Any from pydantic import BaseModel, BeforeValidator, ConfigDict, Field from ..field_model import FieldClassification, classify from ..references import Reference, resolve_reference +from ..timestamps import parse_ev_datetime def _empty_str_to_none(value: Any) -> Any: @@ -27,6 +29,60 @@ def _empty_str_to_none(value: Any) -> Any: """An ``int | None`` field that treats the API's ``""`` sentinel as ``None``.""" +def _empty_str_to_none_datetime(value: Any) -> Any: + """Coerce EasyVista's ``""`` sentinel for an absent date to ``None``. + + Distinct from :func:`_empty_str_to_none`: a *malformed* timestamp must still + raise, so this maps only the two documented absences -- ``None`` (a JSON + ``null``, and the field's own default) and EasyVista's ``""`` sentinel. Every + other value -- including a ``datetime`` handed in directly, not just a string -- + routes through :func:`~easyvista_python_client.timestamps.parse_ev_datetime`, + which normalizes a naive ``datetime`` to UTC and returns ``None`` for + anything it cannot parse. + + When it returns ``None`` this raises ``ValueError``, which pydantic wraps + into a ``ValidationError`` naming the field. It deliberately does **not** + fall through to pydantic's own datetime parser, which is far more permissive + than EasyVista's format and would invent a plausible-but-wrong instant + instead of reporting the mismatch: ``"20260817"`` (ISO-basic, no separators) + becomes ``1970-08-23T12:00:17Z``, 56 years off, and ``1755434441610`` -- + what an epoch-millis format change would look like -- becomes a wholly + credible ``2025-08-17T12:40:41.610Z``. Absorbing the one format change this + guard exists to surface is the opposite of the intended behaviour, so junk + raises here instead. + """ + if value is None: + return None + if isinstance(value, str) and not value.strip(): + return None + parsed = parse_ev_datetime(value) + if parsed is None: + raise ValueError( + f"{value!r} is not an EasyVista timestamp. EasyVista sends ISO 8601 " + "with an explicit UTC offset (or '' for an unset date); anything " + "else is reported rather than guessed at, because pydantic's own " + "parser would turn a numeric or ISO-basic value into a plausible " + "but wrong instant and hide the format change." + ) + return parsed + + +OptionalDateTime = Annotated[ + datetime | None, BeforeValidator(_empty_str_to_none_datetime) +] +"""An aware ``datetime | None`` for an EasyVista timestamp column. + +EasyVista returns ISO 8601 with an explicit UTC offset and millisecond +precision (``2026-08-17T15:40:41.610+02:00``), and ``""`` for an unset date — +verified live 2026-08-17. Python 3.10's ``fromisoformat`` rejects the 3-digit +fraction outright, which is why this goes through +:func:`~easyvista_python_client.parse_ev_datetime` rather than letting pydantic +parse the string itself. A naive ``datetime`` passed in directly (not just a +wire string) is normalized to aware UTC the same way, so the ``| None`` aside, +this type's value is always timezone-aware, never naive. +""" + + class EasyvistaModel(BaseModel): """Base for read models. diff --git a/easyvista_python_client/models/employee.py b/easyvista_python_client/models/employee.py index 19f17c8..40eb2fe 100644 --- a/easyvista_python_client/models/employee.py +++ b/easyvista_python_client/models/employee.py @@ -12,7 +12,7 @@ from pydantic import Field -from .common import EasyvistaModel, EasyvistaWriteModel, OptionalInt +from .common import EasyvistaModel, EasyvistaWriteModel, OptionalDateTime, OptionalInt class Employee(EasyvistaModel): @@ -33,7 +33,7 @@ class Employee(EasyvistaModel): login: str | None = Field(default=None, alias="LOGIN") function_id: OptionalInt = Field(default=None, alias="FUNCTION_ID") language_id: OptionalInt = Field(default=None, alias="LANGUAGE_ID") - last_update: str | None = Field(default=None, alias="LAST_UPDATE") + last_update: OptionalDateTime = Field(default=None, alias="LAST_UPDATE") href: str | None = Field(default=None, alias="HREF") diff --git a/easyvista_python_client/models/request.py b/easyvista_python_client/models/request.py index a2dbf4e..1d6506d 100644 --- a/easyvista_python_client/models/request.py +++ b/easyvista_python_client/models/request.py @@ -13,7 +13,7 @@ from pydantic import Field, model_validator -from .common import EasyvistaModel, EasyvistaWriteModel, OptionalInt +from .common import EasyvistaModel, EasyvistaWriteModel, OptionalDateTime, OptionalInt class Request(EasyvistaModel): @@ -71,23 +71,27 @@ class Request(EasyvistaModel): recipient_id: OptionalInt = Field(default=None, alias="RECIPIENT_ID") owner_id: OptionalInt = Field(default=None, alias="OWNER_ID") - # timestamps and time limits — verified *returned*; their accepted write - # format is NOT verified (both a string and an int probe return HTTP 590), - # so no datetime parsing is claimed here. See spec open item O-590-DATE. + # timestamps — ISO 8601 with an EXPLICIT UTC OFFSET and millisecond + # precision, verified live 2026-08-17 against our own UTC clock, so these + # parse to aware datetimes. Their accepted WRITE format is still NOT + # verified (both a string and an int probe return HTTP 590), which is why + # no write model carries a datetime. An unset date is ``""``, handled by + # OptionalDateTime. Note ``_UT`` does NOT mean UTC-normalized: those columns + # carry the same local offset as LAST_UPDATE. # # These are the OFFICIAL time fields, portable across EasyVista # deployments. The instance-specific GTR/GTI family (``E_GTR_STATUS``, # ``E_GTI_UT``, ``E_DELAI_PEC``…) is deliberately NOT declared: it does not # exist on another deployment, so it belongs in the custom bucket of # :meth:`classify_fields`, reached by name at the call site. - submit_date_ut: str | None = Field(default=None, alias="SUBMIT_DATE_UT") - creation_date_ut: str | None = Field(default=None, alias="CREATION_DATE_UT") - max_resolution_date_ut: str | None = Field( + submit_date_ut: OptionalDateTime = Field(default=None, alias="SUBMIT_DATE_UT") + creation_date_ut: OptionalDateTime = Field(default=None, alias="CREATION_DATE_UT") + max_resolution_date_ut: OptionalDateTime = Field( default=None, alias="MAX_RESOLUTION_DATE_UT" ) - expected_date_ut: str | None = Field(default=None, alias="EXPECTED_DATE_UT") - end_date_ut: str | None = Field(default=None, alias="END_DATE_UT") - last_update: str | None = Field(default=None, alias="LAST_UPDATE") + expected_date_ut: OptionalDateTime = Field(default=None, alias="EXPECTED_DATE_UT") + end_date_ut: OptionalDateTime = Field(default=None, alias="END_DATE_UT") + last_update: OptionalDateTime = Field(default=None, alias="LAST_UPDATE") sla_id: OptionalInt = Field(default=None, alias="SLA_ID") # Verified live (2026-07-28 Phase 0 probe, U6) as a string on every ticket # checked -- never an int -- so no int branch is declared here. @@ -115,13 +119,38 @@ def _derive_rfc_from_href(self) -> Request: class PostRequest(EasyvistaWriteModel): """Payload for creating a ticket. - Field set matches the documented create body (``docs/API_Info.md``), - verified against a live instance: a ticket needs at minimum ``catalog_code`` - plus ``title`` (and typically ``origin`` / ``department_id``). The exact - mandatory fields are configured **per catalog on the EasyVista side**, so the - client cannot know them statically; a missing one is rejected server-side and - surfaces as :class:`EasyvistaValidationError` (HTTP 590, code 2013), not a - retried server error. ``custom_fields`` values are serialized with an ``e_`` + Field set matches the documented create body (``docs/API_Info.md``). **Send + the whole documented set** -- ``catalog_code``, ``origin``, ``title``, + ``description``, ``department_id``, ``urgency_id``, ``impact_id`` -- rather + than a subset. An earlier version of this docstring claimed a ticket needs + "at minimum ``catalog_code`` plus ``title``"; that is wrong, and the way it + is wrong is expensive: + + * the full documented body creates successfully on every catalog tried; + * the same body minus the four ids creates on some catalogs and is rejected + on others -- measured on two catalogs of one instance with the *identical* + remaining bytes, accepted by one and rejected by the other; + * the rejection is HTTP 590 / code 2013 whose message is a **SQL parser + error** (``=(1,35) expected token:( * + - . IDENTIFIER CASE NOT JOIN ...``) + naming no field at all. It is easy to misread as a server-side defect. It + is not: it is what an under-specified create body looks like here. + + Which fields a given catalog can do without is configured per catalog on the + EasyVista side, so the client cannot know it statically -- which is exactly + why sending the documented set is the only reliable shape. + + Ids may be sent as JSON numbers or as strings; both are accepted (measured + side by side). The documented examples quote them; these fields are typed + ``int`` here and serialize as numbers, which the API takes. + + **A rejected create may still have created the ticket.** Measured: 12 + attempts returned 3 ``RFC_NUMBER``s and afterwards all 12 tickets existed -- + 9 of 9 failures had written a row, with the ids they were missing left null. + So a 590 here means *possibly created*, never *not created*: retrying + duplicates, and the caller never learns the id. Reconcile by + ``external_reference`` rather than trusting the error. + + ``custom_fields`` values are serialized with an ``e_`` prefix unless they already start with ``e_`` (see :class:`EasyvistaWriteModel`). ``catalog_code`` is the only verified way to name a catalog here. An earlier @@ -153,20 +182,61 @@ class RequestUpdate(EasyvistaWriteModel): ``docs/API_Info.md`` documents only the create, comment and close bodies, so the update body is not vendor-documented. Every field here is one verified - accepted against a live instance -- ``title`` by the Phase 0 probe and by - ``integration_tests/test_live_ticket_identity.py``. Nothing is added - speculatively: an unaccepted field would silently no-op or raise HTTP 590. + accepted against a live instance **by re-reading the ticket afterwards**, not + by trusting HTTP 200 — that distinction matters on this API, where a write + can return 200 and change nothing. ``description`` writes the ticket's **COMMENT** Memo, not ``DESCRIPTION`` -- - verified live. EasyVista models ``COMMENT`` as the request's justification - and ``DESCRIPTION`` as a separate Memo; which one a deployment actually - populates is a per-instance configuration choice. On the instance this - client was verified against, ``DESCRIPTION`` is empty on every ticket and - ``COMMENT`` carries the body text. Read it back with - ``resolve_memo("requests/{rfc}/comment")``, or take - ``TicketContext.comment``, which resolves it for you. + verified live by reading the text back, and pinned by + ``integration_tests/test_live_ticket_metadata.py``. + + EasyVista models ``COMMENT`` as the request's justification and + ``DESCRIPTION`` as a separate Memo. Which one a deployment populates is a + per-instance configuration choice, and it is **not** reliably detectable at + runtime. A pooled 77-row sample of one instance across four different + orderings found ``COMMENT`` populated on 57 rows, ``DESCRIPTION`` on 27, + *both* on 24 and neither on 17 -- and the proportions flipped depending on + which slice was sampled, so a majority vote over a sample answers whichever + way the sort happened to fall (measured 2026-08-18). An earlier 15-ticket + sample that found ``DESCRIPTION`` empty everywhere was not representative; + do not rely on that being true of any instance. Treat the body memo as + operator configuration, not as something to infer. + + Read ``COMMENT`` back with ``resolve_memo("requests/{rfc}/comment")``, or + take ``TicketContext.comment``, which resolves it for you. + + **Deliberately absent** (verified 2026-08-17): + + * ``status_id`` — there is **no flat status update on this API**. It was a + field here until 2026-08-25 and it never worked: sent alone the PUT is + rejected 590/2013, and sent beside any other field the PUT returns **200, + applies the other field, and drops the status in silence** (measured: + title updated, ``STATUS_ID`` unchanged). A write that reports success and + stores nothing is worse than one that fails, so the field is gone and + ``extra="forbid"`` now makes ``RequestUpdate(status_id=...)`` raise at + construction instead. + + Set a status with :meth:`~easyvista_python_client.EasyvistaClient.set_status`, + which sends the documented ``{"closed": {"status_GUID": ...}}`` body. That + route reaches **every** status, not just terminal ones -- all six statuses + tried landed on exactly the one requested. It is addressed by + ``STATUS_GUID``, not by ``STATUS_ID``. + * ``severity_id`` — ``SEVERITY_ID`` is rejected with HTTP 590 (code 2013). + * ``urgency_id`` — ``URGENCY_ID`` raised HTTP 590 *and the value still + changed*, so the API's behaviour is not one this model can express + honestly. Set it with a raw request and re-read if you must. Note this is + an **update**-path finding only: on the CREATE path ``urgency_id`` is part + of the documented body and lands cleanly (see :class:`PostRequest`). + * a priority field — EasyVista derives priority from urgency x impact rather + than exposing a writable column. + + ``external_reference`` is capped at 50 characters: 50 is accepted and 51 is + rejected server-side (bisected live). The cap is enforced here so the round + trip is saved; over-length is rejected rather than truncated either way. """ - status_id: int | None = None title: str | None = None description: str | None = None + impact_id: int | None = None + owner_id: int | None = None + external_reference: str | None = Field(default=None, max_length=50) diff --git a/easyvista_python_client/models/tests/test_action.py b/easyvista_python_client/models/tests/test_action.py index 767c32f..6b96573 100644 --- a/easyvista_python_client/models/tests/test_action.py +++ b/easyvista_python_client/models/tests/test_action.py @@ -1,5 +1,79 @@ +from datetime import datetime, timedelta, timezone + +import pytest + from easyvista_python_client.models.action import Action, PostAction +_CEST = timezone(timedelta(hours=2)) + +# Trimmed from a real item-level GET (see the spec's Appendix A-2); values are +# synthetic, the KEY NAMES are what this test pins. +_ITEM_PAYLOAD = { + "ACTION_ID": "57483", + "ACTION_NUMBER": "0", + "ACTION_TYPE_ID": "20", + "CREATION_DATE_UT": "2026-08-17T15:40:36.000+02:00", + "LAST_UPDATE": "2026-08-17T15:40:37.653+02:00", + "DONE_BY_ID": "6117", + "GROUP_ID": "57", + "REQUEST_ID": "7743", + "STAGE_ID": "10", + "WORKFLOW_ID": "37", + "PARENT_ACTION_ID": "", + "DONE_BY": {"EMPLOYEE_ID": "6117", "LAST_NAME": "Doe"}, + "DESCRIPTION": {"HREF": "https://ev.test/api/v1/12345/actions/57483/description"}, +} + + +def test_item_level_action_exposes_timestamps_and_author(): + """EV-R1: the fields a Comment model needs all exist on the item GET.""" + action = Action.model_validate(_ITEM_PAYLOAD) + assert action.created_at == datetime(2026, 8, 17, 15, 40, 36, tzinfo=_CEST) + assert action.updated_at == datetime(2026, 8, 17, 15, 40, 37, 653000, tzinfo=_CEST) + assert action.done_by_id == 6117 + assert action.action_type_id == 20 + assert action.group_id == 57 + assert action.request_id == 7743 + + +def test_workflow_context_is_declared_so_generated_actions_are_identifiable(): + """A fresh ticket auto-spawns ~12 workflow actions; these tell them apart.""" + action = Action.model_validate(_ITEM_PAYLOAD) + assert action.stage_id == 10 + assert action.workflow_id == 37 + assert action.parent_action_id is None # "" sentinel -> None + + +@pytest.mark.parametrize( + ("alias", "attr"), + [ + ("DONE_BY_ID", "done_by_id"), + ("ACTION_TYPE_ID", "action_type_id"), + ("GROUP_ID", "group_id"), + ("REQUEST_ID", "request_id"), + ("ACTION_NUMBER", "action_number"), + ("STAGE_ID", "stage_id"), + ("WORKFLOW_ID", "workflow_id"), + ("PARENT_ACTION_ID", "parent_action_id"), + ], +) +def test_the_empty_string_sentinel_maps_to_none_on_every_new_int_field(alias, attr): + """Workflow-generated actions have an EMPTY DONE_BY_ID (measured live).""" + action = Action.model_validate({"ACTION_ID": "1", alias: ""}) + assert getattr(action, attr) is None + + +def test_absent_timestamps_are_none_not_an_error(): + """The list projection omits both date fields entirely.""" + action = Action.model_validate({"ACTION_ID": "1"}) + assert action.created_at is None + assert action.updated_at is None + + +def test_done_by_reference_resolves_through_the_shared_resolver(): + action = Action.model_validate(_ITEM_PAYLOAD) + assert action.reference("DONE_BY").id == "6117" + def test_action_reads_the_item_level_description_memo(): action = Action.model_validate( diff --git a/easyvista_python_client/models/tests/test_common.py b/easyvista_python_client/models/tests/test_common.py index 925e6dc..707da65 100644 --- a/easyvista_python_client/models/tests/test_common.py +++ b/easyvista_python_client/models/tests/test_common.py @@ -1,7 +1,13 @@ +from datetime import datetime, timedelta, timezone + +import pydantic +import pytest + from easyvista_python_client import FieldClassification from easyvista_python_client.models.common import ( EasyvistaModel, EasyvistaWriteModel, + OptionalDateTime, OptionalInt, _empty_str_to_none, ) @@ -64,3 +70,89 @@ def test_base_model_keeps_unknown_fields(): dumped = model.model_dump(by_alias=True) assert dumped["RFC_NUMBER"] == "I123" assert dumped["e_custom1"] == "x" + + +class _Probe(EasyvistaModel): + when: OptionalDateTime = None + + +def test_parses_the_live_easyvista_format(): + got = _Probe.model_validate({"when": "2026-08-17T15:40:41.610+02:00"}).when + assert got == datetime( + 2026, 8, 17, 15, 40, 41, 610000, tzinfo=timezone(timedelta(hours=2)) + ) + + +def test_the_empty_string_sentinel_becomes_none(): + """EasyVista returns "" for every unset date — not null. Measured live.""" + assert _Probe.model_validate({"when": ""}).when is None + assert _Probe.model_validate({"when": " "}).when is None + + +def test_a_missing_key_is_none(): + assert _Probe.model_validate({}).when is None + + +def test_an_explicit_none_is_none_not_an_error(): + """A JSON ``null`` is an ordinary absence on a ``datetime | None`` column. + + Regression guard. When the validator was tightened so a malformed value + raises instead of becoming a bogus epoch instant, it began raising on + ``None`` too -- so a wire payload carrying ``"LAST_UPDATE": null``, or a + caller passing the field's own default explicitly, failed validation. The + tightening is about *junk*; an absence is not junk, and this column's own + type says ``None`` is legal. + """ + assert _Probe.model_validate({"when": None}).when is None + assert _Probe(when=None).when is None + + +def test_an_unparseable_timestamp_raises_rather_than_silently_becoming_none(): + """A malformed date is a real signal; swallowing it would hide a format change. + + Contrast the "" sentinel above, which is EasyVista's documented way of + saying "unset" and is therefore not an error. + """ + with pytest.raises(pydantic.ValidationError) as exc_info: + _Probe.model_validate({"when": "not-a-date"}) + # _empty_str_to_none_datetime raises ValueError, which pydantic wraps into a + # ValidationError naming the field -- confirm it actually does, not just + # that *some* error was raised. + assert exc_info.value.errors()[0]["loc"] == ("when",) + + +@pytest.mark.parametrize( + "junk", + [ + # ISO-basic, no separators. Pydantic's own parser reads this as epoch + # seconds -> 1970-08-23T12:00:17Z, an instant 56 years off, SILENTLY. + "20260817", + # What an epoch-millis format change would look like on the wire. + # Pydantic reads it as 2025-08-17T12:40:41.610Z -- entirely plausible, + # which is exactly why absorbing it would defeat this guard. + 1755434441610, + "1755434441610", + # A plausible alternative "unset" sentinel; pydantic reads it as the + # epoch rather than reporting that EasyVista's sentinel is "". + 0, + "0", + ], +) +def test_a_numeric_shaped_value_raises_instead_of_becoming_an_epoch_instant(junk): + """The guard must not fall through to pydantic's much broader parser. + + Every value here is one pydantic accepts with a credible-looking result, so + a fallthrough would turn the one signal this guard exists to raise -- a + change in EasyVista's timestamp format -- into wrong data with no error. + """ + with pytest.raises(pydantic.ValidationError) as exc_info: + _Probe.model_validate({"when": junk}) + assert exc_info.value.errors()[0]["loc"] == ("when",) + + +def test_a_naive_datetime_input_comes_back_aware(): + """A datetime handed in directly (not a wire string) must still end up + aware -- OptionalDateTime promises "An aware `datetime | None`" for every + accepted input, not only for strings.""" + got = _Probe.model_validate({"when": datetime(2026, 1, 1, 9, 0, 0)}).when + assert got == datetime(2026, 1, 1, 9, 0, 0, tzinfo=timezone.utc) diff --git a/easyvista_python_client/models/tests/test_employee.py b/easyvista_python_client/models/tests/test_employee.py index 3b376c6..f789c92 100644 --- a/easyvista_python_client/models/tests/test_employee.py +++ b/easyvista_python_client/models/tests/test_employee.py @@ -1,3 +1,5 @@ +from datetime import datetime, timedelta, timezone + from easyvista_python_client.models.employee import ( Employee, EmployeeUpdate, @@ -65,3 +67,18 @@ def test_post_employee_to_api(): def test_employee_update_is_write_model(): assert EmployeeUpdate(phone_number="0102").to_api() == {"phone_number": "0102"} + + +def test_employee_last_update_parses_to_an_aware_datetime(): + """BREAKING as of 2026-08-17: last_update was str. EV-R7 proved the format. + + One of the seven fields the retype covers -- this is the only one of the + seven that had no failing pre-existing assertion to fix, so it was + otherwise only exercised transitively (never directly asserted). + """ + emp = Employee.model_validate( + {"EMPLOYEE_ID": 1, "LAST_UPDATE": "2026-08-17T15:40:41.610+02:00"} + ) + assert emp.last_update == datetime( + 2026, 8, 17, 15, 40, 41, 610000, tzinfo=timezone(timedelta(hours=2)) + ) diff --git a/easyvista_python_client/models/tests/test_request.py b/easyvista_python_client/models/tests/test_request.py index 72fc187..e2019e3 100644 --- a/easyvista_python_client/models/tests/test_request.py +++ b/easyvista_python_client/models/tests/test_request.py @@ -1,6 +1,10 @@ +from datetime import datetime, timedelta, timezone + +import pydantic import pytest from pydantic import ValidationError +from easyvista_python_client import ev_since_filter from easyvista_python_client.models.request import PostRequest, Request, RequestUpdate @@ -71,8 +75,8 @@ def test_post_request_custom_fields_get_e_prefix(): def test_request_update_to_api_omits_none(): - update = RequestUpdate(status_id=5) - assert update.to_api() == {"status_id": 5} + update = RequestUpdate(impact_id=5) + assert update.to_api() == {"impact_id": 5} def test_request_declares_title_and_core_scalars(): @@ -114,8 +118,9 @@ def test_request_declares_title_and_core_scalars(): assert req.recipient_id == 13 assert req.owner_id == 14 assert req.external_reference == "REF-1" - assert req.submit_date_ut == "2026-01-01 09:00:00" - assert req.last_update == "2026-01-02 10:30:00" + # No offset in the fixture -> parse_ev_datetime treats it as UTC. + assert req.submit_date_ut == datetime(2026, 1, 1, 9, 0, 0, tzinfo=timezone.utc) + assert req.last_update == datetime(2026, 1, 2, 10, 30, 0, tzinfo=timezone.utc) def test_request_coerces_empty_string_numerics_to_none(): @@ -181,7 +186,7 @@ def test_request_update_serializes_title(): def test_request_update_omits_unset_fields(): - assert RequestUpdate(status_id=3).to_api() == {"status_id": 3} + assert RequestUpdate(impact_id=3).to_api() == {"impact_id": 3} def test_request_declares_the_official_time_fields(): @@ -198,10 +203,17 @@ def test_request_declares_the_official_time_fields(): "TIME_USED_TO_SOLVE_REQUEST": "3600", } ) - assert ticket.creation_date_ut == "2026-07-28 09:00:00" - assert ticket.max_resolution_date_ut == "2026-07-30 09:00:00" - assert ticket.expected_date_ut == "2026-07-29 09:00:00" - assert ticket.end_date_ut == "" + # No offset in the fixtures -> parse_ev_datetime treats them as UTC. + assert ticket.creation_date_ut == datetime( + 2026, 7, 28, 9, 0, 0, tzinfo=timezone.utc + ) + assert ticket.max_resolution_date_ut == datetime( + 2026, 7, 30, 9, 0, 0, tzinfo=timezone.utc + ) + assert ticket.expected_date_ut == datetime( + 2026, 7, 29, 9, 0, 0, tzinfo=timezone.utc + ) + assert ticket.end_date_ut is None # "" sentinel assert ticket.sla_id == 4 assert ticket.time_used_to_solve_request == "3600" @@ -227,3 +239,118 @@ def test_gtr_custom_fields_stay_out_of_the_official_bucket(): assert set(fc.custom) == {"E_GTR_STATUS", "E_GTI_UT"} assert "SLA_ID" in fc.official assert "MAX_RESOLUTION_DATE_UT" in fc.official + + +def test_request_timestamps_are_aware_datetimes(): + """BREAKING as of 2026-08-17: these were str. EV-R7 proved the format.""" + request = Request.model_validate( + { + "RFC_NUMBER": "I240101_0001", + "LAST_UPDATE": "2026-08-17T15:40:41.610+02:00", + "CREATION_DATE_UT": "2026-08-17T15:40:36.383+02:00", + "END_DATE_UT": "", + } + ) + assert request.last_update.tzinfo is not None + assert request.last_update.utcoffset() == timedelta(hours=2) + assert request.creation_date_ut.year == 2026 + assert request.end_date_ut is None # "" sentinel + + +def test_a_request_timestamp_round_trips_into_a_change_window_filter(): + """The retype must not cost the ability to build an interval (EV-R5).""" + request = Request.model_validate( + {"RFC_NUMBER": "I240101_0001", "LAST_UPDATE": "2026-08-17T15:40:41.610+02:00"} + ) + assert ev_since_filter("LAST_UPDATE", request.last_update) == ( + "LAST_UPDATE:(2026-08-17T15:40:41.610+02:00;)" + ) + + +def test_request_update_carries_the_writable_columns(): + """Pins the emitted body SHAPE only -- the client's own lowercase key names. + + This asserts nothing about the wire. A 200 on a PUT is not a receipt on this + API: a field it cannot honour is silently dropped while the request succeeds. + The read-back that does establish these three land lives in + ``integration_tests/test_live_ticket_identity.py`` + ``::test_request_update_writes_impact_owner_and_external_reference``. + """ + body = RequestUpdate( + title="t", + impact_id=1, + owner_id=42, + external_reference="PEER-abc123", + ).to_api() + assert body == { + "title": "t", + "impact_id": 1, + "owner_id": 42, + "external_reference": "PEER-abc123", + } + + +def test_request_update_still_rejects_an_unverified_field(): + """extra="forbid" is the guard that caught SEVERITY_ID before the wire.""" + with pytest.raises(pydantic.ValidationError): + RequestUpdate(severity_id=2) + + +def test_external_reference_longer_than_fifty_characters_is_refused_locally(): + """Bisected live: 50 accepted, 51 -> HTTP 590. Refuse before the round trip. + + Over-length is REJECTED server-side, not truncated, and the previously + stored value survives — so this guard loses nothing and saves a request. + """ + with pytest.raises(pydantic.ValidationError): + RequestUpdate(external_reference="X" * 51) + assert ( + RequestUpdate(external_reference="X" * 50).to_api()["external_reference"] + == "X" * 50 + ) + + +def test_request_update_refuses_status_id(): + """``RequestUpdate(status_id=...)`` must not be constructible. + + A regression guard with teeth, because the field existed and its failure was + invisible: measured live, a flat status write is rejected 590 when sent alone + and -- far worse -- returns 200, applies its companion field and drops the + status silently when sent beside one. Anything that reinstates this field + reinstates a write that reports success and stores nothing. The status route + is ``set_status`` / the ``{"closed": {"status_GUID": ...}}`` envelope. + """ + with pytest.raises(ValidationError) as excinfo: + RequestUpdate(status_id=2) + assert "status_id" in str(excinfo.value) + + +def test_post_request_carries_the_whole_documented_create_body(): + """Every field of the documented create body survives ``to_api``. + + The documented body (``docs/API_Info.md``) is catalog_code + origin + title + + description + department_id + urgency_id + impact_id, and sending a SUBSET is + what produces the 590 whose message is a bare SQL parser error. So the shape + is pinned here: a field silently dropped from this model would reintroduce + exactly that failure, on some catalogs only. + """ + body = PostRequest( + catalog_code="SYNTH_INC_001", + origin=7, + title="t", + description="d", + department_id=9, + urgency_id=7, + impact_id=21, + external_reference="X", + ).to_api() + assert body == { + "catalog_code": "SYNTH_INC_001", + "origin": 7, + "title": "t", + "description": "d", + "department_id": 9, + "urgency_id": 7, + "impact_id": 21, + "external_reference": "X", + } diff --git a/easyvista_python_client/references.py b/easyvista_python_client/references.py index ead676e..ef91ccf 100644 --- a/easyvista_python_client/references.py +++ b/easyvista_python_client/references.py @@ -7,14 +7,22 @@ conventions, so any field — including custom ``e_*`` fields on any instance — resolves the same way with no registry or configuration. -Leaf module: stdlib only, no model/client imports. +Leaf module: only stdlib plus the :mod:`~easyvista_python_client.timestamps` +leaf (no cycle: that module imports nothing from the package either), no +model/client imports. The timestamps import exists so ``.reference()`` on a +timestamp column (``LAST_UPDATE``, …) — now a ``datetime`` since the +2026-08-17 read-path retype — resolves to its rendered value instead of an +empty :class:`Reference`. """ from __future__ import annotations from dataclasses import dataclass +from datetime import datetime from typing import Any +from .timestamps import format_ev_datetime + @dataclass(frozen=True) class Reference: @@ -30,9 +38,23 @@ def display(self) -> str | None: def _scalar(value: Any) -> str | None: - """A non-empty id-like scalar as a string, else ``None`` (bools rejected).""" + """A non-empty id-like scalar as a string, else ``None`` (bools rejected). + + A ``datetime`` renders as EasyVista's own wire format + (:func:`~easyvista_python_client.timestamps.format_ev_datetime`), so + ``.reference("LAST_UPDATE")`` on a retyped timestamp field still resolves + to a populated :class:`Reference` instead of an empty one. That function + raises on a *naive* datetime -- which should not occur for a value that + came through ``OptionalDateTime`` -- but this must never raise, so a naive + value falls back to plain ``.isoformat()`` instead. + """ if isinstance(value, bool): return None + if isinstance(value, datetime): + try: + return format_ev_datetime(value) + except ValueError: + return value.isoformat() if isinstance(value, (str, int)) and str(value).strip(): return str(value).strip() return None diff --git a/easyvista_python_client/reporting.py b/easyvista_python_client/reporting.py index 55c70b6..94945af 100644 --- a/easyvista_python_client/reporting.py +++ b/easyvista_python_client/reporting.py @@ -7,43 +7,18 @@ from __future__ import annotations -import re from collections.abc import Iterable, Sequence from dataclasses import dataclass -from datetime import datetime, timezone +from datetime import datetime from typing import Any from .models.request import Request from .references import resolve_reference +from .timestamps import parse_ev_datetime -_FRACTION_RE = re.compile(r"\.(\d+)") - - -def _parse_iso_datetime(value: Any) -> datetime | None: - """Parse an EasyVista timestamp to a timezone-aware ``datetime``, or ``None``. - - Accepts a ``datetime`` (returned as-is, naive made UTC) or an ISO-8601 string. - Normalizes for Python 3.10's stricter ``fromisoformat``: maps a trailing ``Z`` - to ``+00:00`` and pads/truncates fractional seconds to 6 digits. A naive result - is treated as UTC. Unparseable input returns ``None``. - """ - if isinstance(value, datetime): - return value if value.tzinfo else value.replace(tzinfo=timezone.utc) - if not isinstance(value, str) or not value.strip(): - return None - text = value.strip() - if text.endswith(("Z", "z")): - text = text[:-1] + "+00:00" - match = _FRACTION_RE.search(text) - if match: - frac6 = (match.group(1) + "000000")[:6] - text = text[: match.start()] + "." + frac6 + text[match.end() :] - try: - parsed = datetime.fromisoformat(text) - except ValueError: - return None - return parsed if parsed.tzinfo else parsed.replace(tzinfo=timezone.utc) - +# Kept as a module-level alias so the eight existing tests in +# tests/test_reporting.py keep importing the name they were written against. +_parse_iso_datetime = parse_ev_datetime DEFAULT_DIMENSIONS: tuple[str, ...] = ( "STATUS", @@ -118,6 +93,26 @@ def aggregate_tickets( ``CREATION_DATE_UT`` (a ``datetime`` or ISO string); a ticket with a missing/unparseable date is excluded when a bound is set. Raises ``ValueError`` for a malformed bound string. + + **An offset-less bound is interpreted as UTC**, not as instance-local time, + because it routes through + :func:`~easyvista_python_client.parse_ev_datetime`. On a ``+02:00`` instance + ``created_since="2026-01-01T00:00:00"`` therefore silently excludes every + ticket created between 00:00 and 02:00 local on 1 January -- a two-hour hole + in a bound this docstring calls inclusive. Pass an aware ``datetime``, or an + offset-bearing string, when the boundary matters. Note this filter is + client-side and deliberately more permissive than the *wire* builders, which + refuse an offset-less time outright + (:func:`~easyvista_python_client.ev_since_filter`); making the two agree is a + behaviour change and a candidate follow-up. + + The "unparseable" half of that per-ticket guard is unreachable for a + ``Request`` built the normal way: ``Request.model_validate`` itself now + rejects a malformed ``CREATION_DATE_UT`` before this function ever sees the + ticket (see ``OptionalDateTime`` in ``models/common.py``). It stays as + defence-in-depth for a ``Request`` assembled some other way (e.g. + ``model_construct``, which bypasses validation) -- do not delete it as dead + code. """ since = _bound(created_since, "created_since") until = _bound(created_until, "created_until") diff --git a/easyvista_python_client/resources/actions.py b/easyvista_python_client/resources/actions.py index 88bfb67..660edf8 100644 --- a/easyvista_python_client/resources/actions.py +++ b/easyvista_python_client/resources/actions.py @@ -8,14 +8,14 @@ from __future__ import annotations -from collections.abc import Callable +from collections.abc import Callable, Iterable from typing import Any from .._transport import RequestSpec from ..filters import ev_equals_filter -from ..models.action import Action, PostAction +from ..models.action import Action, ActionUpdate, PostAction from ..pagination import extract_records -from .descriptor import ResourceDescriptor, build_get, build_search +from .descriptor import ResourceDescriptor, build_get, build_search, build_update ACTIONS: ResourceDescriptor[Action] = ResourceDescriptor( path="actions", envelope_key="actions", model=Action @@ -38,6 +38,9 @@ def parse(data: Any) -> Action: def build_list_actions( rfc_number: str, + *, + fields: Iterable[str] | str | None = None, + max_rows: int | None = None, ) -> tuple[RequestSpec, Callable[[Any], list[Action]]]: # Actions are listed via the TOP-LEVEL /actions resource filtered by the # request number, not a nested requests/{rfc}/actions path (which the API @@ -46,10 +49,28 @@ def build_list_actions( # so a raw value could append conditions and list another ticket's actions. A # blank one must raise too — ev_equals_filter returns None for blank input, and # search=None would list every action just as surely. + # + # ``fields`` is honoured by this endpoint and grants every scalar requested + # (verified live 2026-08-17), which is what lets a caller read a PAGE of + # actions' timestamps and authors in ONE request instead of an item fetch + # per action. + # Three limits, all silent: the memo bodies (``DESCRIPTION``, ``COMMENT``) + # come back as HREF objects under any projection — the text is never inlined + # —, ``fields=*`` is NOT a wildcard (it silently reduces to ``ACTION_ID``), + # and this call returns ONE PAGE. It does not paginate: a ticket with more + # actions than ``max_rows`` is truncated with no error, and the envelope's + # ``total_record_count`` is discarded by the parser below, so the caller + # cannot even detect it. ``max_rows`` is nonetheless passed explicitly so + # the cap is the client's configured page size rather than an unstated + # server default (25 on the verified instance) — a caller who is told the + # cap can raise it. Real pagination is a follow-up: it needs live + # verification that this endpoint's ``@next`` behaves like the others'. search = ev_equals_filter("REQUEST.RFC_NUMBER", rfc_number) if search is None: raise ValueError("rfc_number is required to list a ticket's actions") - spec, parse_search = build_search(ACTIONS, search=search) + spec, parse_search = build_search( + ACTIONS, search=search, fields=fields, max_rows=max_rows + ) def parse(data: Any) -> list[Action]: return parse_search(data).records @@ -70,3 +91,14 @@ def build_get_action( way the nested list path is. """ return build_get(ACTIONS, action_id) + + +def build_update_action( + action_id: str | int, payload: ActionUpdate +) -> tuple[RequestSpec, Callable[[Any], Action]]: + """Edit one action, via the TOP-LEVEL ``actions/{id}`` path. + + The nested ``requests/{rfc}/actions/{id}`` form is rejected with HTTP 403, + the same way the nested list and item paths are (verified live). + """ + return build_update(ACTIONS, action_id, payload) diff --git a/easyvista_python_client/resources/documents.py b/easyvista_python_client/resources/documents.py index 3a869f6..d553df0 100644 --- a/easyvista_python_client/resources/documents.py +++ b/easyvista_python_client/resources/documents.py @@ -57,6 +57,26 @@ def build_list_documents( return RequestSpec("GET", f"requests/{rfc_number}/documents"), _all_documents +def build_delete_document(rfc_number: str, document_id: str) -> RequestSpec: + """Delete one attachment, via the per-ticket NESTED path. + + ``DELETE requests/{rfc}/documents/{document_id}`` is live-verified + (2026-08-17): the document count went 5 → 4 and the target was absent from a + re-listing. The top-level ``DELETE documents/{id}`` returns HTTP 403. + + Both identifiers are required and must be non-blank: a blank ``document_id`` + would address the collection rather than an item, which is a very different + request to send by accident. + """ + rfc = str(rfc_number).strip() + if not rfc: + raise ValueError("rfc_number is required to delete a document") + doc = str(document_id).strip() + if not doc: + raise ValueError("document_id is required to delete a document") + return RequestSpec("DELETE", f"requests/{rfc}/documents/{doc}") + + def download_href(document: Document | str) -> str: """The URL to fetch a document's bytes. diff --git a/easyvista_python_client/resources/requests.py b/easyvista_python_client/resources/requests.py index f7bc45a..1dc48ce 100644 --- a/easyvista_python_client/resources/requests.py +++ b/easyvista_python_client/resources/requests.py @@ -68,12 +68,22 @@ def build_close_ticket( delete_actions: int | None = None, comment: str | None = None, ) -> tuple[RequestSpec, Callable[[Any], Request]]: - """Build a close (PUT) spec. - - ``status_guid`` is the instance's "closed" status GUID (EasyVista - ``status_GUID``); without it the API may not actually transition the ticket. - ``delete_actions=1`` drops the ticket's actions on close. Shapes follow the - documented close body (``docs/API_Info.md``), verified live. + """Build the ``{"closed": {...}}`` PUT spec — the API's status-set route. + + Despite the wire name, this envelope is **not limited to closing**. It is the + only working way to set a ticket's status, and it reaches every status: + handed each of six different ``STATUS_GUID``s in turn, a fresh ticket landed + on exactly the status requested every time -- including non-terminal ones + like "A prendre en compte" and "En cours". Nothing was forced to the closed + status. + + Note the addressing: ``status_GUID``, not ``STATUS_ID``. There is no flat + status update on this API -- see :class:`RequestUpdate` for what happens if + you try one. :func:`build_set_status` is the same spec under a name that says + what it does. + + ``delete_actions=1`` drops the ticket's actions. Shapes follow the documented + close body (``docs/API_Info.md``), verified live. """ closed: dict[str, Any] = {} if status_guid is not None: @@ -89,3 +99,19 @@ def parse(data: Any) -> Request: return Request.model_validate(records[0] if records else data) return spec, parse + + +def build_set_status( + rfc_number: str, + *, + status_guid: str, + comment: str | None = None, +) -> tuple[RequestSpec, Callable[[Any], Request]]: + """Build a spec that sets ``rfc_number``'s status to ``status_guid``. + + The same request :func:`build_close_ticket` builds, named for what it + actually does. ``status_guid`` is required here rather than optional: the + envelope without one is a close request with nothing to close to, and making + that unexpressible is the point of having this function at all. + """ + return build_close_ticket(rfc_number, status_guid=status_guid, comment=comment) diff --git a/easyvista_python_client/resources/tests/test_actions.py b/easyvista_python_client/resources/tests/test_actions.py index 5feab92..103eaad 100644 --- a/easyvista_python_client/resources/tests/test_actions.py +++ b/easyvista_python_client/resources/tests/test_actions.py @@ -1,8 +1,12 @@ import pytest -from easyvista_python_client.models.action import Action, PostAction +from easyvista_python_client.models.action import Action, ActionUpdate, PostAction from easyvista_python_client.resources import actions as a -from easyvista_python_client.resources.actions import build_get_action +from easyvista_python_client.resources.actions import ( + build_get_action, + build_list_actions, + build_update_action, +) def test_action_accepts_object_action_type(): @@ -69,6 +73,39 @@ def test_build_list_actions_filters_by_rfc(): assert spec.params["search"] == 'REQUEST.RFC_NUMBER:"I240101_0001"' +def test_list_actions_passes_a_fields_projection_through(): + """EV-R3: the projection is what makes comment metadata 1 request, not N.""" + spec, _parse = build_list_actions( + "I240101_0001", fields=["ACTION_ID", "CREATION_DATE_UT", "LAST_UPDATE"] + ) + assert spec.params["fields"] == "ACTION_ID,CREATION_DATE_UT,LAST_UPDATE" + assert spec.params["search"] == 'REQUEST.RFC_NUMBER:"I240101_0001"' + + +def test_list_actions_accepts_a_bare_string_projection(): + spec, _parse = build_list_actions("I240101_0001", fields="ACTION_ID,LAST_UPDATE") + assert spec.params["fields"] == "ACTION_ID,LAST_UPDATE" + + +def test_list_actions_omits_fields_when_not_requested(): + """Absent, not empty: `fields=` with no value is not the same request.""" + spec, _parse = build_list_actions("I240101_0001") + assert "fields" not in spec.params + + +def test_list_actions_sends_the_row_cap_explicitly_when_given_one(): + """The cap must be the CLIENT's, not the server's unstated default. + + This call returns one page and does not paginate, so whoever owns the cap + owns where the action log gets truncated. Every sibling search on the client + injects ``config.default_max_rows``; this one used to be the single search + that deferred to the server (25 on the verified instance), which a caller + could neither see nor raise. + """ + spec, _parse = build_list_actions("I240101_0001", max_rows=200) + assert spec.params["max_rows"] == 200 + + def test_build_get_action_targets_the_top_level_path(): spec, _ = build_get_action(52990) assert spec.method == "GET" @@ -80,3 +117,16 @@ def test_build_get_action_parses_an_enveloped_record(): _, parse = build_get_action(52990) action = parse({"actions": [{"ACTION_ID": 52990}]}) assert action.action_id == 52990 + + +def test_update_action_uses_the_top_level_path(): + """The nested requests/{rfc}/actions/{id} form returns 403 (verified live).""" + spec, _parse = build_update_action(57483, ActionUpdate(description="edited")) + assert spec.method == "PUT" + assert spec.path == "actions/57483" + assert spec.json == {"description": "edited"} + + +def test_update_action_drops_unset_fields(): + spec, _parse = build_update_action(1, ActionUpdate(description="only this")) + assert "comment" not in spec.json diff --git a/easyvista_python_client/resources/tests/test_documents.py b/easyvista_python_client/resources/tests/test_documents.py index 5e599da..b78d15f 100644 --- a/easyvista_python_client/resources/tests/test_documents.py +++ b/easyvista_python_client/resources/tests/test_documents.py @@ -4,7 +4,10 @@ from easyvista_python_client.models.document import Document from easyvista_python_client.resources import documents as d -from easyvista_python_client.resources.documents import download_href +from easyvista_python_client.resources.documents import ( + build_delete_document, + download_href, +) def test_build_add_document_base64_envelope_and_path(): @@ -82,3 +85,18 @@ def test_download_href_accepts_a_raw_string(): def test_download_href_raises_when_no_url_is_available(): with pytest.raises(ValueError, match="no download URL"): download_href(Document.model_validate({"DOCUMENT": "report.pdf"})) + + +def test_delete_document_uses_the_nested_per_ticket_path(): + """The top-level documents/{id} form returns 403 (verified live).""" + spec = build_delete_document("I240101_0001", "12345_abcdef") + assert spec.method == "DELETE" + assert spec.path == "requests/I240101_0001/documents/12345_abcdef" + assert spec.json is None + + +def test_delete_document_requires_both_identifiers(): + with pytest.raises(ValueError, match="rfc_number"): + build_delete_document("", "12345_abcdef") + with pytest.raises(ValueError, match="document_id"): + build_delete_document("I240101_0001", "") diff --git a/easyvista_python_client/resources/tests/test_requests.py b/easyvista_python_client/resources/tests/test_requests.py index 14794ef..cc84982 100644 --- a/easyvista_python_client/resources/tests/test_requests.py +++ b/easyvista_python_client/resources/tests/test_requests.py @@ -79,10 +79,10 @@ def test_build_search_tickets_includes_offset(): def test_build_update_ticket(): - spec, _parser = r.build_update_ticket("I1", RequestUpdate(status_id=3)) + spec, _parser = r.build_update_ticket("I1", RequestUpdate(impact_id=3)) assert spec.method == "PUT" assert spec.path == "requests/I1" - assert spec.json == {"status_id": 3} + assert spec.json == {"impact_id": 3} def test_build_close_ticket_default_and_comment(): @@ -108,3 +108,24 @@ def test_build_close_ticket_full_documented_shape(): "comment": "resolved", } } + + +def test_build_set_status_sends_the_closed_envelope(): + """``set_status`` is the ``closed`` envelope, addressed by GUID. + + Pins both halves of the call shape that took several wrong turns to find: the + body is wrapped in ``closed`` (not flat), and the key is ``status_GUID`` (not + ``STATUS_ID``). The envelope is not limited to closing -- six different status + GUIDs each landed on exactly the status requested. + """ + spec, _parser = r.build_set_status("I1", status_guid="{G}", comment="c") + assert spec.method == "PUT" + assert spec.path == "requests/I1" + assert spec.json == {"closed": {"status_GUID": "{G}", "comment": "c"}} + + +def test_build_set_status_matches_build_close_ticket(): + """The two builders are the same request; only the name differs.""" + a, _ = r.build_set_status("I1", status_guid="{G}") + b, _ = r.build_close_ticket("I1", status_guid="{G}") + assert (a.method, a.path, a.json) == (b.method, b.path, b.json) diff --git a/easyvista_python_client/testing/test_method_invocation.py b/easyvista_python_client/testing/test_method_invocation.py index d7da089..00bf4ca 100644 --- a/easyvista_python_client/testing/test_method_invocation.py +++ b/easyvista_python_client/testing/test_method_invocation.py @@ -24,6 +24,7 @@ import respx from easyvista_python_client import ( + ActionUpdate, AsyncEasyvistaClient, DepartmentUpdate, EasyvistaClient, @@ -70,6 +71,7 @@ ARGS: dict[str, tuple[tuple, dict]] = { "add_document": (("I1",), {"filename": "d.txt", "content": b"x"}), "close_ticket": (("I1",), {}), + "set_status": (("I1",), {"status_guid": "{0000-0000}"}), "count_tickets": ((), {}), "create_action": (("I1", PostAction()), {}), "create_asset": ((PostAsset(catalog_id=1),), {}), @@ -77,6 +79,7 @@ "create_employee": ((PostEmployee(),), {}), "create_ticket": ((PostRequest(catalog_code="C"),), {}), "create_tickets": (([PostRequest(catalog_code="C")],), {}), + "delete_document": (("I1", "d1"), {}), "download_document": (("requests/I1/documents/1",), {}), "find_departments": (("Acme",), {}), "get_action": ((1,), {}), @@ -98,7 +101,9 @@ "search_departments": ((), {}), "search_employees": ((), {}), "search_tickets": ((), {}), + "stream_document": (("requests/I1/documents/1",), {}), "ticket_statistics": ((), {"max_records": 1}), + "update_action": ((1, ActionUpdate()), {}), "update_department": ((1, DepartmentUpdate()), {}), "update_employee": ((1, EmployeeUpdate()), {}), "update_ticket": (("I1", RequestUpdate()), {}), diff --git a/easyvista_python_client/testing/test_public_api.py b/easyvista_python_client/testing/test_public_api.py index 0a699b8..58b26ef 100644 --- a/easyvista_python_client/testing/test_public_api.py +++ b/easyvista_python_client/testing/test_public_api.py @@ -2,7 +2,7 @@ def test_package_imports_and_has_version(): - assert easyvista_python_client.__version__ == "0.1.0" + assert easyvista_python_client.__version__ == "0.2.0" def test_public_exports_available(): diff --git a/easyvista_python_client/tests/test_context.py b/easyvista_python_client/tests/test_context.py index 5749ef4..87874a2 100644 --- a/easyvista_python_client/tests/test_context.py +++ b/easyvista_python_client/tests/test_context.py @@ -29,6 +29,7 @@ def _ticket() -> Request: "HREF": "https://h/api/v1/12345/catalog-requests/5791", }, "CREATION_DATE_UT": "2025-11-28T11:35:22+01:00", + "LAST_UPDATE": "2025-11-28T16:14:41.133+01:00", } ) @@ -39,6 +40,13 @@ def test_to_markdown_has_title_and_header_labels(): assert "| Status | En cours |" in md assert "| Department | Example Department |" in md assert "| Catalog | [EXAMPLE] - ticket |" in md + # Exact rendered literals, not just "Created"/"Updated" substrings -- a + # presence-only check would still pass on an empty cell (the 2026-08-17 + # retype briefly made these rows vanish entirely: model_dump(by_alias=True) + # yields a datetime for these keys now, and the extractor used to return "" + # for anything that wasn't already a str). + assert "| Created | 2025-11-28T11:35:22.000+01:00 |" in md + assert "| Updated | 2025-11-28T16:14:41.133+01:00 |" in md def test_to_markdown_contains_no_api_url(): diff --git a/easyvista_python_client/tests/test_directory.py b/easyvista_python_client/tests/test_directory.py index 1450077..8692432 100644 --- a/easyvista_python_client/tests/test_directory.py +++ b/easyvista_python_client/tests/test_directory.py @@ -1,4 +1,5 @@ from easyvista_python_client.directory import ( + RECENT_TICKETS_SORT, DepartmentContext, _department_matches, _normalize_name, @@ -36,3 +37,17 @@ def test_department_context_holds_all_parts(): ) assert ctx.department.department_id == 60 assert ctx.employees[0].employee_id == 1 + + +def test_recent_tickets_sort_uses_the_space_separated_form(): + """The colon form is SILENTLY IGNORED by EasyVista (measured 2026-08-17). + + On a date column, `FIELD:DESC` returned rows in the API's default order — + byte-identical to an unsorted page — so a colon-form constant is the + difference between "most recent" and "arbitrary". The rule is syntactic, so + it applies to RFC_NUMBER too; the live guard for this exact token lives in + integration_tests/test_live_change_window.py. A regression here is invisible + at runtime, which is why it is asserted at all. + """ + assert RECENT_TICKETS_SORT == "RFC_NUMBER DESC" + assert ":" not in RECENT_TICKETS_SORT diff --git a/easyvista_python_client/tests/test_fields.py b/easyvista_python_client/tests/test_fields.py index 880fb1c..1a13249 100644 --- a/easyvista_python_client/tests/test_fields.py +++ b/easyvista_python_client/tests/test_fields.py @@ -1,3 +1,5 @@ +from datetime import datetime, timedelta, timezone + from easyvista_python_client._fields import _label, _text @@ -7,6 +9,20 @@ def test_text_strips_strings_and_ignores_non_strings(): assert _text(123) == "" +def test_text_renders_an_aware_datetime_as_the_ev_wire_format(): + value = datetime( + 2026, 8, 17, 15, 40, 41, 610000, tzinfo=timezone(timedelta(hours=2)) + ) + assert _text(value) == "2026-08-17T15:40:41.610+02:00" + + +def test_text_renders_a_naive_datetime_via_isoformat_fallback(): + # format_ev_datetime refuses a naive datetime; _text must never raise, so it + # falls back to plain .isoformat() rather than propagating that ValueError. + value = datetime(2026, 8, 17, 15, 40, 41) + assert _text(value) == "2026-08-17T15:40:41" + + def test_label_prefers_first_non_empty_key_and_drops_href(): obj = {"STATUS_EN": "", "STATUS_FR": "En cours", "HREF": "http://x/api/v1"} assert _label(obj, ("STATUS_EN", "STATUS_FR")) == "En cours" diff --git a/easyvista_python_client/tests/test_filters.py b/easyvista_python_client/tests/test_filters.py index 6d0d542..924e240 100644 --- a/easyvista_python_client/tests/test_filters.py +++ b/easyvista_python_client/tests/test_filters.py @@ -1,12 +1,20 @@ +from datetime import datetime, timedelta, timezone + import pytest from easyvista_python_client import ( escape_ev_value, + ev_between_filter, + ev_contains_filter, ev_equals_filter, ev_in_filter, + ev_since_filter, + ev_starts_with_filter, is_safe_ev_value, ) +_CET = timezone(timedelta(hours=1)) + def test_equals_filter_quotes_the_value(): assert ev_equals_filter("DEPARTMENT_CODE", "ACME") == 'DEPARTMENT_CODE:"ACME"' @@ -55,3 +63,267 @@ def test_equals_filter_rejects_unsafe_value(): def test_is_safe_predicate_never_raises(): assert is_safe_ev_value("ACME") is True assert is_safe_ev_value('X"') is False + + +def test_since_emits_the_open_ended_interval(): + """``FIELD:(a;)`` — the form measured live as a watermark lower bound. + + The bound is NORMALISED, not passed through. The literal below is the most + natural way for a caller to satisfy the offset gate, and measured live + 2026-08-18 it is HTTP 590 exactly as written -- second precision with an + offset is not a rendering the interval grammar accepts. So the builder + re-renders it at millisecond precision, which is. + """ + got = ev_since_filter("LAST_UPDATE", "2025-11-28T16:14:41+01:00") + assert got == "LAST_UPDATE:(2025-11-28T16:14:41.000+01:00;)" + + +def test_since_normalises_a_space_separated_literal_to_the_T_form(): + """``str(aware_datetime)`` uses a space, which is HTTP 590 on the wire. + + Measured live 2026-08-18 (and again in round 1). Normalising is what makes + the most obvious Python rendering of an aware datetime usable at all. + """ + got = ev_since_filter("LAST_UPDATE", "2025-11-28 16:14:41.133+01:00") + assert got == "LAST_UPDATE:(2025-11-28T16:14:41.133+01:00;)" + + +def test_since_accepts_a_lowercase_z_the_read_path_already_accepts(): + """``parse_ev_datetime`` accepts ``z``; the gate must not contradict it. + + Rejecting a value this package's own read path produces -- with a message + reading "is not an EasyVista timestamp" -- would be actively misleading. + """ + got = ev_since_filter("LAST_UPDATE", "2025-11-28T15:14:41.133z") + assert got == "LAST_UPDATE:(2025-11-28T15:14:41.133+00:00;)" + + +def test_the_string_and_datetime_paths_emit_byte_identical_bounds(): + """The last asymmetry between the two input paths, closed by normalisation. + + Before this, the same instant emitted two different literals depending on + whether the caller had already stringified it -- and only one of the two was + a rendering the wire honours. + """ + dt = datetime(2025, 11, 28, 16, 14, 41, 133000, tzinfo=_CET) + assert ev_since_filter("LAST_UPDATE", dt) == ev_since_filter( + "LAST_UPDATE", dt.isoformat() + ) + + +def test_since_accepts_a_datetime_and_formats_the_offset_literal(): + dt = datetime(2025, 11, 28, 16, 14, 41, 133000, tzinfo=_CET) + assert ev_since_filter("LAST_UPDATE", dt) == ( + "LAST_UPDATE:(2025-11-28T16:14:41.133+01:00;)" + ) + + +def test_between_emits_both_bounds(): + got = ev_between_filter("LAST_UPDATE", "2025-11-28", "2099-12-31") + assert got == "LAST_UPDATE:(2025-11-28;2099-12-31)" + + +def test_blank_input_returns_none_so_callers_compose_without_conditionals(): + assert ev_since_filter("LAST_UPDATE", None) is None + assert ev_since_filter("LAST_UPDATE", "") is None + assert ev_between_filter("LAST_UPDATE", None, None) is None + + +def test_between_with_only_an_end_bound_is_open_on_the_left(): + assert ev_between_filter("LAST_UPDATE", None, "2099-12-31") == ( + "LAST_UPDATE:(;2099-12-31)" + ) + + +@pytest.mark.parametrize( + "bad", + [ + '2025-11-28";DEPARTMENT_ID:"9', # quote breakout + "2025-11-28;2099-12-31", # a second bound smuggled in + "2025-11-28)", # closes the interval early + "2025-11-28 or 1=1", + "today", # a real EV token, but not a timestamp + ], +) +def test_interval_refuses_anything_that_is_not_a_timestamp(bad): + """The interval value is UNQUOTED, so ';' and ')' would break out of it. + + ``ev_equals_filter`` can rely on quoting; this one cannot, so it validates + the shape instead. Refuse rather than interpolate. + """ + with pytest.raises(ValueError, match="timestamp"): + ev_since_filter("LAST_UPDATE", bad) + + +@pytest.mark.parametrize( + "bad", + [ + "9999-99-99", # no such month + "2025-02-30", # no such day (February) + "2025-13-45T99:99:99", # no such month/day/time at all + "2025-11-28T25:61:61", # out-of-range time components + "٢٠٢٥-١١-٢٨", # non-ASCII digits # noqa: RUF001 + ], +) +def test_interval_refuses_a_well_shaped_but_impossible_timestamp(bad): + """The regex is a shape gate only; a calendar-invalid value must still be + refused, because a dropped condition returns the whole table rather than + an error (this is what makes a typo'd watermark dangerous). + """ + with pytest.raises(ValueError, match="timestamp"): + ev_since_filter("LAST_UPDATE", bad) + + +@pytest.mark.parametrize( + ("literal", "emitted"), + [ + # A date alone is legal AND is passed through unchanged: day + # granularity, no time to misplace, and measured live as honoured + # (round 1: 4107 rows against a 4316-row table). Re-rendering it would + # invent a midnight instant in some zone. + ("2025-11-28", "2025-11-28"), + # Already the one honoured time rendering: normalisation is a no-op. + ("2025-11-28T16:14:41.133+01:00", "2025-11-28T16:14:41.133+01:00"), + # Microseconds are truncated to EasyVista's own millisecond precision. + ("2025-11-28T16:14:41.133456Z", "2025-11-28T16:14:41.133+00:00"), + ], +) +def test_interval_accepts_every_rendering_measured_live(literal, emitted): + """The guard's acceptance side: a regression here fails CLOSED on real + watermarks, which no rejection test would catch. + + Accepted is not the same as emitted verbatim -- every admitted *time* is + re-rendered into the one form measured live as honoured. See + ``test_since_emits_the_open_ended_interval``. + + The offset-less *time* renderings that round 1 measured as accepted by the + API moved to ``test_interval_refuses_a_time_without_an_offset``: the wire + takes them, but it reads them in another zone. See that test. + """ + assert ev_since_filter("LAST_UPDATE", literal) == f"LAST_UPDATE:({emitted};)" + + +def test_interval_refuses_a_sub_minute_utc_offset_on_either_path(): + """A whole-minute offset is not a given: historical zoneinfo zones break it. + + ``format_ev_datetime`` would render ``+05:53:20``, which no shape this + grammar accepts can express -- and which the string path already refused. + Validating the RENDERED bound is what keeps the datetime path from emitting + something its own sibling path would reject. + + Both paths must give the SAME diagnosis, which is why the string half matches + on "whole number of minutes" rather than merely on "timestamp". The generic + message ("is not an EasyVista timestamp ... pass a datetime to be certain") + would be wrong twice over here: the value IS a valid ISO-8601 timestamp -- it + is what ``isoformat()`` returns for such a zone -- and following the advice + raises on the datetime path for the same underlying reason. + """ + odd = timezone(timedelta(hours=5, minutes=53, seconds=20)) + aware = datetime(2025, 11, 28, 16, 14, 41, tzinfo=odd) + with pytest.raises(ValueError, match="whole number of minutes"): + ev_since_filter("LAST_UPDATE", aware) + with pytest.raises(ValueError, match="whole number of minutes"): + ev_since_filter("LAST_UPDATE", aware.isoformat()) + # The fractional-second variant takes the same branch: `isoformat()` emits + # microseconds whenever they are non-zero, so this is the shape a caller who + # serialised `datetime.now(odd)` would actually hand back. + with pytest.raises(ValueError, match="whole number of minutes"): + ev_since_filter("LAST_UPDATE", aware.replace(microsecond=133999).isoformat()) + # A genuinely unparseable value must keep the generic message -- the new + # branch must not swallow it. + with pytest.raises(ValueError, match="is not an EasyVista timestamp"): + ev_since_filter("LAST_UPDATE", "not-a-timestamp") + + +@pytest.mark.parametrize( + "bad", + [ + "2025-11-28T16:14:41", + "2025-11-28 16:14:41", + "2025-11-28T16:14:41.133", + "2025-11-28T16:14:41.133456", + ], +) +def test_interval_refuses_a_time_without_an_offset(bad): + """An offset-less time SILENTLY SHIFTS the window, so refuse it locally. + + The API *accepts* these -- which is precisely the danger. Measured live + 2026-08-18 against one instance, the same wall-clock text with and without + its offset enumerated 13 rows and 11 rows respectively; the offset-less form + is read in another zone, moving the bound *later* and skipping records with + no error of any kind. A watermark that silently skips is the worst outcome + this grammar can produce. + + ``format_ev_datetime`` already refuses a naive ``datetime`` for exactly this + reason, and its docstring says so. Accepting a naive *string* let the same + hazard reach the wire by the other path, so both paths now refuse. + """ + with pytest.raises(ValueError, match="offset"): + ev_since_filter("LAST_UPDATE", bad) + + +def test_between_refuses_an_offsetless_time_on_either_bound(): + """Both bounds go through the same gate, so neither may be naive.""" + with pytest.raises(ValueError, match="offset"): + ev_between_filter("LAST_UPDATE", "2025-11-28T16:14:41", "2025-12-01") + with pytest.raises(ValueError, match="offset"): + ev_between_filter("LAST_UPDATE", "2025-11-28", "2025-12-01T16:14:41") + + +def test_contains_wraps_the_value_in_wildcards_with_the_tilde_operator(): + """``~`` IS a pattern operator; it needs an explicit ``*`` (measured live).""" + assert ev_contains_filter("ASSET_TAG", "LAPTOP") == 'ASSET_TAG~"*LAPTOP*"' + + +def test_starts_with_anchors_on_the_left_only(): + assert ev_starts_with_filter("RFC_NUMBER", "I26081") == 'RFC_NUMBER~"I26081*"' + + +def test_wildcard_builders_reject_a_double_quote(): + """Same reasoning as escape_ev_value: no escape for '"' exists.""" + with pytest.raises(ValueError): + ev_contains_filter("ASSET_TAG", 'LAP"TOP') + + +@pytest.mark.parametrize( + "bad", + [ + "LAP*TOP", + "LAP%TOP", + # `_` is a SINGLE-character wildcard under `~`, measured live: replacing + # one character of an RFC that matched 1 row gave 9. Underscores are + # pervasive in EasyVista codes, so this is the routine case, not the + # exotic one -- `ASSET_TAG~"*LAPTOP_01*"` also matches `LAPTOP-01`. + "LAP_TOP", + "LAPTOP_01", + # `[` opens a character class; `[0-9]` in the same position also gave 9. + "LAP[0-9]TOP", + # A backslash does NOT escape it (`\\_` returned 0 rows live), so an + # escaped-looking value is refused too rather than silently mismatching. + r"LAP\_TOP", + ], +) +def test_wildcard_builders_reject_a_metacharacter_inside_the_value(bad): + """A metacharacter in the middle silently changes what the caller asked for. + + ``ev_contains_filter("A*B")`` would match "A" then anything then "B" rather + than the literal "A*B", so refuse instead of quietly widening the query. + All four of ``* % _ [`` behave this way under ``~`` (measured live) and none + of them can be escaped, so all four are refused on the identical rationale. + """ + with pytest.raises(ValueError, match="metacharacter"): + ev_contains_filter("ASSET_TAG", bad) + with pytest.raises(ValueError, match="metacharacter"): + ev_starts_with_filter("ASSET_TAG", bad) + + +@pytest.mark.parametrize("blank", [None, "", " "]) +def test_blank_wildcard_value_returns_none_not_a_match_everything_pattern(blank): + """``FIELD~"**"`` would match every row — the exact silent-widening shape. + + ``None`` is included because the signature says ``str | None``: without the + guard, ``str(None)`` would render ``FIELD~"*None*"``, a pattern that both + widens silently and matches on a value no caller ever supplied. + """ + assert ev_contains_filter("ASSET_TAG", blank) is None + assert ev_starts_with_filter("ASSET_TAG", blank) is None diff --git a/easyvista_python_client/tests/test_references.py b/easyvista_python_client/tests/test_references.py index 319d876..aa18d07 100644 --- a/easyvista_python_client/tests/test_references.py +++ b/easyvista_python_client/tests/test_references.py @@ -1,6 +1,9 @@ +from datetime import datetime, timedelta, timezone + from easyvista_python_client.models.request import Request from easyvista_python_client.references import ( Reference, + _scalar, localized_label, resolve_reference, ) @@ -117,6 +120,30 @@ def test_model_reference_missing_field_is_empty(): assert ticket.reference("DEPARTMENT") == Reference(id=None, label=None) +def test_scalar_renders_an_aware_datetime_as_the_ev_wire_format(): + value = datetime( + 2026, 8, 17, 15, 40, 41, 610000, tzinfo=timezone(timedelta(hours=2)) + ) + assert _scalar(value) == "2026-08-17T15:40:41.610+02:00" + + +def test_scalar_renders_a_naive_datetime_via_isoformat_fallback(): + # format_ev_datetime refuses a naive datetime; _scalar must never raise, so + # it falls back to plain .isoformat() rather than propagating that error. + value = datetime(2026, 8, 17, 15, 40, 41) + assert _scalar(value) == "2026-08-17T15:40:41" + + +def test_model_reference_on_a_retyped_timestamp_field_is_populated(): + """Request.reference("LAST_UPDATE") must not regress to an empty Reference + now that last_update is a datetime rather than a str (2026-08-17 retype).""" + ticket = Request.model_validate( + {"RFC_NUMBER": "I1", "LAST_UPDATE": "2026-08-17T15:40:41.610+02:00"} + ) + ref = ticket.reference("LAST_UPDATE") + assert ref.display == "2026-08-17T15:40:41.610+02:00" + + def test_reference_exported_from_package(): import easyvista_python_client as evc diff --git a/easyvista_python_client/tests/test_reporting.py b/easyvista_python_client/tests/test_reporting.py index 750d785..fd013b9 100644 --- a/easyvista_python_client/tests/test_reporting.py +++ b/easyvista_python_client/tests/test_reporting.py @@ -147,11 +147,21 @@ def test_created_since_until_inclusive_bounds(): assert stats.total == 2 -def test_window_excludes_missing_or_unparseable_dates(): +def test_window_excludes_missing_dates(): + """A ticket with no CREATION_DATE_UT is excluded once a window bound is set. + + This test used to also cover a third, "garbage"-dated ticket to exercise the + unparseable-date branch here in ``aggregate_tickets``. As of the 2026-08-17 + read-path retype, ``Request.model_validate`` itself rejects a malformed + ``CREATION_DATE_UT`` (see + ``test_an_unparseable_timestamp_raises_rather_than_silently_becoming_none`` + in ``models/tests/test_common.py``), so a ``Request`` with an unparseable + creation date can no longer be constructed through the validated path this + helper uses -- that sub-case is gone, not weakened. + """ tickets = [ _ticket(RFC_NUMBER="I1", CREATION_DATE_UT="2025-06-15T12:00:00+00:00"), _ticket(RFC_NUMBER="I2"), # no date - _ticket(RFC_NUMBER="I3", CREATION_DATE_UT="garbage"), ] stats = aggregate_tickets( tickets, dimensions=(), created_since="2025-01-01T00:00:00+00:00" diff --git a/easyvista_python_client/tests/test_timestamps.py b/easyvista_python_client/tests/test_timestamps.py new file mode 100644 index 0000000..323ed9e --- /dev/null +++ b/easyvista_python_client/tests/test_timestamps.py @@ -0,0 +1,57 @@ +"""Tests for EasyVista's timestamp format. + +The format was established live on 2026-08-17: ISO 8601 with an explicit UTC +offset and millisecond precision, e.g. ``2026-08-17T15:40:41.610+02:00``. An +unset date comes back as the empty string, not ``null``. +""" + +from __future__ import annotations + +from datetime import datetime, timedelta, timezone + +import pytest + +from easyvista_python_client import format_ev_datetime, parse_ev_datetime + + +def test_parses_the_live_format_with_offset_and_milliseconds(): + """The exact shape measured live (YYYY-MM-DDTHH:MM:SS.mmm+HH:MM).""" + dt = parse_ev_datetime("2026-08-17T15:40:41.610+02:00") + assert dt == datetime( + 2026, 8, 17, 15, 40, 41, 610000, tzinfo=timezone(timedelta(hours=2)) + ) + assert dt.utcoffset() == timedelta(hours=2) + + +def test_parsed_value_is_always_aware(): + """Action.updated_at requires an aware datetime; a naive one is a bug.""" + assert parse_ev_datetime("2026-08-17T15:40:41.610+02:00").tzinfo is not None + + +def test_the_empty_string_sentinel_is_none_not_an_error(): + """An unset EasyVista date is ``""``. Measured on every unpopulated column.""" + assert parse_ev_datetime("") is None + assert parse_ev_datetime(" ") is None + assert parse_ev_datetime(None) is None + + +def test_unparseable_input_is_none_rather_than_raising(): + assert parse_ev_datetime("not-a-date") is None + assert parse_ev_datetime(12345) is None + + +def test_formats_back_to_the_literal_the_interval_grammar_accepts(): + """``LAST_UPDATE:(;)`` was honoured live with exactly this rendering.""" + dt = datetime(2025, 11, 28, 16, 14, 41, 133000, tzinfo=timezone(timedelta(hours=1))) + assert format_ev_datetime(dt) == "2025-11-28T16:14:41.133+01:00" + + +def test_format_round_trips_through_parse(): + literal = "2026-08-17T15:40:41.610+02:00" + assert format_ev_datetime(parse_ev_datetime(literal)) == literal + + +def test_format_refuses_a_naive_datetime(): + """Refuse rather than guess a zone: a naive instant cannot name a moment.""" + with pytest.raises(ValueError, match="timezone-aware"): + format_ev_datetime(datetime(2026, 8, 17, 15, 40, 41)) diff --git a/easyvista_python_client/timestamps.py b/easyvista_python_client/timestamps.py new file mode 100644 index 0000000..6a83a34 --- /dev/null +++ b/easyvista_python_client/timestamps.py @@ -0,0 +1,91 @@ +"""EasyVista's timestamp format, in one place. + +Established against a live instance on 2026-08-17: every returned timestamp is +**ISO 8601 with an explicit UTC offset** and millisecond precision, e.g. +``2026-08-17T15:40:41.610+02:00``. Verified by arithmetic, not inspection — a +write bracketed by our own UTC clock at ``13:40:40.411Z``/``13:40:40.869Z`` +produced ``15:40:41.610+02:00``, which *is* ``13:40:41.610Z``. + +Two consequences worth stating, because both have bitten callers: + +* The ``_UT`` suffix does **not** mean UTC-normalized. ``CREATION_DATE_UT`` and + ``SUBMIT_DATE_UT`` carry the same local offset as ``LAST_UPDATE``. Treat it as + a naming convention, not a zone promise. +* An **unset** date is the empty string, not ``null``. A parser that only guards + ``None`` raises on real data. + +One deliberate asymmetry, stated here because it looks like an inconsistency +otherwise. On the **read** path :func:`parse_ev_datetime` assumes UTC for a +literal that carries no offset, because an extractor must never fail a record +over one column. On the **write/query** path +:func:`easyvista_python_client.filters._interval_bound` *refuses* the identical +shape, because a mis-zoned interval bound silently moves the window and skips +records — the one failure a watermark must not have, and there a ``ValueError`` +costs the caller nothing but a corrected input. So a naive stamp read back from +an instance becomes a confidently offset-bearing ``datetime``: if a deployment +ever returns offset-less timestamps, that guess is laundered past the filter +guard, and the assumption -- not the guard -- is what to revisit. + +This module is a leaf: it imports nothing from the package, so both ``models/`` +and ``filters.py`` can use it without a cycle. +""" + +from __future__ import annotations + +import re +from datetime import datetime, timezone +from typing import Any + +_FRACTION_RE = re.compile(r"\.(\d+)") + + +def parse_ev_datetime(value: Any) -> datetime | None: + """Parse an EasyVista timestamp to a timezone-aware ``datetime``, or ``None``. + + Accepts a ``datetime`` (returned as-is; a naive one is treated as UTC) or an + ISO-8601 string. Normalizes for Python 3.10's stricter ``fromisoformat``: + maps a trailing ``Z`` to ``+00:00`` and pads/truncates fractional seconds to + 6 digits — EasyVista sends 3, which 3.10 rejects outright. Unparseable input + returns ``None`` rather than raising, so a single malformed column never + fails a whole record. + """ + if isinstance(value, datetime): + return value if value.tzinfo else value.replace(tzinfo=timezone.utc) + if not isinstance(value, str) or not value.strip(): + return None + text = value.strip() + if text.endswith(("Z", "z")): + text = text[:-1] + "+00:00" + match = _FRACTION_RE.search(text) + if match: + frac6 = (match.group(1) + "000000")[:6] + text = text[: match.start()] + "." + frac6 + text[match.end() :] + try: + parsed = datetime.fromisoformat(text) + except ValueError: + return None + return parsed if parsed.tzinfo else parsed.replace(tzinfo=timezone.utc) + + +def format_ev_datetime(value: datetime) -> str: + """Render ``value`` as the literal EasyVista's search grammar accepts. + + Millisecond precision with an explicit offset — byte-identical to what the + API itself returns, and verified live as an accepted interval bound + (``LAST_UPDATE:(2025-11-28T16:14:41.133+01:00;…)`` was honoured). + + Raises ``ValueError`` for a naive datetime. ``ValueError``, not an + ``Easyvista*`` error, because nothing reached the API: this is a local input + fault, the same reasoning as :func:`~easyvista_python_client.escape_ev_value`. + Refusing beats guessing a zone — a naive instant does not name a moment, and + silently assuming UTC would shift every bound by the server's offset. + """ + if value.tzinfo is None: + raise ValueError( + "an EasyVista timestamp must be timezone-aware; a naive datetime " + "does not name a unique instant and would silently shift the bound" + ) + return value.isoformat(timespec="milliseconds") + + +__all__ = ["format_ev_datetime", "parse_ev_datetime"] diff --git a/integration_tests/conftest.py b/integration_tests/conftest.py index 9dc3b77..49011db 100644 --- a/integration_tests/conftest.py +++ b/integration_tests/conftest.py @@ -7,12 +7,16 @@ and no ``EASYVISTA_TEST_*`` environment simply skips the suite rather than failing it. -They are not read-only. A full run creates and closes **20 tickets** (one shared -``rich_ticket``, two ``probe_tickets``, and 17 from ``ticket_factory``), plus 7 -actions, 5 document uploads and 3 updates; ``test_live_smoke`` additionally -issues one create the server is *expected to reject*, so no ticket persists from -it. Every created ticket is registered for cleanup before it is asserted on, and -closed in teardown. Point them at a preprod/test instance, never production. +They are not read-only. A full run creates and closes **21 tickets** (one shared +``rich_ticket``, two ``probe_tickets``, and 18 from ``ticket_factory``), plus 8 +actions, 5 document uploads and **6 to 14 ticket updates** (4 fixed PUTs -- +title, rename, description, external reference -- plus the ``IMPACT_ID`` / +``OWNER_ID`` read-back in the ticket-identity test, which tries up to 5 +candidate values per column and stops at the first the instance accepts, some +of which it may reject outright); ``test_live_smoke`` additionally issues one +create the server is *expected to reject*, so no ticket persists from it. Every +created ticket is registered for cleanup before it is asserted on, and closed +in teardown. Point them at a preprod/test instance, never production. Credentials resolve from an uppercase env var first, then a lowercase file under ``secrets/``: diff --git a/integration_tests/test_live_change_window.py b/integration_tests/test_live_change_window.py new file mode 100644 index 0000000..625bce7 --- /dev/null +++ b/integration_tests/test_live_change_window.py @@ -0,0 +1,799 @@ +"""Characterization of EasyVista's interval, sort and wildcard grammars. + +Established live 2026-08-17 and guarded here. These tests assert *relationships* +— never fixed counts — so they hold on any instance. + +The central discipline: a condition this API cannot honour is **silently +dropped** and the whole table comes back. So a single count can never prove a +range filter works — if the chosen instant sits before every record, an honoured +lower bound *also* returns everything. Every interval assertion below is +therefore a **differential across two instants**: strictly more rows at the +earlier one, and both strictly inside (0, baseline). + +Skipped automatically without credentials; never runs in CI. +""" + +from __future__ import annotations + +import uuid +from datetime import timedelta, timezone +from itertools import pairwise + +import pytest + +from easyvista_python_client import ( + Action, + ActionUpdate, + EasyvistaClient, + EasyvistaValidationError, + PostAction, + ev_between_filter, + ev_contains_filter, + ev_since_filter, + ev_starts_with_filter, + format_ev_datetime, + parse_ev_datetime, +) +from easyvista_python_client._html import html_to_text + +pytestmark = pytest.mark.integration + + +def _count(client: EasyvistaClient, search: str | None = None) -> int: + return client.search_tickets(search=search, max_rows=1).total_record_count or 0 + + +@pytest.fixture(scope="session") +def tickets_baseline(live_client: EasyvistaClient) -> int: + """Unfiltered ticket count — the "condition was dropped" tell.""" + total = _count(live_client) + if total < 4: + pytest.skip("need at least 4 tickets to characterize a change window") + return total + + +@pytest.fixture(scope="session") +def split_instants(live_client: EasyvistaClient) -> tuple[str, str]: + """Two LAST_UPDATE literals with tickets between them, early then late. + + Sampled across four pages because the default order is not chronological, so + one page of a large instance is a biased slice and its quartiles may not + actually split the data. + + Returns the API's own accepted rendering (:func:`format_ev_datetime`), not + a Python ``repr``: ``last_update`` is a parsed, timezone-aware ``datetime`` + (Task 5), so collecting it via ``.model_dump(by_alias=True)`` in "python" + mode hands back the ``datetime`` object itself, not a string -- despite + this fixture's own ``-> tuple[str, str]`` annotation. Interpolating that + object straight into an f-string (as the comparison-operator test below + does) renders Python's ``str(datetime)`` -- a space separator and 6-digit + microseconds -- which is NOT a literal this API accepts. Sorting is done + on the ``datetime`` values themselves (correct under differing UTC + offsets), then each endpoint is rendered to a literal on the way out. + """ + stamps: list = [] + for page in range(4): + result = live_client.search_tickets( + max_rows=200, offset=page * 200, fields=["RFC_NUMBER", "LAST_UPDATE"] + ) + if not result.records: + break + for row in result.records: + if row.last_update is not None: + stamps.append(row.last_update) + if len(stamps) < 8: + pytest.skip("too few LAST_UPDATE values sampled to derive split instants") + stamps.sort() + early_dt, late_dt = stamps[len(stamps) // 4], stamps[(3 * len(stamps)) // 4] + if early_dt == late_dt: + pytest.skip("sampled LAST_UPDATE values do not span two distinct instants") + return format_ev_datetime(early_dt), format_ev_datetime(late_dt) + + +def test_last_update_parses_to_an_aware_datetime(live_client: EasyvistaClient): + """EV-R7: the model must hand back an aware datetime from real data.""" + result = live_client.search_tickets( + max_rows=1, fields=["RFC_NUMBER", "LAST_UPDATE"] + ) + if not result.records: + pytest.skip("no tickets on the live instance") + value = result.records[0].last_update + if value is None: + pytest.skip("sampled ticket has no LAST_UPDATE") + # Bind before asserting: an inline assert would render the whole record (P2). + has_zone = value.tzinfo is not None and value.utcoffset() is not None + assert has_zone, "LAST_UPDATE parsed without a timezone offset" + + +def test_the_open_ended_interval_is_honoured_and_monotone( + live_client: EasyvistaClient, split_instants, tickets_baseline +): + """EV-R5, the decisive one: ``FIELD:(instant;)`` really bounds the result. + + Judged by a differential, never a single count. Only a genuinely applied + lower bound returns strictly fewer rows as the instant moves later while + both counts stay strictly inside the table. + """ + early, late = split_instants + search_early = ev_since_filter("LAST_UPDATE", parse_ev_datetime(early)) + search_late = ev_since_filter("LAST_UPDATE", parse_ev_datetime(late)) + # Two SEPARATE asserts, not one `and`-joined check: a compound boolean + # that fails on the second operand would have pytest's rewriter print + # both operands to explain it, and the other filter string here embeds a + # live instant (P2). Neither can actually fail (`_interval_bound` returns + # a non-empty string or raises), but the channel is closed either way. + assert search_early is not None + assert search_late is not None + + count_early = _count(live_client, search_early) + count_late = _count(live_client, search_late) + + assert count_early > count_late, ( + "the interval was not applied: a later lower bound returned at least as " + "many rows as an earlier one" + ) + assert 0 < count_late, "the later bound matched nothing — instants unusable" + assert count_early < tickets_baseline, ( + "the earlier bound returned the whole table, i.e. the condition was " + "silently dropped" + ) + + +def test_the_closed_interval_is_honoured( + live_client: EasyvistaClient, split_instants, tickets_baseline +): + """The upper bound must narrow further than the lower bound alone. + + A single count strictly inside ``(0, baseline)`` cannot distinguish a + genuinely closed interval from an upper bound that was silently dropped: + if ``FIELD:(early;late)`` degraded to the open-ended ``FIELD:(early;)``, + ``got`` would just equal ``count_early`` -- a value + ``test_the_open_ended_interval_is_honoured_and_monotone`` already proves + sits strictly inside ``(0, baseline)`` on its own, so that check alone + would pass for the wrong reason. The extra ``count_early`` query below is + what actually establishes the upper bound narrows the result further. + """ + early, late = split_instants + search = ev_between_filter("LAST_UPDATE", early, late) + search_since_early = ev_since_filter("LAST_UPDATE", early) + # Two separate asserts, not one `and`-joined check: see the equivalent + # note on the open-ended interval test above (P2). + assert search is not None + assert search_since_early is not None + + got = _count(live_client, search) + count_early = _count(live_client, search_since_early) + + assert 0 < got < tickets_baseline + assert got < count_early, ( + "the closed interval matched at least as many rows as the open-ended " + "lower bound alone -- the upper bound may have been silently dropped" + ) + + +def test_a_comparison_operator_never_narrows_the_result( + live_client: EasyvistaClient, split_instants, tickets_baseline +): + """The negative half, pinned: no comparison operator exists on this API — + but measured live 2026-08-17, it fails two DIFFERENT ways, not one. + + The brief this test started from assumed all three renderings are + "silently dropped" (whole table, no error). Measured against this + instance, only the colon-free rendering actually is: ``LAST_UPDATE>="…"`` + does not match ``FIELD:"value"`` at all, so it is structurally + unparseable and takes the same silent-ignore path as + ``test_bare_sql_like_is_silently_ignored`` in ``test_live_search_syntax.py``. + + The other two DO use ``FIELD:"value"`` syntax, and ``LAST_UPDATE`` is a + date-typed column, so the quoted value must actually parse as one — + embedding ``>=`` or a ``[a TO b]`` range inside the quotes instead trips + the **type-mismatch** fate: a hard ``EasyvistaValidationError`` (HTTP 590). + A CONTROL below isolates that claim: a bare, valid ``LAST_UPDATE`` literal + is asserted to be ACCEPTED (no raise), which is what licenses attributing + the two 590s to the embedded comparison syntax specifically, rather than + to ``FIELD:"value"`` being unusable on this column at all. Without that + control, a future release that started honouring ``>=`` but still + rejected this exact rendering's date shape could keep the raises green + while the claim they guard went false -- the same "prose outran evidence" + failure this task exists to catch, one level down. + ``test_live_search_syntax.py`` documents the same paired shape (bogus vs. + type-correct value) for an int column; this generalizes it to a date + column. Asserting ``== tickets_baseline`` for the two raising cases, as + the original version of this test did, is wrong: it happened to fail + loudly with a 590 rather than passing for the wrong reason, but it was + still pinning a false claim. + + Whichever fate applies, a comparison operator never narrows the result — + it either raises or returns the whole table — so the filter builders' + reason for existing still holds. If a future EasyVista release starts + honouring one of these forms, this test fails and the interval builders + can be simplified. + """ + early, _late = split_instants + + # Control: a bare, valid LAST_UPDATE literal must be ACCEPTED. Called + # outside `pytest.raises` on purpose -- if this column rejected + # `FIELD:"value"` syntax outright, this call would itself raise and the + # test would error here, honestly, rather than mis-attributing that + # rejection to the comparison operator in the two raises below. + control = _count(live_client, f'LAST_UPDATE:"{early}"') + # Strict on BOTH sides, and deliberately so. `0 <= control <= baseline` was + # unfalsifiable -- `_count` never returns a negative, and a filtered count on + # one table cannot exceed the unfiltered one -- so it read as a gate while + # gating nothing, and its failure message described a state that could not + # occur. An exact-instant equality match returns about one row, so the + # strict upper bound additionally proves the literal was HONOURED rather + # than merely not rejected; without it, a future release that started + # silently dropping the condition (and returning the whole table) would + # leave this control green and the 590s below misattributed. + assert 0 < control < tickets_baseline, ( + "a bare valid LAST_UPDATE literal was not honoured as an equality " + "match -- the 590s below can no longer be attributed to the embedded " + "comparison syntax" + ) + + with pytest.raises(EasyvistaValidationError) as excinfo: + _count(live_client, f'LAST_UPDATE:">={early}"') + # Bound first: `excinfo.value.status_code` renders the ExceptionInfo, and + # with it the server's own error prose (P2). + status_code = excinfo.value.status_code + assert status_code == 590 + + with pytest.raises(EasyvistaValidationError) as excinfo: + _count(live_client, f'LAST_UPDATE:"[{early} TO *]"') + status_code = excinfo.value.status_code + assert status_code == 590 + + # Only this rendering breaks FIELD:"value" structure altogether (no + # colon), so it is the one that actually reaches the silent-ignore path. + got = _count(live_client, f'LAST_UPDATE>="{early}"') + # Re-measured now, not the session-cached `tickets_baseline`: one concurrent + # create on the shared instance between that fixture's capture and this + # assertion makes the live unfiltered count `tickets_baseline + 1`, and a + # strict `== tickets_baseline` would fail claiming the opposite of what + # happened -- "a bare comparison operator was honoured" -- for an ordinary + # write elsewhere on the instance. Comparing against a same-instant + # unfiltered re-read keeps the property under test (an unparseable + # comparison operator does not narrow the result) without assuming the + # instance is quiescent. + current_unfiltered = _count(live_client) + assert got == current_unfiltered, ( + "a bare comparison operator was honoured — the interval builders may " + "no longer be the only option" + ) + + +def test_only_some_timestamp_renderings_are_accepted_as_an_interval_bound( + live_client: EasyvistaClient, split_instants, tickets_baseline +): + """Pins the ACCEPTED and REJECTED rendering sets, which normalisation rests on. + + ``_interval_bound`` re-renders every admitted time bound through + ``format_ev_datetime`` rather than passing the caller's string through. That + is only justified if the wire really is this picky, so the matrix is measured + here instead of remembered: a bare date and millisecond-precision-with-offset + (or ``Z``) are honoured, while second precision *with* an offset, minute + precision, and a space separator instead of ``T`` each raise HTTP 590. + + The second-precision case is the one that matters most. It is the most + natural way for a caller to satisfy the "a time bound must carry its offset" + rule -- append ``+02:00`` to a stored ``"2026-08-17T20:26:40"`` watermark -- + and the package's own unit test used to pin it as the canonical shape. If a + future release starts accepting it, normalisation becomes optional and this + test says so. + + Built from raw ``search=`` strings on purpose: the builders now emit only the + honoured rendering, so they cannot express the rejected ones. + + Each honoured rendering is a **differential across two bounds** in that same + rendering, never a single count: a silently dropped condition returns the + whole table, so ``0 < got <= baseline`` would pass on the very fate this + module exists to detect. The later bound must return strictly fewer rows than + the earlier one, which no dropped condition can do -- dropped, both counts are + the baseline. The pairs are built so the strict inequality is guaranteed by + construction: ``split_instants``' early instant is itself in the result set of + the early bound (the lower bound is inclusive) and below the later bound. + """ + early, late = split_instants + moment = parse_ev_datetime(early) + later = parse_ev_datetime(late) + assert moment is not None, "split_instants did not yield a parseable literal" + assert later is not None, "split_instants did not yield a parseable literal" + as_utc = moment.astimezone(timezone.utc) + later_utc = later.astimezone(timezone.utc) + # For the date-only rendering the second bound is the day AFTER the late + # instant, not its own day: on an instance whose sampled stamps all fall on + # one day the two dates would otherwise be equal and the differential empty. + day_after_late = (later.date() + timedelta(days=1)).isoformat() + + honoured = { + "date only": (moment.date().isoformat(), day_after_late), + "milliseconds with offset": ( + format_ev_datetime(moment), + format_ev_datetime(later), + ), + "milliseconds with Z": ( + format_ev_datetime(as_utc).replace("+00:00", "Z"), + format_ev_datetime(later_utc).replace("+00:00", "Z"), + ), + } + for name, (low, high) in honoured.items(): + got_low = _count(live_client, f"LAST_UPDATE:({low};)") + got_high = _count(live_client, f"LAST_UPDATE:({high};)") + # `name` is authored here. The counts are NOT kept out of pytest's + # output by binding them to a local -- pytest's assertion rewriter + # reprints BOTH operands of a comparison even when the assert carries a + # message, so `assert got_low <= tickets_baseline` would still emit the + # live integers. Only binding the comparison itself to a bool closes + # that channel, which is why it is done below (the same idiom + # `is_non_increasing` and `colon_did_not_expand` use elsewhere in this + # module). Aggregate row counts are the one live-derived value this + # module accepts printing (P2) -- record fields, RFCs and timestamps + # are not -- and even that is avoided here where it costs nothing. + bound_was_honoured = 0 < got_low <= tickets_baseline + assert bound_was_honoured, ( + f"the {name!r} rendering was expected to be honoured as an interval " + "bound and returned nothing -- format_ev_datetime may no longer emit " + "a literal this grammar accepts" + ) + later_bound_narrowed = got_high < got_low + assert later_bound_narrowed, ( + f"a later bound in the {name!r} rendering did not return strictly " + "fewer rows than an earlier one -- the condition is being silently " + "dropped and the whole table returned, which is what this rendering " + "being honoured is supposed to rule out" + ) + + rejected = { + # The trap: this is what appending an offset to a naive watermark gives. + "seconds with offset": moment.isoformat(timespec="seconds"), + "minutes with offset": moment.isoformat(timespec="minutes"), + # What `str(aware_datetime)` produces. + "space instead of T": format_ev_datetime(moment).replace("T", " "), + } + for name, literal in rejected.items(): + with pytest.raises(EasyvistaValidationError) as excinfo: + _count(live_client, f"LAST_UPDATE:({literal};)") + # Bind first: rendering the ExceptionInfo would print the server's own + # error prose (P2). + status_code = excinfo.value.status_code + assert status_code == 590, ( + f"the {name!r} rendering failed with an unexpected status; the " + "accepted-rendering set may have changed" + ) + + +def test_descending_sort_needs_the_space_separated_token( + live_client: EasyvistaClient, split_instants +): + """EV-R6: `FIELD DESC` sorts; `FIELD:DESC` is silently ignored. + + Comparing against the UNSORTED order is what makes this meaningful — a + monotonicity check alone cannot distinguish "sorted descending" from "the + default order happens to be descending". + + Measured with the change **window applied**, because that is the claim the + docs rest on: ``ev_since_filter``, the user guide and the search-syntax skill + all now tell a caller to sweep + ``iter_tickets(search=ev_since_filter("LAST_UPDATE", w), sort="LAST_UPDATE + DESC")`` — sort and filter together. ``sort`` has the same silent-failure mode + a search condition has (three token shapes are ignored with no error), so + "DESC is honoured when a ``search`` is also present" has to be measured, not + inferred from the unfiltered case. If this instance dropped ``sort`` whenever + ``search`` was set, that entire remedy would be inert while every surface + asserted it works. + """ + proj = ["RFC_NUMBER", "LAST_UPDATE"] + early, _late = split_instants + window = ev_since_filter("LAST_UPDATE", early) + assert window is not None, "split_instants did not yield a usable window bound" + + def rfcs(sort: str | None) -> list[str | None]: + page = live_client.search_tickets( + search=window, sort=sort, fields=proj, max_rows=20 + ) + return [r.rfc_number for r in page.records] + + def stamps(sort: str | None) -> list: + page = live_client.search_tickets( + search=window, sort=sort, fields=proj, max_rows=20 + ) + return [r.last_update for r in page.records if r.last_update is not None] + + unsorted_order = rfcs(None) + if len(unsorted_order) < 4: + pytest.skip("need at least 4 tickets to characterize sorting") + + descending = stamps("LAST_UPDATE DESC") + # `all(...)` over pairwise is True for a list of 0 or 1 element, and the only + # length guard in this test measures `unsorted_order` -- a DIFFERENT list + # from a different query. Skip rather than assert nothing, the same idiom + # `split_instants` uses for its own sample. + if len(descending) < 2: + pytest.skip("too few LAST_UPDATE values on the sorted page to check order") + is_non_increasing = all(a >= b for a, b in pairwise(descending)) + assert is_non_increasing, "'LAST_UPDATE DESC' did not return newest-first" + reordered = rfcs("LAST_UPDATE DESC") != unsorted_order + assert reordered, "'LAST_UPDATE DESC' returned the default order unchanged" + + # Re-take the unsorted snapshot immediately before comparing, rather than + # reusing `unsorted_order` from three round trips ago. With a window + # applied, a ticket whose LAST_UPDATE sits just below the bound can be + # touched by anyone on the shared instance in those seconds and ENTER the + # filtered set within the first 20 rows -- membership differs, not order, + # which the stale snapshot would misattribute to 'LAST_UPDATE:DESC' now + # reordering results. Before the window was added, set membership was + # stable and only reordering could break this equality; a window makes it + # a race. One retry absorbs a second unlucky write in the same gap before + # this fails for the wrong reason. + colon_order = rfcs("LAST_UPDATE:DESC") + fresh_unsorted = rfcs(None) + colon_ignored = colon_order == fresh_unsorted + if not colon_ignored: + colon_order = rfcs("LAST_UPDATE:DESC") + fresh_unsorted = rfcs(None) + colon_ignored = colon_order == fresh_unsorted + assert colon_ignored, ( + "'LAST_UPDATE:DESC' now reorders results — it used to be silently " + "ignored, and RECENT_TICKETS_SORT was changed on that basis" + ) + + +def test_the_ascending_sort_tokens_are_honoured_too( + live_client: EasyvistaClient, split_instants +): + """Pins the two ASCENDING tokens: bare ``FIELD`` and ``FIELD ASC``. + + **Not the recommended sweep form.** The change-window guidance is + ``LAST_UPDATE DESC`` (see ``ev_since_filter``): under offset pagination an + ascending sweep drops a row whose own stamp did *not* change, which falls + below the next watermark and is lost, while descending drops the re-touched + row, whose stamp is above the watermark and is re-selected. This test exists + only because both tokens' *availability* is a documented fact -- the docs name + them when explaining why the direction is a choice -- and a fact this package + states should be measured rather than remembered. + + Measured with the change window applied, the same reasoning as the DESC test + above, and deliberately paired against the UNSORTED order of the same + filtered query: monotonicity alone cannot tell "sorted ascending" apart from + "the default order happens to be ascending". + """ + proj = ["RFC_NUMBER", "LAST_UPDATE"] + early, _late = split_instants + window = ev_since_filter("LAST_UPDATE", early) + assert window is not None, "split_instants did not yield a usable window bound" + + def page(sort: str | None) -> tuple[list[str | None], list]: + result = live_client.search_tickets( + search=window, sort=sort, fields=proj, max_rows=20 + ) + return ( + [r.rfc_number for r in result.records], + [r.last_update for r in result.records if r.last_update is not None], + ) + + unsorted_order, unsorted_stamps = page(None) + if len(unsorted_order) < 4: + pytest.skip("need at least 4 tickets to characterize sorting") + if len(unsorted_stamps) >= 2 and all(a <= b for a, b in pairwise(unsorted_stamps)): + pytest.skip( + "the default page order is already LAST_UPDATE-ascending on this " + "instance -- cannot distinguish an honoured ascending token from a " + "coincidence" + ) + + for token in ("LAST_UPDATE", "LAST_UPDATE ASC"): + order, stamps = page(token) + if len(stamps) < 2: + pytest.skip("too few LAST_UPDATE values on the sorted page to check order") + is_non_decreasing = all(a <= b for a, b in pairwise(stamps)) + # Bind the token into a local: it is a literal authored here, not a + # value read from the instance, so it is printable under P2. + assert is_non_decreasing, f"{token!r} did not return oldest-first" + reordered = order != unsorted_order + assert reordered, f"{token!r} returned the default order unchanged" + + +def test_recent_tickets_sort_token_is_honoured(live_client: EasyvistaClient): + """The exact constant `get_department_context` relies on (O-DIR-1). + + Comparing against the UNSORTED order is what makes this meaningful, the + same reasoning ``test_descending_sort_needs_the_space_separated_token`` + documents: monotonicity alone cannot tell "sorted descending" apart from + "the default order happens to be descending". If the default page is + itself already RFC-descending, this instance cannot discriminate the two + and the test skips rather than passing for a coincidental reason. + """ + from easyvista_python_client.directory import RECENT_TICKETS_SORT + + proj = ["RFC_NUMBER"] + + def rfcs(sort: str | None) -> list[str]: + page = live_client.search_tickets(sort=sort, fields=proj, max_rows=20) + # Filtered consistently on BOTH the sorted and unsorted side: an RFC-less + # row would otherwise make `sorted_rfcs` and its own re-sorted copy + # differ in length and fail the monotonicity check for the wrong reason. + return [r.rfc_number for r in page.records if r.rfc_number] + + unsorted_order = rfcs(None) + if len(unsorted_order) < 4: + pytest.skip("need at least 4 tickets") + if unsorted_order == sorted(unsorted_order, reverse=True): + pytest.skip( + "the default page order is already RFC-descending on this " + "instance -- cannot distinguish an honoured sort token from a " + "coincidence" + ) + + sorted_rfcs = rfcs(RECENT_TICKETS_SORT) + is_descending = sorted_rfcs == sorted(sorted_rfcs, reverse=True) + # "descending RFC_NUMBER", not "newest-first": RFC_NUMBER is a varchar, so + # this proves a string ordering and nothing about dates. See the comment on + # RECENT_TICKETS_SORT in directory.py. + assert is_descending, ( + f"{RECENT_TICKETS_SORT!r} did not return descending RFC_NUMBER order" + ) + reordered = sorted_rfcs != unsorted_order + assert reordered, ( + f"{RECENT_TICKETS_SORT!r} returned the default order unchanged -- it " + "may be silently ignored" + ) + + +def test_tilde_is_a_wildcard_operator_when_given_a_wildcard( + live_client: EasyvistaClient, tickets_baseline +): + """Corrects this suite's own earlier conclusion that `~` is exact-match. + + That held only for wildcard-free inputs. With an explicit `*`, `~` matches a + prefix or a substring; `:` never does. Anchored on a real RFC so the prefix + demonstrably exists. + """ + page = live_client.search_tickets(max_rows=1, fields=["RFC_NUMBER"]) + if not page.records or not page.records[0].rfc_number: + pytest.skip("no RFC to build a wildcard probe from") + rfc = page.records[0].rfc_number + if len(rfc) < 8: + pytest.skip("RFC too short to form a strict prefix") + prefix = rfc[:6] + + exact = _count(live_client, f'RFC_NUMBER:"{rfc}"') + assert exact == 1 + + by_prefix = _count(live_client, ev_starts_with_filter("RFC_NUMBER", prefix)) + if by_prefix <= exact: + # `exact <= by_prefix` was satisfied by by_prefix == exact == 1, in which + # state the test passed while asserting nothing about `~` being a pattern + # operator -- its own headline claim. The sibling '%' test already treats + # this state as inconclusive and skips; agree with it rather than + # reporting green on a degenerate sample. + pytest.skip( + "the sampled prefix matches no more than the exact RFC on this " + "instance -- cannot demonstrate wildcard expansion" + ) + assert by_prefix < tickets_baseline, ( + "the prefix pattern matched the whole table — '~' with a wildcard is " + "not behaving as a pattern operator" + ) + + by_contains = _count(live_client, ev_contains_filter("RFC_NUMBER", prefix)) + assert by_prefix <= by_contains < tickets_baseline + + # ':' does NOT expand a wildcard — an honest 0, not the whole table. + colon_literal = _count(live_client, f'RFC_NUMBER:"{prefix}*"') + assert colon_literal == 0 + + +def test_every_refused_metacharacter_really_is_one_under_tilde( + live_client: EasyvistaClient, tickets_baseline +): + """Settles whether ``%`` is really a wildcard for ``~`` — measured, not assumed. + + ``ev_contains_filter``/``ev_starts_with_filter`` (``filters.py``) reject a + caller-supplied ``%`` on the premise that it is a wildcard character like + ``*``, but the original probe behind that rejection only ever measured + ``*``. Measured live 2026-08-17 on this instance: ``RFC_NUMBER~"%"`` + and ``RFC_NUMBER~"*"`` matched the identical, non-trivial count (32 + of 4317 tickets), both strictly more than the 1-row exact match and strictly + fewer than the whole table. ``%`` behaves exactly as a wildcard here, so the + builders' rejection of it is justified and should stay as is. + + Extended 2026-08-18 to the other two refused metacharacters, ``_`` and + ``[``, on the same reasoning: the builders reject them, so the rejection + needs live justification. ``_`` is a SINGLE-character wildcard — replacing + one character of an exact-matching RFC with it widens the match — and + ``[0-9]`` in that position is evaluated as a character class, while + ``[x]`` still matches only the one row. A backslash does + not escape ``_``; ``\\_`` matches nothing, which is what makes + refusing the only honest option. + + Built with raw ``search=`` strings rather than the builders themselves, + since ``ev_contains_filter``/``ev_starts_with_filter`` raise ``ValueError`` + on any of these in the caller's value by design — that rejection is the very + thing this test is checking the justification for. + """ + page = live_client.search_tickets(max_rows=1, fields=["RFC_NUMBER"]) + if not page.records or not page.records[0].rfc_number: + pytest.skip("no RFC to build a wildcard probe from") + rfc = page.records[0].rfc_number + if len(rfc) < 8: + pytest.skip("RFC too short to form a strict prefix") + prefix = rfc[:6] + + exact = _count(live_client, f'RFC_NUMBER:"{rfc}"') + assert exact == 1 + + by_star = _count(live_client, f'RFC_NUMBER~"{prefix}*"') + if by_star <= exact: + # A data-availability gap (this sampled prefix happens to be unique + # on this instance), not a defect -- skip rather than fail (P1). The + # sibling tilde test's non-strict `exact <= by_prefix` is the + # precedent for treating "no wider than exact" as inconclusive, not + # wrong. + pytest.skip( + "the sampled prefix's '*' match is no wider than the exact RFC " + "on this instance -- cannot use it as the reference point for " + "the '%' comparison" + ) + assert by_star < tickets_baseline, ( + "the '*' prefix pattern matched the whole table -- cannot use it as " + "the reference point for the '%' comparison" + ) + + by_percent = _count(live_client, f'RFC_NUMBER~"{prefix}%"') + assert by_percent == by_star, ( + "'%' no longer matches the same count as '*' under '~' — it may have " + "stopped behaving as a wildcard, which would justify relaxing the " + "builders' rejection of a caller-supplied '%'" + ) + + # `_` and `[` are probed by REPLACING the RFC's final character, so the + # pattern has the same length as the exact value. Three outcomes, all + # distinguishable: 0 means the character was compared LITERALLY (no RFC + # contains it in that position) and is no longer a metacharacter — that is + # the regression this pins; `== exact` means it behaved as a wildcard but + # this sampled stem has no sibling to widen onto, a data gap the module + # skips on elsewhere; `> exact` is the measured behaviour. + stem, last = rfc[:-1], rfc[-1] + widened_by: dict[str, int] = {} + for probe, name in ( + (f'RFC_NUMBER~"{stem}_"', "_"), + (f'RFC_NUMBER~"{stem}[0-9]"', "[0-9]"), + ): + widened = _count(live_client, probe) + widened_by[name] = widened + # `name` is a literal authored here, never a value read from the + # instance, so it is printable under P2. `stem` is NOT printed. + assert widened > 0, ( + f"{name!r} matched nothing where the exact RFC matches one row -- it " + "is being compared literally, i.e. it is no longer a metacharacter " + "under '~', and the builders' refusal of it could be relaxed" + ) + if widened == exact: + pytest.skip( + f"{name!r} behaved as a pattern but this instance has no other " + "record sharing the sampled stem -- cannot demonstrate widening" + ) + assert widened < tickets_baseline, ( + f"{name!r} matched the whole table, which is what a SILENTLY DROPPED " + "condition also looks like -- inconclusive as evidence" + ) + + # The EXIT the builders' error message, the user guide, the README and the + # asset skill now name: `:` does not expand a wildcard, so an exact match on + # a value containing `_` is expressible even though `~` refuses it. Decisive + # because `~` on this very pattern widened above -- if `:` expanded `_` too, + # this count would match that one instead of being far smaller. + by_colon = _count(live_client, f'RFC_NUMBER:"{stem}_"') + colon_did_not_expand = by_colon < widened_by["_"] + assert colon_did_not_expand, ( + "':' expanded '_' as a wildcard (or the condition was dropped and the " + "whole table came back) -- ev_equals_filter is then NOT the exact-match " + "exit that the wildcard builders' error message and the docs point at " + "for a value containing '_'" + ) + + # A one-character class matching only the real final character must behave + # like the exact match: that is what shows the class is evaluated rather + # than `[0-9]` merely being swallowed into some broader match. + single_class = _count(live_client, f'RFC_NUMBER~"{stem}[{last}x]"') + assert single_class == exact, ( + "a one-character class naming only the real final character did not " + "behave like the exact match -- '[' is not being evaluated as a " + "character class the way the wider '[0-9]' probe suggests" + ) + + # No escape exists: the backslash is compared literally, which is why the + # builders refuse a metacharacter rather than escaping it. Decisive only on + # an RFC that really contains an underscore -- then a WORKING escape would + # match that one row, and a literal backslash matches nothing. + if "_" in rfc: + escaped = _count(live_client, 'RFC_NUMBER~"{}"'.format(rfc.replace("_", "\\_"))) + assert escaped == 0, ( + "a backslash now escapes '_' under '~' — the builders could escape a " + "caller-supplied metacharacter instead of refusing it" + ) + + +def test_update_action_writes_the_description_with_model_dump_casing( + live_client: EasyvistaClient, + live_write_client: EasyvistaClient, + ticket_factory, + live_action_config, +): + """``ActionUpdate.to_api()`` ships lowercase keys — verify that lands live. + + The probe behind :meth:`EasyvistaClient.update_action` edited an action by + sending a raw, hand-built ``{"DESCRIPTION": ...}`` body. ``ActionUpdate`` + instead goes through ``EasyvistaWriteModel.to_api()``, which calls + ``model_dump(exclude_none=True)`` with **no aliasing** — so the body this + client actually ships is lowercase ``{"description": ...}``, a casing + nobody had verified live before this test. Creates exactly one action: + an earlier probe found a *second* ``create_action`` on the same ticket can + fail with HTTP 590. + """ + rfc = ticket_factory() + original_marker = f"EVCLI{uuid.uuid4().hex[:10].upper()}ORIGINAL" + updated_marker = f"EVCLI{uuid.uuid4().hex[:10].upper()}UPDATED" + + before = {a.action_id for a in live_client.list_actions(rfc)} + live_write_client.create_action( + rfc, + PostAction( + action_type_id=int(live_action_config["action_type_id"]), + group_id=int(live_action_config["group_id"]), + description=original_marker, + ), + ) + fresh: list[Action] = [ + a for a in live_client.list_actions(rfc) if a.action_id not in before + ] + # Bound first: `assert len(fresh) == 1` would repr the whole list, i.e. + # every live Action record in it (P2). + exactly_one_new = len(fresh) == 1 + assert exactly_one_new, ( + f"expected exactly 1 new action on {rfc} after creating one, got {len(fresh)}" + ) + action_id = fresh[0].action_id + assert action_id is not None, "listed action carries no ACTION_ID" + + returned = live_client.update_action( + action_id, ActionUpdate(description=updated_marker) + ) + # Characterize the PUT's echo, which had never been captured -- the skill + # snippet used to print a field off it. Two shapes are both acceptable and + # both documented: `update_action` parses through `_first_record_parser`, so + # an empty or href-only body yields an Action whose every field is None, + # while a record-bearing body yields this action. What must NEVER happen is + # the third shape -- an echo naming a DIFFERENT action, which would make the + # return value actively misleading rather than merely sparse. Deliberately + # not asserting `== action_id`: that shape is unverified, which is exactly + # why the docstring and the skill now say to re-read with `get_action`. + # `action_id` is already bound and already interpolated in this test's own + # messages, so echoing it is no new P2 exposure. + echo_names_another_action = ( + returned.action_id is not None and returned.action_id != action_id + ) + assert not echo_names_another_action, ( + f"update_action's echo names an action other than {action_id} -- the " + "return value cannot be treated as the edited record at all" + ) + + action = live_client.get_action(action_id) + href = ( + action.description.get("HREF") if isinstance(action.description, dict) else None + ) + assert href, f"action {action_id} carries no DESCRIPTION href after the update" + text = html_to_text(live_client.resolve_memo(href) or "") + + # Both markers are self-authored nonces, so printing them is fine under P2 + # (they name nothing about the live instance), but bind first anyway to + # keep this module's style uniform. + landed = updated_marker in text + stale = original_marker in text + assert landed, ( + "update_action's lowercase-cased body did not change the DESCRIPTION " + "memo -- ActionUpdate.to_api() ships {'description': ...} with no " + "aliasing, and that casing had never been verified live before this" + ) + assert not stale, "the pre-update marker is still present after the edit" diff --git a/integration_tests/test_live_search_syntax.py b/integration_tests/test_live_search_syntax.py index 51ae124..461af64 100644 --- a/integration_tests/test_live_search_syntax.py +++ b/integration_tests/test_live_search_syntax.py @@ -29,9 +29,11 @@ silently *widens* a same-field query. A ``,`` **inside** the quotes is a literal, so escaping the quote is what blocks it. * **``;`` is not a combinator** — it is swallowed into the quoted value. -* **``~`` is exact-match, not "contains"** — identical to ``:``, on code-like - fields (``DEPARTMENT_CODE``, ``ASSET_TAG``) and free-text label fields - (``DEPARTMENT_FR``) alike. The published docs claiming otherwise are wrong. +* **``~`` is a pattern operator, but only with an explicit wildcard.** + ``FIELD~"abc*"`` matches a prefix and ``FIELD~"*abc*"`` a substring (verified + live 2026-08-17); ``%`` works as a wildcard too. Given a *bare* value it is + identical to ``:`` — exact match — which is why this suite once concluded it + was exact-only. ``:`` never expands a wildcard: ``FIELD:"abc*"`` returns 0. * **No escape for an embedded ``"`` was found.** Raw, backslash-escaped, and doubled-quote renderings of a title containing a literal ``"`` all fail to match a ticket verifiably created with that exact title (full table in @@ -378,13 +380,14 @@ def test_semicolon_is_not_a_combinator(live_client, sample_department_row, basel # --- the tilde operator ---------------------------------------------------- -def test_tilde_is_exact_match_not_contains( +def test_tilde_without_a_wildcard_behaves_as_exact_match( live_client, other_department_code, baseline ): - """``FIELD~value`` behaves identically to ``FIELD:"value"``. - - Decisive probe: a strict infix of a code that verifiably exists. A real - "contains" operator must match that code; exact-match cannot. + """``~`` requires an EXPLICIT wildcard to act as a pattern operator. Given a + bare value it is equality, which is what this test pins. The README's + ``ASSET_TAG~LAPTOP`` therefore finds only a tag that *is* ``LAPTOP``; to + mean "contains", write ``ASSET_TAG~"*LAPTOP*"`` — see + ``test_live_change_window.py`` for that behaviour, verified live 2026-08-17. """ code = other_department_code infix = code[1:] @@ -402,16 +405,17 @@ def test_tilde_is_exact_match_not_contains( assert tilde_infix == 0 -def test_tilde_is_exact_match_on_free_text_fields_too( +def test_tilde_without_a_wildcard_is_exact_on_free_text_too( live_client, department_label, baseline ): - """``~`` is not "contains" on free text either — it is exact everywhere. + """``~`` requires an EXPLICIT wildcard to act as a pattern operator. Given a + bare value it is equality, which is what this test pins. The README's + ``ASSET_TAG~LAPTOP`` therefore finds only a tag that *is* ``LAPTOP``; to + mean "contains", write ``ASSET_TAG~"*LAPTOP*"`` — see + ``test_live_change_window.py`` for that behaviour, verified live 2026-08-17. ``DEPARTMENT_FR`` is a human label, not a code, so this rules out the - "``~`` is contains, but only on free-text fields" hypothesis. The published - docs (``user_guide.rst`` calls ``~`` "contains"; the README advertises - ``ASSET_TAG~LAPTOP``) are wrong: ``~LAPTOP`` matches only a tag that *is* - exactly ``LAPTOP``. + "``~`` is contains, but only on free-text fields" hypothesis. """ label = department_label infix = label[1:-1] @@ -446,12 +450,14 @@ def test_comma_inside_a_quoted_value_is_a_literal( assert inside != baseline # and it is not silently ignored either -def test_tilde_on_asset_tag_is_exact_match(live_client): - """The README's advertised ``ASSET_TAG~LAPTOP`` does not do what it implies. +def test_tilde_without_a_wildcard_is_exact_on_asset_tag(live_client): + """``~`` requires an EXPLICIT wildcard to act as a pattern operator. Given a + bare value it is equality, which is what this test pins. The README's + ``ASSET_TAG~LAPTOP`` therefore finds only a tag that *is* ``LAPTOP``; to + mean "contains", write ``ASSET_TAG~"*LAPTOP*"`` — see + ``test_live_change_window.py`` for that behaviour, verified live 2026-08-17. - Probed on the very endpoint and field the README documents. ``~`` is exact - there too, so ``ASSET_TAG~LAPTOP`` finds only an asset tagged exactly - ``LAPTOP`` — not the laptops. + Probed on the very endpoint and field the README documents. """ try: result = live_client.search_assets(max_rows=25) diff --git a/integration_tests/test_live_smoke.py b/integration_tests/test_live_smoke.py index 8186436..09506b4 100644 --- a/integration_tests/test_live_smoke.py +++ b/integration_tests/test_live_smoke.py @@ -4,10 +4,18 @@ env vars or ``secrets/easyvista_test_*`` files. Never runs in CI (which runs ``pytest -m "not integration"``). NEVER point at production. -No ticket persists from this module: it reads, plus issues a single create that -the server is *expected to reject* (``test_missing_mandatory_field_raises_ -validation_error``). The ticket-creating fixture lives in ``conftest.py`` and is -used by ``test_live_search_syntax``. +This module WRITES. It creates up to three tickets and closes every one: + +* one under-specified create the server is expected to reject -- which still + creates the row (measured: 9 of 9 rejected creates left one), so it is + reconciled by its ``external_reference`` marker and closed. An earlier version + of this file claimed "no ticket persists from this module ... read-only-safe by + construction"; that was wrong and leaked one ticket per live run; +* one create with the full documented body, to prove the ids land; +* one from ``ticket_factory`` for the ``set_status`` check. + +The ticket-creating fixture lives in ``conftest.py`` and is also used by +``test_live_search_syntax``. Every assertion here is by shape, and every one routes through ``_assertions`` or a pre-bound local (design principle P2). pytest's assertion rewriter reports @@ -23,6 +31,8 @@ from __future__ import annotations +import uuid + import pytest from easyvista_python_client import ( @@ -30,9 +40,11 @@ Asset, Document, EasyvistaClient, + EasyvistaError, EasyvistaValidationError, PostRequest, Request, + ev_equals_filter, ) from integration_tests._assertions import assert_shape @@ -88,18 +100,203 @@ def test_classify_fields_live_ticket( assert all("AVAILABLE_FIELD_" in k.upper() for k in fc.available) -def test_missing_mandatory_field_raises_validation_error( - live_write_client: EasyvistaClient, sample_catalog_code: str +def test_an_underspecified_create_body_raises_validation_error( + live_client: EasyvistaClient, + live_write_client: EasyvistaClient, + live_write_config: dict[str, str], + sample_catalog_code: str, +) -> None: + """A create missing the documented ids is rejected 590, not retried as 5xx. + + This test used to send ``PostRequest(catalog_code=...)`` and attribute the + 590 to the missing *title*. Both halves of that were wrong, measured: + + * ``title`` is NOT the mandatory field. The full documented body with no + title at all creates successfully. What the old payload actually omitted + was ``origin``/``department_id``/``urgency_id``/``impact_id``, so the + assertion never tested the thing it named. + * it claimed "no ticket created ... read-only-safe by construction". A + rejected create on this API **does** create the row -- 9 of 9 rejections + left one -- so the old test leaked a ticket on every single live run. + + So the omission is now the documented ids (which really are required here), + and the ticket the rejection leaves behind is reconciled and closed. The + ``external_reference`` marker survives the failed insert and is searchable, + which is the only handle available: no ``RFC_NUMBER`` comes back from a + rejected create. + """ + marker = f"EVCLI{uuid.uuid4().hex[:10].upper()}" + try: + with pytest.raises(EasyvistaValidationError) as ei: + live_write_client.create_ticket( + PostRequest( + catalog_code=sample_catalog_code, + title=marker, + description=f"{marker} under-specified create; safe to close", + external_reference=marker, + ) + ) + # Bound first: asserting on `ei.value.status_code` makes the rewriter + # render the ExceptionInfo, and that prints the exception's own message -- + # server prose this suite did not author (P2). + status_code = ei.value.status_code + assert status_code == 590, "an under-specified create did not raise HTTP 590" + finally: + # Runs even when the create unexpectedly SUCCEEDS, because that outcome + # leaves a ticket too and an assertion failure must not also orphan one. + _close_by_marker(live_client, live_write_config, marker) + + +def test_the_documented_create_body_lands_every_id( + live_client: EasyvistaClient, + live_write_client: EasyvistaClient, + live_write_config: dict[str, str], + sample_catalog_code: str, +) -> None: + """The documented create body is accepted and every id it carries persists. + + The counterpart to the test above, and the reason this suite can trust any + create at all. ``origin``/``department_id``/``urgency_id``/``impact_id`` are + read back and compared to what was sent. + + The read is **explicitly projected**. It has to be: these columns are absent + from the default search projection exactly like ``TITLE``, so an unprojected + read returns ``None`` for all of them and would pass this test by comparing + two absences. + """ + cfg = live_write_config + marker = f"EVCLI{uuid.uuid4().hex[:10].upper()}" + try: + created = live_write_client.create_ticket( + PostRequest( + catalog_code=sample_catalog_code, + title=marker, + description=f"{marker} documented create body; safe to close", + origin=int(cfg["origin"]), + department_id=int(cfg["department_id"]), + urgency_id=int(cfg["urgency_id"]), + impact_id=int(cfg["impact_id"]), + external_reference=marker, + ) + ) + rfc = created.rfc_number + assert rfc, "the documented create body returned no RFC_NUMBER" + rows = live_client.search_tickets( + search=ev_equals_filter("RFC_NUMBER", rfc), + fields=[ + "RFC_NUMBER", + "URGENCY_ID", + "IMPACT_ID", + "REQUEST_ORIGIN_ID", + "EXTERNAL_REFERENCE", + ], + max_rows=1, + ) + record = rows.records[0] if rows.records else None + assert record is not None, "the created ticket was not readable back" + # Every comparison binds a bool before asserting, so a mismatch cannot + # print the instance's own values (P2). + urgency_matches = str(record.urgency_id) == str(cfg["urgency_id"]) + impact_matches = str(record.impact_id) == str(cfg["impact_id"]) + marker_matches = record.external_reference == marker + assert urgency_matches, "URGENCY_ID did not survive the documented create" + assert impact_matches, "IMPACT_ID did not survive the documented create" + assert marker_matches, "EXTERNAL_REFERENCE did not survive the create" + finally: + _close_by_marker(live_client, live_write_config, marker) + + +def test_set_status_reaches_a_non_terminal_status( + live_client: EasyvistaClient, + live_write_client: EasyvistaClient, + live_write_config: dict[str, str], + ticket_factory, ) -> None: - # Creating with a catalog but no title is rejected by EasyVista (no ticket - # created), so this stays read-only-safe by construction. The catalog code - # must be *valid* on this instance: the missing title has to be the only - # defect in the payload, or the 590 can't be attributed to it -- an unknown - # catalog would raise 590 too, and the assertion would prove nothing. - with pytest.raises(EasyvistaValidationError) as ei: - live_write_client.create_ticket(PostRequest(catalog_code=sample_catalog_code)) - # Bound first: asserting on `ei.value.status_code` makes the rewriter render - # the ExceptionInfo, and that prints the exception's own message -- server - # prose this suite did not author, on a payload naming a real catalog (P2). - status_code = ei.value.status_code - assert status_code == 590, "creating without a title did not raise HTTP 590" + """``set_status`` sets an arbitrary status, not only a closing one. + + The API has no flat status update -- ``RequestUpdate`` carries no + ``status_id`` for that reason -- and the ``{"closed": {"status_GUID": ...}}`` + envelope is the only route. Its wire name suggests it only closes; measured, + it reaches every status tried. + + This pins the non-terminal case specifically, because that is the surprising + half and the half a future reader is most likely to "simplify" away. The GUID + is read off the instance rather than hardcoded: status GUIDs are per-instance + configuration, so a literal here would be a value this repo must not carry + and would be wrong on any other deployment anyway. + """ + rfc = ticket_factory() + before = live_client.get_ticket(rfc).status_id + target_guid, target_id = _a_different_status(live_client, exclude=before) + if target_guid is None: + pytest.skip("no second status with a readable GUID on this instance") + + live_write_client.set_status( + rfc, status_guid=target_guid, comment="capability-suite status probe" + ) + after = live_client.get_ticket(rfc).status_id + # Bound as bools: the ids are instance configuration, not suite-authored (P2). + moved = str(after) != str(before) + landed_on_target = str(after) == str(target_id) + assert moved, "set_status did not change the ticket's status" + assert landed_on_target, "set_status landed on a status other than the one asked" + + +def _close_by_marker(client: EasyvistaClient, cfg: dict[str, str], marker: str) -> None: + """Close every ticket carrying ``marker``, however it got there. + + The cleanup a rejected create needs. No ``RFC_NUMBER`` comes back from one, + so the marker is the only handle -- and it does survive the failed insert + (measured). Searching rather than tracking is the point: a tracked list can + only hold ids the caller was given, which is precisely the set that excludes + every orphan. + + Never raises. It runs in a ``finally`` beside the assertion that matters, and + a cleanup failure must not replace a real test result. + """ + try: + found = client.search_tickets( + search=ev_equals_filter("EXTERNAL_REFERENCE", marker), + fields=["RFC_NUMBER"], + max_rows=20, + ) + except EasyvistaError: + return + for record in found.records: + rfc = record.rfc_number + if not rfc: + continue + try: + client.close_ticket( + rfc, status_guid=cfg["status_guid"], comment="smoke cleanup" + ) + except EasyvistaError: + continue + + +def _a_different_status( + client: EasyvistaClient, *, exclude: object +) -> tuple[str | None, str | None]: + """Return ``(status_guid, status_id)`` for some status that is not ``exclude``. + + Read off the instance because status GUIDs are per-instance configuration: a + literal would be a value this repo must not carry, and would be wrong on any + other deployment. Found by sampling tickets and taking the first whose status + differs -- the nested ``STATUS`` object carries both the id and the GUID, + but only on an UNPROJECTED read, so no ``fields`` is passed here. + """ + try: + sampled = client.search_tickets(sort="LAST_UPDATE DESC", max_rows=60) + except EasyvistaError: + return None, None + for record in sampled.records: + status = record.model_extra.get("STATUS") if record.model_extra else None + if not isinstance(status, dict): + continue + sid = status.get("STATUS_ID") + guid = status.get("STATUS_GUID") + if sid is None or not guid: + continue + if str(sid) != str(exclude): + return str(guid), str(sid) + return None, None diff --git a/integration_tests/test_live_ticket_identity.py b/integration_tests/test_live_ticket_identity.py index 2598814..0d0572a 100644 --- a/integration_tests/test_live_ticket_identity.py +++ b/integration_tests/test_live_ticket_identity.py @@ -22,7 +22,12 @@ import pytest -from easyvista_python_client import EasyvistaClient, Request, RequestUpdate +from easyvista_python_client import ( + EasyvistaClient, + EasyvistaValidationError, + Request, + RequestUpdate, +) from integration_tests._assertions import assert_populated, assert_shape pytestmark = pytest.mark.integration @@ -73,6 +78,153 @@ def test_title_is_writable(live_client: EasyvistaClient, ticket_factory): assert title_updated, "TITLE was not changed by RequestUpdate(title=...)" +def _other_live_values( + client: EasyvistaClient, column: str, current: object, limit: int = 5 +) -> list[int]: + """Up to ``limit`` distinct ids in use on ``column``, none equal to ``current``. + + Needed because writing back the value a ticket already carries proves + nothing: ``ticket_factory`` sets ``IMPACT_ID`` from ``live_write_config``, so + a read-back against that same id would pass even if the field were silently + dropped. Sampling ids that genuinely exist on the instance avoids hardcoding + an instance-specific value. + + Several candidates, not one: an id that is in use *somewhere* is not + necessarily legal *here*. ``IMPACT_ID`` and ``OWNER_ID`` are foreign keys + constrained by the ticket's catalog entry, severity matrix and domain, so the + first sampled id may be refused for a ticket ``ticket_factory`` just created. + The caller tries them in order (see :func:`_write_first_accepted`). + + An empty list means the sampled page carries no second value at all, which + the caller turns into a skip rather than a failure. + """ + page = client.search_tickets(max_rows=200, fields=["RFC_NUMBER", column]) + values: list[int] = [] + for record in page.records: + value = getattr(record, column.lower(), None) + if value is None or value == current or value in values: + continue + values.append(value) + if len(values) >= limit: + break + return values + + +def _write_first_accepted( + client: EasyvistaClient, rfc: str, column: str, candidates: list[int] +) -> int | None: + """PUT each candidate to ``column`` until the read-back confirms one landed. + + ``None`` when every candidate was refused, which the caller turns into a + skip. A refusal here means "that id is not assignable to this ticket", not + "``RequestUpdate`` cannot write this column" -- reporting it as the latter + would be a false red, and this test is about the latter only. + + The read-back after every PUT is load-bearing, not defensive style: this + API's documented refusal mode -- restated in + :func:`test_request_update_writes_impact_owner_and_external_reference`'s + own docstring -- is a **200 with the field silently dropped**, not a raised + error. Treating "no exception raised" as "the instance accepted it" would + make the first sampled id -- accepted or not -- the last one tried, and a + silently declined candidate would surface as "RequestUpdate cannot write + this column" instead of "that value is not valid for this ticket": exactly + the false red this helper exists to prevent. + + Only ``EasyvistaValidationError`` is caught around the PUT itself, which is + the transport's mapping of the two statuses a *refused value* can still + raise as (400 and 590). An auth failure, a 404 or a 5xx still reddens, + because none of those means "this id is illegal here". + """ + attr = column.lower() + for candidate in candidates: + try: + client.update_ticket(rfc, RequestUpdate(**{attr: candidate})) + except EasyvistaValidationError: + continue + if getattr(client.get_ticket(rfc), attr, None) == candidate: + return candidate + return None + + +def test_request_update_writes_impact_owner_and_external_reference( + live_client: EasyvistaClient, ticket_factory +): + """The three columns ``RequestUpdate`` gained on this branch, read back. + + Their unit test pins only the emitted body shape -- the client's own + lowercase key names -- which under this branch's measured rule proves + nothing: a 200 on a PUT is not a receipt, and a field the API cannot honour + is silently dropped while the request succeeds. Without this, EasyVista + renaming ``EXTERNAL_REFERENCE`` or declining ``OWNER_ID`` on this verb would + leave the whole suite green and the public API still advertising all three. + The same reasoning produced ``ActionUpdate``'s live guard in + ``test_live_change_window.py``. + + One field per PUT, deliberately. A combined body that came back 200 with one + field dropped would be exactly the failure this test exists to catch, and a + combined body that raised would not say which field caused it. + + A PUT the instance *refuses* is not that failure. ``IMPACT_ID`` and + ``OWNER_ID`` are foreign keys whose legal values depend on the ticket's + catalog entry and domain, so an id sampled off another ticket may be rejected + for this one; that is a skip, not a red. ``EXTERNAL_REFERENCE`` needs no + instance-side legality and stays an unconditional assertion. + + P2: ``reference`` is a self-authored nonce, so it may appear in a message. + The impact and owner ids are read off the instance and must not. + """ + rfc = ticket_factory() + before = live_client.get_ticket(rfc) + + # EXTERNAL_REFERENCE first and unconditionally: it is free text, so no + # instance-side legality can stand between the PUT and the read-back, and + # asserting it before the two foreign keys keeps this guard from being + # skipped past when one of those cannot be established. + reference = f"EVCLI{uuid.uuid4().hex[:10].upper()}REF" # 18 chars; cap is 50 + live_client.update_ticket(rfc, RequestUpdate(external_reference=reference)) + reference_landed = live_client.get_ticket(rfc).external_reference == reference + assert reference_landed, ( + f"EXTERNAL_REFERENCE is not {reference} after " + "RequestUpdate(external_reference=...) -- the field was accepted with a " + "200 and silently dropped" + ) + + # IMPACT_ID and OWNER_ID are foreign keys, and an id in use on another ticket + # is not necessarily assignable to this one, so a refusal is a skip: several + # candidates are tried, and only if none is accepted does the column go + # unmeasured. Failing there would report "RequestUpdate cannot write + # OWNER_ID" when what happened is "that owner is not valid for this ticket". + new_impact = _write_first_accepted( + live_client, + rfc, + "IMPACT_ID", + _other_live_values(live_client, "IMPACT_ID", before.impact_id), + ) + new_owner = _write_first_accepted( + live_client, + rfc, + "OWNER_ID", + _other_live_values(live_client, "OWNER_ID", before.owner_id), + ) + if new_impact is None or new_owner is None: + # P2: no sampled id is named, only the column names authored here. + pytest.skip( + "no sampled IMPACT_ID / OWNER_ID differing from this ticket's own " + "was accepted for it -- cannot distinguish an honoured write from a " + "dropped one" + ) + + after = live_client.get_ticket(rfc) + impact_landed = after.impact_id == new_impact + owner_landed = after.owner_id == new_owner + assert impact_landed, ( + "IMPACT_ID does not match the id sent by RequestUpdate(impact_id=...)" + ) + assert owner_landed, ( + "OWNER_ID does not match the id sent by RequestUpdate(owner_id=...)" + ) + + def test_update_does_not_disturb_the_identifier( live_client: EasyvistaClient, ticket_factory ): diff --git a/integration_tests/test_live_ticket_metadata.py b/integration_tests/test_live_ticket_metadata.py index dc4fb22..0579afc 100644 --- a/integration_tests/test_live_ticket_metadata.py +++ b/integration_tests/test_live_ticket_metadata.py @@ -99,10 +99,11 @@ def test_description_round_trips_through_the_comment_memo( ): # Phase 0 follow-up, verified live: `RequestUpdate.description` writes the # ticket's COMMENT memo, not DESCRIPTION, and a description supplied at - # CREATE time is not readable back through either. On this deployment - # DESCRIPTION is empty on every ticket sampled (0/15, portal-created - # included) while COMMENT is populated on all of them -- so COMMENT is - # where a ticket's body text lives here. This pins the path that works. + # CREATE time is not readable back through either. This pins the path that + # works. It does NOT pin which memo an instance populates: a later pooled + # 77-row sample across four orderings found COMMENT on 57, DESCRIPTION on + # 27 and both on 24, so the earlier "DESCRIPTION empty on 0/15" reading was + # a sampling artifact and must not be generalized (see RequestUpdate). rfc = ticket_factory() body = f"EVCLI{uuid.uuid4().hex[:10].upper()}BODY" live_client.update_ticket(rfc, RequestUpdate(description=body)) diff --git a/pyproject.toml b/pyproject.toml index dc3f997..426eba7 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -44,7 +44,7 @@ exclude = [ [project] name = "easyvista-python-client" -version = "0.1.0" +version = "0.2.0" description = "Typed Python client for the EasyVista Service Manager REST API" readme = "README.md" requires-python = ">=3.10" @@ -94,7 +94,10 @@ dev = [ "sphinx>=7.2,<8.2", "sphinx-rtd-theme>=2.0", "tomli>=2.0; python_version < '3.11'", - "twine>=5.1", + # >=7.0 for Metadata-Version 2.5 support: hatchling >=1.32 stamps 2.5, and + # twine <=6.2 monkeypatches packaging's valid-metadata list to end at 2.4, so + # `twine check` fails a perfectly good wheel and sdist (measured). + "twine>=7.0", # Exact-pinned, not a range: unasync_build.py --check is a byte-equality # gate between _async/ and the checked-in _sync/, and a different # generator version can regenerate byte-different-but-equally-"correct" @@ -254,8 +257,10 @@ fail_under = 95 # # The other two entries are pre-emptive, not repairs: as of this commit the # source contains no `if TYPE_CHECKING:` block and no `raise -# NotImplementedError`, and adding them left the total at 1272 statements / -# 99.21% exactly. They are here so the first one written does not read as a +# NotImplementedError`, so adding them excluded nothing and changed no number. +# (For reference, the gate reports 1437 statements / 99.37% here; treat that as +# a snapshot, not a canary -- it moves with every added line.) They are here so +# the first one written does not read as a # coverage gap -- a TYPE_CHECKING body exists for mypy and cannot execute, and # an unimplemented-stub line is not a line anyone can test. `if TYPE_CHECKING:` # excludes the guard and its whole body. diff --git a/scripts/tests/test_skills_contract.py b/scripts/tests/test_skills_contract.py index 4197399..eab92e6 100644 --- a/scripts/tests/test_skills_contract.py +++ b/scripts/tests/test_skills_contract.py @@ -46,6 +46,7 @@ import pytest import easyvista_python_client as ev +from easyvista_python_client.models.common import EasyvistaWriteModel REPO_ROOT = Path(__file__).resolve().parents[2] SKILLS_DIR = REPO_ROOT / "skills" @@ -193,6 +194,7 @@ def test_readme_lists_every_skill() -> None: "PostRequest": ev.PostRequest, "RequestUpdate": ev.RequestUpdate, "PostAction": ev.PostAction, + "ActionUpdate": ev.ActionUpdate, "PostAsset": ev.PostAsset, "PostDepartment": ev.PostDepartment, "DepartmentUpdate": ev.DepartmentUpdate, @@ -456,3 +458,29 @@ def test_snippet_hosts_are_synthetic(skill: Path) -> None: f"{skill.name} carries a non-synthetic URL {url!r}; every " "host in a skill must sit under example.com" ) + + +def test_write_models_map_is_complete() -> None: + """Every exported EasyvistaWriteModel subclass maps in _WRITE_MODELS. + + The ``_WRITE_MODELS`` dict pairs write model names to their classes so that + snippet keyword validation can find them. A write model exported from the + package but missing from the dict has its snippets silently skipped, leaving + typos and dropped keywords undetected. This test converts the hand-maintained + enumeration into a self-checking gate that fails when a new write model is + exported without a map entry. + """ + exported = { + name + for name in ev.__all__ + if ( + inspect.isclass(obj := getattr(ev, name, None)) + and obj is not EasyvistaWriteModel + and issubclass(obj, EasyvistaWriteModel) + ) + } + mapped = set(_WRITE_MODELS.keys()) + assert exported == mapped, ( + f"exported write models {sorted(exported)} do not match " + f"_WRITE_MODELS {sorted(mapped)}; missing from map: {sorted(exported - mapped)}" + ) diff --git a/scripts/validate_docs_examples.py b/scripts/validate_docs_examples.py index cd31016..ffbb351 100644 --- a/scripts/validate_docs_examples.py +++ b/scripts/validate_docs_examples.py @@ -328,6 +328,7 @@ def signatures() -> None: "add_document": {"rfc_number", "filename", "content"}, "list_documents": {"rfc_number"}, "download_document": {"document"}, + "stream_document": {"document", "chunk_size"}, } problems = [] for method, params in expected.items(): @@ -364,6 +365,7 @@ def async_parity() -> None: "add_document", "list_documents", "download_document", + "stream_document", "from_env", } missing = [m for m in public if not hasattr(AsyncEasyvistaClient, m)] @@ -372,6 +374,11 @@ def async_parity() -> None: assert inspect.isasyncgenfunction(AsyncEasyvistaClient.iter_tickets), ( "AsyncEasyvistaClient.iter_tickets is not an async generator" ) + # And so must stream_document: the user guide documents it as something + # you iterate, which on the async surface means `async for`. + assert inspect.isasyncgenfunction(AsyncEasyvistaClient.stream_document), ( + "AsyncEasyvistaClient.stream_document is not an async generator" + ) r.check( "AsyncEasyvistaClient mirrors sync method names [Sync vs async]", diff --git a/skills/README.md b/skills/README.md index 44aebad..e842a6b 100644 --- a/skills/README.md +++ b/skills/README.md @@ -14,10 +14,10 @@ installed wheel, which carries only the `easyvista_python_client` package. | Skill | Use when the agent needs to | Main public API | | --- | --- | --- | | `easyvista-client-setup` | Build and configure an authenticated client | `EasyvistaConfig`, `EasyvistaClient`, `AsyncEasyvistaClient` | -| `easyvista-search-syntax` | Write or debug any `search=` expression | `ev_equals_filter`, `ev_in_filter`, `escape_ev_value`, `is_safe_ev_value` | +| `easyvista-search-syntax` | Write or debug any `search=` expression, including a date/time window | `ev_equals_filter`, `ev_in_filter`, `ev_contains_filter`, `ev_since_filter`, and 4 more filter builders | | `easyvista-ticket-workflow` | Create, read, search, update or close tickets, or read instance-specific columns off any record | `PostRequest`, `Request`, `RequestUpdate`, `SearchResult`, `Reference`, `FieldClassification` | -| `easyvista-ticket-actions` | Read or write a ticket's action log | `PostAction`, `Action`, `resolve_memo` | -| `easyvista-document-workflow` | Attach, list or download ticket files | `Document`, `add_document`, `download_document` | +| `easyvista-ticket-actions` | Read or write a ticket's action log | `PostAction`, `Action`, `ActionUpdate`, `resolve_memo` | +| `easyvista-document-workflow` | Attach, list, download, stream or delete ticket files | `Document`, `add_document`, `download_document`, `stream_document`, `delete_document` | | `easyvista-asset-workflow` | Create, fetch, search or iterate assets | `PostAsset`, `Asset` | | `easyvista-directory` | Resolve or provision departments and employees | `Department`, `Employee`, `find_departments` | | `easyvista-reporting-and-context` | Count and break down tickets, or build one context bundle | `TicketStatistics`, `aggregate_tickets`, `TicketContext`, `DepartmentContext` | @@ -30,7 +30,8 @@ The package ships two clients with one endpoint surface: `await`, `for` over the iterators. - `AsyncEasyvistaClient` — asynchronous, doing real non-blocking I/O. `async with AsyncEasyvistaClient(config) as client`, `await` every method, - `async for` over the iterators. + `async for` over the iterators and over `stream_document`, which is an async + generator rather than a coroutine. Neither wraps the other. `_async/` is hand-written and `_sync/` is generated from it by `unasync_build.py` under a byte-equality CI gate, so the two surfaces diff --git a/skills/easyvista-asset-workflow/SKILL.md b/skills/easyvista-asset-workflow/SKILL.md index e390baa..3a86349 100644 --- a/skills/easyvista-asset-workflow/SKILL.md +++ b/skills/easyvista-asset-workflow/SKILL.md @@ -5,7 +5,7 @@ license: MIT compatibility: "Requires Python 3.10+, easyvista-python-client, network access to an EasyVista Service Manager REST API, and a profile authorized for the assets resource." metadata: package: easyvista-python-client - version: "0.1.0" + version: "0.2.0" --- > **Sync and async.** Examples use `EasyvistaClient`. For `AsyncEasyvistaClient`, @@ -108,13 +108,40 @@ with EasyvistaClient.from_env() as client: print(asset.asset_id, asset.asset_tag) ``` +```python +from easyvista_python_client import EasyvistaClient, ev_contains_filter + +with EasyvistaClient.from_env() as client: + # A bare '~' is exact match, just like ':' -- ev_contains_filter adds the + # explicit wildcard a partial-tag search needs: ASSET_TAG~"*LAPTOP*". + # The value must carry none of * % _ [ -- "LAPTOP_01" raises ValueError. + found = client.search_assets(search=ev_contains_filter("ASSET_TAG", "LAPTOP")) + print(found.total_record_count) +``` + ## Gotchas - `catalog_id` is required by EasyVista and is an `int`; `get_asset` takes a `str` id. The asymmetry is real. -- `ASSET_TAG` filters as **exact match** with `~` as well as `:` — there is - no substring search for a partial tag - (`integration_tests/test_live_search_syntax.py::test_tilde_on_asset_tag_is_exact_match`). +- `ASSET_TAG~"LAPTOP"` (a bare value, no wildcard) is **exact match**, + identical to `ASSET_TAG:"LAPTOP"` — `~` degenerates to equality without an + explicit wildcard. For a partial-tag search use `ev_contains_filter` / + `ev_starts_with_filter`, which add the wildcard for you: + `ev_contains_filter("ASSET_TAG", "LAPTOP")` builds `ASSET_TAG~"*LAPTOP*"` + (verified live + `integration_tests/test_live_search_syntax.py::test_tilde_without_a_wildcard_is_exact_on_asset_tag`; + see `easyvista-search-syntax` for the full grammar). +- **An asset tag containing `_` or `[` cannot go through the pattern builders.** + `*` and `%` are not the only metacharacters under `~`: `_` matches any single + character and `[` opens a character class (measured live), and there is no + escape — a backslash is compared literally. So `ev_contains_filter` / + `ev_starts_with_filter` raise `ValueError` for a value containing any of + `* % _ [`, and `_` is pervasive in asset tags: `ev_contains_filter("ASSET_TAG", + "LAPTOP_01")` raises rather than also matching `LAPTOP-01` and `LAPTOP001` with + HTTP 200 and no hint. For an **exact** match on such a tag use + `ev_equals_filter("ASSET_TAG", "LAPTOP_01")` — `:` does not expand a wildcard. + To pattern-match around one, filter server-side on a wider condition and + compare exactly in Python. - The `Asset` model declares only `asset_id`, `asset_tag`, `serial_number`, `status_id` and `href`; everything else the instance returns is preserved by `extra="allow"` and reachable through `classify_fields()`. `reference()` diff --git a/skills/easyvista-client-setup/SKILL.md b/skills/easyvista-client-setup/SKILL.md index 37ed8d1..55c75a6 100644 --- a/skills/easyvista-client-setup/SKILL.md +++ b/skills/easyvista-client-setup/SKILL.md @@ -5,7 +5,7 @@ license: MIT compatibility: "Requires Python 3.10+, easyvista-python-client, network access to an EasyVista Service Manager REST API, and valid EasyVista credentials." metadata: package: easyvista-python-client - version: "0.1.0" + version: "0.2.0" --- `easyvista_python_client` ships both clients over one surface: @@ -14,6 +14,10 @@ and does real non-blocking I/O. Same method names, same arguments, same results. Pick the one matching the runtime — synchronous script or async event loop — and keep it consistent within one application. +Two exceptions to "returns coroutines": the `iter_*` methods and +`stream_document` are **async generators** on the async client. Iterate them +with `async for`; `await client.stream_document(...)` raises `TypeError`. + ## Procedure 1. Pick the client class: `EasyvistaClient` for synchronous code, diff --git a/skills/easyvista-directory/SKILL.md b/skills/easyvista-directory/SKILL.md index 4d93806..dc4d60c 100644 --- a/skills/easyvista-directory/SKILL.md +++ b/skills/easyvista-directory/SKILL.md @@ -5,7 +5,7 @@ license: MIT compatibility: "Requires Python 3.10+, easyvista-python-client, network access to an EasyVista Service Manager REST API, and a profile authorized for the departments and employees resources (writes are additionally profile-gated)." metadata: package: easyvista-python-client - version: "0.1.0" + version: "0.2.0" --- > **Sync and async.** Examples use `EasyvistaClient`. For `AsyncEasyvistaClient`, diff --git a/skills/easyvista-document-workflow/SKILL.md b/skills/easyvista-document-workflow/SKILL.md index 023bbb1..54e80c5 100644 --- a/skills/easyvista-document-workflow/SKILL.md +++ b/skills/easyvista-document-workflow/SKILL.md @@ -1,22 +1,25 @@ --- name: easyvista-document-workflow -description: "Attach, list and download files on an EasyVista ticket with easyvista_python_client — add_document, list_documents and download_document with the Document model. Use for ticket attachments, uploading evidence or logs to a request, or fetching an attachment's bytes." +description: "Attach, list, download, stream and delete files on an EasyVista ticket with easyvista_python_client — add_document, list_documents, download_document, stream_document and delete_document with the Document model. Use for ticket attachments, uploading evidence or logs to a request, fetching an attachment's bytes whole or chunk by chunk without buffering a large file, or removing one." license: MIT compatibility: "Requires Python 3.10+, easyvista-python-client, network access to an EasyVista Service Manager REST API, and a profile authorized for the documents sub-resource." metadata: package: easyvista-python-client - version: "0.1.0" + version: "0.2.0" --- > **Sync and async.** Examples use `EasyvistaClient`. For `AsyncEasyvistaClient`, > use `async with`, `await` every call, and `async for` over the `iter_*` -> methods — the method names and arguments are identical. See +> methods and `stream_document` — the method names and arguments are identical. +> `stream_document` is an async generator, so `await client.stream_document(...)` +> is a `TypeError`; iterate it. See > `easyvista-client-setup`. -Documents are attachments on a ticket. Three methods: `add_document(rfc, -filename=, content=)`, `list_documents(rfc)`, `download_document(document)`. -All are ticket-scoped — there is no standalone document resource on this -client. +Documents are attachments on a ticket. Five methods: `add_document(rfc, +filename=, content=)`, `list_documents(rfc)`, `download_document(document)`, +`stream_document(document, chunk_size=)` and `delete_document(rfc, +document_id)`. All are ticket-scoped — there is no standalone document +resource on this client. ## Procedure @@ -25,7 +28,13 @@ client. 2. List with `list_documents(rfc)` → `list[Document]`. 3. Download with `download_document(document)` → `bytes`. Pass the `Document` from the list, or a raw href/path. -4. Write the bytes yourself; the client does not touch the filesystem. +4. For a large attachment, iterate `stream_document(document)` instead → byte + chunks (64 KiB by default, `chunk_size=` to change it). Same accepted + inputs and same URL resolution as `download_document`; the file never has + to exist in memory whole. +5. Write the bytes yourself; the client does not touch the filesystem. +6. Remove an attachment with `delete_document(rfc, document.document_id)`. It + returns nothing (the API answers with an empty body) — re-list to confirm. ## The Document model @@ -36,6 +45,7 @@ observed shape), `name`, `document`, `document_id`, `download_href` (the API's `download_document` resolves the URL as `download_href or href` — it prefers `DDL_HREF` and **falls back to `HREF`**. Either field on its own is enough, so a record with an empty `download_href` may still be perfectly downloadable. +`stream_document` resolves it exactly the same way. ## Examples @@ -76,21 +86,82 @@ with EasyvistaClient.from_env() as client: Path(document.filename or "attachment.bin").write_bytes(payload) ``` +```python +from pathlib import Path + +from easyvista_python_client import EasyvistaClient + +# Streaming: the bytes go straight to disk, so a 32 MB attachment never sits +# in memory whole. Only the download streams -- see the Gotchas on upload. +with EasyvistaClient.from_env() as client: + for document in client.list_documents("YOUR_RFC_NUMBER"): + target = Path(document.filename or "attachment.bin") + with target.open("wb") as sink: + for chunk in client.stream_document(document, chunk_size=1024 * 1024): + sink.write(chunk) +``` + +```python +from easyvista_python_client import EasyvistaClient + +with EasyvistaClient.from_env() as client: + rfc = "YOUR_RFC_NUMBER" + documents = client.list_documents(rfc) + # DELETE requests/{rfc}/documents/{document_id} -- nested on the ticket, + # like every other document operation. Returns nothing on success. + client.delete_document(rfc, documents[0].document_id) +``` + ## Gotchas - `content` must be `bytes`. Read files in binary mode. +- **Upload cannot stream, and that is the API's constraint, not a gap here.** + EasyVista takes an attachment as base64 inside a JSON body, so + `add_document` has to materialise the whole payload before it can send + anything. Only the download direction has a chunked form. +- **`stream_document` does not retry a mid-stream failure.** Opening the + download is retried under the usual policy, but from the first chunk onwards + the request is committed and a transport failure raises + `EasyvistaConnectionError` instead of starting over — restarting would hand + you bytes you already have. Nothing resumes a partly consumed stream, so + either discard what you collected and stream again, or use + `download_document`, which retries the whole fetch, when the file is small + enough to buffer. +- `stream_document` is a generator: nothing is requested until you start + iterating, so a `ValueError` for a missing URL or an `EasyvistaError` for a + foreign one surfaces on the first step, not at the call. A non-positive + `chunk_size` raises `ValueError` there too, rather than escaping as an httpx + internal error. +- **Stopping early on the async client needs an explicit close.** After + `break`ing out of an `async for`, the response stays checked out of the + connection pool until the event loop's async-generator finalizer runs — a + garbage-collection cycle later. Use + `contextlib.aclosing(client.stream_document(doc))` or call `aclose()`, or a + prefix-reading fan-out under a bounded `max_connections` will stall on + connections it looks like it released. The sync client releases at the `break`. +- **Streamed bytes are not proof of instance origin.** An absolute URL in a + response *body* is refused when it names another host, but an HTTP *redirect* + off the instance is followed (signed-location hops need it); the credential is + dropped on the foreign request, and the foreign bytes are returned as the + attachment's content. +- It is `stream_document`, not `iter_document`: every `iter_*` method on this + client iterates *records*, and this one iterates the bytes of one document. - `download_document` raises `ValueError` only when **neither** `DDL_HREF` nor `HREF` is set. Guard on both (`download_href is None and href is None`), or catch the `ValueError`. Skipping a record because `download_href` alone is unset silently drops attachments the client would have fetched through `href`. - A download URL pointing outside the configured instance raises - `EasyvistaError`. Downloads follow redirects (signed URLs are common), and - httpx strips the `Authorization` header on a cross-origin redirect, so a - foreign host would receive the request unauthenticated — refusing is - deliberate. -- A 403 on an attachment still surfaces as `EasyvistaAuthError`; the binary - path reuses the same error mapping and retry policy as the JSON one. + `EasyvistaError`, on the streaming path as well as the buffered one. + Downloads follow redirects (signed URLs are common), and httpx strips the + `Authorization` header on a cross-origin redirect, so a foreign host would + receive the request unauthenticated — refusing is deliberate. +- A 403 on an attachment still surfaces as `EasyvistaAuthError`; both binary + paths reuse the same error mapping and the same retry policy as the JSON one. - `filename` is derived, not always sent by the API. Fall back to a literal name before writing to disk. -- There is no delete-document method on this client. +- `delete_document(rfc, document_id)` is **ticket-scoped**: it calls the + nested `DELETE requests/{rfc}/documents/{document_id}`. Both identifiers + must be non-blank — a blank one would silently address the collection + rather than one item, which `delete_document` refuses with `ValueError` + before sending anything. diff --git a/skills/easyvista-reporting-and-context/SKILL.md b/skills/easyvista-reporting-and-context/SKILL.md index 58361f6..b2c002e 100644 --- a/skills/easyvista-reporting-and-context/SKILL.md +++ b/skills/easyvista-reporting-and-context/SKILL.md @@ -5,7 +5,7 @@ license: MIT compatibility: "Requires Python 3.10+, easyvista-python-client, and network access to an EasyVista Service Manager REST API." metadata: package: easyvista-python-client - version: "0.1.0" + version: "0.2.0" --- > **Sync and async.** Examples use `EasyvistaClient`. For `AsyncEasyvistaClient`, @@ -38,6 +38,12 @@ one call, degrading around profile restrictions instead of failing. `CREATION_DATE_UT`, accepting a `datetime` or an ISO-8601 string, applied client-side. A ticket with a missing or unparseable date is excluded when a bound is set. A malformed bound string raises `ValueError`. +- An **offset-less** bound string is interpreted as **UTC**, not instance-local. + On a `+02:00` instance, `created_since="2026-01-01T00:00:00"` silently excludes + tickets created between 00:00 and 02:00 local on 1 January. Pass an aware + `datetime` or an offset-bearing string. (This client-side filter is + deliberately more permissive than `ev_since_filter`, which refuses an + offset-less time outright.) ## Context bundles @@ -166,13 +172,20 @@ with EasyvistaClient.from_env() as client: the async surface lets siblings already in flight settle before the error propagates, so a failing call can issue more requests than the sync surface would. -- `recent_tickets` ordering is best-effort: it depends on the server - honouring the descending-sort token `RECENT_TICKETS_SORT`, and that - dependency — like the assumption that an unknown `sort` falls back to the - default order rather than erroring — is not confirmed against a live - instance (open item O-DIR-1; `easyvista-search-syntax` hedges the same - claim). Until checked against your own instance, treat the ordering as - unconfirmed rather than guaranteed descending. +- `recent_tickets` is genuinely sorted **by descending `RFC_NUMBER`**: + `RECENT_TICKETS_SORT` uses the space-separated descending token + (`RFC_NUMBER DESC`), which EasyVista honours — verified live 2026-08-17 by + `integration_tests/test_live_change_window.py` (closes open item O-DIR-1). + That is newest-first only where RFC numbers are issued monotonically. It is a + varchar (`I240101_0001`), so a descending *string* sort orders by the + request-type prefix first: on an instance issuing more than one prefix letter, + every `R…` ticket outranks every `I…` ticket regardless of date. What is + measured is the descending-ness, not the recency — sort a date column yourself + if you need a date guarantee. + A colon-separated token (`RFC_NUMBER:DESC`), `-RFC_NUMBER` and + `DESC(RFC_NUMBER)` are all silently ignored instead, falling back to the + server's default order with no error — that was the earlier, unconfirmed + form this constant used to rely on. - `TicketContext.to_markdown()` titles the body "Description" whichever memo carried it, and emits both headings only when both memos have text. Do not parse the heading to infer the source field — read `context.description` / diff --git a/skills/easyvista-search-syntax/SKILL.md b/skills/easyvista-search-syntax/SKILL.md index 146ec10..af697a8 100644 --- a/skills/easyvista-search-syntax/SKILL.md +++ b/skills/easyvista-search-syntax/SKILL.md @@ -1,11 +1,11 @@ --- name: easyvista-search-syntax -description: "Write correct EasyVista server-side search expressions for search_tickets, iter_tickets, count_tickets, search_assets, search_departments and search_employees using ev_equals_filter, ev_in_filter, escape_ev_value and is_safe_ev_value. Use whenever building a search= argument, filtering EasyVista records, or debugging a filter that returned everything or nothing — EasyVista silently ignores conditions it cannot honour and returns the whole table." +description: "Write correct EasyVista server-side search expressions for search_tickets, iter_tickets, count_tickets, search_assets, search_departments and search_employees using ev_equals_filter, ev_in_filter, ev_contains_filter, ev_starts_with_filter, ev_since_filter, ev_between_filter, escape_ev_value and is_safe_ev_value. Use whenever building a search= argument, filtering EasyVista records, filtering by a date/time window, or debugging a filter that returned everything or nothing — EasyVista silently ignores conditions it cannot honour and returns the whole table." license: MIT compatibility: "Requires Python 3.10+, easyvista-python-client, and network access to an EasyVista Service Manager REST API." metadata: package: easyvista-python-client - version: "0.1.0" + version: "0.2.0" --- > **Sync and async.** Examples use `EasyvistaClient`. For `AsyncEasyvistaClient`, @@ -15,22 +15,47 @@ metadata: Every `search_*` and `iter_*` method takes the same `search` string. The grammar is small and two of its three failure modes are silent, so this skill -is a prerequisite for any filtering work. Except where a claim below is -explicitly flagged as unconfirmed, everything here was characterized against a -live instance by `integration_tests/test_live_search_syntax.py` — that file is -the authority when something here looks wrong. +is a prerequisite for any filtering work. Everything here was characterized +against a live instance by `integration_tests/test_live_search_syntax.py` +(the base grammar) and `integration_tests/test_live_change_window.py` (the +interval, wildcard and sort grammars) — those files are the authority when +something here looks wrong. ## The grammar - `FIELD:"value"` — exact match. -- `~` is a **synonym for `:`** — exact match, not "contains", on code-like - fields (`DEPARTMENT_CODE`, `ASSET_TAG`) and on free-text label fields - (`DEPARTMENT_FR`) alike. Vendor documentation claiming otherwise is wrong. - **No substring operator has been identified.** -- `%` inside a value is a **literal character**, not a wildcard. +- `~` is a **pattern operator**, not a synonym for `:`. It only behaves like + "contains" or "starts with" when the value carries an **explicit** wildcard: + `*` and `%` both expand (`FIELD~"*abc*"` substring, `FIELD~"abc*"` prefix — + verified live 2026-08-17, and `%` reproduces the same match count as `*`). + Given a **bare** value with no wildcard, `~` degenerates to exact match — + identical to `:` — which is why this skill previously documented it as + exact-match-only; that conclusion held only for the wildcard-free inputs it + was tested with. `:` never expands a wildcard, even when one is present in + the value: `FIELD:"abc*"` matches nothing. Use `ev_contains_filter` / + `ev_starts_with_filter` rather than building the pattern by hand. +- `*` and `%` are not the only metacharacters under `~`. `_` matches any + **single** character and `[` opens a character class — measured live, + replacing one character of an RFC that matched 1 row with `_`, or with + `[0-9]`, matched 9. There is **no escape**: `\_` returned 0 rows, i.e. the + backslash is compared literally. `ev_contains_filter` / + `ev_starts_with_filter` therefore raise `ValueError` for a value containing + any of `* % _ [`, because silently matching more rows is worse than failing. + This bites on ordinary input, not exotic input: `_` is pervasive in EasyVista + codes, and `ev_contains_filter("ASSET_TAG", "LAPTOP_01")` would otherwise also + match `LAPTOP-01` and `LAPTOP001` with HTTP 200 and no hint. For an **exact** + match on such a value use `ev_equals_filter` — `:` does not expand a wildcard, + so a `_` in the value is compared literally there. Only if you need to + pattern-match *around* a literal `_` are you stuck: filter server-side on a + wider condition and match exactly in Python. - `,` combines conditions: **OR** when every condition names the same field, **AND** across different fields. - `;` is **not** a combinator; it is swallowed into the quoted value. +- There is **no comparison operator** (`>=`, `BETWEEN`, `[a TO b]`…). Writing + one fails one of two different ways depending on its exact shape — see fate + 3 below — never by narrowing the result. Use `ev_since_filter` / + `ev_between_filter` for a date/time window instead (see "Filtering by a + change window"). - There is **no escape for a `"` inside a value**. Raw, backslash-escaped and doubled renderings were all tested against a ticket verifiably created with a quote in its title; none matched. @@ -41,18 +66,93 @@ the authority when something here looks wrong. 2. **Silently dropped** — no error. EasyVista removes any condition it cannot honour and applies what is left; with nothing left, it returns **every** row. This happens for structurally unparseable input - (`DEPARTMENT_FR LIKE "%TECH%"`, bare garbage), for an unknown field, and - for a well-formed condition on a returned-but-unsearchable field. Dropping - is **per condition**: in a two-condition search, one can be honoured while - the other vanishes. + (`DEPARTMENT_FR LIKE "%TECH%"`, bare garbage, a colon-free comparison like + `LAST_UPDATE>="2026-01-01"`), for an unknown field, and for a well-formed + condition on a returned-but-unsearchable field. Dropping is **per + condition**: in a two-condition search, one can be honoured while the + other vanishes. 3. **Rejected outright** — `EasyvistaValidationError` (HTTP 590) when the value's *type* does not match the column, e.g. sending a status name to - the integer `STATUS_ID`. This is the friendly failure. + the integer `STATUS_ID`. This is the friendly failure. A comparison + operator embedded *inside* `FIELD:"value"` syntax lands here too — + `LAST_UPDATE:">=2026-01-01"` and `LAST_UPDATE:"[2026-01-01 TO *]"` both + raise HTTP 590, because the quoted text must still parse as `LAST_UPDATE`'s + date type. So a comparison operator has **two** fates, not one: drop the + `FIELD:` colon and it is silently dropped (fate 2); keep the colon and + embed the operator in the value and it is a type mismatch (fate 3). + Neither ever narrows the result. The counter-intuitive case: a **broken quote does not** return the table. `DEPARTMENT_CODE:"X""` still parses as a field expression, the value swallows the junk, and it matches nothing (0 rows). +## Filtering by a change window + +There is no comparison operator, so a range is an interval in the *value* +position: `ev_since_filter("LAST_UPDATE", watermark)` builds +`LAST_UPDATE:(;)`, an open-ended lower bound; `ev_between_filter` +builds a closed `LAST_UPDATE:(a;b)`. Pass a `datetime` (preferred, and what a +`Request` timestamp field already is) or a timestamp string — either bound is +validated as a real timestamp because it is interpolated **unquoted**, so a +stray `;` or `)` inside it would silently change the query rather than being +escaped away. + +**A bound naming a time must carry its UTC offset**, and both builders refuse one +that does not — as a `datetime` or as a string. EasyVista *accepts* an +offset-less literal and reads it in another zone, moving the bound later and +skipping records with no error (measured live: 13 rows with the offset, 11 +without, same wall-clock text). A bare date is fine; it has no time to misplace. + +An admitted string bound naming a **time** is re-rendered to millisecond +precision with an offset, because that is the only time rendering the wire +honours: `LAST_UPDATE:(2025-11-28T16:14:41+01:00;)` — second precision with an +offset, the most obvious way to satisfy the rule above — is **HTTP 590**, as are +minute precision and a space instead of `T` (what `str(aware_datetime)` +produces). So the string and `datetime` paths emit identical bounds; do not +hand-build the literal. + +The lower bound is **inclusive** and milliseconds are honoured (verified live on +three independent boundaries), so a watermark taken as `max(t.last_update)` +re-reads the boundary record on the next sweep. De-duplicate by `rfc_number`. + +**Sort a sweep `LAST_UPDATE DESC`, and de-duplicate.** `iter_*` walks the result +set by *offset*, and the rows a change window selects are by construction the +rows that are changing, so a ticket touched between two pages moves *within the +set being paged*. An unsorted sweep can drop such a row silently — and so can +either sort direction. What differs is where the dropped row's own timestamp +lands relative to the watermark this sweep records: + +- **`LAST_UPDATE DESC`**: the re-touched row jumps to the head, behind the read + cursor, so this sweep misses it — but its stamp is now *above* the watermark, + so the next sweep selects it again. **Deferred, self-healing.** +- **`LAST_UPDATE` / `LAST_UPDATE ASC`**: the re-touched row moves to the tail and + everything behind it shifts one place head-ward, so the row that crosses the + cursor is one whose own stamp did **not** change. It falls *below* the new + watermark and no later sweep selects it. **Permanent miss.** + +Both tokens are honoured (measured live); descending is chosen for the reason +above. De-duplicate by `rfc_number` — the duplicates are the deferred rows +arriving on a later sweep, plus the inclusive-boundary re-read. + +**A sweep that never finishes is a separate trap.** `DESC` yields the newest +row first, so the watermark reaches its *final* value on page 1. A sweep that +is interrupted, or capped with `max_records` (as some pagination examples in +this repo do), still ends up holding the newest stamp — advance the watermark +from that and the next window's `(newest;)` bound permanently excludes every +row the incomplete sweep never read. Only advance the watermark after a sweep +runs to completion. + +If even a deferred miss is unacceptable, do not use `iter_*`: page +`search_tickets` yourself with **keyset** pagination — sort ascending and, after +each page, advance the *window* to `ev_since_filter(field, max(stamps on the +page))` at `offset=0` instead of incrementing an offset. With no offset there is +no cursor for a row to shift past. `iter_tickets` cannot express this because it +owns its own offset. + +(An earlier version of this skill recommended ascending, reasoning that it turns +a permanent miss into a duplicate. That was wrong: the row an ascending sweep +drops is not the re-touched one.) + ## What is searchable Only **top-level scalar columns**. Two families are returned but not @@ -64,6 +164,15 @@ searchable, and naming one matches everything: inside `STATUS`) — they are not top-level columns at all. Filter `STATUS_ID`. +A **dotted path across a relation** is the exception and IS honoured in +`search`: `REQUEST.RFC_NUMBER:""` on `/actions` genuinely scopes, and it is +what `list_actions` is built on (pinned by +`integration_tests/test_live_ticket_history.py::test_list_actions_filters_to_the_requested_ticket`). +What is silently ignored is a **bare** nested sub-key (`STATUS_EN`) and a +`*_PATH` display column — not the dotted form. Note `fields` does not accept the +dotted form even where `search` does: a projection like `DESCRIPTION.HREF` is +silently dropped. + The rule is about **nesting, not language**: `DEPARTMENT_FR` is a top-level column on `departments` and filters correctly. `CATALOG_GUID` is not an instance of this rule — it is not returned at all, so it is merely an unknown @@ -72,9 +181,11 @@ ids are instance-specific. ## Procedure -1. Build every filter with `ev_equals_filter` / `ev_in_filter`. Never +1. Build every filter with a helper: `ev_equals_filter` / `ev_in_filter` for + exact match, `ev_contains_filter` / `ev_starts_with_filter` for a pattern, + `ev_since_filter` / `ev_between_filter` for a date/time window. Never f-string a value into a `search`. -2. Handle `None`: both builders return `None` for a blank or missing value, +2. Handle `None`: every builder returns `None` for a blank or missing value, so `search=None` means unfiltered — guard when that is not what you want. 3. Call `is_safe_ev_value(value)` first when you would rather skip a filter than raise; `escape_ev_value` raises `ValueError` on a value containing @@ -153,6 +264,38 @@ with EasyvistaClient.from_env() as client: result = client.find_departments(user_supplied, limit=10) ``` +```python +from easyvista_python_client import EasyvistaClient, ev_contains_filter + +with EasyvistaClient.from_env() as client: + # A bare '~' is exact match; the wildcard is what makes it "contains". + result = client.search_assets(search=ev_contains_filter("ASSET_TAG", "LAPTOP")) + print(result.total_record_count) +``` + +```python +from easyvista_python_client import EasyvistaClient, ev_since_filter + +with EasyvistaClient.from_env() as client: + ticket = client.get_ticket("I240101_0001") + # ticket.last_update is already an aware datetime -- feed it straight back + # in as a watermark for "everything changed since this ticket". + search = ev_since_filter("LAST_UPDATE", ticket.last_update) + if search is not None: + # The sort direction is load-bearing. Descending on the filtered + # column defers a mid-sweep miss to the next sweep (the row's stamp + # ends up above the watermark); ascending loses it for good. Hence + # the de-duplication -- see "Filtering by a change window". + seen = set() + for changed in client.iter_tickets( + search=search, sort="LAST_UPDATE DESC", page_size=100 + ): + if changed.rfc_number in seen: + continue + seen.add(changed.rfc_number) + print(changed.rfc_number) +``` + ## Gotchas - A `,` reaching the server inside untrusted input **widens** a same-field @@ -161,12 +304,15 @@ with EasyvistaClient.from_env() as client: `escape_ev_value` does. - `ev_equals_filter` returns `None` for a blank value; passing that straight through as `search=` silently means "no filter". -- An unknown `sort` token is believed to be ignored, not rejected, falling - back to the default order — but unlike the rest of this skill, that is - **not** covered by the live suite. It is what +- The sort token must be **space-separated**: `FIELD DESC` (or `field desc`) + genuinely reorders the result, and bare `FIELD` / `FIELD ASC` both sort + ascending — verified live by + `integration_tests/test_live_change_window.py`. `FIELD:DESC`, `-FIELD` and + `DESC(FIELD)` are all silently ignored — the query falls back to the + server's default order with no error, so a sweep written with one of those + looks sorted and is not. Nothing validates the token locally. This is what `easyvista_python_client/directory.py`'s `RECENT_TICKETS_SORT` relies on - (open item O-DIR-1); treat it as unconfirmed until checked against your - own instance. + (closes open item O-DIR-1). - `count_tickets` is the cheap way to check a filter: it sends `max_rows=1` and reads the envelope total without fetching records. - `search_*` returns one page; `iter_*` pages until the server reports no diff --git a/skills/easyvista-ticket-actions/SKILL.md b/skills/easyvista-ticket-actions/SKILL.md index 8465758..174803d 100644 --- a/skills/easyvista-ticket-actions/SKILL.md +++ b/skills/easyvista-ticket-actions/SKILL.md @@ -1,11 +1,11 @@ --- name: easyvista-ticket-actions -description: "Read and write the action log on an EasyVista ticket with easyvista_python_client — create_action, list_actions and get_action with PostAction and Action, including how to recover a created action's id and how to resolve an action's note text, which the list endpoint does not return. Use for ticket followups, work notes, progress entries or any per-ticket action history." +description: "Read and write the action log on an EasyVista ticket with easyvista_python_client — create_action, list_actions, get_action and update_action with PostAction, Action and ActionUpdate, including how to recover a created action's id, how to project timestamps and author onto list_actions with fields=, and how to resolve an action's note text, which the list endpoint does not return. Use for ticket followups, work notes, progress entries or any per-ticket action history." license: MIT compatibility: "Requires Python 3.10+, easyvista-python-client, network access to an EasyVista Service Manager REST API, and a profile authorized for the actions sub-resource." metadata: package: easyvista-python-client - version: "0.1.0" + version: "0.2.0" --- > **Sync and async.** Examples use `EasyvistaClient`. For `AsyncEasyvistaClient`, @@ -14,14 +14,23 @@ metadata: > `easyvista-client-setup`. Actions are EasyVista's per-ticket work log — the closest equivalent to a -followup. Three methods: `create_action(rfc, action)`, `list_actions(rfc)` and -`get_action(action_id)`. The list and item shapes differ substantially, which -is where most mistakes come from. +followup. Four methods: `create_action(rfc, action)`, `list_actions(rfc)`, +`get_action(action_id)` and `update_action(action_id, update)`. The list and +item shapes differ substantially, which is where most mistakes come from. ## Two shapes of the same record -- `list_actions(rfc)` returns a **slim collection record**. It does **not** - carry the note text. +- `list_actions(rfc)` returns a **slim collection record**: by default it + carries `ACTION_ID`, `ACTION_LABEL_FR`, `ACTION_NUMBER`, `DONE_BY_ID` and + `EXPECTED_START_DATE_UT`, but **not** the note text, and not + `CREATION_DATE_UT`/`LAST_UPDATE` either. +- Pass `fields=` to widen that projection in one request instead of an item + fetch per action: `list_actions(rfc, fields=["ACTION_ID", + "ACTION_TYPE_ID", "CREATION_DATE_UT", "LAST_UPDATE", "DONE_BY_ID"])` + returns those columns top-level on every row. The note text stays + unreachable this way — `DESCRIPTION`/`COMMENT` are Memo sub-resources and + come back as HREF objects under any projection — and `fields="*"` is + **not** a wildcard, it silently reduces to `ACTION_ID` alone. - `get_action(action_id)` returns a **fuller item record** whose `DESCRIPTION` and `COMMENT` are memo href objects — that is, `action.description` is a `dict` with an `HREF`, not a string, until something resolves it. @@ -31,6 +40,19 @@ is where most mistakes come from. which fetches each action item-level and resolves its memo for you — see `easyvista-reporting-and-context`. +## Editing an action + +`update_action(action_id, ActionUpdate(description=...))` edits an existing +action's note (`comment` is also available, for a deployment configured the +other way round). Two asymmetries worth knowing: + +- Unlike `create_action`/`list_actions`, which are ticket-scoped + (`rfc_number`), `update_action` is keyed on the **action id alone** — it + calls the top-level `PUT actions/{id}`, not a + `requests/{rfc}/actions/{id}` path. +- An action can be **edited but not deleted**: `DELETE actions/{id}` is + refused with HTTP 403, so there is deliberately no `delete_action`. + ## Discover the ids first `action_type_id` and `group_id` are instance-specific. Read them off existing @@ -56,6 +78,8 @@ with EasyvistaClient.from_env() as client: 5. To read note text, either call `get_action` and resolve the memo href with `resolve_memo`, or take `get_ticket_context(rfc)` and read `context.actions`. +6. To edit an action's note afterwards, call `update_action(action_id, + ActionUpdate(description=...))` — by the action id alone, not the ticket. ## Examples @@ -113,6 +137,33 @@ with EasyvistaClient.from_env() as client: print(action.action_id, action.description) ``` +```python +from easyvista_python_client import ActionUpdate, EasyvistaClient + +with EasyvistaClient.from_env() as client: + # Keyed on the action id alone -- NOT the ticket's rfc_number. + client.update_action( + 12345, ActionUpdate(description="Corrected: printer power-cycled twice.") + ) + # The PUT's own response body has never been captured, so do not read the + # returned Action's fields -- re-read instead. + print(client.get_action(12345).description) +``` + +```python +from easyvista_python_client import EasyvistaClient + +with EasyvistaClient.from_env() as client: + # Project timestamps and author onto the list in one request instead of + # an item fetch per action. + actions = client.list_actions( + "YOUR_RFC_NUMBER", + fields=["ACTION_ID", "ACTION_TYPE_ID", "CREATION_DATE_UT", "DONE_BY_ID"], + ) + for action in actions: + print(action.action_id, action.created_at, action.done_by_id) +``` + ## Gotchas - **`create_action` gives you no usable id.** The live create response is an @@ -132,3 +183,29 @@ with EasyvistaClient.from_env() as client: - A profile restriction on the actions sub-resource surfaces as `EasyvistaAuthError` (403); the context bundle degrades to `[]` rather than failing. +- `update_action` takes only an `action_id`, no `rfc_number` — passing the + nested `requests/{rfc}/actions/{id}` shape (as `create_action` and + `list_actions` do) is not how this one works, and the nested form is + rejected with HTTP 403 anyway. +- `list_actions(fields=...)` has two silent footguns: `fields="*"` is not a + wildcard (it reduces to `ACTION_ID` alone), and a dotted path like + `"DESCRIPTION.HREF"` is silently dropped rather than raising. +- **`list_actions` returns ONE page and does not paginate.** The cap is + `config.default_max_rows`; a ticket with more actions than that is truncated + with **no error**, and the call discards the envelope's total so nothing in the + result reveals it. A freshly created ticket already carries about twelve + actions (most workflow-generated), so a busy ticket crosses a default cap + easily. The same cap truncates `get_ticket_context`'s `actions` and + `TicketContext.to_markdown()`'s rendered log — for a comment sync, that means + silently missing comments on exactly the busiest tickets. Raise + `EasyvistaConfig.default_max_rows` when a whole log matters. +- **`update_action`'s return value is an unverified echo.** The PUT's response + body has never been captured, and the parser falls back to the raw body when + there are no records — so if the API answers empty or HREF-only, you get an + `Action` whose every field is `None`. Re-read with `get_action` instead of + reading fields off the returned object. +- `Action` names its timestamps `created_at`/`updated_at` where `Request` and + `Employee` use `creation_date_ut`/`last_update` for the identical wire + columns. `getattr(record, "last_update")` raises `AttributeError` on an + `Action`; for code spanning record types, go through `classify_fields()` or + `.reference()`, where the wire alias is uniform. diff --git a/skills/easyvista-ticket-workflow/SKILL.md b/skills/easyvista-ticket-workflow/SKILL.md index 5bdb604..e448d31 100644 --- a/skills/easyvista-ticket-workflow/SKILL.md +++ b/skills/easyvista-ticket-workflow/SKILL.md @@ -5,7 +5,7 @@ license: MIT compatibility: "Requires Python 3.10+, easyvista-python-client, network access to an EasyVista Service Manager REST API, and a profile authorized for the requests resource." metadata: package: easyvista-python-client - version: "0.1.0" + version: "0.2.0" --- > **Sync and async.** Examples use `EasyvistaClient`. For `AsyncEasyvistaClient`, @@ -66,7 +66,10 @@ deployment needs before you build a payload for it. 4. Call `create_ticket(ticket)`. It returns a `Request` whose `rfc_number` is usable immediately — see the first Gotcha for why. 5. To set body text you can read back afterwards, follow the create with - `update_ticket(rfc, RequestUpdate(description=...))`. + `update_ticket(rfc, RequestUpdate(description=...))`. `RequestUpdate` also + accepts `title`, `status_id`, `impact_id`, `owner_id` and + `external_reference` (capped at 50 characters) after create — see the + Gotchas for what it deliberately omits. 6. Read one ticket with `get_ticket(rfc)`; search a page with `search_tickets(...)`, which returns a `SearchResult` carrying `.records`, `.record_count` (this page) and `.total_record_count` (every match on the @@ -111,6 +114,21 @@ with EasyvistaClient.from_env() as client: print(updated.rfc_number) ``` +```python +from easyvista_python_client import EasyvistaClient, RequestUpdate + +with EasyvistaClient.from_env() as client: + # impact_id, owner_id and external_reference can all be changed after + # create, not only set at create time. external_reference is capped at + # 50 characters -- 51 raises pydantic's own ValidationError locally, + # before any request is sent. + updated = client.update_ticket( + "YOUR_RFC_NUMBER", + RequestUpdate(impact_id=1, owner_id=1, external_reference="TICKET-REF-0001"), + ) + print(updated.rfc_number) +``` + ```python from easyvista_python_client import EasyvistaClient, ev_equals_filter @@ -154,6 +172,22 @@ with EasyvistaClient.from_env() as client: ## Gotchas +- **Timestamp columns are aware `datetime`, so a record dump is not + JSON-serialisable.** `submit_date_ut`, `creation_date_ut`, + `max_resolution_date_ut`, `expected_date_ut`, `end_date_ut` and `last_update` + are parsed, so `json.dumps(ticket.model_dump(by_alias=True))` and + `json.dumps(ticket.classify_fields().official)` raise `TypeError`. For a dump, + use `model_dump(mode="json")`. `classify_fields()` takes **no arguments**, so + that keyword has nowhere to go there: render the values with + `format_ev_datetime` before serialising the bucket, or re-key a JSON-mode dump + by the bucket's keys — `dumped = ticket.model_dump(mode="json", + by_alias=True)`, then `{k: dumped[k] for k in ticket.classify_fields().official}`. + Only the + **declared** columns are parsed — an instance-specific date reached through + `classify_fields().custom` is still the raw string, so pass it through + `parse_ev_datetime` before comparing the two. No write model accepts a + `datetime`, `custom_fields` included: a `datetime` there fails inside the HTTP + layer with a bare `TypeError`, so render it yourself. - `create_ticket`'s response body is **HREF-only** — the API returns no `RFC_NUMBER`. `Request` derives `rfc_number` from the trailing path segment of the `href` (its own model validator does this, and it is checked against @@ -194,3 +228,9 @@ with EasyvistaClient.from_env() as client: the catalog is misconfigured is a reasonable next thing to try. - The accepted **write** format for the date fields is unverified; both a string and an int probe returned 590. Do not attempt to set them. +- `RequestUpdate` deliberately does **not** expose `severity_id` (`SEVERITY_ID` + is rejected with HTTP 590) or a priority field (EasyVista derives priority + from urgency x impact rather than exposing a writable column). `urgency_id` + is also absent: it raised HTTP 590 while still changing the stored value on + the verified instance, so it is not offered until that is resolved (open + item `O-590-PARTIAL`). diff --git a/unasync_build.py b/unasync_build.py index 9728472..de3fe73 100644 --- a/unasync_build.py +++ b/unasync_build.py @@ -80,12 +80,36 @@ #: * **Third-party naming.** unasync's built-in ``Async*`` -> ``Sync*`` #: convention would produce ``SyncClient``, which does not exist; the real #: httpx name is ``Client``. Same for tenacity's ``AsyncRetrying``. +#: * **httpx's own async method names**, which are spelled with a leading +#: ``a`` rather than the ``Async`` prefix the convention knows about, so +#: nothing infers them. ``aclose``, ``aread`` and ``aiter_bytes`` all have +#: sync twins in httpx that differ only by that letter, and an unmapped one +#: is emitted verbatim into the sync tree. Only ``aclose`` **on the client** +#: then fails loudly: ``httpx.Client`` has no ``aclose``, so that line raises +#: ``AttributeError``. Every other case is *silent*, which is the real reason +#: this dict has to cover them all -- httpx defines both spellings on +#: ``httpx.Response`` (checked against 0.28.1: ``aread``, ``aiter_bytes`` and +#: ``aclose`` are all present on the one class), so the attribute exists in +#: sync code and merely does the wrong thing. ``response.aread()`` without an +#: ``await`` builds a coroutine and drops it -- a ``RuntimeWarning`` and an +#: unread body, after which the next line's ``_raise_for_response`` raises +#: ``httpx.ResponseNotRead`` instead of the mapped EasyVista exception; and +#: ``for chunk in response.aiter_bytes(...)`` raises ``TypeError: +#: 'async_generator' object is not iterable``. A wrong exception escaping the +#: client, not a missing attribute -- exactly the silent-collision class +#: ``testing/test_unasync_codegen.py`` says a diff of the two trees cannot +#: catch, so a new httpx async method must be added here rather than trusted +#: to blow up on its own. #: * **This package's own public class name**, which differs between the two -#: surfaces by design, and ``aclose``, which is public async API. +#: surfaces by design. ``aclose`` is in both categories: httpx's method and +#: this package's public async API. #: #: Everything else -- helpers, module names, the executor's methods -- is #: spelled *identically* in both trees. Keeping this list short is #: deliberate: every entry is a chance for a silent collision. +#: ``testing/test_unasync_codegen.py`` scans the async tree for identifiers +#: that would be rewritten by any of these, so a local or a parameter that +#: happens to share a spelling fails there rather than in production. TOKEN_REPLACEMENTS = { # Intra-tree imports are absolute, so the package segment is itself a # NAME token and rewriting it repoints every one of them at the @@ -95,6 +119,12 @@ "AsyncClient": "Client", "AsyncRetrying": "Retrying", "aclose": "close", + # The streaming download path: `Response.aiter_bytes` yields the body in + # chunks and `Response.aread` materialises it, the latter needed on the + # error path because a streaming response refuses `.content` until it has + # been read. + "aread": "read", + "aiter_bytes": "iter_bytes", } #: The qualified package prefix, and what it becomes in the generated tree.