diff --git a/CONTEXT.md b/CONTEXT.md index a88ea330..0821b7e3 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -44,6 +44,10 @@ _Avoid_: Specimen; Derivation (aliquots do not require a transformation record) Where a specimen's material originated. The primary source is a **Subject** (via a study). Alternate sources include **Control batch**, **Reagent**, **Cell line**, **Plasmid**, and **Standard**. _Avoid_: Origin (too vague); parent (reserved for derivations — use **Parent container**) +**Specimen registration**: +Recording a **Specimen** from a known **Source**, together with zero or more **Containers** (aliquots) and their **Container placement**. One collection event from one Source is one Specimen: registering the same Source, specimen type, and collection date again adds containers to the existing Specimen instead of creating a second one. A registration either records everything it was given or nothing. +_Avoid_: Import (the CSV workflow that leads to registration); creating a second Specimen for the same collection event + ### Controls (quality control specimens) SampleDB supports multiple **Control** types for QC, calibration, and validation. Control specimens use the same **Specimen** and storage hierarchies as study specimens, but provenance runs through definitions and batches instead of studies and subjects. diff --git a/docs/adr/0006-unified-container-inbound-write.md b/docs/adr/0006-unified-container-inbound-write.md index 2df4ab77..08a73392 100644 --- a/docs/adr/0006-unified-container-inbound-write.md +++ b/docs/adr/0006-unified-container-inbound-write.md @@ -57,13 +57,19 @@ Affected routes include specimen container create, bulk combined import, control ## Single API write executor -All persistence funnels through `createContainerForSpecimen` (accepting `ContainerWriteInput`) after `resolveContainerPlacement`. Delete parallel implementations: +All persistence funnels through `createContainerForSpecimen` (accepting `ContainerWriteInput`) after `resolveContainerPlacement`. + +> **Amended by [ADR 0010](./0010-specimen-registration.md):** persistence funnels through `registerSpecimens` in `lib/specimen-registration/`. `createContainerForSpecimen` is a one-container wrapper around it for callers not yet migrated, and `resolveContainerPlacement` is gone. + +Delete parallel implementations: - `batch-with-specimens` `prepareContainerData` / `createContainerSync` - Inline subtype inserts in `derivations.ts` for child containers Placement resolution (sheet find-or-create within box/bag, plate/box resolve) lives in one module — not duplicated across registration, control batch, and derivation CSV. +> **Amended by [ADR 0010](./0010-specimen-registration.md):** that module is `lib/specimen-registration/collections.ts`, and `createContainerForSpecimen` now wraps `registerSpecimens`. It replaced `container-write-placement.ts`, whose lookup fell back from a missing collection barcode to the name; that file is deleted. The derivation CSV and control batch validators still look collections up themselves until their ADR 0010 slices replace them. + ## CSV One contract mapper — `csvRowToContainerWriteInput` — maps flat CSV rows to `ContainerWriteInput`, then Zod validates. @@ -104,6 +110,7 @@ Run `bun run ci:verify` before merge. - [ADR 0005: Wire DTO layer and omit-on-wire JSON](./0005-wire-dto-omit-on-wire.md) — outbound wire shape this mirrors - [ADR 0002: HTTP client response unwrap](./0002-http-client-response-unwrap.md) — web imports shared contract types - [ADR 0003: SQLite schema evolution](./0003-sqlite-schema-evolution.md) — storage layer unchanged; this is request-shape and routing only +- [ADR 0010: Specimen registration](./0010-specimen-registration.md) — amends this ADR: the single write executor and placement module are `lib/specimen-registration/` ## Non-goals diff --git a/docs/adr/0010-specimen-registration.md b/docs/adr/0010-specimen-registration.md new file mode 100644 index 00000000..b80646bf --- /dev/null +++ b/docs/adr/0010-specimen-registration.md @@ -0,0 +1,44 @@ +# Specimen registration: one module, validation as a rolled-back write + +--- +status: accepted +--- + +Every write path that creates a **Container** ran its own checks before calling the shared writer `createContainerForSpecimen`: bulk combined import checked the same rows in four places (`bulk-combined-validate.ts`, its prepare step, an in-transaction revalidate, and the writer), `/specimens/bulk` had its own prepare module, and the derivation CSV had a hand-written validator. The checks drifted from the write. Validation looked plate collections up by barcode without falling back to name while the write fell back, so an occupied well passed validation and failed on import. Validation rejected two Papers on one **Sheet** that the write accepted. The derivation CSV validator assumed `micronix_tube` when no container type was given while the import rejected the row. + +We put **Specimen registration** (see [CONTEXT.md](../../CONTEXT.md)) behind one module, `packages/api/src/lib/specimen-registration/`, and make validation the write itself, rolled back. + +## Interface + +- `registerSpecimens(db, entries, options)` plans every entry, refuses with all errors if any, otherwise writes. Called inside the caller's transaction, it joins it. +- `checkSpecimenRegistration(db, entries, options)` runs the same write in a transaction that is always rolled back and returns the errors. Callers whose own writes come first (bulk combined creates **Subjects**) wrap their whole write in `withRolledBackTransaction` instead. +- An entry is a resolved **Source** (`subjectId` or `controlBatchId`; derivations add a parent container later), a specimen type, an optional collection date, and `ContainerWriteInput`s ([ADR 0006](./0006-unified-container-inbound-write.md)) with optional unit and quantity. An entry may instead name an existing specimen to add containers to it. +- Errors are `{ entryIndex, containerIndex?, field?, message }`, keyed to the module's input. Callers map indexes back to CSV rows or form fields. + +## Rules the module owns + +- **Specimen identity**: (Source, specimen type, collection date), a missing date matching only a missing date. A match is reused and reported as `reused`; entries with the same key in one request become one Specimen. +- **Collection identity**: every identifier given (id, barcode, name) must resolve to the same **Collection**. One that misses while another matches, or two that match different collections, is an error. A Collection is created only when nothing matches and a location is given. A location given for a Collection that exists, or that the request already creates, must be the one it has, so one new plate given two locations fails however the request is split into calls. Sheets are created under their box or bag when missing. +- **Placement**: one **Grid position** per container per collection, checked against the database and the rest of the request by resolved collection, after normalizing positions (`A2` → `A02`). Any number of Papers share a Sheet; a duplicate sublabel on one Sheet is an error. +- **Units**: an omitted unit takes the configured default for the container type; an unknown or disallowed unit is an error. No hardcoded container-type default. +- **Atomicity**: one call writes everything or nothing. Atomicity across calls is the caller's choice: bulk combined `per_subject` mode registers each subject in its own `withWriteTransaction`: a transaction of its own when called at the top level, a savepoint when an outer transaction is open (validate, or a caller's transaction). A failed subject leaves nothing behind, its Subject row included. +- **Errors**: problems found while planning are a `SpecimenRegistrationError` with status 400 and the full error list. A constraint violation while writing is the same error with status 409. + +## Considered options + +- **Validate as a read-only plan, write separately**: rejected. "Validate passes ⇒ write succeeds" then holds only as far as the plan anticipates the schema, which is how the copies drifted. The rolled-back write costs a second set of inserts, which SQLite does in milliseconds for a CSV of thousands of rows. +- **Plan outside the transaction, execute inside**: rejected. The plan goes stale, which is why bulk combined needed an in-transaction revalidate. +- **Module owns Subjects and control batches too**: rejected. Each caller would need its own variant and the interface would grow back to the size of its callers. Callers create their Source records in the same transaction and pass ids. +- **First matching identifier wins**: rejected. It silently placed a tube in a plate whose barcode contradicted the row. + +## Consequences + +- Validate endpoints keep their wire shapes; each route maps module errors to its existing response. +- Behaviour change for every container write, including the callers still on `createContainerForSpecimen` (add-container, `POST /specimens`, derivations, control batches): the collection identifiers a request gives (`id`, `barcode`, `name`, and a sheet's `id` and `name`) must all name the same Collection. Before, an `id` won and a `barcode` or `name` beside it was ignored, and a barcode that matched nothing fell back to the name. Now a request whose identifiers disagree, or whose barcode matches no collection while its name does, is refused with 400 and a message naming the identifier that disagrees. A client that sends a stale name or barcode next to an id must correct it or leave it out. +- Bulk combined validate runs the import in its `atomicMode` and rolls it back. For `per_subject` it runs each subject in its own savepoint inside the rolled-back transaction, rolling back a subject that fails and keeping the ones that pass until the end, as the import does. So validate flags exactly the subjects the import rejects, including a subject that relies on a collection a failed subject would have created. +- Migration is in slices, each deleting the code it replaces: bulk combined import, `/subjects/with-specimens`, and `/specimens/bulk`; `POST /specimens` and add-container; derivations and the derivation CSV; control batches. `/subjects/with-specimens` and `/specimens/bulk` moved into the first slice: the first shares bulk combined's code, and the second's validator was the last user of the old placement resolver (ADR 0006). + +## Related ADRs + +- [ADR 0006: Unified container inbound write](./0006-unified-container-inbound-write.md) — amended by this ADR. Its single write executor is now `registerSpecimens`, with `createContainerForSpecimen` a wrapper for callers not yet migrated, and its one placement module is `lib/specimen-registration/collections.ts`, which replaced `container-write-placement.ts`. `ContainerWriteInput` stays the container shape registration takes. +- [ADR 0007: Specimen source resolution](./0007-specimen-source-resolution.md) — the read side of **Source**; registration takes a Source already resolved to a record id. diff --git a/packages/api/src/__tests__/README.md b/packages/api/src/__tests__/README.md index 845c9fa9..e5ae6ea6 100644 --- a/packages/api/src/__tests__/README.md +++ b/packages/api/src/__tests__/README.md @@ -82,6 +82,7 @@ Available factories: - `createTestControlDefinition` - `createTestSpecimen` - `createTestUnit` +- `createTestSpecimenTypeContainerType`, `createTestContainerTypeUnit` (allow a container type for a specimen type, and a unit for a container type) - `createTestStudy` - `createTestStudySubject` - `createTestControlBatch` diff --git a/packages/api/src/__tests__/helpers/constraint-race.ts b/packages/api/src/__tests__/helpers/constraint-race.ts new file mode 100644 index 00000000..0a26c6dd --- /dev/null +++ b/packages/api/src/__tests__/helpers/constraint-race.ts @@ -0,0 +1,17 @@ +import type { Database as SQLiteDatabase } from 'bun:sqlite' + +/** + * Make every insert into `table` fail with a UNIQUE violation, the way a write fails when a + * concurrent request took the row after the plan checked it. Use it to reach the 409 path of a + * write that validates first. + */ +export function failInsertsWithUniqueViolation(sqlite: SQLiteDatabase, table: string): void { + sqlite.exec(` + CREATE TABLE IF NOT EXISTS test_constraint_race (taken TEXT UNIQUE); + INSERT OR IGNORE INTO test_constraint_race VALUES ('row'); + CREATE TRIGGER test_constraint_race_${table} BEFORE INSERT ON ${table} + BEGIN + INSERT INTO test_constraint_race VALUES ('row'); + END; + `) +} diff --git a/packages/api/src/__tests__/helpers/factories.ts b/packages/api/src/__tests__/helpers/factories.ts index e5d7feb9..57dc4e5a 100644 --- a/packages/api/src/__tests__/helpers/factories.ts +++ b/packages/api/src/__tests__/helpers/factories.ts @@ -21,7 +21,10 @@ import { cryovialTube, staticWell, paper, + specimenTypeContainerType, + containerTypeUnit, } from '../../db/schema' +import type { ContainerType } from '../../lib/container-creation' import { utcNow } from '../../lib/datetime' /** @@ -96,6 +99,28 @@ export async function createTestUnit(db: Database, data: { return result } +/** + * Allow a container type for a specimen type (specimen_type_container_type) + */ +export async function createTestSpecimenTypeContainerType(db: Database, data: { + specimenTypeId: number + containerType: ContainerType +}) { + const [result] = await db.insert(specimenTypeContainerType).values(data).returning() + return result +} + +/** + * Allow a unit for a container type (container_type_unit) + */ +export async function createTestContainerTypeUnit(db: Database, data: { + containerType: ContainerType + unitId: number +}) { + const [result] = await db.insert(containerTypeUnit).values(data).returning() + return result +} + /** * Create a test storage container (for testing "in use" scenarios) * Note: storageContainer requires specimenId and unitId diff --git a/packages/api/src/__tests__/integration/control-batch.test.ts b/packages/api/src/__tests__/integration/control-batch.test.ts index e9171c29..a41f1589 100644 --- a/packages/api/src/__tests__/integration/control-batch.test.ts +++ b/packages/api/src/__tests__/integration/control-batch.test.ts @@ -6,8 +6,16 @@ import { } from '../helpers/authenticated-route-test' import { createTestControlDefinition, + createTestLocation, + createTestMicronixPlate, + createTestSpecimenType, + createTestStorageType, + createTestUnit, } from '../helpers/factories' import { createControlsRoutes } from '../../routes/controls' +import { setContainerDefaults } from '../../lib/settings' +import { containerTypeUnit, controlBatch, specimenTypeContainerType } from '../../db/schema' +import { failInsertsWithUniqueViolation } from '../helpers/constraint-race' describe('Control Batch Creation Integration Tests', () => { let ctx: AuthenticatedRouteTestContext @@ -65,4 +73,64 @@ describe('Control Batch Creation Integration Tests', () => { const data = await response.json() as { error: string } expect(data).toHaveProperty('error') }) + + describe('when a container write hits a constraint', () => { + let definitionId: number + let plateId: number + + beforeEach(async () => { + const unit = await createTestUnit(ctx.db, { symbol: 'uL', name: 'microliter', category: 'volume' }) + await setContainerDefaults(ctx.db, { + micronix_tube: { totalQuantity: 1, remainingQuantity: 1, defaultUnitSymbol: 'uL' }, + }) + await ctx.db.insert(containerTypeUnit).values({ containerType: 'micronix_tube', unitId: unit.id }) + definitionId = (await createTestControlDefinition(ctx.db, { name: 'Def1' })).id + const specimenType = await createTestSpecimenType(ctx.db, { name: 'DNA' }) + await ctx.db.insert(specimenTypeContainerType).values({ specimenTypeId: specimenType.id, containerType: 'micronix_tube' }) + const storageType = await createTestStorageType(ctx.db, { name: 'Freezer' }) + const location = await createTestLocation(ctx.db, { name: 'Loc', storageTypeId: String(storageType.id) }) + plateId = (await createTestMicronixPlate(ctx.db, { name: 'Plate1', locationId: location.id })).id + failInsertsWithUniqueViolation(ctx.sqlite, 'micronix_tube') + }) + + function specimens() { + return [ + { + specimenTypeName: 'DNA', + containers: [{ containerType: 'micronix_tube', barcode: 'MT001', collection: { type: 'micronix_plate', id: plateId, position: 'A01' } }], + }, + ] + } + + it('create-with-specimens returns 409', async () => { + const res = await ctx.request('/api/blood-controls/batches/create-with-specimens', { + method: 'POST', + json: { batch: { controlDefinitionId: definitionId, name: 'Batch1' }, specimens: specimens() }, + }) + + expect(res.status).toBe(409) + expect(await ctx.db.select().from(controlBatch)).toEqual([]) + }) + + it('create-many-with-specimens returns 409', async () => { + const res = await ctx.request('/api/blood-controls/batches/create-many-with-specimens', { + method: 'POST', + json: { batches: [{ batch: { controlDefinitionId: definitionId, name: 'Batch1' }, specimens: specimens() }] }, + }) + + expect(res.status).toBe(409) + expect(await ctx.db.select().from(controlBatch)).toEqual([]) + }) + + it('adding specimens to a batch returns 409', async () => { + const [batch] = await ctx.db.insert(controlBatch).values({ controlDefinitionId: definitionId, name: 'Existing' }).returning() + + const res = await ctx.request(`/api/blood-controls/batches/${batch.id}/specimens/bulk`, { + method: 'POST', + json: { specimens: specimens() }, + }) + + expect(res.status).toBe(409) + }) + }) }) diff --git a/packages/api/src/__tests__/integration/derivations.test.ts b/packages/api/src/__tests__/integration/derivations.test.ts index b0a49f89..9774d92c 100644 --- a/packages/api/src/__tests__/integration/derivations.test.ts +++ b/packages/api/src/__tests__/integration/derivations.test.ts @@ -16,6 +16,7 @@ import { setContainerDefaults } from '../../lib/settings' import { specimenTypeContainerType, containerTypeUnit, storageContainer, micronixTube } from '../../db/schema' import type { Database } from '../../db/client' import { utcNow } from '../../lib/datetime' +import { failInsertsWithUniqueViolation } from '../helpers/constraint-race' const BASE = '/api/derivations' @@ -135,6 +136,27 @@ describe('Derivation Workflow Integration Tests', () => { expect(list.derivations).toHaveLength(1) }) + it('returns 409 when the child container write hits a constraint', async () => { + const { parentContainerId, targetPlateId } = await createParentContainerFixture(ctx.db) + failInsertsWithUniqueViolation(ctx.sqlite, 'micronix_tube') + + const res = await ctx.request(`${BASE}/containers/${parentContainerId}/derive`, { + method: 'POST', + json: { + derivationType: 'aliquot', + specimenTypeName: 'DNA', + container: { + containerType: 'micronix_tube', + barcode: 'MT-CHILD', + collection: { type: 'micronix_plate', id: targetPlateId, position: 'A01' }, + }, + }, + }) + + expect(res.status).toBe(409) + expect(await ctx.db.select().from(storageContainer)).toHaveLength(1) + }) + it('returns derivation source for the child container', async () => { const { parentContainerId, targetPlateId } = await createParentContainerFixture(ctx.db) diff --git a/packages/api/src/bun-sqlite.d.ts b/packages/api/src/bun-sqlite.d.ts index 35eee7f9..be6220fe 100644 --- a/packages/api/src/bun-sqlite.d.ts +++ b/packages/api/src/bun-sqlite.d.ts @@ -9,7 +9,7 @@ declare module 'bun:sqlite' { run(sql: string, ...params: any[]): void serialize(): Uint8Array loadExtension(path: string): void - inTransaction(): boolean + readonly inTransaction: boolean transaction(fn: () => T): T } diff --git a/packages/api/src/db/README.md b/packages/api/src/db/README.md index a71dc81d..c02308f4 100644 --- a/packages/api/src/db/README.md +++ b/packages/api/src/db/README.md @@ -8,12 +8,38 @@ See [ADR-0003: SQLite schema evolution](../../../docs/adr/0003-sqlite-schema-evo Use `openOperationalDatabase(optionalPath?)` from the client module. It resolves the file path, enables WAL, turns on foreign keys, runs schema evolution, and returns Drizzle + raw `bun:sqlite` handles. Importing the client module does **not** open a connection. -Multi-statement writes that `await` must use `withWriteTransaction` from `write-transaction.ts`. Drizzle's bun-sqlite `db.transaction(async () => …)` commits when the callback first yields; it does not await the returned Promise. ESLint (`eslint-rules/no-promise-transaction-callback.mjs`, tests included) rejects any callback passed to `.transaction(...)` whose return type is Promise-like: an `async` function, a named async function, or a sync function that returns a Promise. - Tests should use the same entry point (e.g. `openOperationalDatabase(':memory:')` via `setupTestDatabase()`), except a test that checks the snapshot itself: `__tests__/initial-schema-snapshot.test.ts` applies `initial_schema.sql` alone, because the entry point would run pending migrations on top and hide a stale snapshot. **Exception: read-only inspection.** `scripts/check-integrity.ts` opens the file with `new Database(path, { readonly: true })` instead. It is often pointed at a backup, and `openOperationalDatabase` would migrate it and enable WAL (both are writes). Resolve the path with `resolveOperationalDatabasePath` so it finds the same file. Any other tool that only reads, and must leave the file as it found it, should do the same. +## Transactions + +The API shares one `bun:sqlite` connection across all requests, so a transaction that stays open across `await` is visible to every request that runs while it is open. Use the helpers in `write-transaction.ts`: + +| Helper | Use it to | Ends with | +|--------|-----------|-----------| +| `withWriteTransaction(db, fn)` | Run multi-statement writes that `await` | `COMMIT`, or `ROLLBACK` if `fn` throws | +| `withRolledBackTransaction(db, fn)` | Check that a write would succeed by performing it (validate routes) | `ROLLBACK`, always | + +Drizzle's bun-sqlite `db.transaction(async () => …)` commits when the callback first yields; it does not await the returned Promise. ESLint (`eslint-rules/no-promise-transaction-callback.mjs`, tests included) rejects any callback passed to `.transaction(...)` whose return type is Promise-like: an `async` function, a named async function, or a sync function that returns a Promise. + +### Concurrency model + +- **One transaction at a time per connection.** A process-wide lock per connection serializes the helpers. A call from another async context, such as another request, waits for the lock and then runs its own `BEGIN`. +- **Nesting follows the async context.** The open transaction is kept in an `AsyncLocalStorage`. A helper called from inside another helper's `fn` runs in a `SAVEPOINT` of the open transaction instead of waiting. If the nested `fn` throws, only its own writes roll back. A nested `withRolledBackTransaction` always rolls back to its savepoint and leaves the outer transaction's earlier writes in place. +- **A transaction opened outside the helpers** (a raw `sqlite.exec('BEGIN')`, as some tests do) is not the helpers' to commit: a helper called while one is open runs in a savepoint inside it. + +### What the lock does not cover + +The lock only orders the helpers. A write made outside them runs immediately and synchronously, and if another context has a helper's transaction open at that moment the write lands in that transaction: + +- If that transaction commits, the write commits with it. +- If it rolls back (a failed import, or any `withRolledBackTransaction` validate), the write is lost, though the request that made it has already reported success. + +Writes outside the helpers include plain `db.insert/update/delete` calls (about 220 in 38 files under `routes/`, `lib/`, and `middleware/`, among them session writes on login, rate-limit counters, and the error log that `handleRouteError` writes) and Drizzle's synchronous `db.transaction((tx) => …)`, which bun:sqlite turns into a savepoint inside the open transaction. Routing them through the lock would mean making every write in the API wait on an async lock; it has not been done. The exposure lasts as long as a helper's transaction stays open across `await`. + +When you add a multi-statement write, use `withWriteTransaction` instead of adding to that list. + ## Schema version - Table: `schema_version` with a single row (`version INTEGER NOT NULL`). diff --git a/packages/api/src/db/__tests__/write-transaction.test.ts b/packages/api/src/db/__tests__/write-transaction.test.ts index 6c41ff57..2cc46baf 100644 --- a/packages/api/src/db/__tests__/write-transaction.test.ts +++ b/packages/api/src/db/__tests__/write-transaction.test.ts @@ -2,7 +2,7 @@ import { describe, it, expect, beforeEach, afterEach } from 'vitest' import { setupTestDatabase, cleanupTestDatabase } from '../../__tests__/helpers/db-setup' import { createTestStudy } from '../../__tests__/helpers/factories' import { study } from '../schema' -import { withWriteTransaction } from '../write-transaction' +import { withRolledBackTransaction, withWriteTransaction } from '../write-transaction' import type { Database } from '../client' describe('withWriteTransaction', () => { @@ -53,4 +53,52 @@ describe('withWriteTransaction', () => { const rows = await testDb.select().from(study) expect(rows.find((row) => row.shortCode === 'N1')).toBeUndefined() }) + + it('keeps a write from another request that lands while a validate is open', async () => { + let validateOpen!: () => void + const opened = new Promise((resolve) => (validateOpen = resolve)) + let finishValidate!: () => void + const finish = new Promise((resolve) => (finishValidate = resolve)) + + const validate = withRolledBackTransaction(testDb, async (db) => { + await createTestStudy(db, { title: 'Checked Study', shortCode: 'CHK' }) + validateOpen() + await finish + }) + await opened + // Another request, from its own async context, writes while the validate is still open. + const write = withWriteTransaction(testDb, async (db) => { + await createTestStudy(db, { title: 'Saved Study', shortCode: 'SAVED' }) + }) + finishValidate() + await Promise.all([validate, write]) + + const codes = (await testDb.select().from(study)).map((row) => row.shortCode) + expect(codes).toContain('SAVED') + expect(codes).not.toContain('CHK') + }) + + it('runs overlapping writers in separate transactions', async () => { + let firstOpen!: () => void + const opened = new Promise((resolve) => (firstOpen = resolve)) + let failFirst!: () => void + const fail = new Promise((resolve) => (failFirst = resolve)) + + const first = withWriteTransaction(testDb, async (db) => { + await createTestStudy(db, { title: 'Failed Study', shortCode: 'FAIL' }) + firstOpen() + await fail + throw new Error('first fails') + }) + await opened + const second = withWriteTransaction(testDb, async (db) => { + await createTestStudy(db, { title: 'Second Study', shortCode: 'SECOND' }) + }) + failFirst() + await expect(first).rejects.toThrow('first fails') + await second + + const codes = (await testDb.select().from(study)).map((row) => row.shortCode) + expect(codes).toEqual(['SECOND']) + }) }) diff --git a/packages/api/src/db/write-transaction.ts b/packages/api/src/db/write-transaction.ts index c53e815d..2906dd69 100644 --- a/packages/api/src/db/write-transaction.ts +++ b/packages/api/src/db/write-transaction.ts @@ -1,3 +1,13 @@ +/** + * Transactions that stay open across `await` on the one shared bun:sqlite connection. + * + * A transaction belongs to the async context that opened it. A helper called from inside that + * context (truly nested) runs in a savepoint of the open transaction. A helper called from any + * other context, such as another request, waits for the connection's lock and then opens its + * own transaction, so two requests never share one. See the db README for what the lock does + * not cover. + */ +import { AsyncLocalStorage } from 'node:async_hooks' import type { Database as SQLiteDatabase } from 'bun:sqlite' import type { Database } from './client' @@ -15,17 +25,115 @@ function sqliteClientOf(database: object): SQLiteDatabase | undefined { return undefined } -function isAlreadyInTransactionMessage(error: unknown): boolean { - const message = error instanceof Error ? error.message : String(error) - return message.includes('within a transaction') +/** Serializes transactions on one connection, first come first served. */ +class ConnectionLock { + private tail: Promise = Promise.resolve() + + async acquire(): Promise<() => void> { + let release!: () => void + const released = new Promise((resolve) => (release = resolve)) + const previous = this.tail + this.tail = previous.then(() => released) + await previous + return release + } +} + +const locks = new WeakMap() + +function lockFor(sqlite: SQLiteDatabase): ConnectionLock { + let lock = locks.get(sqlite) + if (!lock) { + lock = new ConnectionLock() + locks.set(sqlite, lock) + } + return lock +} + +/** The transaction the current async context opened, while it is still open. */ +type OpenTransaction = { sqlite: SQLiteDatabase; open: boolean } + +const currentTransaction = new AsyncLocalStorage() + +function joinedTransaction(sqlite: SQLiteDatabase): OpenTransaction | undefined { + const current = currentTransaction.getStore() + return current && current.open && current.sqlite === sqlite ? current : undefined +} + +/** Read fresh each time: statements run inside `fn` can end the transaction. */ +function inTransaction(sqlite: SQLiteDatabase): boolean { + return sqlite.inTransaction +} + +let savepointCounter = 0 + +async function inSavepoint( + sqlite: SQLiteDatabase, + keep: boolean, + fn: () => Promise, +): Promise { + const name = `sp_${++savepointCounter}` + sqlite.exec(`SAVEPOINT ${name}`) + let result: T + try { + result = await fn() + } catch (error) { + sqlite.exec(`ROLLBACK TO ${name}`) + sqlite.exec(`RELEASE ${name}`) + throw error + } + if (!keep) sqlite.exec(`ROLLBACK TO ${name}`) + sqlite.exec(`RELEASE ${name}`) + return result +} + +/** + * Run `fn` in a transaction of its own, then commit (`keep`) or roll back. Nested calls from + * the same async context use a savepoint; calls from other contexts wait their turn. + */ +async function runTransaction( + sqlite: SQLiteDatabase, + keep: boolean, + fn: () => Promise, +): Promise { + if (joinedTransaction(sqlite)) return inSavepoint(sqlite, keep, fn) + + const release = await lockFor(sqlite).acquire() + try { + if (inTransaction(sqlite)) { + // Opened outside these helpers (raw BEGIN): it is not ours to commit, so nest in it. + const transaction: OpenTransaction = { sqlite, open: true } + try { + return await currentTransaction.run(transaction, () => inSavepoint(sqlite, keep, fn)) + } finally { + transaction.open = false + } + } + + const transaction: OpenTransaction = { sqlite, open: true } + sqlite.exec('BEGIN') + let result: T + try { + result = await currentTransaction.run(transaction, fn) + } catch (error) { + transaction.open = false + if (inTransaction(sqlite)) sqlite.exec('ROLLBACK') + throw error + } + transaction.open = false + sqlite.exec(keep ? 'COMMIT' : 'ROLLBACK') + return result + } finally { + release() + } } /** - * Run writes in one SQLite transaction that stays open across `await`. + * Run writes in one SQLite transaction that stays open across `await`, and commit them. * - * Drizzle's bun-sqlite `db.transaction(async () => ...)` commits when the - * callback first yields (it does not await the returned Promise). Callers - * already inside a transaction run `fn` as-is. + * Drizzle's bun-sqlite `db.transaction(async () => ...)` commits when the callback first + * yields (it does not await the returned Promise). Called inside another helper's `fn`, this + * runs in a savepoint: if `fn` throws, only its own writes roll back. */ export async function withWriteTransaction( database: Database, @@ -35,26 +143,21 @@ export async function withWriteTransaction( if (!sqlite) { return fn(database) } + return runTransaction(sqlite, true, () => fn(database)) +} - try { - sqlite.exec('BEGIN') - } catch (error) { - if (isAlreadyInTransactionMessage(error)) { - return fn(database) - } - throw error - } - - try { - const result = await fn(database) - sqlite.exec('COMMIT') - return result - } catch (error) { - try { - sqlite.exec('ROLLBACK') - } catch { - // Connection may already have left the transaction. - } - throw error +/** + * Run writes and always roll them back, returning what `fn` returned. Use it to check + * that a write would succeed by performing it. Inside an open transaction it rolls back + * to a savepoint, leaving the outer transaction's earlier writes in place. + */ +export async function withRolledBackTransaction( + database: Database, + fn: (database: Database) => Promise, +): Promise { + const sqlite = sqliteClientOf(database) + if (!sqlite) { + throw new Error('withRolledBackTransaction needs a bun:sqlite database') } + return runTransaction(sqlite, false, () => fn(database)) } diff --git a/packages/api/src/lib/__tests__/bulk-combined-import.test.ts b/packages/api/src/lib/__tests__/bulk-combined-import.test.ts index 08e44aeb..1c264451 100644 --- a/packages/api/src/lib/__tests__/bulk-combined-import.test.ts +++ b/packages/api/src/lib/__tests__/bulk-combined-import.test.ts @@ -8,16 +8,20 @@ import { createTestLocation, createTestMicronixPlate, createTestUnit, + createTestSpecimenTypeContainerType, + createTestContainerTypeUnit, } from '../../__tests__/helpers/factories' import { setContainerDefaults } from '../settings' import { normalizePosition, runBulkCombinedImport, - prepareSubjectWithSpecimens, + validateBulkCombinedPayload, + type BulkCombinedSubjectInput, } from '../bulk-combined-import' -import { specimenTypeContainerType, containerTypeUnit, studySubject, specimen } from '../../db/schema' +import { studySubject, specimen } from '../../db/schema' import { eq } from 'drizzle-orm' import type { Database } from '../../db/client' +import { withWriteTransaction } from '../../db/write-transaction' describe('bulk-combined-import', () => { let testDb: Database @@ -57,41 +61,6 @@ describe('bulk-combined-import', () => { }) }) - describe('prepareSubjectWithSpecimens', () => { - it('throws when study short code is invalid', async () => { - await expect( - prepareSubjectWithSpecimens(testDb, 'NONEXISTENT', 'Subj1', [ - { specimenTypeName: 'DNA', collectionDate: '2024-01-01' }, - ]) - ).rejects.toThrow(/Invalid study|study/i) - }) - - it('resolves study and specimen type and returns prepared subject without container', async () => { - const study = await createTestStudy(testDb, { title: 'Study 1', shortCode: 'ST1' }) - const specimenType = await createTestSpecimenType(testDb, { name: 'DNA' }) - const prepared = await prepareSubjectWithSpecimens(testDb, study.shortCode, 'NewSubject', [ - { specimenTypeName: specimenType.name, collectionDate: '2024-01-15' }, - ]) - expect(prepared.studyId).toBe(study.id) - expect(prepared.trimmedName).toBe('NewSubject') - expect(prepared.existingSubjectId).toBeNull() - expect(prepared.resolvedSpecimens).toHaveLength(1) - expect(prepared.resolvedSpecimens[0].specimenTypeId).toBe(specimenType.id) - expect(prepared.resolvedSpecimens[0].collectionDate).toBe('2024-01-15') - }) - - it('finds existing subject by name and study', async () => { - const study = await createTestStudy(testDb, { title: 'Study 1', shortCode: 'ST1' }) - const existing = await createTestStudySubject(testDb, { studyId: study.id, name: 'ExistingSubj' }) - const specimenType = await createTestSpecimenType(testDb, { name: 'DNA' }) - const prepared = await prepareSubjectWithSpecimens(testDb, study.shortCode, ' ExistingSubj ', [ - { specimenTypeName: specimenType.name, collectionDate: '2024-01-15' }, - ]) - expect(prepared.existingSubjectId).toBe(existing.id) - expect(prepared.trimmedName).toBe('ExistingSubj') - }) - }) - describe('runBulkCombinedImport', () => { it('throws when study does not exist (full_file mode)', async () => { await expect( @@ -181,6 +150,30 @@ describe('bulk-combined-import', () => { expect(result.summary.specimensCreated).toBe(2) }) + it('per_subject mode inside an open transaction rolls back only the failed subject', async () => { + const study = await createTestStudy(testDb, { title: 'Study 1', shortCode: 'ST1' }) + const specimenType = await createTestSpecimenType(testDb, { name: 'DNA' }) + + const result = await withWriteTransaction(testDb, (tx) => + runBulkCombinedImport( + tx, + { + studyShortCode: study.shortCode, + atomicMode: 'per_subject', + subjects: [ + { subjectName: 'GoodSubj', specimens: [{ specimenTypeName: specimenType.name, collectionDate: '2024-01-15' }] }, + { subjectName: 'BadSubj', specimens: [{ specimenTypeName: 'NonExistentType', collectionDate: '2024-01-15' }] }, + ], + }, + undefined + ) + ) + + expect(result.errors).toHaveLength(1) + const subjects = await testDb.select().from(studySubject).all() + expect(subjects.map((s) => s.name)).toEqual(['GoodSubj']) + }) + for (const atomicMode of ['full_file', 'per_subject'] as const) { it(`places each subject in its own study for a mixed-study plate (${atomicMode})`, async () => { const studyA = await createTestStudy(testDb, { title: 'Study A', shortCode: 'STA' }) @@ -305,4 +298,147 @@ describe('bulk-combined-import', () => { expect(subjects.find((s) => s.name === 'Override')?.studyId).toBe(studyB.id) }) }) + + describe('validate passes exactly when the import succeeds', () => { + let locationId: number + + beforeEach(async () => { + const study = await createTestStudy(testDb, { title: 'Study 1', shortCode: 'ST1' }) + await createTestStudySubject(testDb, { studyId: study.id, name: 'Existing' }) + const specimenType = await createTestSpecimenType(testDb, { name: 'DNA' }) + const storageType = await createTestStorageType(testDb, { name: 'Freezer' }) + locationId = (await createTestLocation(testDb, { name: 'Shelf', storageTypeId: String(storageType.id), canContainCollections: true })).id + const unit = await createTestUnit(testDb, { symbol: 'uL', name: 'microliter', category: 'volume' }) + for (const containerType of ['micronix_tube', 'paper'] as const) { + await createTestSpecimenTypeContainerType(testDb, { specimenTypeId: specimenType.id, containerType }) + await createTestContainerTypeUnit(testDb, { containerType, unitId: unit.id }) + } + await setContainerDefaults(testDb, { + micronix_tube: { totalQuantity: 1, remainingQuantity: 1, defaultUnitSymbol: 'uL' }, + paper: { totalQuantity: 1, remainingQuantity: 1, defaultUnitSymbol: 'uL' }, + }) + const plate = await createTestMicronixPlate(testDb, { name: 'P1', barcode: 'P1-BC', locationId }) + await runBulkCombinedImport( + testDb, + { studyShortCode: 'ST1', atomicMode: 'full_file', subjects: [subject('Existing', tube('TAKEN', { id: plate.id, position: 'A01' }))] }, + undefined + ) + }) + + function tube(barcode: string, collection: Record) { + return { containerType: 'micronix_tube' as const, barcode, collection: { type: 'micronix_plate' as const, ...collection } } + } + + function paperOn(sheetName: string, sublabel: string) { + return { + containerType: 'paper' as const, + sublabel, + collection: { type: 'sheet' as const, name: sheetName, parent: { type: 'box' as const, name: 'PaperBox', locationId } }, + } + } + + function subject(name: string, container?: NonNullable, studyShortCode?: string): BulkCombinedSubjectInput { + return { subjectName: name, studyShortCode, specimens: [{ specimenTypeName: 'DNA', collectionDate: '2024-01-15', container }] } + } + + const fixtures: Array<{ name: string; valid: boolean; subjects: () => BulkCombinedSubjectInput[] }> = [ + { name: 'tubes on an existing and a new plate', valid: true, subjects: () => [subject('A', tube('T1', { name: 'P1', position: 'A02' })), subject('B', tube('T2', { name: 'NEW', locationId, position: 'A01' }))] }, + { name: 'existing plate name with an unknown barcode', valid: false, subjects: () => [subject('A', tube('T1', { name: 'P1', barcode: 'UNKNOWN', position: 'A01' }))] }, + { name: 'two subjects with papers on one sheet', valid: true, subjects: () => [subject('A', paperOn('Sheet-1', 's1')), subject('B', paperOn('Sheet-1', 's2'))] }, + { name: 'one sublabel twice on a sheet across subjects', valid: false, subjects: () => [subject('A', paperOn('Sheet-1', 's1')), subject('B', paperOn('Sheet-1', 's1'))] }, + { name: 'a barcode already in the database', valid: false, subjects: () => [subject('A', tube('TAKEN', { name: 'P1', position: 'B01' }))] }, + { name: 'one well named by plate name and by plate barcode', valid: false, subjects: () => [subject('A', tube('T1', { name: 'P1', position: 'C01' })), subject('B', tube('T2', { barcode: 'P1-BC', position: 'C1' }))] }, + { name: 'an occupied well', valid: false, subjects: () => [subject('A', tube('T1', { barcode: 'P1-BC', position: 'A1' }))] }, + { name: 'a missing study on one subject', valid: false, subjects: () => [subject('A'), subject('B', undefined, 'NOPE')] }, + ] + + // per_subject validate checks each subject the way the import runs it, so it flags exactly + // the subjects the import rejects. + for (const fixture of fixtures) { + it(`${fixture.name} (per_subject)`, async () => { + const payload = { studyShortCode: 'ST1', atomicMode: 'per_subject' as const, subjects: fixture.subjects() } + + const validation = await validateBulkCombinedPayload(testDb, payload) + const imported = await runBulkCombinedImport(testDb, payload, undefined) + + const flagged = [...new Set(validation.errors.map((e) => e.subjectIndex))] + const rejected = (imported.errors ?? []).map((e) => e.index) + expect(validation.valid).toBe(fixture.valid) + expect(flagged).toEqual(rejected) + }) + } + + it('per_subject flags a subject that needed a collection from a subject that failed', async () => { + // B finds the new plate only because A, which fails, would create it. A's rollback drops + // the plate, so B fails too, in validate as in the import. + const payload = { + studyShortCode: 'ST1', + atomicMode: 'per_subject' as const, + subjects: [ + subject('A', tube('TAKEN', { name: 'NEW', locationId, position: 'A01' })), + subject('B', tube('T2', { name: 'NEW', position: 'A02' })), + ], + } + + const validation = await validateBulkCombinedPayload(testDb, payload) + const imported = await runBulkCombinedImport(testDb, payload, undefined) + + expect([...new Set(validation.errors.map((e) => e.subjectIndex))]).toEqual([0, 1]) + expect((imported.errors ?? []).map((e) => e.index)).toEqual([0, 1]) + }) + + it('per_subject flags a second subject that gives a new plate another location', async () => { + const storageType = await createTestStorageType(testDb, { name: 'Rack' }) + const otherLocationId = (await createTestLocation(testDb, { name: 'Other shelf', storageTypeId: String(storageType.id), canContainCollections: true })).id + const payload = { + studyShortCode: 'ST1', + atomicMode: 'per_subject' as const, + subjects: [ + subject('A', tube('T1', { name: 'NEW', locationId, position: 'A01' })), + subject('B', tube('T2', { name: 'NEW', locationId: otherLocationId, position: 'A02' })), + ], + } + + const validation = await validateBulkCombinedPayload(testDb, payload) + const imported = await runBulkCombinedImport(testDb, payload, undefined) + + expect(validation.errors.map((e) => e.subjectIndex)).toEqual([1]) + expect((imported.errors ?? []).map((e) => e.index)).toEqual([1]) + }) + + it('per_subject reports a failed subject with its own index and row', async () => { + const validation = await validateBulkCombinedPayload(testDb, { + studyShortCode: 'ST1', + atomicMode: 'per_subject', + subjects: [subject('A'), { subjectName: 'B', specimens: [{ specimenTypeName: 'NOPE', rowIndex: 7 }] }], + }) + + expect(validation.errors).toEqual([{ subjectIndex: 1, specimenIndex: 0, rowIndex: 7, message: "Specimen type 'NOPE' not found" }]) + }) + + it('says a sublabel repeats in the file', async () => { + const validation = await validateBulkCombinedPayload(testDb, { + studyShortCode: 'ST1', + atomicMode: 'full_file', + subjects: [subject('A', paperOn('Sheet-1', 's1')), subject('B', paperOn('Sheet-1', 's1'))], + }) + + expect(validation.errors.map((e) => e.message)).toEqual(["Sublabel 's1' is used more than once on this sheet in your file."]) + }) + + for (const fixture of fixtures) { + it(fixture.name, async () => { + const payload = { studyShortCode: 'ST1', atomicMode: 'full_file' as const, subjects: fixture.subjects() } + + const validation = await validateBulkCombinedPayload(testDb, payload) + const imported = await runBulkCombinedImport(testDb, payload, undefined).then( + () => true, + () => false + ) + + expect(validation.valid).toBe(fixture.valid) + expect(imported).toBe(validation.valid) + }) + } + }) }) diff --git a/packages/api/src/lib/__tests__/bulk-combined-validate.test.ts b/packages/api/src/lib/__tests__/bulk-combined-validate.test.ts index 4f7ae818..793e8a30 100644 --- a/packages/api/src/lib/__tests__/bulk-combined-validate.test.ts +++ b/packages/api/src/lib/__tests__/bulk-combined-validate.test.ts @@ -9,7 +9,7 @@ import { createTestUnit, } from '../../__tests__/helpers/factories' import { setContainerDefaults } from '../settings' -import { validateBulkCombinedPayload } from '../bulk-combined-validate' +import { validateBulkCombinedPayload } from '../bulk-combined-import' import { specimenTypeContainerType, containerTypeUnit } from '../../db/schema' import type { Database } from '../../db/client' import { utcNow } from '../datetime' @@ -32,6 +32,7 @@ describe('bulk-combined-validate', () => { describe('validateBulkCombinedPayload', () => { it('returns invalid when study short code does not exist', async () => { const result = await validateBulkCombinedPayload(testDb, { + atomicMode: 'full_file', studyShortCode: 'NONEXISTENT', subjects: [ { @@ -48,6 +49,7 @@ describe('bulk-combined-validate', () => { it('returns invalid when specimen type is not found', async () => { const study = await createTestStudy(testDb, { title: 'Study 1', shortCode: 'ST1' }) const result = await validateBulkCombinedPayload(testDb, { + atomicMode: 'full_file', studyShortCode: study.shortCode, subjects: [ { @@ -64,6 +66,7 @@ describe('bulk-combined-validate', () => { const study = await createTestStudy(testDb, { title: 'Study 1', shortCode: 'ST1' }) const specimenType = await createTestSpecimenType(testDb, { name: 'DNA' }) const result = await validateBulkCombinedPayload(testDb, { + atomicMode: 'full_file', studyShortCode: study.shortCode, subjects: [ { @@ -99,6 +102,7 @@ describe('bulk-combined-validate', () => { await testDb.insert(containerTypeUnit).values({ containerType: 'micronix_tube', unitId: unit.id }) const result = await validateBulkCombinedPayload(testDb, { + atomicMode: 'full_file', studyShortCode: study.shortCode, subjects: [ { @@ -130,6 +134,7 @@ describe('bulk-combined-validate', () => { const study = await createTestStudy(testDb, { title: 'Study 1', shortCode: 'ST1' }) const specimenType = await createTestSpecimenType(testDb, { name: 'DNA' }) const result = await validateBulkCombinedPayload(testDb, { + atomicMode: 'full_file', studyShortCode: study.shortCode, subjects: [ { @@ -164,6 +169,7 @@ describe('bulk-combined-validate', () => { await testDb.insert(containerTypeUnit).values({ containerType: 'micronix_tube', unitId: unit.id }) const result = await validateBulkCombinedPayload(testDb, { + atomicMode: 'full_file', studyShortCode: study.shortCode, subjects: [ { @@ -220,6 +226,7 @@ describe('bulk-combined-validate', () => { await testDb.insert(containerTypeUnit).values({ containerType: 'micronix_tube', unitId: unit.id }) const result = await validateBulkCombinedPayload(testDb, { + atomicMode: 'full_file', studyShortCode: study.shortCode, subjects: [ { @@ -269,6 +276,7 @@ describe('bulk-combined-validate', () => { await testDb.insert(containerTypeUnit).values({ containerType: 'micronix_tube', unitId: unit.id }) const result = await validateBulkCombinedPayload(testDb, { + atomicMode: 'full_file', studyShortCode: study.shortCode, subjects: [ { @@ -294,6 +302,7 @@ describe('bulk-combined-validate', () => { it('includes rowIndex in errors when provided in payload', async () => { const study = await createTestStudy(testDb, { title: 'Study 1', shortCode: 'ST1' }) const result = await validateBulkCombinedPayload(testDb, { + atomicMode: 'full_file', studyShortCode: study.shortCode, subjects: [ { @@ -315,6 +324,7 @@ describe('bulk-combined-validate', () => { await createTestStudy(testDb, { title: 'Study B', shortCode: 'STB' }) await createTestSpecimenType(testDb, { name: 'DNA' }) const result = await validateBulkCombinedPayload(testDb, { + atomicMode: 'full_file', subjects: [ { studyShortCode: 'STA', diff --git a/packages/api/src/lib/__tests__/container-creation.test.ts b/packages/api/src/lib/__tests__/container-creation.test.ts index ac89ae86..12b66e51 100644 --- a/packages/api/src/lib/__tests__/container-creation.test.ts +++ b/packages/api/src/lib/__tests__/container-creation.test.ts @@ -1,7 +1,8 @@ import { describe, it, expect, beforeEach, afterEach } from 'vitest' import { setupTestDatabase, cleanupTestDatabase } from '../../__tests__/helpers/db-setup' import type { Database } from '../../db/client' -import { validateContainerWriteFields, createContainerForSpecimen } from '../container-creation' +import { createContainerForSpecimen } from '../container-creation' +import { validateContainerWriteFields } from '../specimen-registration' import { createTestSpecimenType, createTestStorageType, @@ -110,16 +111,6 @@ describe('container-creation', () => { expect(result.valid).toBe(false) expect(result.error).toBe('Sheet name is required for papers') }) - - it('rejects barcode on paper inbound path', () => { - const result = validateContainerWriteFields({ - containerType: 'paper', - collection: { type: 'sheet', name: 'S1', parent: { type: 'box', name: 'Box1' } }, - barcode: 'P-1', - } as Parameters[0]) - expect(result.valid).toBe(false) - expect(result.error).toContain('sublabel') - }) }) describe('static_well', () => { diff --git a/packages/api/src/lib/__tests__/container-placement-validation.test.ts b/packages/api/src/lib/__tests__/container-placement-validation.test.ts deleted file mode 100644 index d23714a2..00000000 --- a/packages/api/src/lib/__tests__/container-placement-validation.test.ts +++ /dev/null @@ -1,93 +0,0 @@ -import { describe, it, expect, beforeEach, afterEach } from 'vitest' -import { setupTestDatabase, cleanupTestDatabase } from '../../__tests__/helpers/db-setup' -import { - createTestStorageType, - createTestLocation, - createTestMicronixPlate, - createTestStorageContainer, -} from '../../__tests__/helpers/factories' -import { collectContainerPlacementErrors } from '../container-placement-validation' -import { micronixTube } from '../../db/schema' -import type { Database } from '../../db/client' -import { utcNow } from '../datetime' - -describe('container-placement-validation', () => { - let testDb: Database - let sqlite: Awaited>['sqlite'] - - beforeEach(async () => { - const setup = await setupTestDatabase() - testDb = setup.db - sqlite = setup.sqlite - }) - - afterEach(() => { - if (sqlite) { - cleanupTestDatabase(sqlite) - } - }) - - describe('collectContainerPlacementErrors', () => { - it('detects duplicate positions within the same collection in payload', async () => { - const storageType = await createTestStorageType(testDb, { name: 'Freezer' }) - const loc = await createTestLocation(testDb, { - name: 'Loc', - storageTypeId: String(storageType.id), - canContainCollections: true, - }) - const plate = await createTestMicronixPlate(testDb, { name: 'Plate1', locationId: loc.id }) - const collectionKey = `micronix_plate-${plate.name}` - - const errors = await collectContainerPlacementErrors(testDb, [ - { - containerType: 'micronix_tube', - collectionId: plate.id, - collectionKey, - normalizedPosition: 'A01', - barcode: 'BC1', - }, - { - containerType: 'micronix_tube', - collectionId: plate.id, - collectionKey, - normalizedPosition: 'A01', - barcode: 'BC2', - }, - ]) - - expect(errors.some((e) => e.message.includes('used more than once in your file'))).toBe(true) - }) - - it('detects position already used in database', async () => { - const storageType = await createTestStorageType(testDb, { name: 'Freezer' }) - const loc = await createTestLocation(testDb, { - name: 'Loc', - storageTypeId: String(storageType.id), - canContainCollections: true, - }) - const plate = await createTestMicronixPlate(testDb, { name: 'Plate1', locationId: loc.id }) - const now = utcNow() - const container = await createTestStorageContainer(testDb) - await testDb.insert(micronixTube).values({ - id: container.id, - collectionId: plate.id, - barcode: 'TAKEN', - position: 'A01', - created: now, - lastUpdated: now, - }) - - const errors = await collectContainerPlacementErrors(testDb, [ - { - containerType: 'micronix_tube', - collectionId: plate.id, - collectionKey: `micronix_plate-${plate.name}`, - normalizedPosition: 'A01', - barcode: 'NEW1', - }, - ]) - - expect(errors.some((e) => e.message.includes('already used in this plate'))).toBe(true) - }) - }) -}) diff --git a/packages/api/src/lib/__tests__/container-registration-parity.test.ts b/packages/api/src/lib/__tests__/container-registration-parity.test.ts index dc387ba0..a42ccfe2 100644 --- a/packages/api/src/lib/__tests__/container-registration-parity.test.ts +++ b/packages/api/src/lib/__tests__/container-registration-parity.test.ts @@ -1,5 +1,6 @@ /** - * Write-path parity: specimens bulk and combined import produce equivalent Containers. + * Write-path parity: every path that creates Containers goes through Specimen registration + * (ADR 0010), so the same container input produces the same rows from each path. */ import { describe, it, expect, beforeEach, afterEach } from 'vitest' import { eq } from 'drizzle-orm' diff --git a/packages/api/src/lib/__tests__/registration-orchestrator.test.ts b/packages/api/src/lib/__tests__/registration-orchestrator.test.ts index f5ee25c9..19aa2a05 100644 --- a/packages/api/src/lib/__tests__/registration-orchestrator.test.ts +++ b/packages/api/src/lib/__tests__/registration-orchestrator.test.ts @@ -11,7 +11,7 @@ import { createTestStorageContainer, } from '../../__tests__/helpers/factories' import { setContainerDefaults } from '../settings' -import { validateBulkSpecimenRows } from '../registration-orchestrator' +import { createBulkSpecimenRows, validateBulkSpecimenRows, type BulkSpecimenValidateRow } from '../registration-orchestrator' import { specimenTypeContainerType, containerTypeUnit, micronixTube, staticWell } from '../../db/schema' import type { Database } from '../../db/client' import { utcNow } from '../datetime' @@ -139,7 +139,7 @@ describe('registration-orchestrator', () => { ]) expect(result.valid).toBe(false) - expect(result.errors.some((e) => e.message.includes('used more than once in your file'))).toBe(true) + expect(result.errors.some((e) => e.message.includes('used more than once in this request'))).toBe(true) }) it('returns invalid when barcode already exists in database', async () => { @@ -227,4 +227,63 @@ describe('registration-orchestrator', () => { expect(result.errors.some((e) => e.message.includes('Position A01 is already used'))).toBe(true) }) }) + + describe('validate passes exactly when the import succeeds', () => { + let locationId: number + + beforeEach(async () => { + const study = await createTestStudy(testDb, { title: 'Study 1', shortCode: 'ST1' }) + await createTestStudySubject(testDb, { studyId: study.id, name: 'Subj1' }) + const specimenType = await createTestSpecimenType(testDb, { name: 'DNA' }) + const storageType = await createTestStorageType(testDb, { name: 'Freezer' }) + locationId = (await createTestLocation(testDb, { name: 'Loc', storageTypeId: String(storageType.id), canContainCollections: true })).id + const unit = await createTestUnit(testDb, { symbol: 'uL', name: 'microliter', category: 'volume' }) + const now = utcNow() + for (const containerType of ['micronix_tube', 'paper'] as const) { + await testDb.insert(specimenTypeContainerType).values({ specimenTypeId: specimenType.id, containerType, created: now, lastUpdated: now }) + await testDb.insert(containerTypeUnit).values({ containerType, unitId: unit.id }) + } + await setContainerDefaults(testDb, { + micronix_tube: { totalQuantity: 1, remainingQuantity: 1, defaultUnitSymbol: 'uL' }, + paper: { totalQuantity: 1, remainingQuantity: 1, defaultUnitSymbol: 'uL' }, + }) + await createTestMicronixPlate(testDb, { name: 'P1', barcode: 'P1-BC', locationId }) + }) + + function row(container?: BulkSpecimenValidateRow['container'], collectionDate = '2024-01-15'): BulkSpecimenValidateRow { + return { sourceType: 'subject', studyShortCode: 'ST1', subjectName: 'Subj1', specimenTypeName: 'DNA', collectionDate, container } + } + + function tube(barcode: string, collection: Record): BulkSpecimenValidateRow['container'] { + return { containerType: 'micronix_tube', barcode, collection: { type: 'micronix_plate', ...collection } } + } + + function paperOn(sheetName: string, sublabel: string): BulkSpecimenValidateRow['container'] { + return { + containerType: 'paper', + sublabel, + collection: { type: 'sheet', name: sheetName, parent: { type: 'box', name: 'PaperBox', locationId } }, + } + } + + const fixtures: Array<{ name: string; valid: boolean; rows: () => BulkSpecimenValidateRow[] }> = [ + { name: 'existing plate name with an unknown barcode', valid: false, rows: () => [row(tube('T1', { name: 'P1', barcode: 'UNKNOWN', position: 'A01' }))] }, + { name: 'two papers on one new sheet', valid: true, rows: () => [row(paperOn('Sheet-1', 's1')), row(paperOn('Sheet-1', 's2'), '2024-02-01')] }, + { name: 'one well named by plate name and by plate barcode', valid: false, rows: () => [row(tube('T1', { name: 'P1', position: 'C01' })), row(tube('T2', { barcode: 'P1-BC', position: 'C1' }), '2024-02-01')] }, + { name: 'a new plate given two locations', valid: false, rows: () => [row(tube('T1', { name: 'NEW', locationId, position: 'A01' })), row(tube('T2', { name: 'NEW', locationId: locationId + 1000, position: 'A02' }), '2024-02-01')] }, + ] + + for (const fixture of fixtures) { + it(fixture.name, async () => { + const validation = await validateBulkSpecimenRows(testDb, fixture.rows()) + const created = await createBulkSpecimenRows(testDb, fixture.rows()).then( + (result) => result.success, + () => false, + ) + + expect(validation.valid).toBe(fixture.valid) + expect(created).toBe(validation.valid) + }) + } + }) }) diff --git a/packages/api/src/lib/bulk-combined-import.ts b/packages/api/src/lib/bulk-combined-import.ts index 83ad00fa..0476853b 100644 --- a/packages/api/src/lib/bulk-combined-import.ts +++ b/packages/api/src/lib/bulk-combined-import.ts @@ -1,52 +1,47 @@ /** * Bulk combined import (subjects + specimens + containers) with configurable atomicity. - * Used by POST /imports/bulk-combined and shares logic with POST /subjects/with-specimens. + * Used by POST /imports/bulk-combined (and its validate route) and POST /subjects/with-specimens. + * Subjects are found or created here; Specimens and Containers go through Specimen registration. + * Validation is the import itself, rolled back (ADR 0010). */ import type { Database } from '../db/client' -import { - studySubject, - specimen, - specimenType, -} from '../db/schema' +import { studySubject, type specimen } from '../db/schema' import { and, eq } from 'drizzle-orm' -import { findExistingStudySpecimen } from './specimen-helpers' -import { - validateStudyShortCode, - validateSubjectName, - validateCollectionDate, - validateContainerTypeForSpecimenType, - validateUnitForContainerType, -} from './validation' -import { getDefaultUnit, getDefaultTotalQuantity, getDefaultRemainingQuantity } from './defaults' +import { validateStudyShortCode, validateSubjectName } from './validation' import { utcNow } from './datetime' -import { resolveSubjectByNameAndStudy, resolveSpecimenTypeByName } from './identifier-resolution' -import { ValidationError } from './error-handler' -import { collectContainerPlacementErrors } from './container-placement-validation' +import { resolveSpecimenTypeByName } from './identifier-resolution' +import { ConflictError, ValidationError } from './error-handler' +import { withRolledBackTransaction, withWriteTransaction } from '../db/write-transaction' import { - bulkCombinedCollectionMessages, - prepareCombinedSubjectContainerBatch, -} from './registration-prepare' -import { createContainerForSpecimen, pickContainerQuantity, validateContainerWriteFields } from './container-creation' -import { - assertWriteInputPlacementResolvable, - toContainerWriteInput, - type BulkCombinedContainerInput, -} from './container-write-placement' -import { withWriteTransaction } from '../db/write-transaction' -import { insertedRow } from './inserted-row' - -export type { BulkCombinedContainerInput } + RefusedWithErrors, + registerSpecimens, + SpecimenRegistrationError, + type RegistrationContainer, + type RegistrationEntry, + type RegistrationResult, +} from './specimen-registration' export { normalizePosition } from './normalize-position' +export interface BulkCombinedSpecimenInput { + specimenTypeName: string + collectionDate?: string + container?: RegistrationContainer + /** CSV row, echoed back on errors. */ + rowIndex?: number +} + +export interface BulkCombinedSubjectInput { + /** Overrides the payload-level study. */ + studyShortCode?: string + subjectName: string + specimens: BulkCombinedSpecimenInput[] +} + export interface WithSpecimensPayload { studyShortCode: string subjectName: string - specimens: Array<{ - specimenTypeName: string - collectionDate?: string - container?: BulkCombinedContainerInput - }> + specimens: BulkCombinedSpecimenInput[] } export interface OneSubjectResult { @@ -58,268 +53,174 @@ export interface OneSubjectResult { containerId?: number specimenCreated: boolean }> - summary: { - subjectsCreated: number - subjectsUpdated: number - specimensCreated: number - containersCreated: number - } + summary: BulkCombinedSummary } -interface PreparedSubject { - studyId: number - existingSubjectId: number | null - trimmedName: string - resolvedSpecimens: Array<{ - specimenTypeId: number - collectionDate?: string - container?: BulkCombinedContainerInput - }> - preparedContainers: Array<{ unitId: number; totalQuantity: number; remainingQuantity: number }> - collectionMap: Map +interface BulkCombinedSummary { + subjectsCreated: number + subjectsUpdated: number + specimensCreated: number + containersCreated: number } -async function revalidatePreparedSubjectInTx( - tx: Database, - prepared: PreparedSubject, +export interface BulkCombinedError { subjectIndex: number -): Promise { - const subjectLabel = `subject ${subjectIndex + 1} ('${prepared.trimmedName}')` - - if (prepared.existingSubjectId) { - const existing = tx.select({ id: studySubject.id }).from(studySubject).where(eq(studySubject.id, prepared.existingSubjectId)).get() - if (!existing) { - throw new ValidationError(`${subjectLabel}: existing subject no longer exists`) - } - } - - for (let i = 0; i < prepared.resolvedSpecimens.length; i++) { - const spec = prepared.resolvedSpecimens[i] - const specimenLabel = `${subjectLabel}, specimen ${i + 1}` - const typeExists = tx.select({ id: specimenType.id }).from(specimenType).where(eq(specimenType.id, spec.specimenTypeId)).get() - if (!typeExists) { - throw new ValidationError(`${specimenLabel}: specimen type no longer exists`) - } + specimenIndex?: number + rowIndex?: number + message: string +} - if (spec.container?.containerType) { - const container = spec.container - const containerTypeValidation = await validateContainerTypeForSpecimenType( - tx, - spec.specimenTypeId, - container.containerType - ) - if (!containerTypeValidation.valid) { - throw new ValidationError(`${specimenLabel}: ${containerTypeValidation.error ?? 'invalid container type for specimen type'}`) - } - const unitValidation = await validateUnitForContainerType( - tx, - container.containerType, - prepared.preparedContainers[i].unitId - ) - if (!unitValidation.valid) { - throw new ValidationError(`${specimenLabel}: ${unitValidation.error ?? 'invalid unit for container type'}`) - } +/** The subjects could not all be registered; nothing was written. */ +class BulkCombinedRejected extends RefusedWithErrors {} - const writeInput = toContainerWriteInput(container) - const fieldValidation = validateContainerWriteFields(writeInput) - if (!fieldValidation.valid) { - throw new ValidationError(`${specimenLabel}: ${fieldValidation.error ?? 'Invalid container data'}`) - } - try { - await assertWriteInputPlacementResolvable(tx, writeInput, prepared.collectionMap) - } catch (error: unknown) { - const message = error instanceof Error ? error.message : 'Invalid container placement' - throw new ValidationError(`${specimenLabel}: ${message}`) - } - } - } +/** A subject's own study wins over the payload-level default. */ +export function resolveSubjectStudyShortCode( + payloadStudyShortCode: string | undefined, + subject: { studyShortCode?: string } +): string | undefined { + return subject.studyShortCode ?? payloadStudyShortCode } -export async function prepareSubjectWithSpecimens( +/** + * Find or create each subject, then register all their specimens in one call. Runs inside the + * caller's transaction and throws BulkCombinedRejected with every problem found. + */ +async function registerSubjects( database: Database, - studyShortCode: string, - subjectName: string, - specimens: WithSpecimensPayload['specimens'] -): Promise { - const studyValidation = await validateStudyShortCode(database, studyShortCode) - if (!studyValidation.valid || !studyValidation.studyId) { - throw new ValidationError(studyValidation.error ?? 'Invalid study') - } - const studyId = studyValidation.studyId - const trimmedName = subjectName.trim() - const existingSubjectId = await resolveSubjectByNameAndStudy(database, trimmedName, studyId) - if (!existingSubjectId) { - const nameValidation = await validateSubjectName(database, studyId, trimmedName) - if (!nameValidation.valid) throw new ValidationError(nameValidation.error ?? 'Invalid subject name') - } - - const resolvedSpecimens: PreparedSubject['resolvedSpecimens'] = [] - for (let i = 0; i < specimens.length; i++) { - const spec = specimens[i] - const specimenTypeId = await resolveSpecimenTypeByName(database, spec.specimenTypeName) - if (!specimenTypeId) { - throw new ValidationError(`Specimen type '${spec.specimenTypeName}' not found`, { specimenIndex: i }) + payloadStudyShortCode: string | undefined, + subjects: BulkCombinedSubjectInput[], + userId: number | undefined, + inputName?: string, +): Promise { + const now = utcNow() + const errors: BulkCombinedError[] = [] + const entries: RegistrationEntry[] = [] + const entryRefs: Array<{ subjectIndex: number; specimenIndex: number; rowIndex?: number }> = [] + const registered: Array<{ subject: typeof studySubject.$inferSelect; created: boolean } | null> = [] + const studyIds = new Map>>() + const specimenTypeIds = new Map() + + for (let subjectIndex = 0; subjectIndex < subjects.length; subjectIndex++) { + const subj = subjects[subjectIndex] + const subjectError = (message: string) => + errors.push({ subjectIndex, specimenIndex: 0, rowIndex: subj.specimens[0]?.rowIndex, message }) + + const code = resolveSubjectStudyShortCode(payloadStudyShortCode, subj) + if (!code) { + subjectError('Study short code is required') + registered.push(null) + continue } - const dateValidation = validateCollectionDate(spec.collectionDate) - if (!dateValidation.valid) { - throw new ValidationError(dateValidation.error, { specimenIndex: i }) + let study = studyIds.get(code) + if (!study) { + study = await validateStudyShortCode(database, code) + studyIds.set(code, study) } - resolvedSpecimens.push({ - specimenTypeId, - collectionDate: dateValidation.normalized, - container: spec.container, - }) - } - - const containerPrep = await prepareCombinedSubjectContainerBatch(database, resolvedSpecimens, { - messages: bulkCombinedCollectionMessages, - }) - if (!containerPrep.valid) { - throw new ValidationError(containerPrep.message, { specimenIndex: containerPrep.specimenIndex }) - } - - const { placementRows, collectionMap } = containerPrep.result - const placementErrors = await collectContainerPlacementErrors(database, placementRows) - if (placementErrors.length > 0) { - throw new ValidationError(placementErrors[0].message, { specimenIndex: placementErrors[0].rowIndex }) - } - - const preparedContainers: PreparedSubject['preparedContainers'] = [] - for (const spec of resolvedSpecimens) { - if (spec.container?.containerType) { - const containerType = spec.container.containerType - const unitId = spec.container.unitId ?? (await getDefaultUnit(database, containerType)) - const unitValidation = await validateUnitForContainerType(database, containerType, unitId) - if (!unitValidation.valid) throw new ValidationError(unitValidation.error ?? 'Invalid unit for container type') - const defaultTotalQty = await getDefaultTotalQuantity(database, containerType) - const defaultRemainingQty = await getDefaultRemainingQuantity(database, containerType) - preparedContainers.push({ - unitId, - totalQuantity: spec.container.totalQuantity ?? defaultTotalQty, - remainingQuantity: spec.container.remainingQuantity ?? spec.container.totalQuantity ?? defaultRemainingQty, + if (!study.valid || !study.studyId) { + subjectError(study.error ?? 'Invalid study') + registered.push(null) + continue + } + const studyId = study.studyId + + // Names are unique per study: a subject named again (in the database or earlier in this + // file) is reused. + const name = subj.subjectName.trim() + const existing = database + .select() + .from(studySubject) + .where(and(eq(studySubject.studyId, studyId), eq(studySubject.name, name))) + .get() + let subject = existing + if (!subject) { + const nameValidation = await validateSubjectName(database, studyId, name) + if (!nameValidation.valid) { + subjectError(nameValidation.error ?? 'Invalid subject name') + registered.push(null) + continue + } + subject = database + .insert(studySubject) + .values({ studyId, name, created: now, lastUpdated: now, createdBy: userId, updatedBy: userId }) + .returning() + .get() + } + registered.push({ subject, created: !existing }) + + for (let specimenIndex = 0; specimenIndex < subj.specimens.length; specimenIndex++) { + const spec = subj.specimens[specimenIndex] + let specimenTypeId = specimenTypeIds.get(spec.specimenTypeName) + if (specimenTypeId === undefined) { + specimenTypeId = await resolveSpecimenTypeByName(database, spec.specimenTypeName) + specimenTypeIds.set(spec.specimenTypeName, specimenTypeId) + } + if (!specimenTypeId) { + errors.push({ + subjectIndex, + specimenIndex, + rowIndex: spec.rowIndex, + message: `Specimen type '${spec.specimenTypeName}' not found`, + }) + continue + } + entries.push({ + source: { subjectId: subject.id }, + specimenTypeId, + collectionDate: spec.collectionDate, + containers: spec.container ? [spec.container] : [], }) - } else { - preparedContainers.push({ unitId: 0, totalQuantity: 0, remainingQuantity: 0 }) + entryRefs.push({ subjectIndex, specimenIndex, rowIndex: spec.rowIndex }) } } - return { - studyId, - existingSubjectId, - trimmedName, - resolvedSpecimens, - preparedContainers, - collectionMap, + let registration: RegistrationResult | null = null + try { + registration = await registerSpecimens(database, entries, { userId, inputName }) + } catch (error) { + if (!(error instanceof SpecimenRegistrationError)) throw error + const mapped = error.errors.map((e) => ({ ...entryRefs[e.entryIndex], message: e.message })) + if (error.status === 409) throw new BulkCombinedRejected(409, [...errors, ...mapped]) + errors.push(...mapped) } -} - -export async function createSubjectWithSpecimensInTx( - tx: Database, - prepared: PreparedSubject, - userId: number | undefined, - now: string -): Promise { - const { studyId, existingSubjectId, trimmedName, resolvedSpecimens, preparedContainers, collectionMap } = prepared - - let subjectId: number - let subject: typeof studySubject.$inferSelect - // A subject prepared as new may already exist by now: an earlier entry in the same file - // with the same trimmed name, or a concurrent import. Reuse it (names are unique per study). - const alreadyInStudy = existingSubjectId - ? undefined - : tx.select().from(studySubject).where(and(eq(studySubject.studyId, studyId), eq(studySubject.name, trimmedName))).get() - const reuseSubjectId = existingSubjectId ?? alreadyInStudy?.id ?? null - if (reuseSubjectId) { - const existing = alreadyInStudy ?? tx.select().from(studySubject).where(eq(studySubject.id, reuseSubjectId)).get() - if (!existing) throw new Error('Subject not found') - subject = existing - subjectId = existing.id - } else { - const newSubjectResult = tx - .insert(studySubject) - .values({ - studyId, - name: trimmedName, - created: now, - lastUpdated: now, - createdBy: userId, - updatedBy: userId, - }) - .returning() - .all() - subject = insertedRow(newSubjectResult, 'study subject') - subjectId = subject.id + if (errors.length > 0 || !registration) { + errors.sort((a, b) => a.subjectIndex - b.subjectIndex || (a.specimenIndex ?? 0) - (b.specimenIndex ?? 0)) + throw new BulkCombinedRejected(400, errors) } - const insertedSpecimens: OneSubjectResult['specimens'] = [] - for (let i = 0; i < resolvedSpecimens.length; i++) { - const spec = resolvedSpecimens[i] - const preparedContainer = preparedContainers[i] - const existingSpecimen = findExistingStudySpecimen(tx, subjectId, spec.specimenTypeId, spec.collectionDate) - let specimenRecord: typeof specimen.$inferSelect - let specimenCreated: boolean - if (existingSpecimen) { - specimenRecord = existingSpecimen - specimenCreated = false - } else { - const newSpecimenResult = tx - .insert(specimen) - .values({ - studySubjectId: subjectId, - specimenTypeId: spec.specimenTypeId, - collectionDate: spec.collectionDate, - created: now, - lastUpdated: now, - createdBy: userId, - updatedBy: userId, - }) - .returning() - .all() - specimenRecord = insertedRow(newSpecimenResult, 'specimen') - specimenCreated = true - } - const specimenId = specimenRecord.id - let containerCreated = false - let containerId: number | undefined - - if (spec.container?.containerType) { - const container = spec.container - const writeInput = toContainerWriteInput(container) - const containerResult = await createContainerForSpecimen(specimenId, writeInput, tx, { - userId, - collectionMap, - quantity: pickContainerQuantity(preparedContainer), - }) - if (!containerResult.success || !containerResult.containerId) { - throw new ValidationError( - containerResult.error ?? 'Failed to create container', - { specimenIndex: i } - ) - } - containerId = containerResult.containerId - containerCreated = true + const results: OneSubjectResult[] = registered.map((r) => { + if (!r) throw new Error('Subject missing after a registration without errors') + return { + subject: r.subject, + subjectCreated: r.created, + specimens: [], + summary: { subjectsCreated: r.created ? 1 : 0, subjectsUpdated: r.created ? 0 : 1, specimensCreated: 0, containersCreated: 0 }, } - - insertedSpecimens.push({ - specimen: specimenRecord, - containerCreated, + }) + registration.entries.forEach((entry, i) => { + const result = results[entryRefs[i].subjectIndex] + const containerId = entry.containerIds.at(0) + result.specimens.push({ + specimen: entry.specimen, + specimenCreated: !entry.reused, + containerCreated: containerId !== undefined, containerId, - specimenCreated, }) - } + if (!entry.reused) result.summary.specimensCreated += 1 + if (containerId !== undefined) result.summary.containersCreated += 1 + }) + return results +} - const subjectCreated = !reuseSubjectId - return { - subject, - subjectCreated, - specimens: insertedSpecimens, - summary: { - subjectsCreated: subjectCreated ? 1 : 0, - subjectsUpdated: subjectCreated ? 0 : 1, - specimensCreated: insertedSpecimens.filter((s) => s.specimenCreated).length, - containersCreated: insertedSpecimens.filter((s) => s.containerCreated).length, - }, - } +function sumSummaries(results: OneSubjectResult[]): BulkCombinedSummary { + return results.reduce( + (acc, r) => ({ + subjectsCreated: acc.subjectsCreated + r.summary.subjectsCreated, + subjectsUpdated: acc.subjectsUpdated + r.summary.subjectsUpdated, + specimensCreated: acc.specimensCreated + r.summary.specimensCreated, + containersCreated: acc.containersCreated + r.summary.containersCreated, + }), + { subjectsCreated: 0, subjectsUpdated: 0, specimensCreated: 0, containersCreated: 0 } + ) } export async function runOneSubjectWithSpecimens( @@ -327,59 +228,31 @@ export async function runOneSubjectWithSpecimens( payload: WithSpecimensPayload, userId: number | undefined ): Promise { - const prepared = await prepareSubjectWithSpecimens( - database, - payload.studyShortCode, - payload.subjectName, - payload.specimens - ) - const now = utcNow() - return withWriteTransaction(database, async (tx) => { - return createSubjectWithSpecimensInTx(tx, prepared, userId, now) - }) + try { + const [result] = await withWriteTransaction(database, (tx) => + registerSubjects(tx, payload.studyShortCode, [{ subjectName: payload.subjectName, specimens: payload.specimens }], userId) + ) + return result + } catch (error) { + if (!(error instanceof BulkCombinedRejected)) throw error + const details = { specimenIndex: error.errors[0]?.specimenIndex, errors: error.errors } + if (error.status === 409) throw new ConflictError(error.message, details) + throw new ValidationError(error.message, details) + } } +/** The bulk combined routes take a CSV import, so a value repeated in it is "used more than once in your file". */ +const CSV_INPUT_NAME = 'your file' + export interface BulkCombinedPayload { /** Default study for subjects that do not set their own studyShortCode. */ studyShortCode?: string atomicMode: 'full_file' | 'per_subject' - subjects: Array<{ - studyShortCode?: string - subjectName: string - specimens: Array<{ - specimenTypeName: string - collectionDate?: string - container?: BulkCombinedContainerInput - }> - }> -} - -/** A subject's own study wins over the payload-level default. */ -export function resolveSubjectStudyShortCode( - payloadStudyShortCode: string | undefined, - subject: { studyShortCode?: string } -): string | undefined { - return subject.studyShortCode ?? payloadStudyShortCode -} - -function requireSubjectStudyShortCode( - payloadStudyShortCode: string | undefined, - subject: { studyShortCode?: string; subjectName: string } -): string { - const studyShortCode = resolveSubjectStudyShortCode(payloadStudyShortCode, subject) - if (!studyShortCode) { - throw new ValidationError(`Subject '${subject.subjectName}': study short code is required`) - } - return studyShortCode + subjects: BulkCombinedSubjectInput[] } export interface BulkCombinedResult { - summary: { - subjectsCreated: number - subjectsUpdated: number - specimensCreated: number - containersCreated: number - } + summary: BulkCombinedSummary results: OneSubjectResult[] errors?: Array<{ index: number; error: string }> } @@ -390,85 +263,60 @@ export async function runBulkCombinedImport( userId: number | undefined ): Promise { const { studyShortCode, atomicMode, subjects } = payload - const results: OneSubjectResult[] = [] - const errors: Array<{ index: number; error: string }> = [] if (atomicMode === 'per_subject') { + const results: OneSubjectResult[] = [] + const errors: Array<{ index: number; error: string }> = [] for (let i = 0; i < subjects.length; i++) { try { - const one = await runOneSubjectWithSpecimens( - database, - { - studyShortCode: requireSubjectStudyShortCode(studyShortCode, subjects[i]), - subjectName: subjects[i].subjectName, - specimens: subjects[i].specimens, - }, - userId - ) + const [one] = await withWriteTransaction(database, (tx) => registerSubjects(tx, studyShortCode, [subjects[i]], userId, CSV_INPUT_NAME)) results.push(one) - } catch (err) { - errors.push({ - index: i, - error: `Subject '${subjects[i].subjectName}': ${err instanceof Error ? err.message : 'failed to create subject with specimens'}`, - }) + } catch (error) { + if (!(error instanceof BulkCombinedRejected)) throw error + errors.push({ index: i, error: `Subject '${subjects[i].subjectName}': ${error.errors.map((e) => e.message).join('; ')}` }) } } - const summary = results.reduce( - (acc, r) => ({ - subjectsCreated: acc.subjectsCreated + r.summary.subjectsCreated, - subjectsUpdated: acc.subjectsUpdated + r.summary.subjectsUpdated, - specimensCreated: acc.specimensCreated + r.summary.specimensCreated, - containersCreated: acc.containersCreated + r.summary.containersCreated, - }), - { subjectsCreated: 0, subjectsUpdated: 0, specimensCreated: 0, containersCreated: 0 } - ) - return { summary, results, errors: errors.length > 0 ? errors : undefined } + return { summary: sumSummaries(results), results, errors: errors.length > 0 ? errors : undefined } } - // full_file: one transaction for all subjects - const allPrepared: PreparedSubject[] = [] - const mergedCollectionMap = new Map() - for (let i = 0; i < subjects.length; i++) { - const prepared = await prepareSubjectWithSpecimens( - database, - requireSubjectStudyShortCode(studyShortCode, subjects[i]), - subjects[i].subjectName, - subjects[i].specimens - ) - for (const [k, v] of prepared.collectionMap) mergedCollectionMap.set(k, v) - allPrepared.push(prepared) + try { + const results = await withWriteTransaction(database, (tx) => registerSubjects(tx, studyShortCode, subjects, userId, CSV_INPUT_NAME)) + return { summary: sumSummaries(results), results } + } catch (error) { + if (!(error instanceof BulkCombinedRejected)) throw error + const first = error.errors.at(0) + const where = first ? `Subject '${subjects[first.subjectIndex].subjectName.trim()}' (index ${first.subjectIndex + 1}) failed: ` : '' + if (error.status === 409) throw new ConflictError(`${where}${error.message}`, { errors: error.errors }) + throw new ValidationError(`${where}${error.message}`, { errors: error.errors }) } +} - const now = utcNow() - const fullResults = await withWriteTransaction(database, async (tx) => { - for (let i = 0; i < allPrepared.length; i++) { - await revalidatePreparedSubjectInTx(tx, allPrepared[i], i) - } - - const out: OneSubjectResult[] = [] - for (let i = 0; i < allPrepared.length; i++) { - const prepared = allPrepared[i] - const prepWithMergedMap = { ...prepared, collectionMap: mergedCollectionMap } +/** + * Check a bulk combined import by running it and rolling it back, in the payload's atomicMode, + * and report every problem. Validate passes exactly when the import would succeed. + * + * per_subject runs each subject in its own savepoint, as the import does: a failed subject's + * writes are rolled back before the next subject runs, so a subject that relies on a collection + * a failed subject would have created is flagged too. + */ +export async function validateBulkCombinedPayload( + database: Database, + payload: BulkCombinedPayload +): Promise<{ valid: boolean; errors: BulkCombinedError[] }> { + const { studyShortCode, atomicMode, subjects } = payload + return withRolledBackTransaction(database, async (tx) => { + const errors: BulkCombinedError[] = [] + const batches = atomicMode === 'per_subject' ? subjects.map((subject, i) => ({ first: i, subjects: [subject] })) : [{ first: 0, subjects }] + for (const batch of batches) { try { - out.push(await createSubjectWithSpecimensInTx(tx, prepWithMergedMap, userId, now)) - } catch (error: unknown) { - const message = error instanceof Error ? error.message : 'Failed to create subject with specimens' - throw new ValidationError(`Subject '${prepared.trimmedName}' (index ${i + 1}) failed: ${message}`) + await withWriteTransaction(tx, (savepoint) => registerSubjects(savepoint, studyShortCode, batch.subjects, undefined, CSV_INPUT_NAME)) + } catch (error) { + if (!(error instanceof BulkCombinedRejected)) throw error + errors.push(...error.errors.map((e) => ({ ...e, subjectIndex: batch.first + e.subjectIndex }))) } } - return out + return { valid: errors.length === 0, errors } }) - - const summary = fullResults.reduce( - (acc, r) => ({ - subjectsCreated: acc.subjectsCreated + r.summary.subjectsCreated, - subjectsUpdated: acc.subjectsUpdated + r.summary.subjectsUpdated, - specimensCreated: acc.specimensCreated + r.summary.specimensCreated, - containersCreated: acc.containersCreated + r.summary.containersCreated, - }), - { subjectsCreated: 0, subjectsUpdated: 0, specimensCreated: 0, containersCreated: 0 } - ) - return { summary, results: fullResults } } /** HTTP response shape for POST /subjects/with-specimens */ diff --git a/packages/api/src/lib/bulk-combined-validate.ts b/packages/api/src/lib/bulk-combined-validate.ts deleted file mode 100644 index 2d7a7f71..00000000 --- a/packages/api/src/lib/bulk-combined-validate.ts +++ /dev/null @@ -1,223 +0,0 @@ -/** - * Upfront validation for bulk-combined import. Runs all DB and payload checks - * without performing any inserts; returns all errors for display. - */ -import type { Database } from '../db/client' -import { - validateStudyShortCode, - validateSubjectName, - validateCollectionDate, - validateUnitForContainerType, -} from './validation' -import { getDefaultUnit } from './defaults' -import { resolveSubjectByNameAndStudy, resolveSpecimenTypeByName } from './identifier-resolution' -import { - resolveSubjectStudyShortCode, - type BulkCombinedContainerInput, - type BulkCombinedPayload, -} from './bulk-combined-import' -import { - bulkCombinedCollectionMessages, - prepareCombinedSubjectContainerBatch, -} from './registration-prepare' -import { - collectContainerPlacementErrors, - type ContainerPlacementCheckRow, -} from './container-placement-validation' -/** Map orchestrator field messages to combined-import UX copy. */ -function combinedContainerErrorMessage(message: string): string { - if (message.includes('Position (well) is required')) { - return 'Position is required for this container type (e.g. A01)' - } - return message -} - -export interface BulkCombinedValidateError { - subjectIndex: number - specimenIndex?: number - rowIndex?: number - message: string -} - -export interface BulkCombinedValidateResult { - valid: boolean - errors: BulkCombinedValidateError[] -} - -/** Payload for validate: same as BulkCombinedPayload but specimens may include optional rowIndex */ -export type BulkCombinedValidatePayload = Omit & { - subjects: Array<{ - studyShortCode?: string - subjectName: string - specimens: Array<{ - specimenTypeName: string - collectionDate?: string - container?: BulkCombinedContainerInput - rowIndex?: number - }> - }> -} - -interface ContainerRowContext { - subjectIndex: number - specimenIndex: number - rowIndex?: number -} - -export async function validateBulkCombinedPayload( - database: Database, - payload: BulkCombinedValidatePayload -): Promise { - const errors: BulkCombinedValidateError[] = [] - const { studyShortCode, subjects } = payload - - const add = (subjectIndex: number, specimenIndex: number, message: string, rowIndex?: number) => { - errors.push({ subjectIndex, specimenIndex, message, ...(rowIndex !== undefined && { rowIndex }) }) - } - - const studyValidations = new Map>>() - const resolveStudyId = async (subjectIndex: number, rowIndex: number | undefined): Promise => { - const code = resolveSubjectStudyShortCode(studyShortCode, subjects[subjectIndex]) - if (!code) { - add(subjectIndex, 0, 'Study short code is required', rowIndex) - return null - } - let studyValidation = studyValidations.get(code) - if (!studyValidation) { - studyValidation = await validateStudyShortCode(database, code) - studyValidations.set(code, studyValidation) - } - if (!studyValidation.valid || !studyValidation.studyId) { - add(subjectIndex, 0, studyValidation.error ?? 'Invalid study', rowIndex) - return null - } - return studyValidation.studyId - } - - const placementRows: ContainerPlacementCheckRow[] = [] - const placementContexts: ContainerRowContext[] = [] - const toBeCreatedKeys = new Set() - - for (let subjectIndex = 0; subjectIndex < subjects.length; subjectIndex++) { - const subj = subjects[subjectIndex] - const trimmedName = subj.subjectName.trim() - const firstRowIndex = subj.specimens[0]?.rowIndex - const studyId = await resolveStudyId(subjectIndex, firstRowIndex) - - if (studyId !== null) { - const existingSubjectId = await resolveSubjectByNameAndStudy(database, trimmedName, studyId) - if (!existingSubjectId) { - const nameValidation = await validateSubjectName(database, studyId, trimmedName) - if (!nameValidation.valid) { - add(subjectIndex, 0, nameValidation.error ?? 'Invalid subject name', firstRowIndex) - } - } - } - - const resolvedForPrepare: Array<{ - specimenTypeId: number - collectionDate?: string - container?: BulkCombinedContainerInput - specimenIndex: number - rowIndex?: number - }> = [] - - for (let specimenIndex = 0; specimenIndex < subj.specimens.length; specimenIndex++) { - const spec = subj.specimens[specimenIndex] - const rowIndex = 'rowIndex' in spec ? spec.rowIndex : undefined - - const specimenTypeId = await resolveSpecimenTypeByName(database, spec.specimenTypeName) - if (!specimenTypeId) { - add(subjectIndex, specimenIndex, `Specimen type '${spec.specimenTypeName}' not found`, rowIndex) - continue - } - - const dateValidation = validateCollectionDate(spec.collectionDate) - if (!dateValidation.valid) { - add(subjectIndex, specimenIndex, dateValidation.error, rowIndex) - continue - } - - if (spec.container?.containerType) { - const containerType = spec.container.containerType - try { - const unitId = spec.container.unitId ?? (await getDefaultUnit(database, containerType)) - const unitValidation = await validateUnitForContainerType(database, containerType, unitId) - if (!unitValidation.valid) { - add(subjectIndex, specimenIndex, unitValidation.error ?? 'Invalid unit for container type', rowIndex) - } - } catch { - add(subjectIndex, specimenIndex, 'Default unit not configured for this container type', rowIndex) - } - } - - resolvedForPrepare.push({ - specimenTypeId, - collectionDate: dateValidation.normalized, - container: spec.container, - specimenIndex, - rowIndex, - }) - } - - if (resolvedForPrepare.length === 0) { - continue - } - - const containerPrep = await prepareCombinedSubjectContainerBatch( - database, - resolvedForPrepare.map(({ specimenTypeId, collectionDate, container }) => ({ - specimenTypeId, - collectionDate, - container, - })), - { messages: bulkCombinedCollectionMessages } - ) - if (!containerPrep.valid) { - const failed = resolvedForPrepare.at(containerPrep.specimenIndex) - add( - subjectIndex, - failed?.specimenIndex ?? containerPrep.specimenIndex, - combinedContainerErrorMessage(containerPrep.message), - failed?.rowIndex - ) - continue - } - - let placementRowIndex = 0 - for (const resolvedRow of resolvedForPrepare) { - if (!resolvedRow.container?.containerType) continue - const prepRow = containerPrep.result.placementRows.at(placementRowIndex) - if (!prepRow) break - - const c = resolvedRow.container.collection - const willCreateByName = - (c && 'locationId' in c && c.locationId != null) || - (c && 'parent' in c && c.parent?.locationId != null) - if (willCreateByName) { - toBeCreatedKeys.add(prepRow.collectionKey) - } - - placementRows.push(prepRow) - placementContexts.push({ - subjectIndex, - specimenIndex: resolvedRow.specimenIndex, - rowIndex: resolvedRow.rowIndex, - }) - placementRowIndex++ - } - } - - const placementErrors = await collectContainerPlacementErrors(database, placementRows) - for (const placementError of placementErrors) { - const context = placementContexts.at(placementError.rowIndex) - if (context) { - add(context.subjectIndex, context.specimenIndex, placementError.message, context.rowIndex) - } - } - - return { - valid: errors.length === 0, - errors, - } -} diff --git a/packages/api/src/lib/container-creation.ts b/packages/api/src/lib/container-creation.ts index 804d23b6..8ffd31fa 100644 --- a/packages/api/src/lib/container-creation.ts +++ b/packages/api/src/lib/container-creation.ts @@ -1,29 +1,8 @@ import type { Database } from '../db/client' -import type { ExtractTablesWithRelations } from 'drizzle-orm' -import type { SQLiteTransaction } from 'drizzle-orm/sqlite-core' -import type * as schema from '../db/schema' -import { - storageContainer, - micronixTube, - cryovialTube, - paper, - staticWell, - specimen, -} from '../db/schema' -import { eq } from 'drizzle-orm' -import { getDefaultUnit, getDefaultTotalQuantity, getDefaultRemainingQuantity } from './defaults' -import { validateUnitForContainerType, validateContainerTypeForSpecimenType } from './validation' -import { utcNow } from './datetime' -import { normalizePosition } from './normalize-position' -import { ValidationError } from './error-handler' import type { ContainerWriteInput } from '@sampledb/contract' -import { ensureContainerPlacement, type EnsuredPlacement } from './container-write-placement' -import { checkGridPositionOccupancy, occupiedGridPositionMessage } from './container-occupancy' -import { insertedRow } from './inserted-row' - -type DatabaseOrTransaction = - | Database - | SQLiteTransaction<'sync', void, typeof schema, ExtractTablesWithRelations> +import type { DatabaseOrTransaction } from './db-types' +import { registerSpecimens, SpecimenRegistrationError } from './specimen-registration' +import { ConflictError } from './error-handler' export type ContainerType = 'micronix_tube' | 'cryovial_tube' | 'paper' | 'static_well' @@ -44,294 +23,14 @@ export function pickContainerQuantity(src: ContainerQuantity): ContainerQuantity export type CreateContainerForSpecimenOptions = { userId?: number - /** Mutable map of collection keys to ids; updated when collections are auto-created. */ - collectionMap?: Map quantity?: ContainerQuantity } -function hasGridCollection(collection: { id?: number; name?: string; barcode?: string } | undefined): boolean { - if (!collection) return false - return collection.id != null || Boolean(collection.name) || Boolean(collection.barcode) -} - /** - * Required ContainerWriteInput fields by type (no DB uniqueness checks). + * Add one Container to an existing Specimen through Specimen registration. + * Callers not yet migrated to registerSpecimens use this; it reports the first problem. + * A write that hits a database constraint throws ConflictError, so every caller replies 409. */ -export function validateContainerWriteFields( - input: ContainerWriteInput -): { valid: boolean; error?: string } { - if (input.containerType === 'micronix_tube') { - if (!input.barcode) { - return { valid: false, error: 'Barcode is required for micronix tubes' } - } - if (!hasGridCollection(input.collection)) { - return { valid: false, error: 'Collection name or barcode is required' } - } - if (!input.collection?.position || String(input.collection.position).trim() === '') { - return { valid: false, error: 'Position (well) is required for micronix tubes.' } - } - } else if (input.containerType === 'cryovial_tube') { - if (!hasGridCollection(input.collection)) { - return { valid: false, error: 'Collection name or barcode is required' } - } - if (!input.collection?.position || String(input.collection.position).trim() === '') { - return { valid: false, error: 'Position (well) is required for cryovial tubes.' } - } - } else if (input.containerType === 'paper') { - const barcode = Reflect.get(input, 'barcode') - if (typeof barcode === 'string' && barcode !== '') { - return { valid: false, error: 'Paper containers use sublabel for spot identifiers, not barcode' } - } - const collection = input.collection - if (!collection) { - return { valid: false, error: 'Collection is required for paper containers' } - } - if (collection.id == null) { - if (!collection.parent || (collection.parent.id == null && !collection.parent.name)) { - return { valid: false, error: 'Collection name is required for papers' } - } - if (!collection.name) { - return { valid: false, error: 'Sheet name is required for papers' } - } - } - } else { - if (!hasGridCollection(input.collection)) { - return { valid: false, error: 'Collection name or barcode is required' } - } - if (!input.collection?.position || String(input.collection.position).trim() === '') { - return { valid: false, error: 'Position (well) is required for static wells.' } - } - } - - return { valid: true } -} - -async function insertStorageContainer( - specimenId: number, - database: DatabaseOrTransaction, - dbForValidation: Database, - containerType: ContainerType, - quantity: ContainerQuantity | undefined, - comment: string | undefined, - userId: number | undefined -): Promise<{ id: number } | { error: string }> { - const defaultUnitId = await getDefaultUnit(dbForValidation, containerType) - const finalUnitId = quantity?.unitId || defaultUnitId - - const unitValidation = await validateUnitForContainerType(dbForValidation, containerType, finalUnitId) - if (!unitValidation.valid) { - return { error: unitValidation.error ?? 'Invalid unit for container type' } - } - - const defaultTotalQty = await getDefaultTotalQuantity(dbForValidation, containerType) - const defaultRemainingQty = await getDefaultRemainingQuantity(dbForValidation, containerType) - - const now = utcNow() - const inserted = await database.insert(storageContainer).values({ - specimenId, - unitId: finalUnitId, - totalQuantity: quantity?.totalQuantity ?? defaultTotalQty, - remainingQuantity: quantity?.remainingQuantity ?? quantity?.totalQuantity ?? defaultRemainingQty, - comment, - created: now, - lastUpdated: now, - createdBy: userId, - updatedBy: userId, - }).returning() - - return { id: insertedRow(inserted, 'container').id } -} - -async function createMicronixTube( - specimenId: number, - placement: Extract, - database: DatabaseOrTransaction, - options?: CreateContainerForSpecimenOptions -): Promise<{ success: boolean; containerId?: number; error?: string }> { - try { - const dbForValidation = database as unknown as Database - const position = normalizePosition(placement.position) - const occupancy = await checkGridPositionOccupancy(dbForValidation, { - collectionKind: 'micronix_plate', - collectionId: placement.collectionId, - position, - }) - if (occupancy.occupied && position) { - return { success: false, error: occupiedGridPositionMessage(position, 'micronix_plate') } - } - - if (placement.barcode) { - const existing = await database - .select({ id: micronixTube.id }) - .from(micronixTube) - .where(eq(micronixTube.barcode, placement.barcode)) - .get() - if (existing) { - return { success: false, error: `Barcode '${placement.barcode}' already exists` } - } - } - - const inserted = await insertStorageContainer( - specimenId, - database, - dbForValidation, - 'micronix_tube', - options?.quantity, - placement.comment, - options?.userId - ) - if ('error' in inserted) return { success: false, error: inserted.error } - - await database.insert(micronixTube).values({ - id: inserted.id, - collectionId: placement.collectionId, - barcode: placement.barcode, - position, - }) - - return { success: true, containerId: inserted.id } - } catch (error: unknown) { - if (error instanceof ValidationError) { - return { success: false, error: error.message } - } - throw error - } -} - -async function createCryovialTube( - specimenId: number, - placement: Extract, - database: DatabaseOrTransaction, - options?: CreateContainerForSpecimenOptions -): Promise<{ success: boolean; containerId?: number; error?: string }> { - try { - const dbForValidation = database as unknown as Database - const position = normalizePosition(placement.position) - const occupancy = await checkGridPositionOccupancy(dbForValidation, { - collectionKind: 'cryovial_box', - collectionId: placement.collectionId, - position, - }) - if (occupancy.occupied && position) { - return { success: false, error: occupiedGridPositionMessage(position, 'cryovial_box') } - } - - if (placement.barcode) { - const existing = await database - .select({ id: cryovialTube.id }) - .from(cryovialTube) - .where(eq(cryovialTube.barcode, placement.barcode)) - .get() - if (existing) { - return { success: false, error: `Barcode '${placement.barcode}' already exists` } - } - } - - const inserted = await insertStorageContainer( - specimenId, - database, - dbForValidation, - 'cryovial_tube', - options?.quantity, - placement.comment, - options?.userId - ) - if ('error' in inserted) return { success: false, error: inserted.error } - - await database.insert(cryovialTube).values({ - id: inserted.id, - collectionId: placement.collectionId, - barcode: placement.barcode || null, - position, - }) - - return { success: true, containerId: inserted.id } - } catch (error: unknown) { - if (error instanceof ValidationError) { - return { success: false, error: error.message } - } - throw error - } -} - -async function createPaper( - specimenId: number, - placement: Extract, - database: DatabaseOrTransaction, - options?: CreateContainerForSpecimenOptions -): Promise<{ success: boolean; containerId?: number; error?: string }> { - try { - const dbForValidation = database as unknown as Database - const inserted = await insertStorageContainer( - specimenId, - database, - dbForValidation, - 'paper', - options?.quantity, - placement.comment, - options?.userId - ) - if ('error' in inserted) return { success: false, error: inserted.error } - - await database.insert(paper).values({ - id: inserted.id, - sheetId: placement.sheetId, - sublabel: placement.sublabel?.trim() || null, - }) - - return { success: true, containerId: inserted.id } - } catch (error: unknown) { - if (error instanceof ValidationError) { - return { success: false, error: error.message } - } - throw error - } -} - -async function createStaticWell( - specimenId: number, - placement: Extract, - database: DatabaseOrTransaction, - options?: CreateContainerForSpecimenOptions -): Promise<{ success: boolean; containerId?: number; error?: string }> { - try { - const dbForValidation = database as unknown as Database - const position = normalizePosition(placement.position) - const occupancy = await checkGridPositionOccupancy(dbForValidation, { - collectionKind: 'micronix_plate', - collectionId: placement.collectionId, - position, - }) - if (occupancy.occupied && position) { - return { success: false, error: occupiedGridPositionMessage(position, 'micronix_plate') } - } - - const inserted = await insertStorageContainer( - specimenId, - database, - dbForValidation, - 'static_well', - options?.quantity, - placement.comment, - options?.userId - ) - if ('error' in inserted) return { success: false, error: inserted.error } - - await database.insert(staticWell).values({ - id: inserted.id, - collectionId: placement.collectionId, - position, - }) - - return { success: true, containerId: inserted.id } - } catch (error: unknown) { - if (error instanceof ValidationError) { - return { success: false, error: error.message } - } - throw error - } -} - export async function createContainerForSpecimen( specimenId: number, data: ContainerWriteInput, @@ -340,55 +39,18 @@ export async function createContainerForSpecimen( ): Promise<{ success: boolean; containerId?: number; error?: string }> { const opts: CreateContainerForSpecimenOptions = typeof options === 'number' ? { userId: options } : (options ?? {}) - const dbForValidation = database as unknown as Database - - const fieldValidation = validateContainerWriteFields(data) - if (!fieldValidation.valid) return { success: false, error: fieldValidation.error } - - const specimenRecord = await database - .select({ specimenTypeId: specimen.specimenTypeId }) - .from(specimen) - .where(eq(specimen.id, specimenId)) - .get() - if (!specimenRecord) { - return { success: false, error: 'Specimen not found' } - } - - const containerTypeValidation = await validateContainerTypeForSpecimenType( - dbForValidation, - specimenRecord.specimenTypeId, - data.containerType - ) - if (!containerTypeValidation.valid) { - return { success: false, error: containerTypeValidation.error } - } - - let placement: EnsuredPlacement try { - placement = await ensureContainerPlacement(database, data, { - collectionMap: opts.collectionMap, - userId: opts.userId, - }) - } catch (error: unknown) { - if (error instanceof ValidationError) { + const result = await registerSpecimens( + database as Database, + [{ specimenId, containers: [{ ...data, ...opts.quantity }] }], + { userId: opts.userId } + ) + return { success: true, containerId: result.entries[0]?.containerIds[0] } + } catch (error) { + if (error instanceof SpecimenRegistrationError) { + if (error.status === 409) throw new ConflictError(error.message, { errors: error.errors }) return { success: false, error: error.message } } throw error } - - switch (placement.containerType) { - case 'micronix_tube': - return createMicronixTube(specimenId, placement, database, opts) - case 'cryovial_tube': - return createCryovialTube(specimenId, placement, database, opts) - case 'paper': - return createPaper(specimenId, placement, database, opts) - case 'static_well': - return createStaticWell(specimenId, placement, database, opts) - default: { - const _exhaustive: never = placement - void _exhaustive - return { success: false, error: 'Unsupported container type' } - } - } } diff --git a/packages/api/src/lib/container-placement-validation.ts b/packages/api/src/lib/container-placement-validation.ts deleted file mode 100644 index b78d4ce2..00000000 --- a/packages/api/src/lib/container-placement-validation.ts +++ /dev/null @@ -1,160 +0,0 @@ -/** - * Barcode, well-position, and sheet-name uniqueness checks for containers in one request. - * Shared by bulk specimen validate, bulk-combined validate/import, and POST /subjects/with-specimens. - */ -import type { Database } from '../db/client' -import { micronixTube, cryovialTube } from '../db/schema' -import { eq } from 'drizzle-orm' -import { normalizePosition } from './normalize-position' -import { checkGridPositionOccupancy, occupiedGridPositionMessage } from './container-occupancy' - -export type PlacementContainerType = 'micronix_tube' | 'cryovial_tube' | 'paper' | 'static_well' - -export type ContainerPlacementCheckRow = { - containerType: PlacementContainerType - collectionId: number | null - collectionKey: string - normalizedPosition: string | null - barcode: string | null - boxKey?: string | null - sheetName?: string | null -} - -export type ContainerPlacementError = { - rowIndex: number - message: string -} - -type PlacementContainer = { - containerType: PlacementContainerType - collectionName?: string - collectionBarcode?: string - barcode?: string - position?: string - sheetName?: string - sublabel?: string -} - -export function buildContainerPlacementCheckRow( - container: PlacementContainer, - collectionId: number | null, - collectionKey: string -): ContainerPlacementCheckRow { - const containerType = container.containerType - return { - containerType, - collectionId, - collectionKey, - normalizedPosition: normalizePosition(container.position), - barcode: - containerType === 'paper' - ? container.sublabel?.trim() || null - : container.barcode?.trim() || null, - boxKey: containerType === 'paper' ? collectionKey : null, - sheetName: containerType === 'paper' ? (container.sheetName ?? 'Sheet-1').trim() : null, - } -} - -export async function collectContainerPlacementErrors( - database: Database, - rows: ContainerPlacementCheckRow[] -): Promise { - const errors: ContainerPlacementError[] = [] - const seenBarcodes = new Set() - const seenPositionByCollection = new Map>() - const seenSheetByBox = new Map>() - - for (let rowIndex = 0; rowIndex < rows.length; rowIndex++) { - const row = rows[rowIndex] - const { - containerType, - collectionId, - collectionKey, - normalizedPosition, - barcode, - boxKey, - sheetName, - } = row - - if (containerType === 'micronix_tube' || containerType === 'cryovial_tube') { - if (barcode) { - const existingInDb = - containerType === 'micronix_tube' - ? await database - .select({ id: micronixTube.id }) - .from(micronixTube) - .where(eq(micronixTube.barcode, barcode)) - .get() - : await database - .select({ id: cryovialTube.id }) - .from(cryovialTube) - .where(eq(cryovialTube.barcode, barcode)) - .get() - if (existingInDb) { - errors.push({ - rowIndex, - message: `Barcode '${barcode}' already exists. Use a different barcode.`, - }) - } - if (seenBarcodes.has(barcode)) { - errors.push({ - rowIndex, - message: `Barcode '${barcode}' is used more than once in your file. Each barcode must be unique.`, - }) - } - seenBarcodes.add(barcode) - } - } - - if ( - normalizedPosition && - (containerType === 'micronix_tube' || containerType === 'cryovial_tube' || containerType === 'static_well') - ) { - if (collectionId !== null) { - // Same rule as the write path: tubes and static wells share plate cells. - const collectionKind = containerType === 'cryovial_tube' ? 'cryovial_box' : 'micronix_plate' - const occupancy = await checkGridPositionOccupancy(database, { - collectionKind, - collectionId, - position: normalizedPosition, - }) - if (occupancy.occupied) { - errors.push({ - rowIndex, - message: occupiedGridPositionMessage(normalizedPosition, collectionKind), - }) - } - } - - let positionSet = seenPositionByCollection.get(collectionKey) - if (!positionSet) { - positionSet = new Set() - seenPositionByCollection.set(collectionKey, positionSet) - } - if (positionSet.has(normalizedPosition)) { - errors.push({ - rowIndex, - message: `Position ${normalizedPosition} in this plate/box is used more than once in your file. Each position can only be used once.`, - }) - } - positionSet.add(normalizedPosition) - } - - if (containerType === 'paper' && boxKey && sheetName) { - let sheetSet = seenSheetByBox.get(boxKey) - if (!sheetSet) { - sheetSet = new Set() - seenSheetByBox.set(boxKey, sheetSet) - } - if (sheetSet.has(sheetName)) { - errors.push({ - rowIndex, - message: `Sheet name '${sheetName}' in box is used more than once in your file.`, - }) - } - sheetSet.add(sheetName) - } - } - - return errors -} diff --git a/packages/api/src/lib/container-write-placement.ts b/packages/api/src/lib/container-write-placement.ts deleted file mode 100644 index a12fd018..00000000 --- a/packages/api/src/lib/container-write-placement.ts +++ /dev/null @@ -1,546 +0,0 @@ -import type { ContainerWriteInput } from '@sampledb/contract' -import type { Database } from '../db/client' -import type { ExtractTablesWithRelations } from 'drizzle-orm' -import type { SQLiteTransaction } from 'drizzle-orm/sqlite-core' -import type * as schema from '../db/schema' -import { and, eq } from 'drizzle-orm' -import { bag, box, cryovialBox, micronixPlate, sheet } from '../db/schema' -import { resolveCollection, resolveCollectionByBarcode } from './collections/collection-resolve' -import { - CollectionLocationNotAllowedError, - CollectionLocationNotFoundError, - createOrResolveCollection, - type CollectionEntityType, -} from './collections/collection-lifecycle' -import { utcNow } from './datetime' -import { ValidationError } from './error-handler' -import { insertedRow } from './inserted-row' - -type DatabaseOrTransaction = - | Database - | SQLiteTransaction<'sync', void, typeof schema, ExtractTablesWithRelations> - -type GridCollection = NonNullable< - Extract['collection'] -> - -export type EnsuredPlacement = - | { - containerType: 'micronix_tube' - collectionId: number - barcode: string - position?: string - comment?: string - } - | { - containerType: 'cryovial_tube' - collectionId: number - barcode?: string - position?: string - comment?: string - } - | { - containerType: 'static_well' - collectionId: number - position?: string - comment?: string - } - | { - containerType: 'paper' - sheetId: number - sublabel?: string - comment?: string - } - -export type BulkCombinedContainerInput = ContainerWriteInput & { - unitId?: number - totalQuantity?: number - remainingQuantity?: number -} - -type EnsureOptions = { - collectionMap?: Map - userId?: number -} - -function commentOf(input: ContainerWriteInput): string | undefined { - return 'comment' in input ? input.comment : undefined -} - -function asDatabase(database: DatabaseOrTransaction): Database { - return database as unknown as Database -} - -async function createCollectionAtLocation( - database: DatabaseOrTransaction, - input: { - type: CollectionEntityType - name: string - locationId: number - barcode?: string | null - userId?: number - } -): Promise { - try { - return await createOrResolveCollection(database, input) - } catch (error: unknown) { - if (error instanceof CollectionLocationNotFoundError || error instanceof CollectionLocationNotAllowedError) { - throw new ValidationError(error.message) - } - throw error - } -} - -function gridCollectionType( - containerType: 'micronix_tube' | 'cryovial_tube' | 'static_well' -): 'micronix_plate' | 'cryovial_box' { - return containerType === 'cryovial_tube' ? 'cryovial_box' : 'micronix_plate' -} - -function gridCollectionMapKey( - collectionType: 'micronix_plate' | 'cryovial_box', - collection: GridCollection -): string { - const identifier = collection.name ?? collection.barcode ?? String(collection.id ?? '') - return `${collectionType}-${identifier}` -} - -async function lookupGridCollectionId( - database: DatabaseOrTransaction, - containerType: 'micronix_tube' | 'cryovial_tube' | 'static_well', - collection: GridCollection, - collectionMap?: Map -): Promise { - const collectionType = gridCollectionType(containerType) - const key = gridCollectionMapKey(collectionType, collection) - if (collectionMap?.has(key)) { - return collectionMap.get(key)! - } - - const db = asDatabase(database) - - if (collection.id != null) { - const existing = - collectionType === 'cryovial_box' - ? await database.select({ id: cryovialBox.id }).from(cryovialBox).where(eq(cryovialBox.id, collection.id)).get() - : await database.select({ id: micronixPlate.id }).from(micronixPlate).where(eq(micronixPlate.id, collection.id)).get() - if (!existing) { - const label = collectionType === 'cryovial_box' ? 'Cryovial box' : 'Micronix plate' - throw new ValidationError(`${label} not found: id ${collection.id}`) - } - collectionMap?.set(key, collection.id) - return collection.id - } - - if (collection.barcode) { - const byBarcode = await resolveCollectionByBarcode(collection.barcode, collectionType, db) - if (byBarcode) { - collectionMap?.set(key, byBarcode) - return byBarcode - } - } - - if (collection.name) { - const existing = await resolveCollection(collection.name, collectionType, db) - if (existing) { - collectionMap?.set(key, existing) - return existing - } - } - - return null -} - -async function ensureGridCollectionId( - database: DatabaseOrTransaction, - containerType: 'micronix_tube' | 'cryovial_tube' | 'static_well', - collection: GridCollection | undefined, - options?: EnsureOptions -): Promise { - const requiredMessage = - containerType === 'cryovial_tube' - ? 'Collection is required for cryovial tubes' - : 'Collection is required for micronix tubes' - if (!collection) { - throw new ValidationError(requiredMessage) - } - - const existing = await lookupGridCollectionId(database, containerType, collection, options?.collectionMap) - if (existing != null) { - return existing - } - - if (collection.locationId == null) { - throw new ValidationError('Collection not found and no location provided') - } - - const name = collection.name ?? collection.barcode - if (!name) { - throw new ValidationError( - containerType === 'cryovial_tube' - ? 'box_name or collection_barcode is required for cryovial_tube' - : 'plate_name or collection_barcode is required for micronix_tube' - ) - } - - const collectionType = gridCollectionType(containerType) - const id = await createCollectionAtLocation(database, { - type: collectionType, - name, - locationId: collection.locationId, - barcode: collection.barcode, - userId: options?.userId, - }) - options?.collectionMap?.set(gridCollectionMapKey(collectionType, collection), id) - return id -} - -/** - * Find an existing box or bag (a paper Sheet's parent) by id or name, or `null` when a - * named one does not exist. Throws when an id does not exist or neither id nor name is given. - */ -export async function lookupParentCollectionId( - database: DatabaseOrTransaction, - parent: { type: 'box' | 'bag'; id?: number; name?: string; locationId?: number }, - collectionMap?: Map -): Promise { - if (parent.id != null) { - if (parent.type === 'box') { - const boxRecord = await database.select({ id: box.id }).from(box).where(eq(box.id, parent.id)).get() - if (!boxRecord) throw new ValidationError(`Box not found: id ${parent.id}`) - return parent.id - } - const bagRecord = await database.select({ id: bag.id }).from(bag).where(eq(bag.id, parent.id)).get() - if (!bagRecord) throw new ValidationError(`Bag not found: id ${parent.id}`) - return parent.id - } - - if (!parent.name) { - throw new ValidationError('box_name or bag_name is required for paper') - } - - const key = `${parent.type}-${parent.name}` - if (collectionMap?.has(key)) { - return collectionMap.get(key)! - } - - if (parent.type === 'box') { - const byName = await database.select({ id: box.id }).from(box).where(eq(box.name, parent.name)).get() - if (byName) { - collectionMap?.set(key, byName.id) - return byName.id - } - } else { - const byName = await database.select({ id: bag.id }).from(bag).where(eq(bag.name, parent.name)).get() - if (byName) { - collectionMap?.set(key, byName.id) - return byName.id - } - } - - return null -} - -async function ensureParentCollectionId( - database: DatabaseOrTransaction, - parent: { type: 'box' | 'bag'; id?: number; name?: string; locationId?: number }, - options?: EnsureOptions -): Promise { - const existing = await lookupParentCollectionId(database, parent, options?.collectionMap) - if (existing != null) { - return existing - } - - if (parent.locationId == null || !parent.name) { - throw new ValidationError(`Collection (${parent.type}) not found: ${parent.name ?? ''}`) - } - - const id = await createCollectionAtLocation(database, { - type: parent.type, - name: parent.name, - locationId: parent.locationId, - userId: options?.userId, - }) - options?.collectionMap?.set(`${parent.type}-${parent.name}`, id) - return id -} - -async function ensureSheetId( - database: DatabaseOrTransaction, - collection: NonNullable['collection']> | undefined, - options?: EnsureOptions -): Promise { - if (!collection) { - throw new ValidationError('Collection is required for paper containers') - } - - if (collection.id != null) { - const sheetRecord = await database - .select({ id: sheet.id }) - .from(sheet) - .where(eq(sheet.id, collection.id)) - .get() - if (!sheetRecord) { - throw new ValidationError(`Sheet not found: id ${collection.id}`) - } - return collection.id - } - - if (!collection.name) { - throw new ValidationError('sheet_name is required for paper') - } - if (!collection.parent) { - throw new ValidationError('box_name or bag_name is required for paper') - } - - const parentId = await ensureParentCollectionId(database, collection.parent, options) - const parentType = collection.parent.type - - const existing = - parentType === 'box' - ? await database - .select({ id: sheet.id }) - .from(sheet) - .where(and(eq(sheet.name, collection.name), eq(sheet.boxId, parentId))) - .get() - : await database - .select({ id: sheet.id }) - .from(sheet) - .where(and(eq(sheet.name, collection.name), eq(sheet.bagId, parentId))) - .get() - - if (existing) { - return existing.id - } - - const now = utcNow() - const inserted = await database - .insert(sheet) - .values({ - name: collection.name, - boxId: parentType === 'box' ? parentId : null, - bagId: parentType === 'bag' ? parentId : null, - created: now, - lastUpdated: now, - createdBy: options?.userId, - updatedBy: options?.userId, - }) - .returning() - const row = insertedRow(inserted, 'sheet') - return row.id -} - -/** Find-or-create Collection placement and return ids. Does not insert the Container. */ -export async function ensureContainerPlacement( - database: DatabaseOrTransaction, - input: ContainerWriteInput, - options?: EnsureOptions -): Promise { - const comment = commentOf(input) - - switch (input.containerType) { - case 'micronix_tube': { - const collectionId = await ensureGridCollectionId(database, 'micronix_tube', input.collection, options) - return { - containerType: 'micronix_tube', - collectionId, - barcode: input.barcode, - position: input.collection?.position, - comment, - } - } - case 'cryovial_tube': { - const collectionId = await ensureGridCollectionId(database, 'cryovial_tube', input.collection, options) - return { - containerType: 'cryovial_tube', - collectionId, - barcode: input.barcode, - position: input.collection?.position, - comment, - } - } - case 'static_well': { - const collectionId = await ensureGridCollectionId(database, 'static_well', input.collection, options) - return { - containerType: 'static_well', - collectionId, - position: input.collection?.position, - comment, - } - } - case 'paper': { - const sheetId = await ensureSheetId(database, input.collection, options) - return { - containerType: 'paper', - sheetId, - sublabel: input.sublabel, - comment, - } - } - default: { - const _exhaustive: never = input - void _exhaustive - throw new ValidationError('Unsupported container type') - } - } -} - -/** Lookup-only: collection exists or can be created (locationId present). Does not insert. */ -export async function assertWriteInputPlacementResolvable( - database: DatabaseOrTransaction, - input: ContainerWriteInput, - collectionMap?: Map -): Promise { - switch (input.containerType) { - case 'micronix_tube': - case 'cryovial_tube': - case 'static_well': { - if (!input.collection) { - throw new ValidationError( - input.containerType === 'cryovial_tube' - ? 'Collection is required for cryovial tubes' - : 'Collection is required for micronix tubes' - ) - } - const existing = await lookupGridCollectionId(database, input.containerType, input.collection, collectionMap) - if (existing == null && input.collection.locationId == null) { - throw new ValidationError('Collection not found and no location provided') - } - return - } - case 'paper': { - const collection = input.collection - if (!collection) { - throw new ValidationError('Collection is required for paper containers') - } - if (collection.id != null) { - const sheetRecord = await database - .select({ id: sheet.id }) - .from(sheet) - .where(eq(sheet.id, collection.id)) - .get() - if (!sheetRecord) { - throw new ValidationError(`Sheet not found: id ${collection.id}`) - } - return - } - if (!collection.name) { - throw new ValidationError('sheet_name is required for paper') - } - if (!collection.parent) { - throw new ValidationError('box_name or bag_name is required for paper') - } - const parentId = await lookupParentCollectionId(database, collection.parent, collectionMap) - if (parentId == null && collection.parent.locationId == null) { - throw new ValidationError( - `Collection (${collection.parent.type}) not found: ${collection.parent.name ?? ''}` - ) - } - return - } - default: { - const _exhaustive: never = input - void _exhaustive - throw new ValidationError('Unsupported container type') - } - } -} - -export function toContainerWriteInput(container: BulkCombinedContainerInput): ContainerWriteInput { - const { unitId: _unitId, totalQuantity: _totalQuantity, remainingQuantity: _remainingQuantity, ...write } = - container - return write -} - -export function collectionKeyFromWriteInput(input: ContainerWriteInput): string { - switch (input.containerType) { - case 'paper': { - const parent = input.collection?.parent - if (parent?.type === 'bag') { - const name = parent.name ?? String(parent.id ?? '') - return `bag-${name}` - } - const boxName = parent?.name ?? input.collection?.name ?? String(input.collection?.id ?? '') - return `box-${boxName}` - } - case 'cryovial_tube': { - const c = input.collection - const identifier = c?.name ?? c?.barcode ?? String(c?.id ?? '') - return `cryovial_box-${identifier}` - } - case 'micronix_tube': - case 'static_well': { - const c = input.collection - const identifier = c?.name ?? c?.barcode ?? String(c?.id ?? '') - return `micronix_plate-${identifier}` - } - default: - return 'unknown' - } -} - -export async function lookupWriteInputCollectionId( - database: DatabaseOrTransaction, - input: ContainerWriteInput, - collectionMap?: Map -): Promise<{ collectionKey: string; collectionId: number | null }> { - const collectionKey = collectionKeyFromWriteInput(input) - if (collectionMap?.has(collectionKey)) { - return { collectionKey, collectionId: collectionMap.get(collectionKey)! } - } - - const dbForResolve = asDatabase(database) - - if (input.containerType === 'paper') { - return { collectionKey, collectionId: null } - } - - const collection = input.collection - if (!collection) { - return { collectionKey, collectionId: null } - } - - if (collection.id != null) { - return { collectionKey, collectionId: collection.id } - } - - if (input.containerType === 'cryovial_tube') { - if (collection.barcode) { - const id = await resolveCollectionByBarcode(collection.barcode, 'cryovial_box', dbForResolve) - return { collectionKey, collectionId: id } - } - if (collection.name) { - const id = await resolveCollection(collection.name, 'cryovial_box', dbForResolve) - return { collectionKey, collectionId: id } - } - } else { - if (collection.barcode) { - const id = await resolveCollectionByBarcode(collection.barcode, 'micronix_plate', dbForResolve) - return { collectionKey, collectionId: id } - } - if (collection.name) { - const id = await resolveCollection(collection.name, 'micronix_plate', dbForResolve) - return { collectionKey, collectionId: id } - } - } - - return { collectionKey, collectionId: null } -} - -export function placementContainerFromWriteInput(input: ContainerWriteInput) { - if (input.containerType === 'paper') { - return { - containerType: 'paper' as const, - collectionName: input.collection?.parent?.name, - sublabel: input.sublabel, - sheetName: input.collection?.name, - } - } - return { - containerType: input.containerType, - collectionName: input.collection?.name, - collectionBarcode: input.collection?.barcode, - barcode: 'barcode' in input ? input.barcode : undefined, - position: input.collection?.position, - } -} diff --git a/packages/api/src/lib/controls/__tests__/batch-with-specimens.test.ts b/packages/api/src/lib/controls/__tests__/batch-with-specimens.test.ts index d0265e3c..dcb38332 100644 --- a/packages/api/src/lib/controls/__tests__/batch-with-specimens.test.ts +++ b/packages/api/src/lib/controls/__tests__/batch-with-specimens.test.ts @@ -12,7 +12,7 @@ import { import { setContainerDefaults, clearSettingsCache } from '../../settings' import { clearDefaultsCache } from '../../defaults' import { createBatchWithSpecimens, createBatchesWithSpecimens } from '../batch-with-specimens' -import { specimenTypeContainerType, containerTypeUnit, storageContainer, box, controlBatch } from '../../../db/schema' +import { specimenTypeContainerType, containerTypeUnit, storageContainer, box, controlBatch, micronixPlate } from '../../../db/schema' import { eq } from 'drizzle-orm' import type { Database } from '../../../db/client' import { utcNow } from '../../datetime' @@ -127,6 +127,33 @@ describe('batch-with-specimens', () => { expect(result.specimens[0].containerIds).toHaveLength(1) }) + it('rejects one new plate given two locations, though each container is registered on its own', async () => { + const unit = await createTestUnit(testDb, { symbol: 'uL', name: 'microliter', category: 'volume' }) + await setContainerDefaults(testDb, { micronix_tube: { totalQuantity: 1, remainingQuantity: 1, defaultUnitSymbol: 'uL' } }) + await testDb.insert(containerTypeUnit).values({ containerType: 'micronix_tube', unitId: unit.id }) + const definition = await createTestControlDefinition(testDb, { name: 'Def1' }) + const specimenType = await createTestSpecimenType(testDb, { name: 'DNA' }) + await testDb.insert(specimenTypeContainerType).values({ specimenTypeId: specimenType.id, containerType: 'micronix_tube', created: utcNow() }) + const storageType = await createTestStorageType(testDb, { name: 'Freezer' }) + const shelf = await createTestLocation(testDb, { name: 'Shelf', storageTypeId: String(storageType.id), canContainCollections: true }) + const rack = await createTestLocation(testDb, { name: 'Rack', storageTypeId: String(storageType.id), canContainCollections: true }) + const onNewPlate = (barcode: string, locationId: number, position: string) => ({ + containerType: 'micronix_tube' as const, + barcode, + collection: { type: 'micronix_plate' as const, name: 'NEW-PLATE', locationId, position }, + }) + + await expect( + createBatchWithSpecimens(testDb, { + batch: { controlDefinitionId: definition.id, name: 'Batch1' }, + specimens: [{ specimenTypeName: 'DNA', containers: [onNewPlate('MT001', shelf.id, 'A01'), onNewPlate('MT002', rack.id, 'A02')] }], + }) + ).rejects.toThrow("Micronix plate 'NEW-PLATE' is stored at a different location from the one given.") + + expect(await testDb.select().from(controlBatch)).toEqual([]) + expect(await testDb.select().from(micronixPlate)).toEqual([]) + }) + it('throws when paper container is missing sheet name', async () => { const unit = await createTestUnit(testDb, { symbol: 'uL', name: 'microliter', category: 'volume' }) await setContainerDefaults(testDb, { diff --git a/packages/api/src/lib/controls/batch-with-specimens.ts b/packages/api/src/lib/controls/batch-with-specimens.ts index 2f41e999..256c233c 100644 --- a/packages/api/src/lib/controls/batch-with-specimens.ts +++ b/packages/api/src/lib/controls/batch-with-specimens.ts @@ -203,7 +203,6 @@ export async function createBatchWithSpecimens( specimens: CreatedSpecimen[] createdCollections: CreatedCollection[] }> { - const collectionMap = new Map() const definition = await database .select() @@ -260,7 +259,6 @@ export async function createBatchWithSpecimens( const containerIds: number[] = [] for (const prepared of preparedContainers) { const containerResult = await createContainerForSpecimen(specimenId, prepared.writeInput, tx, { - collectionMap, quantity: prepared.quantity, }) if (!containerResult.success || containerResult.containerId == null) { @@ -314,7 +312,6 @@ export async function addSpecimensToBatch( throw new Error(`Batch not found: ${batchId}`) } - const collectionMap = new Map() const preparedSpecimens = await prepareSpecimensForBatch(database, data.specimens) return withWriteTransaction(database, async (tx) => { @@ -348,7 +345,6 @@ export async function addSpecimensToBatch( const containerIds: number[] = [] for (const prepared of preparedContainers) { const containerResult = await createContainerForSpecimen(specimenId, prepared.writeInput, tx, { - collectionMap, quantity: prepared.quantity, }) if (!containerResult.success || containerResult.containerId == null) { diff --git a/packages/api/src/lib/controls/strain-composition.ts b/packages/api/src/lib/controls/strain-composition.ts index d627dd3f..da931c9f 100644 --- a/packages/api/src/lib/controls/strain-composition.ts +++ b/packages/api/src/lib/controls/strain-composition.ts @@ -176,7 +176,7 @@ export async function resolveStrainComposition( /** 409 for a blood definition whose density and strain mix another definition already has. */ export class DuplicateControlCompositionError extends ConflictError { constructor(existingDefinition: typeof controlDefinition.$inferSelect) { - super('A control definition with this combination of density and strains already exists', { + super('A control definition with this combination of density and strains already exists', undefined, { existingDefinition, }) this.name = 'DuplicateControlCompositionError' diff --git a/packages/api/src/lib/derivations-csv.ts b/packages/api/src/lib/derivations-csv.ts index 65323316..119a86e1 100644 --- a/packages/api/src/lib/derivations-csv.ts +++ b/packages/api/src/lib/derivations-csv.ts @@ -1,5 +1,4 @@ import type { Database } from '../db/client' -import type { DatabaseOrTransaction } from './db-types' import { containerDerivation, cryovialBox, @@ -26,7 +25,7 @@ import { withWriteTransaction } from '../db/write-transaction' import { resolveParentContainerId } from './derivations/parent-container-resolver' import { normalizePosition } from './normalize-position' import { checkGridPositionOccupancy } from './container-occupancy' -import { lookupParentCollectionId } from './container-write-placement' +import { CollectionPlanner } from './specimen-registration/collections' /** * Map technical DB/constraint errors to messages that are clear for non-technical users. @@ -306,7 +305,7 @@ export function parseCsv(text: string): DerivationCsvRow[] { } async function resolveCollectionId( - database: DatabaseOrTransaction, + database: Database, containerType: DerivationCsvRow['container_type'], { name: collectionName, barcode: collectionBarcode, paperParentType = 'box' }: ChildCollectionRef, ): Promise<{ id?: number; status: 'existing' | 'will_be_created' }> { @@ -358,11 +357,13 @@ async function resolveCollectionId( // Paper goes on a sheet in the named box or bag; the sheet is created if missing, // but the box or bag itself must already exist. if (!collectionName) return { status: 'will_be_created' } - const parentId = await lookupParentCollectionId(database, { + const resolution = await new CollectionPlanner(database).resolveParent({ type: paperParentType, name: collectionName.trim(), }) - return parentId != null ? { id: parentId, status: 'existing' } : { status: 'will_be_created' } + return 'target' in resolution && 'existingId' in resolution.target + ? { id: resolution.target.existingId, status: 'existing' } + : { status: 'will_be_created' } } return { status: 'will_be_created' } diff --git a/packages/api/src/lib/derivations.ts b/packages/api/src/lib/derivations.ts index e04e051e..7855e70f 100644 --- a/packages/api/src/lib/derivations.ts +++ b/packages/api/src/lib/derivations.ts @@ -235,12 +235,10 @@ async function createDerivationInTx( throw new Error(containerTypeValidation.error || 'Container type not allowed for this specimen type') } - const collectionMap = new Map() const unitId = await resolveUnitIdForChild(database, input.container.containerType, input.unitSymbol) const quantity = input.quantity ?? 1.0 const containerResult = await createContainerForSpecimen(derivedSpecimenId, input.container, database, { - collectionMap, quantity: pickContainerQuantity({ unitId, totalQuantity: quantity, remainingQuantity: quantity }), }) diff --git a/packages/api/src/lib/error-handler.ts b/packages/api/src/lib/error-handler.ts index f6a47049..dd4282b1 100644 --- a/packages/api/src/lib/error-handler.ts +++ b/packages/api/src/lib/error-handler.ts @@ -125,7 +125,8 @@ export function handleRouteError(error: unknown, c: Context): Response { return c.json({ ...error.fields, error: error.message, - errorCode: 'CONFLICT' + errorCode: 'CONFLICT', + ...(error.details !== undefined && { details: error.details }), }, 409) } @@ -204,8 +205,12 @@ export class NotFoundError extends Error { } export class ConflictError extends Error { - /** `fields` are added to the 409 body next to `error` (e.g. the record that conflicts). */ - constructor(message: string, public readonly fields: Record = {}) { + /** `details` goes in the 409 body as `details`; `fields` are added next to `error` (e.g. the record that conflicts). */ + constructor( + message: string, + public readonly details?: unknown, + public readonly fields: Record = {} + ) { super(message) this.name = 'ConflictError' } diff --git a/packages/api/src/lib/registration-orchestrator.ts b/packages/api/src/lib/registration-orchestrator.ts index 50f2b54b..90b36358 100644 --- a/packages/api/src/lib/registration-orchestrator.ts +++ b/packages/api/src/lib/registration-orchestrator.ts @@ -1,154 +1,156 @@ /** - * Registration orchestrator — validate/create entry points for bulk specimen registration. - * Prepare logic lives in registration-prepare.ts. + * POST /specimens/bulk and its validate route. Each row's Source is resolved here; Specimens and + * Containers go through Specimen registration (ADR 0010), and validation is the same + * registration, rolled back. */ import type { Database } from '../db/client' -import { specimen } from '../db/schema' -import { withWriteTransaction } from '../db/write-transaction' -import { createContainerForSpecimen, pickContainerQuantity } from './container-creation' -import { findExistingStudySpecimen, findExistingControlSpecimen } from './specimen-helpers' -import { utcNow } from './datetime' +import type { specimen } from '../db/schema' +import { ConflictError } from './error-handler' +import { validateSpecimenData } from './validation' import { - toContainerWriteInput, - type BulkCombinedContainerInput, -} from './container-write-placement' -import { - bulkSpecimenPayloadKey, - prepareRegistrationBatchForSpecimens, - type BulkSpecimenValidateRow, - type BulkSpecimenValidateError, - type BulkSpecimenValidateResult, - type BulkSpecimenCreateResult, -} from './registration-prepare' + checkSpecimenRegistration, + registerSpecimens, + SpecimenRegistrationError, + type RegistrationContainer, + type RegistrationEntry, + type RegistrationError, +} from './specimen-registration' + +export type BulkSpecimenContainerInput = Omit & { + containerType?: RegistrationContainer['containerType'] +} + +export type BulkSpecimenValidateRow = { + sourceType: 'subject' | 'control' + sourceId?: number + studyShortCode?: string + subjectName?: string + specimenTypeName: string + collectionDate?: string + container?: BulkSpecimenContainerInput +} + +export type BulkSpecimenValidateError = { + index: number + message: string +} + +export type BulkSpecimenValidateResult = { + valid: boolean + errors: BulkSpecimenValidateError[] +} + +export type BulkSpecimenCreateResult = { + specimens: Array + created: number + containersCreated: number +} + +/** A row's container, if it names a container type; rows without one register only the Specimen. */ +function rowContainers(row: BulkSpecimenValidateRow): RegistrationContainer[] { + const container = row.container + if (!container?.containerType) return [] + return [container as RegistrationContainer] +} + +/** Resolve each row's Source and specimen type into a registration entry, or a row error. */ +async function toRegistrationEntries( + database: Database, + rows: BulkSpecimenValidateRow[], +): Promise<{ entries: RegistrationEntry[]; rowOfEntry: number[]; errors: BulkSpecimenValidateError[] }> { + const entries: RegistrationEntry[] = [] + const rowOfEntry: number[] = [] + const errors: BulkSpecimenValidateError[] = [] + + for (let index = 0; index < rows.length; index++) { + const row = rows[index] + const validation = await validateSpecimenData( + { + sourceType: row.sourceType, + sourceId: row.sourceId, + studyShortCode: row.studyShortCode, + subjectName: row.subjectName, + specimenTypeName: row.specimenTypeName, + collectionDate: row.collectionDate, + }, + database, + ) + const resolved = validation.resolved + if (!validation.valid || !resolved) { + errors.push({ index, message: validation.error ?? 'Invalid specimen data' }) + continue + } + const source = + resolved.studySubjectId != null + ? { subjectId: resolved.studySubjectId } + : resolved.controlBatchId != null + ? { controlBatchId: resolved.controlBatchId } + : null + if (!source) { + errors.push({ index, message: 'Invalid specimen data' }) + continue + } + entries.push({ + source, + specimenTypeId: resolved.specimenTypeId, + collectionDate: resolved.collectionDate, + containers: rowContainers(row), + }) + rowOfEntry.push(index) + } -export type { - BulkSpecimenContainerInput, - BulkSpecimenValidateRow, - BulkSpecimenValidateError, - BulkSpecimenValidateResult, - BulkSpecimenCreateResult, - PreparedBulkSpecimenRow, - SpecimenContainerRegistrationResult, -} from './registration-prepare' + return { entries, rowOfEntry, errors } +} -export { - bulkCombinedCollectionMessages, - bulkSpecimenPayloadKey, - prepareRegistrationBatchForSpecimens, - prepareCombinedSubjectContainerBatch, - validateSpecimenContainerRegistration, -} from './registration-prepare' -import { insertedRow } from './inserted-row' +function rowErrors(errors: RegistrationError[], rowOfEntry: number[]): BulkSpecimenValidateError[] { + return errors.map((error) => ({ index: rowOfEntry[error.entryIndex], message: error.message })) +} /** - * Validate bulk specimen rows without creating records. - * Used by POST /specimens/bulk/validate. + * Validate bulk specimen rows without creating records: the registration POST /specimens/bulk + * would run, rolled back. Used by POST /specimens/bulk/validate. */ export async function validateBulkSpecimenRows( database: Database, - rows: BulkSpecimenValidateRow[] + rows: BulkSpecimenValidateRow[], ): Promise { - const result = await prepareRegistrationBatchForSpecimens(database, rows) - return { - valid: result.valid, - errors: result.valid ? [] : result.errors, - } + const { entries, rowOfEntry, errors } = await toRegistrationEntries(database, rows) + errors.push(...rowErrors(await checkSpecimenRegistration(database, entries), rowOfEntry)) + errors.sort((a, b) => a.index - b.index) + return { valid: errors.length === 0, errors } } /** - * Create bulk specimen rows after the same validation as validateBulkSpecimenRows. - * Used by POST /specimens/bulk. + * Register bulk specimen rows, all or nothing. Used by POST /specimens/bulk. + * Throws ConflictError when a write hits a database constraint. */ export async function createBulkSpecimenRows( database: Database, rows: BulkSpecimenValidateRow[], - userId?: number + userId?: number, ): Promise< | { success: true; result: BulkSpecimenCreateResult } | { success: false; errors: BulkSpecimenValidateError[] } > { - const prep = await prepareRegistrationBatchForSpecimens(database, rows) - if (!prep.valid) { - return { success: false, errors: prep.errors } + const { entries, rowOfEntry, errors } = await toRegistrationEntries(database, rows) + if (errors.length > 0) { + return { success: false, errors } } - const { prepared } = prep - const now = utcNow() - - const orderedPayloadKeys = prepared.map(({ row, resolved }) => bulkSpecimenPayloadKey(row, resolved)) - const payloadKeyToFirstIndex = new Map() - for (let idx = 0; idx < orderedPayloadKeys.length; idx++) { - const key = orderedPayloadKeys[idx] - if (!payloadKeyToFirstIndex.has(key)) payloadKeyToFirstIndex.set(key, idx) - } - const uniqueSpecimenOrder: number[] = [] - const indexToUniqueIndex: number[] = [] - for (let idx = 0; idx < orderedPayloadKeys.length; idx++) { - const first = payloadKeyToFirstIndex.get(orderedPayloadKeys[idx]) ?? idx - if (first === idx) uniqueSpecimenOrder.push(idx) - indexToUniqueIndex.push(uniqueSpecimenOrder.indexOf(first)) - } - - // Specimens and their containers commit together: a container failure must not leave bare specimens. - return withWriteTransaction(database, async (dbTx) => { - const specimenRecordsByUniqueIndex: Array = [] - let newCount = 0 - for (const uniqueIdx of uniqueSpecimenOrder) { - const { row, resolved } = prepared[uniqueIdx] - const studySubjectId = resolved.studySubjectId - const existing = - row.sourceType === 'subject' && studySubjectId != null - ? findExistingStudySpecimen(dbTx, studySubjectId, resolved.specimenTypeId, row.collectionDate) - : row.sourceType === 'control' && resolved.controlBatchId != null - ? findExistingControlSpecimen(dbTx, resolved.controlBatchId, resolved.specimenTypeId, row.collectionDate) - : null - if (existing) { - specimenRecordsByUniqueIndex.push(existing) - } else { - const insertResult = dbTx - .insert(specimen) - .values({ - studySubjectId: resolved.studySubjectId, - controlBatchId: resolved.controlBatchId, - specimenTypeId: resolved.specimenTypeId, - collectionDate: row.collectionDate ?? null, - created: now, - lastUpdated: now, - createdBy: userId, - updatedBy: userId, - }) - .returning() - .all() - specimenRecordsByUniqueIndex.push(insertedRow(insertResult, 'specimen')) - newCount += 1 - } - } - - const specimensOut = indexToUniqueIndex.map((ui) => specimenRecordsByUniqueIndex[ui]) - let containersCount = 0 - for (let i = 0; i < prepared.length; i++) { - const { row, index } = prepared[i] - const specimenRecord = specimensOut[i] - if (!row.container?.containerType) continue - const container = row.container as BulkCombinedContainerInput - const writeInput = toContainerWriteInput(container) - const containerResult = await createContainerForSpecimen(specimenRecord.id, writeInput, dbTx, { - userId, - quantity: pickContainerQuantity(container), - }) - if (!containerResult.success || !containerResult.containerId) { - throw new Error(containerResult.error ?? `Row ${index}: failed to create container`) - } - containersCount += 1 - } - + try { + const result = await registerSpecimens(database, entries, { userId }) return { - success: true as const, + success: true, result: { - specimens: specimensOut, - created: newCount, - containersCreated: containersCount, + specimens: result.entries.map((entry) => entry.specimen), + created: result.entries.filter((entry) => !entry.reused).length, + containersCreated: result.entries.reduce((sum, entry) => sum + entry.containerIds.length, 0), }, } - }) + } catch (error) { + if (!(error instanceof SpecimenRegistrationError)) throw error + const mapped = rowErrors(error.errors, rowOfEntry) + if (error.status === 409) throw new ConflictError(error.message, { errors: mapped }) + return { success: false, errors: mapped } + } } diff --git a/packages/api/src/lib/registration-prepare.ts b/packages/api/src/lib/registration-prepare.ts deleted file mode 100644 index 4f9a1a73..00000000 --- a/packages/api/src/lib/registration-prepare.ts +++ /dev/null @@ -1,283 +0,0 @@ -/** - * Registration batch prepare — provenance resolution and container placement checks - * shared by bulk specimen validate/create and combined import validate/import. - */ -import type { Database } from '../db/client' -import { specimen } from '../db/schema' -import { validateSpecimenData, validateContainerTypeForSpecimenType } from './validation' -import { validateContainerWriteFields } from './container-creation' -import { - buildContainerPlacementCheckRow, - collectContainerPlacementErrors, - type ContainerPlacementCheckRow, -} from './container-placement-validation' -import { - assertLocationCanContainCollections, - CollectionLocationNotAllowedError, - CollectionLocationNotFoundError, - LOCATION_CANNOT_CONTAIN_COLLECTIONS, -} from './collections/collection-lifecycle' -import { - assertWriteInputPlacementResolvable, - lookupWriteInputCollectionId, - placementContainerFromWriteInput, - toContainerWriteInput, - type BulkCombinedContainerInput, -} from './container-write-placement' - -export type BulkSpecimenContainerInput = Omit & { - containerType?: BulkCombinedContainerInput['containerType'] -} - -export type BulkSpecimenValidateRow = { - sourceType: 'subject' | 'control' - sourceId?: number - studyShortCode?: string - subjectName?: string - specimenTypeName: string - collectionDate?: string - container?: BulkSpecimenContainerInput -} - -export type BulkSpecimenValidateError = { - index: number - message: string -} - -export type BulkSpecimenValidateResult = { - valid: boolean - errors: BulkSpecimenValidateError[] -} - -export type BulkSpecimenCreateResult = { - specimens: Array - created: number - containersCreated: number -} - -type ResolvedSpecimen = NonNullable>['resolved']> - -export type PreparedBulkSpecimenRow = { - index: number - row: BulkSpecimenValidateRow - resolved: ResolvedSpecimen -} - -type CollectionResolutionMessages = { - collectionNotFound: (identifier: string) => string - boxNotFound: (name: string) => string -} - -export const bulkCombinedCollectionMessages: CollectionResolutionMessages = { - collectionNotFound: (identifier) => - `Collection '${identifier}' not found. Provide collection.locationId to create it.`, - boxNotFound: (name) => `Box '${name}' not found. Provide collection.parent.locationId to create it.`, -} - -export type SpecimenContainerRegistrationResult = - | { - valid: true - collectionId: number | null - collectionKey: string - placementRow: ContainerPlacementCheckRow - } - | { valid: false; error: string } - -export async function validateSpecimenContainerRegistration( - database: Database, - specimenTypeId: number, - container: BulkCombinedContainerInput, - _options?: { messages?: CollectionResolutionMessages } -): Promise { - const writeInput = toContainerWriteInput(container) - const containerType = writeInput.containerType - const containerTypeValidation = await validateContainerTypeForSpecimenType( - database, - specimenTypeId, - containerType - ) - if (!containerTypeValidation.valid) { - return { - valid: false, - error: containerTypeValidation.error || 'Invalid container type for specimen type', - } - } - - const containerValidation = validateContainerWriteFields(writeInput) - if (!containerValidation.valid) { - return { valid: false, error: containerValidation.error || 'Invalid container data' } - } - - try { - await assertWriteInputPlacementResolvable(database, writeInput) - } catch (error: unknown) { - const message = error instanceof Error ? error.message : 'Invalid container placement' - return { valid: false, error: message } - } - - const collection = writeInput.collection - const locationId = - (collection && 'locationId' in collection ? collection.locationId : undefined) ?? - (collection && 'parent' in collection ? collection.parent?.locationId : undefined) - if (locationId != null) { - try { - assertLocationCanContainCollections(database, locationId) - } catch (error) { - if (error instanceof CollectionLocationNotFoundError) { - return { valid: false, error: 'Location not found' } - } - if (error instanceof CollectionLocationNotAllowedError) { - return { valid: false, error: LOCATION_CANNOT_CONTAIN_COLLECTIONS } - } - throw error - } - } - - const collectionResolution = await lookupWriteInputCollectionId(database, writeInput) - return { - valid: true, - collectionId: collectionResolution.collectionId, - collectionKey: collectionResolution.collectionKey, - placementRow: buildContainerPlacementCheckRow( - placementContainerFromWriteInput(writeInput), - collectionResolution.collectionId, - collectionResolution.collectionKey - ), - } -} - -/** Stable dedupe key for bulk specimen rows (subject/control + type + date). */ -export function bulkSpecimenPayloadKey(row: BulkSpecimenValidateRow, resolved: ResolvedSpecimen): string { - return row.sourceType === 'subject' - ? `s:${String(row.studyShortCode ?? '')}:${String(row.subjectName ?? '')}:${String(row.specimenTypeName)}:${String(row.collectionDate ?? '')}` - : `c:${String(resolved.controlBatchId ?? '')}:${String(resolved.specimenTypeId)}:${String(row.collectionDate ?? '')}` -} - -export type CombinedSubjectSpecimenRow = { - specimenTypeId: number - collectionDate?: string - container?: BulkCombinedContainerInput -} - -export type CombinedContainerPrepareResult = { - placementRows: ContainerPlacementCheckRow[] - collectionMap: Map -} - -/** - * Validate containers for combined-import subject specimens with resolved specimen types. - * Builds placement rows and a collection key map for downstream placement checks. - */ -export async function prepareCombinedSubjectContainerBatch( - database: Database, - specimens: CombinedSubjectSpecimenRow[], - options?: { messages?: CollectionResolutionMessages } -): Promise< - | { valid: true; result: CombinedContainerPrepareResult } - | { valid: false; specimenIndex: number; message: string } -> { - const messages = options?.messages ?? bulkCombinedCollectionMessages - const placementRows: ContainerPlacementCheckRow[] = [] - const collectionMap = new Map() - - for (let specimenIndex = 0; specimenIndex < specimens.length; specimenIndex++) { - const spec = specimens[specimenIndex] - if (!spec.container?.containerType) { - continue - } - - const containerRegistration = await validateSpecimenContainerRegistration( - database, - spec.specimenTypeId, - spec.container, - { messages } - ) - if (!containerRegistration.valid) { - return { valid: false, specimenIndex, message: containerRegistration.error } - } - - if (containerRegistration.collectionId !== null) { - collectionMap.set(containerRegistration.collectionKey, containerRegistration.collectionId) - } - - placementRows.push(containerRegistration.placementRow) - } - - return { valid: true, result: { placementRows, collectionMap } } -} - -/** - * Resolve provenance and container placement for flat bulk specimen rows. - */ -export async function prepareRegistrationBatchForSpecimens( - database: Database, - rows: BulkSpecimenValidateRow[] -): Promise< - | { valid: true; prepared: PreparedBulkSpecimenRow[] } - | { valid: false; errors: BulkSpecimenValidateError[] } -> { - const errors: BulkSpecimenValidateError[] = [] - const prepared: PreparedBulkSpecimenRow[] = [] - const placementRows: ContainerPlacementCheckRow[] = [] - const placementIndexToRowIndex: number[] = [] - - for (let index = 0; index < rows.length; index++) { - const row = rows[index] - try { - const validation = await validateSpecimenData( - { - sourceType: row.sourceType, - sourceId: row.sourceId, - studyShortCode: row.studyShortCode, - subjectName: row.subjectName, - specimenTypeName: row.specimenTypeName, - collectionDate: row.collectionDate, - }, - database - ) - if (!validation.valid || !validation.resolved) { - errors.push({ index, message: validation.error || 'Invalid specimen data' }) - continue - } - - // Carry the normalized date so dedup keys and inserts use the same form. - prepared.push({ index, row: { ...row, collectionDate: validation.resolved.collectionDate }, resolved: validation.resolved }) - - if (!row.container?.containerType) { - continue - } - - const containerRegistration = await validateSpecimenContainerRegistration( - database, - validation.resolved.specimenTypeId, - row.container as BulkCombinedContainerInput - ) - if (!containerRegistration.valid) { - errors.push({ index, message: containerRegistration.error }) - continue - } - - placementRows.push(containerRegistration.placementRow) - placementIndexToRowIndex.push(index) - } catch (error: unknown) { - errors.push({ - index, - message: error instanceof Error ? error.message : 'Validation failed', - }) - } - } - - const placementErrors = await collectContainerPlacementErrors(database, placementRows) - for (const placementError of placementErrors) { - const originalIndex = placementIndexToRowIndex.at(placementError.rowIndex) - if (originalIndex !== undefined) { - errors.push({ index: originalIndex, message: placementError.message }) - } - } - - if (errors.length > 0) { - return { valid: false, errors } - } - - return { valid: true, prepared } -} diff --git a/packages/api/src/lib/specimen-registration/__tests__/specimen-registration.test.ts b/packages/api/src/lib/specimen-registration/__tests__/specimen-registration.test.ts new file mode 100644 index 00000000..34100513 --- /dev/null +++ b/packages/api/src/lib/specimen-registration/__tests__/specimen-registration.test.ts @@ -0,0 +1,383 @@ +import { describe, it, expect, beforeEach, afterEach } from 'vitest' +import { eq } from 'drizzle-orm' +import { setupTestDatabase, cleanupTestDatabase } from '../../../__tests__/helpers/db-setup' +import { + createTestLocation, + createTestMicronixPlate, + createTestSpecimenType, + createTestStorageType, + createTestStudy, + createTestStudySubject, + createTestUnit, + createTestSpecimenTypeContainerType, + createTestContainerTypeUnit, +} from '../../../__tests__/helpers/factories' +import type { Database } from '../../../db/client' +import { + micronixPlate, + micronixTube, + paper, + sheet, + specimen, + storageContainer, +} from '../../../db/schema' +import { setContainerDefaults } from '../../settings' +import { + checkSpecimenRegistration, + registerSpecimens, + SpecimenRegistrationError, + type RegistrationContainer, + type RegistrationEntry, +} from '..' +import { planRegistration } from '../plan' +import { writeRegistration } from '../write' + +const CONTAINER_TYPES = ['micronix_tube', 'cryovial_tube', 'paper', 'static_well'] as const + +describe('Specimen registration', () => { + let db: Database + let sqlite: Awaited>['sqlite'] + let subjectId: number + let specimenTypeId: number + let locationId: number + let unitId: number + let plate: { id: number; name: string } + + beforeEach(async () => { + const setup = await setupTestDatabase() + db = setup.db + sqlite = setup.sqlite + + const study = await createTestStudy(db, { title: 'Study 1', shortCode: 'ST1' }) + subjectId = (await createTestStudySubject(db, { studyId: study.id, name: 'S1' })).id + specimenTypeId = (await createTestSpecimenType(db, { name: 'DNA' })).id + const storageType = await createTestStorageType(db, { name: 'Freezer' }) + locationId = (await createTestLocation(db, { name: 'Shelf', storageTypeId: String(storageType.id), canContainCollections: true })).id + unitId = (await createTestUnit(db, { symbol: 'uL', name: 'microliter', category: 'volume' })).id + for (const containerType of CONTAINER_TYPES) { + await createTestSpecimenTypeContainerType(db, { specimenTypeId, containerType }) + await createTestContainerTypeUnit(db, { containerType, unitId }) + } + await setContainerDefaults(db, { + micronix_tube: { totalQuantity: 1, remainingQuantity: 1, defaultUnitSymbol: 'uL' }, + cryovial_tube: { totalQuantity: 1, remainingQuantity: 1, defaultUnitSymbol: 'uL' }, + paper: { totalQuantity: 1, remainingQuantity: 1, defaultUnitSymbol: 'uL' }, + static_well: { totalQuantity: 1, remainingQuantity: 1, defaultUnitSymbol: 'uL' }, + }) + plate = await createTestMicronixPlate(db, { name: 'P1', barcode: 'P1-BC', locationId }) + }) + + afterEach(() => { + cleanupTestDatabase(sqlite) + }) + + const tube = (barcode: string, collection: Record): RegistrationContainer => ({ + containerType: 'micronix_tube', + barcode, + collection: { type: 'micronix_plate', ...collection }, + }) + + const paperOn = (sheetName: string, sublabel?: string, boxLocation = true): RegistrationContainer => ({ + containerType: 'paper', + sublabel, + collection: { + type: 'sheet', + name: sheetName, + parent: { type: 'box', name: 'PaperBox', ...(boxLocation ? { locationId } : {}) }, + }, + }) + + const entry = (containers: RegistrationContainer[], collectionDate = '2024-01-15'): RegistrationEntry => ({ + source: { subjectId }, + specimenTypeId, + collectionDate, + containers, + }) + + async function rejection(entries: RegistrationEntry[]): Promise { + const error = await registerSpecimens(db, entries).then( + () => null, + (e: unknown) => e, + ) + if (!(error instanceof SpecimenRegistrationError)) throw new Error(`expected a registration error, got ${String(error)}`) + return error + } + + const count = async (table: typeof specimen | typeof storageContainer | typeof micronixPlate | typeof sheet) => + (await db.select().from(table)).length + + it('registers a specimen with a tube, normalizing the grid position', async () => { + const result = await registerSpecimens(db, [entry([tube('T1', { name: 'P1', position: 'a1' })])]) + + expect(result.entries).toHaveLength(1) + expect(result.entries[0].reused).toBe(false) + const [row] = await db.select().from(micronixTube).where(eq(micronixTube.id, result.entries[0].containerIds[0])) + expect(row).toMatchObject({ collectionId: plate.id, barcode: 'T1', position: 'A01' }) + }) + + it('reuses the specimen for the same Source, type, and date', async () => { + await registerSpecimens(db, [entry([])]) + const result = await registerSpecimens(db, [ + entry([tube('T1', { name: 'P1', position: 'A01' })]), + entry([tube('T2', { name: 'P1', position: 'A02' })], '1/15/2024'), + ]) + + expect(result.entries.map((e) => e.reused)).toEqual([true, true]) + expect(result.entries[0].specimen.id).toBe(result.entries[1].specimen.id) + expect(await count(specimen)).toBe(1) + expect(await count(storageContainer)).toBe(2) + }) + + it('merges entries for one collection event into one new specimen', async () => { + const result = await registerSpecimens(db, [entry([]), entry([])]) + + expect(result.entries.map((e) => e.reused)).toEqual([false, true]) + expect(await count(specimen)).toBe(1) + }) + + it('adds containers to an existing specimen named by id', async () => { + const [first] = (await registerSpecimens(db, [entry([])])).entries + const result = await registerSpecimens(db, [{ specimenId: first.specimen.id, containers: [tube('T1', { name: 'P1', position: 'A01' })] }]) + + expect(result.entries[0]).toMatchObject({ reused: true, specimen: { id: first.specimen.id } }) + expect(result.entries[0].containerIds).toHaveLength(1) + }) + + it('reports every problem in the request and writes nothing', async () => { + await registerSpecimens(db, [entry([tube('TAKEN', { name: 'P1', position: 'A01' })], '2024-01-01')]) + const before = await count(storageContainer) + + const error = await rejection([ + { source: { subjectId }, specimenTypeId: 9999, containers: [] }, + entry([], 'not-a-date'), + entry([tube('T2', { name: 'P1', position: 'A01' })]), + entry([tube('TAKEN', { name: 'P1', position: 'B01' })]), + { specimenId: 9999, containers: [] }, + ]) + + expect(error.status).toBe(400) + expect(error.errors).toEqual([ + { entryIndex: 0, field: 'specimenTypeId', message: 'Specimen type not found' }, + { entryIndex: 1, field: 'collectionDate', message: expect.stringContaining('Invalid date') }, + { entryIndex: 2, containerIndex: 0, field: 'position', message: expect.stringContaining('Position A01 is already used') }, + { entryIndex: 3, containerIndex: 0, field: 'barcode', message: expect.stringContaining("Barcode 'TAKEN' already exists") }, + { entryIndex: 4, field: 'specimenId', message: 'Specimen not found' }, + ]) + expect(await count(storageContainer)).toBe(before) + }) + + describe('Collection identity', () => { + it('rejects a barcode that does not match the plate the name finds', async () => { + const error = await rejection([entry([tube('T1', { name: 'P1', barcode: 'UNKNOWN', position: 'A02' })])]) + + expect(error.errors[0]).toMatchObject({ field: 'collection', message: "Micronix plate 'P1' has barcode 'P1-BC', not 'UNKNOWN'." }) + expect(await count(storageContainer)).toBe(0) + }) + + it('rejects a name that does not match the plate the barcode finds', async () => { + await createTestMicronixPlate(db, { name: 'P2', barcode: 'P2-BC', locationId }) + + const error = await rejection([entry([tube('T1', { name: 'P2', barcode: 'P1-BC', position: 'A01' })])]) + + expect(error.errors[0].message).toContain("point to different micronix plates: 'P1' and 'P2'") + }) + + it('names the id when the plate it finds has another name', async () => { + const error = await rejection([entry([tube('T1', { id: plate.id, name: 'OTHER', position: 'A01' })])]) + + expect(error.errors[0].message).toBe(`Micronix plate id ${plate.id} is named 'P1', not 'OTHER'.`) + }) + + it('names the barcode when the plate it finds has another name', async () => { + const error = await rejection([entry([tube('T1', { barcode: 'P1-BC', name: 'OTHER', position: 'A01' })])]) + + expect(error.errors[0].message).toBe("Micronix plate with barcode 'P1-BC' is named 'P1', not 'OTHER'.") + }) + + it('treats a plate named by name and by barcode as one plate for position clashes', async () => { + const error = await rejection([ + entry([tube('T1', { name: 'P1', position: 'A01' })]), + entry([tube('T2', { barcode: 'P1-BC', position: 'A1' })]), + ]) + + expect(error.errors).toEqual([ + expect.objectContaining({ entryIndex: 1, field: 'position', message: expect.stringContaining('more than once in this request') }), + ]) + }) + + it('creates a missing plate once and lets later rows find it by name', async () => { + const result = await registerSpecimens(db, [ + entry([tube('T1', { name: 'NEW', barcode: 'NEW-BC', locationId, position: 'A01' })]), + entry([tube('T2', { name: 'NEW', position: 'A02' })]), + ]) + + expect(result.collectionsCreated).toEqual([{ type: 'micronix_plate', id: expect.any(Number), name: 'NEW' }]) + const [created] = await db.select().from(micronixPlate).where(eq(micronixPlate.name, 'NEW')) + expect(created).toMatchObject({ barcode: 'NEW-BC', locationId }) + }) + + it('refuses to create a plate without a location', async () => { + const error = await rejection([entry([tube('T1', { name: 'NOPE', position: 'A01' })])]) + + expect(error.errors[0].message).toBe('Collection not found and no location provided') + }) + + it('rejects two locations for one new plate', async () => { + const storageType = await createTestStorageType(db, { name: 'Fridge' }) + const other = await createTestLocation(db, { name: 'Other', storageTypeId: String(storageType.id), canContainCollections: true }) + + const error = await rejection([ + entry([tube('T1', { name: 'NEW', locationId, position: 'A01' })]), + entry([tube('T2', { name: 'NEW', locationId: other.id, position: 'A02' })]), + ]) + + expect(error.errors[0]).toMatchObject({ entryIndex: 1, message: "Micronix plate 'NEW' is given two different locations in this request." }) + }) + }) + + describe('Collection location', () => { + let other: number + + beforeEach(async () => { + const storageType = await createTestStorageType(db, { name: 'Fridge' }) + other = (await createTestLocation(db, { name: 'Other', storageTypeId: String(storageType.id), canContainCollections: true })).id + }) + + it('rejects a new plate given another location by a later call', async () => { + await registerSpecimens(db, [entry([tube('T1', { name: 'NEW', locationId, position: 'A01' })])]) + + const error = await rejection([entry([tube('T2', { name: 'NEW', locationId: other, position: 'A02' })])]) + + expect(error.errors[0]).toMatchObject({ field: 'collection', message: "Micronix plate 'NEW' is stored at a different location from the one given." }) + }) + + it('rejects a box given another location than the one it is stored at', async () => { + await registerSpecimens(db, [entry([paperOn('Sheet-1')])]) + + const error = await rejection([ + entry([{ containerType: 'paper', collection: { type: 'sheet', name: 'Sheet-2', parent: { type: 'box', name: 'PaperBox', locationId: other } } }]), + ]) + + expect(error.errors[0].message).toBe("Box 'PaperBox' is stored at a different location from the one given.") + }) + + it('accepts an existing plate given its own location or none', async () => { + await expect( + registerSpecimens(db, [entry([tube('T1', { name: 'P1', locationId, position: 'A02' }), tube('T2', { name: 'P1', position: 'A03' })])]), + ).resolves.toBeDefined() + }) + }) + + describe('Paper on Sheets', () => { + it('puts several papers on one new sheet', async () => { + const result = await registerSpecimens(db, [ + entry([paperOn('Sheet-1', 's1')]), + entry([paperOn('Sheet-1', 's2', false)], '2024-01-16'), + ]) + + expect(result.collectionsCreated.map((c) => c.type)).toEqual(['box', 'sheet']) + const papers = await db.select().from(paper) + expect(papers.map((p) => p.sheetId)).toEqual([papers[0].sheetId, papers[0].sheetId]) + }) + + it('rejects a sublabel used twice on one sheet', async () => { + await registerSpecimens(db, [entry([paperOn('Sheet-1', 's1')])]) + + const error = await rejection([entry([paperOn('Sheet-1', 's1'), paperOn('Sheet-1', 's2'), paperOn('Sheet-1', 's2')])]) + + expect(error.errors.map((e) => [e.containerIndex, e.message])).toEqual([ + [0, "Sublabel 's1' is already used on this sheet."], + [2, "Sublabel 's2' is used more than once on this sheet in this request."], + ]) + }) + }) + + it('names the input a value repeats in as the caller asks', async () => { + const entries = [entry([tube('T1', { name: 'P1', position: 'A02' }), tube('T1', { name: 'P1', position: 'A03' })])] + + const error = await registerSpecimens(db, entries, { inputName: 'your file' }).catch((e: unknown) => e) + + expect((error as SpecimenRegistrationError).errors[0].message).toBe( + "Barcode 'T1' is used more than once in your file. Each barcode must be unique.", + ) + }) + + it('keys a missing position to the position field', async () => { + const error = await rejection([entry([tube('T1', { name: 'P1' })])]) + + expect(error.errors).toEqual([ + { entryIndex: 0, containerIndex: 0, field: 'position', message: 'Position is required for micronix tubes (e.g. A01).' }, + ]) + }) + + it('rejects a unit not allowed for the container type', async () => { + const other = await createTestUnit(db, { symbol: 'mg', name: 'milligram', category: 'mass' }) + + const error = await rejection([entry([{ ...tube('T1', { name: 'P1', position: 'A01' }), unitId: other.id }])]) + + expect(error.errors[0]).toMatchObject({ field: 'unitId', message: expect.stringContaining("Unit 'mg' is not allowed") }) + }) + + describe('checkSpecimenRegistration', () => { + it('writes nothing and reports no errors for a registration that would succeed', async () => { + const entries = [entry([tube('T1', { name: 'NEW', locationId, position: 'A01' }), paperOn('Sheet-1')])] + + expect(await checkSpecimenRegistration(db, entries)).toEqual([]) + expect([await count(specimen), await count(micronixPlate), await count(sheet)]).toEqual([0, 1, 0]) + await expect(registerSpecimens(db, entries)).resolves.toBeDefined() + }) + + it('returns the errors registerSpecimens would throw', async () => { + const errors = await checkSpecimenRegistration(db, [entry([tube('T1', { name: 'P1', barcode: 'UNKNOWN', position: 'A01' })])]) + + expect(errors).toHaveLength(1) + }) + + it('rolls back to a savepoint inside an open transaction', async () => { + sqlite.exec('BEGIN') + await registerSpecimens(db, [entry([])]) + await checkSpecimenRegistration(db, [entry([tube('T1', { name: 'P1', position: 'A01' })], '2024-02-01')]) + sqlite.exec('COMMIT') + + expect([await count(specimen), await count(storageContainer)]).toEqual([1, 0]) + }) + }) + + it('reports a constraint hit while writing as a 409', async () => { + // Plan, then let another write take the well before this one writes: the race a plan cannot see. + const { plan, errors } = await planRegistration(db, [entry([tube('T1', { name: 'P1', position: 'A01' })])]) + expect(errors).toEqual([]) + await registerSpecimens(db, [entry([tube('T0', { name: 'P1', position: 'A01' })], '2024-01-01')]) + + const error = await writeRegistration(db, plan, undefined).then( + () => null, + (e: unknown) => e, + ) + + expect(error).toBeInstanceOf(SpecimenRegistrationError) + expect(error).toMatchObject({ status: 409, errors: [{ entryIndex: 0, containerIndex: 0 }] }) + }) + + it('blames a constraint hit on a new collection on the row that asked for it', async () => { + // Rows 0 and 1 use the existing plate; row 2 creates P2. Another write creates P2 first. + const entries = [ + entry([tube('T1', { name: 'P1', position: 'A01' })]), + entry([paperOn('Sheet-1'), tube('T2', { name: 'P2', locationId, position: 'A01' })], '2024-02-01'), + ] + const { plan, errors } = await planRegistration(db, entries) + expect(errors).toEqual([]) + await createTestMicronixPlate(db, { name: 'P2', barcode: 'P2-BC', locationId }) + sqlite.exec('CREATE UNIQUE INDEX test_plate_name ON micronix_plate(name)') + + const error = await writeRegistration(db, plan, undefined).then( + () => null, + (e: unknown) => e, + ) + + expect(error).toBeInstanceOf(SpecimenRegistrationError) + expect(error).toMatchObject({ status: 409 }) + expect((error as SpecimenRegistrationError).errors).toEqual([ + { entryIndex: 1, containerIndex: 1, message: 'A record with this name already exists.' }, + ]) + }) +}) diff --git a/packages/api/src/lib/specimen-registration/collections.ts b/packages/api/src/lib/specimen-registration/collections.ts new file mode 100644 index 00000000..699328b6 --- /dev/null +++ b/packages/api/src/lib/specimen-registration/collections.ts @@ -0,0 +1,270 @@ +/** + * Collection identity for registration: which Collection a container goes into, and which + * Collections the registration has to create. Every identifier a row gives must name the + * same Collection; nothing is created unless nothing matches and a location is given. + */ +import { and, eq } from 'drizzle-orm' +import type { ContainerWriteInput } from '@sampledb/contract' +import type { Database } from '../../db/client' +import { sheet } from '../../db/schema' +import { GRID_COLLECTION_TABLES, PARENT_COLLECTION_TABLES, type GridCollectionType, type ParentCollectionType } from './tables' +import { + assertLocationCanContainCollections, + CollectionLocationNotAllowedError, + CollectionLocationNotFoundError, +} from '../collections/collection-lifecycle' + +export type { GridCollectionType, ParentCollectionType } from './tables' + +type GridCollectionInput = NonNullable< + Extract['collection'] +> +type SheetCollectionInput = NonNullable['collection']> +type ParentCollectionInput = NonNullable + +/** A Collection that exists, or one this registration will create (index into `planned`). */ +export type CollectionTarget = { existingId: number } | { plannedIndex: number } + +export type PlannedCollection = + | { type: GridCollectionType; name: string; barcode?: string; locationId: number } + | { type: ParentCollectionType; name: string; locationId: number } + | { type: 'sheet'; name: string; parentType: ParentCollectionType; parent: CollectionTarget } + +export type Resolution = { target: CollectionTarget } | { error: string } + +type Candidate = { + target: CollectionTarget + type: GridCollectionType | ParentCollectionType + name: string + barcode: string | null + locationId: number | null +} + +const LABELS: Record = { + micronix_plate: 'Micronix plate', + cryovial_box: 'Cryovial box', + box: 'Box', + bag: 'Bag', +} + +export function targetKey(target: CollectionTarget): string { + return 'existingId' in target ? `e${target.existingId}` : `p${target.plannedIndex}` +} + +function distinctCandidates(candidates: Candidate[]): Candidate[] { + const seen = new Map() + for (const candidate of candidates) seen.set(targetKey(candidate.target), candidate) + return [...seen.values()] +} + +function locationError(locationId: number, database: Database): string | null { + try { + assertLocationCanContainCollections(database, locationId) + return null + } catch (error) { + if (error instanceof CollectionLocationNotFoundError || error instanceof CollectionLocationNotAllowedError) { + return error.message + } + throw error + } +} + +/** Resolves Collections for one registration, remembering the ones it plans to create. */ +export class CollectionPlanner { + readonly planned: PlannedCollection[] = [] + + constructor(private readonly database: Database) {} + + private plannedCandidates( + type: GridCollectionType | ParentCollectionType, + match: (planned: PlannedCollection) => boolean, + ): Candidate[] { + const out: Candidate[] = [] + this.planned.forEach((planned, plannedIndex) => { + if (planned.type === type && match(planned)) { + out.push({ + target: { plannedIndex }, + type, + name: planned.name, + barcode: 'barcode' in planned ? (planned.barcode ?? null) : null, + locationId: 'locationId' in planned ? planned.locationId : null, + }) + } + }) + return out + } + + /** + * A location given for a Collection that exists or is already planned must be the one it has. + * Checking stored Collections too makes the rule hold when a request is split over several calls. + */ + private locationConflict(match: Candidate, locationId: number | undefined): string | null { + if (locationId == null || match.locationId === locationId) return null + return 'plannedIndex' in match.target + ? `${LABELS[match.type]} '${match.name}' is given two different locations in this request.` + : `${LABELS[match.type]} '${match.name}' is stored at a different location from the one given.` + } + + /** The stored grid Collection whose id, barcode, or name equals `value`. */ + private storedGrid(type: GridCollectionType, by: 'id' | 'barcode' | 'name', value: number | string): Candidate | null { + const table = GRID_COLLECTION_TABLES[type] + const row = this.database + .select({ id: table.id, name: table.name, barcode: table.barcode, locationId: table.locationId }) + .from(table) + .where(eq(table[by], value)) + .get() + return row ? { target: { existingId: row.id }, type, name: row.name, barcode: row.barcode, locationId: row.locationId } : null + } + + /** The stored box or bag whose id or name equals `value`. */ + private storedParent(type: ParentCollectionType, by: 'id' | 'name', value: number | string): Candidate | null { + const table = PARENT_COLLECTION_TABLES[type] + const row = this.database + .select({ id: table.id, name: table.name, locationId: table.locationId }) + .from(table) + .where(eq(table[by], value)) + .get() + return row ? { target: { existingId: row.id }, type, name: row.name, barcode: null, locationId: row.locationId } : null + } + + async resolveGrid(type: GridCollectionType, collection: GridCollectionInput): Promise { + const label = LABELS[type] + const candidates: Candidate[] = [] + + if (collection.id != null) { + const byId = this.storedGrid(type, 'id', collection.id) + if (!byId) return { error: `${label} not found: id ${collection.id}` } + candidates.push(byId) + } + const { barcode, name } = collection + if (barcode) { + const stored = this.storedGrid(type, 'barcode', barcode) + candidates.push(...(stored ? [stored] : this.plannedCandidates(type, (p) => 'barcode' in p && p.barcode === barcode))) + } + if (name) { + const stored = this.storedGrid(type, 'name', name) + candidates.push(...(stored ? [stored] : this.plannedCandidates(type, (p) => p.name === name))) + } + + const matches = distinctCandidates(candidates) + if (matches.length > 1) { + return { + error: `Collection name and barcode point to different ${label.toLowerCase()}s: ${matches + .map((m) => `'${m.name}'`) + .join(' and ')}.`, + } + } + const match = matches.at(0) + if (match) { + if (collection.barcode && match.barcode !== collection.barcode) { + const actual = match.barcode ? `has barcode '${match.barcode}'` : 'has no barcode' + return { error: `${label} '${match.name}' ${actual}, not '${collection.barcode}'.` } + } + if (collection.name && match.name !== collection.name) { + // The name matched nothing, so the id or, failing that, the barcode found this one. + const found = collection.id != null ? `${label} id ${collection.id}` : `${label} with barcode '${collection.barcode ?? ''}'` + return { error: `${found} is named '${match.name}', not '${collection.name}'.` } + } + const conflict = this.locationConflict(match, collection.locationId) + if (conflict) return { error: conflict } + return { target: match.target } + } + + if (collection.locationId == null) { + return { error: 'Collection not found and no location provided' } + } + const newName = name ?? barcode + if (!newName) { + return { error: `A new ${label.toLowerCase()} needs a name or a barcode.` } + } + const badLocation = locationError(collection.locationId, this.database) + if (badLocation) return { error: badLocation } + this.planned.push({ type, name: newName, barcode, locationId: collection.locationId }) + return { target: { plannedIndex: this.planned.length - 1 } } + } + + async resolveParent(parent: ParentCollectionInput): Promise { + const type = parent.type + const label = LABELS[type] + const candidates: Candidate[] = [] + + if (parent.id != null) { + const byId = this.storedParent(type, 'id', parent.id) + if (!byId) return { error: `${label} not found: id ${parent.id}` } + candidates.push(byId) + } + const name = parent.name + if (name) { + const stored = this.storedParent(type, 'name', name) + candidates.push(...(stored ? [stored] : this.plannedCandidates(type, (p) => p.name === name))) + } + + const matches = distinctCandidates(candidates) + const match = matches.at(0) + if (match) { + if (matches.length > 1 || (parent.name && match.name !== parent.name)) { + return { error: `${label} id ${parent.id ?? ''} is named '${match.name}', not '${parent.name ?? ''}'.` } + } + const conflict = this.locationConflict(match, parent.locationId) + if (conflict) return { error: conflict } + return { target: match.target } + } + + if (!parent.name) { + return { error: `A new ${label.toLowerCase()} needs a name.` } + } + if (parent.locationId == null) { + return { error: `Collection (${type}) not found: ${parent.name}` } + } + const badLocation = locationError(parent.locationId, this.database) + if (badLocation) return { error: badLocation } + this.planned.push({ type, name: parent.name, locationId: parent.locationId }) + return { target: { plannedIndex: this.planned.length - 1 } } + } + + async resolveSheet(collection: SheetCollectionInput): Promise { + if (collection.id != null) { + const row = await this.database.select().from(sheet).where(eq(sheet.id, collection.id)).get() + if (!row) return { error: `Sheet not found: id ${collection.id}` } + if (collection.name && collection.name !== row.name) { + return { error: `Sheet id ${collection.id} is named '${row.name}', not '${collection.name}'.` } + } + if (collection.parent) { + const parent = await this.resolveParent(collection.parent) + if ('error' in parent) return parent + const parentId = collection.parent.type === 'box' ? row.boxId : row.bagId + if (!('existingId' in parent.target) || parent.target.existingId !== parentId) { + return { error: `Sheet id ${collection.id} is not in that ${collection.parent.type}.` } + } + } + return { target: { existingId: row.id } } + } + + if (!collection.name) return { error: 'A sheet needs an id or a name.' } + if (!collection.parent) return { error: 'A sheet named without an id needs its box or bag.' } + + const name = collection.name + const parentType = collection.parent.type + const parent = await this.resolveParent(collection.parent) + if ('error' in parent) return parent + + if ('existingId' in parent.target) { + const parentColumn = parentType === 'box' ? sheet.boxId : sheet.bagId + const row = await this.database + .select({ id: sheet.id }) + .from(sheet) + .where(and(eq(sheet.name, name), eq(parentColumn, parent.target.existingId))) + .get() + if (row) return { target: { existingId: row.id } } + } + + const parentKey = targetKey(parent.target) + const plannedIndex = this.planned.findIndex( + (p) => p.type === 'sheet' && p.name === name && p.parentType === parentType && targetKey(p.parent) === parentKey, + ) + if (plannedIndex >= 0) return { target: { plannedIndex } } + + this.planned.push({ type: 'sheet', name, parentType, parent: parent.target }) + return { target: { plannedIndex: this.planned.length - 1 } } + } +} diff --git a/packages/api/src/lib/specimen-registration/container-fields.ts b/packages/api/src/lib/specimen-registration/container-fields.ts new file mode 100644 index 00000000..9135a139 --- /dev/null +++ b/packages/api/src/lib/specimen-registration/container-fields.ts @@ -0,0 +1,59 @@ +import type { ContainerWriteInput } from '@sampledb/contract' + +function hasGridCollection(collection: { id?: number; name?: string; barcode?: string } | undefined): boolean { + if (!collection) return false + return collection.id != null || Boolean(collection.name) || Boolean(collection.barcode) +} + +/** The input field a missing or misplaced value belongs to. */ +export type ContainerWriteField = 'barcode' | 'sublabel' | 'collection' | 'position' + +function hasPosition(collection: { position?: string } | undefined): boolean { + return Boolean(collection?.position?.trim()) +} + +function positionRequired(containers: string) { + return { valid: false, field: 'position', error: `Position is required for ${containers} (e.g. A01).` } as const +} + +/** + * Required ContainerWriteInput fields by type (no DB uniqueness checks). + */ +export function validateContainerWriteFields( + input: ContainerWriteInput +): { valid: true } | { valid: false; field: ContainerWriteField; error: string } { + if (input.containerType === 'micronix_tube') { + if (!input.barcode) { + return { valid: false, field: 'barcode', error: 'Barcode is required for micronix tubes' } + } + if (!hasGridCollection(input.collection)) { + return { valid: false, field: 'collection', error: 'Collection name or barcode is required' } + } + if (!hasPosition(input.collection)) return positionRequired('micronix tubes') + } else if (input.containerType === 'cryovial_tube') { + if (!hasGridCollection(input.collection)) { + return { valid: false, field: 'collection', error: 'Collection name or barcode is required' } + } + if (!hasPosition(input.collection)) return positionRequired('cryovial tubes') + } else if (input.containerType === 'paper') { + const collection = input.collection + if (!collection) { + return { valid: false, field: 'collection', error: 'Collection is required for paper containers' } + } + if (collection.id == null) { + if (!collection.parent || (collection.parent.id == null && !collection.parent.name)) { + return { valid: false, field: 'collection', error: 'Collection name is required for papers' } + } + if (!collection.name) { + return { valid: false, field: 'collection', error: 'Sheet name is required for papers' } + } + } + } else { + if (!hasGridCollection(input.collection)) { + return { valid: false, field: 'collection', error: 'Collection name or barcode is required' } + } + if (!hasPosition(input.collection)) return positionRequired('static wells') + } + + return { valid: true } +} diff --git a/packages/api/src/lib/specimen-registration/index.ts b/packages/api/src/lib/specimen-registration/index.ts new file mode 100644 index 00000000..0542b156 --- /dev/null +++ b/packages/api/src/lib/specimen-registration/index.ts @@ -0,0 +1,52 @@ +/** + * Specimen registration (CONTEXT.md, ADR 0010): record Specimens from a known Source with + * their Containers and placement, all or nothing. The check is the same write, rolled back, + * so a registration that passes the check succeeds when written. + */ +import type { Database } from '../../db/client' +import { withRolledBackTransaction, withWriteTransaction } from '../../db/write-transaction' +import { planRegistration } from './plan' +import { writeRegistration } from './write' +import { + SpecimenRegistrationError, + type RegistrationEntry, + type RegistrationError, + type RegistrationOptions, + type RegistrationResult, +} from './types' + +export * from './types' +export { validateContainerWriteFields } from './container-fields' + +/** + * Register every entry, or nothing. Throws SpecimenRegistrationError listing every problem + * (400), or the constraint a write hit (409). Inside an open transaction, joins it. + */ +export async function registerSpecimens( + database: Database, + entries: RegistrationEntry[], + options: RegistrationOptions = {}, +): Promise { + return withWriteTransaction(database, async (db) => { + const { plan, errors } = await planRegistration(db, entries, options.inputName) + if (errors.length > 0) throw new SpecimenRegistrationError(400, errors) + return writeRegistration(db, plan, options.userId) + }) +} + +/** Run registerSpecimens and roll it back, returning the errors it found (none if it would succeed). */ +export async function checkSpecimenRegistration( + database: Database, + entries: RegistrationEntry[], + options: RegistrationOptions = {}, +): Promise { + return withRolledBackTransaction(database, async (db) => { + try { + await registerSpecimens(db, entries, options) + return [] + } catch (error) { + if (error instanceof SpecimenRegistrationError) return error.errors + throw error + } + }) +} diff --git a/packages/api/src/lib/specimen-registration/plan.ts b/packages/api/src/lib/specimen-registration/plan.ts new file mode 100644 index 00000000..5f9e07e2 --- /dev/null +++ b/packages/api/src/lib/specimen-registration/plan.ts @@ -0,0 +1,324 @@ +/** + * Plan a registration: resolve every Specimen, Collection, unit, and placement, and collect + * every problem. Reads only; the write step trusts the plan. + */ +import { and, eq, isNull } from 'drizzle-orm' +import type { Database } from '../../db/client' +import { + controlBatch, + paper, + specimen, + specimenType, + studySubject, + type Specimen, +} from '../../db/schema' +import { getDefaultRemainingQuantity, getDefaultTotalQuantity, getDefaultUnit } from '../defaults' +import { normalizePosition } from '../normalize-position' +import { checkGridPositionOccupancy, occupiedGridPositionMessage } from '../container-occupancy' +import { + validateCollectionDate, + validateContainerTypeForSpecimenType, + validateUnitForContainerType, +} from '../validation' +import { GRID_COLLECTION_OF, TUBE_TABLES } from './tables' +import { CollectionPlanner, targetKey, type CollectionTarget, type GridCollectionType, type PlannedCollection } from './collections' +import { validateContainerWriteFields } from './container-fields' +import type { RegistrationContainer, RegistrationEntry, RegistrationError, RegistrationSource } from './types' + +export type PlannedSpecimen = + | { existing: Specimen } + | { + existing: null + source: RegistrationSource + specimenTypeId: number + collectionDate: string | null + } + +export type PlannedContainer = { + entryIndex: number + containerIndex: number + input: RegistrationContainer + unitId: number + totalQuantity: number + remainingQuantity: number + /** Plate or box for tubes and wells, Sheet for paper. */ + collection: CollectionTarget + position: string | null +} + +export type RegistrationPlan = { + specimens: PlannedSpecimen[] + /** entries[i] registers specimens[specimenIndexOfEntry[i]]; undefined when the entry has errors. */ + specimenIndexOfEntry: Array + containers: PlannedContainer[] + collections: PlannedCollection[] +} + +function findSpecimen( + database: Database, + source: RegistrationSource, + specimenTypeId: number, + collectionDate: string | null, +): Specimen | null { + const sourceCondition = + 'subjectId' in source + ? eq(specimen.studySubjectId, source.subjectId) + : eq(specimen.controlBatchId, source.controlBatchId) + const dateCondition = collectionDate == null ? isNull(specimen.collectionDate) : eq(specimen.collectionDate, collectionDate) + return ( + database + .select() + .from(specimen) + .where(and(sourceCondition, eq(specimen.specimenTypeId, specimenTypeId), dateCondition)) + .get() ?? null + ) +} + +/** What makes two entries the same Specimen (ADR 0010, Specimen identity). */ +type SpecimenIdentity = + | { kind: 'existing'; specimenId: number } + | { kind: 'new'; source: RegistrationSource; specimenTypeId: number; collectionDate: string | null } + +function specimenIdentityKey(identity: SpecimenIdentity): string { + if (identity.kind === 'existing') return `id:${identity.specimenId}` + const { source, specimenTypeId, collectionDate } = identity + const sourceKey = 'subjectId' in source ? `subject:${source.subjectId}` : `control:${source.controlBatchId}` + return `${sourceKey}:${specimenTypeId}:${collectionDate ?? ''}` +} + +/** Something a request may use only once. */ +type Claim = + | { kind: 'barcode'; containerType: 'micronix_tube' | 'cryovial_tube'; barcode: string } + | { kind: 'position'; gridType: GridCollectionType; collection: CollectionTarget; position: string } + | { kind: 'sublabel'; sheet: CollectionTarget; sublabel: string } + +function claimKey(claim: Claim): string { + switch (claim.kind) { + case 'barcode': + return `barcode:${claim.containerType}:${claim.barcode}` + case 'position': + return `position:${claim.gridType}:${targetKey(claim.collection)}:${claim.position}` + case 'sublabel': + return `sublabel:${targetKey(claim.sheet)}:${claim.sublabel}` + } +} + +/** Per-request uniqueness: barcodes, Grid positions per Collection, sublabels per Sheet. */ +class RequestClaims { + private readonly taken = new Set() + + /** How messages name the input a claim repeats in (RegistrationOptions.inputName). */ + constructor(readonly inputName: string) {} + + /** Returns false when the claim was already taken earlier in the request. */ + take(claim: Claim): boolean { + const key = claimKey(claim) + if (this.taken.has(key)) return false + this.taken.add(key) + return true + } +} + +export async function planRegistration( + database: Database, + entries: RegistrationEntry[], + inputName = 'this request', +): Promise<{ plan: RegistrationPlan; errors: RegistrationError[] }> { + const errors: RegistrationError[] = [] + const planner = new CollectionPlanner(database) + const claims = new RequestClaims(inputName) + const specimens: PlannedSpecimen[] = [] + const specimenIndexByKey = new Map() + const specimenIndexOfEntry: Array = [] + const containers: PlannedContainer[] = [] + + /** The index in `specimens` of the Specimen with this identity, planning it on first sight. */ + const specimenIndexFor = (identity: SpecimenIdentity, make: () => PlannedSpecimen): number => { + const key = specimenIdentityKey(identity) + const known = specimenIndexByKey.get(key) + if (known !== undefined) return known + specimens.push(make()) + specimenIndexByKey.set(key, specimens.length - 1) + return specimens.length - 1 + } + + for (let entryIndex = 0; entryIndex < entries.length; entryIndex++) { + const entry = entries[entryIndex] + const entryError = (field: string, message: string) => errors.push({ entryIndex, field, message }) + let specimenTypeId: number | null = null + + if ('specimenId' in entry) { + const existing = database.select().from(specimen).where(eq(specimen.id, entry.specimenId)).get() + if (!existing) { + entryError('specimenId', 'Specimen not found') + specimenIndexOfEntry.push(undefined) + continue + } + specimenTypeId = existing.specimenTypeId + specimenIndexOfEntry.push(specimenIndexFor({ kind: 'existing', specimenId: existing.id }, () => ({ existing }))) + } else { + let sourceOk = true + if ('subjectId' in entry.source) { + const subjectId = entry.source.subjectId + if (!database.select({ id: studySubject.id }).from(studySubject).where(eq(studySubject.id, subjectId)).get()) { + entryError('source', 'Subject not found') + sourceOk = false + } + } else { + const batchId = entry.source.controlBatchId + if (!database.select({ id: controlBatch.id }).from(controlBatch).where(eq(controlBatch.id, batchId)).get()) { + entryError('source', 'Control batch not found') + sourceOk = false + } + } + const typeExists = database + .select({ id: specimenType.id }) + .from(specimenType) + .where(eq(specimenType.id, entry.specimenTypeId)) + .get() + if (typeExists) specimenTypeId = entry.specimenTypeId + else entryError('specimenTypeId', 'Specimen type not found') + + const date = validateCollectionDate(entry.collectionDate) + if (!date.valid) entryError('collectionDate', date.error) + + if (sourceOk && specimenTypeId !== null && date.valid) { + const identity = { + kind: 'new' as const, + source: entry.source, + specimenTypeId, + collectionDate: date.normalized ?? null, + } + specimenIndexOfEntry.push( + specimenIndexFor(identity, () => { + const { source, specimenTypeId: typeId, collectionDate } = identity + const existing = findSpecimen(database, source, typeId, collectionDate) + return existing ? { existing } : { existing: null, source, specimenTypeId: typeId, collectionDate } + }), + ) + } else { + specimenIndexOfEntry.push(undefined) + } + } + + for (let containerIndex = 0; containerIndex < entry.containers.length; containerIndex++) { + const planned = await planContainer(database, planner, claims, entry.containers[containerIndex], specimenTypeId) + if ('errors' in planned) { + for (const e of planned.errors) errors.push({ entryIndex, containerIndex, ...e }) + } else { + containers.push({ entryIndex, containerIndex, ...planned }) + } + } + } + + return { plan: { specimens, specimenIndexOfEntry, containers, collections: planner.planned }, errors } +} + +type ContainerProblem = { field: string; message: string } + +async function planContainer( + database: Database, + planner: CollectionPlanner, + claims: RequestClaims, + input: RegistrationContainer, + specimenTypeId: number | null, +): Promise | { errors: ContainerProblem[] }> { + const problems: ContainerProblem[] = [] + const containerType = input.containerType + + const fields = validateContainerWriteFields(input) + if (!fields.valid) { + return { errors: [{ field: fields.field, message: fields.error }] } + } + + if (specimenTypeId !== null) { + const allowed = await validateContainerTypeForSpecimenType(database, specimenTypeId, containerType) + if (!allowed.valid) problems.push({ field: 'containerType', message: allowed.error ?? 'Container type not allowed' }) + } + + let unitId = input.unitId ?? null + let totalQuantity = 0 + let remainingQuantity = 0 + try { + unitId ??= await getDefaultUnit(database, containerType) + const unitAllowed = await validateUnitForContainerType(database, containerType, unitId) + if (!unitAllowed.valid) problems.push({ field: 'unitId', message: unitAllowed.error ?? 'Unit not allowed' }) + totalQuantity = input.totalQuantity ?? (await getDefaultTotalQuantity(database, containerType)) + remainingQuantity = + input.remainingQuantity ?? input.totalQuantity ?? (await getDefaultRemainingQuantity(database, containerType)) + } catch (error) { + problems.push({ field: 'unitId', message: error instanceof Error ? error.message : 'Container defaults are not configured' }) + } + + let collection: CollectionTarget | null = null + let position: string | null = null + + if (input.containerType === 'paper') { + // validateContainerWriteFields guarantees a collection. + const resolved = input.collection ? await planner.resolveSheet(input.collection) : { error: 'Collection is required for paper containers' } + if ('error' in resolved) { + problems.push({ field: 'collection', message: resolved.error }) + } else { + collection = resolved.target + const sublabel = input.sublabel?.trim() + if (sublabel) { + if ('existingId' in collection) { + const taken = database + .select({ id: paper.id }) + .from(paper) + .where(and(eq(paper.sheetId, collection.existingId), eq(paper.sublabel, sublabel))) + .get() + if (taken) problems.push({ field: 'sublabel', message: `Sublabel '${sublabel}' is already used on this sheet.` }) + } + if (!claims.take({ kind: 'sublabel', sheet: collection, sublabel })) { + problems.push({ field: 'sublabel', message: `Sublabel '${sublabel}' is used more than once on this sheet in ${claims.inputName}.` }) + } + } + } + } else { + const gridType = GRID_COLLECTION_OF[input.containerType] + const resolved = input.collection + ? await planner.resolveGrid(gridType, input.collection) + : { error: 'Collection name or barcode is required' } + if ('error' in resolved) { + problems.push({ field: 'collection', message: resolved.error }) + } else { + collection = resolved.target + position = normalizePosition(input.collection?.position) + if (position) { + if ('existingId' in collection) { + const occupancy = await checkGridPositionOccupancy(database, { + collectionKind: gridType, + collectionId: collection.existingId, + position, + }) + if (occupancy.occupied) problems.push({ field: 'position', message: occupiedGridPositionMessage(position, gridType) }) + } + if (!claims.take({ kind: 'position', gridType, collection, position })) { + problems.push({ + field: 'position', + message: `Position ${position} in this plate/box is used more than once in ${claims.inputName}. Each position can only be used once.`, + }) + } + } + } + + const barcode = 'barcode' in input ? input.barcode?.trim() : undefined + if (barcode && (input.containerType === 'micronix_tube' || input.containerType === 'cryovial_tube')) { + const table = TUBE_TABLES[input.containerType] + const taken = await database.select({ id: table.id }).from(table).where(eq(table.barcode, barcode)).get() + if (taken) problems.push({ field: 'barcode', message: `Barcode '${barcode}' already exists. Use a different barcode.` }) + if (!claims.take({ kind: 'barcode', containerType: input.containerType, barcode })) { + problems.push({ + field: 'barcode', + message: `Barcode '${barcode}' is used more than once in ${claims.inputName}. Each barcode must be unique.`, + }) + } + } + } + + if (problems.length > 0 || collection === null || unitId === null) { + return { errors: problems } + } + return { input, unitId, totalQuantity, remainingQuantity, collection, position } +} diff --git a/packages/api/src/lib/specimen-registration/tables.ts b/packages/api/src/lib/specimen-registration/tables.ts new file mode 100644 index 00000000..2cec3789 --- /dev/null +++ b/packages/api/src/lib/specimen-registration/tables.ts @@ -0,0 +1,22 @@ +/** Which Collection each container type goes in, and the table behind each type. One place. */ +import { bag, box, cryovialBox, cryovialTube, micronixPlate, micronixTube, staticWell } from '../../db/schema' + +export type GridContainerType = 'micronix_tube' | 'cryovial_tube' | 'static_well' +export type GridCollectionType = 'micronix_plate' | 'cryovial_box' +export type ParentCollectionType = 'box' | 'bag' + +/** The plate or box a tube or well goes in. */ +export const GRID_COLLECTION_OF: Record = { + micronix_tube: 'micronix_plate', + cryovial_tube: 'cryovial_box', + static_well: 'micronix_plate', +} + +export const GRID_COLLECTION_TABLES = { micronix_plate: micronixPlate, cryovial_box: cryovialBox } as const + +export const PARENT_COLLECTION_TABLES = { box, bag } as const + +/** Tube and well tables; only tubes carry a barcode. */ +export const GRID_CONTAINER_TABLES = { micronix_tube: micronixTube, cryovial_tube: cryovialTube, static_well: staticWell } as const + +export const TUBE_TABLES = { micronix_tube: micronixTube, cryovial_tube: cryovialTube } as const diff --git a/packages/api/src/lib/specimen-registration/types.ts b/packages/api/src/lib/specimen-registration/types.ts new file mode 100644 index 00000000..4d5ae384 --- /dev/null +++ b/packages/api/src/lib/specimen-registration/types.ts @@ -0,0 +1,83 @@ +import type { ContainerWriteInput } from '@sampledb/contract' +import type { Specimen } from '../../db/schema' +import type { GridCollectionType, ParentCollectionType } from './tables' + +/** A container to register: the unified write shape plus optional unit and quantity overrides. */ +export type RegistrationContainer = ContainerWriteInput & { + unitId?: number + totalQuantity?: number + remainingQuantity?: number +} + +/** Where the specimen's material came from, already resolved to a record id. */ +export type RegistrationSource = { subjectId: number } | { controlBatchId: number } + +/** + * One collection event and its containers. Name the Source, specimen type, and date to find or + * create the Specimen, or name an existing Specimen to add containers (aliquots) to it. + */ +export type RegistrationEntry = + | { + source: RegistrationSource + specimenTypeId: number + /** Optional; validated and normalized to YYYY-MM-DD. */ + collectionDate?: string + containers: RegistrationContainer[] + } + | { + specimenId: number + containers: RegistrationContainer[] + } + +export type RegistrationOptions = { + userId?: number + /** + * How errors name the caller's input when a value repeats in it, for example 'your file' + * for a CSV import. Defaults to 'this request'. + */ + inputName?: string +} + +/** Keyed to the input: entries[entryIndex].containers[containerIndex]. */ +export type RegistrationError = { + entryIndex: number + containerIndex?: number + field?: string + message: string +} + +export type RegisteredEntry = { + specimen: Specimen + /** True when the Specimen existed before this entry, in the database or from an earlier entry. */ + reused: boolean + containerIds: number[] +} + +export type CreatedCollection = { + type: GridCollectionType | ParentCollectionType | 'sheet' + id: number + name: string +} + +export type RegistrationResult = { + entries: RegisteredEntry[] + collectionsCreated: CreatedCollection[] +} + +/** + * A write refused with every problem found, keyed however the caller keys its input. + * Status 400: the input has problems, all listed. Status 409: a write hit a database constraint + * the plan did not foresee (for example a concurrent registration). + */ +export class RefusedWithErrors extends Error { + constructor( + readonly status: 400 | 409, + readonly errors: E[], + ) { + super(errors[0]?.message ?? 'Request refused') + this.name = new.target.name + } +} + +/** Registration refused; errors are keyed to the registration's entries. */ +export class SpecimenRegistrationError extends RefusedWithErrors {} diff --git a/packages/api/src/lib/specimen-registration/write.ts b/packages/api/src/lib/specimen-registration/write.ts new file mode 100644 index 00000000..087bd5d5 --- /dev/null +++ b/packages/api/src/lib/specimen-registration/write.ts @@ -0,0 +1,151 @@ +/** Carry out a registration plan that has no errors. Call inside a write transaction. */ +import type { Database } from '../../db/client' +import { paper, sheet, specimen, storageContainer, type Specimen } from '../../db/schema' +import { utcNow } from '../datetime' +import { sqliteConstraintConflict } from '../error-handler' +import { insertedRow } from '../inserted-row' +import type { CollectionTarget, PlannedCollection } from './collections' +import type { RegistrationPlan } from './plan' +import { GRID_COLLECTION_TABLES, GRID_CONTAINER_TABLES, PARENT_COLLECTION_TABLES } from './tables' +import { + SpecimenRegistrationError, + type CreatedCollection, + type RegisteredEntry, + type RegistrationResult, +} from './types' + +type ContainerRef = { entryIndex: number; containerIndex?: number } + +/** The first container that needs each planned Collection: directly, or as its Sheet's box or bag. */ +function collectionRequesters(plan: RegistrationPlan): ContainerRef[] { + const requesters: Array = plan.collections.map(() => undefined) + for (const { entryIndex, containerIndex, collection } of plan.containers) { + let target: CollectionTarget | null = collection + while (target && 'plannedIndex' in target) { + requesters[target.plannedIndex] ??= { entryIndex, containerIndex } + const planned: PlannedCollection = plan.collections[target.plannedIndex] + target = planned.type === 'sheet' ? planned.parent : null + } + } + return requesters.map((requester) => { + if (!requester) throw new Error('Planned collection is not used by any container') + return requester + }) +} + +export async function writeRegistration( + database: Database, + plan: RegistrationPlan, + userId: number | undefined, +): Promise { + const now = utcNow() + const audit = { created: now, lastUpdated: now, createdBy: userId, updatedBy: userId } + const requesters = collectionRequesters(plan) + let current: ContainerRef = { entryIndex: 0 } + + try { + const collectionIds: number[] = [] + const collectionsCreated: CreatedCollection[] = [] + const idOf = (target: CollectionTarget): number => + 'existingId' in target ? target.existingId : collectionIds[target.plannedIndex] + + for (const [plannedIndex, planned] of plan.collections.entries()) { + current = requesters[plannedIndex] + let id: number + switch (planned.type) { + case 'micronix_plate': + case 'cryovial_box': { + const table = GRID_COLLECTION_TABLES[planned.type] + const values = { name: planned.name, barcode: planned.barcode ?? null, locationId: planned.locationId, ...audit } + id = insertedRow(await database.insert(table).values(values).returning({ id: table.id }), planned.type).id + break + } + case 'box': + case 'bag': { + const table = PARENT_COLLECTION_TABLES[planned.type] + const values = { name: planned.name, locationId: planned.locationId, ...audit } + id = insertedRow(await database.insert(table).values(values).returning({ id: table.id }), planned.type).id + break + } + case 'sheet': { + const parentId = idOf(planned.parent) + const values = { + name: planned.name, + boxId: planned.parentType === 'box' ? parentId : null, + bagId: planned.parentType === 'bag' ? parentId : null, + ...audit, + } + id = insertedRow(await database.insert(sheet).values(values).returning({ id: sheet.id }), 'sheet').id + break + } + } + collectionIds.push(id) + collectionsCreated.push({ type: planned.type, id, name: planned.name }) + } + + const specimenRows: Array = [] + const entries: RegisteredEntry[] = [] + for (let entryIndex = 0; entryIndex < plan.specimenIndexOfEntry.length; entryIndex++) { + current = { entryIndex } + const index = plan.specimenIndexOfEntry[entryIndex] + if (index === undefined) throw new Error('Registration plan has an entry without a specimen') + const planned = plan.specimens[index] + let row = specimenRows[index] + let reused = true + if (!row) { + if (planned.existing) { + row = planned.existing + } else { + const values = { + studySubjectId: 'subjectId' in planned.source ? planned.source.subjectId : null, + controlBatchId: 'controlBatchId' in planned.source ? planned.source.controlBatchId : null, + specimenTypeId: planned.specimenTypeId, + collectionDate: planned.collectionDate, + ...audit, + } + row = insertedRow(await database.insert(specimen).values(values).returning(), 'specimen') + reused = false + } + specimenRows[index] = row + } + entries.push({ specimen: row, reused, containerIds: [] }) + } + + for (const container of plan.containers) { + current = { entryIndex: container.entryIndex, containerIndex: container.containerIndex } + const entry = entries[container.entryIndex] + const input = container.input + const { id } = insertedRow( + await database + .insert(storageContainer) + .values({ + specimenId: entry.specimen.id, + unitId: container.unitId, + totalQuantity: container.totalQuantity, + remainingQuantity: container.remainingQuantity, + comment: input.comment, + ...audit, + }) + .returning({ id: storageContainer.id }), + 'container', + ) + const collectionId = idOf(container.collection) + if (input.containerType === 'paper') { + await database.insert(paper).values({ id, sheetId: collectionId, sublabel: input.sublabel?.trim() || null }) + } else { + // The plan checked that a micronix tube has a barcode. + const barcode = 'barcode' in input ? input.barcode?.trim() || null : null + await database + .insert(GRID_CONTAINER_TABLES[input.containerType]) + .values({ id, collectionId, position: container.position, ...(barcode !== null && { barcode }) }) + } + entry.containerIds.push(id) + } + + return { entries, collectionsCreated } + } catch (error) { + const conflict = error instanceof Error ? sqliteConstraintConflict(error) : null + if (conflict) throw new SpecimenRegistrationError(409, [{ ...current, message: conflict }]) + throw error + } +} diff --git a/packages/api/src/routes/__tests__/imports.test.ts b/packages/api/src/routes/__tests__/imports.test.ts index d077e221..48c5cb05 100644 --- a/packages/api/src/routes/__tests__/imports.test.ts +++ b/packages/api/src/routes/__tests__/imports.test.ts @@ -15,6 +15,7 @@ import { import { studySubject, specimen, specimenTypeContainerType, containerTypeUnit, micronixPlate, cryovialBox, settings, storageContainer } from '../../db/schema' import { setContainerDefaults } from '../../lib/settings' import { eq } from 'drizzle-orm' +import { failInsertsWithUniqueViolation } from '../../__tests__/helpers/constraint-race' describe('Imports API', () => { let ctx: AuthenticatedRouteTestContext @@ -366,6 +367,24 @@ describe('Imports API', () => { expect(created).toBeDefined() expect(created!.unitId).toBe(testUnit.id) }) + it('returns 409 with every error when a full_file write hits a constraint', async () => { + await createTestStudy(ctx.db, { title: 'Race Study', shortCode: 'RACE' }) + await createTestSpecimenType(ctx.db, { name: 'Whole Blood' }) + failInsertsWithUniqueViolation(ctx.sqlite, 'specimen') + + const res = await ctx.request('/api/imports/bulk-combined', { + method: 'POST', + json: { + studyShortCode: 'RACE', + atomicMode: 'full_file', + subjects: [{ subjectName: 'SUBJ1', specimens: [{ specimenTypeName: 'Whole Blood', collectionDate: '2025-01-01' }] }], + }, + }) + + expect(res.status).toBe(409) + const body = (await res.json()) as { details?: { errors?: unknown[] } } + expect(body.details?.errors).toEqual([expect.objectContaining({ subjectIndex: 0, specimenIndex: 0 })]) + }) }) describe('POST /api/imports/bulk-combined/validate', () => { diff --git a/packages/api/src/routes/__tests__/specimens.test.ts b/packages/api/src/routes/__tests__/specimens.test.ts index 8406b3f4..b13fc9fb 100644 --- a/packages/api/src/routes/__tests__/specimens.test.ts +++ b/packages/api/src/routes/__tests__/specimens.test.ts @@ -30,6 +30,7 @@ import { paper, } from '../../db/schema' import { eq } from 'drizzle-orm' +import { failInsertsWithUniqueViolation } from '../../__tests__/helpers/constraint-race' describe('Specimens API', () => { let ctx: AuthenticatedRouteTestContext @@ -251,6 +252,56 @@ describe('Specimens API', () => { const afterSpecimens = await ctx.db.select({ id: specimen.id }).from(specimen) expect(afterSpecimens.length).toBe(beforeSpecimens.length) }) + + it('POST /api/specimens returns 409 when the container write hits a constraint', async () => { + failInsertsWithUniqueViolation(ctx.sqlite, 'micronix_tube') + + const res = await ctx.request('/api/specimens', { + method: 'POST', + json: { + sourceType: 'subject', + studyShortCode: 'BLK01', + subjectName: 'BULK-SUBJ', + specimenTypeName: 'Whole Blood', + collectionDate: '2024-06-03', + container: { + containerType: 'micronix_tube', + barcode: 'RACE-001', + collection: { type: 'micronix_plate', name: 'BULK-PLATE', position: 'C01' }, + }, + }, + }) + + expect(res.status).toBe(409) + expect(await ctx.db.select().from(specimen).where(eq(specimen.collectionDate, '2024-06-03'))).toEqual([]) + }) + + it('POST /api/specimens/bulk returns 409 when the container write hits a constraint', async () => { + failInsertsWithUniqueViolation(ctx.sqlite, 'micronix_tube') + + const res = await ctx.request('/api/specimens/bulk', { + method: 'POST', + json: { + specimens: [ + { + sourceType: 'subject', + studyShortCode: 'BLK01', + subjectName: 'BULK-SUBJ', + specimenTypeName: 'Whole Blood', + collectionDate: '2024-06-04', + container: { + containerType: 'micronix_tube', + barcode: 'RACE-002', + collection: { type: 'micronix_plate', name: 'BULK-PLATE', position: 'C02' }, + }, + }, + ], + }, + }) + + expect(res.status).toBe(409) + expect(await ctx.db.select().from(specimen).where(eq(specimen.collectionDate, '2024-06-04'))).toEqual([]) + }) }) describe('POST /api/specimens/bulk/validate', () => { @@ -422,6 +473,38 @@ describe('Specimens API', () => { expect(tubes[0].position).toBe('A01') }) + it('returns 409 when the container write hits a constraint', async () => { + failInsertsWithUniqueViolation(ctx.sqlite, 'micronix_tube') + + const res = await ctx.request(`/api/specimens/${testSpecimen.id}/containers`, { + method: 'POST', + json: { + containerType: 'micronix_tube', + barcode: 'RACE-TUBE-001', + collection: { type: 'micronix_plate', name: 'ADD-PLATE', position: 'A01' }, + }, + }) + + expect(res.status).toBe(409) + expect(await ctx.db.select().from(storageContainer).where(eq(storageContainer.specimenId, testSpecimen.id))).toEqual([]) + }) + + it('rejects a barcode on a paper container at the schema (ADR 0006)', async () => { + const res = await ctx.request(`/api/specimens/${testSpecimen.id}/containers`, { + method: 'POST', + json: { + containerType: 'paper', + barcode: 'LEGACY', + sublabel: 'Spot-1', + collection: { type: 'sheet', name: 'Sheet-1', parent: { type: 'box', name: 'PaperBox', locationId: 1 } }, + }, + }) + + expect(res.status).toBe(400) + expect(JSON.stringify(await res.json())).toContain('not barcode') + expect(await ctx.db.select().from(storageContainer).where(eq(storageContainer.specimenId, testSpecimen.id))).toEqual([]) + }) + it('accepts unitId and quantities from container defaults (web payload)', async () => { const [uL] = await ctx.db.select().from(unit).where(eq(unit.symbol, 'uL')) const res = await ctx.request(`/api/specimens/${testSpecimen.id}/containers`, { diff --git a/packages/api/src/routes/imports.ts b/packages/api/src/routes/imports.ts index e9584de6..573ce14d 100644 --- a/packages/api/src/routes/imports.ts +++ b/packages/api/src/routes/imports.ts @@ -2,8 +2,7 @@ import { Hono } from 'hono' import { z } from 'zod' import type { Database } from '../db/client' import { importDerivationsFromCsv, validateDerivationsCsv, type BulkDerivationSettings } from '../lib/derivations-csv' -import { runBulkCombinedImport, type BulkCombinedContainerInput } from '../lib/bulk-combined-import' -import { validateBulkCombinedPayload } from '../lib/bulk-combined-validate' +import { runBulkCombinedImport, validateBulkCombinedPayload } from '../lib/bulk-combined-import' import { createAuthMiddleware, createMemberMiddleware } from '../middleware/auth' import { handleRouteError } from '../lib/error-handler' import { @@ -86,19 +85,7 @@ imports.post('/bulk-combined', memberMiddleware, async (c) => { return c.json({ error: 'No subjects provided' }, 400) } const user = c.get('user') - const result = await runBulkCombinedImport(database, { - studyShortCode: data.studyShortCode, - atomicMode: data.atomicMode, - subjects: data.subjects.map((s) => ({ - studyShortCode: s.studyShortCode, - subjectName: s.subjectName, - specimens: s.specimens.map((sp) => ({ - specimenTypeName: sp.specimenTypeName, - collectionDate: sp.collectionDate, - container: sp.container as BulkCombinedContainerInput | undefined, - })), - })), - }, user?.id) + const result = await runBulkCombinedImport(database, data, user?.id) return c.json({ summary: result.summary, results: result.results.map((r) => ({ @@ -125,20 +112,7 @@ imports.post('/bulk-combined/validate', memberMiddleware, async (c) => { if (data.subjects.length === 0) { return c.json({ error: 'No subjects provided' }, 400) } - const result = await validateBulkCombinedPayload(database, { - studyShortCode: data.studyShortCode, - atomicMode: data.atomicMode, - subjects: data.subjects.map((s) => ({ - studyShortCode: s.studyShortCode, - subjectName: s.subjectName, - specimens: s.specimens.map((sp) => ({ - specimenTypeName: sp.specimenTypeName, - collectionDate: sp.collectionDate, - container: sp.container as BulkCombinedContainerInput | undefined, - rowIndex: sp.rowIndex, - })), - })), - }) + const result = await validateBulkCombinedPayload(database, data) return c.json(result) } catch (error) { return handleRouteError(error, c) diff --git a/packages/api/src/routes/specimens.ts b/packages/api/src/routes/specimens.ts index 55c421b0..67715734 100644 --- a/packages/api/src/routes/specimens.ts +++ b/packages/api/src/routes/specimens.ts @@ -6,11 +6,8 @@ import { z } from 'zod' import { listSpecimens } from '../lib/specimens/specimen-read' import { validateSpecimenData } from '../lib/validation' import { createContainerForSpecimen, pickContainerQuantity } from '../lib/container-creation' -import { - toContainerWriteInput, - type BulkCombinedContainerInput, -} from '../lib/container-write-placement' -import { handleRouteError, NotFoundError, ValidationError } from '../lib/error-handler' +import type { RegistrationContainer } from '../lib/specimen-registration' +import { ConflictError, handleRouteError, NotFoundError, ValidationError } from '../lib/error-handler' import { containerSchema, containerSchemaWithLocation } from '../lib/schemas' import { createAuthMiddleware, createMemberMiddleware } from '../middleware/auth' import { utcNow } from '../lib/datetime' @@ -113,10 +110,10 @@ specimens.post('/:id/containers', memberMiddleware, async (c) => { if (parsed == null) { throw new ValidationError('Container is required') } - const container = parsed as BulkCombinedContainerInput + const container = parsed as RegistrationContainer const user = c.get('user') - const result = await createContainerForSpecimen(id, toContainerWriteInput(container), dbInstance, { + const result = await createContainerForSpecimen(id, container, dbInstance, { userId: user?.id, quantity: pickContainerQuantity(container), }) @@ -190,9 +187,8 @@ specimens.post('/', memberMiddleware, async (c) => { if (data.container) { try { - const container = data.container as BulkCombinedContainerInput - const writeInput = toContainerWriteInput(container) - const result = await createContainerForSpecimen(newSpecimen.id, writeInput, dbInstance, { + const container = data.container as RegistrationContainer + const result = await createContainerForSpecimen(newSpecimen.id, container, dbInstance, { userId: user?.id, quantity: pickContainerQuantity(container), }) @@ -205,7 +201,8 @@ specimens.post('/', memberMiddleware, async (c) => { } catch (err) { await dbInstance.delete(storageContainer).where(eq(storageContainer.specimenId, newSpecimen.id)) await dbInstance.delete(specimen).where(eq(specimen.id, newSpecimen.id)) - throw err instanceof ValidationError ? err : new ValidationError(err instanceof Error ? err.message : 'Failed to create container') + if (err instanceof ValidationError || err instanceof ConflictError) throw err + throw new ValidationError(err instanceof Error ? err.message : 'Failed to create container') } } diff --git a/packages/api/src/routes/subjects.ts b/packages/api/src/routes/subjects.ts index e8d8c7e9..67dd5444 100644 --- a/packages/api/src/routes/subjects.ts +++ b/packages/api/src/routes/subjects.ts @@ -23,7 +23,6 @@ import { requireParam } from '../lib/common-validators' import { runOneSubjectWithSpecimens, formatOneSubjectWithSpecimensResponse, - type BulkCombinedContainerInput, } from '../lib/bulk-combined-import' import { withSpecimensRequestSchema } from '../lib/schemas' import { mergeSubjects } from '../lib/subjects/merge' @@ -335,19 +334,7 @@ subjects.post('/with-specimens', memberMiddleware, async (c) => { const body = await c.req.json() const data = withSpecimensRequestSchema.parse(body) const user = c.get('user') - const result = await runOneSubjectWithSpecimens( - dbInstance, - { - studyShortCode: data.studyShortCode, - subjectName: data.subjectName, - specimens: data.specimens.map((sp) => ({ - specimenTypeName: sp.specimenTypeName, - collectionDate: sp.collectionDate, - container: sp.container as BulkCombinedContainerInput | undefined, - })), - }, - user?.id - ) + const result = await runOneSubjectWithSpecimens(dbInstance, data, user?.id) return c.json(formatOneSubjectWithSpecimensResponse(result), 201) } catch (error) { if (error instanceof z.ZodError) {