4.0: breaking changes and features #53

The plan for 4.0: breaking changes deferred from 3.x, where they would break working code (in 3.x these are warnings or opt-in options), and larger features.

Packaging

• Require knex ^3.3.0 as peer dependency, the tested version. 3.x requires >=2.3.0 <4 (Bound the knex peer dependency range #95).
• ESM-only package, vendored db-errors and vitest: done, but on the next branch, see Convert objection to an ES module package #140.
• Switch from npm to pnpm: packageManager field, pnpm-lock.yaml, onlyBuiltDependencies for sqlite3, CI with pnpm install --frozen-lockfile, scripts and contributing docs, and check that pnpm pack produces the same package (was Switch from npm to pnpm #136, PR Switch from npm to pnpm #201). With the old Node versions dropped, no workaround for pnpm's Node requirement is needed.
• Require MySQL 8.0 or newer, and remove the MySQL 5.7 code paths (limitPerParent without window functions) and its CI job (Drop MySQL 5.7 support in 4.0 #292), PR Drop support for MySQL 5.7 (#292) #294).
• Before 4.0 work lands on main: create a 3.x branch and deploy the docs from it, so the docs workflow on main can't deploy the 4.0 docs over the 3.x ones (was The docs workflow on main can still deploy the 4.0 docs #134).

Errors instead of 3.x warnings

• Throw when onConflict(), ignore() or merge() is used with insertGraph() / upsertGraph(). 3.x ignores the clause with a warning (InsertGraph does not work with onConflict Vincit/objection.js#2156, PR Throw on onConflict() with insertGraph() and upsertGraph() (#2156) #199).
• Throw on unknown graph options, e.g. noInset. 3.x warns (Validate graph options and reject unknown keys #84, PR Throw on unknown graph options (#84) #259).
• The relation expression parser throws for anything after * or ^, like 'a.^.b' or '*.a', in withGraphFetched() and graph options alike. 3.x silently drops the rest (Smaller graph issues: uuid case, recursive option expressions, #ref left in results #221), PR Throw for anything after * or ^ in relation expressions (#221) #304).
• Remove the deprecated skipUndefined() (PR Remove the deprecated skipUndefined() #200).
• Throw in insertGraph() with relate for new models below related models, which are ignored today. 3.9.0 warns (insertGraph() with relate silently drops new children below related nodes #217), PR Throw in insertGraph() for new models below related models (#217) #295).
• Throw for MySQL delete() with orderBy() and limit(): knex drops the order, so the limit deletes arbitrary rows. 3.9.0 warns (Smaller relation write issues: MySQL delete with order and limit, extras-only patch count, duplicate join rows #235, PR Throw for offset() in updates and deletes and for ordered, limited deletes on MySQL (#286, #235) #298).
• BelongsToOne unrelate() applies the relation's modify / filter also without a where clause, like HasMany and ManyToMany (BelongsToOne unrelate() ignores the relation's modify without a where clause #234, PR Apply the relation's modify to BelongsToOne unrelate() without a where clause (#234) #249), PR Apply the relation's modify to BelongsToOne unrelate() without a where clause (#234) #301).
• Owner models loaded from the database without their join property throw when a relation is fetched for them. Models built in memory, e.g. with fromJson(), silently get no related models. A hidden flag marks models built from database rows ($fetchGraph doesn't load relation if the foreign key column is not selected Vincit/objection.js#1832, Decide which 3.x warnings become errors in 4.0 #272, PR Throw for missing and array join properties when fetching relations (#1832, #2258, #114) #302).
• Related rows without their join property throw, naming the relation and the column ($fetchGraph returning empty array when it should return data Vincit/objection.js#2258, Decide which 3.x warnings become errors in 4.0 #272, PR Throw for missing and array join properties when fetching relations (#1832, #2258, #114) #302).
• Owner join properties holding an array of keys throw, pointing to a relation through a join table or the array-of-keys relation (Warn when a relation's owner key is an array #114, Solution for Fixing Binding Issues with Array in JSON Field in HasManyRelation Vincit/objection.js#2667, Decide which 3.x warnings become errors in 4.0 #272, PR Throw for missing and array join properties when fetching relations (#1832, #2258, #114) #302).
• JSON field expressions throw on the known databases that don't support them (SQL Server, Oracle, and MySQL / SQLite until the feature below lands), and keep warning on unknown clients (JSON field expressions and whereJson*() fail or silently misbehave on MySQL #113, JSON support for mysql and sqlite3 Vincit/objection.js#276, Decide which 3.x warnings become errors in 4.0 #272, PR Throw for JSON field expressions on MySQL, SQLite, SQL Server and Oracle (#113) #305).

Behaviour changes

• Don't snake-case JSON keys in field expressions by default. 3.x has an opt-in option (snakeCaseMappers works differently for objects and FieldExpressions Vincit/objection.js#1089, PR Preserve the JSON keys of field expressions in snakeCaseMappers() by default (#1089) #231).
• snakeCaseMappers() converts colon keys declared in jsonSchema.properties as a whole, so columns like ns:foo_bar round-trip with preserveJsonKeys (PR Preserve the JSON keys of field expressions in snakeCaseMappers() by default (#1089) #231), PR Convert declared keys with a colon as a whole in snakeCaseMappers() (#1089) #297).
• Give result in static hooks the value the caller gets: after* hooks run last, after first() and *AndFetch, with the real shape (model, array, number or undefined), and a returned value is used as is. before* hooks get undefined (result in static hook arguments is always an array (Vincit#1842) #66, StaticHookArguments#result is always an array Vincit/objection.js#1842, PR Pass static after hooks the result in the shape that the query returns (#66) #308).
• withGraphFetched() assigns related models after their static afterFind hook, so a returned value is used. withGraphJoined() calls the hook too, for changes in place: a returned value is ignored there and documented, with a tracking issue for using it. Do it together with the after* result change (withGraphFetched didn't return staticHook afterFind result Vincit/objection.js#1963, PRs Assign what the relation queries of withGraphFetched() return to the owners (#1963) #312, Call the afterFind hooks of the related model classes in withGraphJoined() (#1963) #313).
• $setRelated() / $appendRelated() keep foreign keys in sync: the key follows the relation, and becomes null when the relation is cleared, when the related model has no key yet, and for models removed from a has-many relation. Conflicting keys set explicitly are overwritten. fromJson() / $setJson() don't sync. Inserts through $relatedQuery() may set the key in memory before their relate query runs (Keep foreign keys in sync in $setRelated() and $appendRelated() (#2001) #100, Feature request: setRelated modifies associated id column Vincit/objection.js#2001, PR Keep foreign keys in sync in $setRelated() and $appendRelated() (#2001) #100).
• Don't treat aggregates (count(), sum(), …) as column selections in findSelection() / hasSelection(), which changes the columns root and withGraphFetched() queries add: aggregates match by their alias only, and the workarounds in RelationJoiner go (Aggregates in withGraphJoined() modifiers are treated as column selections (Vincit#2219) #111, withGraphJoined doesn't select root object's columns when using modifiers and more than 1 relation Vincit/objection.js#2219), PR Match aggregates only by their alias in selections (#111) #303).
• Apply limit() / offset() / first() in graph modifiers per parent by default. 3.x has the opt-in limitPerParent graph option (limit()/first() in graph modifiers apply to all parents instead of per parent (Vincit#429) #127, modifyEager#builder.limit for eager limits entire N+1 Vincit/objection.js#429, PR Apply limit(), offset() and first() in graph modifiers per parent by default (#127) #229).
• With limitPerParent, find out whether a relation query has a limit without building it twice, so its modify and onBuild hooks run once (PR Apply limit(), offset() and first() in graph modifiers per parent by default (#127) #229), PR Drop support for MySQL 5.7 (#292) #294).
• Run the graph operations of insertGraph() / upsertGraph() one after another on MySQL (upsertGraph() without a transaction can deadlock on MySQL 8 #135) → not needed: the deadlock was fixed in 3.6.0 (patches by id), and the docs already say graph writes aren't atomic without a transaction.
• Throw for relation expressions that name a relation twice, like '[movies.reviews, movies.actors]', instead of silently using only the last one: in withGraphFetched() / withGraphJoined() and other relation expressions (3.x warns), and in graph options (Accept relation expressions for graph options like noInsert #131, PR Throw for relation expressions that name a relation twice (#131) #251).
• update() rejects field expression keys like 'meta:color', so it always replaces whole values. 3.x writes them with jsonb_set and validates them like a patch (update() with field expressions: validation is wrong, a column default can wipe the JSON column #160, PR Reject field expression keys in update() (#160) #230).
• insert() rejects field expression keys with the same ValidationError as update() (PR Reject field expression keys in update() (#160) #230), PR Reject field expression keys in insert() (#160) #296).
• asArray() returns [] for undefined, and the result of static hooks is [] where it was [null] (asArray(): return [] for undefined #129, PR Return an empty array from asArray() for undefined (#129) #210).
• upsertGraph() always clears the owner's foreign key before deleting a BelongsToOneRelation row, like unrelate(), whatever the owner's schema says. 3.9.0 only does it when the schema allows null (PR Set nullable foreign keys to null before upsertGraph() deletes the rows of BelongsToOneRelations (#212) #250) and otherwise deletes first, leaving the reference to the database. Schemas or columns that don't allow null then throw instead of cascading the owner away; the schema check of PR Set nullable foreign keys to null before upsertGraph() deletes the rows of BelongsToOneRelations (#212) #250 goes (upsertGraph() removing a BelongsToOne model deletes the row before clearing the owner's foreign key #212), PR Always clear the foreign key before upsertGraph() deletes the row of a BelongsToOneRelation (#212) #299).
• upsertGraph() ignores a #unrelate / #delete marker on a HasOneRelation / HasOneThroughRelation model that isn't the current one and keeps the current model, like BelongsToOneRelation since 3.9.0. Arrays of to-many relations still replace the list (Markers on a HasOne model that isn't the current one delete the current model #261), PR Keep the current HasOneRelation model in upsertGraph() when the graph ignores a different model (#261) #300).
• upsertGraph() runs has-many relates before deletes, so rows moved in a tree with ON DELETE CASCADE, or up a level while their old parent is deleted, aren't deleted with their old parent or fail. Rows moved below related models are kept since 3.9.0 (PR Don't delete rows that upsertGraph() moves below related models (#214) #278). Unique foreign keys replaced in the same graph (e.g. HasOneRelation) must keep working (upsertGraph() deletes rows that the same graph relates to another owner #214). Design this as one order of the graph operations, and fold the separate step that deletes replaced BelongsToOneRelation rows after the patches (PR Delete replaced BelongsToOneRelation rows in upsertGraph() after pointing their owners to the new rows #228) into it. (PRs Plan the actions of upsertGraph() before the first write (#214) #307, Relate moved rows in upsertGraph() before deleting the removed rows (#214) #309, Throw in upsertGraph() when a relate or update finds no row (#214) #310)
• findById() and findByIds() put their id condition in its own where group, like $query() and the *ById() writes, so findById(1).patch(…).orWhere(b) can't write other rows. "Id or slug" lookups become where('id', x).orWhere('slug', x) (deleteById() and findById().patch()/delete() let a user's orWhere() write other rows #232, Decide whether findById() keeps its id condition in its own where group #271, PR Keep the id condition of findById() and findByIds() in its own where group (#271) #315).
• Fetch the related ids of many-to-many patches with join table extras in one form on all databases: select the distinct ids from the filter query, and pick the related rows for limit() on MySQL and SQL Server without the outer id in (…) query, which needs a second form today (PR Select the related ids of many-to-many patches with join table extras from their filter query (#161) #266, PR Pick the related ids of limited many-to-many patches from their filter query (#161) #322).
• Leave JSON attributes as the Postgres drivers (pg, pgnative, cockroachdb) return them, so JSON-looking strings stay strings; other drivers keep the 3.x parsing. static jsonColumnFormat = 'auto' | 'text' | 'parsed' overrides it, e.g. 'text' for MariaDB or JSON in Postgres text columns (Postgres: JSON columns holding a JSON-looking string are parsed twice #205, PR Leave JSON attributes as the Postgres drivers return them, and add jsonColumnFormat (#205) #321).

Typings

Strict mode, narrowing after select() and the rebind slot change how builder types carry the model, result and column scope, so they should be designed together. Details in #16.

• Strict mode, on by default, with TypeConfig { strict: false } as the opt-out (prototype Prototype opt-in strict typings with column scopes and FromSchema #94). Before 4.0 ships, fix the gaps that cause false errors: snakeCaseMappers(), generic builders followed by withGraph*(), and column scopes that depend on call order. Column names in string variables need ref() or a cast. FromSchema works in both modes. Measure the check time on a real codebase before the release. Done in PR Check the columns and values of queries against the models in the typings (#16) #327: columns checked against the relation graph, strict at 1.1–1.4× loose check time. Generic code followed by withGraph*() needs ref() for column methods (accepted for 4.0).
• Require TypeScript 5.8 or newer (5.8, 6 and 7), and run the typings tests on 5.8 and 7 in CI (PR Require TypeScript 5.8 or newer #293). The strict tests join the 5.8 check when strict mode lands.
• Narrowing after select() (PR Narrow the result rows of select() and the aggregates in the typings (#16) #329).
• The rebind slot for custom query builders, replacing the five *QueryBuilderType members, which are removed (draft Add a rebind slot for custom query builder types (#319) #125). Designed together with strict mode, as one way builder types carry the model, result and column scope. The check-time measurement covers both; if the combined cost is too high, keep the five members instead (PR Declare custom query builder types through a single RebindType member (#16) #317).
• Profile the typings and fix the hotspots: about 30–40% faster checks (PR Speed up type checking of query builders #276). QueryBuilder<out M> is left to the strict mode design.
• $relatedQuery() / relatedQuery() of one-relations resolve to M | undefined (TypeScript return type from $relatedQuery is not optional for optional relation Vincit/objection.js#2355), and toJSON() returns a recursive JSON type, so relations aren't typed as model instances (ModelObject with relations Vincit/objection.js#2380) (PRs Type $relatedQuery() of to-one relations as possibly undefined (#2355) #330, Type the result of toJSON() as the model's JSON, with related models as JSON (#2380) #331).
• withGraph narrowing: type to-one relations as T | null → kept as in 3.x: null only where the model declares the relation nullable (parent?: Person | null), as otherwise narrowed results no longer fit their declared model type.
• Type the rows of relate().returning() as Pojo[] instead of any[]. Later, once typed relation mappings are common: precise row types per relation kind (related rows for HasMany, the owner row for BelongsToOne, join rows for ManyToMany) (PR Type the rows of relate().returning() as plain objects (#53) #332).
• Reject relation data in the insert() / patch() types, as both ignore it (PR Reject relation data in the types of insert(), patch() and update() (#53) #320).
• Remove the deprecated allowMarkers from InsertGraphOptions. insertGraph() rejects markers since 3.8.0 (PR Remove the deprecated allowMarkers from InsertGraphOptions #208).

Features

• A relation through an array of keys held by the owner, e.g. tagIds = [1, 4, 7], as its own relation class; read-only first is an option ( Solution for Fixing Binding Issues with Array in JSON Field in HasManyRelation Vincit/objection.js#2667, from Upstream backlog: features for later and reports needing a repro #103). Done as BelongsToManyRelation (PRs Add BelongsToManyRelation for owners that hold an array of keys (#2667) #319, Support relate(), unrelate(), insert(), delete() and graph writes for BelongsToManyRelation (#2667) #323).
• JSON field expressions, ref() and the whereJson* methods for MySQL and SQLite: objection renders the SQL per database, SQL Server throws (JSON support for mysql and sqlite3 Vincit/objection.js#276, from Upstream backlog: features for later and reports needing a repro #103, PRs Render the JSON field expressions of ref() for MySQL and SQLite (#276) #324–Render the whereJson*() methods for MySQL and SQLite (#276) #326).
• modifyJoin(relationPath, (join, { owner, related, joinTable }) => …) to adjust the ON clause in joinRelated() and withGraphJoined(), like modifyGraph() (Add a way to modify the "on clause" when using joinRelated. Vincit/objection.js#1975, from Upstream backlog: features for later and reports needing a repro #103, PR Add modifyJoin() to modify the ON clause of relation joins (#1975) #314).