Joist 2.2 Released!
Joist 2.2 is out, and we didn’t get around to a 2.1 release announcement 😬, so we’re going to cover both the 2.1 and 2.2 releases.
As usual, Joist releases are driven by the problems we’re solving in our day-to-day application work at Homebound, so this release has a mix of headline features, API cleanup, performance work, and correctness fixes.
The biggest changes are:
- Collection filters now use
EXISTSclauses, avoiding join fanoutem.findqueries - Joist’s Postgres driver now uses
node-pgdirectly instead of Knex - Soft-deleted rows can now be resurrected by
upsertandfindOrCreate - Recursive relations now support self-referential many-to-many tables
- Derived property APIs were renamed, with new async-capable properties
collection.set(...)now works without preloading the collection
Faster Collection Queries with EXISTS
Section titled “Faster Collection Queries with EXISTS”The biggest 2.2 change is #1743, which changes how Joist renders many collection filters.
Previously, a query like this:
// Find authorsawait em.find(Author, { // Where they have a "child book" with this title books: { title: "Domain Modeling" }, // And also a "child comment" with this text comments: { text: "great post" },});Would result in multiple LEFT JOINs into the books and comments tables, followed by DISTINCT to collapse duplicate Author rows.
That worked, but can create a lot of intermediate rows: if an author has 10 books and 20 comments, the database can end up considering 200 joined rows for that one author before deduping. Usually this fan out didn’t matter—until the combined rows got really big, at which point it could adversely affect performance on the database server.
Joist now renders collection filters as correlated EXISTS subqueries where possible:
WHERE EXISTS ( SELECT 1 FROM books b WHERE b.author_id = a.id AND b.title = 'Domain Modeling')AND EXISTS ( SELECT 1 FROM comments c WHERE c.parent_author_id = a.id AND c.text = 'great post')The original PR measured this from 1.4x to 10.9x faster than join-plus-distinct in benchmarked cases, with one match-all case reducing planner rows processed by up to 542x.
There are a few behavior changes worth calling out:
- Queries that still require multiple collection
LEFT JOINs (due to having complexWHEREconditions thatORacross both child collections in a single clause) may now throw unlessallowMultipleLeftJoins: trueis set. beforeFindplugins still see the “pre-EXISTS” AST — this lets the plugin stay focused on theLEFT JOINas the “logical child collection” and let Joist do anyEXISTSoptimizations after the fact
Goodbye Knex, Hello node-pg
Section titled “Goodbye Knex, Hello node-pg”The largest 2.1 breaking change is #1754: Joist’s PostgresDriver now uses node-pg directly instead of taking a Knex instance.
New setup looks like:
import pg from "pg";import { PostgresDriver } from "joist-orm";
const pool = new pg.Pool(config);const driver = new PostgresDriver(pool, { onQuery(sql) { console.log(sql); },});If you still want Knex for custom query building (i.e. starting with an em.find to build 90% of your query, and then customize the last 10%—a common pattern we use internally), it is still available via joist-orm/knex:
import { createKnex } from "joist-orm/knex";const knex = createKnex(pool);Soft-Delete Resurrection
Section titled “Soft-Delete Resurrection”Soft deletes are great until a unique constraint says “you already have a row for that” and your operation blows up.
Joist 2.2 adds “resurrection” support in #1802, with follow-up fixes in #1821 and #1822.
Now, when upsert or findOrCreate would create a new row, Joist first checks for a “matching” soft-deleted row. If one exists, Joist clears it’s deleted_at columns, and returns/updates that row instead:
const author = await em.upsert(Author, { ssn: "123-45-6789", firstName: "Jane", lastName: "Austen",});In this example, if a soft-deleted Author with ssn = "123-45-6789" already exists, Joist resurrects that existing row instead of trying to insert a new one.
For upsert, the resurrection identity comes from uniqueBy in the joist-config.json:
{ "entities": { "Author": { "uniqueBy": "ssn" }, "Book": { "uniqueBy": ["author", "title"] } }}Resurrection is intentionally treated as an update:
- The existing id is preserved
beforeUpdaterunsbeforeCreatedoes not run- Ordinary duplicate rows still fail as expected
Recursive Many-to-Many Relations
Section titled “Recursive Many-to-Many Relations”Joist has supported recursive relations for self-referential foreign keys, i.e. an employee.manager.get will also have a employee.managersRecursive.get backed by Postgres recursive CTE queries.
Joist 2.1 extends that support to self-referential many-to-many tables in #1739.
For example, if users can have many parent users through a join table, Joist can now generate recursive collections for the full ancestry/descendant graph:
// Given [u5] has parents [u4, u3]// And parent [u4] has parents [u2, u1]
// When accesing u5's parentsRecursiveconst u5 = await em.load(User, "u:5", "parentsRecursive");
// Then we see u4, u3, u2, 1u5.parentsRecursive.get;// [u4, u4, u2, u1]Recursive m2m collections support:
- Batched loading for multiple seed entities.
- Respect in-memory, unflushed changes.
- Cycle detection.
- Reactive hints, added in #1744.
Cycle handling was also cleaned up and documented in #1745, #1746, and #1747:
config.addCycleRule( "parentsRecursive", (u) => `User ${u.name} has a cycle in their parent hierarchy`,);Derived Property API Cleanup
Section titled “Derived Property API Cleanup”2.2 also cleans up Joist’s derived-property naming in #1789 and #1819, while adding new SQL-backed derived properties in #1781.
The old hasAsyncProperty API is now hasProperty:
class Author extends AuthorCodegen { readonly numberOfBooks = hasProperty("books", (a) => a.books.get.length);}The rationale is that properties like numberOfBooks are “just properties” because their lambda is synchronous, and the hasAsyncProperty term (their previous name) is better reserved for properties that use the async keyword for their lambda.
These new “actually async” hasAsyncProperty’s, with their lambda being async, means they can use async database queries to their calculate values:
class Publisher extends PublisherCodegen { readonly numberOfBookReviews = hasAsyncProperty((p) => p.em.findCount(BookReview, { book: { author: { publisher: p.id } } }), );}Similarly, ReactiveQueryField is now named AsyncReactiveField, for reactive fields whose recalculation is backed by a database query:
class Publisher extends PublisherCodegen { readonly numberOfBookReviews = hasAsyncReactiveField( "id", { authors: { books: "reviews" } }, (p) => p.em.findCount(BookReview, { book: { author: { publisher: p.id } } }), );}collection.set Works Unloaded
Section titled “collection.set Works Unloaded”In 2.1, set no longer requires a preloaded collection for either many-to-many (#1722) or one-to-many (#1725) relations.
This means you can replace a collection directly:
const book = await em.load(Book, "b:2");const [t4, t5] = await em.loadAll(Tag, ["t:4", "t:5"]);
book.tags.set([t4, t5]);await em.flush();Without first doing:
await book.tags.load();Joist tracks the intended final set, flushes the diff, composes later .add / .remove calls correctly, and still triggers reactive validation rules from the unloaded set.
For one-to-many relations, Joist also percolates the many-to-one side immediately for loaded or new children:
const publisher = await em.load(Publisher, "p:1");const [a2, a3] = await em.loadAll(Author, ["a:2", "a:3"]);
publisher.authors.set([a2, a3]);await em.flush();This is a small API change but big DX win: most set calls were already saying “make the final collection exactly this”, so requiring a pre-load was unnecessary noise.
EntityManager.pendingChanges
Section titled “EntityManager.pendingChanges”2.1 adds a new debugging/tooling API in #1748: em.pendingChanges.
It exposes a high-level snapshot of the unflushed work in the current unit of work:
const a = newAuthor(em);
em.pendingChanges;// [{ kind: "create", entity: a }]It also includes relation changes:
a.tags.add(t);
em.pendingChanges;// [{ kind: "m2m", op: "add", joinTableName: "authors_to_tags", entities: [a, t] }]This is especially useful for tests, debugging tools, audit-style logic, or any code that wants to inspect what flush() is about to do without poking at internal state.
Markdown-Backed Entity Docs
Section titled “Markdown-Backed Entity Docs”#1749 adds codegen support for syncing entity documentation between Markdown files and generated TypeScript docs.
The new config options are:
{ "docs": true, "outputDocs": true}With this enabled, entity docs can live in Markdown:
## Overview
The Author entity represents a writer who can publish books.
## Fields
### numberOfBooks
The number of books this author has written.And codegen can sync them into JSDoc and emit runtime metadata:
export const docs = { Author: { comment: "The Author entity represents a writer who can publish books.", fields: { numberOfBooks: "The number of books this author has written.", }, },} as const;This should make it easier to keep entity docs readable in-repo while still making them available to GraphQL, admin UIs, or other runtime tooling.
Smaller But Notable Changes
Section titled “Smaller But Notable Changes”A few other changes are worth calling out:
- #1762 adds
isaliases in test factories, so tests can assert stable factory ids and reference in-flight entities by factory id. - #1718 fixes
findOrCreatesupport fornullvalues. - #1758 removes the implicit
entityLimitandDISTINCTfromfindIds. - #1784 and #1786 make m2m ordering deterministic across loading paths and processes.
- #1798 avoids stack overflows in
BatchLoader.loadAllwith very large key arrays. - #1820 adds plugin hooks around validation:
beforeValidateandafterValidate. - #1825 runs default values in Joist’s trusted context, so plugins/field logging don’t treat default initialization as user writes.
There are also many correctness fixes in reaction handling, load hints, subtype matching, stale preload caches, soft-delete joins, docs formatting, and dependency bumps across both release trains.
Upgrading
Section titled “Upgrading”The main migration notes are:
- Update
PostgresDriversetup to pass apg.Poolinstead of Knex. - Rename old
hasAsyncPropertyusages tohasProperty(codemod). - Rename
ReactiveQueryFieldusages toAsyncReactiveField(codemod). - Review queries that intentionally rely on multiple collection
LEFT JOINs; most should be faster automatically, but unusual cases may needallowMultipleLeftJoins: trueoroptimizeJoinsToExists: false.
Thanks to everyone filing issues, testing next releases, and pushing on Joist’s edge cases.