Joist 2.4 adds em.query and em.execute for the SQL-shaped queries and bulk writes that don't fit em.find and entity-based updates, plus lazy fields, pipelining on by default, and aligned soft-delete behavior.
This post explains pruning, a novel feature of the em.find query DSL in Joist, an entity-based ORM for TypeScript.
Specifically, we show how ergonomic pruning is for “semi-dynamic” endpoints that return the same kind of data on every request, such as GET /authors, but vary their filters, joins, and WHERE clauses according to the request parameters.
A quick example of Joist’s query DSL for GET /authors is:
// Supports multiple parameters, for example:
// - `GET /authors?firstName=a1`
// - `GET /authors?publisherId=1`
// - `GET /authors?publisherName=p1`
asyncfunctiongetAuthors(params: {
firstName?:string;
publisherId?:string;
publisherName?:string;
}):Promise<Author[]> {
// Each variable is `string | undefined`, depending on what is sent
A key feature of any ORM, whether it uses a Knex-style query builder or a Joist-style domain model, is its query API: the DSL (domain-specific language) used to find rows in the database.
An ORM’s query DSL is usually one of its defining features. For query-builder libraries such as Knex, Drizzle, and Kysely, it is essentially the entire product, so query DSLs are a common area of exploration and innovation that can make or break the ergonomics of the library.
These DSLs can be as simple as “slightly decorated” SQL strings in postgres.js:
const rows = await sql`SELECT * FROM users WHERE name = ${name}`;
They can also use fluent, method-chaining APIs such as Knex, Kysely, or Drizzle:
Each ORM generally tries to provide “the most idiomatic query DSL possible,” but this can become a contentious topic because the interpretation of “idiomatic” depends on:
What “looks idiomatic” in the host programming language
That is, an idiomatic TypeScript DSL differs from an idiomatic Java DSL, which in turn differs from an idiomatic Go DSL.
In my opinion, this is why Prisma’s circa-2021 experiment with a Rust-based engine shared by TypeScript and Go clients was quizzical from the start: Go and TypeScript differ sharply in taste and style.
What “looks idiomatic” depends on where the ORM sits on the “driver => query builder => domain model” spectrum.
That is, what looks idiomatic for a low-level node-postgres or postgres.js driver differs from what looks idiomatic for a higher-level Active Record or entity-based ORM such as MikroORM or Joist.
What “looks idiomatic” is fundamentally a personal judgment call.
(…except on Reddit, where the only correct answer is “raw SQL.” 💀)
That disclaimer aside, we’ll argue that Joist is idiomatic for an ORM at the domain-model end of the spectrum by examining two query patterns:
Static queries: the query shape is always exactly the same, differing only in parameter values.
Dynamic queries: the query shape varies, with different relationship paths and conditions.
Here, “shape” means which columns and conditions the query uses and which tables or relationship paths it traverses.
For brevity, these signatures use Book and Author throughout the post. postgres.js, Knex, Drizzle, and Prisma return row or record objects; MikroORM and Joist return hydrated entity instances.
MikroORM and Joist take a fundamentally different approach: the query condition is a POJO literal. This is more succinct but also more limited because their em.find APIs return entities rather than arbitrary projected row shapes.
For these initial static-query examples, in my opinion the queries are all fairly simple/standard and could each be considered idiomatic in their own way.
We include a dummy 1 = 1 so that all our conditional predicates can always use an AND prefix.
Without this workaround, we would have to track which predicate comes first so that only subsequent predicates receive an AND prefix, which would be even more tedious. Disclaimer: this workaround was ChatGPT’s idea. 😅
Each conditional join check needs to know not only whether filters from its table are in use (if bookStatus is used, include books) but also whether filters from downstream tables are in use (if highRating is used, include books so that the book_reviews join works).
Next is Knex, which should be representative of method-chaining-style query builders:
Next is Prisma, using its typed relation filters. This version never passes an explicit undefined, so it also works with Prisma’s strictUndefinedChecks option:
Joist’s insight is that, for most dynamic-query use cases, the maximal structure of the query remains essentially static, just like the static queries in the previous section, and then individual parts within that structure are turned on or off according to the input.
Several of the other query DSLs can omit individual “leaf” predicates that are undefined (Drizzle supports undefined conditions, Prisma omits them by default or uses Prisma.skip, and MikroORM can opt into similar behavior). However, these DSLs generally do not remove both unused conditions and now-unused relationship paths from one stable nested query shape, so application code still handles that boilerplate with if statements, ternaries, or conditional method chaining.
Joist instead determines which conditions and relationship paths to retain by tracking the usage of each one.
To do that, Joist leverages one of JavaScript’s core wat-isms: “Oh no, both null and undefined?!? 😱” For em.find:
Any condition with null, such as firstName: null, becomes IS NULL.
Any condition with undefined, such as firstName: undefined, is unused.
Any relationship path referenced only by unused conditions is also unused.
After dropping all unused conditions and relationship paths, Joist generates SQL that contains only the necessary predicates and traversals.
Here are some conditions that are pruned or retained based on their runtime values:
Joist
// Given two variables, e.g. from incoming query parameters
const firstName = undefined;
const publisherId = null;
// When this `em.find` call executes, each condition is...
So far we’ve looked at how Joist prunes its em.find “inline conditions,” which are predicates contained directly in the object literal passed to em.find; all of them become top-level AND conditions in the query.
For complex AND and OR expressions, Joist has a conditions property that accepts { and: [...] } and { or: [...] } structures. These also support pruning:
Joist
const [a, b] = aliases(Author, Book);
// Find authors matching either a.lastName or b.title.
await em.find(
Author,
// This second argument is the "join literal" that declares the maximal
// relationship structure rooted at the `authors` table.
{
// `as` binds the `authors` table to `a` for use in conditions below.
as: a,
// Inline conditions become top-level AND conditions.
firstName: { eq: "bob" }, // not pruned
// The books path remains because b.title is used below. Joist may
// implement this collection filter as a correlated EXISTS subquery.
books: { as: b },
},
// This third argument contains complex AND/OR expressions.
{
conditions: {
or: [
a.lastName.eq(undefined), // pruned
b.title.eq("foo"), // not pruned
],
},
},
);
// Here is the same query without inline comments.
await em.find(
Author,
{ as: a, firstName: { eq: "bob" }, books: { as: b } },
To show how succinct Joist queries are in practice, here are two queries copied from our production codebase.
Both are GraphQL query resolvers because our primary codebase is a GraphQL monolith, but the patterns apply equally to REST endpoints, gRPC services, and other APIs.
The first resolver implements query { items } with a filter; ItemFilter defines five optional fields that clients can combine freely:
// If tradePartnerIds was provided, filter by those IDs.
or: [
c.tradePartner.in(tradePartnerIds),
cco.tradePartner.in(tradePartnerIds),
bc.tradePartner.in(tradePartnerIds),
],
},
orderBy: { number: "ASC" },
},
);
},
};
This query is not a one-liner 😅, but it is still relatively clean: the join literal defines the static structure, and the or: [...] expression makes tradePartnerIds conditional without cluttering the code.
It is also worth calling out the easy-to-miss ...others spread, which sends the remaining directly mapped GraphQL filters to Joist for default handling.
This highlights a key Joist principle: most API fields map directly to database fields. Therefore:
Common one-to-one cases should be succinct and boilerplate-free.
One-off cases should remain possible without fighting the framework.
Joist’s em.find DSL has one major limitation: it always returns entities, such as Author or Book, rather than arbitrary projected row shapes.
This means it does not currently support arbitrary SQL queries that use features such as SUM and GROUP BY to return arbitrary POJOs.
This does not mean that applications cannot use those SQL features–we of course have a small percentage queries that need them, and currently we just use a lower-level query builder such as Knex for those queries.
I.e. we’re not dogmatic about “Joist must do everything” (…but maybe someday 😅🤞).
Historically, this has just been a pragmatic/ROI decision: because of Joist’s focus on entities, literally 95% of our codebase’s queries are all em.finds (yes we counted 🤣), and so we’ve not yet prioritized a low-level SQL builder.
See Issue #188 for tracking that idea, which we’re optimistic will be able to leverage the same core alias and usage-tracking/pruning features of the em.find API.
Personally, I feel like “building dynamic queries” is often overlooked when ORM authors design query DSLs, almost as if they were an afterthought.
My admittedly anecdotal evidence for this is:
Dynamic queries are at best boilerplately in most ORMs, and downright painful in a few.
The postgres.js, Knex, Drizzle, Prisma, and MikroORM examples all show how dynamic queries can expand query-construction code with boilerplate, particularly around conditional relationship paths/joins.
To me, this is a tell that the scenario was not treated as a first-class problem when the DSL was designed.
ORM tutorials and overviews often omit examples of how to add three or four relationship paths conditionally, relegating the topic to FAQs or leaving users to work it out themselves.
Granted, Joist might have gotten lucky in knowing “this is a problem to solve” because idiomatic GraphQL queries frequently have optional filters, which made this an acute pain point for us while developing our primary codebase.
But we also took our time designing the em.find API, refining it over several years of day-to-day/in-the-weeds feature development, before ending up at its current form–which we’re quite happy with! 😀
Joist 2.3 is out, headlined by Rails-style scopes, batched paginated finds, enum collections, native Node 26 Temporal support, smarter batched queries, and a large round of hot-path performance work.