Partial Update APIs
Joist has built-in support for building "partial update"-style APIs on top of your domain model.
Partial update APIs, whether they are implemented over REST or GraphQL or GRPC, typically follow the conventions of:
- A create or update can include only a subset of fields, and any field not included in the subset is left as-is
- An update can use
null
as a marker to mean "unset this field" - Updating a parent's collection of children can be done incrementally, i.e. w/o knowing the full set of children
Joist has dedicated EntityManager.createOrUpdatePartial
and Entity.setPartial
APIs to help implement APIs that follow these conventions with as little boilerplate as possible.
These methods are particularly useful for implementing GraphQL APIs, where Joist's normal EntityManager.create
and Entity.set
methods have TypeScript types that are too strict for GraphQL's looser/less strict type system.
For example, a SaveAuthorInput
firstName
that is generated by graphql-code-generator might be typed as firstName: string | null | undefined
which does not match Joist's more idiomatic Author.firstName: string | undefined
typing.
Unsetting Fields with null
Marker
A common pattern for partial-update style APIs is to treat null
and undefined
differently, i.e. { firstName: null }
specifically means "unset the firstName
property", while firstName
being not present (i.e. either { firstName: undefined }
or an empty {}
) means "do not change firstName
".
As mentioned, in a GraphQL input type this might be typed as SaveAuthorInput
having a firstName: string | null | undefined
field.
Normally the null
-ness of input's firstName
will cause issues with Joist's "preferred undefined
" convention:
// This is typed as string | null | undefined
const { firstName } = saveAuthorInput;
const author = em.load(Author, "1");
// The normal `.set` will not compile
author.set({ firstName });
}
Instead, Joist provides a setPartial
that allows code to opt-in to the partial-semantic behavior and an RPC layer's potentially-less-strict typing:
// This is typed as string | null | undefined
const { firstName } = saveAuthorInput;
const author = em.load(Author, "1");
// Compiles successfully
author.setPartial({ firstName });
}
Specifically, the semantics of Entity.setPartial
is that:
- For a required field
firstName
:{ firstName: "foo" }
will updatefirstName
{ firstName: undefined }
will do nothing{ firstName: null }
will cause a validation error b/cfirstName
is required and cannot benull
- For an optional field
lastName
:{ lastName: "bar" }
will updatelastName
{ lastName: undefined }
will do nothing{ lastName: null }
will unsetlastName
(i.e. set it asundefined
)
- For collections like
books
:{ books: [b1] }
will set the collection to justb1
{ books: null }
will set the collection to[]
{ books: undefined }
will do nothing
The EntityManager.createPartial
and EntityManager.createOrUpdatePartial
methods both have these semantics as well.
Arguably, the ideal partial-update type for Author
in this scenario would be:
interface SaveAuthorInput {
firstName: string | undefined;
lastName: string | null | undefined;
}
But sometimes it's a challenge to get the RPC framework, e.g. GraphQL in this instance, to generate a type that exactly matches this, which is why Joist offers these more flexible setPartial
methods.
Saving Parents with Children
To save both a parent and multiple potentially-new-or-existing children, Joist provides EntityManager.createOrUpdatePartial
.
An example usage is:
// Given some RPC types (in this instance GraphQL)
interface SaveAuthorInput {
id?: string | null;
firstName?: string | null;
books?: SaveBookInput[];
}
interface SaveBookInput {
id?: string | null;
title?: string | null;
}
// When the client's request comes in...
const input: SaveAuthorInput = {
// Updating author 1
id: "a:1",
// To have 3 books
books: [
// And the 1st book is new
{ title: "new book" },
// And the 2nd book already exists but has no changes
{ id: "b:1" },
// And the 3rd book already exists but has a new title
{ id: "b:2", title: "updated" },
]
}
// Then we can apply all of those changes via
// a single call
await em.createOrUpdatePartial(Author, input);
Admittedly, your RPC/GraphQL API convention for parent/children inputs has to fairly closely follow what Joist's own partial update / createOrUpdatePartial
convention, but assuming you do so, Joist can reduce a very large amount of CRUD boilerplate in an RPC/GraphQL API.
Unlike EntityManager.create
, which is synchronous, EntityManager.createOrUpdatePartial
is async and needs to be await
-d because it may require SQL calls to look up existing entities, e.g. the b:1
and b:2
IDs in the above example.
Joist's createOrUpdatePartial
behavior, while developed independently, is effectively similar Objection.js's upsertGraph
operation.
Incremental Collection Updates
Joist's default behavior for any collection set (e.g. Entity.set
, Entity.setPartial
, EntityManger.createOrUpdatePartial
, etc.) is for the collection to be exhaustively set to the new value, for example:
const author = newAuthor(em);
// Start out with two books
author.set({ books: [b1, b2] });
// Later set with a third book
author.set({ books: [b3] });
// Passes b/c `set` has implicitly removed the 1st two books
expect(author.books.get.length).toEqual(1);
However, when partially updating entities via an RPC call, it's often convenient to change only a single child of the collection, especially for APIs where the child itself doesn't have a dedicated operation (i.e. saving an invoice line item can only be done via the invoice API).
To support these APIs, setPartial
and createOrUpdatePartial
will both opt-in to incremental collection semantics if they detect an extra op
hint key on the children. For example:
const author = newAuthor(em);
// Start out with two books
author.set({ books: [b1, b2] });
// Later add a third book
author.setPartial({ books: [
// And include the `op` hint
{ op: "include", title: "b3" }
]});
// Passes b/c `setPartial` saw `op` and worked incrementally
expect(author.books.get.length).toEqual(3);
The valid values of an op
keyword are:
{ op: "delete", id: ... }
will remove the child andEntityManager.delete
it for hard deletion{ op: "remove", id: ... }
will remove the child but not callEntityManager.delete
{ op: "include", id: ... }
will include (i.e. add if needed, or just update if it already an existing child) the child
If any child has an op
key, then all children must have an op
key, to be the most clear/consistent to the caller that incremental semantics are being used.
The op
key is not an actual column stored in your database or domain model, i.e. the Book
entity should not have an op
field.
Instead, op
can be just an extra key on the RPC/GraphQL input types to specifically driven Joist's incremental collection semantics.
The key name op
was chosen for both succinctness and also low probably of overlapping with real fields in the domain model.
Because incremental semantics are enabled by the presence of an op
key, if the collection is empty it will fundamentally look like an exhaustive set that clears the collection.
// An empty collection will always clear the books, and can't be
// treated as a incremental operation
author.setPartial({ books: [] });
This means that if a client is trying to optimize its call by sending "only changed children", when no children have been changed then it should not include the child key in the payload at all, to avoid unintentionally clearing the collection.
Technically, Joist currently supports a 4th { op: "incremental" }
keyword that can act as a "fake child" and will be ignored (i.e. not treated as an entity to add/remove from the collection), but will still enable incremental semantics and so avoid the "empty list clears the collection" gotcha.
If you're working in React, the form-state library has built-in knowledge of Joist's op
keyword/incremental collection semantics and makes it easy to craft optimized/only-what-changed payloads with a single form.changedValue
call.
Legacy Incremental Collection Updates
Joist's initial version of incremental updates used separate delete
and remove
keys instead of op
:
const author = newAuthor(em);
// Start out with three books
author.set({ books: [b1, b2, b3] });
// Then delete b1, remove b2, leave b3 alone, and add b4
author.setPartial({ books: [
{ id: "b:1", delete: true },
{ id: "b:2", remove: true },
{ id: "b:4" },
]});
These are still supported, but are soft-deprecated and the op
key is preferred going forward.
We moved away from delete
and remove
keywords for two reasons:
- They are more likely to overlap with existing fields in the domain model, and
- It is ergonomically easier for our frontend UI to bind to an always-present
op
key, and just flip its value frominclude
/delete
when the user toggles adding/removing rows, instead of adding/removing entiredelete
/remove
keys.