Lazy Fields for 30x speedup without Decorators or Transforms
Joist is an ActiveRecord-style TypeScript ORM, such that we purposefully mimic how Rails declares relations, i.e. as class fields:
// simplified exampleclass Author { books = hasMany("books");}
These class fields are very ergonomic, i.e. no decorators (which we purposefully avoid or other boilerplate), so much so that at Homebound, we’ve written (or codegen-d) literally thousands of these class fields in our domain models.
Fields are Expensive at Scale
Section titled “Fields are Expensive at Scale”This approach has worked really well, with the only downside being that JavaScript’s class fields are eagerly initialized.
This means when new Author
is called, all the fields in class Author
, like books = hasMany(...)
above, and any others like publisher = hasOne(...)
, etc., are immediately created & allocated in memory.
These class fields are created & assigned every time, for every author instance, even if the code that called new Author
doesn’t end up using the books
relation (maybe it only uses publisher
).
This eager-ness is fine when Author
has a handful of relations, but in sufficiently-complicated domain models, some of our core entities have 30 relations, and the cost of creating 30 relations x
1,000 of rows can start to add up.
So our goal is to keep the books = hasMany("books")
syntax, but somehow make the fields lazy, so we only pay for what we use.
First Approach: Codegen
Section titled “First Approach: Codegen”Not surprisingly, our first solution was to leverage code generation.
For an application’s default, foreign-key-based relations like one-to-many, many-to-one, etc., Joist already generates a AuthorCodegen
class that has the boilerplate relations defined for free.
I.e. in Joist, you get getters & setters like author.firstName
, and relations like author.books
and publisher.author
all for free, from the generated AuthorCodegen
base class.
Because these ...Codegen
classes are not handwritten, it was easy to just swap the field-based code over to getters:
// in AuthorCodegen.ts, generated by joist-codegenclass AuthorCodegen { // lazy cache of relations #relations: Record<string, any> = {};
// Not a field anymore, but looks like one to callers get books(): OneToMany<Book> { return (this.#relations ??= hasMany("books")); }}
// in Author.ts, written by the userclass Author extends AuthorCodegen { // we get the lazy relations for free}
Because get books()
is a method in a JavaScript class
, the books
function is installed just once on the Author.prototype
, not each individual author
instance, and so when we create new Author
the only field to initialize is a single shared #relations
cache, and there “nothing to do” for books
—it doesn’t need an allocation, it’s just waiting to be called, on the prototype—it’s lazy.
This works great! We get lazy initialization, hasMany("books")
is only invoked when books
is accessed, and no one cares that the generated output is a little longer/more verbose.
However, Joist also has a robust set of user-defined relations, for rich domain modeling, i.e. things like ReactiveField:
// in Author.ts, written by the userclass Author extends AuthorCodegen { numberOfBooks = hasReactiveField(...);}
And because this line is fundamentally a user-/engineer-written relation, we couldn’t apply our usual codegen hammer.
Second Approach: AST Transformers
Section titled “Second Approach: AST Transformers”Given we want “this field…but rewritten into a getter”, that sounded like a good job for AST rewriting!
So our next approach was using ts-patch to run a custom transform during the tsc
compilation process that scanned the AST for class fields following our has...
convention, and rewrote them to getters.
So the source Author.ts
file would have the books = hasBooks
field syntax, but the tsc
emitted Author.js
file would have a get books() { ... }
getter, very similar to the AuthorCodegen
getters from the previous approach.
This was pretty good!
The pros/cons were:
- Pro: We achieved lazy initialization :tada:
- Pro: Engineers kept writing the ergonomic
relation = hasSome(...)
field syntax - Pro: No runtime overhead
- Con: Potentially different test vs. production behavior, if tests did not use
ts-jest
& invoke the transformer - Con: We’re tightly coupled to
ts-patch
andtsc
-driven output
This last con has been becoming more painful, as more alterative performance-improving solutions to “transpile/load TypeScript” have been popping up.
We’re fans of tsx, Bun, and Node’s built-in TypeScript support—all of which are amazingly fast, but all have spotty or missing transform support (likely on purpose, b/c transforms themselves would only slow things down), and so we’ve not been able to leverage these newer tools.
This had been fine, but as our majestic monolith codebase grows (and grows), we’re more & more motivated to switch to a faster build/load process—without giving up the lazy fields performance optimization.
Final Approach: JavaScript Prototypes
Section titled “Final Approach: JavaScript Prototypes”We spent quite awhile brainstorming how to “keep fields…but not have them initialized”—literally months of “here & there” thinking about it, since this was not our top priority, but always an itch, lurking, tingling, just waiting to be scratched.
We were willing to use proxies, or subtypes, or really any magic—but could not think of a way to avoid the hasMany
function in books = hasMany
from running when the constructor is called.
Stepping back, we established that, tactically, we need to achieve two things:
- Create an instance without the fields
- Move the relations onto the prototype
When articulated this way, a native-JavaScript solution starts to emerge, and even become obvious in retrospect.
Skipping field initializers with Object.create
Section titled “Skipping field initializers with Object.create”The insight to avoid field initializers is they happen “when the constructor is called”—what if we just don’t call the constructor? I.e. don’t call new Author
?
Ideally we’d like to create an empty author
instance (i.e. with no fields assigned, so no hasMany
functions invoked), but hooked up to the Author
class’s prototype, so it would still “quack like an Author”.
Turns out in JavaScript this is a one-liner:
const empty = Object.create(Author.prototype);
And it has all the behaviors we’re looking for:
console.log(empty instanceof Author); // true!console.log(empty.someGetterOnPrototype); // worksconsole.log(empty.someClassField); // undefined but expected
Granted, “don’t use new Author
” sounds like a weird approach, but serendipitously Joist already requires all entity creation to go through the EntityManager
, i.e.:
// Creating new authors must use em.createconst newAuthor = em.create(Author);// Loading existing authors must use em.load/em.findconst oldAuthor = await em.load(Author, "a:1");
So our codebases were already decoupled from the new
operator, and using em.create
/ em.load
instead, as Joist uses these em.create
& em.load
methods to precisely control (and optimize) the entity creation/lifecycle.
This was lucky! Our em.create
/ em.load
API meant we already had a single choke point to swap out an optimized Object.create
-based instantiation flow (instead of using new
), and have the entire codebase benefit, with very few/ideally zero changes.
So far this is almost too easy—we’ve got an empty author
, but of course author.books.load()
does not work yet (it was a field that we skipped), so how do we get that books
relation, and all the other relations, back?
Moving Relations to the Prototype
Section titled “Moving Relations to the Prototype”We need a caller accessing author.books
to still work, and ideally “find books” on the Author
’s prototype, which means something like:
- The JavaScript runtime looks for a
"books"
key defined directly on theauthor
instance, but does not find one (because we skipped thenew Author
constructor) - The JavaScript runtime looks up the prototype chain, for a
"books"
key, and findsAuthor.prototype["books"]
, and invokes that, withauthor
bound tothis
This flow would allow us to define books
“just once” on the Author.prototype
and have all author instances “get the books
key” basically for free: they would inherit the "books"
key from their prototype, without any extra per-author
instantiation costs.
Granted, different authors can have different books, so we don’t want Author.prototype.books
to be a literal Book[]
array of books—but we could define it as a function that, when called with this=author1
or this=author2
, returns the books for that specific author.
…which is essentially a getter (i.e. we’ve just described how getter methods in JavaScript classes work: they are just syntax sugar for “put this function on the prototype”).
So, how do we get this Author.prototype.books
getter defined?
Because everything in JavaScript is “just an object”, including prototypes, we can dynamically define keys on the prototype with just an Object.defineProperty
:
Object.defineProperty(Author.prototype, "books", { configurable: true, get(this: any) { return (this.relations["books"] ??= hasBooks()); },});
So easy!
We just need to call this defineProperty
for author.books
, author.publisher
, and all other relations on Author
—but how do we know what those relations are?
Probing for Relation Metadata
Section titled “Probing for Relation Metadata”We want to figure out “what are the fields on Author
that we should move to the prototype”? Ideally just at runtime, i.e. without any AST parsing.
Who can tell us what the fields are? …the constructor!
This is ironic, because we’ve been trying so hard to avoid the constructor (avoid calling new
)—but if we just call it once, during boot, we can use it as a one-time “probe” to discover the fields, and then never call it again.
We end up with a process like:
- When Joist boots, create a single “fake”
new Author
to let the constructor assign fields likebooks = hasMany(...)
during the traditional object instantiation process - Teach
hasBooks(...)
to return aLazyField
marker/wrapper that isn’t the true relation (it cannot actually hold/fetchBook
s), but instead a lazy version that lets us:- a) identify “this
author.books
value is a lazy field”, and - b) later ask it to create the live relation, only when needed/accessed.
- a) identify “this
- For every
Object.entries(author)
wherevalue intanceof LazyField
, do anObject.defineProperties
to “move” that field to the prototype, and calllazyField.create()
when lazily accessed:
Object.defineProperty(Author.prototype, lazyField.fieldName, { configurable: true, get(this: any) { return (this.__fields[lazyField.fieldName] ??= lazyField.create(this)); },});
And that’s it—we’ve done a one-time/on-boot “probe” of new Author
, to find its fields/relations, moved them to the prototype, and now going forward we can do our constructor-avoiding Object.create(Author.prototype)
call.
When callers access the relations like author.books
, author.publisher
they will still be there, and still “look like fields” to the type system, but they’ll be lazily created.
Non-Relation Transient Fields
Section titled “Non-Relation Transient Fields”The approach so far works great for “class fields that are relations”, like hasMany
or hasReactiveField
, but what about “just regular fields”, like if you want to have some extra/random class field on an Author?
Joist already had an established convention for these “not actually persisted but sometimes useful fields”, which was wrapping them in a transientFields
object:
class Author extends AuthorCodegen { transientFields = { someSpecialFlag: false, };
// use the transient field in some business logic get someLogic(): boolean { return this.transientFields.someSpecialFlag ? 1 : 2; }}
We created this convention solely for developer ergonomics: we want it to be abundantly clear that someSpecialFlag
is not a database column, and wrapping these one-off fields in a transientFields
object has been a great way to communicate that.
Here again we got lucky—because transientFields
is a known convention/hardcoded name, even though we’re skipping the new
operator (which normally defines the transientFields
POJO on each instance), we can apply the same “move to the prototype” trick.
We defineProperty
a Author.prototype.transientFields
getter to lazily create each instances’ transientFields
POJO on first access:
Object.defineProperty(cstr.prototype, "transientFields", { get(this: any) { // Give each instance its own lazily-created copy of transientFields const copy = structuredClone(value); // Once defined on the instance, this prototype getter won't be hit // again, b/c author.transientFields will immediately find its instance // level copy Object.defineProperty(this, "transientFields", { value: copy }); return copy; },});
So everything still works!
Performance Achieved
Section titled “Performance Achieved”So, why do all this?
We recorded benchmarks from our internal codebase, creating one of our core entities, which has 30+ relations, across these 3 approaches:
- The vanilla
new cstr()
with no field optimizations at all - Our previous
joist-transform-properties
ts-patch approach - The new
Object.create
+ prototype approach
benchmark avg (min … max) p75 / p99 (min … top 1%)---------------------------------------------------- -------------------------------
new cstr() vanilla js 4.98 µs/iter 5.00 µs █▅ (4.11 µs … 176.31 µs) 8.92 µs ██▃ ( 32.00 b … 1.16 mb) 29.87 kb ▁████▅▃▂▂▁▁▁▁▁▁▁▁▁▁▁▁
joist-transform-properties 258.26 ns/iter 258.62 ns █ (248.18 ns … 331.49 ns) 279.27 ns ██ (296.04 b … 661.47 b) 496.61 b ▂▁▂▂████▃▄▃▂▂▂▂▁▁▁▁▁▁
Object.create() 157.89 ns/iter 156.26 ns ██ (152.97 ns … 305.13 ns) 222.49 ns ██ (351.71 b … 708.99 b) 441.30 b ██▃▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
Note that the 1st results 4.98µs is microseconds, while the 2nd and 3rd are nanoseconds, so after normalizing & comparing them:
- Our new
Object.create
is 30x faster that vanillanew Author
- The new
Object.create
is also 1.6x faster than our previous AST-transform approach (likely b/ctransientFields
is lazy now too)
Perhaps more importantly, for memory usage:
- The new
Object.create
uses 70x less memory than vanillanew Author
- The new
Object.create
uses basically the same memory as the AST approach (expected)
We’re really happy with these results!
Onward
Section titled “Onward”Looking at what we’ve achieved, we’ve made all Joist relations/class fields lazy, with no decorators like @lazy
or @ManyToOne
, or our prior AST transforms; instead, just by taking the time to study the problem, and then leaning into one of JavaScript’s core native features—prototypes!
This work unblocks our current priority: improving the DX of our large, 500-table majestic monolith codebase, to feel as light & nimble as smaller codebases—and doing so without regressing real-world application performance.