Hi David, (apologies I wrote this between many meetings so publishing got delayed)
Does the relational layer replace Proxies, or sit alongside them?
Depends which side of the wall youâre standing on.
Iâm leading with that because the honest answer isnât the tidy one, and pretending otherwise would only cost you a paragraph later. Below the wall, in the bundle itself, Proxies as a storage concept do not exist. There is no Proxy table. Whatâs there instead is a flat entity-attribute-value store plus a graph laid beside it: synthetic nodes and typed, directed relations, catalogued and queried by SQL. SUBELEMENT connects one object to another. ON_LEVEL connects an object to a level. HAS_MATERIAL connects geometry to a material. IN_COLLECTION connects an object to a collection. That part is genuinely new, genuinely relational, and genuinely not something you traverse.
Above the wall, where your code actually lives, operations.receive still hands you a tree. And â this is the part I got wrong the first time I answered this, so Iâm not going to pretend I didnât â that tree still carries renderMaterialProxies and instanceDefinitionProxies, the same keys your unpacker already looks for. Theyâre kept there on purpose, specifically so existing Proxy-consuming code doesnât have to change on day one. So: replaced underneath, retained at the boundary you touch, for now. Both halves of that sentence are true today, which is precisely what makes it a worse answer than âit replaces them,â and also the correct one. Whether âthe boundary you touchâ still looks like a tree in two years is a different question, and Iâm not going to promise you an answer to that one.
How will traversal work, and how will existing Proxy-based code need to change?
Two different questions now, with two different answers, which is the tax for asking a two-part question about a two-layer system.
For the storage layer: no traversal API, because the new architecture there is SQL. Consumers of the low-level query surface will not begin at a root Base, recursively walk its closure, and hope the thing they need hasnât been detached somewhere with an @ prefix. Theyâll query object, property, node and relation tables and join what they need. Moving from an object to its level is a join across relations and nodes. Finding everything assigned to a material is the same relationship queried in the other direction. None of that is traversal. Itâs SQL. Thatâs the layer I was describing last time, and on its own terms I stand by it.
For the receive layer, the one most existing code actually calls: the call itself doesnât change â you still call receive, you still get a tree, you still walk it, your Proxy-reading code still finds Proxies. What changes is fidelity, not the method of calling it, and thatâs a real distinction, not a hedge: the tree arrives in the v3 DataObject idiom rather than the old typed classes, id stops being a content hash, and parameter metadata collapses to scalar values rather than the old typed dict. If your code pattern-matches on typed classes, caches anything keyed on Base.id as if it were stable and content-addressed, or reads parameter metadata as a structured dict, thatâs where youâll actually feel this â not at the call site, but in whatever you do with what comes back. If you want the new relational query surface instead of the tree, itâs there, opt-in, alongside receive rather than instead of it: query(refs, sql), raw readonly SQL over the bundle.
Within a version, objects are represented by a dense integer object_index. That part I had right the first time: itâs the identity used for joins, grouping and filtering inside the bundle, with the stable applicationId still available at the boundary where external or cross-version identity is needed.
Existing Proxy-based code therefore doesnât need to change on day one, and eventually gets a second option rather than a forced rewrite: keep reading Proxies off the received tree as before, or, if you want the underlying relational detail Proxies were always an approximation of, drop down to the query surface and ask for it directly.
Will I still need to pull the entire model to edit a few rows in a table?
For now, yes.
This particular schema change does not itself deliver server-side filtering. Server-side SQL execution remains the intended solution, and work on it has started, but it has taken a back seat while the object, identity and relation schema settles.
The ordering matters. A selective query must return a coherent set, and âcoherentâ depends on stable answers to several irritatingly reasonable questions. If three objects match a filter, which relations must accompany them? Do they require their containment path, level, materials, geometry, definitions or instances? Which records are necessary to understand the result, and which would merely be the rest of the model arriving under another name?
Finalising server-side filtering before settling those semantics would have produced a filtering contract against an intermediate schema, followed by another filtering contract against the final schema, followed by a compatibility layer between the two, followed by Jonathon explaining on the Community forum that both are technically correct depending on when the model was published. Nobody has requested this outcome, although historically that has not always prevented it.
The architecture already separates the query contract from where the query executes. Its public data surface is SQL in and typed rows or columns out. Queries can initially run locally against an attached version bundle, while the same surface is designed to support server-side execution later without exposing DuckDB connections, storage paths or worker machinery to consumers.
The eventual workflow is therefore selective retrieval rather than graph reconstruction. The server filters the object and relation records, returns the required columns and supporting relations, the consumer applies changes against stable identities, and the result is published as a new version.
Versions remain immutable. âEdit three rowsâ does not mean mutating three records inside an existing version. It means retrieving the small working set, changing it, and publishing the resulting state without first transferring the full model merely to discover that the desired rows were, as previously suspected, three.
I canât give you a delivery commitment for server-side execution yet. But the âfew table rows without pulling the whole modelâ workflow is directly within its intended scope. The schema work is not the finished capability; it is the dependency that needs to settle before that capability can be finished without immediately becoming its own migration problem.
P.S. There are some genuine terms waiting in the codebase behind this work: the three-artefact bundle, the migration envelope, envelope topology, the relation catalogue, native object grain, the grain rule, the materialisation ceiling, full acquisition inversion, the scene cube, the cube frontier, demand-driven tier deepening, in-place grafts, the container forest, the whale path, and reference rematerialisation. These are all real! The migration envelope cannot be posted. The container forest contains no trees. The whale path contains no whales. The scene cube is not the cube you previously instructed us to keep alive, although long live the cube. Reference rematerialisation does not create an object from pure information while engineers record the temperature; it finds referenced objects in earlier packfiles, copies them into the consuming version and creates their property-index rows. This is useful and correct, which is not really what the phrase âreference rematerialisationâ promised. The materialisation ceiling remains in place. It has not been replaced by a typed relation. Not everything is being replaced. Proxies, underneath the wall, are â though youâll still find them waiting for you on the other side of receive.
Excited to see the model architecture is improving. One of things that always felt a bit strange was pulling a whole model down to make only a few table changes and then package that model back up into a commit, so if there is a way to push these changes in a more performant way, all for it!