Best practice for isolating user-coloured objects

[mivalek]

Hello,

First of all, amazing work guys! Second, happy new year!

Description

Now, I am writing a web app (Next.js) that uses various analyses stored in a Speckle stream. To visualise these, I am using the speckle-viewer package (see screenshot).

I have an interactive parallel coordinates plot that shows variables used in some analysis and a viewer instance with geometry divided into block. These blocks are coloured according to the value of one of the variables. When user selects a range on the PC pot (light blue rectangle on the left hand side in the screenshot), the viewer isolates the corresponding blocks.

Problem

My current approach is as follows:
The pointer event on the PC plot calls back a function that

• retrieves the IDs within the selected range
• cancels previous selection with viewer.resetFilters
• colours the objects using viewer.setUserObjectColors
• isolates selected objects with viewer.isolateObjects

The problem is that this approach is rather slow which makes the UX a bit janky. I thought I could setUserObjectColors on all Objects and then just (un)isolate them as needed but I can’t seem to be able to make this work. Once the objects are coloured, the (un)isolateObjects functions seem to have no effect.

Question

Is there a better way to update coloured isolation that doesn’t require traversing the data tree multiple times as is the case in my approach?

Thanks a lot!

[alex]

Hello @mivalek

Happy New Year to you too!

We’re always very excited to see people build cool stuff with speckle, and we’re keen on helping them get the best results!

My first question would be which viewer package version are you using? if it’s not API 2.0 I would strongly suggest you switch to that. We’ve been pushing update alpha versions for API 2.0 under the viewer-next tag on npm.

Once you get API 2.0, you’ll have more than one option to solving your problem. If you provide me with a stream and some more details about behavior I could help you with a code sandbox example

Cheers

[mivalek]

Hi @alex,

Thanks for the quick reply. I have been using the old API (by mistake more than by choice). I just had a quick read of the API 2.0 documentation and it seems like the old functions are mostly gone?

Unfortunately, I am not sure I am at the liberty of sharing the data but would really appreciate if you could help me understand - if you can - the basic logic of how the combined colouring/isolating could be achieved with the new API if all I have is an array of object IDs. The documentation is a little too bare bones for me to make much sense of it

Many thanks!

[alex]

Hi @mivalek

It’s fine if you can’t share the data. I’ll try to be as helpful as I can either way. I know API 2.0 documentation is bare bones, but we’re working on a full fledged one as we speak

With API 2.0, you can choose to use the same filtering functionality that exists in the old version( we’ll revamp filtering in a future iteration of the viewer), or you can also set object materials selectively. One such example is this sandbox which uses that API 2.0 functionality.

I can further help you with maybe another sandbox that does something more close to what you’re trying to do, but I need more details. So far I got that you need to set a group of objects a particular color, and also have them isolated, as in, every other object is ghosted? If the scenarios is more complex than that, please let me know in a reply and we’ll sort it out together

Cheers

[mivalek]

Hi @alex , your sandbox was exactly what I needed to set me on the right track, thank you very much! What I did was loop over all ids and set the corresponding object’s material to the right colour if it’s among the selected ids and to a transparent material otherwise. The result seems a little snappier than it was before with the 3-step procedure I described in my OP. I’m reasonable happy about the performance but if you can see a way to further optimise it, I’d be super interested. Here’s the code:

export async function isolateColour(selection: string[]) {
  ...
  const renderer = viewer.getRenderer();
  const rt = viewer.getWorldTree().getRenderTree();
  const colourGroups = ... // just an object with id as key and hex string as value
  const colourMaterial = {
    id: "id",
    color: "#ffffff",
    opacity: 1,
    metalness: 0,
    roughness: 1,
    vertexColors: false,
  };
  const ghostMaterial = {
    id: "id",
    color: "#aaaaaa",
    opacity: 0.1,
    metalness: 0,
    roughness: 1,
    vertexColors: false,
  };
  for (let id of ids) {
    let material;
    if (selection.includes(id)) {
      material = colourMaterial;
      material.color = colourGroups[id];
    } else {
      material = ghostMaterial;
    }
    const rvs = rt.getRenderViewsForNodeId(id);
    renderer.setMaterial(rvs, material);
  }
  await viewer.requestRender();
}

Apropos of what you said about being able to use the same functions. I just looked into the code and see that there’s indeed filteringExtension in the package but I didn’t see it in the quick reference. That makes me wonder if I’m reading the correct document. Is this the most up-to-date documentation?
Don’t get me wrong, I appreciate that you guys are currently working hard on the API so I’ll have to wait for a more comprehensive document.

If I may make a request for future reference, I think it would be very useful to define the terminology you’re using. I don’t come from a graphics background and have no idea if terms such as “world/data/render tree” or “render view(s)” are standard jargon. All I know is that I don’t have a clue what they are

In any case, thank you so much for your help!

[alex]

Hi @mivalek

I’m glad to hear you managed to get it working!

Generally speaking, unless you have tens of thousands (or more) of ids you need to assign materials to, you shouldn’t encounter performance issues, but regardless, here are some ideas for improving on the code you shared:

• Instead of calling setMaterial foreach id, I would accumulate all rvs for each particular material, then call setMaterial once for each material with the accumulated rvs
• I would cache results from each rt.getRenderViewsForNodeId(id) call, since there is a finite number of them
• You are probably calling isolateColour from a UI control handler. If that’s the case you should limit the amount the calls you make to a maximum 1 per frame. Generally UI handlers fire multiple times per frame, which is pointless for anything graphics related. In order to do this, I recommend you turn this function into an Extension and use the extension’s update or render callbacks to limit the amount of calls per frame similar to how it’s done in this extension with a frameLock variable.

I realize the current available documentation is too bare bones, but it’s meant only as a quick reference for the API. The official documentation is being put together as we speak and will be available soon

The soon to be available documentation will cover all the terminology and explain what is what I realize that without proper explanation, the terms we are using might be confusing. Until then, here’s a quick intro:

• WorldTree: The tree-like data structure we store all loaded data into. It consists of nodes organized in a tree fashion. We use this to determine hierarchic relations between the objects
• DataTree: This will become obsolete in API 2.0, so you don’t need to worry about it
• RenderTree: It can be either the entire WorldTree, or a subtree of it, with added rendering related functionality
• RenderView: Represents an atomic renderable entity and it’s viewer specific. Speckle objects can contain multiple renderable objects, but need to be treated both as a whole, and as separate parts.

Thank you very much for your feedback, and please let us know any other thoughts you have on the viewer and it’s API!

Cheers

[mivalek]

Excellent tips, thank you @alex! I’ll be sure to look into all your recommendations. And thanks as well for the quick tailor-made glossary, it makes more sense now! I’m looking forward to the more fleshed-out version of the documentation but the amazing support you’ve provided me makes up for any shortfall on that front!

I’m sure I’ll be back with more questions but, in the meantime, thank you again and thanks to all the Speckle team for the awesome job

[zm1072223921 (_-5upl1ke)]

@alex haven’t seen you for a long time！

[zm1072223921 (_-5upl1ke)]

@alex When I use the ‘isolateObjects’ function, I find that after I import a large model (about 7 million triangles), this function will cause my program to be very stuck, and in some cases the length of the JavaScript array will appear. For overflow errors, I looked at the filter module’s implementation of ‘isolateObjects’ and found that it was implemented by replacing materials. So in this extreme case, how should I deal with the lag problem?
At the same time, I found that the two functions ‘hideObjects’ and ‘showObjects’ cannot achieve the correct effect on many models. This problem is especially likely to occur when I pass in ‘ids’ in batches, and these two methods are different from ’ isolateObjects’, the same performance problem will occur when encountering more complex models. I tried to rewrite ‘hideObjects’ and ‘showObjects’ to make them independent from the ‘filter’ module, and it can work normally. show’ or ‘hide’ the model I specified, but it still cannot solve the performance problem.

This is the method of ‘hideobject’ that I overridden:

 public hideObjects(ids: string[]) {
    if (!ids || ids?.length < 0) return

    this.hideObjectList = Array.from(new Set([...this.hideObjectList, ...ids]))
    let list: any[] = []
    ids.forEach((item) => {
      const nodes = this.WTI.findId(item)
      if (!nodes || nodes.length <= 0) return
      nodes.forEach((node: TreeNode) => {
        const rvsNodes = this.WTI.getRenderTree()
          .getRenderViewNodesForNode(node, node)
          .map((rvNode) => rvNode.model.renderView)
        list = [...list, ...rvsNodes]
      })
    })

    list.forEach((item) => {
      if (!this.selectionMaterials[item.guid]) {
        this.selectionMaterials[item.guid] = this.viewer.getRenderer().getMaterial(item)
      }

      this.viewer.getRenderer().setMaterial([item], this.hideMaterialData)
    })
    this.selectionRvs = {}
    this.viewer.requestRender()
  }

[alex]

Hi @zm1072223921,

What viewer version are you using? I would recommend you move to the latest: 2.17.0-alpha.11 which has some bugfixes regarding hiding objects.

Regarding the performance issues for isolateObjects or hideObjects I’ll look into it, but it would be of help if you give me some numbers:

• How many ids are you passing to the function in the ids array?
• Are you calling hideObjects/isolateObjects each frame? Maybe multiple times per frame?
• Approximately how many objects does you stream have?

What would really help me to replicate your issue, would be the actual stream you’re working on, or even better, a code sandbox with an example

Cheers

[zm1072223921 (_-5upl1ke)]

Hi @alex
When I wanted to upload a complex model to you, I thought that your database has an upload size limit. Then we pulled your source code locally and canceled this limit, so I can only preview this complex model locally. When I try to upload on speckle’s official page it fails, do you have any good solutions?
This is the error log:
SpeckleCoreLog20240119.txt (1.7 MB)

[gjedlicska (Gergő Jedlicska)]

Hey @zm1072223921

you issue from the logs is due to a performance limitation in the server, we’re working on fixing.

Is your local server deployed with docker compose?

I’ve scaled our backend to allow for more burst load. Could you retry the send operation to XYZ?

Thanks,
Gergő

[zm1072223921 (_-5upl1ke)]

@gjedlicska Thanks! I have now successfully uploaded a larger model to the platform.
@alex When I looked at this large model, I found that many functions would become very laggy. For example, when I called ‘hideObejcts’, I took out about 16,000 ‘objectis’ from the data and then passed them to ‘hideobjects’. Then my browser will freeze completely for about 30 seconds, and then a similar problem will occur when I call ‘showobjects’. Do you have any good solutions? Here is the link to the model I looked at:
https://speckle.xyz/streams/a1c2b52e1d/commits/cd5c2fbb68

[alex]

Hi @zm1072223921

If you open the same model with FE2: Viewer - b1 - Speckle you will see that the hide operations as well as loading times are several orders of magnitude faster. Generally everything should be better in FE2

Cheers,
Alex

[zm1072223921 (_-5upl1ke)]

@alex I am currently using fe2. I pulled the source code for speckle from GitHub and used the yarn dev command in the frondend2 file directory. Then, when I called the hideObjects method, I passed in ‘ids’ equivalent to the entire scene, and my page experienced a graphics lag of up to 30 seconds. You can try reproducing this operation in the console.
Cheers, _-5upl1ke.

[alex]

If I got to the FE2 link from the post above, hiding all objects is instantaneous compared to FE1 where indeed it took a very large amount of time

[zm1072223921 (_-5upl1ke)]

@alex Have you ever tried to pass in 16000 objects simultaneously?

[zm1072223921 (_-5upl1ke)]

@alex Did I describe this issue clearly? Can you reproduce the problem I encountered on your end？

[jonathon]

@zm1072223921 Besides offering premium support to the Speckle Community, @alex is an excellent full-time engineer building our amazing viewer and API. He will have received a notification of your reply, and when he can, I’m sure, offer more great answers.

[alex]

@zm1072223921

I usually do my best to answer forum questions that are viewer related, however some answers require more attention and often additional work on my part before a proper reply can be posted. When it takes me more time to answer somebody’s posts usually means just that

Did I describe this issue clearly? Can you reproduce the problem I encountered on your end？

Yes you did, and I cannot perfectly reproduce the problem you’re describing on my end, but I may still know what’s going.

Have you ever tried to pass in 16000 objects simultaneously?

Yes I have, even made a code sandbox for this purpose

Because you are usually cryptic in describing the bigger picture of what you are trying to do, I’ll have to make some assumptions:

• You have forked(one way or another) frontend-2 and you are either changing the application, to turn it into your own, either making a new one from scratch with frontend-2 as inspiration.
• You still haven’t answered which viewer version you are using, so I’ll have to assume that you’re using the latest API 2.0 version

I’ve made this sandbox where I load a large model, about the same size as your model (which I cannot access because you removed access), then I get ID_COUNT number of random object ids (default is 64000) which you can hide via a button. It takes 1-2 seconds to hide those 64000 ids. If I am to get 16000 random ids, like in you post, hiding them is super fast, so I upped your number so I can get an idea on how much time it takes. If you increase that number of ids, the time it takes also increases, but in order to get to your 30 seconds worth if waiting, you probably have to send millions.

We could execute all of these operations faster, and we’ll probably work on that in one of the next iterations on the viewer. But until then, there are also things developers can do to speed things up. I’ll talk next about that

You’ve stated:

Then, when I called the hideObjects method, I passed in ‘ids’ equivalent to the entire scene

The viewer itself does not stop you from doing that, but what you end up doing is pilling up redundancy, and here’s why: Internally, when passing an id to the viewer in order to hide/select/isolate/etc, there will eventually be a call to getRenderViewNodesForNode or getRenderViewsForNode which starts at the node with the id you passed in and walks downwards in the tree and returns all nodes/renderviews that are displayable. If you take all your ids from the scene and pass them to hide/isolate/select/etc what will happen is that the displayable nodes will get collected more than once (possibly a lot more). It’s almost like telling it to hide/isolate/select everything 100 times one after the other.

If you do want to hide everything at once, all you need to do is pass the subtree root id (which will typically be the adress of the stream’s root object if it’s a speckle stream) to the hide function. This action is exemplified in the sandbox I made. You will see that hiding everything this way, even if internally it still needs to go over all displayables and act on them it’s fast