Accessing threejs objects through viewer

Help
,
1

hi :wave: question to the web dev crew :melting_face:! Is it possible to access and modify threejs geometry / material attributes from the @speckle/viewer package? I’m working on a dashboard that will display a scene that is constructed with viewer.loadObjectAsync() and after the scene is loaded I will need to access a specific point cloud object and modify the points and colors during runtime. This is where I’m stuck, as I can’t seem to figure out how I could possibly do this with the viewer :neutral_face:

For context, here is the gist of what I’m doing

// loading!
await viewer.loadObjectAsync(...)

// this seems to be sepckle stuff
const tree = viewer.getDataTree();

// this seems to be three stuff
const render = world?.getRenderTree();

// find the point cloud node
const renderNode = render?.getRenderableNodes(SpeckleType.Pointcloud);

// no impact when modifying the model 
renderNode[0].model.raw.colors = Array(pointCount).fill('rgb(255, 255, 0)');
viewer.requestRender();

And this is sorta where I’m stuck :melting_face:

I’ve looked through the notion page and dug around in the repo as well for any hints. I’ve also been logging out the rendering/world/data trees to see if I can locate any three geometry within the nodes to maybe call something directly to the three object like cloudGeo.setAttribute('color', colors). If it is possible to use the viewer to access the raw three-js geometry, how would the crew recommend that? or is the viewer only meant for viewing :upside_down_face:

tks <3

2

Hi @haitheredavid

It’s possible to accomplish what you set out to do via de following:

// create a DebugViewer instance instead of a Viewer one
const viewer = new DebugViewer(...)

// do regular init stuff
...

// load what you need to
await viewer.loadObjectAsync(...)

// get the speckle world tree
const worldTree = viewer.getWorldTree()

// get a render tree instance
const renderTree = worldTree.getRenderTree()

// get the render views you want to manipulate
cons rvs = renderTree.getRenderableRenderViews(SpeckleType.Pointcloud)

// let's say you want to manipulate the first one
const pointCloudRv = rvs[0]

// get the three.js object containing the speckle object. In this case it will a 'Points' three object
const threeObj = viewer.getRenderer().scene.getObjectByProperty('uuid', pointCloudRv.batchId)

// get the three.js geometry
const geometry = threeObj.geometry

// get the vertex attribute array you want to change, let's say the color
const colorAttr = geometry.getAttribute('color)

// set the new color values
const newColors = new Float32Array([ ... ])
colorAttr.set(newColors, pointCloudRv.vertStart * colorAttr.itemSize)
colorAttr.needsUpdate = true

You will need to use a DebugViewer in order to get access to the renderer. It works exactly the same as a regular Viewer.

The viewer batches multiple speckle renderable objects into fewer three.js renderable objects. We’ll be using the speckle viewer’s NodeRenderView concept. All render views hold a batchId property which point to a three.js object inside the scene with that same uuid. That’s what we use to get the containing three.js Points object. Afterwards we do regular three.js stuff and update the color vertex attribute buffer. One thing important here is the render view’s vertStart property which always points to the object’s vertex attribute offset iniside the batch. Please make sure the new color array length you set does not exceed the render view’s vertEnd - vertStart otherwise you might overflow to other objects inside the batch. All of this is of course valid even if you have a single point cloud object (or any geometry type) in the scene

Just as a reminder, if you only need to change the color of point clouds, you can also do this via the filtering API by using setUserObjectColors. Keep in mind though, that any possible vertex colors of the point cloud will still be taken into account even you set a specific color via it’s material

Finally, we want to let you know that we’ve working on a new version for the viewer’s API which will allow much, much more flexibility and ease for any client app using the viewer library. Once this new API will go public we will deprecate a lot of the current API, gracefully or otherwise. The documentation for the current (old) viewer API has lagged behind because of the coming of this newer API.

Let me know if you need additional help on this!
Cheers

4 Likes
3

@alex :pray: thank you for this insight! This helped me get exactly what I needed for this prototype.

There was one piece I needed to add in order to render the vertex color. By default the point material has vertexColor= false set by default, but I modified the value in the same call with
threeObj.material[0].vertexColors = true;`

For others who want to copy + :spaghetti:

// typical loading of viewer

const worldTree = viewer.current?.getWorldTree();

const renderTree = worldTree?.getRenderTree();

const renderViews = renderTree?.getRenderableRenderViews(SpeckleType.Pointcloud);

const happyCloud= renderViews[0];

const threeObj = viewer.current
    ?.getRenderer()
    .scene.getObjectByProperty('uuid', happyCloud.batchId);

// @ts-ignore
threeObj.material[0].vertexColors = true;

// @ts-ignore
const geometry = threeObj.geometry;

const pointCount = geometry.getAttribute('position').count;
const colors = new Float32Array(pointCount * 3);
for (let i = 0; i < pointCount * 3; i++) {
    colors[i] = Math.random();
}

geometry.setAttribute('color', new THREE.BufferAttribute(colors, 3));
geometry.attributes.color.needsUpdate = true;
1 Like
4

I got a couple of follow up questions regarding the viewer details you mentioned :melting_face: Defiantly nothing urgent below :upside_down_face:

I think I get the gist of this, but just to confirm this is meant to make sure we don’t exceed the amount values that can be stored in a buffer? The start and end should give us the range of data to use and the offset index for the passing the buffer to the attribute, correct? I’m curious cause I was following your recommendation of checking that the the length of the new buffer array against the vert values, and when I logged these values out and they all came back as undefined while there were some values with batch (batchStart=0, batchCount=62865, and batchEnd=62865)…

Does any of this relate to chunkable attribute from speckle-sharp ?

Excited to hear more about this! gracefully or otherwise :melting_face:

1 Like
5

Hi @haitheredavid

Glad to be of help! :slight_smile:

Yes, that is correct. In cases where a speckle renderable object is the single one in the batch, it will fill up the vertex attribute buffers entirely, like you seem to have in your example. But generally, the vertex attribute buffer hold the vertex attributes for more than a single object, that’s why when writing to them you need to keep between the bounds of the render view’s vertStart and vertEnd

Not all render views have batch information stored. Only the ones which are renderable.

No, all the batch related stuff is viewer specific and it’s strictly graphics related. We batch things up to cut down on draw calls.

I’m very glad to hear this. More info on this coming very soon! :wink:

Cheers

1 Like
6

excellent info @alex! :heart_eyes_cat: thanks my man :pray:

1 Like