Extract the geometry or location from speckle object for further labeling

Nikita_Sneg · 23 January 2024 11:02

Hello,

I am trying to extract the three.js geometry from a Speckle object in the viewer. Ideally, I want to access the geometry using the Speckle object ID or Node ID.

So far, my attempts have been unsuccessful. I’ve tried the discussion on how to achieve this (Accessing threejs objects through viewer). However, this method requires accessing certain rendering view, but I received a bunch of these views. My challenge is identifying the correct object geometry, as the three.js geometry object varies in each view that i got somehow. Any code as to achieve it correctly will be best.
My attempt:

const worldTree = speckleViewer.getWorldTree();

const worldNodeById = worldTree.findId('82ab4d34ff57a4020912f4dd8841fefb')[0];//giving here the  nodeID wich is the same as speckleObjectId 

const rvsObj = renderTree.getRenderViewsForNode(worldNodeById);//getting multiple views for single node

const renderView = rvsObj[0];//getting first one as an example, but i need to understand logic which will be correct?

const threeObject = treeScene.getObjectByProperty('uuid',renderView.batchId);

I also tried to obtain a single view with the method:
const rvsObj = renderTree.getRenderViewForNodeId(id: string),
but it consistently returns [null].

Another approach I attempted was based on a suggestion here (Best approach to add CSS2DRenderer objects to speckle viewer). This involved locating the exact point of a user click. According to the documentation, the method
const pointLocation = selectionInfo.hits[0].point;
should return a Vector3 type representing a point in 3d-space. So there is (X,Y,Z) The next step involves projecting this onto the user screen using
speckleViewer.Utils.NDCToScreen(x: number, y: number), which requires only two coordinates but we got 3 in the previous method. I am currently confused about how to proceed effectively.

The ultimate goal is to dynamically label a bunch of elements. I’ve also found an example on the official three.js page that seems straightforward. For those facing similar issues, here are some sources I’ve found and will leave here for the future:
https://threejs.org/examples/?q=label#css2d_label

github.com

mrdoob/three.js/blob/02d4c5aa091ea8b6137db5bf1ad38819308246e3/examples/css2d_label.html

<!DOCTYPE html>
<html>
	<head>
		<meta charset="utf-8">
		<meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0">
		<title>three.js css2d - label</title>
		<link type="text/css" rel="stylesheet" href="main.css">
		<style>
			.label {
				color: #FFF;
				font-family: sans-serif;
				padding: 2px;
				background: rgba( 0, 0, 0, .6 );
			}
		</style>
	</head>
	<body>
		<div id="info"><a href="https://threejs.org" target="_blank" rel="noopener">three.js</a> css2d - label</div>

		<script type="importmap">

This file has been truncated. show original

github.com

tamani-coding/threejs-html-text/blob/9dad6f5842e040435689879882a80df655f280e0/src/index.ts

import * as THREE from 'three';
import { Vector3 } from 'three';
import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls';
import { GLTFLoader, GLTF } from 'three/examples/jsm/loaders/GLTFLoader.js';

// SCENE
const scene = new THREE.Scene();
scene.background = new THREE.Color(0xa8def0);

// CAMERA
const camera = new THREE.PerspectiveCamera(45, 
        window.innerWidth / window.innerHeight, 0.1, 1000);
camera.position.y = 2;
camera.position.z = -5;
camera.position.x = 5;

// RENDERER
const renderer = new THREE.WebGLRenderer({ antialias: true });
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.setPixelRatio(window.devicePixelRatio);

This file has been truncated. show original

haitheredavid · 23 January 2024 12:44

hi @Nikita_Sneg

Can you explain this part? Do you have multiple instances of the speckle viewer?

Nikita_Sneg · 23 January 2024 13:29

Hey.
Here is part of the code that call the renderView:

const rvsObj = renderTree.getRenderViewsForNode(worldNodeById);
console.log('Render Views For Node:',rvsObj);
const renderView = rvsObj[0];
const threeObject = treeScene.getObjectByProperty('uuid',renderView.batchId);
console.log('Three Object:',threeObject);

And here is the pic. what i get in the console for small model:

And here what is the first view contain:

So as I said there is 116 rendering views for just one tree node.

I tried when I load a one object to viewer that was 2 views. And when I load a big model i got an almost thousand rendering view for one node. Can you explain what i have done wrong?

haitheredavid · 23 January 2024 15:03

I believe that is the expected behavior, I think you might be confusing what a view vs SpeckleViewer. I’m sure I’m going to mix up some details here, but hopefully @alex will continue to be our guiding light .

The SpeckleViewer is the main component that ties together all of the three-js magic. When you load a speckle object into that viewer it will create a view of that model you loaded.

A view is the rendered output of each object with a displayable value (i.e. a mesh). If you look at the screen shot from the rvsObj list you’ll see there are some Speckle props showing up in first item along with some of the rendering data for three. As you pointed out that list of views will get bigger if the model has more objects.

I’m still unsure on what problem you’re hitting. Selecting by ID’s is something supported within the viewer. I’m not sure what you mean by “extract the three.js geometry” could you explain?

alex · 24 January 2024 00:01

Hi @Nikita_Sneg

I can gladly be of help.

First thing what we need to do, is use the latest version of viewer API 2.0.
Next, I’ll try to go over some ideas from your post one by one with hope of providing some clarity.
Finally, I’ll link a live code sandbox where I’ve implemented some things with the viewer which will hopefully prove useful for your use case.

I am trying to extract the three.js geometry from a Speckle object in the viewer. Ideally, I want to access the geometry using the Speckle object ID or Node ID.

That’s great news, because that’s the way things usually work with the viewer As an additional note, unless the id belongs to an instance, the node ID will be the same with it’s speckle object id

However, this method requires accessing certain rendering view, but I received a bunch of these views

Like @haitheredavid explained, a RenderView is just an abstraction the viewer makes on objects that are displayable in any way. Because the objects are organized as a tree which preserves their original hierarchy, calling getRenderViewsForNode for a node high up in the hierarchy will get you back all the render views that belong to it’s children, it’s childrens children, and so on.

My challenge is identifying the correct object geometry, as the three.js geometry object varies in each view that i got somehow

The viewer batches all objects automatically. This means that

const threeObject = treeScene.getObjectByProperty('uuid',renderView.batchId);

Will not return the three object associated with the render view because that doesn’t exist. It will return the batch the render view is part of. If you really need the actual geometry of the speckle object, the render view can help you there.

The next step involves projecting this onto the user screen using
speckleViewer.Utils.NDCToScreen(x: number, y: number), which requires only two coordinates but we got 3 in the previous method. I am currently confused about how to proceed effectively.

This function is a helper to transform NDC (Normalized Device Coordinates) values which are in the range [-1, 1] to pixel (screen) values. The value you get from selectionInfo.hits[0].point is in world space, and cannot be used directly into Utils.NDCToScreen. You’d need to unproject it with a query but the code sandbox linked below takes an easier route. In case you really want an explanation and example on queries and coordinate space transformation, let me know and I’ll gladly help.

I tried when I load a one object to viewer that was 2 views. And when I load a big model i got an almost thousand rendering view for one node. Can you explain what i have done wrong?

As @haitheredavid also states, you’ve done nothing wrong Getting the render views for the root node of a large model, will return a lot of objects

In order to better exemplify, I've made this code sandbox for you:

https://codesandbox.io/p/sandbox/dawn-bird-ft36lr?file=%2Fsrc%2FLabelling.ts

It uses the three.js code you linked in your post to draw dynamic labels and it shows two things:

How you can get the render views of objects for several cases with a varying level of hierarchical depth. This adds three labels at startup with the text ‘Rv Count’
How you can get the point you clicked on objects and add a label dynamically. This adds a label each time you click an object, at the location you clicked it with the primitive count as text.

Viewer’s API 2.0 is being properly documented as we speak, and full API reference along with examples and other resources will be available very soon. Until then, there is an older
quick reference which might be of help.

We appreciate your feedback, and in case you need further help with anything don’t hesitate to ask!

Cheers,
Alex

Nikita_Sneg · 24 January 2024 14:50

Thank you very much!
You are truly a genius, even maybe a God. Your detailed explanation was beyond my expectations. Everything seems to be working now. I’ll attempt to incorporate it into my application.