How to covert an Assembly with re-used instances to Speckle?

I am making progress on a Speckle Python-connector for our CAD system, i can finally visualise meshes. But i still struggle with some basics and would be glad if somebody could help.

Currently i create one mesh like so:

block = Mesh(vertices=verts, faces=faces)
new_stream_id = client.stream.create(name="triangle")
new_stream = client.stream.get(id=new_stream_id)
transport = ServerTransport(client=client, stream_id=new_stream_id)
hash = operations.send(base=block, transports=[transport])
commid_id = client.commit.create(stream_id=new_stream_id, object_id=hash, message="block made in speckle-py")

How would i:

  1. create a scene graph with groups containing n meshes
  2. re-use or instance re-occuring meshes
  3. apply position, scale, rotation or matrix transforms, Speckle.Mesh doesn’t seem to have any of those attributes

And maybe just to make sure, does Speckle-Py have vector math helpers? I would need something to help with that. Using Python for the first time, i only used JS for the last couple of years. :face_with_peeking_eye:

Great that you are making progress and learning some of the idiosyncrasies of Speckle.

Aside from mesh definition, Speckle also distinguishes itself from other 3D environments in several ways, particularly in how it handles both Transforms, Instances and its underlying data structure. While other 3D environments might support transforms at every node, Speckle employs a different strategy that prioritizes data interoperability, efficiency, and collaboration.

Directed Acyclic Graph (DAG) Structure

Speckle uses a directed acyclic graph (DAG) to manage its objects. This structure ensures that each object points to other objects in a hierarchical manner, with no cycles. The benefits of a DAG include:

•	Efficiency: By avoiding cyclic dependencies, Speckle ensures efficient data storage and retrieval.
•	Non-redundancy: Objects can be reused across the graph without duplication, reducing data redundancy.
•	Clarity: The hierarchical nature of a DAG makes it easier to understand the relationships between different objects.

Definitions and Instances

In Speckle, reusable components are managed through Definition and Instance objects:

•	`Definition`: Acts as a template for reusable components, such as windows or doors, that appear multiple times in your model.
•	`Instance`: Represents each occurrence of a defined component, including a transformation matrix to specify its position, rotation, and scale.

This approach contrasts with other 3D environments where transformations might be applied at every node, potentially leading to complex and less efficient hierarchies.

Objects, Custom Objects and Display Values

For arguments sake let’s say all Objects in Speckle can include a displayValue property, which is a list of meshes* representing the visual aspect of the object. This allows for flexible and detailed representation of components without the need for transformations at every level.

Collections and Hierarchical Organization

Speckle uses Collection objects to group related elements, which can include other collections, custom objects, and in turn be geometric definitions.

This hierarchical organization supports:

•	Modular Design: By grouping related components, Speckle encourages a modular approach to model design.
•	Reusability: Components defined once can be reused across different parts of the model, maintaining consistency and reducing duplication.

Example: Building Model with Reusable Assemblies

To illustrate how Speckle’s approach works in practice, let’s consider creating a building model with reusable window instances:

1.	Define the Mesh for a Window: Create the geometry for a window using a Mesh object.
2.	Create a Window Definition: Define a reusable window component using a Definition.
3.	Create Window Instances: Instantiate the window with different transformations for placement in the building.
4.	Define Custom Objects (e.g., Walls): Create custom objects that include window instances in their displayValue.
5.	Group Elements Using Collections: Use a Collection to group all elements of the building assembly.
6.	Send Data to Speckle: Serialize and send the organized data to Speckle for visualization and collaboration.

SpeckleMesh Specification

As you have been discovering a SpeckleMesh stores vertices and faces in a flat list format:

•	Vertices: A flat list of `x`, `y`, `z` coordinates. The length should be a multiple of 3.
•	Faces: A flat list of integers representing polygon faces. The first integer indicates the number of vertices (cardinality) of the face.

Code Example

Here’s how you might implement this in Speckle:


from specklepy.objects.geometry import Mesh
from specklepy.objects.other import Definition, Instance, Transform, Collection
from specklepy.objects.base import Base
from specklepy.api.client import SpeckleClient
from specklepy.api.credentials import get_default_account
from specklepy.api.operations import operations
from specklepy.transports.server import ServerTransport
import numpy as np

# Initialize the client and transport
client = SpeckleClient(host="https://speckle.xyz")
account = get_default_account()
client.authenticate_with_account(account)

# Example vertices and faces for a window mesh
window_vertices = [0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0]
window_faces = [3, 0, 1, 2, 4, 3, 4, 5, 6]

# Create the mesh
window_mesh = Mesh(vertices=window_vertices, faces=window_faces)

# Create a reusable window definition
window_definition = Definition(name="Window", geometry=[window_mesh])

# Function to create a column-major transformation matrix
def create_transform_matrix(translation, rotation=np.eye(3), scale=[1, 1, 1]):
    transform_matrix = np.identity(4)
    transform_matrix[:3, :3] = np.diag(scale) @ rotation
    transform_matrix[:3, 3] = translation
    return transform_matrix.flatten().tolist()  # Flatten for row-major order in Speckle

# Transformations for window instances
window_transform1 = Transform(
    value=create_transform_matrix([10, 0, 0])
)
window_transform2 = Transform(
    value=create_transform_matrix([20, 0, 0])
)

# Create window instances
window_instance1 = Instance(
    definition=window_definition,
    transform=window_transform1.matrix
)
window_instance2 = Instance(
    definition=window_definition,
    transform=window_transform2.matrix
)

# Create custom objects with window instances
class CustomObject(Base):
    def __init__(self, name, display_meshes):
        super().__init__()
        self.name = name
        self.displayValue = display_meshes

wall_with_windows = CustomObject(name="Wall with Windows", display_meshes=[window_instance1, window_instance2])

# Group the instances into a collection
building_collection = Collection(
    name="Building Assembly",
    elements=[wall_with_windows]
)

# Create a definition for the building assembly
building_definition = Definition(
    name="Building Assembly Definition",
    geometry=[building_collection]
)

# Create transformation matrices for building instances
building_transform1 = Transform(
    translation=[0, 0, 0],
    rotation=np.eye(3).flatten().tolist(),
    scale=[1, 1, 1]
)
building_transform2 = Transform(
    translation=[0, 50, 0],
    rotation=np.eye(3).flatten().tolist(),
    scale=[1, 1, 1]
)

# Create instances of the building assembly
building_instance1 = Instance(
    definition=building_definition,
    transform=building_transform1.matrix
)
building_instance2 = Instance(
    definition=building_definition,
    transform=building_transform2.matrix
)

# Group all building instances into a final collection
final_collection = Collection(
    name="Building Model",
    elements=[building_instance1, building_instance2]
)

# Send the final collection to Speckle
new_stream_id = client.stream.create(name="Building Model with Reusable Assembly")
new_stream = client.stream.get(id=new_stream_id)
transport = ServerTransport(client=client, stream_id=new_stream_id)
hash_final_collection = operations.send(base=final_collection, transports=[transport])
commit_id_final_collection = client.commit.create(stream_id=new_stream_id, object_id=hash_final_collection, message="Building model with reusable assembly instances")

Conclusion

Speckle’s approach, with its DAG structure and use of Definition and Instance objects, offers a streamlined and efficient way to manage complex assemblies. While it differs from other 3D environments that may support transforms at every node, Speckle’s method prioritises data integrity, reusability, and clarity, making it a powerful tool for collaboration and data management in AEC and other industries.

While specklepy itself does not offer built-in matrix or vector math helpers for complex geometric transformations, it can be effectively combined with other libraries such as Trimesh for more advanced geometry manipulation. This allows users to leverage the strengths of both Speckle for data interoperability and external libraries for geometric operations.

I can add an example of conversion from speckle mesh to trimesh if it helps

thank you @jonathon this will most likely bring me to the finishing line. might i ask where i could find all these constructs? Introduction | Speckle Docs is quite synthetic and i notice i have some trouble getting by on my own, i wouldn’t have figured this out without help.

Indeed, we were having a debate on which direction our documentation should go recently and in response to your question.

In Objects terms the original source of truth is speckle-sharp, but we try very hard to ensure specklepy isn’t too far off:

Collections

Definitions and Instances

Transforms

it seems there’s one last snag, if i use this format, translation, rotation and scale, then it crashes the send. i tried with our data (position, xrot, yrot, zrot) and the code above to make sure.

if i use Transform(matrix=[…]) op.send goes through, though i have trouble figuring the matrix format out.

You may of course be right - my eyeballed method of Transform construction direct in the forum editor is by no means definitive, just my quick response.

The schema for the Transform is what is important - if you have a better method that is working - don’t take my example as the way to do things

The source for transform just says translation and scale,

    """The 4x4 transformation matrix

    The 3x3 sub-matrix determines scaling.
    The 4th column defines translation,
    where the last value is a divisor (usually equal to 1).
    """

there’s no property for rotation. I haven’t been able to construct a matrix that moves the parts where they belong.

this is our data

        p = csys[0]
        x = csys[1]
        y = csys[2]
        z = csys[3]

i’m trying this:

        matrix = [
            x[0], y[0], z[0], p[0],
            x[1], y[1], z[1], p[1],
            x[2], y[2], z[2], p[2],
            0, 0, 0, 1
        ]

it doesn’t work. is “scale” the same as rotation? perhaps i should use something else and not “Transform” ?

https://speckle.guide/dev/objects.html#instances-transforms-and-definitions
Might help for some more in depth explanation of the instance, transform, and definition objects: it’s missing a few python details but worth reading through!

1 Like

thank you @jonathon @clrkng i’m almost there and the matrix format is correct now. there’s just one open question left, in threejs as well as in our CAD every group or part or assembly has itself a local transform, and if i, for instance, have a group in threejs and add other things in it, the transforms will multiply naturally.

in the following example the group has a local transform, foo and bar also, and their transforms will add up and move with the group, if it’s transform changes.

const group = new THREE.Group()
group.position.set(1, 2, 3)
group.scale.set(2, 2, 2)
group.rotation.set(Math.PI / 2, 0, 0)

const foo = new THREE.Mesh(geometry, material)
foo.position.set(0, 0, 10)
foo.scale.set(4, 4, 4)
foo.rotation.set(0, 0, Math.PI)

const bar = new THREE.Mesh(geometry, material)
bar.position.set(100, 0, 10)
bar.scale.set(10, 10, 10)
bar.rotation.set(0, Math.PI, 0)

group.add(foo)
group.add(bar)

in speckle a collection does not seem to have a transform. does this mean that all instance transforms have to to be global? do i have to traverse the path structure back and multiply all previous matrices for each instance?

what is also still odd is that whatever i do, even hardcoding transforms to exaggerated numbers, i never see an instance move or rotate, they’re all at position 0/0/0.

NAUO1 (the blue plate) should be shifted to xyz=1000/1000/1000, the transform format should be correct:

[
  1, 0, 0, 1000,
  0, 1, 0, 1000,
  0, 0, 1, 1000,
  0, 0, 0, 1
]

in speckle a collection does not seem to have a transform. does this mean that all instance transforms have to to be global? do i have to traverse the path structure back and multiply all previous matrices for each instance?

Yes, we do not support transforms on any type of object other than Instance currently. Instance transforms are local, with the top-level instance transform therefore being global for any geometry inside the geometry property on the instance. What this means is if you had an object like:

Instance A {
  transform: tA,
  definition: defA { 
    geometry: {
      geoA,
      Instance B {
        transform: tB,
        definition: defB {
          geometry: geoB 
        }
      }
    }
  }
}

Then

  • tB would be the local transform of Instance B to Instance A
  • tA would be the local/global transform of geomA
  • tA*tB would be the global transform of all geomB

What I’d recommend for your group with a transform is to create another instance representing the group with its transform, an instance for foo and an instance for bar with their respective local transforms & geom in the definition, and put the foo and bar instances in the geom of the group definition.

It’s hard for me to say why your hardcoded transform isn’t shifting the instance, I can take a quick peek if you drop a link to that commit with the blue plate :slight_smile:

thank you! i appreciate the willingness to help in the speckle community!
https://app.speckle.systems/projects/d645862bab/models/cfae666774

Hmm at a glance, the class doesn’t look right: make sure you are using the specklepy BlockInstance class and not the abstract Instance class if you are creating a BlockDefinition.

Here’s a snippet from @Jedd in the Blender connector if you’d like an explicit example: speckle-blender/bpy_speckle/convert/to_speckle.py at 6fd4571d340e1e800441dcb41efbabcab1de2b4d · specklesystems/speckle-blender · GitHub

2 Likes