Kit and Converter support for Unity

Hi :wave:

I recently just went through the process of stripping out my project code from the speckle-unity repo and exporting the speckle-unity project as a one off unity package that I could import into my other unity project. All is well in good with that :grinning_face_with_smiling_eyes: but it made me realize that most of that hassle I went through was separating out the ISpeckleConverter class for unity and how that get’s integrated within the SpeckleManagerConnectorThing. It would be helpful to consider how a client, like unity, might deal with handling kits and converters (mainly converters).

I thought it would be good to have a lil’ convo around this process. Here are some of things I think a solution would need to address.

Serializing the Converter in Unity :writing_hand:
Right now in order for a converter to work in Unity a user needs to build out that converter, reference it in the ConnectorUnity namespace, and then modify the Receiver class to use that converter. What would be ideal is creating a simple object that any receiver or sender could have access to and rely on that for converting (just like it does in any other connector) without the need of hardcoding it. But we would need to create a unity friendly object that it could rely on for storing and accessing the different converters. For example, ConverterUnity is the current Object converter that is just a simple object that can be instanced in unity during runtime or editor, but it won’t be serialized since unity doesn’t chill like that with objects. We could attach MonoBehaviour or a ScritableObject class to it, and then unity would be a-okay with serializing that.

I did a pass at addressing this issue by creating an abstract object that a converter could inherit from and then adding those to a container scriptable object. Here is the UnityConverter with that class implemented. The kit and converter object really just a way for the speckle manager to know what converter is should utilize.

image

Here is an epic gif of the manager with reference to those converters (flexing that logo).

lawl

I’m sure there is a more elegant solution that could be built out with reflections, robots, maybe even a skylight, but I’m more of a peon type of developer.

Packaging Converters and Objects :package:
Currently speckle converters and objects need to be added directly to the project. I’m sure on the unity-connector roadmap there some plan to migrate the project into the package manager, but I think it would be helpful to have objects and their converters be separated too. I’ve been using OpenUPM to package my viewobjs and I plan on doing the same with the unity-converter. However, my unity converter piggy backs off the speckle one (ConverterUnity, no need to repeat mesh building code…) so I can’t separate that out from the main project repo. This structure would be a bit more relatable to the current structure of other connectors too.

anndd we out

1 Like

Hi David,

YES! This is definitely worth a discussion.

It sounds similar to a bit of a dilemma I’m having on the Speckle-Unreal side also.

I’m going to take this opportunity to piggyback some of my issues into this discussion because I imagine they will be the same in Unity.

An issue I’m facing with Speckle-Unreal, is there are several types of native meshes that a user might want to convert to.

Secondly, I want to allow users to write their own converters without having to modify the Speckle-Unreal plugin’s source code.

Thirdly, some objects, like point-clouds and breps aren’t natively supported by Unreal Engine, but they are through a plugin. I’m hesitant to make the Speckle-Unreal connector dependent on plugins (particularly third party ones) that most users won’t even need, and so I’m currently toying with this idea of modular converters.


In my (half baked) idea of modular converters, we would have Mesh, Camera, Blocks, Point-Cloud etc conversion, all in separate classes implementing an interface (eg. ISpeckleTypeConverter)

We would store the active ISpeckleTypeConverters in a TMap (c# equivalent would be a Dictionary) that maps a Speckle Type (eg. "Objects.Geometry.Mesh") as a string, to an ISpeckleTypeConverter.

During receive, we then use the Speckle Type of our object to get its converter using the TMap.

I think Unity has some similar issue with point clouds being dependent on a third party plugin. And the need to modify source code to extend conversion.

In Unity, these ISpeckleTypeConverter could be serialisable, and grouped into kits using ScriptableObjects.

What do you think (have I gone too far? is this a separate discussion?)

1 Like

@Jedd ah you bring many good points! I agree that breaking things down into Type converters could lead to some handy solutions with unity and unreal. For unity we currently convert and pass things back as Gameobjects, but we could modify that to be component or mono based (i.e. you could pass back Unity.Mesh instead of gameobject with all the bells and whistles on it. I think any of the default converters for clients like unity or unreal should be as lean as possible, with little to no additional third-party plugins. I was thinking of pushing the point cloud object over to the particle system or storing all them in a computebuffer field that the vfx graph could play with. If down the road I plan to use a third-party plugin for something like point clouds, I would probably specify it for a singular unity project.

From my experience so far with building out different converters for my viewobjs, I’ve always incorporated a reference to a default converter (usually the main Object converter) in order to handle passing data conversions that already handled by the peeps like yourself on the Speckle Team. So I would be curious to know how you might consider a use case like mine being able to incorporate different component based converters?