Unity plugin help

[eugeneida (Eugene)]

Hi there,

I have some questions regarding the speckle functions. Specifically centering around SpeckleReceiver.ReceiveAndConvert_Async and SpeckleReceiver.ReceiveAndConvert_Routine.

Some background:
I first started out using the ReceiveAndConvert_Async version and basically followed in how it was used in the SpeckleReceiverTests class. Everything worked, when loading test models. However, as we grew the test size, Unity is freezing every time this function is called, and ReceiveAndConvert_Async seems to be not async.

Changing it as detailed below (in SpecklerReceiver), allows Unity to not freeze during the receive period, however it does freeze after, during the convert period.

public async void ReceiveAndConvert_Async(
    Transform? parent = null,
    Predicate<TraversalContext>? predicate = null
)
{
    try
    {
        BeginOperation();
        //Base commitObject = await ReceiveAsync(CancellationToken).ConfigureAwait(true);
        Base commitObject = await Task.Run( async () => await ReceiveAsync(CancellationToken).ConfigureAwait(true));
....

Consequently, I’ve tried SpeckleReceiver.ReceiveAndConvert_Routine, and Unity doesn’t lock up, and everything is responsive. However there is a major drawback.

This is the timing for my model, using the first solution (with the edited async)

As you can see this takes less than 15 mins to load in. However the whole process using SpeckleReceiver.ReceiveAndConvert_Routine is still going on after 1.5hrs. Clearly this is problematic and I am rather stumped.

Questions:

1. Is the edited code alright? Am I missing something? Would you like a PR?
2. Is there a way for ReceiveAndConvert_Async to convert async as well? (I suspect there are some unity threading limitations.)
3. Why is ReceiveAndConvert_Routine taking so long? Is there something that can be done about it?

If you need to, you can DM me and I can share the model for your testing. We have bigger projects as well and I am sure others do too, so I must be doing something wrong. Thanks for helping.

[Jedd (Jedd Morgan)]

Hi @eugeneida

As you’ve likely discovered, running the ReceiveAndConvert_Routine directly as a co-routine is not very performant, since you only get 1 object converted per frame. (I wonder if this function is not so useful then… )

I’ll quickly go over the broad requirements, and then point you towards a method for using co-routines performantly to perform conversion of multiple objects per frame.

1. Conversion needs to be done on unity’s main thread.
   Either using the RecursivelyConvertToNative_Sync (blocks the main thread)
   or using RecursivelyConvertToNative_Enumerable (if setup right with a coroutine, can be spread across multiple frames)
2. Everything else Speckle related, can be on worker threads (e.g. Task.Run and ContinueAwait(false))

---

I assume you are looking for as much of a non-blocking experience as possible?

I recommend doing something like this speckle-unity-vr/Assets/Scripts/Speckle Helpers/VRReceiver.cs at 3d541484c71f2b263940342dadfddffc4ba3051d · specklesystems/speckle-unity-vr · GitHub

We are enumerating over the IEnumerable return of the frameTimeBudget of 14ms, and only yield the coroutine if we have run over our 14ms budget.

We then call start this in a coroutine like so, just after the RecieveAsync is called.

That way, while we download and deserialize objects from the server, we are running that all on worker threads. And when we get to conversion (which needs to be on the main thread), we convert as many objects per frame, without dropping the framerate too low.

You could increase this budget if a faster receive is desired, and if a lower FPS is acceptable.

---

There may be other better ways to achieve this, async await in Unity is not a first class experience. There are a couple third part packages that make dealing with async await a bit easier. See GitHub - Cysharp/UniTask: Provides an efficient allocation free async/await integration for Unity.

Let us know if you can spot ways we can make the unity connector better…

[eugeneida (Eugene)]

Hi Jedd,

Thanks again for your instructive replies. I have managed to follow through quite easily, as I’ve already rewrote the Receiver myself to something similar and needed that little bit to understand that basically its generating 1 per frame.

So initially when its taking more than 1.5hrs to load, its now taking about 20-30 mins on a 100ms budget. This was still not very usable for us, so I whipped out the profiler to chase down the problem.

As you can see above, 50(!)%, or thereabouts as it fluctuates, of the total CPU time was on OnAterDeserialize alone. I have went to comment out the function ( as a quick test) and seem to have discovered the main bottleneck(?)
This is the following profiler screenshot :

There seem to be other possible optimisations as you can see it seems to still take up 30% of total time. However, this single change now makes the model download AND load in 4 mins flat, a huge change to actually being usable for us.

What does this actually do? I see it serialise then only to deserialise later, and I am not too sure of this purpose, since it already stores the data in the dictionary. All speckle properties do appear normally.

On a related note, as I was trying to experiment ways to lower the load times, I realise Speckle doesn’t have a GameObject pool, and it could perhaps help more:

1. Potentially you could offload the gameobject creation time to a less busy or important period, for example, when you are first loading the game, or in some kind of lobby or menu selection screen, and later use the created objects in e.g Converter.Unity.Mesh.MeshesToNative();
2. Created Objects, then can be released for a subsequent loading of another model, instead of destroying then creating them again, which is expensive.
   I tried to do a pool outside of speckle, but I cannot control the creation of the gameobject, so I did a small hack to include a object pool inside ConverterUnity. Results were inconclusive but might be more significant if we decide to move up to more complex models. (Above screenshot is with the object pool)

Would appreciate the input for the previous points, and any further advice or comment, regarding either this or how to proceed.

Thank you for your time again.

[Jedd (Jedd Morgan)]

wow, impressive investigation!

@eugeneida

What does this actually do?

Attaching Speckle Properties is a feature that will write custom properties, revit parameters, and other non-geometry data to the converted unity objects.
With this setting enable, you should see a SpeckleProperties component that exposes a dictionary of these properties.

The OnDeserialzie call back is required to persist this data between scene changes and hotreloads,

Some Speckle models tend to be quite heavy with parameters (e.g. large models from Revit),
so its quite memory intensive to store all this data in the unity scene, as you’ve discovered.

If you don’t care about these properties, you can disable this feature quite easily from the inspector:

[Jedd (Jedd Morgan)]

@eugeneida

have a GameObject pool, and it could perhaps help more:

You’re likely correct, there’s probably several optimisations that we could be doing to increase conversion, particularly for runtime receives.

Please let me know if you experiment further and find significant performance savings by pooling objects.

[eugeneida (Eugene)]

Hey Jedd,

Yes, we’re working on something interactive, so runtime receive load times are important.

Regarding attaching properties:
We would actually prefer to have the properties attached ( I do note the example to load the properties on demand instead), and do not require it to persist between scenes. We haven’t experimented with hot loading yet, and admittedly we’ve just be reloading it. (I’ve noticed the ReceiveMode option in the ConverterUnity class, however I do not seem to find where its implemented so as to switch the mode. Perhaps you could point me to an example?)

The bottleneck seems to be the serialisation rather than the deserialisation (I haven’t investigated the cause) as shown in the previous post.
There are probably workarounds for this:

1. Have the option to not serialise, with the drawback that data doesn’t persist on a reload
2. Not have the serialisation during the loading, and defer it to a less busy period.
3. Similar but slightly different batch load first, then batch serialise later, probably less performant overall as it loops twice instead of once.

As for now, probably we’ll try the “on demand”, or if there’s feedback about the wait time to display information, we might try something else.

These are some sample runs, (just for loading, no downloading, in ms), using the same building I’ve been using to test. Maybe later runs were cached, and the first run was especially slow, so I’d discount it, as well as the second run. Also there is some difference but not that significant as mentioned before
no pool
1 - 32650
2 - 26283
3 - 17173
4 - 17042
5 - 19297

with
1 - 18878
2 - 15347
3 - 17171

Also, I’ve been getting a lot of this message:

Failed to convert Objects.BuiltElements.Revit.RevitWall: System.ArgumentException: Expected at least one Mesh

Is there something that might be wrong with the revit model that we have?

I am doing this to set the property to false

receiver = Manager.AddComponent<SpeckleReceiver>();
((ConverterUnity)(receiver.Converter.ConverterInstance)).shouldAttachProperties = false;

I wonder if this is what I was meant to do I wanted to do it programmatically

[Jedd (Jedd Morgan)]

@eugeneida

Failed to convert Objects.BuiltElements.Revit.RevitWall: System.ArgumentException: Expected at least one Mesh

Likely you have one wall object that has no geometry (i.e. zero meshes)
I think some Revit elements are like this, having no mesh representation. So, unless you’re seeing missing geometry, you can probably safely ignore this warning.