Differing Behavior Between Sender, Sync Sender and Objects for GH Trees

[zb_anderson (Alex Anderson)]

I wanted to post here first to make sure I was using these GH modules as expected before opening an issue on Github. We are evaluating the use of speckle for our GH workflows now that it is supported via Rhino Compute (this was an exciting update for us). But I have run into an issue when sending data that includes data that is a single item or list along with data that is wrapped in trees. Our workflow requires the use of trees so we don’t have an option to get rid of them as a work around.

The problem I am running into though is that the Sender, Sync Sender, and Speckle Object blocks seem to behave differently when building a speckle object. I can send a mixed set of data (single objects and a tree for example) via the inputs on the Sender block and I can get back data that is the same format that I sent it in. But I cannot use this block in compute, even with the auto-sending on, since it the compute instance closes the operation before the Sender module can execute its commit because it is non-blocking.

That means I need to use the Sync Sender but if I send this data either via directly piping the data into the data input (D) on the Sync Sender or by building a speckle object before I pass the data into the Sync Sender I get really weird output data structures that seem to nest even single object multiple times in trees. I have tried every combo I can think of to build the speckle object to try and retain the right structure (separate speckle objects, list vs no list input, detached, vs not detached, object groups, etc) but I still end up with garbage output that is multiple tree layers deep with repeated inputs which results in ballooning file sizes of some commits with heavy meshes being repeated hundreds of times.

To summarize, it looks like the regular Sender module has access to trees in GH but neither the speckle object or Sync Sender block have the same level of access. Below is an image of an example setup along with the GH file if that helps with debugging. I am also running Rhino 7 and speckle 2.14.

SpeckleDebug.gh (19.0 KB)

What I am looking for is clarification that I am going about testing these other object packaging methods for the Sync Sender correctly, and if I am then I can go and open an issue on Github because this difference between these approaches is hamstringing our implementation of speckle. If I am using them incorrectly (I hope that I am ) then please let me know and I will try a different approach ASAP.

[AlanRynne]

Hey @zb_anderson!

Nice to see you poking at our Grasshopper connector! The answer to this has much more to do with Grasshopper data tree handling and Rhino.Compute than it does with Speckle. And what you’re describing is in fact the intended behaviour.

I’ll go one at a time

Create Speckle Object (or extend…)
The data access of these nodes is modelled from the Python/C# script approach (where it’s the user responsibility to set their access between item, list and tree).
The only caveat is that our Object nodes only support Item and List access. This is intended, as allowing it will in de-facto allow “DataTrees within DataTrees”, which is not a concept supported in Grasshopper.

As such, you can only attach an individual value (item access) or a list/branch of values (list access).

This is not a limitation of Speckle, but rather a design decision for the Grasshopper Connector. If you know what you’re doing, you can still send them with some extra work. But we don’t want to incentivise people to start doing so, as it was not designed for this and there will be unintended side-effects.

So, in your case, when you’re plugging a data tree with List access, Grasshopper will run the component once for every branch in the tree and will repeat any other inputs as needed.
This is what leads to your massive duplication of items in the list and single inputs of your example

FYI, even if we supported Tree access, the data repetition issue would still exist if you were to, for example, input more than one item on the single input, as this is the way Grasshopper operates in it’s core.

Difference between Sync Send/Receive and normal Send/Receive

For these, I think you’re conclusion is not correct. All send/receive nodes have their data inputs set to tree access. Meaning it will gobble up all data inputed in a single iteration (because we want to “send just once”, and not allow GH to possibly create thousands of send operations)

The difference you’re experiencing is also by design:

Normal Send node has the ability to set multiple data trees as inputs, which the Normal receive will “auto-expand” (this option can be disabled). But when operating on Rhino.Compute, you neither have the option of “creating new inputs”, nor you can “rewire the receiver with more outputs”, as you’re running it in the background. This makes the “multiple input/output option” of the normal send/receive nodes turn from a feature to a problem (you modify the inputs of the Send node, and your Rhino.Compute script will suddenly break, or have an input that’s empty… etc)

So the Sync send/receive nodes are designed with a single input/output DataTree to prevent this particular scenario.

Sorry for the long reply! Couldn’t make it any shorter

Hope this helps get you going! But happy to discuss more your use-case and how we could help unlocking the use of Speckle in your practice!

[zb_anderson (Alex Anderson)]

Alan, thank you for the detailed response! As with almost any tool like this, there usually is a reason behind these decisions that makes sense once it is explained.

In our particular use case we are sending the same data as outputs/inputs for each compute operation so unless something more serious breaks, we are able to get the correct outputs/inputs every time. Or put another way, I can send the proper trees using the regular sender module, then receive them with they sync receiver while inside of compute and it auto-expands with all of the expected outputs.

A work around for this I have found (to populate the correct inputs for a sync receiver in a compute targeted script) is to make a separate commit (in either a different branch or repo) with the expected inputs and outputs but will all the values as null. I can then use this “dummy” commit to build my grasshopper script since the sync receiver will auto-expand to include all the inputs/outputs of that specific module even though they are null at the time. Then when I run my compute script I replace the default commit url which is pointing to my placeholder commit, with an updated url, via compute’s/grasshoppers “get string” function, that has the same named values but is populated with live data. This also allows myself and our other designers to take a look at the placeholder commit (which also has a version number tied to it) and see what the inputs and outputs of a specific compute block are expected to be, similar to how a docstring for a function would indicate to a developer what the expected inputs and outputs are. I understand this is a bit of an odd setup but we are working with very complicated grasshopper workflows multiple compute sequences chained together, anywhere from 2 to 8 distinct operations.

In summary though, it seems like there currently isn’t a way of sending a mix of single, list, and tree packaged data in a single commit in a compute context? Would that be a correct statement to make?

Thanks again for all your insight and help!

[AlanRynne]

For now, if you want to achieve something similar to this, you can do so by playing a bit smart on the GH+Speckle combo.

This is where we started

Option 1

Collapse the data tree into a list. For this, you can input the data tree into a CSO component with a single input set as list access. This will create one object per branch, leaving you with a flat list of objects, each representing one branch.

Since this is a flat list, you can then connect it to the tree input (set in list access also) and all should be good

On receive, you’ll need an extra Expand Speckle Object component to retrieve the branch info.

Option 2

Manually creating the tree object

This is basically replicating what we do internally. It’s fairly similar to the previous one, with the added advantage that it preserves the data tree branch indices

I cannot stress enough that this should not be mis-used!! It should be OK though, if you do it as a final step to collapse data trees just to plug things to the send node. And only if you plug in one tree per input or things will get messy on the receiving end

You can create a single object for the entire data tree, each key of this object should be the branch identifier, and the values should be the items within each branch.

Using Create Speckle Object by Key Value and the Speckle Group components, it’s rather easy to achieve.

Here’s a sample file I made to get this working I’ll admit this may not be very “user friendly” but it should work! I’m currently re-thinking these assumptions based on our conversation, so I’ll keep you posted (and may even ping you for some more feedback on how you would like it to work)

SendingTreesInCompute_Workaround.gh (35.1 KB)