I’m with the Arup team and working on the v2 connector for GSA. We’ll soon be looking at the v2 speed of sending large models (with say 80,000-100,000 structural objects) as well as their results, which will be tables of data for each of those elements.
Here’s an early model I sent to the xyz server today (which @mishaelnuh will recognise and chuckle that it is still being used):
It’s a small model, and not many results were chosen to be sent; it took around 11 minutes to send this to the xyz server, so on first inspection looks to be slower than v1.
Work is ongoing, and there are many variables at play here, but just flagging that this will be something that we might focus on, and look to collaborate on, in the coming weeks.
(I imagine that @Reynold_Chan will be looking to speed up sending of large data sets to a Speckle v2 server too with his ETABS work)
Is anyone else dealing with transmitting large data sets to/from server transports, or is going to be, for v2?
Not sure what is going on here, but I’ve had a look at your sent data and found some weird things going on. Most of your GSAElement2D contain empty Parent and BaseMesh objects, that could be increasing the conversion/serialisation time considerably (see screenshot)
Could you give us some extra info on what you are trying to send and how?
I noticed there are a bunch of displayMesh instances that contain very little vertices (squares/triangles).
On a different note, I recommend you detach your objects as much as possible (preceding their keys by @) to also speed up the process (and allow us to pull just parts of the model to debug it).
trying to receive that stream now might cause issues as the Objects/Structural schema has changed since I sent that stream, and is still undergoing change
stepping through the code in the GSA connector, I can see that the conversion doesn’t take too long; most of the duration was taken by the Core’s Operations.Send() method - I have encountered some errors from the server when sending larger amounts of result data (which dwarf model data in size in many instances)
data sent from the Design Layer in GSA will, by design (mind the pun), have simplistic meshes; since the 2D elements in the Design Layer only have perimeter nodes and no FE mesh, the converter code has an algorithm that creates simple meshes so that you can at least see something in the viewer when sending from the Design Layer
Regarding your second point, the bulk of the time from when you hit “Send” to when it shows up on the server is actually taken up by serialisation (which happens within Operations.Send before the objects can get uploaded to the server). The conversion and the upload do not occupy a lot of the total perceived “Send” time (unless I suppose you have deployed a very slow server ). Serialisation time will of course increase the more data you send since it requires traversing each of your objects and converting them to a JSON string. We have recently greatly improved the efficiency of the serialiser (thanks @cristi!) so I’d recommend giving the latest version of Core a try
Regarding your third point, it’s up to you if you feel you want to send these small meshes to represent your Design Layer. However, what @AlanRynne was pointing out was that a lot of these objects were not simple meshes, but were in fact empty meshes (0 faces and vertices as you see in his screenshot). This of course will bloat your stream and waste time in serialisation as you’re serialising a bunch of empty objects. At a minimum, I would strongly suggest removing all these empty meshes as they are not providing any visualisation and are only slowing you down.
If you really want to include these visualisation meshes, I would recommend combining them into one large display mesh to represent the whole Design Layer. This will be much more efficient meaning faster sending and smoother performance in the viewer.
Hope that was helpful! Please let us know how you get on with the latest Core
EDIT: If you’d like us to help further, you’ll need to send an updated version of the stream and always let us know what version of the object model you’re using. Otherwise, if we can’t receive the stream there really isn’t much that we can do other than inspect the raw data like Alan did.
in terms of result data that are being sent, I am curious to see how the GSA Connector is handling elements vs member data. From the ETABS side of things, just modeling practice is that elements are drawn as a single member and rarely broken down into single elements at every node as it is GSA. This can get out of hand in particular as I recalled back when it came to 2D elements with occasional manual meshing. If cases like this arise in the conversion process to speckle, there will be inevitable large structural results being sent from a GSA perspective into the analysis model. ETABS might be a lot less since the conventional modeling practice is to model things as a single member and retrieve results for it.
Maybe the solution here is not to send the analysis model that GSA generates and just the design member forces since a majority of other structural programs behave this way? Not sure though whether the community would agree with me on this one.
The main benefit I guess would come to visualization inside rhino/grasshopper, but usually, if we’re doing some cool visualization with the grasshopper, I guess the question to ask ourselves is how much structural results are needed? Let me know what you think!
Cheers for the tip about the latest Core - is there any parallelisation of processing, of serialisation or otherwise? I look forward to trying it.
Any invalid data - meshes or otherwise - are bugs that we’ll fix, they’re not what we ultimately intend to include in the streamed data. Hopefully once they’re ironed out we can see that reflected in the serialisation time.
I hadn’t thought of sending one display mesh for the entire model for the Design layer - that’s a novel idea. Do you mean essentially having a separation between the display and structural 2D elements, so that there is a 2D mesh purely for display purposes and other 2D elements that maintain a valid structural model but which have no display meshes?
Yes, that’s exactly what I’m suggestion regarding your design layer. Creating a single display mesh will greatly reduce serialisation time, sending / receiving time, and the size of your stream. It will also boost the performance (fps / perceived smoothness) in the viewer. Note that 2D elements (eg points and lines) will still be visible in the viewer as they are now.
). Serialisation time will of course increase the more data you send since it requires traversing each of your objects and converting them to a JSON string. We have recently greatly improved the efficiency of the serialiser (thanks 
