Hey folks, I have some code for an app, where I’m uploading a tree of 250 solids from one stream to another. The specifics of the commit aren’t that important right now, as I’ve had this issue with other large commits - in my logs (attached below), I’m seeing many long calls to https://latest.speckle.systems//api/diff/019def77fa in baches of 4 (threads I assume).
I’ve had the same issue when trying to put many BlockInstances into a diff (1000 or so), where the whole process gets bogged down with diff calls.
It’s becoming a bit of a bottleneck, and I was wondering if there was any light that could be shed onto how to minimise the issue. The final log line says “Finished sending 2735 objects after 54.4186851” which doesn’t seem that huge.
I’m also interested that the calls are taking similar amounts of time to execute, but I won’t speculate on why that might be the case.
For Review - Speckle
partial-logs.txt (16.0 KB)
Thanks for your help!
Sorry this has gone unanswered for so long, @chris.welch .
Is your question, what are some diffs quicker than others, or what is the cause of the slower calls?
In summary, these are all successful publish events, but your concern is that it isn’t quick enough. Is that it?
Yeah specifically it’s that the documentation on the end point suggests baching diff requests in chunks of 1000 ( REST API | Speckle Docs), I’m sending 2500 objects, and it’s making 25 calls to the endpoint.
It’s only an issue when speed is a concern, for example, when trying to quickly update a model that a client is looking at.
Note also that in the example I mention in the first post, I’m copying a tree of solids from one commit to another, so every object will already be created - it’s purely the calls to diff/ that take time
And in the other example, where I have 2000+ objects, they are all object instances, and as such would be ideally much faster to process (as that’s what they are for, for example, a km of railway sleepers)
As an aside, maybe the answer to the “huge array of instances” thing would instead to have a objects type for “instance array” that allowed for tighter control of serialisation
Right, are you making your REST API calls, or is the log output from a Speckle connector? It looks like the latter, but it seems you are deep-diving here, too.
Yep this is just the log output from the dotnet connector out of the box.
1 Like
Hey chris, this comes after i’ve wasted last weekend looking at this. If you’re curious:
Funnily,
What i would literally pay for in whatever currency you want is a sample model.
2 Likes
There’s the polyglot notebook I used as a template → migrateStream.dib (2.1 KB)
There’s the 250 top level Breps where I noticed the lag, it gets increasingly obvious when copying in the same project → Viewer - For Review - Speckle
2 Likes
And here’s an example with a shitload of instances → Viewer - Instances - Speckle
2 Likes
