PowerBI Visual not loading all objects from Stream

Hi Alan, I have a problem with the 3D Visualization in PowerBI.

This is the picture of the speckle stream:

And this is a picture of PowerBI 3D Visualization

For some reason, objects are not positioned how they should.

For this example, I exported the folowing categories from Revit 2022. (Categories: Floors, Walls, Structural Framing and Structural Foundation. 3372 elements.)

In Speckle data there are 3372 Objects
But in PowerBI Data there are 3372 Objects + 3372 Objects.Geometry.Mesh. Is this a desired behaviour?

Best Regards

P.S.:
Stream: Speckle
PowerBI: Microsoft Power BI

2 Likes

Hi @gjordanp,

Thanks for testing our PowerBI connector.

Objects.Geometry.Mesh represents what you see on the viewer. For each data, you have a geometry in viewer. So i guess yes, this is the expected behaviour.

I think there is no problem in positioning of the objects but rather, viewer doesn’t show all the elements. Maybe there is a limit to the number of elements viewer can show? I will leave the final call to the makers👑, @AlanRynne @Reynold_Chan .

Yes it seems that not all elements are showing. I have made another example with a diferente building (smaller one) this time from Revit 2020 and I have the same results.(missing elements)

Also I would like to say that Querys take a really long time. Every time I make a change it would take aproximately 2.5 minutes to refresh an this is a small building (1634 elements)

In spite of that I really like what you have done. This Will be a very nice tool in the future.

Thanks a lot Speckle

1 Like

Hi @gjordanp!

I moved this into a new thread, since the powerBI one is getting a bit unmanageable :slight_smile:

Thanks for bringing this up! To expand on @gokermu’s answer:

For each thing you can see, i.e. a wall, we store the properties of that object (it’s height, level, etc…), and also create a “fallback representation” of that object as a Mesh, that will be used to display that object in any place that does not understand what “a wall” is.

I’ll investigate the missing objects issue, as I suspect we may be hitting some limit on the dataRole input that I was unaware of.

As for the super-slow query speeds. We’re aware of the issue, and have some ideas on how to solve it. Basically, right now the “query” will be complete once we fetch all objects from a stream, one at a time (power query M cannot do multi-threading, so we cannot fetch multiple objects at once).

We’ve got some ideas on how to “fix” (or at least, ease) this problem. The current query returns a table, but we could return a NavigationTable, which has the nice ability to delay some of the queries until they’re needed, meaning you’ll be able to (in theory) get the commit object and navigate through it way faster than the current approach.

Thanks for the feedback :heart: It really helps!

I’ll open an issue for this on GitHub and keep you posted here :wink:

3 Likes

Hi there!

We just released version 2.0.0-alpha3 which bumps up the limit for objects being loaded into a custom visual to 30.000, instead of the default 1000.

It also addresses a critical bug in how the objects were being loaded, so we really recommend you upgrade to the latest version as soon as possible, as you will experience errors when loading in the previous versions from now on.

1 Like

Thanks Alan!! I will try it. Has been any development in improving PowerBI database conector speed?

Regards

We’re hoping to start working on that for the following release, but it’s going to take a full re-factor of the current PowerQuery M code, so it may take a while until we have something viable. :smiling_face_with_tear:

1 Like

Hi Alan,

I’m testing it but takes really long time to load the model.

Here is a video.

Regards

1 Like

Yep, that’s a side-effect of the fix we introduced. To prevent the server from being flooded with requests, we’re now pulling them synchronously, we’ll be increasing the loading speed in the coming releases.

But we had to fix this ASAP to prevent anyone from bringing our servers (or anywones for that matter) offline.

Great thanks! It took a while to load them all, but I still have the same issue (not loading all element). Do i need to try with a new stream uploaded with speckle 2.9.2?

Best Regards

Hey @gjordanp thx for getting back with this! I’ll have to investigate a bit more, I did some initial tests on my side and the component was loading 1k+ items, but there may be some edge cases I was unaware of.

We already have your files so I’ll try to reproduce it sometime today :wink:

No problem! Thanks you and Speckle Team
Regards

Hi @gjordanp, sorry for the delay looking into this. I was out at BILT Europe 2022 for a couple days.

It seems that alpha-3 had the data reduction settings under the values. I couldn’t find any documentation around why/how is it different from adding it to the categories directly, but it appears there are cases where the limit is not raised:

I could reproduce your issue, and after moving the data reduction setting to the categories instead, it seemed to have done the trick.

I would love for you to confirm if this is the case before we release a new version. You’ll find attached the latest build with the fix :wink:

powerbiSpeckleVisualAA98F06515D847E8ACB33BAB487244E0.2.0.0-alpha4.pbiviz (1.4 MB)

Great thanks, I will try it today. I will let you know.

Hi Alan
I test it, but I still get missing elements.
I don’t know if I should update the database, I just updated the visual.
Regards
:sob:

My bad! Now I delete and then create a new Visual, and it works!!!
Great work!!

2 Likes

Awesome!

I was just starting to think I was going mad :rofl: :raised_hands:t3:

I’ll prepare a new release with this fix before the end of the week.

Do keep letting us know any other issues you may experience. I’m hoping to dedicate some time next month to upgrade to the new viewer implementation, and we’re still investigating how the visual should work so any guidance on your preferences is greatly appreciated!!

2 Likes

One feature I will really appreciate on the viewer is the ability look under the model, currently camera only allows to look from horizontal to fully vertical looking down. I would like to be hable to watch te model from its bottom to inspect foundations for example.

Regards

3 Likes