QGIS Connector WIP

Making Speckle
41

In case someone is interested, I created a pull request for sending raster layers around: WIP: send raster layers by tlmnrnhrdt · Pull Request #13 · specklesystems/speckle-qgis · GitHub

I am creating 1mb chunks if possible as discussed here: Uploads Max File Size - why is that value chosen? - #5 by dimitrie

3 Likes
42

Hey @tlmn this is great :slight_smile:

I’ll be doing some QGIS work myself this week to improve the UI so I’ll also check this out! Had a quick peek now and it looks good :+1:t3:

I’m interested to explore how this would play out with other connectors (specially Grasshopper/Dynamo) and hear about what workflows (if any) would this enable that were not possible before.

As I understand it, we may need to have some sort of conversion logic for it to “understand” and display what this Raster data is… similar to how we deal with BIM objects. So there’s a deeper discussion to be had here on supporting GIS data across Speckle and how would that look like.

2 Likes
43

Currently we are sending a height map (raster data with some real heights attached to it) as a string in the native format. The user takes that string in GH and converts it into a terrain mesh (reducing the resolution).

We could also imagine displaying that data in GH as an image (as done in QGIS), but then we would need some kind of raster band(not sure if that´s the right term?!) with a color <-> height (or any other raster data) mapping.

So I would see two possibilities atm:

  1. Sending generic raster data with 1…n bands (?) to be displayed as an image in GH
  2. Sending some speckle object that specifically represents a height map that can be directly converted into a mesh (which might be ending up way to big without reducing the resolution)

Maybe solution 1.) with the additional information that it´s about heights might be reasonable?

2 Likes
44

Hi everyone,

since I saw this post on twitter lately (https://twitter.com/SpeckleSystems/status/1465753964230721551) I wanted to ask if there has been some recent development that hasn´t made into the repository yet?

Is somebody working on new features in general and what are the plans for the near future?

We thought that it could be quite useful to add the functionality to pick a specific branch for sending receiving.

1 Like
45

Hey @tlmn! Nice to hear from you

I’m actually wrapping up some things to finalise a “stable release” of QGIS https://github.com/specklesystems/admin/issues/216

I’ve been implementing the Receiving part of things, and updated the UI with better controls (including send/receive to/from a specific branch).

This was already doable before, just by pasting the URL of the branch, but now it will be easier to manage.

I also just commented on your PR (sorry I had it a bit abandoned…) to check if there was anything else pending to move it out of WIP status, if not, i’ll be happy to merge it in right away so it will be available on the following release :slight_smile:

Once we have sending/receiving working, plans will depend on community feature requests :wink:

2 Likes
46

Sounds awesome!
I am sure we can provide you with feature requests in the future… :wink:

2 Likes
47

Do keep them coming!! We also accept “hate mail” when things don’t work :wink: :raised_hands:t3: Thx for the great contribution by the way!

1 Like
48

Hi there, thanks for you great work!

I admit I still have to properly test the connector (especially the latest update), but I would like to share my use case.

I’m using QGIS as a base to prepare/collect data in urban environments (buildings, structures, utilities) near a planned tunnel excavation; the data is then exported (via CSV=>Excel with WKT geometry :fearful: ) and used in an internal python tool that performs a Building Risk Assessment and outputs Excel, SHP and DXF files for further analysis and reports.

The option to move away from Excel data exchange has been in the to do list for ages, and I planned to build a qgis plugin for this. Now that I found you :heart:, I suppose I can exploit your work (and help you as much as I can, of course!)

This is how the connector can be useful:

  • import the alignment from Civil3D data (or other CAD software, the important thing is to know the CRS used);
  • Import available CAD drawings geometries (mostly buildings - now it’s a manual task using QGIS dwg/dxf import)
  • Integrate Excel data: usually my geotechnical colleagues are not fond of working in QGIS, and are happier editing the exported Excel file to update structures information (building heights, foundations elevation, vulnerability classes, and so on), but these results in data drift.
    Our current excel “data model” is to have one layer (type of structure) per excel sheet (since they have slightly different parameters to populate) - we usually have the code attribute to uniquely identify a structure (and allow us to do data joining).

We also planned to implement other workflows to bring GIS and BIM data together and have better sense of the project constraints; I suppose they will mostly use the sender part of the connector, but a 2 way data flow is not to exclude. I will report new use cases as soon as I discover them.

From my first tests I ran into some troubles (QGIS 3.20):

  • I tried to receive the alignment sent by Civil 3D connector in an empty QGIS project, but nothing comes up (speckle log says "successfully received "). I suppose creating new layers/features is not yet supported, isn’t it?
  • I created a new branch in the web app to send some QGIS feature, but the (empty) branch doesn’t show in the branch dropdown, even after clicking “Reload” multiple times - I had to close and reopen QGIS
  • QGIS geometries are successfully received by Civil3D connector, but layer names are somewhat cryptic
    image
    image728×114 7.02 KB
    • linestring features are in the first layer
    • (multi)polygons are both in $boundary and $displayValue layers (as closed polylines)
  • I tinkered a bit with the QGIS <-> Excel round trip without success, but then I saw this discussion and realized that the excel connector is not made for property updates! I’ll try to force my colleagues to use QGIS for the time being :stuck_out_tongue:

That said, keep up the good work!

7 Likes
49

Welcome to our community and thanks a lot for the feedback @sanzoghenzo !

It’s really helpful when people explain their use cases in detail so we can tailor our testing and future planning to their needs.

I have to admit the QGIS <> Civil3D is not a flow that we have tested much - if not at all, but it sounds like the issues you’ve found might be simple fixes on our end! @AlanRynne and @clrkng can track these internally and we’ll let you know when they’re fixed.

In regards to Excel, that thread is quite Revit specific and it might be indeed possible to update some QGIS properties. Do you have a very simple model or stream for us to test with?

3 Likes
50

Thanks @teocomi for you reply!

Here I created a stream with some buildings of my town downloaded from OSM.

Somehow I’ve managed to mess the features attributes, but it can be a good example of the use case: being able to make edits to the attributes in excel and send it back to QGIS.

Unfortunately the data chunks and __closure fields don’t play well with this kind of task!

For the Civil 3D → QGIS part, if I understood the current code correctly, the qgis plugin only looks for Layer objects (and its children) from the stream, so it only works with its own data (and indeed I was able to copy the polygons to an empty qgis project).

For my specific use case, I don’t care much if the received alignment(s) will be placed in a new or existing layer, but I can imagine some cases where, e.g., CAD layers need to be transposed to qgis layers - if they have only a single geometry type!

1 Like
51

Hello,

I am trying to install the speckle plugin for QGIS.
Running into the same issue than @tlmn
I did pip install specklepy. Is it because my version of QGIS is too recent ? Currently installing 3.16.13

image
image1162×711 77.5 KB

2 Likes
52

Hi @pauldotnet

Are you trying to install the connector via the package manager from QGIS? or directly from code?

We added a dependency installer that should check if specklepy is installed upon load and if it isn’t it should install it itself (or upgrade it if needed)

Not sure about the QGIS version, I use 3.22 but I’m aware Kateryna does use 3.16 so in theory it should be fine with both.

If you’re installing the connector directly from the github repo, try installing the latest main branch.

EDIT: Also, what system are you running QGIS from?

53

thanks @AlanRynne for the instant reply. I tried to install from QGIS’ package manager. Will try from the github repo then, hopefully the dependency installer with solve this.

If you mean Mac or Windows, I am using Windows

54

I’ll try to debug on your specific version of QGIS to check if it has anything to do with that.

The latest version in the QGIS package manager should have the dependency installer… so I’m leaning towards considering this an unknown bug for now. Thanks for bringing it up.

On a sidenote, just to try to pinpoint the issue a bit more. In windows there are several ways to install QGIS afaik, I know of:

  • Using the Standalone Version installer
  • Using the OSGEO4W installer
  • Maybe there’s more?

Which one did you use? This will help me reproduce your steps and hopefully get the same issue

55

Thank you for investigating further. I installed QGIS from this Standalone

2 Likes
56

Hi @sanzoghenzo ! Sorry for a long time to respond. We are working on a new release right now which would allow receiving data from other software than only GIS. Would you mind sharing a sample of the data you sent from Civil3d, here or in PM? If you can make me a stream collaborator for existing or a new test stream would be great!

57

Hello, I’ve reinstalled QGIS and and still getting the same error as posted above. Any new tips for bypassing this ?

58 
ModuleNotFoundError: No module named 'specklepy' 
Traceback (most recent call last):
  File "C:\PROGRA~1/QGIS32~1.7/apps/qgis-ltr/./python\qgis\utils.py", line 423, in _startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "C:\Users/PPoinet/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\speckle-qgis\__init__.py", line 51, in classFactory
    from speckle_qgis import SpeckleQGIS
  File "C:\PROGRA~1/QGIS32~1.7/apps/qgis-ltr/./python\qgis\utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "C:\Users\PPoinet\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\speckle-qgis\speckle_qgis.py", line 25, in 
    from specklepy.api import operations
  File "C:\PROGRA~1/QGIS32~1.7/apps/qgis-ltr/./python\qgis\utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'specklepy'


Version de Python : 3.9.5 (tags/v3.9.5:0a7dcbd, May  3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] 
Version de QGIS : 3.22.7-Białowieża Białowieża, 5a65627a 

Chemin Python :
C:\Users\PPoinet\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\speckle-qgis
C:/PROGRA~1/QGIS32~1.7/apps/qgis-ltr/./python
C:/Users/PPoinet/AppData/Roaming/QGIS/QGIS3\profiles\default/python
C:/Users/PPoinet/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins
C:/PROGRA~1/QGIS32~1.7/apps/qgis-ltr/./python/plugins
C:\Program Files\QGIS 3.22.7\bin\python39.zip
C:\PROGRA~1\QGIS32~1.7\apps\Python39\DLLs
C:\PROGRA~1\QGIS32~1.7\apps\Python39\lib
C:\Program Files\QGIS 3.22.7\bin
C:\PROGRA~1\QGIS32~1.7\apps\Python39
C:\PROGRA~1\QGIS32~1.7\apps\Python39\lib\site-packages
C:\PROGRA~1\QGIS32~1.7\apps\Python39\lib\site-packages\GDAL-3.4.3-py3.9-win-amd64.egg
C:\PROGRA~1\QGIS32~1.7\apps\Python39\lib\site-packages\win32
C:\PROGRA~1\QGIS32~1.7\apps\Python39\lib\site-packages\win32\lib
C:\PROGRA~1\QGIS32~1.7\apps\Python39\lib\site-packages\Pythonwin
C:/Users/PPoinet/AppData/Roaming/QGIS/QGIS3\profiles\default/python
2 Likes
59

Hey @pauldotnet,

Could you try installing specklepy directly from your QGIS python environment? Just to ensure it is, in fact, being installed.

I cannot manage to reproduce your issue with your exact version, so I’ll need to look into it a bit.

Quick question, once QGIS opens (and after that exception), do you have a “Speckle” message panel in QGIS? And if so, does it show the message?

If it doesn’t, it may be that the “auto-installer” we included is not doing the trick for you for some reason. I’m going to look into this further, but i’d be happy to schedule a call to look into your particular case as I’m sure you’re not the only one affected by it, just the only one that brought it up :raised_hands:t3:

And sorry for the late reply on this! It fell through the cracks of Discourse :sweat:

2 Likes
60

Hi Alan,
Many thanks for the answer.
I finally successfully installed the plugin after pip install in the command prompt. I thought I tried last time but didn’t work back then. So yeah, not sure why but in my case Speckle hasn’t installed it automatically.
All good!

Sorry, couldn’t replicate this one after the plug-in got installed :stuck_out_tongue:

2 Likes