Model checker error

Help
,
1

Hi, I am testing the Model Checker function in automate and got the following error:

[prepare] 2025/04/02 14:52:47 Entrypoint initialization[c8f5484979-e4e81da91f-a6723e] Reporting run status with content: {'projectId': 'e2a7b596f2', 'functionRunId': 'd7fed99ca971de5', 'status': 'RUNNING', 'statusMessage': None, 'results': None, 'contextView': None}[c8f5484979-e4e81da91f-a6723e] It took 2.53 seconds to receive  the speckle version 7b60acf7a6[c8f5484979-e4e81da91f-a6723e] e[92mDetected Speckle object schema version: 3e[0m[c8f5484979-e4e81da91f-a6723e] Traceback (most recent call last):[c8f5484979-e4e81da91f-a6723e]   File "/usr/local/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3805, in get_loc[c8f5484979-e4e81da91f-a6723e]     return self._engine.get_loc(casted_key)[c8f5484979-e4e81da91f-a6723e]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^[c8f5484979-e4e81da91f-a6723e]   File "index.pyx", line 167, in pandas._libs.index.IndexEngine.get_loc[c8f5484979-e4e81da91f-a6723e]   File "index.pyx", line 196, in pandas._libs.index.IndexEngine.get_loc[c8f5484979-e4e81da91f-a6723e]   File "pandas/_libs/hashtable_class_helper.pxi", line 7081, in pandas._libs.hashtable.PyObjectHashTable.get_item[c8f5484979-e4e81da91f-a6723e]   File "pandas/_libs/hashtable_class_helper.pxi", line 7089, in pandas._libs.hashtable.PyObjectHashTable.get_item[c8f5484979-e4e81da91f-a6723e] KeyError: 'Logic'[c8f5484979-e4e81da91f-a6723e] [c8f5484979-e4e81da91f-a6723e] The above exception was the direct cause of the following exception:[c8f5484979-e4e81da91f-a6723e] [c8f5484979-e4e81da91f-a6723e] Traceback (most recent call last):[c8f5484979-e4e81da91f-a6723e]   File "/home/speckle/src/spreadsheet.py", line 161, in read_rules_from_spreadsheet[c8f5484979-e4e81da91f-a6723e]     df = process_rule_numbers(df)[c8f5484979-e4e81da91f-a6723e]          ^^^^^^^^^^^^^^^^^^^^^^^^[c8f5484979-e4e81da91f-a6723e]   File "/home/speckle/src/spreadsheet.py", line 57, in process_rule_numbers[c8f5484979-e4e81da91f-a6723e]     where_indices = df[df["Logic"].str.upper() == "WHERE"].index[c8f5484979-e4e81da91f-a6723e]                        ~~^^^^^^^^^[c8f5484979-e4e81da91f-a6723e]   File "/usr/local/lib/python3.11/site-packages/pandas/core/frame.py", line 4102, in __getitem__[c8f5484979-e4e81da91f-a6723e]     indexer = self.columns.get_loc(key)[c8f5484979-e4e81da91f-a6723e]               ^^^^^^^^^^^^^^^^^^^^^^^^^[c8f5484979-e4e81da91f-a6723e]   File "/usr/local/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3812, in get_loc[c8f5484979-e4e81da91f-a6723e]     raise KeyError(key) from err[c8f5484979-e4e81da91f-a6723e] KeyError: 'Logic'[c8f5484979-e4e81da91f-a6723e] e[92mFailed to read the TSV from the URL: 'Logic':'Logic'e[0m[c8f5484979-e4e81da91f-a6723e] Automation run EXCEPTION after 3.28 seconds.[c8f5484979-e4e81da91f-a6723e] Failed to process rules[c8f5484979-e4e81da91f-a6723e] Reporting run status with content: {'projectId': 'e2a7b596f2', 'functionRunId': 'd7fed99ca971de5', 'status': 'EXCEPTION', 'statusMessage': 'Failed to process rules', 'results': None, 'contextView': '/projects/e2a7b596f2/models/49108465fb@7b60acf7a6'}

I have used the template rules without any change and used the following public google drive link: Copy of QAQC Checker Rules - template - Rules.tsv - Google Drive

Can you please help?

On another topic, can we share it to a specific single address that need access so it’s not public?

Thanks,

2

Thanks for checking this out. We are seeing a repeating issue with making this work.

When you made a copy of the Sheet, did you see a Speckle menu appear?

image
image1380×660 90 KB

I suspect assumptions about what non-Google Sheets users have access to were wrong.

You can publish a TSV from Google Sheets - not share the file.

image
image850×1171 61 KB

While we were motivated to be as light touch as possible and so use Google Sheets, this is becoming a blocker. To resolve this, I have an early alpha of a UI application (beware, it’s slow)

image
image1917×896 72.5 KB

image
image1921×1088 178 KB

image
image1915×1770 245 KB

This will also resolve your question:

can we share it to a specific single address that need access so it’s not public?

3

Hi Jonathon,

Thanks for your prompt reply, the Speckle menu does not show on google sheets for me. I will try the alternative option,

Regards,

1 Like
4

Keep the questions coming :cool_spockle:

5

Hi Jonathon,

I managed to find the speckle bar in Google Sheets, which needs activated, but the access requirements are a bit scary as it needs access to all files.

image
image459×270 17.8 KB

On another hand I also tried the web application, which seems great but I can’t see how to use it in a project?

Thanks,

1 Like
6

Hey @bmc that’s fair and a sad side effect of the Apps Script just needing to make a copy of a file into your drive - you can of course do all these steps manually.

I’m confident a bit of polish and the model checker UI will be the way forward.

If it is not immediately apparent how to use the Model Checker in a project context, then we have work to do to help you.

Model Checker is for people who:
:white_check_mark: have models
:white_check_mark: have data in those models
:cross_mark: can’t be 100% sure that data is top notch

If that’s you, you’re in the right place. All we need to do is define a few rules you care about, and we’ll help you turn them into automated checks.

Here’s one way to get started:

  • Think of a rule you’d normally check manually. Something like: “All light switches must be 1.2m above floor level” or “Every element must have a classification assigned.”
  • We’ll help you turn that into a ruleset. Once it’s set up, you can run it automatically on your models every time they change.

Would you be willing to share one or two of the checks you typically run on a project? We can turn those into a starting point for a function you can use immediately in Automate.

7

Hi Jonathon,

That makes sense, sorry I should have been more clear. I managed to create a rule but was not sure how to run it in the model checker. Since then I tried some buttons and figured that I needed to ‘share’ and then copy the link into the automation :smiley:

image
image484×194 13.4 KB

Now that it works, I looked in the guidance here(GitHub - specklesystems/speckle-automate-checker) and while trying some random checks for different scenarios I am not getting the checks to return the expected results.

E.g. From these rules:

image
image1170×343 46.5 KB

This is what I get:

image
image271×492 30.3 KB

While I was expecting to get:

rule # 2 it’s correct to match 1 element but why does it say skipped?
rule # 3 it’s correct that 1 element has mm. But should not this return back all the ones that do not contain mm?
rule # 4 to return 5 incorrect.
rule #5 to return 8 incorrect.
rule #6 to return 50.

What am I doing wrong?

8

When you ask what you’re doing wrong, I turn the question around to how I can better demonstrate it.

I’ll break it down a bit (though I’m not sure exactly which model you’re using):

Rule 2: Why is it marked as skipped but with one element?

This is a quirk we’re aware of — the only way I’ve been able to get a rule to report as “skipped” is when it matched an object that doesn’t exist (i.e., the rule matched 1 nonexistent element). We have a ticket in the backlog to improve how this is handled and shown.

That said, it’s worth checking: should it be matching anything?

You’re strongly matching on parameters.Classification.Uniclass..., but the correct path would likely be:

Properties.Parameters.Type Parameters.Classification.Uniclass...

Rather than get caught up in the full path, try something like:

Classification.Uniclass.Pr.Number

This will search more broadly and is often enough.

Rule 3: Why doesn’t it show everything that doesn’t contain mm?

This rule is working correctly based on how Automate works. You’re seeing two result groups:

  • :white_check_mark: 1 element that does contain mm
  • :warning: 14 that don’t, flagged as Warning

So of 15 doors, that checks out. What might be confusing is what’s displayed — when setting up the automation, you can configure it to only report Warnings and Errors (not Passes), which might be why you’re not seeing the whole picture at first glance.

I’m open to better ways of showing this in the UI. Suggestions are very welcome!

Rule 4 (and Rule 5): Why more than expected?

This follows the same logic as Rule 2. Of the 15 doors, none have a value in parameters.COBie.Type.Name, so technically none contain DOR- — meaning all 15 are flagged as failing the check.

If you want a more straightforward rule that catches this better, try:

COBie.Type.Name

…without the full path prefix. Same goes for Rule 5 (COBie.Component.Name).

Rule 6: Expected 50, got 34?

Try simplifying the rule to just:

materialName CONTAINS Wood

Depending on your model, the full path (properties.materialName) might be over-specific. It’s often best to start with a loose match and tighten it only if needed. However, I will check this as it finds 34 things.

Let’s jump on a quick call or walk through one together. I’m always happy to iterate on this with you. And again, thanks for digging in; this kind of feedback is gold.

9

Hi Jonathon, happy to go on a call. I did simplify the rules and returns the same values.