GraphQL: how to query a speckle object

Hi,

I am new to GraphQL and Speckle as well. I am trying to figure out how to write a query that allows me to filter the data I want. I use the native Speckle’s GraphQL manager online. I manage through some examples to query a certain stream with a certain object:

query {
  stream(id:"c6b0c4077a"){
     	id
      name
    object(id:"0d0a4abc6a5fcc763e6c850dd3d5ecab"){
      id
      createdAt
      totalChildrenCount
      children(limit:5){
        cursor
        totalCount
        objects{
          id
          data
        }
      }
    }
  }     
  }

However, under the “children” and down the graph, I am unable to figure out how to filter anything. It looks like the “children” object could allow for some filtering capabilities, but I am not able to figure it out.
Below is some response data from the query above. The data is a stream from Revit for a Standard Window. What I want to do is to be able to do a query on a certain “applicationId” for example. However, I haven’t figured out how to write such a query, since it does not seem like I can write a filter clause for what ever field I want. Of course I would be able to run through the response and in my application and filter it, but I want to be able to do that directly through the query itself.

"id": "017db146c79cbd1d7acbea6b82dc6455",
              "data": {
                "id": "017db146c79cbd1d7acbea6b82dc6455",
                "type": "Standard",
                "level": {
                  "id": "03504aabeb160292ff17c88f13d6bafe",
                  "name": "Level 1 Living Rm.",
                  "units": "mm",
                  "elementId": "515270",
                  "elevation": -550,
                  "createView": true,
                  "parameters": [
                    {
                      "id": "b9d966405c3d7b3cfcb54985b088e8bc",
                      "name": "Building Story",
                      "value": true,
                      "isShared": false,
                      "isReadOnly": false,
                      "speckle_type": "Objects.BuiltElements.Revit.Parameter",
                      **"applicationId"**: "LEVEL_IS_BUILDING_STORY",
                      "revitUnitType": "UT_Number",
                      "isTypeParameter": false,
                      "totalChildrenCount": 0
                    },
1 Like

Hi @henkon & welcome to the forum!

Yes, I think we’re due to write some more docs on how to use the api :sweat_smile:, in the meantime you can have a look at our tests for samples on how to query the gql data. The one below should be doing almost exactly what you’re after:

Hi, first of all thanks for your reply. I actually did look at that example. Unfortunately I could not quite grasp the syntax, or rather what actually goes into [JSONObject!], which seams to be assigned to $query variable that is used to filter in the example. I kind of interpreted that the query needs to be formatted as json?, so I tried several versions of the following change
children(limit:5, query:"{applicationId: ‘LEVEL_IS_BUILDING_STORY’, type: ‘Parameter’}" )

Did not help much. I get the error:
“Expected type [JSONObject!], found “{applicationId: ‘LEVEL_IS_BUILDING_STORY’, type: ‘Parameter’}”; Cannot read property ‘forEach’ of undefined”

I tried to add brackets in my “json”, since I thought it might expect an array, but that did not work either.
Any help would be appreciated.

Hi @henkon! We desperately need to create some legible examples on querying. There’s probably a lot we still need to do on this aspect, as we haven’t “flexed” these parts of the server too much yet.

Specifically to your error, that’s coming because the query parameter expects to be a valid json string - and it’s not: you will need to quote all property parameters too, not just their values.

I’m going to add some examples against this commit’s referenced object. I’m starting with selecting specific fields (it’s easier), and I’ll move to actual queries.

Selecting Specific Fields

Using the select clause is super easy: it expects a string array.

query{
  stream(id:"7b253e5c4c"){
    object(id:"f4a0e0922fff27bc884ad149c828a9c8"){
      totalChildrenCount
      children(select:["type", "family", "category"] limit:100 depth: 100){
        totalCount
        cursor
        objects{
          id
          data
        }
      }
    }
  }
}

You can also use dot notation inside the select clause to select specific subfields. Below I’m asking for the level name and elevation of each object. Note: since we’re querying an object graph, some might have these properties, and some might not.

query{
  stream(id:"7b253e5c4c"){
    object(id:"f4a0e0922fff27bc884ad149c828a9c8"){
      totalChildrenCount
      children(select:["type", "family", "category", "level.name", "level.elevation", "level.id"] limit:100 depth: 100){
        totalCount
        cursor
        objects{
          id
          data
        }
      }
    }
  }
}

Here’s what an object returned by the query above looks like:

{
  "id": "03a7b6b5fb8328d6372fba11c4827433",
  "data": {
    "type": "φ1100",
    "family": "PHC",
    "category": "Structural Foundations",
    "level": {
      "name": "設計GL",
      "elevation": -100.00000000000108,
      "id": "8f6a0b889cb29b5a52da5573543fa95c"
    }
  }
},

Querying

The query parameter expects a json object - which essentially translates to a nicely formatted json string representation of it. Depending on the client you’ll be using, you might get away with passing a normal js object, or you might need to call JSON.stringify() on it. If you pass it directly in the query itself, and not as a query variable, the latter is a given.

Note that the query parameter does, as you said, expect an array of JSONObjects. What is the format of that JSON Object? Read below.

For example, to get all the elements that are on the level called “5FL” - which I assume in that model corresponds to the “5th floor”, you need the following query:

{
  "myQuery": [
    {
      "field": "level.name",
      "value": "5FL",
      "operator":"="
    }
  ]
}

The anatomy of an object is rather simple: what field you want to operate against, what value of that field you want to reference against, and with what operator.

How do you pass this query in then? That’s a bit more tricky. The best practice way is to add it as a query variable. For this, we’ll need to edit the top of our original query a bit:

# notice how we've declared up here the 'myQuery' variable as having the same type as the one of the query field in the children node
query($myQuery: [JSONObject!]){
  stream(id:"7b253e5c4c"){
    object(id:"f4a0e0922fff27bc884ad149c828a9c8"){
      totalChildrenCount
      # ... and note how we're using it here
      children(query: $myQuery select:["speckle_type","type", "family", "category", "level.name", "level.elevation", "level.id"] limit:100 depth: 100){
        totalCount
        cursor
        objects{
          id
          data
        }
      }
    }
  }
}

To actually pass it in, if you’re using the graphql playground, down below you have a tab called “Query Variables”:

And voila! You’re now querying for all the objects on the 5th floor! Please note that for numeric values, you do not need to have them in quotes!

For example, let’s say we want to get all elements that are on a level with an elevation higher than 16900 (mm in this case), the query would become:

{
  "myQuery": [
    {
      "field": "level.elevation",
      "value": 16900,
      "operator": ">="
    }
  ]
}

What about multiple constraints? Easy! Let’s say we want to get all the columns that are on the 5th floor level, the one we used earlier. The query then becomes:

{
  "myQuery": [
    {
      "field": "level.name",
      "value": "5FL",
      "operator":"="
    },
    {
      "field":"speckle_type",
      "value":"Objects.BuiltElements.Column:Objects.BuiltElements.Revit.RevitColumn",
      "operator": "="
    }
  ]
}

There’s more to querying but I’m a bit out of breath for now. I’ll move to another important aspect, pagination.

Pagination

The query above was limiting the number of return objects to 100. Using the same query, I’m going to restrict it to 5 objects only for demonstration purposes:

query($myQuery:[JSONObject!]){
  stream(id:"7b253e5c4c"){
    object(id:"f4a0e0922fff27bc884ad149c828a9c8"){
      totalChildrenCount
      # note the limit param below! it's only 5 now.
      children(limit:5 query: $myQuery select:["speckle_type","type", "family", "category", "level.name"] ){
        totalCount
        cursor
        objects{
          id
          data
        }
      }
    }
  }
}

What you get back is the following:

{
  "data": {
    "stream": {
      "object": {
        "totalChildrenCount": 6625,
        "children": {
          "totalCount": 23,
          "cursor": "eyJmaWVsZCI6ImlkIiwib3BlcmF0b3IiOiI+IiwidmFsdWUiOiI0MjBjZDUwZjA0OTE0MTVlNDE1MTZiZjJiY2VmZjQ3OSJ9",
          "objects": [ ... ]

You can see that your query’s total matching result is 23. That’s why the API provides you with a cursor that allows you to get the next “page” of results. It’s essentially an opaque index to your previous end position. To get the next set of results, you just need to pass in that cursor to your query parameters. This would look something like this:

query($myQuery:[JSONObject!]){
  stream(id:"7b253e5c4c"){
    object(id:"f4a0e0922fff27bc884ad149c828a9c8"){
      totalChildrenCount
      # note the addition of the cursor. 
      children(limit:5 cursor:"eyJmaWVsZCI6ImlkIiwib3BlcmF0b3IiOiI+IiwidmFsdWUiOiI0MjBjZDUwZjA0OTE0MTVlNDE1MTZiZjJiY2VmZjQ3OSJ9" query: $myQuery select:["speckle_type","type", "family", "category", "level.name"] ){
        totalCount
        cursor
        objects{
          id
          data
        }
      }
    }
  }
}

If you’re doing a frontend application, it’s usually good practice to keep track of all your previous cursors, so you can easily go “back” to the previous page. Most importantly, it’s usually good practice to not request all the objects at the same time as this may result in really large payloads being sent to the client - we don’t want that, it makes your app feel slow!

As a bonus for reading so far, here’s the model I’m using in the examples above:

4 Likes

Hi @dimitrie
Thanks a lot, I can’t complain about that answer, and I guess it will be of help for others as well :slight_smile:

I have played around with what you wrote, it all makes sense. I found the “Query Variables” tab, though hard to spot. I managed to re-create all you have done.
I still however, after following your instructions, do not get the query that I am trying to apply to work properly. No errors, just don’t get back data I know is there. I will continue to test to see if I have done a mistake. I will however paste in what I have done, in case you are able to quicly review it. Slightly embarassing, I was sure I would manage with the great explanations, once again thanks!
So, just to quickly recap, what I am trying to get as response, is one “speckle_type”: “Objects.BuiltElements.Revit.Parameter” with a certain applicationId

query($myQuery: [JSONObject!])
{
  stream(id: "c6b0c4077a") {
    id
    name
    object(id: "0d0a4abc6a5fcc763e6c850dd3d5ecab") {
      id
      createdAt
      totalChildrenCount
      children
      (
        query:$myQuery 
        select:["parameters.id", "parameters.name", "parameters.value", "parameters.speckle_type", "parameters.applicationId"]
      )
      {
        cursor
        totalCount
        objects {
          id
          data
        }
      }
    }
  }
}

{
  "myQuery": [
    {
      "field": "parameters.applicationId",
      "value": "566ab2f0-e46d-4f8f-8902-b624ce7929d3",
      "operator": "="
    }
  ]
}

returning:

{
  "data": {
    "stream": {
      "id": "c6b0c4077a",
      "name": "Revit Windows1",
      "object": {
        "id": "0d0a4abc6a5fcc763e6c850dd3d5ecab",
        "createdAt": "2021-05-10T19:50:01.286Z",
        "totalChildrenCount": 57,
        "children": {
          "cursor": null,
          "totalCount": 0,
          "objects": []
        }
      }
    }
  }
}

Hey @henkon, no worries - this is how I get coerced into writing docs! Happy it’s clear. Let me first try to understand what you’re trying to do: are you trying to get a specific revit parameter out of all the objects that are present in a given commit object? or just a specific one, for a specific object?

Here’s actually for that specific object (i’ve replaced stuff with your streamId - it’s public, so make it private if it’s private!):

Later edit: You can also query by a specific parameter value. The query below returns all objects with an area smaller than “5”:

2 Likes

Hi there Speckle community,
I’ve been playing with Speckle V2 over the last couple of days and specifically with the graphql queries - kudos to everyone involved in the development, awesome job :clap:
I was wondering about the operators in the query variables. I see we can use “=”, “>” and similar but what about if I want to:

  1. Filter fields that contain a specific string value? → something like “value”: “startsWith%”
  2. Assign an array to the value? → something like “value”: [“A”, “B”, “C”], “operator”: “in”
    In other words, what is the full list of operators we can use ?
    Thanks a lot!

Mariela

1 Like

Hi @Mariela_Tsopanova, thanks for the praise!

When I’ve written down the object queries, I haven’t spent much time expanding the type of queries as I wanted to first get them on an efficient level with pagination, the see what data in Speckle will actually look like first (there were no connectors when i wrote that code!), and then what querying needs we will need based on your feedback. Now that we’ve gotten some, we can move it forward!

To your points:

(1) Currently supported operators and queries are not that many, and, unfortunately, the best place to look is in the service itself:

The server tests on this topic can be useful too.

(2) Extra operators are welcome. We’ll see what we can do here, as we have to work with what the json path language that postgres works with. It’s quite flexible though, so we can expand more what you can do, inc. searches based on inner array queries, etc.

I’ve flagged an issue here. It would help us a lot if you’d describe a bit the real-life case that you’re grappling with at the moment/or what you’re trying to do (data from revit/gh, what kind of data, what query you’re thinking of, etc.)!

3 Likes

This is cool!

Any plans on improving the performance of queries and making them more plentiful in the future.
I am thinking of storing extensive multi-hiearchical graph models and querying only specific views on the data. For that, the kind of object queries outlined here would need to be very fast.

I haven’t looked at the underlying code yet, but how’s the object structure stored? Are the JSON-dumps just stored as mongoDB-documents per object?

Thanks already!

1 Like