How to list more than 100 commits with Specklepy and GraphQL?

Help Developers
,
1

Trying to get a list of commits from a stream, im doing:

commits = client.commit.list(stream_id, 100)

But that seems like the limit? If I raise it to 101, I get a GraphQLException:

GraphQLException: Failed to execute the GraphQL commit request. Errors: [{‘message’: ‘Cannot return more than 100 items, please use pagination.’, ‘locations’: [{‘line’: 3, ‘column’: 5}], ‘path’: [‘stream’, ‘commits’], ‘extensions’: {‘code’: ‘BAD_USER_INPUT’}}]

Is there a way to get around this? What would be an example of using pagination with specklePY?

2

100 is the server limit. But you can add the last commit id received as the cursor property for subsequent queries ( this may be a timestamp cursor- sorry I’m replying from a taxi )

3

Thanks Jonathon.

Trying to do:
response = client.commit.list(stream_id, 100, cursor=cursor)

Getting:
TypeError: list() got an unexpected keyword argument ‘cursor’

4

Right.

But, that’s not strictly the Graphql API though that’s our python SDK Client. Apologies, I should have twigged that is what your frame of reference was.

Specifically, that resource interface doesn’t support using a cursor for pagination. It probably could/should.

In the meantime, you can craft something yourself.

Inspecting the code inside the SDK method:

commits = client.commit.list(wrapper.stream_id)

uses this query

query Commits($stream_id: String!, $limit: Int!) {
    stream(id: $stream_id) {
        commits(limit: $limit) {
            items {
                id
                message
                referencedObject
                authorName
                authorId
                authorName
                authorAvatar
                branchName
                createdAt
                sourceApplication
                totalChildrenCount
                parents
            }
        }
    }
}

we can amend the query and then wrap it in a new function (I’m making the response request briefer just for example):

query PaginatedCommits($stream_id: String!, $limit: Int!, $cursor: String) {
    stream(id: $stream_id) {
        commits(limit: $limit, cursor: $cursor) {
            cursor
            items {
                id
                message
                referencedObject
            }
        }
    }
}

The function is pretty simple:

from gql import gql

def get_paginated_commits(stream_id, limit, cursor):
    query = gql(
        """
            query PaginatedCommits($stream_id: String!, $limit: Int!, $cursor: String) {
                stream(id: $stream_id) {
                    commits(limit: $limit, cursor: $cursor) {
                        cursor
                        items {
                            id
                            message
                            referencedObject
                        }
                    }
                }
            }
            """
    )
    params = {"stream_id": stream_id, "limit": limit, "cursor": cursor}

    response = client.server.make_request(query=query, params=params, return_type=[])

    return (
        response["stream"]["commits"]["cursor"],
        response["stream"]["commits"]["items"],
    )

stream_id = wrapper.stream_id
limit = 3
cursor = ""

cursor, commits = get_paginated_commits(stream_id, limit, cursor)

'2023-09-12T17:30:24.232Z'

[{'id': '528d915022',
  'message': 'Sent 18 elements from Navisworks.',
  'referencedObject': 'f5d040c732c92869e93f710df29196d3'},
 {'id': '0105cde21b',
  'message': 'Sent 18 elements from Navisworks.',
  'referencedObject': 'f5d040c732c92869e93f710df29196d3'},
 {'id': '0e2d07497d',
  'message': 'Sent 4 elements from Navisworks.',
  'referencedObject': '53ec42b5f6e6f70d77b0eb0f59cf8d84'}]

and then calling that function again

'2023-09-12T17:30:24.232Z'

[{'id': 'd08855adb0',
   'message': 'Sent 18 elements from Navisworks.',
   'referencedObject': '24e46827fdc2fb783b1fac2684fb8b44'},
  {'id': '715636467d',
   'message': 'Sent 18 elements from Navisworks.',
   'referencedObject': '30726c32e80251e74a0c7b11674d233c'},
  {'id': 'aaaf0a9789',
   'message': 'Sent 18 elements from Navisworks.',
   'referencedObject': '293c8f60f220ce1d2c84d9b5ea755b1c'}]
1 Like
5

Beautiful, really appreciate this :heart:

2 Likes