Ids in speckle server not matching those retrieved from API

Help
,
1

The ids shown in the data view next to the 3d viewer in speckle server appear to be hex uuids which I expect to align with the ids that are returned in the object obtained using the python api. I’m unable to find any elements by the id shown in the speckle server. All other attributes match, it is just the ids that are the issue. Am I missing something?

2

Could we have a sample stream/some way of reproduction? This is, in theory, impossible. There are a few places we could look afterwards:

  • the viewer object display adds some cosmetics on top of the raw data
  • desertialsiation in our could affect the id on receive
  • we need to make sure we’re talking about the same id (vs. applicationId)

(And welcome to the forum, by the way!)

3

Thanks @dimitrie . Is there a public stream that I can have read access on to create an MRE? I tried looking around the documentation but wasn’t able to find anything.

4

Feel free to use this one: Speckle

5

Thanks jonathon! Here’s some code to reproduce the issue that I’m seeing.

from specklepy.api.client import SpeckleClient
from specklepy.transports.server import ServerTransport
from specklepy.api import operations
from specklepy.serialization.base_object_serializer import BaseObjectSerializer

import json
import os

client = SpeckleClient(host="https://speckle.xyz/") 
client.authenticate_with_token(os.environ['SPECKLE_TOKEN_XYZ'])
transport = ServerTransport(client = client, stream_id = '538fcacdbe')

obj = operations.receive(obj_id = 'd17c15ea132db9d91126fc33ebbe9a94',remote_transport=transport)

serializer = BaseObjectSerializer()
hash, obj_dict = serializer.traverse_base(obj)

ids = [x['id'] for x in obj_dict['elements']]
##Check for id for 'Walls' element as displayed in browser
'fafbda8723b404b01420605f8f0f8785' in ids
#False

##Get id of 'Walls' element as returned by serializer
[x['id'] for x in obj_dict['elements'] if x['name'] == 'Walls']
#['c59f3a45114957a266dd0c2fa4ae158a']

I expect to see the id (fafbda8723b404b01420605f8f0f8785) in the screenshot below as one of the ids in the obj_dict dictionary. However, it looks like the serializer gets a different id for that element, ‘c59f3a45114957a266dd0c2fa4ae158a’.

ID_MRE
ID_MRE518×627 16.9 KB

6

Hey @RyanHaunfelder ,

Your example shows that you are retrieving objects and directly running them through the serializer. Essentially with that, you are using specklepy to create a new hash id for objects ready to send to Speckle. The traverse_base adds ids for all convertible objects within a root Base object.

Where an object is serialized in Revit (c#) and sent to Speckle, if you receive in Python and serialize it again, it will be a new object and hence a new id.

If you are instead trying to retrieve the objects for processing in Python, there are a couple of ways.

I include below a simple flatten example:

def flatten(obj, visited=None):
    visited = visited or set()

    if obj in visited:
        return

    visited.add(obj)
    yield obj

    for value in obj.__dict__.values():
        if value is None:
            continue

        if isinstance(value, Base) or isinstance(value, Iterable) and any(isinstance(item, Base) for item in value):
            yield from flatten(value, visited)

This retrieves all Base objects, which you may not want to do… Changing the yield in that function to be particular might be

    should_include = any([
        hasattr(obj, "speckle_type") and obj.speckle_type == "Speckle.Core.Models.Collection",
        ]
    )

    if should_include:
        yield obj

Nevertheless, with the first definition, getting all the Collections named Walls per your example

elements = list(flatten(obj)) # this is the `obj` you created in your code above
ids = [
    x["id"]
    for x in elements
    if x.speckle_type == "Speckle.Core.Models.Collection" and x["name"] == "Walls"
]

'fafbda8723b404b01420605f8f0f8785' in ids
#True

But, given that we already have all the objects at this point, perhaps you don’t care about the Revit Category collection but want the meat in the sandwich:

[
    (x["id"], x.speckle_type)
    for x in elements
    if   x.speckle_type == "Objects.BuiltElements.Wall:Objects.BuiltElements.Revit.RevitWall"
]

# [('05813d9f9fa83de33fe834b4c2ce8b50', 'Objects.BuiltElements.Wall:Objects.BuiltElements.Revit.RevitWall'),
#  ('5ecd14b3c1d8fa170fd681f0d308b591', 'Objects.BuiltElements.Wall:Objects.BuiltElements.Revit.RevitWall'),
#  ('e8d105ed0e3db144b095fecfd2e35303', 'Objects.BuiltElements.Wall:Objects.BuiltElements.Revit.RevitWall'),
#  ('4caf4da981937ef5eb98beb837c9ccf3', 'Objects.BuiltElements.Wall:Objects.BuiltElements.Revit.RevitWall'),
#  ('a26a6e470736d473782a578028515def', 'Objects.BuiltElements.Wall:Objects.BuiltElements.Revit.RevitWall'),
#  ('207cd30854d925c41a861e1b09487ba2', 'Objects.BuiltElements.Wall:Objects.BuiltElements.Revit.RevitWall'),
#  ('83920a088eb1911a74c167f0206e1063', 'Objects.BuiltElements.Wall:Objects.BuiltElements.Revit.RevitWall'),
#  ('077eab1399935bbb558e01f2c1afb009', 'Objects.BuiltElements.Wall:Objects.BuiltElements.Revit.RevitWall'),
#  ('5a675fc787e6860820e85ce2f0fdea8a', 'Objects.BuiltElements.Wall:Objects.BuiltElements.Revit.RevitWall')]

@Jedd is working on a universal traversal approach to retrieving objects from a commit, but let me know if this suffices?

1 Like
7

Thank you @jonathon ! That makes perfect sense. I’m trying to convert the entire speckle object to a dictionary and misunderstood the purpose of the serializer. I think the flatten function will be useful so I appreciate the response. After reflecting on it, I think just recursively getting the “__dict__” attribute of each object is what I’m after. Something like,

elements = json.loads(json.dumps(obj, default=lambda o: o.__dict__ if hasattr(o, '__dict__') else None))

Is there a more straightforward way of doing this conversion?

8

Perhaps it is because I am familiar with the Speckle data model, but don’t have context of what you are looking to next, I’m not sure I can offer a better alternative.

While there are alternative approaches to achieve this conversion, using json.dumps() and json.loads() with a custom default function is a commonly used method to parse complex object hierarchies into a dictionary.

Here are a few alternative approaches that you can consider:

  1. Implement a custom conversion/traversal/ecursive method (@jedd will be publishing some examples soon in our docs)
  2. Utilise third-party libraries: There are third-party libraries available, such as dataclasses-json and attrs , that offer built-in functionalities for converting Python objects to JSON and vice versa.

But json is built in and if you are getting what you want, that’s fine also.

9

I appreciate the response @jonathon. I’m really enjoying speckle and it’s great to have an active development team.

2 Likes