Not able to set `speckle_type` when dynamically creating Speckle classes using Pydantic

Hello @izzylys @gjedlicska,

We are currently setting up Speckle kits to get our own Python classes to be recognized in Grasshopper, after exchanging them via Speckle. We start from Python, where we dynamically create Speckle classes based on the classes we already defined, during import. For example, we have a Typology class, for that we dynamically create a SpeckleTypology class, to which we copy the attributes of Typology, and that inherits from the Speckle Base class. Then, when sending to Speckle, we convert all Typology instances to SpeckleTypology instances, to send these to the Speckle server.

However, it doesn’t seem possible to dynamically create a Speckle class (using pydantic.create_model()), with specification of the speckle_type argument:

speckle_class = pydantic.create_model(speckle_class_name, __base__=base, **fields, speckle_type=(ClassVar, speckle_type))

The issue is that the working of pydantic.create_model() is sort of clashing with the use of __init_subclass__ in _RegisteringBase that sets the speckle_type and registers the type in the internal _type_registry. The speckle_type keyword is namely taken as a field by create_model and directly added to the attributes of the new class, and is not passed to the __init_subclass__ method. In the end, this means that the speckle_class_name will be used to set speckle_type. The problem is thus that the speckle_type cannot be set to match the exact type as defined in the C# kit, Objects.Other.Typology in this case. A possible workaround is to provide this directly as class name, so the speckle_type will match, but this isn’t very pretty. Another one is to first create the class, to then overrule the speckle_type and adjust the _type_registry, also not pretty.

I also tried to create the class using type(). This did actually work, as it takes a dictionary with attributes, instead of collecting any keyword argument as in pydantic.create_model(). This namely allows to specify the speckle_type argument separately, which is then passed all the way to __init_subclass__. It gives the desired result which you can see in the picture.

speckle_class = type(speckle_class_name, (base, ), fields, speckle_type=speckle_type)

However, apart from this, it is definitely preferable to use pydantic. Therefore I hope that you might have a better suggestion to fix this issue. Of course you can’t change pydantic, but maybe I’m overlooking something there. Another option would be a method in _RegisteringBase to update the speckle_type of a class, also updating the registry, though I can imagine this is not something you want to enable. Any advice is welcome!

1 Like

Hey @Rob

This is def an advanced topic we haven’t considered supporting officially so far.
But your assumptions, that this should work are correct.

So i went ahead and tried to create a repro case for your issue. See the code below. Unfortunately (or maybe not so) my script below works and the dynamically created type can be serialized and deserialized by our BaseSerializer. This means, send and receive should also work.

Maybe I got your problem wrong. If that is the case, could you modify my script, to reproduce the issue you are facing?

import json
import pydantic

from specklepy.objects import Base
from specklepy.serialization.base_object_serializer import BaseObjectSerializer

speckle_type = "Speckle.Foo"

attributes = {
  "bar": (str, ...),
  # even if the attribute is added to the attributes
  # and has a default value, the code below works
  "speckle_type": (str,speckle_type)
}


SpeckleFoo = pydantic.create_model(speckle_type, __base__=Base, **attributes)

print("speckle_type on SpeckleFoo CLASS", SpeckleFoo.speckle_type)

speckle_foo = SpeckleFoo(bar="speckle_bar")

print("speckle_type on speckle_foo INSTANCE", SpeckleFoo.speckle_type)

serializer = BaseObjectSerializer()

obj_id, data = serializer.write_json(speckle_foo)

# thins includes the proper speckle_type
print("speckle_type in the json data", json.loads(data)["speckle_type"])

recomposed = serializer.read_json(data)

print("speckle_type on recomposed INSTANCE", recomposed.speckle_type)
print("Can this recompose to the proper type?", "Yes" if isinstance(recomposed, SpeckleFoo) else "No")
print("Is the dynamic type registered on Base Type Reg?", "Yes" if Base._type_registry.get(speckle_type) == SpeckleFoo else "No")

Hi @gjedlicska,

Thanks for your reply and the repro, think this will indeed help to get it more clear. It is indeed not reproducing the exact issue. However, with two minor changes it does, see below:

import json
import pydantic

from specklepy.objects import Base
from specklepy.serialization.base_object_serializer import BaseObjectSerializer

# EDIT 1: Desired name of the Speckle class is not equal to the desired Speckle type,
# which should match C# namespace and class name.
speckle_class = "SpeckleFoo"
speckle_type = "Objects.Other.Foo"

attributes = {
  "bar": (str, ...),
  # even if the attribute is added to the attributes
  # and has a default value, the code below works
  "speckle_type": (str, speckle_type)
}

# EDIT 2: Use 'speckle_class' as model name, with 'speckle_type' in **attributes
SpeckleFoo = pydantic.create_model(speckle_class, __base__=Base, **attributes)

print("speckle_type on SpeckleFoo CLASS", SpeckleFoo.speckle_type)

speckle_foo = SpeckleFoo(bar="speckle_bar")

print("speckle_type on speckle_foo INSTANCE", speckle_foo.speckle_type)

serializer = BaseObjectSerializer()

obj_id, data = serializer.write_json(speckle_foo)

# this includes the proper speckle_type
print("speckle_type in the json data", json.loads(data)["speckle_type"])

recomposed = serializer.read_json(data)

Running above script will show the issue. speckle_type of class and instance are both equal to the given speckle_class instead of given speckle_type. Cause is that the speckle_type input is not passed to the __init_subclass__ method, which results in SpeckleFoo.speckle_type being overruled by the class name. Hope that fully demonstrates the arising issue.

Ok, now I have a proper example, that I can investigate, thanks.

In the meantime, what is the reason, for not using the speckle_type as the value of the speckle_class? You can name the generated type with any name you’d like.

image
prints:
image

And if I get your usecase corretly, the SpeckleFoo class would not be used manually anyhow, its just a dynamic converter of your class definitions to something inheriting from the Speckle Base object.
Making the name of the class almost irrelevant right?

I think I’ve made it work @Rob

replacing your call with this, should do the job:

SpeckleFoo = pydantic.create_model(speckle_class, __base__=Base, **attributes, __cls_kwargs__={"speckle_type": speckle_type})
1 Like

Very nice! This indeed fixes it :smiley:

I did have to update my pydantic version, as this __cls_kwargs__ argument is only available since version 1.9.1, therefore I didn’t notice this option before. Unfortunately it’s also not mentioned on their docs. Anyway, this is exactly what we needed, thanks for the help!

1 Like

Awesome, glad i could help :slight_smile:

Happy speckling :slight_smile:

1 Like