Gotta catch-em-all
The stated objective of the July’s Community Contest was to virtually catch all the pokemon for the July Community challenge.
The winner was set to be the one who managed to 3D Comment on the most Pokémon. There were 4 possible strategies for finding the Pokémon.
- The amateur sleuth
-
The cunning section plane,
-
The Grasshopper as ETL tool approach,
-
Or, this, the hijack the server method.
The Basics
The basic install for any good speckle adventure with python: specklepy
pip install specklepy
The simplest way to quickly get a hold of speckly data is to use the super handy StreamWrapper.
Construct a StreamWrapper with a stream, branch, commit, or object URL. The corresponding ids will be stored in the wrapper. If you have local accounts on the machine, you can use the get_account and get_client methods to get a local account for the server. You can also pass a token into get_client if you don’t have a corresponding local account for the server.
I already know I’ve the Pokémon stream is public and I have an account active in the Manager, so I can whack in the bare stream Url.
from specklepy.api.wrapper import StreamWrapper
wrapper = StreamWrapper(
"https://speckle.xyz/streams/d0176180f3/commits/3eae9e5dc6" )
client = wrapper.get_client()
# Always worth checking the client for errors, it saves a headache later.
print(wrapper.get_client())
> SpeckleClient( server: https://speckle.xyz, authenticated: True )
Now we have an authenticated client; we can start enquiring.
The wrapper object carries a bunch of stream attributes; while we know these from the URL, worth using the resolved ids from there. The client can list all the commits. These are in age order, so the latest is always first - you can limit the query to 1. Or we use a specific commit if we referenced it in the wrapper URL.
I can then get that Commit object. The output shows that a Commit is a Base object like many others.
# latest commit from main branch
latest_commit = client.commit.list(wrapper.stream_id, 1)
commit_id = latest_commit[0].id
# alternatively, you can use the wrapper to get a specific commit
commit_id = wrapper.commit_id
commit = client.commit.get(wrapper.stream_id, commit_id)
# I'll pump out that result just to make sure it's working
print(commit_id)
> 3eae9e5dc6
As we’ll be running an analysis to find those Pokébeasts, the next step is to receive the data for the commit. This is quite straightforward with the also handy Transport operations.
I don’t know if I have the stream cached and will only be getting this blob once, so I’m using a ServerTransport. More details about transports can be found in the Docs
The key information from the Commit was that referencedObject id. Really a Commit is a Base Object defining the message and metadata. The referencedObject holds the committed data.
from specklepy.transports.server import ServerTransport
from specklepy.api import operations
from specklepy.objects import Base
transport = wrapper.get_transport()
base_obj = Base()
received = operations.receive(
obj_id=commit.referencedObject,
remote_transport=transport )
That takes long enough to be a bunch of data; let’s check.
> Base(
id: 5ae00e06e0d62a4a8e3c46a184cd5181,
speckle_type: Base, totalChildrenCount: 14404)
That’s server interaction done with for now and we have a massive Base object in memory.
We can use this Base object (the operations deserialized it into python native data structures) to work our magic.
If you’ve followed my Grasshopper example for caputuring the Pokémon, you could go into all the individual objects and inspect them for mesh vertex counts in the same way. You may also have noticed from that or from the data explorer in the web viewer, the biggest worst kept secret.
The Secret
ALL THE POKEMON ARE ON A SINGLE DATA BRANCH!!!
The last time around, I ignored that in the name of purist adventure. Now I’m saving time.
The Method
data = received['@Data']
# the first two are a random building, I'll skip them
pokemon_branch = data['@{0;0;0}'][2:]
# I'll set an arbitrary number as a maximum pokémon yield.
how_many_to_catch = 1000
I’m so lazy at this point, I don’t actually care about the detailed geometry at all.
Bounding boxes have been added by the carefully constructed Speckle Connectors. That’s all we’ll need for positional information.
So let’s ensure we’re only dealing with objects with a bounding box (bbox) attribute.
And then a couple of helper functions for the position and size of the box.
def get_bbox(obj):
# this literally just returns if there is a bbox, could easily be a
# more rigourous logic check for any of the data properties
return getattr(obj, 'bbox', None) is not None
def get_midpoint(interval):
return (interval.start + interval.end)/2
def get_centroid(bbox):
return {
'x': get_midpoint(bbox.xSize),
'y': get_midpoint(bbox.ySize),
'z': get_midpoint(bbox.zSize) }
def get_size(bbox):
return {
'x': bbox.xSize.end - bbox.xSize.start,
'y': bbox.ySize.end - bbox.ySize.start,
'z': bbox.zSize.end - bbox.zSize.start }
# pokemon_branch is a list of objects
pokemon_boxes = list(filter(get_bbox, pokemon_branch))
The Schema
The specklepy library doesn’t have any methods for dealing with comments yet (#182)
But we can send a GraphQL mutation query to the Speckle server directly. We just need to be nimble and keep up with the evolving schema 
!!
Making a hand-spun server query isn’t documented yet, but the main client functions implement the same resource call methods. I’ll just piggyback on the server.
This is better than rolling your own HTTP POST request handler as the error checking and fallbacks are already included in that make_request method.
def add_comment(query):
result = client.server.make_request(
query=query,
params={},
return_type=[] )
Consider the code block below to be the schema documentation of Comments for any of the APIs you might be using. 
Many are nullable, but mandatory. I’ll use as many as possible just to demonstrate each feature.
I’ll craft it as a reusable template. Each $attr is an indicator of what data we need to hydrate it with.
from gql import gql
from string import Template
make_comment = Template(
"""
mutation CommentCreateInput {
commentCreate(
input: {
streamId: "$STREAM_ID",
resources: [
{
resourceType: $RESOURCE_TYPE
resourceId: "$RESOURCE_ID",
}
],
blobIds: []
screenshot: null
text: {
type: "doc",
content :[
{
type: "paragraph",
content: [{type:"text",text:"$COMMENT_TEXT"}]
}
]
},
data: {
location: {
x: $COMMENT_PIN_X,
y: $COMMENT_PIN_Y,
z: $COMMENT_PIN_Z
},
filters: {
filterBy: {
__parents: {
includes: [
"$OBJECT_ID"
]
}
}
},
camPos: [
$CAMERA_X,
$CAMERA_Y,
$CAMERA_Z,
$TARGET_X,
$TARGET_Y,
$TARGET_Z,
1,
1
],
selection: null,
sectionBox: {
min:{
x: $SECTION_MIN_X,
y: $SECTION_MIN_Y,
z: $SECTION_MIN_Z
},
max: {
x: $SECTION_MAX_X,
y: $SECTION_MAX_Y,
z: $SECTION_MAX_Z
}
}
}
}
)
}
""")
from assets.random_comment import random_comment
A couple more helper functions to process each BBox to help fill that template.
- Pin the comment to the centre of the Pokémon.
- Look at the Pokémon from up and to the side.
- Filter the view to highlight the captured beast.
- Constrain the objects to a section box 4 times the size.
Section boxes and filters together don’t result in the view you think they might, but it’s good for the demo.
def get_camera_offset(bbox):
centroid = get_centroid(bbox)
size = get_size(bbox)
return {
'x': centroid['x'] - size['x']*2,
'y': centroid['y'] - size['y']*2,
'z': centroid['z'] + size['z']*2
}
def get_section_box(bbox):
centroid = get_centroid(bbox)
size = get_size(bbox)
return {
'min': {
'x': centroid['x'] - size['x']*2,
'y': centroid['y'] - size['y']*2,
'z': centroid['z'] - size['z']*2
},
'max': {
'x': centroid['x'] + size['x']*2,
'y': centroid['y'] + size['y']*2,
'z': centroid['z'] + size['z']*2
}
}
Enough already, catch me some Pokémon.!! 
This really is more fun if you look at the stream while this runs 
for p, pokemon in enumerate(pokemon_boxes):
bbox = pokemon['bbox']
centroid = get_centroid(bbox)
camera = get_camera_offset(bbox)
section_box = get_section_box(bbox)
params = {
# The resource data
'STREAM_ID': wrapper.stream_id,
'RESOURCE_TYPE': 'commit',
'RESOURCE_ID': commit_id,
# The comment data
'COMMENT_TEXT': random_comment(),
'COMMENT_PIN_X': centroid[ 'x' ],
'COMMENT_PIN_Y': centroid[ 'y' ],
'COMMENT_PIN_Z': centroid[ 'z' ],
# The filter data
'OBJECT_ID': pokemon[ 'id' ],
# The camera data
'CAMERA_X': centroid[ 'x' ],
'CAMERA_Y': centroid[ 'y' ],
'CAMERA_Z': centroid[ 'z' ],
'TARGET_X': camera[ 'x' ],
'TARGET_Y': camera[ 'y' ],
'TARGET_Z': camera[ 'z' ],
# The section box data
'SECTION_MIN_X': section_box[ 'min' ][ 'x' ],
'SECTION_MIN_Y': section_box[ 'min' ][ 'y' ],
'SECTION_MIN_Z': section_box[ 'min' ][ 'z' ],
'SECTION_MAX_X': section_box[ 'max' ][ 'x' ],
'SECTION_MAX_Y': section_box[ 'max' ][ 'y' ],
'SECTION_MAX_Z': section_box[ 'max' ][ 'z' ]
}
hydrated_comment = gql(make_comment.substitute(params))
limit = min(len(pokemon_boxes), how_many_to_catch)
limit
if p < limit:
try:
add_comment(hydrated_comment)
finally:
print(p, end='\r')
The clean-up
For the sake of completeness, it’s worth showing the inverse and how we might tidy up from this nonsense.
Again, we use the wrapper to get the stream_id and the author_id. To delete all comments by an author, we first need to query all the comments and then filter over those by the author.
The comment query only returns batches of 25; this seems hardwired or is at least non-configurable with the make_request resource method. So we just while over the inner loop until nothing is left.
A delete query per resultant comment will rip them from the view as quickly as they landed.
def delete_comment_by_author(stream_id, author_id):
query = Template(
"""
query {
comments(streamId:"$stream_id") {
items {
id
authorId
}
}
}
""")
deleteTemplate = Template(
"""
mutation {
commentArchive(streamId:"$stream_id", commentId:"$comment_id")
}
""")
count = 1
while count > 0:
resource = client.server
params = {}
result = resource.make_request(
query=gql(
query.substitute({'stream_id':stream_id})),
params=params,
return_type=['comments', 'items'] )
count = len(result)
if count == 0:
break
author_comments = filter(lambda x: x['authorId'] == author_id, result)
for comment in author_comments:
comment_id = comment['id']
hydrated_delete = gql(
deleteTemplate.substitute(
{'stream_id':stream_id, 'comment_id':comment_id} ) )
data = resource.make_request(
query=hydrated_delete,
params=params, return_type=[] )
print (count, end="\r")
delete_comment_by_author(
stream_id=wrapper.stream_id,
author_id=wrapper.get_account().userInfo.id )
