Speckle account problem while deploying app in Streamlit cloud

Help Developers
1

Hi all,

I’m working on a project with the help of Speckle + Streamlit. I worked on my local so far. Now I’m trying to deploy it in Streamlit cloud. I used StreamWrapper to get all data for the spesific commit while running locally. I’m getting this error in the cloud now:

 /home/adminuser/venv/lib/python3.8/site-packages/streamlit/runtime/scriptrun  

  ner/script_runner.py:535 in _run_script                                       

                                                                                

  /mount/src/project_beam_mvp/pages/Explore_Projects.py:94 in <module>          

                                                                                

     91 │                                                                       

     92 │   # Get data using object id of selected commit                       

     93 │   obj_id = sCommit.referencedObject                                   

  ❱  94 │   commit_data = operations.receive(                                   

     95 │   │   obj_id=obj_id, remote_transport=wrapper.get_transport()         

     96 │   )                                                                   

     97                                                                         

                                                                                

  /home/adminuser/venv/lib/python3.8/site-packages/specklepy/api/operations.py  

  :53 in receive                                                                

                                                                                

    50 │   │   Base -- the base object                                          

    51 │   """                                                                  

    52 │   metrics.track(metrics.RECEIVE, getattr(remote_transport, "account",  

  ❱ 53 │   return _untracked_receive(obj_id, remote_transport, local_transport  

    54                                                                          

    55                                                                          

    56 def serialize(base: Base, write_transports: List[AbstractTransport] = [  

                                                                                

  /home/adminuser/venv/lib/python3.8/site-packages/specklepy/core/api/operatio  

  ns.py:86 in receive                                                           

                                                                                

     83 │   │   │   )                                                           

     84 │   │   )                                                               

     85 │                                                                       

  ❱  86 │   obj_string = remote_transport.copy_object_and_children(             

     87 │   │   id=obj_id, target_transport=local_transport                     

     88 │   )                                                                   

     89                                                                         

                                                                                

  /home/adminuser/venv/lib/python3.8/site-packages/specklepy/transports/server  

  /server.py:145 in copy_object_and_children                                    

                                                                                

    142 │   │   r.encoding = "utf-8"                                            

    143 │   │                                                                   

    144 │   │   if r.status_code != 200:                                        

  ❱ 145 │   │   │   raise SpeckleException(                                     

    146 │   │   │   │   f"Can't get object {self.stream_id}/{id}: HTTP error"   

    147 │   │   │   │   f" {r.status_code} ({r.text[:1000]})"                   

    148 │   │   │   )                                                           

────────────────────────────────────────────────────────────────────────────────

SpeckleException: SpeckleException: Can't get object 

.....: HTTP error 403 ({"error":"Your 

token is not valid."})

I parsed the commit data using the speckle-data-pack examples and developed the project using this. To be honest, I didn’t focus on the “local account” mentioned in the docs and didn’t question how it works at the time :). It is the problem, am I right? Should I use SpeckleClient instead of StreamWrapper?

2

Hi @Elif_Sinem_Aktas ,

Yes, you are right, StreamWrapper works with your Speckle Manager(locally). Use SpeckleClient if you want to deploy and make sure on the Developer Setting when you generate the Access Token/Application with the permissions you need.

If you have more questions please let us know.

image
image1657×913 73.7 KB

3

Hi @Nikos,

Thank you for your quick response and confirmation😊 Yes, I created token and SpeckleClient works well in the cloud. I can list streams, branchs, commits etc. All I need is refactor my code using SpeckleClient. I’ll pop back here if I run into any issues. Thanks✌️

1 Like
4

Hi @Nikos,

It didn’t take long for me to come back here :sweat_smile:

I’ve changed my code :

# Get data using object id of selected commit
obj_id = sCommit.referencedObject
commit_data = project_client.object.get(stream.id, obj_id)

While testing in local, I’m not getting error while parsing this collection.

image
image1074×648 32.9 KB

In the cloud, I can get collection:
image

While parsing, it says:

/home/adminuser/venv/lib/python3.8/site-packages/specklepy/serialization/base_object_serializer.py:432: SpeckleWarning: Could not find the referenced child object of id `83331fa50142777b0638114eb4261e8a` in the given read transport: SQLite

for child object ids, then getting key error for “elements”. I need to get all the data once and then parse it. What am I missing? Object cache dbs?

5

Hi @Elif_Sinem_Aktas

Would it be possible for you to share the Speckle project with us?

6

Hi @Jedd

We granted you access for the test stream. I guess you can reach now :pray:

7

Hi @Elif_Sinem_Aktas,

The Exception that you describe, which part of your code was throwing it?

I cannot reproduce with the below code

client = SpeckleClient()
client.authenticate_with_token("...") # token redacted
result = client.object.get("a1c5ef416f", "b0eba07e92720a17435636db62a89cce")
print(result["elements"])
8

Hi @Jedd,

I did not want to overwhelm you with the log, I am sharing the logs of streamlit cloud as txt file in the attachment. You can see the exact line in the logs :pray:

(For same stream/branch/commit)
speckle-streamlitcloud-logs.txt (30.6 KB)

I’m using the following code blocks:

In Welcome.py

client = authenticate_client(speckleServer, speckleToken)

In Explore_Projects.py

try:
    project_client = st.session_state['client']
except:
    switch_page("Welcome")

------

# Get data using object id of selected commit
obj_id = sCommit.referencedObject
commit_data = project_client.object.get(stream.id, obj_id)

# Add print to check collection
st.write(commit_data)

# Parse the data read from speckle
data = create_material_table(commit_data)

In helpers.py

def authenticate_client(server_url, token):
    client = SpeckleClient(host=server_url)
    client.authenticate_with_token(token)
    return client

def create_material_table(commit_data):
    table = {'Element': [], 'Element_ID': [], 'Type': [], 'Level': [], 'Material': [], 'Volume': [], 'Area': []}
    for element in commit_data["elements"]:
        for el in element['elements']:
            # Some Elements don't have each field we defined, we'll ignore
            for mat in el['materialQuantities']:
                table['Element'].append(element.name)
                table['Element_ID'].append(el.elementId)
                table['Type'].append(el['type'])
                table['Level'].append(el['level']['name'])
                table['Material'].append(mat['material']['name'])
                table['Volume'].append(float(mat['volume']) / 1.0E+9)
                table['Area'].append(float(mat['area']) / 1.0E+6)
    df = pd.DataFrame(table)
    return df

As I mentioned, it works when I try it on the local machine. I am sorry for taking your time. The problem may be simple but I can’t see it. Thank you for your support :blush:

9

I wonder if perhaps you are seeing a problem with filesystem access. I’m not familiar with the restrictions of streamlit in this regard.

Perhaps you could try the below code, using operations.receive and passing in a MemoryTransport instead of the default SQLiteTransport

from specklepy.transports.server import ServerTransport
from specklepy.transports.memory import MemoryTransport
from specklepy.core.api import operations
from specklepy.core.api.client import SpeckleClient

def FetchSpeckleData(object_id: str, stream_id : str, token : Optional[str]) -> Base:
    client = SpeckleClient()
    client.authenticate_with_token("...") # token redacted

    remote_transport = ServerTransport(stream_id, client)
    local_transport = MemoryTransport()

    return operations.receive(object_id, remote_transport, local_transport)

result = FetchSpeckleData("a1c5ef416f", "b0eba07e92720a17435636db62a89cce")
print(result["elements"])
4 Likes
10

I suspected about the same thing (about filesystem access). I needed your help as I couldn’t fully work on the methods and background process in Specklepy. Thank you for the solution you provided, I’ll try and give feedback :pray:

11

@Jedd, it worked :blush: I should read more about transport :sweat_smile:

Thank you so much! :partying_face:

2 Likes
12

Amazing, glad to hear it works.

We have some docs on transports that you might find an exciting read :grin:.

But as a basic premise, operations.receive will do two main things,

  1. Copy objects from a remote data store to a local one
  2. Deserialize the JSON blobs as Base objects.

The step 1 is where transports come into play, as they provide (read/write) access to a data store of speckle objects.
e.g. a remote store like a Speckle Server, or a local store like a SQLite database, or an in-memory dictionary.

Hence ServerTransport, SQLiteTransport, and MemoryTransport

By default, our SDKs will default to using a SQLite on disk. But if that doesn’t work for the machine you’re running on, a MemoryTransport is a good alternative local transport.

2 Likes