Grasshopper and jupyter notebook (pandas)

ssw · 5 January 2023 17:36

Hello,

I find that when loading in excel or csv files into grasshopper- its hard to work with the data aswell as you could with say pandas. I know there are ways around this- however- i wondered if its possible to create a direct stream between grasshoppee data and a jupyter notebook- where i could manipulate data in a much easier environment and either visualise or even pass back sorter data?

I suppose the workaround if this is not possible is to just deal with excel files- but it would be really nice if there was some back and forward workflow?

Thankyou

jonathon · 5 January 2023 17:39

Hey @ssw,

It’s unclear if this is a Speckle data question or more a general using Jupyter/Pandas with Grasshopper question.

What are you looking to achieve? What data manipulations are you attempting?

ssw · 5 January 2023 18:27

It is a speckle question- i want to be able to pass data from grasshopper into jupyter notebook
And wondered if there is a speckle connector to do this? I saw a tutorial about creating a dashboard that uses pandas with speckle- but i would like to be able to use jupyter to have visual feedback-
Does that make sense?

jonathon · 5 January 2023 18:38

ok gotcha

Your first post described manipulating excel data more easily within Jupyter than Grasshopper, if you want to do that your best approach to using Jupyter-manipulated data within Grasshopper is probably to push the results to a speckle branch and consume it in Grasshopper.

How you get the information into jupyter is up to you. If the source is simply excel/vs then the basic csv library would probably suit.

df = pandas.read_csv("data.csv")

and then you can use specklepy to publish that to a stream.

In your follow-up, you want to pass grasshopper data to Jupyter. Again simply importing specklepy into your notebook to consume a stream/branch/commit produced in Grasshopper

ssw · 5 January 2023 19:35

Thankyou- silly question but is there a way i can get say a list from grasshopper directly into jupyter? Or would i need to take the gh list write it to a cvs or excel and then read it with pandas?

Thankyou for your help

gokermu · 6 January 2023 01:54

Yes, there is. As @jonathon mentioned, you can use speckle’s Python SDK, specklepy. Take a look at the following links, I’ll try to share some code in the morning.
https://speckle.systems/tag/specklepy/

jonathon · 6 January 2023 07:48

But not directly to the GH runtime, you will have export a file of some sort.

AlanRynne · 6 January 2023 11:50

Not sure this would fit what you want. Trying to get it to work on a Jupyter notebook may require a bit more headaches, but if you only need to run pandas stuff on a script, this may actually do the trick for you way better than anything you were planning:

Hops is a new plugin for Grasshopper that enables using Rhino.Compute for some crazy complex use-cases, it is quite powerful but I’d confess I haven’t poked around with it much

jonathon · 6 January 2023 12:36

@AlanRynne and @ssw I’m tempted to try this out now

ssw · 6 January 2023 12:49

That looks great, the reason I like jupyter notebook is i can see graphical feedback on both my data frames and any visualisations I do- do you think that would be possible using this?

gokermu · 6 January 2023 13:46

Hey @ssw ,

I created a simple POC that gets the latest commit sent from GH to the below stream. Take a look👀

github.com

specklesystems/speckle-examples/blob/main/python/speckle-jupyter-notebooks/grasshopper-to-jupyter.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "17e1ee48",
   "metadata": {},
   "source": [
    "# Working with GH Data in Jupyter via SpecklePy🐍"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b19579d1",
   "metadata": {},
   "source": [
    "### Purpose\n",
    "\n",
    "We want to access data produced in Grasshopper and sent to a Speckle Stream. The intended destination is run some analytics using pandas' dataframes.\n",

This file has been truncated. show original

GH_Data

Received in Jupyter

Stream Used

https://speckle.xyz/streams/275f928307

Once received in Jupyter, you can do whatever you want with it. Use pandas, matplotlib etc.

ssw · 6 January 2023 15:34

Amazing- can i then go back the other way with speckle- rather than exporting our a csv for example?

jonathon · 6 January 2023 15:36

absolutely, we’ll expand this a little bit and probably make a tutorial post or at the least a community forum post about it.

ssw · 11 January 2023 22:06

Would this tutorial you have made- work with this workflow?

The reason i ask- would be- could you then be able to graphically see the dataframes? How does pandas display just in something like visual studio?

gokermu · 12 January 2023 04:57

I see no reason for it to not work. I did use stremlit for the UI in that tutorial. So modify the code according to Jupyter and you’ll be good to go.

This is how DataFrames look like in VSC👇.