Production Speckle Server deployment

Hi dimitrie,

it is already visible to www with an ip so it is not for localhost use.

We want to run it in development mode.
Since programmers and end-users are at their homes and experimenting with it, it should be visible from everywhere.

So I am proceeding with the “Front-end” package in development mode.

Where should I put the variable “NODE_ENV=development” ? in .env ?

If it’s run with npm run dev, it will be in dev mode! Just run then npm install && npm run build in the frontend package and let me know if a frontend pops up!

1. Install

It has some issues but it seems ok.

npm install 



> core-js@3.8.1 postinstall /var/www/html/speckle/packages/frontend/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"

Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!

The project needs your help! Please consider supporting of core-js on Open Collective or Patreon: 
> https://opencollective.com/core-js 
> https://www.patreon.com/zloirock 

Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)


> ejs@2.7.4 postinstall /var/www/html/speckle/packages/frontend/node_modules/ejs
> node ./postinstall.js

Thank you for installing EJS: built with the Jake JavaScript build tool (https://jakejs.com/)


> @apollo/protobufjs@1.0.5 postinstall /var/www/html/speckle/packages/frontend/node_modules/@apollo/protobufjs
> node scripts/postinstall


> nodemon@1.19.4 postinstall /var/www/html/speckle/packages/frontend/node_modules/nodemon
> node bin/postinstall || exit 0

npm WARN camera-controls@1.28.0 requires a peer of three@^0.126.1 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.13 (node_modules/webpack-dev-server/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.13 (node_modules/watchpack-chokidar2/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.13 (node_modules/nodemon/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 1704 packages from 1064 contributors and audited 1712 packages in 44.367s

57 packages are looking for funding
  run `npm fund` for details

found 16 vulnerabilities (3 low, 5 moderate, 8 high)
  run `npm audit fix` to fix them, or `npm audit` for details


2. BUILD

Also here are some warnings but it seems ok

npm run build

> @speckle/frontend@2.2.1 build /var/www/html/speckle/packages/frontend
> vue-cli-service build


â ™  Building for production...

 WARNING  Compiled with 9 warnings                                                                                                                                                                                                           11:47:57 AM

 warning  

chunk chunk-2de4fc0d [mini-css-extract-plugin]
Conflicting order. Following module has been added:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VCard/VCard.sass
despite it was not able to fulfill desired ordering with these modules:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/directives/ripple/VRipple.sass
   - couldn't fulfill desired order of chunk group(s) , , , , , , , , , , , , 
   - while fulfilling desired order of chunk group(s) , , , , 

 warning  

chunk chunk-2de4fc0d [mini-css-extract-plugin]
Conflicting order. Following module has been added:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VProgressLinear/VProgressLinear.sass
despite it was not able to fulfill desired ordering with these modules:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/directives/ripple/VRipple.sass
   - couldn't fulfill desired order of chunk group(s) , , , , , , , , , , , , 
   - while fulfilling desired order of chunk group(s) , , , , 

 warning  

chunk chunk-39862bf6 [mini-css-extract-plugin]
Conflicting order. Following module has been added:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VInput/VInput.sass
despite it was not able to fulfill desired ordering with these modules:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VTextField/VTextField.sass
   - couldn't fulfill desired order of chunk group(s) , , , , , , , , , , , , , 

 warning  

chunk chunk-39862bf6 [mini-css-extract-plugin]
Conflicting order. Following module has been added:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VLabel/VLabel.sass
despite it was not able to fulfill desired ordering with these modules:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VTextField/VTextField.sass
   - couldn't fulfill desired order of chunk group(s) , , , , , , , , , , , , , 

 warning  

chunk chunk-39862bf6 [mini-css-extract-plugin]
Conflicting order. Following module has been added:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VTextField/VTextField.sass
despite it was not able to fulfill desired ordering with these modules:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VMessages/VMessages.sass
   - couldn't fulfill desired order of chunk group(s) 
   - while fulfilling desired order of chunk group(s) , , , , , , , , , , , , , 

 warning  

chunk chunk-7265f96a [mini-css-extract-plugin]
Conflicting order. Following module has been added:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VBtn/VBtn.sass
despite it was not able to fulfill desired ordering with these modules:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VIcon/VIcon.sass
   - couldn't fulfill desired order of chunk group(s) 
   - while fulfilling desired order of chunk group(s) , , , , , , , , , , , , , , , , 

 warning  

chunk chunk-7265f96a [mini-css-extract-plugin]
Conflicting order. Following module has been added:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VProgressCircular/VProgressCircular.sass
despite it was not able to fulfill desired ordering with these modules:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-3-1!./node_modules/postcss-loader/src??ref--9-oneOf-3-2!./node_modules/sass-loader/dist/cjs.js??ref--9-oneOf-3-3!./node_modules/vuetify/src/components/VIcon/VIcon.sass
   - couldn't fulfill desired order of chunk group(s) 
   - while fulfilling desired order of chunk group(s) , , , , , , , , , , , , , , , , 

 warning  

asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets: 
  fonts/materialdesignicons-webfont.4187121a.woff2 (312 KiB)
  fonts/materialdesignicons-webfont.fec1b66a.woff (446 KiB)
  fonts/materialdesignicons-webfont.3d1f8fa2.eot (982 KiB)
  fonts/materialdesignicons-webfont.3e722fd5.ttf (981 KiB)
  img/onboardingsplash.2f7ac815.png (322 KiB)
  js/chunk-97dcf120.c005d14e.js (927 KiB)
  js/chunk-9fd2d8fe.52aaf3af.js (811 KiB)
  js/chunk-b5664fd0.67e4c1bb.js (856 KiB)
  css/chunk-vendors.78cc2cdf.css (525 KiB)
  js/chunk-vendors.dbe842ea.js (1.15 MiB)

 warning  

entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
  app (1.68 MiB)
      css/chunk-vendors.78cc2cdf.css
      js/chunk-vendors.dbe842ea.js
      css/app.286e3040.css
      js/app.b7c808fe.js


  File                                    Size              Gzipped

  dist/js/chunk-vendors.dbe842ea.js       1179.42 KiB       363.16 KiB
  dist/js/chunk-97dcf120.c005d14e.js      926.81 KiB        235.17 KiB
  dist/js/chunk-b5664fd0.67e4c1bb.js      856.49 KiB        212.19 KiB
  dist/js/chunk-9fd2d8fe.52aaf3af.js      811.17 KiB        201.02 KiB
  dist/js/chunk-5e276b1e.5e4d16bc.js      162.88 KiB        42.78 KiB
  dist/js/chunk-59c2a73e.25c00f8b.js      127.37 KiB        32.34 KiB
  dist/js/chunk-d51158c6.81097395.js      121.15 KiB        31.63 KiB
  dist/js/chunk-35728398.a7e08557.js      112.73 KiB        30.40 KiB
  dist/js/chunk-7a7da7de.a4eafdb5.js      97.63 KiB         26.90 KiB
  dist/js/chunk-73c9b9a8.04e40100.js      94.28 KiB         25.48 KiB
  dist/js/chunk-5308cdee.d9dc7761.js      93.36 KiB         25.11 KiB
  dist/js/chunk-3aadb008.b62a3919.js      54.36 KiB         15.19 KiB
  dist/js/chunk-194bd293.c0811b3c.js      35.17 KiB         10.16 KiB
  dist/js/chunk-28ffb2ef.87a87270.js      24.33 KiB         7.94 KiB
  dist/js/chunk-39862bf6.45ffa45e.js      21.71 KiB         6.27 KiB
  dist/js/chunk-c20ae132.8d82ed3e.js      19.73 KiB         6.54 KiB
  dist/js/chunk-2de4fc0d.302252b6.js      18.78 KiB         5.67 KiB
  dist/js/app.b7c808fe.js                 16.91 KiB         5.19 KiB
  dist/js/chunk-6365c9b5.12a97951.js      16.35 KiB         5.43 KiB
  dist/js/chunk-6cc4f16d.208f9762.js      15.29 KiB         5.03 KiB
  dist/js/chunk-7380123c.f253979d.js      13.43 KiB         4.77 KiB
  dist/js/chunk-7265f96a.befbbfa0.js      11.26 KiB         3.75 KiB
  dist/js/chunk-a1707080.c0afb415.js      9.54 KiB          3.43 KiB
  dist/js/chunk-06f72f4a.f788c72c.js      7.31 KiB          2.82 KiB
  dist/js/chunk-defab8a4.b8edb57c.js      4.47 KiB          1.90 KiB
  dist/js/chunk-2d0c8ff6.beb06388.js      2.56 KiB          1.16 KiB
  dist/js/chunk-2d22cdf7.dbf53099.js      2.43 KiB          1.14 KiB
  dist/js/chunk-2d0c8c18.7c241438.js      0.57 KiB          0.38 KiB
  dist/css/chunk-vendors.78cc2cdf.css     525.10 KiB        72.54 KiB
  dist/css/chunk-5e276b1e.0429c530.css    79.62 KiB         10.34 KiB
  dist/css/chunk-97dcf120.04f4151a.css    69.94 KiB         9.12 KiB
  dist/css/chunk-b5664fd0.180bab9e.css    61.00 KiB         8.46 KiB
  dist/css/chunk-73c9b9a8.c1a9306c.css    61.00 KiB         7.98 KiB
  dist/css/chunk-d51158c6.6eb2c579.css    59.65 KiB         7.84 KiB
  dist/css/chunk-3aadb008.d0d754b7.css    57.54 KiB         7.16 KiB
  dist/css/chunk-35728398.93a42a47.css    55.70 KiB         8.04 KiB
  dist/css/chunk-59c2a73e.fe41332e.css    54.65 KiB         7.72 KiB
  dist/css/chunk-5308cdee.943e7c6d.css    52.57 KiB         7.36 KiB
  dist/css/chunk-7a7da7de.c982246f.css    51.23 KiB         6.73 KiB
  dist/css/chunk-9fd2d8fe.a302e4bd.css    26.60 KiB         4.24 KiB
  dist/css/chunk-39862bf6.4b83cbbc.css    22.39 KiB         3.04 KiB
  dist/css/chunk-6365c9b5.875b4c62.css    21.42 KiB         3.02 KiB
  dist/css/chunk-06f72f4a.08a54220.css    18.10 KiB         2.39 KiB
  dist/css/chunk-defab8a4.b5c9c2ff.css    16.87 KiB         1.98 KiB
  dist/css/chunk-6cc4f16d.2b364ad1.css    12.05 KiB         1.59 KiB
  dist/css/chunk-7265f96a.c04b5267.css    9.82 KiB          2.14 KiB
  dist/css/chunk-194bd293.dfe851b7.css    9.70 KiB          1.71 KiB
  dist/css/chunk-2de4fc0d.138af52c.css    9.55 KiB          1.89 KiB
  dist/css/chunk-28ffb2ef.9814ad34.css    4.65 KiB          1.17 KiB
  dist/css/chunk-c20ae132.ffe22fa2.css    4.65 KiB          1.17 KiB
  dist/css/chunk-7380123c.9deb4d24.css    3.14 KiB          0.80 KiB
  dist/css/chunk-a1707080.9deb4d24.css    3.14 KiB          0.80 KiB
  dist/css/app.286e3040.css               2.17 KiB          0.63 KiB

  Images and other types of assets omitted.

 DONE  Build complete. The dist directory is ready to be deployed.
 INFO  Check out deployment instructions at https://cli.vuejs.org/guide/deployment.html

I tried the following to call the front-end

“http://[ip]:3000/explorer/”

which says that I should login:

If I press login I get black screen with the following:

“Error occured while trying to proxy to: ip:3000/authn/verify/explorer/c6qdx4tg93azokxr1rnuja”

Apologies, gave you some wrong instructions there… npm run dev in the frontend package folder, and the frontend should be accessible then at the root url!

I’ll go make my second coffee to be properly awake.

In development mode, the server package (running at port 3000) proxies frontend requests to the frontend host:port, as configured in the .env file (FRONTEND_HOST, FRONTEND_PORT)

Btw, running packages in development mode exposed on the internet is not recommended (security issues).

“npm run dev”
worked fine

indeed ip:3000 is working now visualizing the front-end

Next thing is to check if commiting from several people is working, and how we can merge some test work first in Rhino and Grasshoper, and then in Revit.

PS: I went for a walk since too many coffees harm.

Hi cristi,

in order to run in production mode
I have first to run server with “build” mode

server > npm install
server > npm run build

and then also the front-end with “build” ?

front-end > npm install
front-end > npm run build

what will not work in build mode ?

ip:3000\graphql ?
pgAdmin ?
ip:3000\profile
ip:3000\explorer

Are there any other open windows ?

One of my tasks is to semantically search for information inside rvt (IFC actually). So, can I query one of the tables of the database to get the rvt model to transform it IFC ? We have found ways to transform IFC to RDF and do semantics queries.

             List of relations

Schema | Name | Type | Owner
--------±------------------------±------±--------
public | api_tokens | table | speckle
public | authorization_codes | table | speckle
public | branch_commits | table | speckle
public | branches | table | speckle
public | commits | table | speckle
public | knex_migrations | table | speckle
public | knex_migrations_lock | table | speckle
public | object_children_closure | table | speckle
public | objects | table | speckle
public | personal_api_tokens | table | speckle
public | pwdreset_tokens | table | speckle
public | refresh_tokens | table | speckle
public | scopes | table | speckle
public | server_acl | table | speckle
public | server_apps | table | speckle
public | server_apps_scopes | table | speckle
public | server_config | table | speckle
public | server_invites | table | speckle
public | stream_acl | table | speckle
public | stream_commits | table | speckle
public | streams | table | speckle
public | token_scopes | table | speckle
public | user_roles | table | speckle
public | user_server_app_tokens | table | speckle
public | users | table | speckle
(25 rows)

Best,
Dimitrios@PrismArch

Hi,

For the frontend, npm run build only compiles the Vue app and doesn’t run any service with it.
npm run build is used to build the package, not run it (the run in npm run is just for executing a script, and build is the name of the script, defined in package.json, under scripts)

Again, these are features useful for development of the packages. If you want to be able to access them in the browser, you must use nginx or the nodejs process to serve the files in development mode (npm run serve)

If you just want to run a Speckle server, we automated the build and running processes and created a docker-compose file that you can use to get up and running without issues like these.

Maybe try Deploying a local Server | Speckle Docs ?

So first, make sure you get the latest version from git.
Then, edit the docker-compose-speckle.yml:

  • change the environment variables for speckle-server: POSTGRES_URL to 127.0.0.1, REDIS_URL to 127.0.0.1, CANONICAL_URL and SESSION_SECRET.
  • if you run postgres and redis on the VM directly, add: network_mode: host right below restart: always to both frontend definition and the server definition (with the same level of indentation as restart: always)

Then run the docker-compose up command as described in the documentation. You can check logs with docker-compose -f docker-compose-speckle.yml logs

You should be able to access the speckle server in the browser at http://PUBLIC_IP_ADDRESS:3000/

ok,

so the production mode is dockerized, so no changes can be easily done. Since our project is exploratory in virtual reality, we will stick to dev mode (for the time being)

I have tried to invite new members from

but I get this:

Should the user be already registered to the server to send the invite ?

Sorry probably due to the fact that I forgot to setup mails in .env

Should I rebuild or just edit and run dev ?

D.

Ha, yes! you beat me to it.

You’ll need to provide the following env variables to be able to send emails:

We’re using mailjet. There are other smtp providers, such as sendgrid, amazon ses, etc. that you could try. Alternatively, just ask everyone to join… manually :slight_smile:

PS: and yes, you will need to restart the server process once those are filled in.

Hmmm,

when I am trying to commit from Rhino @home @mylaptop, It does not seem to connect to the server which is at the lab.

After a while it stops without commiting

Since the port is open, I suspect some priviledges issues ???

D.

Adding collaborators (as owners of the stream) in front-end works, but none of us can do any commits. Is there a log file location in the server ?

D.

Hey @Dimitrios.Ververidis seems like some misconfiguration on your server, are you able to use our public server at https://speckle.xyz/?

Yes I am uploading and downloading properly from speckle.xyz . Also my collaborators can send and receive in the same stream (we are testing Rhino). However, sometimes Rhino needs to restart in order to visualize the latest commit. Is there any refresh rendering button ?

I suspect that my server redirects the query to somewhere else (there is also an Apache with SSL server, next to Redis). Do you know which url calls exactly the SpeckleManager http://myip:3000/???
I am thinking to use a sniffer to see what is going on over internet. Does speckle manager have a console, logfile or any other inspection capability?

Best,
D.

There is a small refresh icon in the UI which should trigger a refresh of the streams and local accounts, let us know if you still experience issues with new commits and we’ll investigate!

image

Regarding Speckle Manager, I’m not sure what problems you are experiencing? The only things it does is setting up accounts and installing connectors, it’s not being used when sending or receiving data :slight_smile:
Are you having issues setting up accounts from your local server or from XYZ?

Thanks, I will check tomorrow about the refresh button.

As regards the second question, the problems regard the communication between Rhino and the speckle-server instance on our server. For your xyz server it works fine. We will play a little with your server as it is difficult to install it (moreover to update and maintain it) on our server. How can I persuade the Architects and the Engineers that xyz server is safe ? Do you have any ISO or any document about data safety-security?

Good point! There’s quite a list of security measures that are in place already at both application level and infrastructure level. Being still in a private beta, we haven’t gotten to “brag” about this. @cristi’s has been making speckle a safer place for quite a while now (it helps having a former security expert/devops ninja on the team!). I’ll let him rattle off when he’s got the time.

What are your concerns here? It will help with our answers!

To be clear: any data that goes into speckle.xyz is 100% yours only, visible only to you, and whoever else you choose to share with; speckle.xyz is bound by our terms of service and privacy policy which do not allow us to do anything with it beyond what you instruct us to.

Ah, as well, before I forget: because Speckle’s been used on a number of large infrastructure projects in the UK, Centre for the Protection of National Infrastructure has kindly allowed us to put their logo on our website since last January, after a technical conversation/audit. We never got to doing it, which kind of is a miss on our part, isn’t it?

It should be on the home page of the website. This is the first thing that Architects seek to.