Speckle server can't be built

dmuravskyi · 8 January 2025 09:24

Hello.

I’m investigating building and customizing of speckle server.
I use WLS2 (Ubuntu 22.04) with Windows 10 pro and took tag 2.23.4 from git.

However, when I try to follow " Local Development Environment" steps, I have an error on “yarn build” stage.

Other notable messages are

and

There are indeed circular dependencies in viewer, but , I suspect, everyone would suffer this ussie if this was the cause.

Few month ago, I already tried to build the server. I cloned repo, switched from master to 2.20.2 and got similar issue, nothing helped, like “rm -rf node_module” and “yarn cache clear”, what helped is to delete WSL distro, create new, clone directly 2.20.2 and build it.

But now, even this doesn’t help.

I tried using “docker compose -f docker-compose-speckle.yml up --build” and it actually built it.
I did minimal changes to server, everything was fine, but when I added minimum changes to frontend - it stopped working as well. I deleted images, switched back to 2.23.4 (there is no changes) but error persisted.

I’m confused as I can’t understand how to reliably develop, I try to delete everything again, but I want to understand that is the cause to avoid this issue in the future.
Can you explain:

What is the supported workflow of building the project for day to day development: docker compose or yarn
Are the any details related to build process I should know to avoid this issues?
What can I try to fix them?

I use node 18.20.5, docker 27.4.1. Other packages I installed are gcc/g++ v11.4 and cmake 3.22.1

dmuravskyi · 8 January 2025 16:18

I tried today to do step by step again:
wsl --install Ubuntu-22.04
git clone
sudo apt-get update
sudo apt-get install gcc g++ cmake // this is not in instruction but without it there is an error that compiler version can’t be detected during “yarn”)

nvm install 18

corepack enable
yarn
yarn build

It just said done without errors
After that I setup docker

yarn dev:docker:up
cp packages/server/.env-example packages/server/.env
cp packages/server/.env.test-example packages/server/.env.test
cp packages/frontend-2/.env.example packages/frontend-2/.env
cp packages/dui3/.env.example packages/dui3/.env
yarn dev

But still it failed to start

5:51:14 PM - Starting compilation in watch mode...
[@speckle/monitor-deployment]: [build:tsc:watch]
[@speckle/preview-service]: [run:watch] (node:4604) ExperimentalWarning: `--experimental-loader` may be removed in the future; instead use `register()`:
[@speckle/preview-service]: [run:watch] --import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("./dist/src/aliasLoader.js", pathToFileURL("./"));'
[@speckle/preview-service]: [run:watch] (Use `node --trace-warnings ...` to show where the warning was created)
[@speckle/objectloader]: (!) Unresolved dependencies
[@speckle/objectloader]: guide/en/warning-treating-module-as-external-dependency
[@speckle/objectloader]: #lodash (imported by src/index.js)
[@speckle/objectloader]: created dist/objectloader.esm.js, dist/objectloader.cjs in 5.2s
[@speckle/monitor-deployment]: [run:watch] (node:4616) ExperimentalWarning: `--experimental-loader` may be removed in the future; instead use `register()`:
[@speckle/monitor-deployment]: [run:watch] --import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("./dist/src/aliasLoader.js", pathToFileURL("./"));'
[@speckle/monitor-deployment]: [run:watch] (Use `node --trace-warnings ...` to show where the warning was created)
[@speckle/frontend-2]: [0]
[@speckle/frontend-2]: [0]   ➜ Local:    127.0.0.1:8081/
[@speckle/frontend-2]: [0]   ➜ Network:  use --host to expose
[@speckle/frontend-2]: [0]
[@speckle/monitor-deployment]: [run:watch] (node:4616) ExperimentalWarning: `--experimental-loader` may be removed in the future; instead use `register()`:
[@speckle/monitor-deployment]: [run:watch] --import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("./dist/src/aliasLoader.js", pathToFileURL("./"));'
[@speckle/monitor-deployment]: [run:watch] (Use `node --trace-warnings ...` to show where the warning was created)
[@speckle/monitor-deployment]: [run:watch] INFO: Initializing Prometheus metrics... {"component":"monitor-deployment"}
[@speckle/monitor-deployment]: [run:watch] INFO: Initializing monitoring metrics... {"component":"monitor-deployment"}
[@speckle/monitor-deployment]: [run:watch] INFO: 📊 Started Database monitor server {"component":"monitor-deployment/server"}
[@speckle/monitor-deployment]: [run:watch] INFO: Listening on port 9092 {"component":"monitor-deployment/server"}
[@speckle/dui3]: [0]
[@speckle/dui3]: [0]   > Local:    localhost:8082/
[@speckle/dui3]: [0]   > Network:  172.18.0.1:8082/
[@speckle/dui3]: [0]   > Network:  172.23.245.106:8082/
[@speckle/dui3]: [0]
[@speckle/preview-service]: [run:watch] (node:4604) ExperimentalWarning: `--experimental-loader` may be removed in the future; instead use `register()`:
[@speckle/preview-service]: [run:watch] --import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("./dist/src/aliasLoader.js", pathToFileURL("./"));'
[@speckle/preview-service]: [run:watch] (Use `node --trace-warnings ...` to show where the warning was created)
[@speckle/server]: [gqlgen] [STARTED] Parse Configuration
[@speckle/server]: [server] INFO: 🖼️  Serving for frontend-2...  {"phase":"startup"}
[@speckle/preview-service]: [build:webpack:watch] [BABEL] Note: The code generator has deoptimised the styling of /home/ambr/speckle-server/packages/viewer/dist/index.js as it exceeds the max of 500KB.
[@speckle/fileimport-service]: ERROR: Error executing task {"component":"fileimport-service"}
[@speckle/fileimport-service]:     err: {
[@speckle/fileimport-service]:       "type": "DatabaseError",
[@speckle/fileimport-service]:       "message": "\n    UPDATE file_uploads\n    SET\n      \"convertedStatus\" = 1,\n      \"convertedLastUpdate\" = NOW()\n    FROM (\n      SELECT \"id\" FROM file_uploads\n      WHERE \"convertedStatus\" = 0 AND \"uploadComplete\" = 't'\n      ORDER BY \"convertedLastUpdate\" ASC\n      LIMIT 1\n    ) as task\n    WHERE file_uploads.\"id\" = task.\"id\"\n    RETURNING file_uploads.\"id\"\n   - relation \"file_uploads\" does not exist",
[@speckle/fileimport-service]:       "stack":
[@speckle/fileimport-service]:           error:
[@speckle/fileimport-service]:               at Object.raw (/home/ambr/speckle-server/node_modules/knex/lib/knex-builder/make-knex.js:115:30)
[@speckle/fileimport-service]:               at Function.value (/home/ambr/speckle-server/node_modules/knex/lib/knex-builder/make-knex.js:91:29)
[@speckle/fileimport-service]:               at startTask (/home/ambr/speckle-server/packages/fileimport-service/src/daemon.js:36:31)
[@speckle/fileimport-service]:               at doStuff (/home/ambr/speckle-server/packages/fileimport-service/src/daemon.js:361:26)
[@speckle/fileimport-service]:               at async main (/home/ambr/speckle-server/packages/fileimport-service/src/daemon.js:386:3)
[@speckle/fileimport-service]:       "length": 111,
[@speckle/fileimport-service]:       "name": "error",
[@speckle/fileimport-service]:       "severity": "ERROR",
[@speckle/fileimport-service]:       "code": "42P01",
[@speckle/fileimport-service]:       "position": "13",
[@speckle/fileimport-service]:       "file": "parse_relation.c",
[@speckle/fileimport-service]:       "line": "1449",
[@speckle/fileimport-service]:       "routine": "parserOpenTable",
[@speckle/fileimport-service]:       "originalStack": "error: \n    UPDATE file_uploads\n    SET\n      \"convertedStatus\" = 1,\n      \"convertedLastUpdate\" = NOW()\n    FROM (\n      SELECT \"id\" FROM file_uploads\n      WHERE \"convertedStatus\" = 0 AND \"uploadComplete\" = 't'\n      ORDER BY \"convertedLastUpdate\" ASC\n      LIMIT 1\n    ) as task\n    WHERE file_uploads.\"id\" = task.\"id\"\n    RETURNING file_uploads.\"id\"\n   - relation \"file_uploads\" does not exist\n    at Parser.parseErrorMessage (/home/ambr/speckle-server/node_modules/pg-protocol/dist/parser.js:287:98)\n    at Parser.handlePacket (/home/ambr/speckle-server/node_modules/pg-protocol/dist/parser.js:126:29)\n    at Parser.parse (/home/ambr/speckle-server/node_modules/pg-protocol/dist/parser.js:39:38)\n    at Socket.<anonymous> (/home/ambr/speckle-server/node_modules/pg-protocol/dist/index.js:11:42)\n    at Socket.emit (node:events:517:28)\n    at addChunk (node:internal/streams/readable:368:12)\n    at readableAddChunk (node:internal/streams/readable:341:9)\n    at Readable.push (node:internal/streams/readable:278:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)"
[@speckle/fileimport-service]:     }
[@speckle/ui-components]: @storybook/cli v8.1.10
[@speckle/ui-components]:
[@speckle/dui3]: [1] [STARTED] Parse Configuration
[@speckle/dui3]: [1] [SUCCESS] Parse Configuration
[@speckle/dui3]: [1] [STARTED] Generate outputs
[@speckle/dui3]: [1] [STARTED] Generate to ./lib/common/generated/gql/
[@speckle/dui3]: [0] [nuxt:tailwindcss] ℹ Using Tailwind CSS from ~/assets/css/tailwind.css
[@speckle/server]: [gqlgen] [SUCCESS] Parse Configuration
[@speckle/server]: [gqlgen] [STARTED] Generate outputs
[@speckle/server]: [gqlgen] [STARTED] Generate to modules/core/graph/generated/graphql.ts
[@speckle/server]: [gqlgen] [STARTED] Generate to modules/cross-server-sync/graph/generated/graphql.ts
[@speckle/server]: [gqlgen] [STARTED] Generate to test/graphql/generated/graphql.ts
[@speckle/server]: [gqlgen] [STARTED] Load GraphQL schemas
[@speckle/server]: [gqlgen] [STARTED] Load GraphQL schemas
[@speckle/server]: [gqlgen] [STARTED] Load GraphQL schemas
[@speckle/preview-service]: [run:watch] INFO: Initializing Prometheus metrics... {"component":"preview-service"}
[@speckle/preview-service]: [run:watch] INFO: 📸 Starting Preview Service background worker {"component":"preview-service/backgroundWorker"}
[@speckle/preview-service]: [run:watch] INFO: Initializing Prometheus metrics... {"component":"preview-service"}
[@speckle/preview-service]: [run:watch] INFO: Prometheus metrics already initialized {"component":"preview-service"}
[@speckle/preview-service]: [run:watch] INFO: 📡 Started Preview Service server {"component":"preview-service/server"}
[@speckle/preview-service]: [run:watch] INFO: Listening on port 3001 {"component":"preview-service/server"}
[@speckle/preview-service]: [run:watch] INFO: 📊 Started Preview Service metrics server {"component":"preview-service/server"}
[@speckle/preview-service]: [run:watch] INFO: Listening on port 9094 {"component":"preview-service/server"}
[@speckle/preview-service]: [run:watch] ERROR: Error executing task {"component":"preview-service/backgroundWorker"}
[@speckle/preview-service]: [run:watch]     err: {
[@speckle/preview-service]: [run:watch]       "type": "DatabaseError",
[@speckle/preview-service]: [run:watch]       "message": "\n    UPDATE object_preview\n    SET\n      \"previewStatus\" = 1,\n      \"lastUpdate\" = NOW()\n    FROM (\n      SELECT \"streamId\", \"objectId\" FROM object_preview\n      WHERE \"previewStatus\" = 0 OR (\"previewStatus\" = 1 AND \"lastUpdate\" < NOW() - INTERVAL '1 WEEK')\n      ORDER BY \"priority\" ASC, \"lastUpdate\" ASC\n      LIMIT 1\n    ) as task\n    WHERE object_preview.\"streamId\" = task.\"streamId\" AND object_preview.\"objectId\" = task.\"objectId\"\n    RETURNING object_preview.\"streamId\", object_preview.\"objectId\"\n   - relation \"object_preview\" does not exist",
[@speckle/preview-service]: [run:watch]       "stack":
[@speckle/preview-service]: [run:watch]           error:
[@speckle/preview-service]: [run:watch]               at Object.raw (/home/ambr/speckle-server/node_modules/knex/lib/knex-builder/make-knex.js:115:30)
[@speckle/preview-service]: [run:watch]               at Function.value (/home/ambr/speckle-server/node_modules/knex/lib/knex-builder/make-knex.js:91:29)
[@speckle/preview-service]: [run:watch]               at Object.getNextUnstartedObjectPreview (file:///home/ambr/speckle-server/packages/preview-service/dist/src/repositories/objectPreview.js:4:43)
[@speckle/preview-service]: [run:watch]               at file:///home/ambr/speckle-server/packages/preview-service/dist/src/services/pollForPreview.js:5:33
[@speckle/preview-service]: [run:watch]               at startPreviewService (file:///home/ambr/speckle-server/packages/preview-service/dist/src/server/background.js:52:11)
[@speckle/preview-service]: [run:watch]               at async start (file:///home/ambr/speckle-server/packages/preview-service/dist/src/bin.js:6:5)
[@speckle/preview-service]: [run:watch]       "length": 113,
[@speckle/preview-service]: [run:watch]       "name": "error",
[@speckle/preview-service]: [run:watch]       "severity": "ERROR",
[@speckle/preview-service]: [run:watch]       "code": "42P01",
[@speckle/preview-service]: [run:watch]       "position": "13",
[@speckle/preview-service]: [run:watch]       "file": "parse_relation.c",
[@speckle/preview-service]: [run:watch]       "line": "1449",
[@speckle/preview-service]: [run:watch]       "routine": "parserOpenTable",
[@speckle/preview-service]: [run:watch]       "originalStack": "error: \n    UPDATE object_preview\n    SET\n      \"previewStatus\" = 1,\n      \"lastUpdate\" = NOW()\n    FROM (\n      SELECT \"streamId\", \"objectId\" FROM object_preview\n      WHERE \"previewStatus\" = 0 OR (\"previewStatus\" = 1 AND \"lastUpdate\" < NOW() - INTERVAL '1 WEEK')\n      ORDER BY \"priority\" ASC, \"lastUpdate\" ASC\n      LIMIT 1\n    ) as task\n    WHERE object_preview.\"streamId\" = task.\"streamId\" AND object_preview.\"objectId\" = task.\"objectId\"\n    RETURNING object_preview.\"streamId\", object_preview.\"objectId\"\n   - relation \"object_preview\" does not exist\n    at Parser.parseErrorMessage (/home/ambr/speckle-server/node_modules/pg-protocol/dist/parser.js:287:98)\n    at Parser.handlePacket (/home/ambr/speckle-server/node_modules/pg-protocol/dist/parser.js:126:29)\n    at Parser.parse (/home/ambr/speckle-server/node_modules/pg-protocol/dist/parser.js:39:38)\n    at Socket.<anonymous> (/home/ambr/speckle-server/node_modules/pg-protocol/dist/index.js:11:42)\n    at Socket.emit (node:events:517:28)\n    at addChunk (node:internal/streams/readable:368:12)\n    at readableAddChunk (node:internal/streams/readable:341:9)\n    at Readable.push (node:internal/streams/readable:278:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)"
[@speckle/preview-service]: [run:watch]     }
[@speckle/dui3]: [1] [STARTED] Load GraphQL schemas
[@speckle/dui3]: [1] [FAILED]
[@speckle/dui3]: [1] [FAILED]         Failed to load schema from 127.0.0.1:3000/graphql:
[@speckle/dui3]: [1] [FAILED]
[@speckle/dui3]: [1] [FAILED]         connect ECONNREFUSED 127.0.0.1:3000
[@speckle/dui3]: [1] [FAILED]         Error: connect ECONNREFUSED 127.0.0.1:3000
[@speckle/dui3]: [1] [FAILED]     at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16)
[@speckle/dui3]: [1] [FAILED]
[@speckle/dui3]: [1] [FAILED]         GraphQL Code Generator supports:
[@speckle/dui3]: [1] [FAILED]           - ES Modules and CommonJS exports (export as default or named export "schema")
[@speckle/dui3]: [1] [FAILED]           - Introspection JSON File
[@speckle/dui3]: [1] [FAILED]           - URL of GraphQL endpoint
[@speckle/dui3]: [1] [FAILED]           - Multiple files with type definitions (glob expression)
[@speckle/dui3]: [1] [FAILED]           - String in config file
[@speckle/dui3]: [1] [FAILED]
[@speckle/dui3]: [1] [FAILED]         Try to use one of above options and run codegen again.
[@speckle/dui3]: [1] [FAILED]
[@speckle/dui3]: [1] [FAILED]
[@speckle/dui3]: [1] [SUCCESS] Generate outputs
[@speckle/dui3]: [1]   ℹ Watching for changes in /home/ambr/speckle-server/packages/dui3...
[@speckle/frontend-2]: [0]
[@speckle/frontend-2]: [0]  WARN  [nuxt] Expected @nuxt/kit to be at least 3.12.2 but got 3.8.2. This might lead to unexpected behavior. Check your package.json or refresh your lockfile.
[@speckle/frontend-2]: [0]
[@speckle/frontend-2]: [0]
[@speckle/frontend-2]: [0]  WARN  [nuxt] Expected @nuxt/schema to be at least 3.12.2 but got 3.8.2. This might lead to unexpected behavior. Check your package.json or refresh your lockfile.
[@speckle/frontend-2]: [0]
[@speckle/frontend-2]: [1] [STARTED] Parse Configuration
[@speckle/frontend-2]: [1] [SUCCESS] Parse Configuration
[@speckle/frontend-2]: [1] [STARTED] Generate outputs
[@speckle/frontend-2]: [1] [STARTED] Generate to ./lib/common/generated/gql/
[@speckle/frontend-2]: [1] [STARTED] Load GraphQL schemas
[@speckle/frontend-2]: [1] [FAILED]
[@speckle/frontend-2]: [1] [FAILED]         Failed to load schema from 127.0.0.1:3000/graphql:
[@speckle/frontend-2]: [1] [FAILED]
[@speckle/frontend-2]: [1] [FAILED]         connect ECONNREFUSED 127.0.0.1:3000
[@speckle/frontend-2]: [1] [FAILED]         Error: connect ECONNREFUSED 127.0.0.1:3000
[@speckle/frontend-2]: [1] [FAILED]     at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16)
[@speckle/frontend-2]: [1] [FAILED]
[@speckle/frontend-2]: [1] [FAILED]         GraphQL Code Generator supports:
[@speckle/frontend-2]: [1] [FAILED]           - ES Modules and CommonJS exports (export as default or named export "schema")
[@speckle/frontend-2]: [1] [FAILED]           - Introspection JSON File
[@speckle/frontend-2]: [1] [FAILED]           - URL of GraphQL endpoint
[@speckle/frontend-2]: [1] [FAILED]           - Multiple files with type definitions (glob expression)
[@speckle/frontend-2]: [1] [FAILED]           - String in config file
[@speckle/frontend-2]: [1] [FAILED]
[@speckle/frontend-2]: [1] [FAILED]         Try to use one of above options and run codegen again.
[@speckle/frontend-2]: [1] [FAILED]
[@speckle/frontend-2]: [1] [FAILED]
[@speckle/frontend-2]: [1] [SUCCESS] Generate outputs
[@speckle/frontend-2]: [1]   ℹ Watching for changes in /home/ambr/speckle-server/packages/frontend-2...

dmuravskyi · 13 January 2025 09:46

The issue was related to WSL setup, by default it set 8 gb of memory and 2 gb swap. After I increased swap memory size to 8 gigabites everything was fixed.

dimitrie · 13 January 2025 16:39

@dmuravskyi we love when you solve your own problem thank you for reporting back the solution. It’s something of a given probably, partially why i upgraded my windows box to:

and i have some appropriately ridiculous allocation in my .wslconfig…