Server Setup with external managed postgres db

Help
1

Hello dear Specklers,

I’m currently trying to set up our production server. Maybe one of you guys can give me a hint, what I’m doing wrong. I have created a managed db of my provider of choice. Somehow Speckle does not find the connection, but PGadmin does.

  POSTGRES_URL: "name.postgres.database.azure.com"
  POSTGRES_USER: "adminpostgres"
  POSTGRES_PASSWORD: "xxxxx"
  POSTGRES_DB: "speckle"

the same login worked for me with PGadmin. With reference to this post
SSL Connection for Redis and Postgres - Help - Speckle Community

I have also tried to remove the ENV for user and password and entered the whole postgres://
into it. Did not work.

Checklist:

  • port 5432 of the firewall is open

Would be cool if you could lead me into the right direction :slight_smile:.

Thanks and all the best!
Alex

2

Hi @AlexHofbeck

Are you deploying the server with docker compose?

The speckle server container should generate detailed error logs. You can post them here if you need assistance interpreting them (though make sure to redact any sensitive info from them first!)

We’ve had some issues in the past where Azure includes symbols, such as @, within the username or password. If used within the POSTGRES_URL, these must be url encoded or they will be mis-interpreted.

3

[INFO wait] --------------------------------------------------------
[INFO wait] docker-compose-wait 2.8.0
[INFO wait] ---------------------------
[DEBUG wait] Starting with configuration:
[DEBUG wait] - Hosts to be waiting for: [postgres:5432, redis:6379, minio:9000]
[DEBUG wait] - Timeout before failure: 30 seconds
[DEBUG wait] - TCP connection timeout before retry: 5 seconds
[DEBUG wait] - Sleeping time before checking for hosts availability: 0 seconds
[DEBUG wait] - Sleeping time once all hosts are available: 0 seconds
[DEBUG wait] - Sleeping time between retries: 1 seconds
[DEBUG wait] --------------------------------------------------------
[INFO wait] Checking availability of postgres:5432
[INFO wait] Host postgres:5432 not yet available…

Sadly not so much to work with :frowning:

And username and password were straightforward … the worst thing happening was a minus in the password

4

by running docker-compose without the -d

attaching to speckle_webhook-service_1, speckle_speckle-server_1, speckle_fileimport-service_1, speckle_speckle-frontend_1, speckle_preview-service_1
speckle-frontend_1 | Starting nginx environment template rendering with ${HOSTNAME} ${PWD} ${OS_FLAVOUR} ${NODE_ENV} ${HOME} ${SHLVL} ${FILE_SIZE_LIMIT_MB} ${BITNAMI_APP_NAME} ${APP_VERSION} ${OS_NAME} ${PATH} ${OS_ARCH} ${}
speckle-frontend_1 | Nginx conf rendered, starting server…
speckle-frontend_1 | openresty 12:49:23.80
speckle-frontend_1 | openresty 12:49:23.80 Welcome to the Bitnami openresty container
speckle-frontend_1 | openresty 12:49:23.80 Subscribe to project updates by watching GitHub - bitnami/containers: Bitnami container images
speckle-frontend_1 | openresty 12:49:23.80 Submit issues and feature requests at Issues · bitnami/containers · GitHub
speckle-frontend_1 | openresty 12:49:23.80
speckle-frontend_1 | openresty 12:49:23.80 INFO ==> ** Starting OpenResty setup **
speckle-frontend_1 | openresty 12:49:23.81 INFO ==> Validating settings in OPENRESTY* env vars
speckle-frontend_1 | openresty 12:49:23.82 INFO ==> Initializing OpenResty
speckle-frontend_1 | realpath: /bitnami/openresty/conf/vhosts: No such file or directory
speckle-server_1 | [INFO wait] --------------------------------------------------------
speckle-server_1 | [INFO wait] docker-compose-wait 2.8.0
speckle-server_1 | [INFO wait] ---------------------------
speckle-server_1 | [DEBUG wait] Starting with configuration:
speckle-server_1 | [DEBUG wait] - Hosts to be waiting for: [postgres:5432, redis:6379, minio:9000]
speckle-server_1 | [DEBUG wait] - Timeout before failure: 30 seconds
speckle-server_1 | [DEBUG wait] - TCP connection timeout before retry: 5 seconds
speckle-server_1 | [DEBUG wait] - Sleeping time before checking for hosts availability: 0 seconds
speckle-server_1 | [DEBUG wait] - Sleeping time once all hosts are available: 0 seconds
speckle-server_1 | [DEBUG wait] - Sleeping time between retries: 1 seconds
speckle-server_1 | [DEBUG wait] --------------------------------------------------------
speckle-server_1 | [INFO wait] Checking availability of postgres:5432
speckle-server_1 | [INFO wait] Host postgres:5432 not yet available…
webhook-service_1 | {“level”:“info”,“time”:“2023-03-08T12:49:23.714Z”,“component”:“webhook-service”,“msg”:“Starting Webhook Service…”}
webhook-service_1 | {“level”:“error”,“time”:“2023-03-08T12:49:23.784Z”,“component”:“webhook-service”,“err”:{“type”:“Error”,“message”:“getaddrinfo ENOTFOUND postgres”,“stack”:“Error: getaddrinfo ENOTFOUND postgres\n at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26)”,“errno”:-3008,“code”:“ENOTFOUND”,“syscall”:“getaddrinfo”,“hostname”:“postgres”},“msg”:“Error executing task”}
speckle-frontend_1 | openresty 12:49:23.83 INFO ==> ** OpenResty setup finished! **
speckle-frontend_1 |
speckle-frontend_1 | openresty 12:49:23.83 INFO ==> ** Starting OpenResty **
speckle-frontend_1 | 2023/03/08 12:49:23 [notice] 1#1: using the “epoll” event method
speckle-frontend_1 | 2023/03/08 12:49:23 [notice] 1#1: openresty/1.21.4.1
speckle-frontend_1 | 2023/03/08 12:49:23 [notice] 1#1: OS: Linux 5.15.0-1033-azure
speckle-frontend_1 | 2023/03/08 12:49:23 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
speckle-frontend_1 | 2023/03/08 12:49:23 [notice] 1#1: start worker processes
speckle-frontend_1 | 2023/03/08 12:49:23 [notice] 1#1: start worker process 33
fileimport-service_1 | {“level”:“info”,“time”:“2023-03-08T12:49:24.025Z”,“component”:“fileimport-service”,“msg”:“Starting FileUploads Service…”}
fileimport-service_1 | {“level”:“error”,“time”:“2023-03-08T12:49:24.050Z”,“component”:“fileimport-service”,“err”:{“type”:“Error”,“message”:“getaddrinfo ENOTFOUND postgres”,“stack”:“Error: getaddrinfo ENOTFOUND postgres\n at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26)”,“errno”:-3008,“code”:“ENOTFOUND”,“syscall”:“getaddrinfo”,“hostname”:“postgres”},“msg”:“Error executing task”}
preview-service_1 | {“level”:“info”,“time”:“2023-03-08T12:49:24.132Z”,“component”:“preview-service”,“component”:“preview-service/server”,“msg”:“Listening on port 3001”}
preview-service_1 | {“level”:“info”,“time”:“2023-03-08T12:49:24.133Z”,“component”:“preview-service”,“msg”:“:camera_flash: Started Preview Service”}
preview-service_1 | {“level”:“error”,“time”:“2023-03-08T12:49:24.151Z”,“component”:“preview-service”,“err”:{“type”:“Error”,“message”:“getaddrinfo ENOTFOUND postgres”,“stack”:“Error: getaddrinfo ENOTFOUND postgres\n at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26)”,“errno”:-3008,“code”:“ENOTFOUND”,“syscall”:“getaddrinfo”,“hostname”:“postgres”},“msg”:“Error executing task”}
speckle-server_1 | [INFO wait] Host postgres:5432 not yet available…
speckle-server_1 | [INFO wait] Host postgres:5432 not yet available…
speckle-server_1 | [INFO wait] Host postgres:5432 not yet available…
speckle-server_1 | [INFO wait] Host postgres:5432 not yet available…
speckle-server_1 | [INFO wait] Host postgres:5432 not yet available…
webhook-service_1 | {“level”:“error”,“time”:“2023-03-08T12:49:28.788Z”,“component”:“webhook-service”,“err”:{“type”:“Error”,“message”:“getaddrinfo ENOTFOUND postgres”,“stack”:“Error: getaddrinfo ENOTFOUND postgres\n at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26)”,“errno”:-3008,“code”:“ENOTFOUND”,“syscall”:“getaddrinfo”,“hostname”:“postgres”},“msg”:“Error executing task”}
fileimport-service_1 | {“level”:“error”,“time”:“2023-03-08T12:49:29.060Z”,“component”:“fileimport-service”,“err”:{“type”:“Error”,“message”:“getaddrinfo ENOTFOUND postgres”,“stack”:“Error: getaddrinfo ENOTFOUND postgres\n at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26)”,“errno”:-3008,“code”:“ENOTFOUND”,“syscall”:“getaddrinfo”,“hostname”:“postgres”},“msg”:“Error executing task”}
preview-service_1 | {“level”:“error”,“time”:“2023-03-08T12:49:29.156Z”,“component”:“preview-service”,“err”:{“type”:“Error”,“message”:“getaddrinfo ENOTFOUND postgres”,“stack”:“Error: getaddrinfo ENOTFOUND postgres\n at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26)”,“errno”:-3008,“code”:“ENOTFOUND”,“syscall”:“getaddrinfo”,“hostname”:“postgres”},“msg”:“Error executing task”}

5

It appears that docker-compose is waiting for the postgres container.
This looks like you may have ran docker-compose-deps.yml, and removed the postgres block (with the intention of replacing it with a hosted Azure service), but didn’t remove the references to postgres in the various depends_on statements within the file?

6

From the logged hostname in the above statements, they suggest that the POSTGRES_URL in the docker-compose file for all these services is still set as postgres?

1 Like
7

And that was exactly the issue
… the references to the non-existing containers.
… and the wrong link at every container
→ using the right docker-compose.yml from the github solved the basic issues.

After that I had to cope (and struggle massively) with the right redis link and postgres link naming. Worked out in the end :slight_smile:. Thanks @iainsproat

1 Like
8

Glad you got it sorted @AlexHofbeck !

Did you find the broken docker-compose file somewhere in our documentation (perhaps I need to update something somewhere)?

9

It was more the classic user issue … Your documentation was not the issue.
It was laziness paired with half-knowledge :sweat_smile:. Starting to get the hang of it though :smile:

3 Likes