Using Docker Speckle 2 Img

markcichy · 26 July 2021 15:52

I’m having an issue where I am not able to login while setting up a Speckle 2.0 instance within my own infrastructure (ie NOT using DO).

I have followed the instructions/docs posted here:
“Deploying a Server - manual setup”
https://speckle.guide/dev/server-manualsetup.html

*Note: docs should probably include a note to comment out lines 57 through 66 in your sample docker-compose.yml as including them prevents the container from starting - need to choose live or preview-service, cannot be both

DOMAIN https://speckletwo.mobiusnode.io

The issue appears to be one of authentication, as in no .env file has been defined allowing “at least one auth method” as is and was standard with Speckle 1.0 and other builds. Though, the docs did not mention where to store and how to ref in the docker-compose file so I left out.

SCREENSHOT

OTHER DETAILS
The server is accessed via an Nginx reverse proxy that forwards traffic to the server running Docker/Compose - encrypting as connections are made. Nothing strange or unique here, I can confirm that forward, https, and server block all good with no errors.

markcichy · 26 July 2021 15:53

@cristi @dimitrie @teocomi any immediate help ya’ll can offer is much appreciated! M

cristi · 26 July 2021 16:27

@markcichy i just tested on a new VM:

installed docker and docker-compose with linux package managers (you can also install the latest version as described in their docs)
copy and pasted the docker-compose.yml file from the docs in a file on the VM
edited the CANONICAL_URL env variable
ran docker-compose up -d
Result: http://178.128.167.29/

To debug your problem, can you paste here the docker-compose.yml that you use and the nginx reverse proxy config for this domain?

Also, can you run docker-compose pull to make sure you have the latest docker images on that VM

markcichy · 26 July 2021 16:48

@cristi here’s is my docker-compose.yml

version: "3"
services:
  ####
  # Speckle Server dependencies
  #######
  postgres:
    image: "postgres:13.1-alpine"
    restart: always
    environment:
      POSTGRES_DB: speckle
      POSTGRES_USER: speckle
      POSTGRES_PASSWORD: <MASKED>
    volumes:
      - ./postgres-data:/var/lib/postgresql/data/
    ports:
      - "127.0.0.1:5432:5432"

  redis:
    image: "redis:6.0-alpine"
    restart: always
    volumes:
      - ./redis-data:/data
    ports:
      - "127.0.0.1:6379:6379"

  ####
  # Speckle Server
  #######
  speckle-frontend:
    image: speckle/speckle-frontend:2
    restart: always
    ports:
      - "0.0.0.0:80:80"

  speckle-server:
    image: speckle/speckle-server:2
    restart: always
    command: ["bash", "-c", "/wait && node bin/www"]
    environment:
      # TODO: Change this to the URL of the speckle server, as accessed from the network
      CANONICAL_URL: "https://speckletwo.mobiusnode.io"

      SESSION_SECRET: "<MASKED>"

      STRATEGY_LOCAL: "true"
      DEBUG: "speckle:*"

      POSTGRES_URL: "postgres"
      POSTGRES_USER: "speckle"
      POSTGRES_PASSWORD: "<MASKED>"
      POSTGRES_DB: "speckle"

      REDIS_URL: "redis://redis"
      WAIT_HOSTS: postgres:5432, redis:6379

# preview-service:
#     image: speckle/speckle-preview-service:2
#     restart: always
#     mem_limit: "1000m"
#     memswap_limit: "1000m"
#     command: ["bash", "-c", "/wait && node bin/www"]
#     environment:
#       DEBUG: "preview-service:*"
#       PG_CONNECTION_STRING: "postgres://speckle:speckle@postgres/speckle"
#       WAIT_HOSTS: postgres:5432

And my Nginx server block

                                      
server {
        # ssl
        listen  443 ssl  http2;
        listen [::]:443 ssl http2;

        server_name speckletwo.mobiusnode.io;

        location / {
                #http trafic
                proxy_pass http://192.168.86.36;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header Host $host;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

                #ws support
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
                server_tokens off;
        }

    ssl_certificate /etc/letsencrypt/live/speckletwo.mobiusnode.io/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/speckletwo.mobiusnode.io/privkey.pem; # managed by Certbot

}

server {
    if ($host = speckletwo.mobiusnode.io) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    listen 80;
    listen [::]:80;

    server_name speckletwo.mobiusnode.io;
    return 404; # managed by Certbot

}

I can confirm: docker image is up to date - was prior to your reply.

markcichy · 26 July 2021 16:59

@cristi I’m wondering if my Nginx proxy is stripping the CSS and headers of the Speckle server, preventing me from being able to auth.

cristi · 26 July 2021 17:01

@markcichy

The probable reason the preview-service container gives an error is because you have version: "3" as first line of the file, instead of version: "2" as in the docs. But this seems unrelated to the issue.

Can you run docker-compose logs speckle-server and paste the output?

It looks like all your server URLs are returning 404 errors

markcichy · 26 July 2021 17:05

@cristi output exceeds body limit!
Here is the txt:
https://drive.google.com/file/d/1LWoA2DuCq7ndv1dQC66EMbDXVM6gME91/view?usp=sharing

cristi · 26 July 2021 17:06

@markcichy and also, more importantly, some of the last lines of docker-compose logs speckle-frontend, some of the lines that contain graphql

(
if there is too much output, you can filter with:

docker-compose logs speckle-frontend | grep graphql | tail

)

markcichy · 26 July 2021 17:07

@cristi tail output:

speckle-frontend_1  | 2021/07/26 17:01:35 [error] 7#7: *593 speckle-server could not be resolved (2: Server failure), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 2021/07/26 17:01:35 [error] 7#7: *593 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 2021/07/26 17:01:35 [error] 7#7: *592 speckle-server could not be resolved (2: Server failure), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/"
speckle-frontend_1  | 2021/07/26 17:01:35 [error] 7#7: *592 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/"
speckle-frontend_1  | 2021/07/26 17:01:35 [error] 7#7: *594 speckle-server could not be resolved (2: Server failure), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 2021/07/26 17:01:35 [error] 7#7: *594 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 192.168.86.101 - - [26/Jul/2021:17:01:35 +0000] "POST /graphql HTTP/1.1" 404 561 "https://speckletwo.mobiusnode.io/authn/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36"
speckle-frontend_1  | 2021/07/26 17:01:46 [error] 7#7: *596 speckle-server could not be resolved (2: Server failure), client: 192.168.86.101, server: , request: "GET /graphql HTTP/1.1", host: "speckletwo.mobiusnode.io"
speckle-frontend_1  | 2021/07/26 17:01:46 [error] 7#7: *596 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.86.101, server: , request: "GET /graphql HTTP/1.1", host: "speckletwo.mobiusnode.io"
speckle-frontend_1  | 192.168.86.101 - - [26/Jul/2021:17:01:46 +0000] "GET /graphql HTTP/1.1" 404 561 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36"

cristi · 26 July 2021 17:09

@markcichy
Oh, the server looks like it cannot start because of invalid Postgres connection parameters.

Most probably because your password contains special characters that are not escaped properly somewhere.

Can you try changing the postgres password to either the default or some password that contains only letters and numbers?

Anyway, can you mention the special character(s) that you used in the password so that we look into fixing the encoding issue?

markcichy · 26 July 2021 17:10

Okay, one moment… will take down and bring back up.
@cristi does the same hold true for the session secret?

cristi · 26 July 2021 17:14

No, it shouldn’t, but we haven’t tested with all the special characters.

But after all, we pass them to other libraries and can’t know all limitations

markcichy · 26 July 2021 17:16

Seem to be the same issue, still not working that is https://speckletwo.mobiusnode.io

speckle-frontend_1  | 192.168.86.101 - - [26/Jul/2021:17:15:45 +0000] "POST /graphql HTTP/1.1" 404 159 "https://speckletwo.mobiusnode.io/authn/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1 Safari/605.1.15"
speckle-frontend_1  | 2021/07/26 17:15:45 [error] 7#7: *627 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 2021/07/26 17:15:45 [error] 7#7: *627 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 192.168.86.101 - - [26/Jul/2021:17:15:45 +0000] "POST /graphql HTTP/1.1" 404 159 "https://speckletwo.mobiusnode.io/authn/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1 Safari/605.1.15"
speckle-frontend_1  | 2021/07/26 17:15:45 [error] 7#7: *628 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 2021/07/26 17:15:45 [error] 7#7: *628 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 192.168.86.101 - - [26/Jul/2021:17:15:45 +0000] "POST /graphql HTTP/1.1" 404 159 "https://speckletwo.mobiusnode.io/authn/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1 Safari/605.1.15"
speckle-frontend_1  | 2021/07/26 17:15:45 [error] 7#7: *631 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.86.101, server: , request: "GET /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io"
speckle-frontend_1  | 2021/07/26 17:15:45 [error] 7#7: *631 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.86.101, server: , request: "GET /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io"
speckle-frontend_1  | 192.168.86.101 - - [26/Jul/2021:17:15:45 +0000] "GET /graphql HTTP/1.1" 404 159 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1 Safari/605.1.15"

cristi · 26 July 2021 17:18

@markcichy
hmm, the previous error was clearly about the postgres connection url being invalid.

Can you paste the current error in the server logs?

markcichy · 26 July 2021 17:19

speckle-frontend_1  | 192.168.86.101 - - [26/Jul/2021:17:18:40 +0000] "GET /graphql HTTP/1.1" 404 561 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36"
speckle-frontend_1  | 2021/07/26 17:18:40 [error] 7#7: *34 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/"
speckle-frontend_1  | 2021/07/26 17:18:40 [error] 7#7: *34 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/"
speckle-frontend_1  | 192.168.86.101 - - [26/Jul/2021:17:18:40 +0000] "POST /graphql HTTP/1.1" 404 561 "https://speckletwo.mobiusnode.io/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36"
speckle-frontend_1  | 2021/07/26 17:18:40 [error] 7#7: *36 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 2021/07/26 17:18:40 [error] 7#7: *36 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 2021/07/26 17:18:40 [error] 7#7: *37 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 2021/07/26 17:18:40 [error] 7#7: *37 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.86.101, server: , request: "POST /graphql HTTP/1.1", upstream: "http://172.18.0.2:3000/graphql", host: "speckletwo.mobiusnode.io", referrer: "https://speckletwo.mobiusnode.io/authn/login"
speckle-frontend_1  | 192.168.86.101 - - [26/Jul/2021:17:18:40 +0000] "POST /graphql HTTP/1.1" 404 561 "https://speckletwo.mobiusnode.io/authn/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36"
speckle-frontend_1  | 192.168.86.101 - - [26/Jul/2021:17:18:40 +0000] "POST /graphql HTTP/1.1" 404 561 "https://speckletwo.mobiusnode.io/authn/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36"

markcichy · 26 July 2021 17:20

Attaching to speckle_speckle-server_1
speckle-server_1    | [INFO  wait] --------------------------------------------------------
speckle-server_1    | [INFO  wait]  docker-compose-wait 2.8.0
speckle-server_1    | [INFO  wait] ---------------------------
speckle-server_1    | [DEBUG wait] Starting with configuration:
speckle-server_1    | [DEBUG wait]  - Hosts to be waiting for: [postgres:5432, redis:6379]
speckle-server_1    | [DEBUG wait]  - Timeout before failure: 30 seconds 
speckle-server_1    | [DEBUG wait]  - TCP connection timeout before retry: 5 seconds 
speckle-server_1    | [DEBUG wait]  - Sleeping time before checking for hosts availability: 0 seconds
speckle-server_1    | [DEBUG wait]  - Sleeping time once all hosts are available: 0 seconds
speckle-server_1    | [DEBUG wait]  - Sleeping time between retries: 1 seconds
speckle-server_1    | [DEBUG wait] --------------------------------------------------------
speckle-server_1    | [INFO  wait] Checking availability of postgres:5432
speckle-server_1    | [INFO  wait] Host postgres:5432 is now available!
speckle-server_1    | [INFO  wait] --------------------------------------------------------
speckle-server_1    | [INFO  wait] Checking availability of  redis:6379
speckle-server_1    | [INFO  wait] Host  redis:6379 is now available!
speckle-server_1    | [INFO  wait] --------------------------------------------------------
speckle-server_1    | [INFO  wait] docker-compose-wait - Everything's fine, the application can now start!
speckle-server_1    | [INFO  wait] --------------------------------------------------------
speckle-server_1    | 2021-07-26T17:18:16.996Z speckle:db-startup Loaded knex conf for production
speckle-server_1    | 2021-07-26T17:18:17.347Z speckle:www error: password authentication failed for user "speckle"
speckle-server_1    |     at Parser.parseErrorMessage (/app/node_modules/pg-protocol/dist/parser.js:278:15)
speckle-server_1    |     at Parser.handlePacket (/app/node_modules/pg-protocol/dist/parser.js:126:29)
speckle-server_1    |     at Parser.parse (/app/node_modules/pg-protocol/dist/parser.js:39:38)
speckle-server_1    |     at Socket.<anonymous> (/app/node_modules/pg-protocol/dist/index.js:10:42)
speckle-server_1    |     at Socket.emit (events.js:315:20)
speckle-server_1    |     at Socket.EventEmitter.emit (domain.js:467:12)
speckle-server_1    |     at addChunk (internal/streams/readable.js:309:12)
speckle-server_1    |     at readableAddChunk (internal/streams/readable.js:284:9)
speckle-server_1    |     at Socket.Readable.push (internal/streams/readable.js:223:10)
speckle-server_1    |     at TCP.onStreamRead (internal/stream_base_commons.js:188:23)

markcichy · 26 July 2021 17:21

Password auth failed, because the postgres user is still trying to see the old pwd…
Can I blow away the old build and purge all file then recompose?
OR, just delete data folders?

cristi · 26 July 2021 17:22

@markcichy

Looks like the password is invalid.
Make sure you modifed it in both postgres container and speckle-server container env variables, and if it still doesn’t work, you can delete the data directory that postgres created to start with a clean slate: rm -rf postgres-data

markcichy · 26 July 2021 17:25

Worked!!!
Was 100% the password character issue.

Really appreciate your help!

markcichy · 26 July 2021 17:28

@cristi RE the special characters used were:
* > + %
*I simply used 1Password to generate a 24 string random password with no restrictions