Invite Mails cannot be send from our own deployment

AlexHofbeck · 16 July 2022 20:50

Hello guys and gals,

I’m currently setting up a Speckle Server for testing on our VM hosted on Azure. It works like a charm except for the mail invites … the server is not able to send anything to the outside world. Also, I have double-checked for invites in the spam of the recipients … no invite mail there. I have also double-checked the other posts here quite intensively, but they also could not help me with my issue.

Maybe you can give me a hint, about what I forgot to think about … still trying to figure out the whole thing beginning with the ABC of everything .

Step 1:
I have opened the SMTP Port 587 for the VM and also checked the open port via telnet. Port on the VM is fine

azureuser@speckle2-dev:~/opt/speckle$ telnet smtp.office365.com 587
Trying 52.98.152.194...
Connected to HHN-efz.ms-acdc.office.com.
Escape character is '^]'.
220 FR0P281CA0012.outlook.office365.com Microsoft ESMTP MAIL Service ready at Sat, 16 Jul 2022 20:32:22 +0000

Step 2:
In order to invite users, I’ve added the Mail Services to the docker-compose.yml. I took care of the right indention … I believe it was 6 spaces

  EMAIL: "true"
  EMAIL_HOST: "smtp.office365.com"
  EMAIL_PORT: "587"
  EMAIL_USERNAME: "address@bollinger-grohmann.de"
  EMAIL_PASSWORD: "xyz"
  EMAIL_FROM: "address@bollinger-grohmann.de"

as this didn’t work so far, I thought that maybe the docker container of the server needs an open port 587 as well:

  speckle-server:
    image: speckle/speckle-server:2
    restart: always
    ports:
      - "127.0.0.1:587:587"

I have added the localhost with port mapping.

and checked it in the “docker container ls”:

> bc3c1b82e6c7 speckle/speckle-server:2 "docker-entrypoint.s…" 12 minutes ago Up 12 minutes 127.0.0.1:587->587/tcp speckle_speckle-server_1

sadly still no mail … maybe you have an idea, where this issue might be … or is this mail service attached to another service and not to the speckle-server? As there is already a port given in the environment variable … it should work without adding anything else to the yml.

Thanks in advance for your help on this matter

gjedlicska · 19 July 2022 09:20

Hello Alex,

I feel your pain there, email in general can be complicated to deal with.

The STMP port doesn’t need to be open on the Speckle server side, in this context, the Speckle server is just a client to the SMTP server. And clients can make requests to any port open on a server, in this case the SMTP server.

So you should set back the port configuration changes to the default, otherwise the server might not work properly.

I’d try to go at the problem from 2 ways, from the client (Speckle server) config side and the Email server side.

does the Speckle server email part work not using your Office365 email service? To do this, i recommend testing with something like Ethereal, a free throwaway email testing service. ( This is also what we use to test stuff ) If you can catch the invite mails like this, that means that the mailing functionality works on the server.
If step 1 works, try to configure it back to your Office365 email. If that doesn’t work, you should check the logs of the server container with something like:
$ docker-compose logs speckle-server-speckle-server-1.
You can look up the name of the server container with $ docker ps
At this point to be more helpful. I’d have to see the error message produced by the server, but I’d look in a direction like this.

Hope this helps, and lmk if it doesn’t solve the issue.

AlexHofbeck · 19 July 2022 09:30

Thank you Gergő! Will dive into this and let you know if it fixes the thing

AlexHofbeck · 19 July 2022 11:57

I have tried this on my private gmail account as well. Ethereal also did not show anything. So the mail was not contacted by the VM :(.

Here is the section of the log of the docker-compose

speckle-server_1      | 2022-07-19T11:35:13.982Z speckle:db-startup Loaded knex conf for production
speckle-server_1      | 2022-07-19T11:35:14.896Z speckle:modules 💥 Init core module
speckle-server_1      | 2022-07-19T11:35:14.965Z speckle:modules 🔑 Init auth module
speckle-server_1      | 2022-07-19T11:35:15.119Z speckle:modules 💅 Init graphql api explorer module
speckle-server_1      | 2022-07-19T11:35:15.119Z speckle:modules 📧 Init emails module
speckle-server_1      | 2022-07-19T11:35:15.120Z speckle:modules 📧 Email provider is not configured. Server functionality will be limited.
speckle-server_1      | 2022-07-19T11:35:15.122Z speckle:modules ♻️  Init pwd reset module
speckle-server_1      | 2022-07-19T11:35:15.124Z speckle:modules 💌 Init invites module
speckle-server_1      | 2022-07-19T11:35:15.177Z speckle:modules 📸 Init object preview module
speckle-server_1      | 2022-07-19T11:35:15.262Z speckle:modules 📄 Init FileUploads module
speckle-server_1      | 2022-07-19T11:35:15.290Z speckle:modules 🗣  Init comments module (barebones)
speckle-server_1      | 2022-07-19T11:35:15.736Z speckle:startup 🚀 My name is Speckle Server, and I'm running at 0.0.0.0:3000
speckle-server_1      | 2022-07-19T11:35:39.542Z speckle:errors No email transport present. Cannot send emails.

Although the provider input was entered, it seems that the server doesn’t recognize it. I have added the environment variables for mail below the webservices … as it was somewhere also shown like this.

Also I did not define them from the beginning. Is this the issue?

Sorry for this noob questions ^^

gjedlicska · 19 July 2022 12:42

Hey Alex,

there are no noob questions here
And also this stuff can be a lot of trial and error sometimes.
I’ve tested again if if works for me starting from the docker-compose file we give out as an example on our dev docs, and it works for me.

Here is my working example, with throwaway values. Maybe double check your values and syntax, cause the email transport should work.

version: "2"
services:
  ####
  # Speckle Server dependencies
  #######
  postgres:
    image: "postgres:13.1-alpine"
    restart: always
    environment:
      POSTGRES_DB: speckle
      POSTGRES_USER: speckle
      POSTGRES_PASSWORD: speckle
    volumes:
      - ./postgres-data:/var/lib/postgresql/data/
    ports:
      - "127.0.0.1:5432:5432"

  redis:
    image: "redis:6.0-alpine"
    restart: always
    volumes:
      - ./redis-data:/data
    ports:
      - "127.0.0.1:6379:6379"

  minio:
    image: "minio/minio"
    command: server /data --console-address ":9001"
    restart: always
    volumes:
      - ./minio-data:/data
    ports:
      - "127.0.0.1:9000:9000"
      - "127.0.0.1:9001:9001"

  ####
  # Speckle Server
  #######
  speckle-frontend:
    image: speckle/speckle-frontend:2
    restart: always
    ports:
      - "0.0.0.0:80:80"

  speckle-server:
    image: speckle/speckle-server:2
    restart: always
    command: ["bash", "-c", "/wait && node bin/www"]
    environment:
      # TODO: Change this to the URL of the speckle server, as accessed from the network
      CANONICAL_URL: "http://localhost"

      # TODO: Change this to a unique secret for this server
      SESSION_SECRET: "TODO:ReplaceWithLongString"

      STRATEGY_LOCAL: "true"
      DEBUG: "speckle:*"

      POSTGRES_URL: "postgres"
      POSTGRES_USER: "speckle"
      POSTGRES_PASSWORD: "speckle"
      POSTGRES_DB: "speckle"

      REDIS_URL: "redis://redis"
      
      S3_ENDPOINT: "http://minio:9000"
      S3_ACCESS_KEY: "minioadmin"
      S3_SECRET_KEY: "minioadmin"
      S3_BUCKET: "speckle-server"
      S3_CREATE_BUCKET: "true"
      
      WAIT_HOSTS: postgres:5432, redis:6379, minio:9000
      EMAIL: true
      EMAIL_HOST: "smtp.ethereal.email"
      EMAIL_PORT: "587"
      EMAIL_USERNAME: "jerad.quigley@ethereal.email"
      EMAIL_PASSWORD: "6p14P6kSFTFTzwFX5Z"
      EMAIL_FROM: "jerad.quigley@ethereal.email"

  preview-service:
    image: speckle/speckle-preview-service:2
    restart: always
    mem_limit: "1000m"
    memswap_limit: "1000m"
    command: ["bash", "-c", "/wait && node bin/www"]
    environment:
      DEBUG: "preview-service:*"
      PG_CONNECTION_STRING: "postgres://speckle:speckle@postgres/speckle"
      WAIT_HOSTS: postgres:5432

  webhook-service:
    image: speckle/speckle-webhook-service:2
    restart: always
    command: ["bash", "-c", "/wait && node main.js"]
    environment:
      DEBUG: "webhook-service:*"
      PG_CONNECTION_STRING: "postgres://speckle:speckle@postgres/speckle"
      WAIT_HOSTS: postgres:5432

  fileimport-service:
    image: speckle/speckle-fileimport-service:2
    restart: always
    command: ["bash", "-c", "/wait && node src/daemon.js"]
    environment:
      DEBUG: "fileimport-service:*"
      PG_CONNECTION_STRING: "postgres://speckle:speckle@postgres/speckle"
      WAIT_HOSTS: postgres:5432

      S3_ENDPOINT: "http://minio:9000"
      S3_ACCESS_KEY: "minioadmin"
      S3_SECRET_KEY: "minioadmin"
      S3_BUCKET: "speckle-server"

      SPECKLE_SERVER_URL: "http://speckle-server:3000"

dimitrie · 19 July 2022 13:02

Just chiming in to say that you probably can get quite far on a free account from mailjet, sendgrid or alternatives, in case everything else fails! This part of the setup is always a PITA. It’s something we ourselves have happily outsourced to a third party service (using a nice neutral api, of course!).

On another note, you cannot really use your email credentials that easy to send away email like from an email client (not sure if you’ve set something up using Azure or you’re using your existing details). If this ends up working, I’ll eat humble pie

AlexHofbeck · 19 July 2022 13:04

It did the trick … thank you for pushing me into the right direction @gjedlicska … I put it in the wrong place. I put it at the end of the document … and I don’t know where I have seen that . Classic user error

gjedlicska · 19 July 2022 13:24

TADAA, happened to all of us

Is it working with the Office365 email too?

AlexHofbeck · 19 July 2022 13:49

works pretty good

joyshmitz · 14 July 2023 12:05

Hello everyone,

I wanted to share some insights about a topic that has been a significant part of my recent work - using Google’s SMTP server with less secure apps.

For those who may not be familiar, SMTP stands for Simple Mail Transfer Protocol, and it’s the standard for sending email across the internet. Google provides an SMTP server that anyone with a Gmail account can use. This can be particularly useful for sending automated emails from your application.

However, Google has implemented strict security measures to protect its users’ data. As a result, by default, Google blocks attempts to send emails through its SMTP server from what it considers “less secure apps”. These are applications that do not use OAuth 2.0 for authentication.

Despite the name, “less secure” doesn’t necessarily mean these apps are vulnerable or risky to use. It simply means they use a less secure method of logging in to your account, typically by asking for your username and password directly.

To use Google’s SMTP server with such an app, you need to enable “Less Secure Apps” in your Google account settings. This allows the app to send emails through Google’s SMTP server using your Gmail account.

Fill in the part responsible for interaction, with email in the docker-compose file, like I did

EMAIL: "true"
      EMAIL_HOST: "smtp.gmail.com"
      EMAIL_PORT: "587"
      EMAIL_USERNAME: "you-speckle@youdomain.com"
      EMAIL_PASSWORD: "you-password"
      EMAIL_FROM: "no-reply@youdomain.com"

That’s all for now. I hope this post was informative and helpful.

Less secure apps & your Google Account

jonathon · 24 August 2024 21:52