Hi All, please may I have your help with my Speckle installation on Azure Kubernetes. When I get the 1st splash screen it prompts me to register a new user and password which is the admin user. Unfortunately I get “Failed to Fetch” error. When I look in the backend on the Postgres database I can see the user has been persisted to the user table. When I attempt to register again I get the user already exists but when I attempt to login, I get the same error “Failed to Fetch”.
Hi @shiangoli
Is there a more detailed error message to be found within the console of your browser’s developer tools?
Hi @iainsproat the issue was with my domain name in the values file. I changed it to include .com and this resolved the issue
3 Likes
That’s great, I’m glad you got it sorted! 
1 Like
Hi @iainsproat when the load balancer is behind an Application Gateway we’ve configured this to take the value of a private address for AKS which is domain-name.io but we still get failed to fetch error for the 1st registered user. Is there an issue with using *.io ?
It should be agnostic to the tld, so the .io tld is unlikely to be the issue.
Could you share the exact wording, or screenshot, of the fetch error? I assume it is thrown by the frontend?
My suspicions are it is due to the tls certificates having an invalid hostname or perhaps CORS, but we may be able to determine from the error message.
Hi Ian, it is the frontend, there isn’t much in the frontend screen shot please see attached. I couldn’t see the anything in the frontend logs or server logs to provide more detail
ah, sorry - I should have been more explicit. I meant a screenshot of the output in the console of your browser’s developer tools.
In @chuongmep’s case, I believe this is due to the frontend being requested via http. The frontend makes a request to the backend via https. Unfortunately the certificate for https was not correctly generated, and this likely resulted in the net::ERR_CONNECTION_REFUSED message we see in the console of the frontend. See https://speckle.community/t/re-issue-with-connect-domain-server/6279/6 for a possible solution for this issue in @chuongmep’s environment (Docker Compose deployed to a DigitalOcean Droplet).
It’s likely that the problem @shiangoli reported has similar roots, but without the error message from the console it is not possible to say for sure.
1 Like
My problem resolved by fill email to get certificate, cheer 
2 Likes
Hi @iainsproat, please see attached it appears to expect https to be used. We have configured SSL termination at the gateway and the backend non SSL. I would prefer if we were able to use SSL passthrough but struggled see the following
post Application Gateway fronting Speckle on Kubernetes - #3 by shiangoli
Thanks @shiangoli - it looks like the frontend is attempting to connect to nginx directly (via http), bypassing your domain gateway. Your browser will then flag the mixed https and http as insecure content and block the http call.
To help with my understanding of your configuration, can you confirm the values of domain and ssl_canonical_url in your helm chart values?
Hi @iainsproat the domain is set to the private url not the url in the public DNS and ssl_canonical_url is set to false
domain: nginx.privatelink.uksouth.azmk8s.io
ssl_canonical_url: false
Have you tried setting the domain value to that of your public url? What was the error/outcome with that configuration?
Hi @iainsproat we tried this before and just tried this again now and couldn’t get this to work 502 bad gateway error. It would be nice to know what would be the right configuration for fronting the service using application gateway and the different options SSL termination or pass through?
Thanks for trying that combination. Was the 502 error being generated by nginx or your gateway? What did the nginx logs show?
A gateway with separate domain and TLS termination is not something we’ve had to support before so hasn’t been tested. I’m not sure what would work, if anything.
There are a number of combinations here that you could work through to determine if any are possible. Which combinations have you already tried, and what were the outcomes?
- SSL
- passthrough
- termination at gateway, http to speckle-server nginx
- termination at gateway, with onward request using tls with self-signed certificates to speckle-server nginx (tls certificate is added to nginx)
-
domainvalue in helm chart- gateway url
- nginx url
-
ssl_canonical_urlvalue in helm chart- false (I think this only makes sense with termination at gateway)
- true
1 Like
Hi @iainsproat we’ve been experimenting with the various options
In the previous post we proved that “termination at gateway, http to speckle-server nginx” didn’t work. We subsequently tried the self signed certificate (3rd option above) on the back end we implemented akv2k8s on AKS to synchronise certificates from the KV which is a nice way to sync certificates. In the ingress-nginx and speckle namespaces you have the following
NAME VAULT VAULT OBJECT SECRET NAME SYNCHED AGE
azurekeyvaultsecret.spv.no/certificate-sync my-kv-dev-uksouth nginx-privatelink-uksouth-azmk8s-io server-tls 7s 6h
and when you describe the above you get the following
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Synced 2m45s (x413 over 3h28m) azurekeyvaultcontroller AzureKeyVaultSecret synced to Kubernetes Secret successfully with change from Azure Key Vault
When we browse via the gateway we still get 501 error and backend health status as follows
The Intermediate certificate is missing from the backend server chain. Please ensure that the certificate chain is complete and correctly ordered on the backend server.
Looking for possible backend issues I noticed this on the frontend-2 pod logs with a failed reason: self signed certificate
Apollo Client error {
operation: {
variables: {},
extensions: {},
operationName: 'ActiveUserMainMetadata',
query: { kind: 'Document', definitions: [Array] }
},
networkError: FetchError: request to https://nginxprivateurl/graphql failed, reason: self signed certificate
at ClientRequest.<anonymous> (file:///speckle-server/server/node_modules/node-fetch-native/dist/shared/node-fetch-native.2b047dc1.mjs:6371:11)
at ClientRequest.emit (node:events:525:35)
at TLSSocket.socketErrorListener (node:_http_client:494:9)
at TLSSocket.emit (node:events:513:28)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
type: 'system',
errno: 'DEPTH_ZERO_SELF_SIGNED_CERT',
code: 'DEPTH_ZERO_SELF_SIGNED_CERT',
erroredSysCall: undefined
},
graphQLErrors: undefined,
forward: [Function (anonymous)]
}
I’m wondering if the above has proved Speckle does not work with self signed certificates? Please let me know if this is true? If so the only other option available to us is the 1st option SSL Pass Thru (Digicert cert for the public and private urls)
This sounds like a positive step forward, and glad you have a clearer picture as to what will work.
From this error message it seems we also need to add the CA Certificate to Frontend 2 so that the requests it makes can validate the self-signed certificate that is being run.
We will have to amend Frontend 2 pod to support the NODE_EXTRA_CA_CERTS environment variable, and allow the certificate to be mounted into it.
In the meantime, Node will trust a Digicert signed certificate so that may also be an option for you.
Hi @iainsproat,
We have implemented the 3rd scenario https and termination at the gateway then issuing another https request to the ingress-nginx load balancer. We also created a certificate with sub domains hence the public facing is the parent and the child domains sits in the private network. From a certificate stand point we have got this resolved and have experimented with other backend test applications to prove this.
When on a debug probe I can run a curl on the private dns direct to the ingress-nginx load balancer using https and the page loads successfully however when we try to run this from the frontend we get the initial registration screen but on registration we get failed to fetch error. Please see below. The error is it cannot resolve the private DNS which was expected. The helm values file holds the private DNS and if we change this to the public domain then it would appear to have a problem with the certificate, the error
The Common Name of the leaf certificate presented by the backend server does not match the Probe or Backend Setting hostname of the application gateway. Stuck atm it would appear fronting it with a gateway is not supported?
1 Like