Application Gateway fronting Speckle on Kubernetes

Help
1

We have created an Azure Application Gateway that fronts Speckle. Speckle is running on Azure Kubernetes (AKS). All of the services in AKS are running in a private subnet inside a vnet. The DNS is pointing to the application gateway with a public ip address. On the gateway we have a backend pool that points to the private ip address of the ingress nginx load balancer. The backend health is unhealthy :slight_smile: with the following error

The Intermediate certificate is missing from the backend server chain. Please ensure that the certificate chain is complete and correctly ordered on the backend server.

We have setup speckle such that the ingress-nginx load balancer is pointing to the same DNS url and I’m wondering this might be the issue and should be something else? We really need to change this to use a private dns with a different address name and terminate the SSL at the gateway. Then how would this work with speckle? The helm values yaml file will have this new private dns, and there will be no SSL certificate but speckle expects it? If any speckler has been involved with such a configuration and would be happy to share there experience it would be greatly appreciated. Thank you

1 Like
2

Hi @shiangoli

Have you tried running Speckle with ssl_canonical_url set to false?

Iain

3

Hi @iainsproat yes we have but we get the following on the application gateway backend health

Received invalid status code: 404 in the backend server’s HTTP response. As per the health probe configuration, 200-399 is the acceptable status code. Either modify probe configuration or resolve backend issues.

On the application gateway the backend pool is pointing to my-private-dns.io
I created a debug pod and run a curl on the private dns name (please see below) and got nginx which suggests the private dns has been resolved. Also the same dns is in the values file and when I look at the speckle namespace ingress its using the same dns

root@aks-nodepool1-12581264-vmss000000:/# curl http://my-private-dns.io
<html>
<head><title>308 Permanent Redirect</title></head>
<body>
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>nginx</center>
</body>
</html>

Not sure what is the issue

4

I suspect the 308 is that nginx is likely trying to redirect the http to https, but we’re unable to say for sure with the given output. If you run the same curl with the verbose flag enabled -v, you should be able to view the headers that were returned. The Location header in the response will tell you where it is trying to redirect you.

This may be why the gateway is receiving a 404, but your curl is receiving 308? (assuming you are curl-ing the same url that is configured for the gateway). Perhaps the gateway is following the redirect location header, but curl isn’t?

Another avenue to explore are the annotations on the nginx cluster configuration. I understand for AWS, for example, it requires some custom annotations to work - the same may be true of Azure or your particular setup on Azure.

5

Hi @iainsproat we managed to get this to work where TLS is up to the gateway. The only issue is we need to manually edit the ingress on the speckle namespace to remove the tls section. Is there a switch in the values yaml helm file or else where to turn this off?

For reference to get to this point

  1. TLS is up to the Azure Application Gateway
  2. In backend settings on the gateway in the host name override we “Pick host name from backend target”
  3. We created a private DNS for AKS ingress-nginx load balancer and configured the speckle helm values file hostname to point to this
  4. Backend pool in the gateway points to a target that is this private address
1 Like
6

Yes, you can set the cert_manager_issuer field to a blank value, e.g. "", and helm will omit the TLS section.

Hope this helps

1 Like