Architecture Advice: Best Practices for Hosting, Scaling, and Maintaining Speckle Server (Large IFCs, 500+ Users)

General discussion
,
1

Hi Speckle Community!

I’m looking for some advice and best practices on how to properly host, maintain, update, and back up a self-hosted Speckle Server. We primarily use it for uploading and downloading 3D models.

Current Setup & The Problem: Right now, I have a Speckle Server running (4 Cores x86, 16 GB RAM), but everything is deployed on a single machine. It works for basic tasks, but it struggles heavily under load. Whenever I upload larger files, the server hits up to 400% CPU usage.

My Goals: I want to build a more robust and scalable infrastructure that allows me to operate the server “easily” without constant babysitting. Specifically, my requirements are:

  • Large File Support: I need to be able to reliably upload IFC files up to 500MB in size without crashing the server or maxing out the CPU for too long.
  • Future Scaling: I want the architecture to be capable of scaling up to 500 users in the future.
  • Operations: I need a solid strategy for seamless updates and reliable backups.

Questions for the Community:

  1. Infrastructure: What does your architecture look like for a similar scale? Do you recommend sticking to Docker Compose with beefier specs, or is migrating to Kubernetes the best path for this amount of users?
  2. Resource Management: How do you handle high CPU spikes during heavy IFC uploads and processing? Are there specific worker configurations or separate nodes you use for processing?
  3. Storage & Databases: Do you recommend separating PostgreSQL, Redis, and Object Storage (like AWS S3 or dedicated MinIO) onto different servers?
  4. Maintenance: What are your best practices or automated workflows for painless version updates and disaster recovery (backups)?

Any pointers, architectural tips, or shared experiences would be hugely appreciated. Thanks in advance for your help!

2

Hey, good questions, and congrats on hitting the “400% CPU” milestone; that’s usually the moment self-hosting stops being a hobby and starts being a job :slightly_smiling_face:

Some honest context first: this takes quite a bit of tinkering. On our end (app.speckle.systems) we run Kubernetes with autoscaling worker queues, file imports, previews, etc. all run as separate workloads that scale out on demand and scale back to zero-ish when idle. Spiky loads (someone dropping a 500MB IFC on a Friday afternoon) are absorbed by the queue rather than by your API server’s cpu

On your specific questions:

Infra: If you go down this path, set it up with Terraform (or similar) from day one; use managed services wherever you can: managed pg (with point-in-time recovery, which solves most of your backup question for free), managed Redis, and S3-compatible object storage with versioning/replication.

Resource management: Separate the file-import workers from the main server. Docker Compose can technically do this on a second machine

Users: With 500 users — especially across departments or companies - workspaces will save you a lot of grief on permissions, project organization, and knowing who’s who. Strongly recommend structuring around them early rather than untangling it later.

And you know, if this not fun, we offer managed enterprise services, on prem too, and you get quite a bit more mileage out of speckle (eg, model validation and bunch of other goodies)

1 Like