I came across this paper and I’m quite interested in trying something out (not exactly sure yet) with the idea of procedural generation + LLM. We could potentially develop something for the hackathon or just discuss it and further develop this idea.
If you are still building this, I would love to join your team for the hackathon. I have built something similar in the past regarding to creating 3D models of objects as what you described.