Our project is moving along and we are kind of look at speckle more deeply now. We haven’t decided on anything yet but I’ll post here first in case there is something we missed, or something that takes time to answer, and is quite exploratory.
In unreal, there isnt a “Send” to speckle option and looking at it, there is only ServerTransport which is a receive. So at this point some questions pops up:
Is ServerTransport 2-way ?
If so, then there is no send function, how would we include this without breaking future updates from speckle?
if not, how would you name the other class so we avoid a clash in an update from speckle?
The crux of what we want to do is to add a send function, but the main gist of this post is to ask about how to structure the setup to do so. A couple of ideas came to me, for example:
inherit the ServerTransport class and add Send functionality
create a fork of speckle-unreal and add required files there and merge in changes
So right now I am trying to find out how to even structure the project(s) to even have updates to speckle integrated when they come in, while maintaining features that we want to implement inside speckle as speckle intends for us to easily extend the functionality.
I realised I also forgot to add this:
What is your development environment like? Why I ask this question is that if I open this in MSVC as a folder, I do not have the benefit of intellisense (especially useful as I need the references as well to all the unreal and speckle classes). However, if I open it as part of an Unreal project (after its already compiled) I can make use of the intellisense to guide me around and I can do on the fly changes to help me experiment/develop with it. This however has a drawback that it is already a compiled version and not the git version in which i can directly commit a change for example.
Your exploration into extending the Speckle connector for Unreal Engine with send functionality is exciting. Given the specific needs of your project and the broader potential benefits for the Speckle community, one pathway to consider is contributing this send functionality back to Speckle.
Currently, ServerTransport is designed only for receiving data in the Unreal connector. To implement sending capabilities, you’ll need to extend or modify the existing framework, as ServerTransport does not inherently support sending data to Speckle.
While developing for your specific use case is a great start, contributing to Speckle could offer several advantages:
Future Compatibility: Integrating your send functionality into the Speckle ecosystem, you help ensure it remains compatible with future updates and changes. This proactive approach can mitigate potential integration challenges down the line.
Community Benefit: Your contribution could fill a gap in the current Speckle functionality for Unreal Engine, providing a valuable tool for other developers facing similar challenges.
Decoupled Architecture: We recommend clearly separating the transport mechanism (ServerTransport) and Speckle object data handling. Decoupling the ServerTransport mechanism from Speckle object data handling adheres to best practices in software architecture. It significantly benefits the review and integration process from our standpoint as the repository owners. Following this architectural principle, your contribution would be a strong addition to the Speckle ecosystem.
We understand that contributing to an open-source project involves additional steps, including ensuring the generic applicability of your implementation and aligning with Speckle’s development guidelines. However, the Speckle team is here to support you through this process, providing guidance and feedback to facilitate your contribution.
If you’re open to this path, we’re excited to work with you to integrate your send functionality into Speckle. This could significantly contribute to the Speckle community, enhancing the Unreal Engine connector for all users.
If contributing directly to Speckle might not be immediately feasible for you, a streamlined approach to maintain flexibility and compatibility with the Speckle Unreal connector:
Fork the Speckle Repository: Create your own version of the Speckle Unreal connector to develop independently.
Create a Separate Branch for Custom Developments: Keep your custom work on a distinct branch to simplify tracking and management.
Use Subclassing and Extensions: Extend existing classes for your new features to maintain compatibility with the base code.
Regularly Sync with the Original Repository: Periodically update your fork with changes from the main Speckle repository to stay current.
Document Your Changes: Clearly document any customizations or additions you make for future reference.
By adhering to these steps, you can ensure that your extension or fork remains a clean, flexible, and up-to-date enhancement of the Speckle Unreal connector.
This approach will enable flexible development alongside ongoing updates to the Speckle Unreal connector. It also offers a path in the future that could be a community contribution.
Decoupled Architecture: We recommend clearly separating the transport mechanism (ServerTransport) and Speckle object data handling. Decoupling the ServerTransport mechanism from Speckle object data handling adheres to best practices in software architecture. It significantly benefits the review and integration process from our standpoint as the repository owners. Following this architectural principle, your contribution would be a strong addition to the Speckle ecosystem.
I think I understand the intent and the general thrust of this, however I am not sure I am able to formulate the architecture … as I am not familiar with plenty of things and would need a lot of help. Which is why I am asking also about the development setup that you might be using as detailed above about not having intellisense for the speckle project if opened as a folder in MSVC. I would need some help to even get started.
Currently, ServerTransport is designed only for receiving data in the Unreal connector. To implement sending capabilities, you’ll need to extend or modify the existing framework, as ServerTransport does not inherently support sending data to Speckle.
I have looked through this and yes it doesn’t and I am not sure about the approach to consider. Currently ServerTransport implements ITransport which doesn’t seem to support sending. Changing this would mean changing all transports, and the name implies that it should have. Or should there be a… ClientTransport that implements an IClientTransport ?
We understand that contributing to an open-source project involves additional steps, including ensuring the generic applicability of your implementation and aligning with Speckle’s development guidelines. However, the Speckle team is here to support you through this process, providing guidance and feedback to facilitate your contribution.
Currently, we are not opposed to contributing to the project and I think we can start first until there is a different directive from management.
We understand that contributing to an open-source project involves additional steps, including ensuring the generic applicability of your implementation and aligning with Speckle’s development guidelines. However, the Speckle team is here to support you through this process, providing guidance and feedback to facilitate your contribution.
You will have to do a lot of guiding I assume I’ve looked at the “how to contribute” How to Contribute to Speckle link and don’t seem to be able to find a lot of information apart from posting in the forum, so here I am!
The Unreal connector is certainly one of the more community-supported and less mature plugins we provide.
The team working on it here in Speckle uses Rider as the IDE of choice, but VS should be equally valid.
This raises the question of how much development for the Unreal environment you have experience with. We could offer some guidance, but complete tutorials of C++ dev for Unreal are beyond our capabilities purely down to time and how wide the team’s knowledge extends.
I fully understand your reluctance to do more than what you need now - contributing to Speckle was simply a failsafe to avoid rework down the line. We have heard so few people asking for Send from Unreal that it hasn’t been a priority.
This raises the question of how much development for the Unreal environment you have experience with. We could offer some guidance, but complete tutorials of C++ dev for Unreal are beyond our capabilities purely down to time and how wide the team’s knowledge extends.
Yes, of course, I’m not asking for c++/Unreal help. I will have to find that out myself. I’m asking how to architecture the classes so that it fits in with the design, thus my questions about the ServerTransport above. e.g Do you guys want a separate interface for send because ITransport as an interface doesn’t support it? or do you want to add functionality to the ITransport interface but this results in changes to all other classes that implement it? This are some starting fundamental questions before I can even get started… and this is what I mean.
Again, I am not asking for C++ help, I’m specifically asking for Speckle design help, which there is no other place to get.
We would be looking for something that aligns with our other SDK’s ITransport interfaces (Py, C#). A SaveObject function that adds to an internal queue. And then some sort of WriteComplete or end_write function to complete the actual web requests (ideally threaded).
The Python implementation is likely the simplest to follow, and I’d recommend starting there trying to replicate as much of that in C++ as possible. How closely we stick to the Python implementation is flexible, likely we’d need to deviate a fair bit for Unreal’s/C++ sake. Getting something working here is key.
If we think the current options will be too complex, then we could consider a purely synchronous way of sending. But both performance and thread blocking would be a bit of a concern here.
Once the server transport work is done. There’s still more work to do before we can send Unreal geometry. But this can be a separate discussion if you’d like.
I’ve looked at the “how to contribute”