Function Inputs are encrypted now! What's the deal

Making Speckle Insiders - Automate
1

Hey folks, we noticed overnight that function inputs are encrypted now! Makes total sense if you want to be able to store things like API keys in there. But how can we interact with that programmatically?

1 Like
2

Let’s look at that together on our call…

3

So as best as I can tell there is now a /api/v1/automations/:automationId/keys endpoint that will return an array of string public keys for the function. And now when we POST to /api/v1/automation/:automationId/versions the functions array object needs to include encryptionPublicKey which is just one of the items returned from the keys endpoint and some algorithm is used to encrypt the functionInputs with the public key?

4

So for context for the dev team, your are deploying Automations via the API and not the UI.

The aspect that has broken for you is the submission of inputs as part of that? Or are you seeking to review inputs external from the function runtime?

5

We have some code written that automatically provisions Automations via the API, and it’s the submission of inputs bit that no longer works

2 Likes
6

Hi @sanchez & @chris.welch

Apologies for making that breaking change without first communicating it! I was still working on the basis that Automate’s UI was the only client. That is some really impressive work getting your own client up and running in the few days that you have had access to Automate! We’ll endeavour to communicate breaking changes to the API in advance from now on.

As we’re still in ‘beta’ with Automate there will be breaking changes in the future, and we won’t be versioning the API or providing backwards compatibility for legacy endpoints just yet. We will communicate the changes in advance though.

Regarding this particular breaking change, we have introduced asymmetric key encryption of user inputs to an Automation. This adds extra security when dealing with API keys or other user secrets provided to Automate. User inputs are encrypted in the client and remain encrypted at rest in the database. They are decrypted only in the server, and only at the time of being requested by the user to start an Automation run or when the original user retrieves their input via the API or UI.

For additional security, input to a property which has the writeOnly annotation added to the Function’s json schema will be redacted whenever displayed in the UI or retrieved from the API. This means that writeOnly properties cannot be seen again by the user once entered.

For encryption, we are using a libsodium sealed box, similar to how GitHub encrypt secrets are sent to their API.

As you discovered, the public key to encrypt the secret can be retrieved per Automation from /api/v1/automations/:automationId/keys. The endpoint will return an array of public keys, but it will only ever be an array of one item.

You will then need to stringify and then encrypt the function inputs, which should look a little like this partial javascript:

const keyFetchResponse = await fetch('https://automate.speckle.dev/api/v1/automations/123456/keys')
const keyFetchBody = await keyFetchResponse.json()
const publicKey = keyFetchBody[0]

await sodium.ready
const binaryKey = sodium.from_base64(publicKey, sodium.base64_variants.ORIGINAL)

const data = JSON.stringify(functionInputs)
const binaryData = sodium.from_string(data)

const encBytes = sodium.crypto_box_seal(binaryData, binaryKey)
const encryptedBase64Data = sodium.to_base64(
      encBytes,
      sodium.base64_variants.ORIGINAL
    )
automationVersionData = {
...,
functions: [
      {
        functionId,
        versionId: functionVersionId,
        functionInputs: encryptedBase64Data,
        encryptionPublicKey: publicKey
      }
    ]
}

LibSodium clients exist for many languages and frameworks.

Sorry once again for making the breaking change without warning, and we’ll aim to improve communication in the future.

Iain

3 Likes
7

Thanks for the details and code snippet @iainsproat, I have been able to implement those changes and it appears we have restored functionality now!

Cheers,
Daniel

2 Likes
8

A thought I had for a future version. What if on build of the docker container, the docker container creates the public and private key and pushes the public key to Speckle Automate. Leaving the private key within the docker container per version of the image, then on container execute the function inputs are decrypted within the container?

2 Likes
9

Reasons we consider this not to be a great idea.

  • The private key would be then retrievable from the registry, which becomes another entity to protect
  • The key couldn’t ever be rotated in the event of a breach
  • Automate wouldn’t be able to display results without first retrieving the, possibly very large, image
  • It places a greater burden on the function author

But because we are always open-minded - What do you think the advantages of the suggestion would be @sanchez ?

3 Likes
10

It was just an interesting alternative I thought of.

Thanks so much for explaining the reasons, I appreciate the time and details!