How to run an enclave

Every send and withdraw transaction on zk.money needs a signature from an Oxide enclave. No operator can move your funds, and no privileged admin role exists in the system. However, an operator like an Oxide enclave or a relayer can go offline, so this page tells you how to run the enclave software yourself if need be.

This is an advanced operation, and you do not need to do this to use zk.money on web or on desktop. It is written for developers (or their agents) and assumes you already know something about AWS, Ethereum, and the Aztec Network. Running your own enclave enables you to exit zk.money in the event that every enclave operator stops running. Running an enclave costs you AWS fees, and registering it costs gas on Ethereum and fees on Aztec.

An enclave must be registered to the portal contract on Ethereum and the token contract on the Aztec Network (steps 5 to 7 below).

What you need

  • An AWS account.
  • An EC2 instance type that supports Nitro Enclaves, launched with the enclave option turned on. The Oxide fleet uses c5.xlarge: 4 vCPUs and 8 GiB of memory, of which the enclave gets 2 vCPUs and 5 GiB.
  • Amazon Linux 2023 on that instance.

How it fits together

   your computer                    your EC2 instance
+------------------+     +-----------------------------------------------+
| zk.money Desktop |---->| tee-proxy  ->  socat  ->  enclave (oxide-tee) |
|                  | HTTP|  :8080          :5001      vsock port 5000     |
+------------------+     +-----------------------------------------------+

The enclave has no network. It answers only on a vsock port. socat connects that port to a TCP port on the instance, and the Oxide proxy puts an HTTP endpoint in front of it. The proxy only moves bytes. Every request is sealed to the enclave's own key, so the proxy cannot read it.

Prepare the instance

nitro-cli is the AWS Nitro Enclaves command-line tool. It reads an enclave image file (EIF) and shows its measurements, and it starts, lists and stops enclaves on the instance. Install it before step 1, together with socat, Node.js 24 and git for the proxy, and jq for reading a measurement:

sudo dnf install -y aws-nitro-enclaves-cli aws-nitro-enclaves-cli-devel socat \
  nodejs24 nodejs24-npm git jq
sudo usermod -aG ne "$USER"          # then log out and in again

# Give the enclave 2 vCPUs and 5 GiB, then start the allocator.
printf -- '---\nmemory_mib: 5120\ncpu_count: 2\n' | sudo tee /etc/nitro_enclaves/allocator.yaml
sudo systemctl enable --now nitro-enclaves-allocator

The ne group lets your user run nitro-cli without sudo. The allocator sets aside the CPUs and memory the enclave gets, so the instance cannot use them.

1. Pick the image

Pick an image from the table in "Enclave images" at the end of this page. For the network the wallet uses now, take the row marked Current. Download oxide-tee.eif and its measurements.json onto the instance, and check both.

IMAGE=<the image link from the table, without the file name>   # …/enclaves/<SHA-256>
curl -fsSO "$IMAGE/oxide-tee.eif"
curl -fsSO "$IMAGE/measurements.json"

sha256sum oxide-tee.eif                  # must equal the SHA-256 in the table
nitro-cli describe-eif --eif-path oxide-tee.eif | jq -r .Measurements.PCR0

The PCR0 that describe-eif shows must equal the PCR0 in the table and in measurements.json. The PCR0 is the measurement of the enclave software. The portal registers only enclaves whose PCR0 it already approves, so you do not have to trust this page or the download: a changed image has a different PCR0 and cannot register.

2. Start the enclave

nitro-cli run-enclave --eif-path oxide-tee.eif --cpu-count 2 --memory 5120 --enclave-cid 16
nitro-cli describe-enclaves          # State must be RUNNING, Flags must be NONE

The enclave makes its keys when it starts and keeps them only in its memory. If it stops, its keys are gone, and a new start is a new enclave that must be registered again.

3. Connect the enclave to a TCP port

socat TCP-LISTEN:5001,reuseaddr,fork,bind=127.0.0.1 VSOCK-CONNECT:16:5000

Keep it running, for example in its own terminal or as a service.

4. Start the proxy

The proxy is a small Node.js program with no runtime dependencies. Its source is in the zk.money public repository at vendor/oxide/yarn-project/tee-proxy.

git clone --recurse-submodules https://github.com/aztec-labs-eng/zkmoney-public.git
cd zkmoney-public/vendor/oxide/yarn-project/tee-proxy
npm install
npx tsc -p tsconfig.json
node dest/proxy.js                   # listens on :8080, forwards to 127.0.0.1:5001

OXIDE_PROXY_HTTP_PORT, OXIDE_PROXY_TCP_HOST and OXIDE_PROXY_TCP_PORT change the defaults. Check that it answers:

curl http://127.0.0.1:8080/health    # prints OK

/health tells you only that the proxy runs. The enclave's own endpoint is /rpc.

5. Get the registration data

Registration needs the enclave's attestation: a document from AWS Nitro, signed by the AWS root certificate, that holds the enclave's PCR0 and a hash of its two public keys. The enclave answers one request without encryption, getAttestation. Every request to /rpc is one frame: a 4-byte big-endian length, then the JSON body.

cat > get-attestation.mjs <<'JS'
const body = Buffer.from(JSON.stringify({ method: 'getAttestation' }))
const frame = Buffer.concat([Buffer.alloc(4), body])
frame.writeUInt32BE(body.length, 0)
const res = await fetch('http://127.0.0.1:8080/rpc', {
  method: 'POST',
  headers: { 'content-type': 'application/octet-stream' },
  body: frame,
})
process.stdout.write(Buffer.from(await res.arrayBuffer()).subarray(4))
JS
node get-attestation.mjs > attestation.json

The reply is { "ok": true, "result": { "attestation": …, "userData": { … } } }:

  • attestation is the Nitro attestation document (COSE_Sign1), encoded as base64. It carries the certificate chain (cabundle and certificate), the signed payload and its signature. Decode it with Buffer.from(result.attestation, 'base64').
  • userData holds the four key coordinates you register: publicKeyX and publicKeyY (the secp256k1 key the enclave signs with) and encPubKeyX and encPubKeyY (the P-256 key requests are sealed to). The enclave's Ethereum address is derived from the secp256k1 key.

The enclave takes its attestation once, when it starts, and the portal refuses an attestation that is more than one hour old. Send the registerTee transaction of step 6 within one hour of starting the enclave. If you miss it, restart the enclave and begin again.

6. Register on Ethereum

Send these transactions to the portal address of the deployment, from the table below or from the deployment's entry in Oxide's manifest. Anyone can send them, and you pay the gas. Verifying the attestation is too large for one transaction, so the portal verifies it in stages and caches each result:

  1. verifyTeeCACert(cert, parentCertHash) for each certificate in cabundle after the first one, which is the AWS root, in order. parentCertHash is the keccak256 of the certificate before it.
  2. verifyTeeClientCert(cert, parentCertHash) for the enclave's own certificate, with the keccak256 of the last cabundle certificate. Its own keccak256 is the leafCertHash of step 4.
  3. verifyTeeAttestationHash(attestationTbs), the signed payload of the COSE_Sign1 document.
  4. verifyTeeAttestationSig(attestationTbsKeccak, signature, leafCertHash).
  5. registerTee(attestationTbs, signature, publicKeyX, publicKeyY, encPubKeyX, encPubKeyY).

Each staging step has a read function (isTeeCertStaged, isTeeAttestationHashStaged, isTeeAttestationSigStaged), so you can skip a step a previous attempt already did. registerTee fails unless isPcr0Approved is true for the image's PCR0, which is true for every image in the table that a deployment uses. On success it emits TEEAdded with a messageKey and a leafIndex for step 7.

7. Approve the signer on Aztec

registerTee also sends a message from Ethereum to Aztec. When that message is ready, call consume_signer_registration(pub_key_x_hi, pub_key_x_lo, pub_key_y_hi, pub_key_y_lo, message_leaf_index) on the deployment's l2Token contract. The key arguments are publicKeyX and publicKeyY, each split into its high and low 128 bits, and the last one is the leafIndex from TEEAdded. Any Aztec account that can pay the fee can send it. Wait until the transaction is in a checkpointed block: a block that is only proposed can still be dropped, together with the transaction.

The helpers that split the attestation into the values above, getCertStagingEntries and decodeAttestationTbs, are in vendor/oxide/yarn-project/oxide-lib/src/attestation.

8. Point zk.money Desktop at it

Open the endpoint settings page of zk.money Desktop, and set Enclave URL to the /rpc address of your proxy. The wallet runs on an HTTPS page, so the address must be HTTPS or a local one. The simplest way is an SSH tunnel from your computer, which keeps the proxy off the internet:

ssh -N -L 8080:127.0.0.1:8080 ec2-user@<your instance>

Then use http://localhost:8080/rpc. The wallet checks the enclave's attestation before it seals anything to it, so a wrong address fails closed.

Enclave images

One row for each deployment on this network, with the published image its portal approves. Current marks the deployment the wallet uses now.