Skip to content

Secrets

Introduction

Metaport encrypts sensitive data such as API keys and tokens for use with various integrations. Historically, Metaport CE simply referenced a single AES Data Encryption Key (DEK) stored in a restricted file on the file-system and used that as a global encryption key.

Since Metaport is designed to be multi-tenant, one encryption key for everything is a serious security and maintenance problem:

  • One DEK is responsible for every tenant. Key loss, corruption, or disclosure affects every tenant
  • DEK key rotation is made very complex and requires dedicated migration pathways

Metaport also supports Key Encryption Keys (KEK - also known as "transit keys") and DEK's managed through an adaptor pattern with Hashicorp Vault the only currently available backend and which uses the envelope encryption pattern.

Vault is reached through a pluggable secrets backend adaptor: Metaport\Security\Secrets\MetaportSecretsService delegates to whichever backend is named by the MP_SECRETS_BACKEND environment variable (Vault selects Metaport\Security\Secrets\Backend\Vault). Each backend implements the MetaportSecretsBackendProvider contract so alternative providers such as AWS Secrets Manager can be added without touching the calling code. Vault is the reference implementation; the rest of this document describes it in the context of Vault, but the process should be adaptable to any backend.

Workflow

Whether Vault is invoked at all is decided based on whether the application's MP_VAULT_* environment variables (see Requirements below) are set.

  • Vault configured - the per-tenant Vault-wrapped DEK is the only key used. There is no fall-back to the legacy filesystem master key; if a record's Organisation has no wrapped DEK, the operation fails+logs rather than use the wrong key.
  • Vault not configured - all encryption and decryption uses the single file-system master key (referenced via MP_MASTER_KEY_PATH), exactly as Metaport CE has always worked. This is the default for non-Vault, on-premise and open-source installations.

Provisioning a DEK (once per Organisation)

  • When an Organisation record is first created, Metaport automatically calls Vault's transit/datakey/plaintext/<key> endpoint (where <key> is the transit key named by MP_VAULT_TRANSIT_KEY, e.g. metaport-dev-dek-kek in the example below).
  • Vault generates a fresh DEK and returns it in one response as both a plaintext DEK and a "wrapped" (KEK-encrypted) DEK.
  • Metaport persists only the wrapped DEK to the WrappedDEK field on the Organisation record and immediately discards the plaintext copy. The plaintext DEK is never stored.

Using a DEK (every encryption or decryption operation)

  • To encrypt or decrypt data, Metaport resolves the acting record's Organisation and reads the value of its stored WrappedDEK field.
  • Metaport sends the wrapped DEK to Vault's transit/decrypt/<key> endpoint, which returns the plaintext DEK.
  • The plaintext DEK is used for that symmetric operation and then discarded immediately - it is never persisted.

Set-up

Requirements

  • A standalone container running a stable release of Vault (in development environments you can use docker-compose-vault.yml)
  • A TLS certificate for inter-container communication (self-signed in development environments is A-OK - see below)
  • The application authenticates to Vault itself via AppRole: on constructing its Vault service it logs in with a role_id + secret_id and uses the returned token for that unit of work. The application therefore needs the four environment variables below.
  • The following environment variables available to the Metaport application container:

MP_SECRETS_BACKEND: The name of the secrets backend in-use e.g. Vault. MP_VAULT_HOST: The full hostname and https scheme where your Vault host can be found. MP_VAULT_TRANSIT_KEY: The name of the stored "transit key" in Vault e.g. organisation-encryption. MP_VAULT_ROLE_ID: The AppRole role_id (an identifier, not itself a secret). MP_VAULT_SECRET_ID: The AppRole secret_id. Runtime-injected, never baked into the image (see Provisioning below).

  • The following environment variables available to the Vault container:

VAULT_HOST: The full hostname and https scheme where your vault host can be found. VAULT_TOKEN: A Vault token for running the vault CLI setup commands below (not used by the application). VAULT_TRANSIT_KEY: The name of the stored "transit key" in Vault e.g. organisation-encryption. VAULT_SKIP_VERIFY: Skip verifiying TLS certs (allows self-signed certs).

Note

In development environments: - Generate your own self-signed SSL certificate files into .dev/services/vault/tls - Don't forget to setup an AppRole and create a transit-key in Vault (See Startup below)

Startup

!! note You'll need to run vault operator unseal x3 times, each time passing it one of the unsealing tokens you were presented with after running vault operator init.

#> docker compose -f docker-compose-vault.yml up -d --build #> docker compose -f docker-compose-vault.yml exec vault sh vault> vault operator init vault> vault operator unseal vault> vault login vault> vault secrets enable transit vault> vault write -f transit/keys/metaport-dev-dek-kek vault> vault auth enable approle vault> vault policy write metaport /etc/vault/policy/metaport.hcl vault> vault write auth/approle/role/metaport token_policies="metaport" token_ttl=1h token_max_ttl=4h vault> vault read -field=role_id auth/approle/role/metaport/role-id vault> vault write -f -field=secret_id auth/approle/role/metaport/secret-id

!! note Assuming production Vault servers are configured to auto-restart via e.g. systemd then after a machine reboot, only the manual unseal operation above need be performed unless you've configured auto-unseal on your Vault server.

The last two commands emit the role_id and secret_id. Set them as the application's MP_VAULT_ROLE_ID and MP_VAULT_SECRET_ID environment variables (in development, in .env). The application logs-in with them itself - there is no token to copy anywhere.

Authentication

The Vault backend (Metaport\Security\Secrets\Backend\Vault) authenticates to Vault directly: on construction it POSTs role_id + secret_id to Vault's auth/approle/login endpoint, and uses the returned client token as X-Vault-Token for that unit of work. The backend is constructed per cryptographic operation, so a login happens per operation - acceptable because cryptographic operations are infrequent. No token is cached or persisted.

Provisioning the secret_id (production)

role_id is an identifier, not a secret, and may remain static. secret_id is sensitive - anyone holding it can mint tokens for the metaport AppRole - so it must be runtime-injected, never baked into the image:

  • Store it as a GitLab CI/CD protected, masked variable (private repo).
  • The deploy job injects it into the running container at deploy time (docker run --env MP_VAULT_SECRET_ID='$CI_VAR' ...) - never via --build-arg / ENV (an image layer is immutable, distributed to every pull and backup, and cannot be rotated). The application reads it with getenv() unchanged.

Rotation: no scheduled rotation. The secret_id lives in a hardened, low-leak store and its blast radius is deliberately floored (its tokens can only datakey / decrypt a single transit key - useless without a separate database breach). Rotate on demand as incident response - generate a new secret_id, update the CI variable, redeploy - triggered by team-member offboarding, a suspected compromise, or any change to the CI-issuing credential. That CI-issuing credential (scoped so it can only issue metaport secret_ids) is the high-value root of trust to safeguard, not the low-blast-radius secret_id.

Self Signed Certs

This should be run before building the local container. It will ensure TLS transport within the container through loopback, and from outside the container through the vault.metaport.dev hostname.

openssl req \
  -subj "/C=GB/CN=vault.metaport.dev" \
  -addext "subjectAltName = DNS:vault.metaport.dev,IP:127.0.0.1" \
  -x509 \
  -nodes \
  -days 365 \
  -newkey rsa:2048 \
  -keyout .dev/services/vault/tls/privkey.pem \
  -out .dev/services/vault/tls/fullchain.pem