Postgres Transparent Data Encryption: pg_vault_tde 1.7.2
Encryption at rest is usually the cloud provider’s checkbox: an encrypted EBS volume, encrypted managed storage, a disk you stop thinking about. Then an auditor asks who can read the base backup sitting in object storage, or a snapshot leaves the account, or an operator with a shell on the host runs strings against a relation file. Volume-level encryption answers none of those questions. Database-level transparent data encryption (TDE) does, because the bytes are ciphertext before they ever reach the storage manager — the heap file, the WAL record and the base backup are all unreadable without the key.
The open-source option most teams evaluate is pg_vault_tde, which published version 1.7.2 on 7 October with a set of critical crash fixes and a new on-disk tuple layout. Read that release note as an operations document, not a changelog: it changes the on-disk format and the WAL resource manager ID, and neither change tolerates the usual upgrade habits. Here is how to move to it without an unplanned outage.
What it encrypts — and what stays in the clear
pg_vault_tde is a pure extension, with no core patches. It registers a table access method, encrypted_heap, and an index access method, tde_btree, through the normal extension API. A table created with USING encrypted_heap has every user column of every tuple encrypted with AES-256-GCM before heap_insert() and decrypted on the way out at the scan layer. Because the work happens inside the table access method, it stays transparent to pg_dump, logical replication and the executor.
Read the threat model before you promise a security team anything:
- Encrypted: tuple user data, TOAST values, WAL payloads, relation files, and — through
pg_basebackup_tde— standby base backups. - Not encrypted: the
HeapTupleHeader(xmin, xmax, ctid, infomask — MVCC needs it plaintext), WAL structural metadata such as LSNs and block numbers, andpg_statisticrows, so most-common-values and histograms still leak value distributions. Network traffic is not covered either: keepssl = on.
Index keys are the subtle part. B-tree keys use AES-256-SIV, deterministic encryption so equality lookups keep working. Other access methods — GIN, GiST, Hash, BRIN — are rejected on an encrypted table by default, unless you set pg_vault_tde.allow_plaintext_index = on and accept an index sitting on disk in the clear. Design encrypted schemas around equality access, and treat index-only scans as unavailable.
The configuration that matters
The extension is a shared preload plus a key provider. kms_provider has no default, so choose deliberately between HashiCorp Vault/OpenBao, a local PKCS#12 wallet, or a PKCS#11 HSM (added in 1.7). It requires PostgreSQL 17 or 18 (19 is planned) plus OpenSSL 3 and libcurl.
# postgresql.conf — Vault / OpenBao provider
shared_preload_libraries = 'pg_vault_tde'
pg_vault_tde.kms_provider = 'vault'
pg_vault_tde.vault_url = 'https://vault.internal:8200'
pg_vault_tde.vault_transit_mount = 'transit'
pg_vault_tde.vault_key_name = 'pg-tde-dek'
pg_vault_tde.vault_ca_cert = '/etc/ssl/vault/ca.pem'
pg_vault_tde.enabled = on # PGC_POSTMASTER: full restart to change
With an HSM the wrapping key never leaves the token, and only the environment variable name appears in configuration — never the PIN itself:
pg_vault_tde.kms_provider = 'pkcs11'
pg_vault_tde.pkcs11_library = '/usr/lib/softhsm/libsofthsm2.so'
pg_vault_tde.pkcs11_token_label = 'pgtde' # preferred over pkcs11_slot_id
# pg_vault_tde.pkcs11_pin_env = 'PG_TDE_PKCS11_PIN' # env var NAME, set before PG starts
pg_vault_tde.enabled = on
Per-relation data-encryption keys are cached in shared memory, so the hot path is a memcpy and one crypto call per tuple, not a round-trip to the KMS. All KMS-provider GUCs are SUSET, which means a single cluster can give tenant_a its own transit key and tenant_b a local wallet through ALTER DATABASE ... SET.
1.7.2 is an upgrade event, not a patch
Three changes in this release dictate how you deploy it. First, the custom WAL resource manager ID moved from 128 to the registered ID 161, so the upgrade requires a clean shutdown. Second, if you enabled pg_vault_tde.toast_custom_rmgr, the primary and every standby must be upgraded together — a rolling upgrade is explicitly not possible. Third, the new v5 on-disk tuple layout, which fixes a segmentation fault on UPDATE of tables with indexed variable-length columns, requires a VACUUM FULL on every encrypted table afterwards.
That combination retires the comfortable plan of “patch the replica, promote, then patch the old primary”. Budget a maintenance window and rehearse it on a restored copy first. The rest of the release is a list of reasons to move quickly: a TOAST segfault when a value crossed the threshold only after being encrypted, a bug that left a table unreadable after an all-NULL row, and three key-management fixes covering concurrent rotate_online() access, local-wallet KEK rotation and vault-to-wallet migrations — each of which could lose or corrupt data under the wrong interleaving.
One behavioural change earns its own line. tde_btree now answers equality queries only, closing a defect where range operators, ORDER BY and merge joins could return incorrect rows. If you leaned on range scans through an encrypted index, those plans have to change.
A safe 1.7.2 upgrade runbook
- Record the current build on every node:
SELECT pg_vault_tde_build_version(); - Take a verified physical backup with
pg_basebackup_tdeand a logical one withpg_dump_tde; test-restore both. - Announce a window. Stop writes, shut the primary down cleanly, and stop every standby — no node stays running.
- Install the 1.7.2 binaries on all nodes, start the primary, then run
VACUUM FULLon every encrypted table. - Restart the standbys, let replication catch up, confirm replay is current, then re-open writes.
Rotation and day-two operations
Key rotation is where TDE stops being a configuration exercise. Rotating a table’s data-encryption key re-encrypts every tuple in place under a RowExclusiveLock — online — and can be batched and monitored:
SELECT pg_vault_tde_rotate_online('secrets', 1000);
SELECT * FROM pg_vault_tde_rotation_status('secrets');
Rotating the KEK is cheaper. SELECT pg_vault_tde_rotate_kek(); re-wraps every per-table DEK under a new key-encryption key without touching tuple data, works across the Vault, wallet and PKCS#11 providers, and keeps each prior generation so old data always decrypts. Changing a wallet passphrase rotates the KEK as a side effect — do not call both.
Backups and standbys
Encryption reaches into the backup path, which is often the whole point. pg_dump_tde writes an AES-256-GCM-wrapped archive with the DEK sealed in its header, pg_restore_tde reads it back, and pg_basebackup_tde covers physical backups — including a standby’s base backup, which matters when the replica’s disk is the thing you do not trust. One caveat: the standalone dump tools do not yet handle a PKCS#11 session and PIN, so an HSM-backed cluster needs a provider those tools can reach at restore time.
Monitoring checklist
- Alert when
pg_vault_tde_build_version()differs across nodes after any maintenance. - Track
pg_vault_tde_rotation_status()for slow rotations instead of assuming they finish. - Watch logs for
allow_plaintext_indexwarnings — each one is an index readable on disk. - Put key-provider availability on the same dashboard as the database: a backend that cannot reach the KMS cannot start.
Why this matters in Houston
Gulf Coast energy, healthcare and financial operators are the natural audience. Upstream and midstream data, protected health information under HIPAA and cardholder data under PCI all demand protection at rest, and per-database KMS scoping lets one cluster isolate subsidiaries or joint ventures. The compliance win is real, but so is the operational cost: TDE turns an outage of your key provider into an outage of your database.
If your high-availability story is not solid yet, fix that first — Patroni and automatic failover and archive_mode and backups are prerequisites, not alternatives. When TDE’s no-rolling-upgrade constraint forces a fresh target cluster instead, logical-replication cutover is the pattern to follow. Then run VACUUM FULL, verify with pg_vault_tde_build_version(), and keep your key provider on the pager.
