On September 30, 2026, AWS announced that Aurora PostgreSQL can query Apache Iceberg and Parquet data directly in your data lake — no reverse-ETL pipeline, no duplicated copies, no second endpoint for applications to learn. The engine doing the analytical scan is DuckDB, embedded inside Aurora PostgreSQL itself. If you have ever kept a nightly job alive just to copy object-store files into a table so an app could read history, this announcement was aimed at you.

The same pattern has been available to self-hosted Postgres for a while through pg_duckdb, now at v1.1.1. Below: what actually shipped, the SQL that works today, and the failure modes to plan for before you point it at production.

What AWS actually shipped in Aurora

Per the AWS launch notice, direct querying of Iceberg and Parquet is generally available on Aurora PostgreSQL starting with versions 17.11 and 18.6 in all commercial and GovCloud regions, at no additional charge — you pay for the incremental Aurora compute your scans consume plus S3 request costs. DuckLabs, the team behind DuckDB, recently joined AWS, and this is the first visible product of that.

The mechanics are deliberately boring, which is the right call. You attach an IAM role with the AuroraAnalytics feature, flip aurora_analytics.enabled to true in the cluster parameter group (it is a dynamic parameter, so no reboot), create the extension, then create foreign tables that point at Parquet files or Iceberg tables in S3, S3 Tables, or the Glue Data Catalog. Data stays where it is; Aurora reads it on demand. Iceberg tables in external Iceberg REST Catalog-compatible catalogs are reachable through Glue Data Catalog federation.

-- 1. Cluster parameter group (applies immediately, no reboot)
--    aurora_analytics.enabled = true

-- 2. In each database that needs data lake access
CREATE EXTENSION aurora_analytics;

-- 3. A foreign table with automatic schema inference from file metadata
CREATE FOREIGN TABLE transaction_history ()
SERVER aurora_analytics_server
OPTIONS (
    location 's3://analytics-lake/finance/transactions/',
    format   'parquet'
);

-- Or point at an Iceberg table registered in Glue (format auto-detected)
CREATE FOREIGN TABLE ft_orders ()
SERVER aurora_analytics_server
OPTIONS (location 'arn:aws:glue:us-east-1:123456789012:table/analytics/orders');

Note the empty parentheses. Aurora infers the schema from the Parquet metadata or the catalog, so you are not hand-maintaining a column list that drifts from the data team’s output. With IMPORT FOREIGN SCHEMA you can bulk-create foreign tables for an entire Glue database in one statement. Iceberg adds time travel through the snapshot and timestamp options on the foreign table definition — timestamp is resolved at table-creation time, so a “yesterday” view is one DDL statement, not a pipeline.

The query side is where this gets interesting. DuckDB handles the columnar scan of the S3 files while Aurora handles operational rows, including uncommitted writes on the writer instance. Predicate pushdown and column pruning mean only relevant columns and row groups are read, and frequently touched data is cached.

-- One statement: live orders in Aurora + five years of history in S3
SELECT a.customer_id, a.amount, a.txn_date, 'recent' AS source
FROM   recent_transactions a
WHERE  a.customer_id = 'C-1001'
UNION ALL
SELECT h.customer_id, h.amount, h.txn_date, 'historical' AS source
FROM   transaction_history h
WHERE  h.customer_id = 'C-1001'
ORDER  BY txn_date DESC
LIMIT  15;

-- Need millisecond latency for a hot subset? Materialize it (writer only)
CREATE TABLE hot_customers AS
SELECT * FROM ft_orders WHERE order_date >= CURRENT_DATE - INTERVAL '90 days';

-- Inspect cost per query: rows scanned, bytes read from S3, cache hits
SELECT * FROM aurora_analytics_stat_statements();

Two operational details matter. Read queries can run on any instance in the cluster, writer or reader, so you can push analytical scans off your operational workload; the materialization commands write, so they run on the writer. And aurora_analytics_stat_statements() is the function to watch — bytes read from S3 is the number that turns “we deleted an ETL pipeline” into a budget conversation.

The self-hosted equivalent: pg_duckdb

If you run Postgres on your own metal or in a container — common in Houston energy and industrial shops where telemetry cannot leave the VPC — pg_duckdb brings the same architecture. It is built by DuckDB Labs with Hydra and MotherDuck, and the current release is v1.1.1. Version 1.1.0 moved the extension to DuckDB 1.4.2 and made PostgreSQL 18 the default in the published Docker images.

CREATE EXTENSION pg_duckdb;

-- Credentials live in DuckDB secrets, not in your queries
SELECT duckdb.create_simple_secret(
  type   := 'S3',
  key_id := 'AKIA...',
  secret := '...',
  region := 'us-east-1'
);

-- Aggregate a year of sensor Parquet without loading it
SELECT r['asset_id'],
       date_trunc('hour', r['ts']) AS hour,
       avg(r['vibration_in'])      AS avg_vibration
FROM   read_parquet('s3://houston-ops/scada/turbine-*/2026-*.parquet') r
GROUP  BY 1, 2
ORDER  BY hour DESC;

The ergonomics differ from Aurora’s foreign tables in one important way: with read_parquet(), read_csv(), and friends you must access columns with the r['column_name'] syntax — the alias is required, and it trips people up in their first ten minutes. Standard SQL works unchanged on your own tables; set duckdb.force_execution = true when you want DuckDB’s engine to run an analytical query over regular Postgres tables instead of the planner.

-- Iceberg and Delta are DuckDB extensions, installed from SQL
SELECT duckdb.install_extension('iceberg');
SELECT * FROM iceberg_scan('s3://warehouse/well_production', version := '2026-09-01-snapshot');

-- Joining live operational tables with a data lake scan in one plan
SELECT w.api_number, w.status, p.oil_bbl
FROM   wells w
JOIN   read_parquet('s3://houston-ops/production/*.parquet') p
       ON p['api_number'] = w.api_number
WHERE  w.county = 'Harris';

-- PgBouncer transaction pooling: force_execution must be tracked
-- pg_duckdb 1.1.0 marks it GUC_REPORT for exactly this reason
-- track_extra_parameters = duckdb.force_execution

Five gotchas before you ship this

1. Columnar engines are not OLTP engines. DuckDB is excellent at scans, aggregations, and joins over large files. It is not a replacement for an index lookup on your hot path. Keep data lake queries on analytical endpoints and read replicas; do not sprinkle them into request-path code just because it compiled.

2. Pooler settings get lost silently. With PgBouncer in transaction pooling mode, session-level settings need to be tracked. pg_duckdb 1.1.0 explicitly marks duckdb.force_execution as GUC_REPORT so it syncs across pooled connections — if you skipped track_extra_parameters entirely, expect inconsistent behavior that looks like a flaky query rather than a config bug.

3. Privileges and row-level security. Reading Postgres tables that have RLS enabled only landed in 1.1.0, and the same release fixed a privilege-check gap where a user could read restricted tables through a view. If you run anything older, pin and upgrade — this is an access-control issue, not a cosmetic one.

4. Small behavioral differences compound. The 1.1.0 release also changed NULL ordering to match Postgres semantics for ORDER BY col DESC (NULLS FIRST). If you validated a report against an older version, re-run the comparison before you declare the migration clean.

5. Watch egress, secrets, and scope. In Aurora, prefer the IAM role over long-lived keys and grant read access narrowly per prefix — s3:GetObject for a single file, adding s3:ListBucket only when you point at a directory or glob. Multi-Region Access Points are not supported as foreign table locations, which matters if your DR design leans on them. Private subnets need S3 gateway and Glue interface VPC endpoints.

Houston use cases that actually fit

Energy: SCADA and turbine telemetry lands as Parquet in S3 by the day. Instead of copying five years of it into Postgres so a maintenance dashboard can trend it, define one foreign table and join it against your live asset and work-order tables. Port and logistics: drayage and container manifests arrive as Parquet from partners; join them to live shipment rows for ETA and settlement reporting without owning an ingestion service. Healthcare and research in the Texas Medical Center: Iceberg tables in Glue hold encounter history while PHI-bearing operational rows stay in Aurora; self-hosted teams can use pg_duckdb to keep both inside the VPC.

What to do this week

If you are on Aurora PostgreSQL 17.11 or 18.6 or later: attach the role, set the parameter, and point exactly one foreign table at a real production Parquet prefix. Run EXPLAIN ANALYZE, then aurora_analytics_stat_statements(), and note rows scanned and bytes read. Compare that against the runtime and failure rate of the ETL job you were maintaining. If you are self-hosted, do the same with pg_duckdb 1.1.1 on a staging replica, with force_execution enabled only for the queries you have measured. Either way, the decision is the same one you should make about any extension: does it delete a system you currently operate?

Also this week in the Postgres ecosystem: Databricks shipped Lakebase Search, adding BM25 full-text search (lakebase_text, lakebase_bm25 indexes) and approximate vector search (lakebase_vector, lakebase_ann) to Postgres; and Supabase raised $150M while acquiring Turso to push further into agent-facing database infrastructure.