OpenAI’s engineers recently shared how PostgreSQL sits at the core of ChatGPT’s infrastructure, serving 800 million users. You do not need ChatGPT-scale traffic for the lesson to apply. In Houston, the same failure pattern shows up in energy trading dashboards, medical claims pipelines, and logistics telematics fleets: the database slows down under load long before CPU or disk max out. When that happens, I start triage at the connection layer, not the query layer. Here is why, and the fix I recommend first.

Why connection count kills throughput

Every Postgres backend is an OS process, not a thread. Each one reserves memory for its working state, and the executor stack is not free. With the default max_connections = 100, pointing a 300-connection application pool at Postgres does not get 3x the work done — it gets less. Backends spend cycles contending for locks, cache lines, and CPU. Add a connection storm after a deploy or a failover, and you get the classic symptom: FATAL: sorry, too many clients already.

The fix is not to raise max_connections and hand out more RAM. The fix is to multiplex many client sessions over a small number of real database connections.

Fix it with PgBouncer in transaction mode

PgBouncer is the boring, battle-tested answer. Run it in transaction pooling mode so a client holds a backend only for the duration of one transaction:

[databases]
app = host=127.0.0.1 port=5432 dbname=app

[pgbouncer]
listen_addr = 127.0.0.1
listen_port = 6432
auth_type = md5
pool_mode = transaction
max_client_conn = 1000
default_pool_size = 40

Point the application at port 6432 instead of 5432 and restart. One caveat: transaction mode does not play well with session-level features like LISTEN/NOTIFY or unnamed prepared statements. Most application workloads — REST APIs, dashboards, queue workers — are fine with it, and the memory savings are dramatic.

Right-size the pool, don’t guess

A common rule of thumb for a transaction pool is 2-4 connections per CPU core on the database server, plus a little headroom for maintenance tasks. For an 8-core instance, default_pool_size = 24 to 32 is a sane starting point; then watch queue time and wait events in pg_stat_activity. If the application has its own framework pool on top, cap it. 200 app-side connections to PgBouncer is fine; 200 direct connections to Postgres is not.

Confirm the effect in one query:

SELECT count(*) FILTER (WHERE state = 'active') AS active,
       count(*) FILTER (WHERE state = 'idle') AS idle
FROM pg_stat_activity;

Before pooling you will see hundreds of idle sessions eating memory. After pooling, the same workload shows a small, steady set of backends.

Then check vacuum and slow queries

Pooling removes the first bottleneck; it does not remove the other two I see daily: bloat and bad plans. Autovacuum keeps dead tuples from inflating tables and indexes. Check that it is keeping up:

SELECT relname, n_live_tup, n_dead_tup, last_autovacuum
FROM pg_stat_user_tables
ORDER BY n_dead_tup DESC
LIMIT 5;

If dead tuples climb on a hot table, tune it individually instead of globally:

ALTER TABLE events SET (autovacuum_vacuum_scale_factor = 0.05);

Then pull the slowest statements from pg_stat_statements and read the plan:

EXPLAIN (ANALYZE, BUFFERS) SELECT ...;

Look for sequential scans on large tables, high Buffers: shared hit/read counts, and a large gap between estimated and actual rows. A missing index or a rewritten join usually closes that gap. Amazon’s own guidance on improving query performance with EXPLAIN plans follows the same discipline: read the plan before you change the hardware.

The Houston takeaway

Before you buy a bigger instance or migrate to a cluster, fix the connection layer. It is the highest-leverage performance change for most workloads, and it is exactly the lesson in OpenAI’s scale-up story. Right-size a PgBouncer pool, keep autovacuum healthy, and measure with EXPLAIN (ANALYZE, BUFFERS). Your energy dashboards, claims pipelines, and telematics streams will thank you at the next spike.