Persistent State Made My Agent Loop 40% Faster

Before every HTTP call, my coding loop sent the entire conversation back to the model: the system prompt, tool schemas, and every previous turn. The payload grew on every iteration.

Moving that request from HTTP to a WebSocket does not solve the problem by itself. A WebSocket keeps a connection open. It does not decide which context either side needs to send.

The actual optimisation

The context only gets smaller when the client and server agree to retain the previous state and exchange only what changed.

The first turn still sends the full context. Later turns send a delta and a stable reference to the prior state, such as previous_response_id. The server can recover the earlier conversation instead of asking the client to serialise it again.

That is the distinction. Connection persistence and context persistence are separate decisions. OpenAI adding a WebSocket mode does not reduce context transfer unless the protocol also supports sending only the change.

What changed in my loop

I changed the conversation protocol so the server retained the prior response and the next request carried only the new turn plus its reference. That reduced serialisation, network traffic, and context reconstruction.

The change cut end-to-end loop time by about 40% in my coding loop. That is a measurement of the incremental state protocol, not a claim that WebSockets alone produce the same result.

The trade-off

State must live somewhere. The system needs an expiry policy, a way to recover after a restart, and a fallback when the state reference is unavailable. Stateless requests remain useful when replay and independent recovery matter more than per-turn latency.

Use a WebSocket when streaming or a long-lived connection helps. Use incremental state when repeated calls share most of their context. The second choice is what removes the bytes.