🚦 Limits: concurrency, rate & bandwidth

Rama gives you three distinct kinds of limits, all composable with the same service stack machinery:

1. Concurrency: at most N inputs in flight at once;
2. Rate: at most N inputs per period (“100 requests per second”);
3. Bandwidth: at most N bytes per second on a stream (traffic shaping), or pacing datagrams at a byte/packet budget.

All three are backed by first-class rama primitives — the rate and bandwidth ones by one shared sans-IO token bucket (rama::utils::rate), which takes time as an argument instead of reading a clock: no_std, deterministic to test, and embeddable in your own sans-IO state machines.

Concurrency & rate: the limit layer

LimitLayer limits any service by a Policy. Rama ships two: ConcurrentPolicy bounds the number of in-flight inputs (optionally with a backoff), and RatePolicy bounds inputs per period. A rate policy runs in one of two modes: an abort mode, where over-budget inputs fail with RateLimitReached (it carries a retry_after, and .into_response() turns it into a 429 Too Many Requests with a Retry-After header), and a wait mode, where over-budget inputs are paced instead of rejected.

Policies compose like any other rama building block: wrap one in Option to toggle it, combine variants with the Either* combinators, and scope them per route or per source with matcher policy maps. Limit layers also stack. Place an aborting rate limit outside a concurrency limit to reject over-budget inputs before they take a concurrency slot. Put a waiting rate limit outside for the same reason: pacing then happens before admission, rather than while the input occupies a concurrency slot.

Per-client fairness

KeyedRatePolicy (rama::net::rate) is the per-key sibling of RatePolicy: every key gets its own lazily-created bucket, stored in a bounded, idle-evicting cache (size the key bound above your expected client count, so it stays a memory backstop rather than an availability limit). At capacity, a new key fails closed instead of evicting a live bucket and resetting its budget. Key on the client IP with ClientIpRateKey — it honours Forwarded information over the socket peer address and aggregates IPv6 clients to a /64 by default, so a single client cannot escape its bucket by rotating within its allocation. Size aggregation and capacity together: one /48 contains 65 536 /64s, exactly the default key capacity. Deployments serving such allocations should use a broader prefix, raise the key bound, or both. You can also key on anything else with a plain closure extractor.

See /examples/src/http_rate_limit.rs for all of the above in action, including the matcher policy map and the 429 + Retry-After mapping.

Bandwidth: throttling streams

ThrottleLayer (rama::net::stream::layer) wraps a connection’s IO in a ThrottledIo that paces reads and/or writes against a byte-rate bucket. Read-side throttling back-pressures the peer through transport flow control; write-side throttling paces your egress.

ThrottleMode chooses the scope: PerConn gives every connection its own budget, while Shared spends one aggregate cap across all connections holding the handle. When both directions are enabled, PerConn gives them independent buckets; Shared makes reads and writes spend that same aggregate budget. Throttling drops into any transport stack (TCP, TLS-wrapped, proxied bridges) exactly like the byte-tracker layers, and an OutgoingThrottleLayer covers the client-connector side.

The same layers take a QUIC Connection, such as one serving HTTP/3: all the streams and datagrams of that connection then spend from its one budget per direction, with PerConn and Shared scoping it as for TCP. Throttle layers stack: a byte stream nests one ThrottledIo per layer, a QUIC connection adds one budget per layer, and every one applies. Stream data waits for budget. A datagram, such as an HTTP/3 datagram, spends from the same budget of its direction but never waits: one the budget cannot pay for at once is dropped and counted in the connection’s ConnectionStats::gated_datagrams. A QUIC connection paces its streams and admits its datagrams through the gates added to it, so the same hooks serve other policies, such as byte accounting. HTTP/1.1 and HTTP/2 run over one byte stream instead: its throttle paces all the requests of a connection together, as HTTP/2 streams share that budget.

See /examples/src/tcp_listener_layers.rs.

Pacing datagrams

PacedSink (rama::stream) paces any framed Sink — a ConnectedUdpFramed, a UdpFramed, a unix datagram codec — by each item’s byte cost, or per item via a cost closure. Items are never split or dropped: sending stays atomic and the budget is repaid before the next item is accepted. The Stream half is passed through untouched, so duplex transports stay bridgeable.

See /examples/src/udp_codec.rs.

UDP tunnels: CONNECT-UDP

A MASQUE proxy limits its CONNECT-UDP tunnels at three scopes. Every limit is opt-in except the first-reply allowance below:

• The proxy: rama_masque::limits::TunnelLimits::with_max_tunnels caps the tunnels running at once. A full proxy answers 503 with Proxy-Status: connection_limit_reached, before it resolves or connects anything.
• A principal: with_max_tunnels_per_principal caps the tunnels of one principal, and with_principal_budget (PrincipalBudget) sets packet and byte rates per direction and a quota per window, shared by all of its tunnels. The principal is the authenticated user, else the client IP, unless a keyer of your own (any InputToRateKey) says otherwise. Principals are tracked in a bounded map (65 536 by default) that, like KeyedRatePolicy, refuses a new principal when full instead of evicting a live one. A principal is forgotten only once it has been idle for a while, its rates have refilled and its quota window has ended, so closing and reopening tunnels gains it nothing.
• A tunnel: TunnelBudget sets packet and byte rates per direction and a lifetime quota. Until the target answers, it also allows only a small first-reply allowance (64 packets or 64 KiB by default), so a tunnel cannot spray a target that never replies.

Every datagram is charged once where it enters the proxy, to the principal and then to the tunnel. Empty, unknown-context and oversized datagrams are charged too, so a client cannot send junk for free. A datagram over budget is dropped and counted in the tunnel’s ConnectUdpStats; it never waits, because a delayed datagram would only add latency to a protocol that recovers loss itself. The HTTP/3 connection carrying tunnels can also take a throttle: its datagrams then spend from the connection’s budget in the same drop-not-wait way.

On the CLI

The rama serve subcommands (echo, fp, http-test, ip, fs, proxy, discard) expose these limits as flags:

• --rate <N>: requests per second for http-serving commands and modes, new TCP/TLS connections per second for the raw transport ones (0 disables it); UDP discard mode rejects this flag because its service is one aggregate datagram stream rather than one service call per peer; http rejections are 429 responses with a Retry-After header;
• --throttle <BYTES_PER_SEC>: per-connection byte-rate shaping, applied to each direction independently (0 disables it); an HTTP/3 connection is throttled as one, across all its streams. UDP discard mode has no connection boundary, so its read throttle is one aggregate socket budget.

For the same reason, discard mode’s --concurrent and --timeout settings apply only to TCP/TLS connections; --concurrent is rejected in UDP mode and the connection timeout is ignored there.