Module posted_recv 

Available on crate features std and tcp only.

Keep data that arrives right before a TCP reset.

When a peer sends data and then resets the connection before the application read that data, Windows discards it: the next read fails with WSAECONNRESET, or WSAECONNABORTED after a send to a peer that already closed. Linux, FreeBSD and macOS hand over the queued bytes first and only then report the reset.

Tokio on Windows waits for readiness and reads afterwards, so it never has a read with a real buffer waiting on the socket. A server that replies and resets right away can therefore lose its whole reply.

PostedRecv closes that gap. On Windows it keeps overlapped receives with their own buffers posted on the socket, so bytes land in user space as they arrive, and only reports the end of the stream or the reset once every received byte was read. On every other platform it passes reads straight through, with the same API.

Only raw TCP streams can be wrapped (see RawTcpStream): wrapping the socket under a TLS or peek layer would bypass that layer.

Limits

Bytes are kept whatever follows them if a receive is posted when they arrive. Each receive completes with whatever arrived, up to slot_size, and is posted again once a completion thread handled it, which takes microseconds. A part of a reply that arrives on its own fills ⌈len / slot_size⌉ receives, the last one maybe only partly, so a reply is kept for sure while its parts fill no more receives than there are slots: by default for instance one part of up to 32 KiB, or two of up to 16 KiB. Bytes beyond that, even a small third part, can arrive while no receive is posted and wait in the kernel, already acknowledged; a peer that resets right after they are acknowledged can beat the completion thread, and those bytes are lost. Size the slots for the replies that must survive a reset, with some room for how the network splits them: more slots for replies sent in many writes, larger ones for large replies. How busy the tokio runtime is does not matter.

No receive is posted while more than max_buffered bytes wait for the reader, so a reader that falls further behind is exposed again. If the system runs out of buffers for a receive, reads pass through tokio until receives can be posted again, with the same exposure.

Memory, per flow and with the default configuration:

• idle: the posted buffers, slots × slot_size = 32 KiB, locked by the kernel while posted;
• reader lagging: up to max_buffered plus the posted buffers of data, about 96 KiB, in buffers of slot_size. A peer sending segments of just over a quarter slot can make those buffers take up to four times the data, about 384 KiB.

The receives of the process complete on a pool of threads that grows while completions queue up and shrinks when idle, see CompletionThreads. A single stream still receives slower than through tokio on its own; round-trip latency stays the same.

Example

use rama_tcp::{TcpStream, posted_recv::PostedRecv};
use tokio::io::{AsyncReadExt, AsyncWriteExt};

let stream = tokio::net::TcpStream::connect("127.0.0.1:88").await?;
let mut stream = PostedRecv::new(TcpStream::new(stream));
stream.write_all(b"request").await?;
let mut reply = Vec::new();
stream.read_to_end(&mut reply).await?;

Modules

dial9dial9

Pre-defined dial9 events of PostedRecv.

Structs

CompletionThreads

How many threads complete the posted receives of the process.

PostedRecv

A raw TCP stream whose reads keep the data that arrives right before a reset.

PostedRecvConfig

Configuration of a PostedRecv.

PostedRecvConnector

A connector that wraps every connection its inner TCP connector establishes in a PostedRecv.

PostedRecvLayer

A Layer that wraps every connection a TCP connector establishes in a PostedRecv.

Enums

ThreadStartReason

Why a completion thread started.

ThreadStopReason

Why a completion thread stopped.

Traits

RawTcpStream

A raw TCP stream that PostedRecv can take over reading from.

Functions

completion_threads

The configuration of the completion threads of the process.

running_completion_threads

How many completion threads run right now; always 0 off Windows.

set_completion_threads

Configure the completion threads of the process, see CompletionThreads.