Add a StreamReaderBufferedProtocol #85451

Activity

1. tontinton commented on Jul 11, 2020

   

   tontintonmannequin

   on Jul 11, 2020

   MannequinAuthor

   More actions

   This will greatly increase performance, from my internal tests it was about 150% on linux.

   Using read_into instead of read will make it so we do not allocate a new buffer each time data is received.
2. tirkarthi

   added

   topic-asyncio

   on Jul 11, 2020
3. tontinton commented on Aug 29, 2020

   

   tontintonmannequin

   on Aug 29, 2020

   MannequinAuthor

   More actions

   bump
4. tontinton

   changed the title [-]Convert StreamReaderProtocol to a BufferedProtocol[/-] [+]Add a StreamReaderBufferedProtocol[/+] on Aug 29, 2020
5. tontinton

   changed the title [-]Convert StreamReaderProtocol to a BufferedProtocol[/-] [+]Add a StreamReaderBufferedProtocol[/+] on Aug 29, 2020
6. ezio-melotti

   transferred this issue fromon Apr 10, 2022
7. ezio-melotti

   added this to asyncioon Jul 17, 2022
8. ezio-melotti

   moved this to Todo in asyncioon Jul 17, 2022
9. webknjaz

   linked a pull request that will close this issuegh-85451: Add StreamReaderBufferedProtocol #21446on Jan 16, 2026
10. picnixz

    added

    stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

    type-featureA feature request or enhancementA feature request or enhancement

    on Jan 18, 2026
11. picnixz commented on Jan 18, 2026

    

    picnixz

    on Jan 18, 2026

    Last edited by picnixz

    Member

    More actions

    @webknjaz FTR, we do not link PRs to issues in general because of backports or if we need to do more things afterwards. We usually leave the responsibility of closing the issue to the core dev who merged the PR.

    cc @kumaraditya303 for the validity and relevence of this FR.
12. webknjaz

    removed a link to a pull requestgh-85451: Add StreamReaderBufferedProtocol #21446on Jan 18, 2026
13. webknjaz commented on Jan 18, 2026

    

    webknjaz

    on Jan 18, 2026

    Member

    More actions

    @picnixz ah, I didn't realize, reverted that. Thanks! I was going through a bunch of issues and noticed incorrect links in the OP, I corrected that by commenting one out (left a note in the OP to see if you click Edit) and assumed this was a part of the breakage. I'll stop linking PRs, then (there may be one or two others that I linked left, though).
14. yoch commented on Sep 8, 2026

    

    yoch

    on Sep 8, 2026 – with ChatGPT Codex Connector

    More actions

    I found a severe Linux allocator/pathological-latency case that makes the BufferedProtocol / recv_into() direction here more than a steady-state micro-optimization. It is reproducible without any third-party library.

    Environment where it reproduces reliably enough to characterize:

    • CPython 3.14.7
    • Linux aarch64, Raspberry Pi 5 (Cortex-A76)
    • glibc 2.43
    • selector event loop

    The selector transport still has max_size = 256 * 1024 and the ordinary Protocol path calls sock.recv(self.max_size). With small socket arrivals, socket.recv(262144) can enter a process-layout-dependent allocator regime where each small read effectively causes mmap -> recvfrom -> mremap -> munmap.

    A minimal raw-socket test is enough to reproduce it. The important shape is one small payload outstanding at a time and a fresh client process per repetition:

    # server process
    import socket
    
    ls = socket.socket()
    ls.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    ls.bind(("127.0.0.1", 9999))
    ls.listen()
    while True:
        c, _ = ls.accept()
        with c:
            c.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
            while True:
                c.sendall(b"x" * 256)
                if not c.recv(1):
                    break

    # run this client as a fresh process repeatedly
    import resource, socket, time
    
    s = socket.create_connection(("127.0.0.1", 9999))
    s.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
    for _ in range(2000):
        assert len(s.recv(262144)) == 256
        s.sendall(b"A")
    
    a = resource.getrusage(resource.RUSAGE_SELF)
    t0 = time.perf_counter()
    for _ in range(20000):
        assert len(s.recv(262144)) == 256
        s.sendall(b"A")
    dt = time.perf_counter() - t0
    b = resource.getrusage(resource.RUSAGE_SELF)
    print("us/read", dt * 1e6 / 20000)
    print("minor_faults/read", (b.ru_minflt - a.ru_minflt) / 20000)
    print("stime_us/read", (b.ru_stime - a.ru_stime) * 1e6 / 20000)

    On the same host, 8 fresh-process repetitions of raw socket.recv(262144) split into 5 fast processes at ~12.45-12.57 us/read with 0 minor faults/read and 3 slow processes at ~14.87-15.05 us/read with exactly ~2.0 minor faults/read. Major faults were always 0.

    Controls:

    • socket.recv(65536): 8/8 at 0 minor faults/read, ~12.49 us/read median.
    • reusable bytearray(262144) + socket.recv_into(): 8/8 at 0 minor faults/read and ~12.27 us/read median.
    • asyncio StreamReader / selector with the default max_size=262144: 2/16 fresh processes at exactly ~2.0 minor faults/read, ~35.5-36.0 us/read vs ~22.2-23.0 us/read in the fast regime.
    • the same asyncio path with transport.max_size=65536: 16/16 at 0 minor faults/read, ~22.36 us/read median.
    • setarch aarch64 -R with the 256 KiB read: 16/16 at 0 minor faults/read.
    • GLIBC_TUNABLES=glibc.malloc.mmap_threshold=1048576 with the 256 KiB read: 16/16 at 0 minor faults/read.

    A slow raw recv(262144) run under strace -c for 6000 measured reads produced:

    6000  mremap
    6079  mmap
    6049  munmap
    6000  recvfrom
    

    The recv_into() and 64 KiB variants had no per-read mremap; only normal startup mappings remained.

    This looks closely related to #121313: recv(n) has to allocate a bytes for the requested size before it knows how much the kernel will return, then shrink it. At 256 KiB this crosses glibc's mmap territory, and depending on process/allocator layout the allocate-short-read-resize-free cycle can become dramatically more expensive. The process-layout dependence explains why it may be invisible on many machines; I could not reproduce the bimodality on an x86-64 KVM host under the same application-level workload.

    The full controlled run and raw evidence are public here: https://github.com/yoch/mqttium/actions/runs/34188738358 . Although that repository is MQTT-related, this particular run contains a standalone socket reproducer and does not import MQTTium or use an MQTT broker.

    So there seem to be two possible upstream remedies worth considering:

    1. the structural solution proposed by this issue/PR: use a reusable buffer + recv_into() for streams; or
    2. independently, reconsider the selector transport's 256 KiB default read size for ordinary Protocols, since a 64 KiB cap completely eliminated this allocator regime in the experiment without requiring a persistent per-stream 256 KiB buffer.

    I can provide a smaller CPython-repo benchmark/test if useful.