Oblique Peer Review — a 1+N pattern, and the place it breaks

Fabry

I want to put a pattern on the bench, not announce a finished thing. It’s written up as a preprint — Oblique Peer Review: Extending Pyragogy Design Patterns through Structural Isolation and the Limits of Blind Convergence (Zenodo, DOI 10.5281/zenodo.20544658, CC BY 4.0) — but the paper is the long version. This post is the short version, and it’s here because Pattern Workshops is where things get tested, extended, or refuted. I’d rather it got refuted here than admired.

The pattern, in one breath

1 + N. One human orchestrator, N AI agents from different vendors, and one rule: the agents never read each other. Information moves only through the human, who strips it of its source and its argumentative framing before passing it on. No agent knows whose reasoning it’s building on, or whether it’s the first voice or the fourth.

The point isn’t to make the agents disagree. It’s to keep their analytical axes independent, so they cross the problem at different angles instead of collapsing into one voice. The friction is geometric — a property of how the trajectories intersect — not hostility between critics. I called it oblique for that reason: not parallel (redundant), not frontally opposed (sterile), but transversal.

It’s the Peeragogy move applied to synthetic peers: when the peer has no stakes and no accountability, the epistemic value can’t live in the agent’s judgment. It has to be engineered into the architecture that arranges the agents.

Where it breaks — and why that’s the actual contribution

Here’s the part I’d defend least confidently, which is exactly why it’s the part worth workshopping.

The pattern is built on the premise that independent, mutually-blind agents supply friction a single model can’t. That premise does real work — but it has a hard limit, and the paper’s central finding is that limit:

Blindness removes imitation, not error. Independent agents can converge on the same wrong answer, for the same reason independent instruments can share a systematic bias — not because they talk to each other, but because they’re exposed to the same salient feature and respond to it the same way. When that happens, their agreement feels like corroboration and is, from the inside, indistinguishable from the real thing.

I caught this the hard way. Three independent models — different vendors — converged on a fix that looked perfect and was endorsed by all of them at once. It was wrong. It would have made the system blind to short-form defamation. Three converging AIs didn’t catch it; a check run against recorded data, outside the loop of agents, did — and only because the check was aimed deliberately at the cases the fix might break, not the cases it was built to fix.

So the stopping rule can’t be agreement. Convergence among the agents is a quality signal only after a check outside the loop, never in place of one.

The two ways the human fails

Both failures land on the same node — the one in the 1+N:

Cognitive Impedance Mismatch. The agents generate faster and denser than the human can integrate. Past a threshold the operator keeps steering but stops absorbing — delegated comprehension — and the loop has quietly become the automation it was meant to prevent. A limit of bandwidth.
False convergence. The one just described — the shared error that reads as proof, and lulls the operator into trusting it. A limit of discernment.

One overwhelms the human; the other reassures him. A real operator can hit either without noticing.

What I’m bringing to the bench

This is n=1 — one practitioner, one body of work, observed from the inside, by the same person who designed the pattern. That’s a genuine limit, and I’d rather state it than have it pointed out. The reflexive circularity is real too: the paper was itself produced through the pattern it describes.

So the honest questions are the ones I can’t answer alone:

Does it survive other hands? The pattern needs a knowledgeable human at the centre — the operator’s domain knowledge repeatedly supplied the ground truth the agents lacked. Run by someone other than its designer, on problems outside software, does it still do anything?
Can the false-convergence check be generalized? I could only build a hand-made, case-specific discriminator — did the independent voices touch distinct features of the problem, or pile onto the same one? Does that distinction hold beyond the single case? Can it be detected without a human reading the underlying material?
Is “delegated comprehension” detectable before it’s too late? The CIM is named here as an observed boundary, not measured. What would it look like to instrument the operator’s integration load instead of relying on his own report of when he started slipping?

If you’ve run something like 1+N — even informally, even just bouncing one decision between two models — I want to hear where it held and where it didn’t. Especially where it didn’t.

The paper is the formal carry. This is the invitation to break it.