UnPeeragogy: An AI-Assisted Audit of the Peeragogy Handbook

Fabry

Status: Completed — first audit cycle
Date started: 2026-08
Date completed: 2026-09
Participants: 1 human researcher + 6 AI agents
Corpus: Peeragogy Handbook — 88 nodes

Hypothesis

What happens when the patterns of the Peeragogy Handbook are systematically examined against documented evidence and real-world practice?

The goal was not to prove that Peeragogy is wrong.

The goal was to find where its theoretical patterns remain well supported, where evidence introduces tension, where claims are underspecified, and where important questions remain unanswered.

The experiment started from a simple principle:

The model may propose. The evidence must dispose.

Method

The original experiment was designed as a human research laboratory.

Practitioners were invited to contribute critical incidents from their own experience: situations in which peer-learning theory met real conditions, including friction, failure, contradiction, or unexpected outcomes.

The laboratory remained open.

Nobody submitted a critical incident.

Rather than treating that silence as the end of the experiment, I changed the experimental conditions.

I built a machine-mediated research branch.

The resulting pipeline uses six AI agents, coordinated across five analytical stages, with different responsibilities:

Theory Interpretation — identifies the claims and assumptions expressed by each pattern.
Evidence Hunting — searches for relevant empirical, documented, or research-based evidence.
Grounding Gate — verifies whether proposed evidence actually supports the claim being examined.
Critical Analysis — evaluates support, contradiction, ambiguity, and limitations.
Tension Analysis & Documentation — compares interpretations, records tensions, and produces the audit trail.

The system does not treat an LLM’s internal knowledge as evidence.

Evidence must be grounded in identifiable sources.

What Happened

The complete Peeragogy Handbook corpus was processed across 88 nodes.

Each node was examined against external evidence and analysed through the multi-agent pipeline.

The resulting audit produced a structured assessment rather than a binary classification of true or false.

The audit records different relationships between theoretical claims and available evidence:

Support
Tension
Contradiction
Ambiguity
Insufficient evidence

This distinction matters.

A lack of contradictory evidence is not evidence that a theoretical claim is true.

Likewise, evidence of tension does not automatically make a pattern false.

It may indicate that a pattern works under some conditions but not others, that its assumptions are underspecified, or that the available evidence is insufficient to establish a stronger conclusion.

What Broke

The first experimental condition broke before the research itself really began.

The human participation model produced no field reports.

The laboratory was open, but nobody walked through the door.

That forced a methodological change.

Instead of pretending that human participation had occurred, I recorded the absence of participation as part of the experiment and built a second branch using AI-mediated systematic analysis.

This changed the nature of the experiment.

The machine branch is not equivalent to human participation.

AI agents do not have lived experience, social stakes, or personal involvement in the situations described by practitioners.

What they can provide is something different: systematic comparison, source retrieval, structured critique, and repeated examination of a theoretical corpus.

That distinction remains central to the interpretation of the results.

Surprises

The most important surprise was that the failed participation model did not end the experiment.

It changed it.

The absence of human contributions became the condition that led to the construction of the multi-agent pipeline.

The second surprise was how different the resulting activity felt from ordinary AI-assisted writing.

The agents were not used simply to generate a report.

They were assigned different functions and asked to challenge one another’s outputs.

One agent could propose an interpretation.

Another could search for evidence.

Another could reject weak grounding.

Another could identify tensions.

The orchestrator then had to reconcile the resulting outputs.

The result was less like asking an AI for an answer and more like constructing a small adversarial research process.

Results

Corpus

88 — Peeragogy Handbook nodes audited

Evidence

304 — evidence items identified
52 — entities represented in the evidence
50 — distinct source URLs

Analysis

6 — AI agents
5 — analytical stages
1 — orchestrated research pipeline

The full quantitative and qualitative results are documented in the audit report.

What Changed Mid-Experiment

The initial hypothesis assumed that the most valuable evidence would come from practitioners contributing critical incidents.

That condition did not materialise.

So the experiment changed from:

Theory → Human Critical Incidents → Analysis

to:

Theory → External Evidence → Multi-Agent Analysis

This was not treated as a correction to the data.

It was treated as a change in experimental conditions.

That distinction is important because the machine-mediated branch answers a different question from the original human laboratory.

The first asks:

What happens when practitioners report what happened to them?

The second asks:

What happens when a theoretical corpus is systematically interrogated against available evidence?

UnPeeragogy became the name for the second experiment.

Analysis

The audit should not be read as a verdict on Peeragogy.

Its purpose is to make theoretical claims more exposed to evidence, criticism, and revision.

A pattern with near-zero tension may not be true — it may simply lack sufficient evidence to challenge it.

A pattern with high tension may not be false — it may operate well under conditions that the corpus does not adequately describe.

The most useful result is therefore not a ranking of “good” and “bad” patterns.

It is a map of where the relationship between theory and evidence becomes interesting.

The central research loop is:

Theory → Evidence → Interpretation → Counter-evidence → Revision

The audit is not a conclusion.

It is a starting point.

What This Experiment Changed

I started with a question about Peeragogy.

I ended up building a reusable research protocol for moving between theoretical claims and external evidence.

The experiment also changed my understanding of participation.

I originally thought the laboratory required people to enter the room before anything meaningful could happen.

It turned out that the laboratory could also be a set of conditions, tools, protocols, and questions that remain active even when nobody enters.

That does not make the human laboratory unnecessary.

It makes the distinction between human experience and machine-mediated analysis more explicit.

The door remains open.

Human field reports are still the evidence that can bring lived experience back into the system.

Artifacts

Quick Links

Resource	Link
Dedicated Audit Page	unpeeragogy.pyragogy.org/audit
Full Report (Zenodo)	doi.org/10.5281/zenodo.22309102
Pipeline Repository	github.com/pyragogy/UnPeeragogy
UnPeeragogy Protocol	unpeeragogy.pyragogy.org/protocol
Background Blog — Part I	blog.pyragogy.org/posts/2dd675a5
Background Blog — Part II	blog.pyragogy.org/posts/e5f507dd
Peeragogy Handbook	peeragogy.org
License	CC Zero — Public Domain

Get Involved

UnPeeragogy is built to be used, challenged, and improved.

If you’re applying peer-learning patterns in your own context — a classroom, a community, an open-source project, or another collaborative environment — your field reports are the evidence that makes the audit meaningful over time.

The machine-mediated audit can systematically examine a corpus.

It cannot replace lived experience.

That is where the next experimental cycle begins.

Submit a field report via the UnPeeragogy repository
Discuss the protocol on the Pyragogy Forum
Fork the pipeline and run it on your own corpus
Challenge the results by providing counter-evidence or alternative interpretations

Next Steps

The first audit cycle is complete.

The laboratory is not.

The next step is to bring external practitioners back into the loop.

Field reports, counter-evidence, alternative interpretations, and independent runs of the pipeline can all become inputs for subsequent iterations.

The experiment therefore remains open to revision.

The audit is not a conclusion. It is a starting point.

And perhaps that is the most useful result of the experiment so far:

The first experimental condition failed to produce participation.
The experiment did not stop. The conditions changed.