UnPeeragogy: An AI-Assisted Audit of the Peeragogy Handbook
-
Status: Completed — first audit cycle
Date started: 2026-08
Date completed: 2026-09
Participants: 1 human researcher + 6 AI agents
Corpus: Peeragogy Handbook — 88 nodes
Hypothesis
What happens when the patterns of the Peeragogy Handbook are systematically examined against documented evidence and real-world practice?
The goal was not to prove that Peeragogy is wrong.
The goal was to find where its theoretical patterns remain well supported, where evidence introduces tension, where claims are underspecified, and where important questions remain unanswered.
The experiment started from a simple principle:
The model may propose. The evidence must dispose.
Method
The original experiment was designed as a human research laboratory.
Practitioners were invited to contribute critical incidents from their own experience: situations in which peer-learning theory met real conditions, including friction, failure, contradiction, or unexpected outcomes.
The laboratory remained open.
Nobody submitted a critical incident.
Rather than treating that silence as the end of the experiment, I changed the experimental conditions.
I built a machine-mediated research branch.
The resulting pipeline uses six AI agents, coordinated across five analytical stages, with different responsibilities:
- Theory Interpretation — identifies the claims and assumptions expressed by each pattern.
- Evidence Hunting — searches for relevant empirical, documented, or research-based evidence.
- Grounding Gate — verifies whether proposed evidence actually supports the claim being examined.
- Critical Analysis — evaluates support, contradiction, ambiguity, and limitations.
- Tension Analysis & Documentation — compares interpretations, records tensions, and produces the audit trail.
The system does not treat an LLM’s internal knowledge as evidence.
Evidence must be grounded in identifiable sources.
What Happened
The complete Peeragogy Handbook corpus was processed across 88 nodes.
Each node was examined against external evidence and analysed through the multi-agent pipeline.
The resulting audit produced a structured assessment rather than a binary classification of true or false.
The audit records different relationships between theoretical claims and available evidence:
- Support
- Tension
- Contradiction
- Ambiguity
- Insufficient evidence
This distinction matters.
A lack of contradictory evidence is not evidence that a theoretical claim is true.
Likewise, evidence of tension does not automatically make a pattern false.
It may indicate that a pattern works under some conditions but not others, that its assumptions are underspecified, or that the available evidence is insufficient to establish a stronger conclusion.
What Broke
The first experimental condition broke before the research itself really began.
The human participation model produced no field reports.
The laboratory was open, but nobody walked through the door.
That forced a methodological change.
Instead of pretending that human participation had occurred, I recorded the absence of participation as part of the experiment and built a second branch using AI-mediated systematic analysis.
This changed the nature of the experiment.
The machine branch is not equivalent to human participation.
AI agents do not have lived experience, social stakes, or personal involvement in the situations described by practitioners.
What they can provide is something different: systematic comparison, source retrieval, structured critique, and repeated examination of a theoretical corpus.
That distinction remains central to the interpretation of the results.
Surprises
The most important surprise was that the failed participation model did not end the experiment.
It changed it.
The absence of human contributions became the condition that led to the construction of the multi-agent pipeline.
The second surprise was how different the resulting activity felt from ordinary AI-assisted writing.
The agents were not used simply to generate a report.
They were assigned different functions and asked to challenge one another’s outputs.
One agent could propose an interpretation.
Another could search for evidence.
Another could reject weak grounding.
Another could identify tensions.
The orchestrator then had to reconcile the resulting outputs.
The result was less like asking an AI for an answer and more like constructing a small adversarial research process.
Results
Corpus
88 — Peeragogy Handbook nodes audited
Evidence
304 — evidence items identified
52 — entities represented in the evidence
50 — distinct source URLsAnalysis
6 — AI agents
5 — analytical stages
1 — orchestrated research pipelineThe full quantitative and qualitative results are documented in the audit report.
What Changed Mid-Experiment
The initial hypothesis assumed that the most valuable evidence would come from practitioners contributing critical incidents.
That condition did not materialise.
So the experiment changed from:
Theory → Human Critical Incidents → Analysis
to:
Theory → External Evidence → Multi-Agent Analysis
This was not treated as a correction to the data.
It was treated as a change in experimental conditions.
That distinction is important because the machine-mediated branch answers a different question from the original human laboratory.
The first asks:
What happens when practitioners report what happened to them?
The second asks:
What happens when a theoretical corpus is systematically interrogated against available evidence?
UnPeeragogy became the name for the second experiment.
Analysis
The audit should not be read as a verdict on Peeragogy.
Its purpose is to make theoretical claims more exposed to evidence, criticism, and revision.
A pattern with near-zero tension may not be true — it may simply lack sufficient evidence to challenge it.
A pattern with high tension may not be false — it may operate well under conditions that the corpus does not adequately describe.
The most useful result is therefore not a ranking of “good” and “bad” patterns.
It is a map of where the relationship between theory and evidence becomes interesting.
The central research loop is:
Theory → Evidence → Interpretation → Counter-evidence → Revision
The audit is not a conclusion.
It is a starting point.
What This Experiment Changed
I started with a question about Peeragogy.
I ended up building a reusable research protocol for moving between theoretical claims and external evidence.
The experiment also changed my understanding of participation.
I originally thought the laboratory required people to enter the room before anything meaningful could happen.
It turned out that the laboratory could also be a set of conditions, tools, protocols, and questions that remain active even when nobody enters.
That does not make the human laboratory unnecessary.
It makes the distinction between human experience and machine-mediated analysis more explicit.
The door remains open.
Human field reports are still the evidence that can bring lived experience back into the system.
Artifacts
Quick Links
Resource Link Dedicated Audit Page unpeeragogy.pyragogy.org/audit Full Report (Zenodo) doi.org/10.5281/zenodo.22309102 Pipeline Repository github.com/pyragogy/UnPeeragogy UnPeeragogy Protocol unpeeragogy.pyragogy.org/protocol Background Blog — Part I blog.pyragogy.org/posts/2dd675a5 Background Blog — Part II blog.pyragogy.org/posts/e5f507dd Peeragogy Handbook peeragogy.org License CC Zero — Public Domain
Get Involved
UnPeeragogy is built to be used, challenged, and improved.
If you’re applying peer-learning patterns in your own context — a classroom, a community, an open-source project, or another collaborative environment — your field reports are the evidence that makes the audit meaningful over time.
The machine-mediated audit can systematically examine a corpus.
It cannot replace lived experience.
That is where the next experimental cycle begins.
- Submit a field report via the UnPeeragogy repository
- Discuss the protocol on the Pyragogy Forum
- Fork the pipeline and run it on your own corpus
- Challenge the results by providing counter-evidence or alternative interpretations
Next Steps
The first audit cycle is complete.
The laboratory is not.
The next step is to bring external practitioners back into the loop.
Field reports, counter-evidence, alternative interpretations, and independent runs of the pipeline can all become inputs for subsequent iterations.
The experiment therefore remains open to revision.
The audit is not a conclusion. It is a starting point.
And perhaps that is the most useful result of the experiment so far:
The first experimental condition failed to produce participation.
The experiment did not stop. The conditions changed.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login