The Index Is Not the World
A language model reading the Epistolæ corpus of medieval women's letters recovered 203 named individuals across 30 letters, compared to only 54 people connected in the traditional metadata-based network graph 136 of those 167 real people (after excluding scriptural/ancient figures and institutions) appeared nowhere in the metadata graph as senders or recipients — they were entirely invisible to the index-based network The metadata graph conflates roles: a single undifferentiated edge cannot dist
Analysis
TL;DR
- A language model reading the Epistolæ corpus of medieval women's letters recovered 203 named individuals across 30 letters, compared to only 54 people connected in the traditional metadata-based network graph
- 136 of those 167 real people (after excluding scriptural/ancient figures and institutions) appeared nowhere in the metadata graph as senders or recipients — they were entirely invisible to the index-based network
- The metadata graph conflates roles: a single undifferentiated edge cannot distinguish between a requester, a mediator, and a beneficiary in an act of intercession, whereas reading the text recovers these distinct relational roles
- The gap is most dramatic in administrative documents (charters, oaths, grants), where the oath of Murel alone accounts for 59 of the 136 hidden people, but even personal letters roughly double the recovered population
- The method uses an LLM strictly for extraction (naming people and identifying acts of intercession), while all counting and analysis is done by transparent, inspectable code — the model reads but never concludes
Why It Matters
This work demonstrates a concrete, measurable limitation of the dominant approach in digital humanities and computational social history: building networks from metadata indices discards the majority of the social world encoded in letter corpora. For AI practitioners working in historical or archival domains, it shows that LLMs can serve as scalable reading instruments that recover structured social data — not just names but roles and relational acts — from unstructured text, enabling close reading at corpus scale while remaining auditable.
Technical Details
- Corpus: Epistolæ, Medieval Women's Letters, a collection of just over 2,000 letters to and from women spanning the 4th–13th centuries, compiled and translated by Joan Ferrante
- Metadata graph: Built directly from the index — one node per sender/recipient, one edge per letter — yielding 978 people and 1,187 links across the full corpus
- Reading pipeline: An LLM was used to extract, in fixed form, every person named in each letter and every act of intercession (one person asking another to help or harm a third party). The model was constrained to extraction only — it never counted, inferred, or drew conclusions
- Sample design: 30 letters — 8 hand-picked for calibration, 22 drawn via seeded random generation (including the oath of Murel) — to avoid selection bias
- Validation: 8 of the 30 letters were read a second time and checked name-by-name against the source; the model neither invented people nor silently dropped them. Name matching was deliberately strict, making the hidden population count a conservative estimate
- Role extraction: The reading identified 11 acts of intercession across the sample, recovering three distinct roles (requester, mediator, beneficiary) that a single metadata edge cannot represent
- Reproducibility: All arithmetic and extracted lists are in a public repository; the reading itself was conducted inside a session and published as data rather than a scripted pipeline
Industry Insight
- AI for digital humanities should prioritize extraction over inference: The most defensible use of LLMs in archival work is as transparent extraction engines — pulling structured data from text while leaving all interpretation to auditable code. This model avoids the hallucination risk of generative analysis while still capturing semantic richness that pattern matching cannot.
- Network-based analysis of historical corpora systematically undercounts social participation: Researchers building graphs from indices should report the gap between indexed actors and text-recovered actors, especially when working with administrative, legal, or governance documents where witness lists and oaths inflate the difference. The loss is not uniform — it concentrates in document types that historians care about most.
- Role-aware graph construction is feasible and valuable: The intercession framework (requester → mediator → beneficiary) offers a generalizable schema for enriching metadata graphs with typed, multi-party edges. Any corpus where power, favor, or agency flows through text — not just between named correspondents — stands to gain from this approach, suggesting broader applicability beyond medieval studies to legal archives, diplomatic correspondence, and institutional records.
Disclaimer: The above content is generated by AI and is for reference only.