- The article describes a benchmark for extracting genealogical data from Portuguese parish records using AI, focusing on structured JSON output of people and relationships.
- The benchmark consists of 84 images from 17 parish books, resulting in 434 records across baptisms, marriages, and deaths, spanning the 17th to 19th centuries.
- Scoring uses ordered and unordered metrics, with field-level partial credit based on Levenshtein distance for names and temporal distance for dates, and a record-type mismatch zeroes all fields.
- Experiments compare single-pass extraction (T0) against an agentic harness (T1) on Gemini 3.5 Flash Lite and Gemini 3.6 Flash, showing T1 improves scores but at higher cost.
- Older (17th-century) records score lower due to faded ink, dense abbreviations, and physical damage, with the 17th century showing a clear penalty across all model-environment pairs.
- Agentic T1 helps decompose complex images (e.g., splitting pages, adjusting contrast) but can introduce errors like misreading dates (e.g., 1606 vs 1666).
- Limitations include small dataset size, lack of independent verification, no repeated sampling, and private dataset to avoid training data contamination.