Vatican Document Scanning Algorithm
Overview
The Vatican scanning system processes scanned historical documents through a multi-phase pipeline that separates metadata extraction from body text transcription and verification. The result is a clean, machine-readable XML structure suitable for preservation and further processing.
Architecture
Phases
The scanning process consists of several sequential phases:
✅ Phase 1: Single-Page Scanning [COMPLETE] — Scan individual pages, extract metadata and body text, and organize as individual XML files. - ✅ Phase 1a: Metadata Extraction [COMPLETE] — Extract title, page number, header, footer, etc. - ✅ Phase 1b: Body Text Extraction [COMPLETE] — Extract and group main content - ✅ Phase 1c: Paragraph Gap Detection [COMPLETE] — Detect and fill missing paragraphs between consecutive transcribed paragraphs - ✅ Phase 1d: Transcription Verification [COMPLETE] — Verify accuracy paragraph-by-paragraph
✅ Phase 2: Document Consolidation [COMPLETE] — Collate all XML page documents and generate a single unified output.LL.xml document.
� Phase 3: Translation into Multiple Languages [IN PROGRESS] — Translate consolidated document into Italian, French, English, Portuguese, and Latin.
Status Legend: 🔵 = Open, 🟡 = In Progress, 🟢 = Review, ✅ = Complete
Each phase processes one page at a time and produces an XML file (output.PP.es.xml, where PP is the page number and es is the language code for Spanish).
Phase 1: Single-Page Scanning [Description]
<?xml version="1.0" encoding="utf-8"?>
<root>
<meta>
<title>Carta IV</title>
<page_number>1</page_number>
<header></header>
<footer></footer>
<location>Barquilla</location>
<date>Sept. 6 de 1914</date>
<!-- Additional metadata fields as extracted -->
</meta>
<body>
<p id="p-1">...</p>
<!-- Additional paragraphs as extracted -->
</body>
</root>
[TRANSLATED TO SPANISH]
You are a paleographer extracting metadata from historical documents.
TASK: Identify and extract metadata from this page image.
EXTRACT (if visible):
1. title: Document or section title
2. page_number: Page number (as displayed)
3. header: Any header text
4. footer: Any footer text
5. location: Geographic location (if mentioned)
6. date: Date (if present)
7. salutation: Opening greeting (if present)
8. any other distinct metadata elements
OUTPUT FORMAT (strict JSON):
{
"title": "...",
"page_number": "...",
"header": "...",
...
}
Return ONLY valid JSON. Do not include any other text.
` blocks with unique sequence IDs ### Output - Updates `output.{page_number}.es.xml` - Adds `
` element with child `` elements - Each `
` has a `pid` attribute: `p-01`, `p-02`, etc. (zero-padded 2-digit numbers) ### XML Structure (Phase 1b Result)
<?xml version="1.0" encoding="utf-8"?>
<root>
<meta>
<title>Carta IV</title>
<page_number>1</page_number>
<!-- ... metadata ... -->
</meta>
<body>
<p pid="p-01">Hoy hace ocho días os dirigía la tercera carta familiar, esforzándome para inspirar en vuestros corazones un amor cual se merece la Disciplina religiosa.</p>
<p pid="p-02">Os prometía continuar escribiéndoos semanalmente sobre la misma hasta poder daros una idea exacta sobre su importancia y necesidad.</p>
<p pid="p-03">Vamos pues al grano y contando con el auxilio de Dios y vuestra benevolencia en escucharme, con poca palabra os diré en esta de las ventajas, por consiguiente veréis la necesidad también, de la disciplina.</p>
<!-- ... more paragraphs ... -->
</body>
</root>
[TRANSLATED TO SPANISH]
You are an accurate paleographer transcribing historical Spanish documents.
TASK: Transcribe the MAIN BODY TEXT from this page image, excluding metadata, titles, page numbers, headers, and footers.
OUTPUT FORMAT (strict):
Provide text with natural paragraph breaks. Separate paragraphs with blank lines.
Preserve original spelling, abbreviations, and punctuation exactly.
Return ONLY the transcribed text, with no additional commentary or markers.
RULES:
- Preserve line breaks that indicate natural sentence/clause boundaries
- Group text logically by paragraph
- When in doubt, err on the side of paragraph breaks (more paragraphs, not fewer)
- Do NOT translate or correct spelling
- Include all visible text (marginal notes, cross-outs as [crossed-out: ...])
` elements) ### Process 1. Load all `
` elements from the XML body 2. Iterate through consecutive pairs `(p_0, p_1)`, `(p_1, p_2)`, …, `(p_{N-2}, p_{N-1})` 3. For each pair call the LLM with the image and both paragraph texts 4. Parse LLM response: - **"OK"** → no gap, advance to next pair - **"INSERT: [missing_text]"** → insert a new `
` between the two paragraphs, **renumber all `
` IDs sequentially**, and **restart from pair `(p_0, p_1)` again** 5. Repeat until a full pass over all pairs produces no insertions 6. Save updated XML; pass the updated paragraph list to Phase 1d ### Why Restart After Each Insertion After inserting a paragraph the total paragraph count changes and all subsequent IDs have shifted. Restarting from the beginning ensures: - The newly inserted text is itself compared against its neighbours - No gap between any two consecutive paragraphs is ever missed - The final list is internally consistent before Phase 1c begins ### Output - Updates `output.{page_number}.es.xml` - Adds zero or more new `
` elements (without `verified` attribute — Phase 1d handles that) - All `
` IDs are sequential from `p-01` after the last pass ### XML Structure (Phase 1c result — one insertion example)
<!-- Before gap detection (Phase 1b output, after Phase 1c begins) -->
<body>
<p pid="p-01">First paragraph from phase 1b.</p>
<p pid="p-02">Third paragraph from phase 1b (gap detected before this one).</p>
</body>
<!-- After Phase 1c: missing paragraph inserted, IDs renumbered -->
<body>
<p pid="p-01">First paragraph from phase 1b.</p>
<p pid="p-02">Inserted missing paragraph transcribed from image.</p>
<p pid="p-03">Third paragraph from phase 1b (now renumbered).</p>
</body>
[TRANSLATED TO SPANISH]
You are an accurate paleographer reviewing a historical document transcription.
TASK: Examine the page image and these two consecutive paragraphs that were transcribed from it.
Determine whether any visible text from the image is MISSING between them.
PARAGRAPH A (comes first):
"%PARAGRAPH_A_TEXT%"
PARAGRAPH B (immediately follows A in the transcription):
"%PARAGRAPH_B_TEXT%"
Look carefully at the image between the end of paragraph A and the start of paragraph B.
RESPOND WITH ONE OF:
- OK (if no text is missing between them)
- INSERT: [missing_text] (full text of the missing passage, transcribed from the image)
Do NOT provide explanations. Respond ONLY with OK or INSERT: ...
` elements) ### Process 1. For each `
` element in the XML:
- Extract the paragraph text
- Call LLM with the image and a verification prompt focusing on that paragraph's region
- Request verification: "OK" or "REPLACE: ` element with verified text or mark with verification status
### Output
- Updates `output.{page_number}.es.xml`
- Marks each ` ` with `verified="true/false"` attribute
- Optional: Adds `
### Verification Prompt Template (Phase 1d)
<?xml version="1.0" encoding="utf-8"?>
<root>
<meta>
<title>Carta IV</title>
<page_number>1</page_number>
<!-- ... metadata ... -->
</meta>
<verification>
<status>in-progress</status>
<verified_paragraphs>5</verified_paragraphs>
<total_paragraphs>8</total_paragraphs>
</verification>
<body>
<p pid="p-01" verified="true">Hoy hace ocho días os dirigía la tercera carta familiar, esforzándome para inspirar en vuestros corazones un amor cual se merece la Disciplina religiosa.</p>
<p pid="p-02" verified="true">Os prometía continuar escribiéndoos semanalmente sobre la misma hasta poder daros una idea exacta sobre su importancia y necesidad.</p>
<!-- ... more paragraphs, some verified ... -->
</body>
</root>
[TRANSLATED TO SPANISH]
You are a paleographer verifying historical Spanish document transcriptions.
TASK: Review this paragraph transcribed from a historical document and verify its accuracy:
PARAGRAPH TO VERIFY:
"[PARAGRAPH_TEXT]"
Examine the image and confirm:
1. All words are spelled correctly (original spelling preserved)
2. All punctuation is correct
3. No words are missing or misread
RESPOND WITH ONE OF:
- OK (if transcription is correct)
- REPLACE: [corrected_text] (if changes needed, include the full corrected paragraph)
Do NOT provide explanations or reasoning. Respond ONLY with OK or REPLACE: ...
Phase 2: Document Consolidation
` element read from source files: - If the source XML already contains `page_nr` attribute (from Phase 1): Preserve it as-is - If missing: Extract from the source filename `output.{page_number}.{language}.xml` and add `page_nr="{page_number}"` to the `
` element - Result: Every `
` in consolidated document has accurate `page_nr` indicating source page 5. **Page Boundary Analysis:** For each page transition (after page 1): - Identify the last `
` element of the previous page - Identify the first `
` element of the current page - Call LLM with both paragraphs to determine if they form a continuous logical paragraph (accidentally split across page break) - If LLM determines continuation: Mark first `
` of current page with `continues_previous_paragraph="true"` attribute - Result will be used during text export to merge these paragraphs 6. Merge all paragraphs into a single unified XML structure 7. Assign global paragraph IDs across all pages (preserve page-aware numbering) 8. Write consolidated file as `output.LL.xml` (where LL is language code, e.g., `output.es.xml`) ### Page Boundary Continuation Check #### Purpose Historical documents often have logical paragraphs that span multiple pages. When the text ends on one page and continues on the next, the paragraph boundaries may not align with the page break. This step detects these cases automatically. #### Process per Page Transition 1. Extract last paragraph from previous page (`P_last`) 2. Extract first paragraph from current page (`P_first`) 3. Call LLM with a continuation detection prompt 4. LLM evaluates: "Are these clearly parts of one logical paragraph interrupted by a page break?" 5. Response: "YES" or "NO" - If "YES": Add `continues_previous_paragraph="true"` to `P_first` - If "NO": Leave as separate paragraphs #### Continuation Detection Prompt Template
[TRANSLATED TO SPANISH]
You are a document analyzer specializing in historical texts.
TASK: Determine if these two text excerpts form ONE LOGICAL PARAGRAPH interrupted by a page break, or are SEPARATE paragraphs.
PREVIOUS PAGE - LAST PARAGRAPH:
"[LAST_PARAGRAPH_OF_PREVIOUS_PAGE]"
CURRENT PAGE - FIRST PARAGRAPH:
"[FIRST_PARAGRAPH_OF_CURRENT_PAGE]"
ANALYSIS:
- Do they form a natural continuation (same topic, sentence structure)?
- Does P_first start mid-sentence or continuation of P_last's thought?
- Are there clear paragraph boundary markers (new indentation, topic shift)?
RESPOND WITH ONLY ONE:
- YES (if they form one logical paragraph split by page break)
- NO (if they are separate paragraphs)
Respond ONLY with YES or NO, nothing else.
` of the current page is marked:
<!-- Page 1, last paragraph -->
<p pid="p-02" page_nr="1" verified="true">
...and this is how the matter stands at the end of page one.
</p>
<!-- Page 2, first paragraph marked as continuation -->
<p pid="p-03" page_nr="2" verified="true" continues_previous_paragraph="true">
Yet we must also consider the opposing view that follows...
</p>
<p pid="p-02" page_nr="1">...end of first thought.</p>
<p pid="p-03" page_nr="2" continues_previous_paragraph="true">Continuation of same thought.</p>
...end of first thought. Continuation of same thought.
` element maintains traceability of paragraphs to their source pages during consolidation. This is essential for: - Identifying where each paragraph originated in the source document - Enabling page-specific queries and filtering in downstream processing - Supporting citation formats that require page references - Validating document continuity and structure ##### Processing Steps 1. **Reading Phase 1 Output:** - Each `output.{page_number}.{language}.xml` file contains paragraphs from a single page - If Phase 1 properly assigned `page_nr` attributes, extract and preserve them - Example: `
...
` 2. **Validation and Assignment:** - For each `` element extracted: - Check if `page_nr` attribute exists - If exists: Preserve as-is (already assigned in Phase 1) - If missing: Derive from source XML filename `output.{page_number}.{language}.xml` and add `page_nr="{page_number}"` - Result: Every paragraph in consolidated output has a valid `page_nr` 3. **Consolidation Phase:** - When merging paragraphs from multiple source files, maintain `page_nr` values - Do NOT renumber or reassign `page_nr` during consolidation - Preserve the original page source for each paragraph - Example sequence: ```xml
Paragraph from page 1
Another from page 1
Paragraph from page 2
Another from page 2
``` ##### Attribute Format<!-- Basic format -->
<p pid="p-01" page_nr="1">Text content</p>
<!-- With verification and continuation attributes -->
<p pid="p-03" page_nr="2" verified="true" continues_previous_paragraph="true">Text content</p>
<!-- All attributes together -->
<p pid="p-15" page_nr="5" verified="true" continues_previous_paragraph="false">Text content</p>
<body>
<!-- Page 1 -->
<page page_nr="1" />
<p pid="p-01" page_nr="1">First paragraph from page 1</p>
<p pid="p-02" page_nr="1">Second paragraph from page 1</p>
<!-- Page 2 -->
<page page_nr="2" />
<p pid="p-03" page_nr="2" continues_previous_paragraph="true">First paragraph from page 2</p>
<p pid="p-04" page_nr="2">Second paragraph from page 2</p>
<!-- Page 3 -->
<page page_nr="3" />
<p pid="p-05" page_nr="3">First paragraph from page 3</p>
<p pid="p-06" page_nr="3">Second paragraph from page 3</p>
</body>
<?xml version="1.0" encoding="utf-8"?>
<root>
<document>
<metadata>
<title>Carta IV</title>
<total_pages>6</total_pages>
<language>es</language>
<consolidated_timestamp>2026-04-06T12:00:00Z</consolidated_timestamp>
</metadata>
<body>
<!-- Page 1 -->
<page page_nr="1" />
<p pid="p-01" page_nr="1">First paragraph from page 1</p>
<p pid="p-02" page_nr="1">Second paragraph from page 1</p>
<!-- Page 2 -->
<page page_nr="2" />
<p pid="p-03" page_nr="2" continues_previous_paragraph="true">First paragraph from page 2 (continues from page 1)</p>
<p pid="p-04" page_nr="2">Second paragraph from page 2</p>
<!-- Page 3 -->
<page page_nr="3" />
<p pid="p-05" page_nr="3">First paragraph from page 3</p>
<p pid="p-06" page_nr="3">Second paragraph from page 3</p>
<p pid="p-07" page_nr="3">Third paragraph from page 3</p>
<!-- ... remaining pages ... -->
</body>
</document>
</root>
Phase 3: Translation into Multiple Languages
` elements in order
3. **For each target language** (`it`, `fr`, `en`, `pt`, `la`):
a. Initialise an empty `translations_so_far` list
b. Copy document XML skeleton (metadata updated with ` ` to output XML, preserving all attributes (`pid`, `page_nr`, `continues_previous_paragraph`, etc.) and adding `source_text` attribute
- Append `(pid, translated_text)` to `translations_so_far`
d. Write `output.{lang}.xml`
### Output
- Multiple consolidated XML files, one per language:
- `output.it.xml` — Italian translation
- `output.fr.xml` — French translation
- `output.en.xml` — English translation
- `output.pt.xml` — Portuguese translation
- `output.la.xml` — Latin translation
### System Prompt Template (English, translated to target language)
### User Message Template (English, translated to target language)
You are a professional translator specializing in historical Catholic documents
from early 20th century Latin America.
Your task: translate a single paragraph from %SOURCE_LANGUAGE_NAME% to %TARGET_LANGUAGE_NAME%.
You are provided with:
1. The full original document in %SOURCE_LANGUAGE_NAME% — for context and consistent meaning
2. All paragraphs translated so far in %TARGET_LANGUAGE_NAME% — for style and
terminology consistency
3. The specific paragraph you must translate now
TRANSLATION REQUIREMENTS:
- Preserve the original meaning, tone, and register (formal, religious)
- Maintain consistent terminology matching the translations provided so far
- Keep proper names, place names, dates, and titles exactly as they appear in the source
- Respect the formal and devotional style of the source text
CRITICAL: Respond with ONLY the translated text of the requested paragraph.
Do NOT include labels, footnotes, explanations, or any additional content.
Placeholders `%ORIGINAL_PARAGRAPHS%`, `%PREVIOUS_TRANSLATIONS%`, and `%CURRENT_PARAGRAPH%` are substituted with live document content **after** template translation and are never cached.
### XML Structure (Phase 3 Result)
FULL ORIGINAL DOCUMENT (%SOURCE_LANGUAGE_NAME%):
%ORIGINAL_PARAGRAPHS%
TRANSLATIONS TO %TARGET_LANGUAGE_NAME% SO FAR:
%PREVIOUS_TRANSLATIONS%
TRANSLATE THE FOLLOWING PARAGRAPH FROM %SOURCE_LANGUAGE_NAME% TO %TARGET_LANGUAGE_NAME%:
%CURRENT_PARAGRAPH%
Respond with ONLY the translated text of this paragraph.
### Notes
- **Prompt templates are translated** to the target language (same convention as Phases 1 and 2)
- The `source_text` attribute stores the original Spanish text for each paragraph
- All structural attributes (`pid`, `page_nr`, `continues_previous_paragraph`) are copied unchanged
- If the LLM returns an empty response for a paragraph, the source text is used as fallback and a warning is logged
<?xml version="1.0" encoding="utf-8"?>
<root>
<document>
<metadata>
<title>Carta IV</title>
<total_pages>6</total_pages>
<language>it</language> <!-- updated to target language -->
<source_language>es</source_language> <!-- added by Phase 3 -->
<translated_timestamp>2026-04-06T13:00:00Z</translated_timestamp>
</metadata>
<body>
<!-- Page markers preserved from Phase 2 -->
<page page_nr="1" />
<p pid="p-01" page_nr="1" source_text="Hoy hace ocho días os dirigía...">Oggi, otto giorni fa, vi inviavo...</p>
<p pid="p-02" page_nr="1" source_text="Os prometía continuar...">Vi promettevo di continuare...</p>
<page page_nr="2" />
<p pid="p-03" page_nr="2" continues_previous_paragraph="true"
source_text="...el auxilio de Dios...">...l'aiuto di Dio...</p>
<!-- ... remaining translated paragraphs ... -->
</body>
</document>
</root>
Phase 4: Text Export with Metadata Header
` elements - Format as plain text with: - Metadata header at top (title, language, page count, timestamps, etc.) - Separator line - Body text with paragraphs preserved - Write to text file: `doc.{language_code}.txt` ### Output - Multiple plain text files, one per language: - `doc.es.txt` — Spanish consolidated document (text format) - `doc.it.txt` — Italian consolidated document (text format) - `doc.fr.txt` — French consolidated document (text format) - `doc.en.txt` — English consolidated document (text format) - `doc.pt.txt` — Portuguese consolidated document (text format) - `doc.la.txt` — Latin consolidated document (text format) ### Text Format (Phase 4 Result)
================================================================================
DOCUMENT METADATA
================================================================================
Title: Carta IV
Language: es (Spanish)
Total Pages: 6
Source Language: es
Consolidated: 2026-04-06T12:00:00Z
Translated: 2026-04-06T13:30:00Z
================================================================================
DOCUMENT BODY
================================================================================
Hoy hace ocho días os dirigía la tercera carta familiar, esforzándome para
inspirar en vuestros corazones un amor cual se merece la Disciplina religiosa.
Os prometía continuar escribiéndoos semanalmente sobre la misma hasta poder
daros una idea exacta sobre su importancia y necesidad.
Vamos pues al grano y contando con el auxilio de Dios y vuestra benevolencia
en escucharme, con poca palabra os diré en esta de las ventajas, por
consiguiente veréis la necesidad también, de la disciplina.
[... more paragraphs ...]
Title: Extracted from <metadata><title>
Language: Extracted from <metadata><language>
Language Name: Human-readable language name (Spanish, Italian, French, etc.)
Total Pages: Extracted from <metadata><total_pages>
Source Language: Extracted from <metadata><source_language> (if present)
Status: Extracted from <metadata><status> (if present)
Consolidated: Extracted from <metadata><consolidated_timestamp>
Translated: Extracted from <metadata><translated_timestamp> (if present)
` elements in order - Preserve paragraph breaks (blank line between paragraphs) - No paragraph IDs or XML tags in output - Clean text only 3. **Language Name Mapping:** - es → Spanish - it → Italian - fr → French - en → English - pt → Portuguese - la → Latin 4. **File Encoding:** - UTF-8 encoding with BOM (for Excel compatibility) - Line endings: CRLF (Windows) or LF (Unix, based on platform) ### Text Export Prompt (None Required) Text export does not require LLM calls—it is a deterministic XML-to-text conversion.
Implementation Details
from src.services.translation_service import get_translation_service
translation_service = get_translation_service()
# Translate a static English prompt template (with permissible placeholders)
prompt_es = translation_service.translate(prompt_en, from_lang='en', to_lang='es')
# Later: substitute the placeholders with actual content
final_prompt = prompt_es.replace('%LANGUAGE%', 'Spanish').replace('%CONTENT%', actual_content)
` element in all XML documents (both single-page and consolidated) carries a `page_nr` attribute indicating its source page number:
<p pid="p-01" page_nr="1" verified="true">
Hoy hace ocho días os dirigía la tercera carta familiar...
</p>
<p pid="p-02" page_nr="1" verified="true">
Os prometía continuar escribiéndoos semanalmente...
</p>
<p pid="p-03" page_nr="2" verified="false">
Vamos pues al grano y contando con el auxilio de Dios...
</p>
` elements are created with `page_nr="{page_number}"` attribute - This is a simple assignment since all paragraphs in `output.{page_number}.es.xml` come from the same page **Phase 2 (Document Consolidation):** - When merging paragraphs from multiple single-page XML files: - Read source file `output.{page_number}.es.xml` - Extract all `
` elements with their `page_nr` attributes - Preserve `page_nr` when adding to consolidated document - Result: `
` elements in `output.es.xml` retain accurate page references **Phase 3 (Translation):** - When translating paragraphs from one language to another: - Preserve all attributes during translation (including `page_nr`) - Copy `page_nr` from source language XML to target language XML - Add `source_text` attribute instead of modifying existing attributes - Result: `
` elements in all translated files (`output.it.xml`, `output.fr.xml`, etc.) maintain consistent `page_nr` values **Phase 4 (Text Export):** - When converting XML to plain text: - Extract `page_nr` from each `
` element - Optionally include page markers in the exported text (e.g., `[Page 1]`, `[Page 2]`) - Or simply use for internal tracking/validation - Result: Plain text files can reference page numbers for citations #### Benefits 1. **Traceability:** Every paragraph can be traced back to its original source page 2. **Citation Support:** Readers can cite specific passages with page numbers (e.g., "as stated on page 2, paragraph 3") 3. **Quality Control:** Easier to identify and fix transcription errors by knowing page context 4. **Workflow Flexibility:** Supports filtering, sorting, and querying by page number 5. **Archive Metadata:** Preserves essential document structure metadata for preservation systems #### Example Workflow
INPUT: Document with 6 pages
Phase 1a: Extract metadata (page_number = 1)
Phase 1b: Create <p pid="p-01" page_nr="1">...</p> for all paragraphs on page 1
[Repeat for pages 2-6]
Phase 2: Consolidate
Page 1 paragraphs: <p pid="p-01" page_nr="1">, <p pid="p-02" page_nr="1">, ...
Page 2 paragraphs: <p pid="p-03" page_nr="2">, <p pid="p-04" page_nr="2">, ...
→ All page_nr attributes preserved in consolidated output.es.xml
Phase 3: Translate to Italian
<p pid="p-01" page_nr="1">Oggi, otto giorni fa...</p> (page_nr preserved)
Phase 4: Export to text
[Page 1] Oggi, otto giorni fa...
[Page 2] (translated content)
<!-- Phase 1 (Single-page XML) -->
<p pid="p-01" page_nr="1" verified="true">
Text content...
</p>
<!-- Phase 2 (Consolidated XML) -->
<p pid="p-01" page_nr="1">
Text content from page 1...
</p>
<!-- Phase 3 (Translated XML) -->
<p pid="p-01" page_nr="1" source_text="Original text...">
Translated text...
</p>
import xml.etree.ElementTree as ET
# Create structure
root = ET.Element('root')
meta = ET.SubElement(root, 'meta')
title = ET.SubElement(meta, 'title')
title.text = 'Carta IV'
# Write to file
tree = ET.ElementTree(root)
tree.write('output.1.xml', encoding='utf-8', xml_declaration=True)
# Read and modify
tree = ET.parse('output.1.xml')
root = tree.getroot()
body = ET.SubElement(root, 'body')
INPUT: Archivo de Cartas (folder with 6 page images)
PHASE 1a (Page 1): Extract metadata
→ output.1.es.xml (with <meta> block)
PHASE 1b (Page 1): Extract body text
→ output.1.es.xml (adds <body> with 8 <p> blocks)
PHASE 1c (Page 1): Verify paragraphs
→ output.1.es.xml (marks each <p> verified="true/false")
[Repeat for pages 2-6]
RESULT: 6 XML files, each with complete page structure
- output.1.es.xml through output.6.es.xml
- Ready for consolidation into doc.es.txt
SCANNING_CONFIG = {
"phase_1a_enabled": True, # Extract metadata
"phase_1b_enabled": True, # Extract body text
"phase_1c_enabled": True, # Verify transcriptions
"target_language": "es", # Spanish
"line_grouping_mode": "strict", # strict | lenient
"max_retries_per_phase": 2,
"save_intermediate_xml": True,
}
</details>