Tencent Open-Sources WeChat End-to-End Document Parsing Model WeVisDoc: Direct Markdown Output from a Single Image

Tencent's WeChat Vision Team has open-sourced WeVisDoc in 2B and 4B versions. Give it an image of a document page, and it directly outputs full-page Markdown, with formulas in LaTeX and tables in HTML.

Key WeVisDoc Features

• Convert a full-page image directly to Markdown: The model processes the entire page in one pass, without first cropping titles, body text, tables, and formulas into separate regions.
• Preserve complex document structure: Body text becomes regular Markdown, formulas become LaTeX, tables become HTML, and the reading order is reconstructed from the page.
• Use one model for the whole parse: Layout understanding, text recognition, formula recognition, and table reconstruction are handled by one vision-language model, with no separate layout detector or OCR engine.
• Choose between 2B and 4B versions: The 2B model suits constrained resources and mostly clean pages; the 4B model has a clearer advantage on photographed and recaptured pages.
• Run locally or as a service: The Apache-2.0 weights and inference code support local Transformers inference or a vLLM service.

End-to-End Parsing and WeVisDoc's Positioning

When building knowledge base retrieval (RAG), automated report analysis, and multimodal workflows, converting PDFs and scans with complex layouts into LLM-friendly Markdown is an inescapable fundamental step. Currently, the industry pursues two starkly different engineering paradigms:

One is pipeline systems (Pipeline-based), represented by solutions such as MinerU and PaddleOCR. Such systems decompose the task into multiple stages: first using an object detection model to identify layout regions (distinguishing titles, body text, figures, tables, formulas), then dispatching cropped image patches to different specialized OCR engines, table reconstruction tools, and LaTeX formula recognition models, and finally stitching them into the final reading order relying on rules or heuristic algorithms. This modular architecture allows specialized models at each step, but at the cost of a long pipeline, requiring the deployment and maintenance of several models; once layout segmentation fails in an earlier step, downstream recognition inevitably compounds the error.

The other is end-to-end models (End-to-End). An end-to-end model feeds the entire document image directly into a vision-language model (VLM), enabling the model to output full-page Markdown, HTML tables, and LaTeX symbols all at once through autoregressive decoding. This approach eliminates intermediate stitching stages and requires deploying only a single model; whether recognition is accurate depends entirely on the model's own capability.

In the end-to-end specialized model category, WeVisDoc-4B achieved leading positions on OmniDocBench v1.6 as well as across the clean, digitally degraded, and real-world captured tracks of PureDocBench (Figure: WeVisDoc technical report)

WeVisDoc adopts a purely end-to-end route. The models directly adopt Qwen3-VL-2B-Instruct and Qwen3-VL-4B-Instruct as backbones, without introducing any external detectors or specialized decoding architectures.

Evaluations focus on two benchmarks. OmniDocBench v1.6 contains 1,651 natural document pages spanning different document types, layouts, and languages. PureDocBench takes 1,475 source pages and produces clean, digitally degraded, and real-capture versions of each, using the same ground truth to show how image quality changes performance.

Both benchmarks report character/text edit distance TextEdit (lower is better), formula score FormulaCDM, and table structural similarity TableTEDS, computing the overall score as Overall = (100 × (1 − TextEdit) + FormulaCDM + TableTEDS) ÷ 3; reading order edit distance ROEdit is reported separately and not included in Overall. PureDocBench's three-track average is the mean of the Overall scores across the three tracks.

In a comprehensive evaluation encompassing general large models, pipeline systems, and end-to-end models, WeVisDoc-4B's rankings across different scopes are as follows (Clean, Digital, and Real are PureDocBench's three tracks; "All Models" refers to the three groups combined from the paper's table):

The two benchmarks rank systems differently. Two pipeline systems score above WeVisDoc-4B on the natural-document OmniDocBench. On PureDocBench, WeVisDoc-4B ranks first for the three-track average, clean pages, and digitally degraded pages, while ranking third on real captures. Which result matters more depends on whether your documents resemble clear but diverse natural pages or degraded scans, photographs, and screen recaptures.

Sorting by sub-metrics from Table 3 of the paper reveals that across all three tracks of PureDocBench, WeVisDoc-4B's plain text recognition and reading order are not first in the end-to-end group; its first-place overall score stems from ranking first in both formulas and tables. The Overall score is an unweighted average of text, formulas, and tables, while reading order is reported separately and excluded. Consequently, for text-centric documents (contracts, reports, books), one should inspect the text edit distance metric rather than relying solely on the overall score.

WeVisDoc-4B's sub-metric rankings within the PureDocBench end-to-end group (21 models):

The pattern shifts on OmniDocBench: Within the end-to-end group, its text edit distance (0.036), formulas, and reading order are all ranked 1st, but table TableTEDS at 92.95 ranks 3rd, trailing the 1B-parameter HunyuanOCR-1.5 (93.67) and its own 2B sibling (93.03).

The 4B model does not improve every dimension by a large margin: on OmniDocBench it is only 0.64 points ahead of the previous best end-to-end model. The 2B model is already close on most clean inputs, while the 4B advantage grows as image degradation becomes more severe. Try 2B first for clean digital pages, and favor 4B when photographs and recaptures are common.

Two-Stage Training Mechanism: From Broad Coverage to Residual-Based Rebalancing

The technical report attributes the primary challenge to training data: existing training corpora skew heavily toward common document types and clean digital pages, and simply expanding data coverage fails to inform how to remedy a model's remaining deficiencies. Rather than altering model architecture, WeVisDoc splits training into two stages: Stage I builds broad geometric and semantic coverage, while Stage II implements targeted refinement based on error diagnostics.

Stage I data construction covers source balancing, multi-model joint annotation, executable webpage synthesis, and multi-path appearance degradation simulation (Figure: WeVisDoc technical report)

Stage I: Building Broad Support with 40 Million Records

The core objective of the first stage is to broaden the model's foundational coverage across semantics, typographical layout structures, and appearance degradations, drawing from a training pool containing approximately 40 million records. Both the vision encoder (ViT) and the language model (LLM) backbone are updated during this stage.

Training data originates from four channels: open-source datasets (existing annotations for articles, books, forms, receipts, tables, formulas), internal collections (supplementing production-environment documents and capture conditions rare in benchmarks), targeted web and PDF scrapes (rare types such as textbooks, newspapers, handwriting, multilingual mixtures), and synthetic pages. Diverse degraded versions of the same page share a deduplication group, counted only once when computing coverage rather than treated as novel content. In terms of language, it covers Simplified Chinese, English, Traditional Chinese, and multilingual mixtures, though the paper does not break down scores by language.

Training granularity is not restricted to full pages. Prior to sampling, roughly 40% of records are full pages, 25% are regional crops of plain text or mixed content, 25% are isolated tables, and 10% are isolated formulas. The division of labor across granularities is explicit: full pages teach reading order and relationships between elements; text crops ensure recognition resolution for fine, dense characters; and standalone table and formula crops specifically drill structural syntax.

Faced with disparate public and proprietary sources, the training process incorporates three critical mechanisms:

Budgeting by target tokens: In autoregressive parsing tasks, supervisory signals vary wildly across samples. A densely laid out academic paper page may contain thousands of target tokens, whereas a cropped standalone formula patch contains only dozens of tokens. If sampling ratios were determined by sample counts, the actual training allocation across data categories would diverge: sampling 100 long pages yields far more output tokens than 100 formula crops. The team therefore calculates training allocation based on the number of output tokens participating in loss computation. They first establish the desired token share for each data category, then convert this into sampling probabilities based on each category's average output length—categories with shorter outputs must be sampled more frequently to attain their allotted token share. Loss is likewise averaged per token rather than computed per sample and then averaged.

Three-model cross-annotation and tiered verification: For real-world pages lacking precise annotations, the system concurrently runs three industry expert models—MinerU2.5-Pro, PaddleOCR-VL-1.6, and dots.mocr—to generate candidate parsing results. When character edit distance differences among the three fall below a preset threshold, the consensus result is adopted directly as clean annotation. When two models agree while one diverges, image verification is triggered. If all three diverge, the page does not enter Stage I directly; instead, it is routed to higher-tier multimodal LLMs and human review pipelines to serve as candidate hard examples for Stage II. Notably, MinerU2.5-Pro and PaddleOCR-VL-1.6 are the two pipeline systems that scored higher than WeVisDoc-4B on OmniDocBench; the report does not discuss the potential implications of this setup on results.

Dual-compiled executable webpage synthesis: To supply extreme layouts rare in the wild (such as nested tables and multi-column formula derivations), the system employs executable web program generation. A single program undergoes two compilation pipelines simultaneously: one side renders high-resolution webpage images, while the other traverses the identical Document Object Model (DOM) tree to generate unambiguous Markdown, HTML tables, and LaTeX source code. Both image and ground truth originate from the same source code, ensuring ground truth precision independent of human or model annotation.

Original version (left) and aged, yellowed texture overlay version (right) of the same page of handwritten English mathematics notes, expanding the visual feature space while preserving text legibility (Figure: WeVisDoc technical report)

To ensure the model is not confined to standard digital high-definition PDFs, synthetic clean pages and collected real-world pages are further fed into a degradation pipeline. The pipeline simulates two categories of effects: paper and propagation processes, including paper texture and aging, stains, ink fading or bleeding, halftone dots and streaks from printing/copying/scanning, and downscaling/compression during transmission; and physical camera capture, including perspective distortion, paper curvature and creases, uneven illumination, moiré patterns from screen photography, and motion or depth-of-field blur. Only when all target content remains fully visible and legible does the degraded image retain its original annotation for training; illegible instances are either discarded or cropped to retain only legible portions with re-matched annotations.

Diagnosing Residual Errors: How Stage II Allocates Training Budget

Following Stage I training, the model acquires general page parsing capabilities, yet continues to exhibit systematic errors across specific page types. The report notes that residual errors (the errors remaining after Stage I) do not inherently signal a data deficit: failures can also stem from erroneous annotations, illegible images, repetitive decoding loops, or fundamental model capacity ceilings where adding similar samples yields no benefit. Stage II therefore first diagnoses root causes before deciding whether to supplement data or train more.

Stage II of WeVisDoc constructs a refinement pool of approximately 5 million records and freezes the vision encoder, fine-tuning only the language model backbone. The core rationale of this stage is to measure residual deficiencies from Stage I and tilt computational resources accordingly.

Clustered feature cluster of colorful textbooks: containing section borders, example blocks, embedded figures, and exercise areas, exhibiting highly consistent visual and structural patterns (Figure: WeVisDoc technical report)

The complete diagnosis and rebalancing workflow unfolds across four steps:

Independent probe set residual measurement: An independent "probe set" is established, completely excluded from training, hard-example mining, and final benchmark sets. Checkpoints obtained from Stage I are run across this probe set. While full-page edit distance incorporates tables and formulas, localized parsing failures exert minimal impact on full-page scores (for instance, on a photographed contract where all surrounding text is recognized correctly but the formula is omitted entirely, full-page edit distance hardly reflects an issue). Consequently, diagnostics compute text, tables, and formulas separately—text uses edit distance, tables use 1 − TEDS, and formulas use 1 − CDM—surfacing localized errors otherwise masked by full-page aggregates.

Visual-structural feature clustering: A frozen SigLIP2 encoder extracts global visual representations of pages. Following dimensionality reduction on visual features, structural information—element composition, content density, language, layout, acquisition condition, and target output length—is concatenated for joint clustering, partitioning the data space into a fixed number of feature clusters (e.g., "handwritten notes cluster," "colorful textbooks cluster"). For clusters with sparse samples, error estimates are shrunk toward the global mean (the fewer the samples, the closer to the average), avoiding distortion from outlier samples; if the probe set contains zero samples for a cluster, the global mean serves as a prior rather than treating it as an empirically verified weakness.

Root-cause audit of high-error samples: Cluster priority is computed via a weighted combination of data scarcity, overall residual error, and the most prominent individual residual among text, tables, and formulas. This ranking serves solely to schedule human review rather than automatically injecting training data: adding new data requires auditing to confirm the error pattern and verifying that existing data is indeed insufficient; clusters with ample data where the model still errs are assigned to train more. High-error samples are also screened for three other failure modes: annotations with missing content, inverted order, or corrupt formatting are corrected first; outputs trapped in repetitive decoding loops are marked as decoding failures; and samples illegible due to cropping, blur, occlusion, or low resolution are excluded from hard-example training, reserved only for robustness evaluation. Hard examples undergo further verification by humans and stronger multimodal models; only when the model clearly fails despite a correct ground truth does it qualify as a true hard example.

Bounded token budget reallocation: Once residual severity per cluster is determined, Stage II training token proportions are adjusted based on error magnitude, allocating more tokens to higher-error clusters. To prevent drastic distribution swings, each cluster's new proportion is constrained within a bounded multiplier range relative to its natural share (specific upper and lower bound multipliers are undisclosed in the report); simultaneously, high-quality Stage I data is replayed so the model is less likely to forget what it has already learned.

Figure 8 from the paper: Gray dashed line represents before reallocation, green solid line represents after reallocation; three panels show page element types, layout and structural difficulty, and appearance and acquisition conditions, with each panel normalized to an average of 1.0 (Figure: WeVisDoc technical report)

Examining the reallocation curves, the total volume under both curves remains identical (the token budget for this stage is fixed). The overarching effect shifts training volume from dominant, common types to rarer types, pulling the overall distribution toward uniformity: Regarding page elements, plain text blocks originally stood at about 1.5 times the mean and mixed text-image layouts at about 1.3 times; both decrease to around 1.0 post-reallocation. Handwritten annotations, cross-element nesting, and multilingual text originally stood at only about 0.6–0.8, rising to near 1.0 post-reallocation. Regarding layout, linear single-column originally stood at about 1.5 times and regular multi-column at about 1.3 times, declining to around 1.0; nested/spanning row-column structures, non-linear reading order, and freeform layouts rose from about 0.7–0.8 to near 1.0. Regarding appearance, standard print scans originally stood at about 1.7 times, declining to around 1.0, while blur, perspective/curvature, and uneven lighting increased slightly.

Targeted synthetic dense formula derivation page generated for Stage I formula parsing residuals (Figure: WeVisDoc technical report)

For clusters confirmed by audit to suffer from genuinely deficient data coverage and model failure (examples cited in the report include Chinese mathematical text, nested tables, and cross-region reading order relationships), the data pipeline uses controllable web generation programs to synthesize targeted data. For clusters with sufficient coverage where the model has yet to master the pattern, the exposure frequency of existing high-quality samples is increased in Stage II.

It must be emphasized that the overall improvement delivered by Stage II is the outcome of a composite intervention. In experimental evaluations, Stage II incorporates cumulative training steps, residual weight reallocation, high-quality old data replay, true hard-example mining, and targeted new data synthesis. While this comparison demonstrates that the refinement pipeline yields gains, the score increase cannot be attributed exclusively to "residual-based reallocation."

Where the Score Gains Fall, and 12 Recovery Cases

Because PureDocBench's three tracks utilize the identical underlying pages across varying degradation levels, they reveal where performance shifts concentrate: whether scaling model size or introducing Stage II, gains manifest primarily on degraded pages. The table below lists all figures from Table 4 of the report (Overall scores, averaged across three inferences):

When expanding the model from 2B to 4B, performance improves by 0.45 points on clean native pages (Clean), 1.12 points on algorithmically degraded pages (Digital), and 3.48 points on real-world captured pages (Real). The report infers from this that more severe appearance variations yield greater returns from increased model capacity. From a model selection perspective, 2B and 4B differ by only 0.32 points on OmniDocBench and 0.45 points on Clean, with the primary gap residing in the 3.48-point difference on Real. The project homepage recommends starting with the smaller 2B, stepping up to 4B when seeking peak accuracy. Thus, if a production workload consists predominantly of digital, clean pages, 2B is largely sufficient; if photographs and screen captures are prevalent, 4B is recommended.

Meanwhile, progressing from Stage I to Stage II, the 4B model gains only 0.49 points on Clean, but achieves leaps of 2.55 points on Digital and 4.03 points on Real. This aligns with Stage II's design intent of targeting rare types and camera capture degradations; nevertheless, as noted above, this comparison cannot isolate the impact of any individual measure.

The project homepage and report appendix showcase 12 before-and-after comparison cases, with 3 from each test set. Selection methodology: scanning Stage II single-page scores from high to low, retaining pages where Stage I scored below 70 and exhibited a single, distinct failure cause; displayed outputs represent the median-scoring run across three inferences. Consequently, they showcase typical successful recoveries, and the report explicitly cautions that they cannot be used to estimate how frequently such errors occur.

Missed tables and boundary recovery: In a book page featuring a 9×9 Sudoku grid (Case 1), Stage I missed the entire table, scoring just 0.333 on TableTEDS; Stage II reconstructed the complete <table> matrix, bringing the score to 1.000. In a consolidated operating report (Case 8), Stage I extended table cells downward into the footnotes, audit notes, and signature block; Stage II identified the correct table boundary, restoring the lower annotations as standard body paragraphs.

Reading order correction: In an event handbook's airport pickup schedule (Case 5) consisting of three parallel columns of flight cards (CA1501, MU5101, NH919...), Stage I read vertically down the first column before moving to the next, yielding an output sequence of CA1501, KE897, BA169, UA857 and scrambling the timeline. Stage II read row-wise from left to right, restoring the proper sequence of CA1501, MU5101, NH919, KE897. Full-page Overall jumped from 49.00 to 94.64, and text edit distance dropped from 0.510 to 0.054.

Formula recovery: In a factory maintenance and crew management manual (Case 7), a downtime risk formula was flattened in Stage I into plain text Downtime Risk = Fault Frequency × Recovery Time, yielding a FormulaCDM of 0; Stage II formatted it as a display LaTeX block wrapped in $$. In a diagonally photographed Chinese Equity Transfer Agreement (Case 11), Stage I committed two errors simultaneously: misrecognizing Party B's executive partner 「林嘉铭」(Lin Jiaming) as 「林露铭」(Lin Luming), and outputting Clause 7's liquidated damages formula as plain text. Stage II corrected both, raising FormulaCDM from 0.024 to 1.000.

Hallucinations and residual errors under physical capture: On a photographed newspaper page (Case 10), Stage I drifted onto an unrelated document, fabricating irrelevant Chinese corporate cash management notices and repetitive mathematical derivations, dropping the page score to 31.43. Stage II suppressed the hallucination, restoring text to the newspaper's actual Federal Reserve rate decision report (Overall rebounded to 94.13).

The report also notes that in several real-world capture cases, Stage II restored overall structure while character- and digit-level recognition errors persisted. For example, in the clinical form case (Case 12), Stage II misread the cause-of-death hierarchy levels 「1a / 1b / 1c」 as 「la / lb / Ic」, mistaking the digit 1 for letters. Structural recovery does not guarantee that every character is recognized correctly.

12 Cases · Comparison of Original Pages and Two-Stage Outputs

Where Stage I Failed and How Stage II Corrected It

Select a case: first examine the red boxes on the original page, then compare the outputs of the two stages below. The ellipsis (…) in outputs is original from the project homepage.

Case 01 · OmniDocBench v1.6

Missed Table

Stage I missed the 9×9 Sudoku grid; Stage II completely reconstructed the table.

Overall 65.87 → 99.22TableTEDS 0.333 → 1.000

Stage I Output

Fig. 10.3 Solved sudoku puzzle.

## 10.4 Hybrid Optimization
Hybrid methods may be required to solve particularly difficult real-world optimization problems. Implementation of hybrid methods typically requires non-trivial scripting
…

Stage II Output

<table><tr><td>5</td><td>3</td><td>4</td><td>6</td><td>7</td><td>8</td><td>9</td><td>1</td><td>2</td></tr>
…
<tr><td>3</td><td>4</td><td>5</td><td>2</td><td>8</td><td>6</td><td>1</td><td>7</td><td>9</td></tr></table>

Fig. 10.3 Solved sudoku puzzle.

### 10.4 Hybrid Optimization
…

Case 02 · OmniDocBench v1.6

Missed Diagram Hierarchy

Stage I missed an intermediate hierarchy level in the diagram; Stage II restored it.

Overall 58.98 → 100.00TextEdit 0.410 → 0.000 (lower is better)

Stage I Output

## 语气的概述

语气可以分为陈述语气、祈使语气和虚拟语气。虚拟语气又有七种基本结构。

## 一 陈述语气
陈述语气表示所说的话是陈述事实、提出想法。
…

Stage II Output

## 语气的概述

语气可以分为陈述语气、祈使语气和虚拟语气。虚拟语气又有七种基本结构。

陈述语气
祈使语气
语气
动词原形
动词的过去式
had + 过去分词
虚拟语气
should + 动词原形
…
## 一 陈述语气
…

Case 03 · OmniDocBench v1.6

Missed Marginalia

Stage I missed the page-level figure title, footer, and page number; Stage II restored them.

Overall 67.66 → 100.00TextEdit 0.323 → 0.000 (lower is better)

Stage I Output

## 企业依赖复杂、多层和互通的架构
案例:半导体、计算机和电子产品以及通讯设备

## 戴尔
2019年收入：900亿美金
…
## 联想
2019年收入：510亿美金
…
资料来源:彭博供应链数据库；麦肯锡全球研究所分析

Stage II Output

## 图E3 即使在同一个行业中，企业也可能有非常不同的供应链结构和显著的重叠

企业依赖复杂、多层和互通的架构
案例:半导体、计算机和电子产品以及通讯设备

戴尔
2019年收入：900亿美金
…
联想
2019年收入：510亿美金
…
资料来源:彭博供应链数据库；麦肯锡全球研究所分析

全球价值链中的风险、韧性和再平衡
9

Case 04 · PureDocBench · Clean

Heading Hierarchy Flattened into Table

Stage I compressed the entire page's heading hierarchy into a single table; Stage II restored the headings and formatted only genuine tables as tables.

Overall 3.46 → 98.30TextEdit 0.996 → 0.033 (lower is better)

Stage I Output

<table><tr><td>PART A — MOTOR VEHICLE EMISSION AND FUEL STANDARDS</td></tr><tr><td>§7521. Emission standards for new motor vehicles or new motor vehicle engines</td></tr>
…
<tr><td>Table 1 — National Ambient Air Quality Standards (NAAQS)</td></tr><tr><td>Pollutant</td><td>Primary Standard(Level)</td><td>Secondary Standard(Level)</td><td>Averaging Time</td><td>Form</td></tr>
…
</table>

Stage II Output

## PART A — MOTOR VEHICLE EMISSION AND FUEL STANDARDS

## §7521. Emission standards for new motor vehicles or new motor vehicle engines

(a) Authority of Administrator to prescribe by regulation.— Except as otherwise provided in subsection (b), the Administrator shall by regulation prescribe
…
## §7409. National Ambient Air Quality Standards
…
Table 1 — National Ambient Air Quality Standards (NAAQS)
<table><tr><td>Pollutant</td><td>Primary Standard (Level)</td><td>Secondary Standard (Level)</td><td>Averaging Time</td><td>Form</td></tr>
…

Case 05 · PureDocBench · Clean

Reading Order

Stage I read the three columns of airport pickup cards vertically column-by-column; Stage II restored the row-wise left-to-right order.

Overall 49.00 → 94.64TextEdit 0.510 → 0.054 (lower is better)

Stage I Output

…
## Airport Pickup Matrix 接机安排 (Apr 19-20)

CA1501 PEK → PVG
Apr 19 14:20 | 8 pax | Shuttle A1

KE897 ICN → PVG
Apr 19 17:10 | 8 pax | Shuttle B1

BA169 LHR → PVG
Apr 20 06:15 | 15 pax | Shuttle C1

UA857 SFO → PVG
Apr 20 07:45 | 14 pax | Shuttle C2
…

Stage II Output

…
## Airport Pickup Matrix 接机安排 (Apr 19-20)

CA1501 PEK → PVG
Apr 19 14:20 | 8 pax | Shuttle A1

MU5101 CAN → PVG
Apr 19 15:45 | 6 pax | Shuttle A1

NH919 NRT → PVG
Apr 19 16:35 | 12 pax | Shuttle B1

KE897 ICN → PVG
Apr 19 17:10 | 8 pax | Shuttle B1
…

Case 06 · PureDocBench · Clean

Missed Measurement Tables

Stage I missed two carotid artery blood flow measurement tables; Stage II restored them.

Overall 48.88 → 91.96TableTEDS 0.366 → 0.901

Stage I Output

…
左侧颈动脉 Left Carotid

斑块分析 Plaque Analysis (Left)
狭窄计算 Stenosis Calculation
<table><tr><td>编号</td><td>位置</td><td>大小mm</td><td>回声</td><td>表面</td><td>GSM</td><td>溃疡</td></tr>
…
</table>
…
右侧颈动脉 Right Carotid ★

斑块分析 Plaque An
<table><tr><td>编号</td><td>位置</td><td>大小mm</td><td>回声</td><td>表面</td><td>GSM</td><td>溃疡</td></tr>
…

Stage II Output

…
## 左侧颈动脉 Left Carotid
<table><tr><th>血管节段</th><th>IMT mm</th><th>ID mm</th><th>PSV cm/s</th><th>EDV cm/s</th><th>RI</th></tr><tr><td>CCA近段</td><td>0.8</td><td>6.4</td><td>76.5</td><td>21.2</td><td>0.72</td></tr>
…
<tr><td>VA V3段</td><td>-</td><td>3.2</td><td>44.8</td><td>15.2</td><td>0.66</td></tr></table>

斑块分析 Plaque Analysis (Left)
…

Case 07 · PureDocBench · Digital Degraded

Formula Rendered as Plain Text

Stage I output display formulas as plain text; Stage II formatted them as LaTeX formula blocks.

Overall 65.28 → 98.70FormulaCDM 0.000 → 1.000

Stage I Output

…
## 5.2 LOTO 与 Permit-to-Work

执行检修、换模、开盖、进入防护罩内作业前，必须进行 Lockout / Tagout。
…
Downtime Risk

Downtime Risk = Fault Frequency × Recovery Time

用于决定异常响应是否升级为 maintenance escalation 或生产会议通报项。

## 5.3 班组 KPI 与 Toolbox Meeting
…

Stage II Output

…
### 5.2 LOTO 与 Permit-to-Work

执行检修、换模、开盖、进入防护罩内作业前，必须进行 Lockout / Tagout。
…
#### Downtime Risk

$$Downtime\ Risk = Fault\ Frequency \times Recovery\ Time$$

用于决定异常响应是否升级为 maintenance escalation 或生产会议通报项。

### 5.3 班组 KPI 与 Toolbox Meeting
…

Case 08 · PureDocBench · Digital Degraded

Table Boundary Overshoot

Stage I extended the table all the way into the notes and signature block below; Stage II terminated the table at the correct boundary.

Overall 51.76 → 98.59TextEdit 0.862 → 0.027 (lower is better)

Stage I Output

…
<tr><td>Diluted EPSNote 2</td><td>$3.28</td><td>$2.54</td><td>$1.25</td><td>+29.1%</td></tr>
…
<tr><td colspan="5">NOTE: This consolidated statement of operations reflects combined performance across all operating segments including:
…
<tr><td colspan="5">AUDITOR&#x27;S CERTIFICATION: This Consolidated Statement of Operations, including all notes and supplementary schedules, has been audited …</td></tr><tr><td>Chief Financial Officer</td><td>Controller</td><td></td><td>Chief Executive Officer</td><td></td></tr>
…

Stage II Output

…
<tr><td>Diluted EPSNote 2</td><td>$3.28</td><td>$2.54</td><td>$1.25</td><td>+29.1%</td></tr>
…
</table>

NOTE: This consolidated statement of operations reflects combined performance across all operating segments including:
…
AUDITOR'S CERTIFICATION: This Consolidated Statement of Operations, including all notes and supplementary schedules, has been audited
…
Chief Financial Officer
…
Controller
…
Chief Executive Officer
…

Case 09 · PureDocBench · Digital Degraded

Fields and Values Splintered

Stage I separated firmware information field names from their values; Stage II re-paired them one by one.

Overall 57.25 → 92.47TextEdit 0.428 → 0.075 (lower is better)

Stage I Output

Device Model 设备型号:
Current FW 当前固件:
Target FW 目标固件:
Build Date 构建日期:
Architecture 架构:
Image Size 镜像大小:

GW-IoT-4200 Edge Gateway
v4.1.8 (build 20250911)
v4.2.1 (build 20260401)
2026-04-01 08:30:00 UTC
ARMv8-A Cortex-A72
48.7 MB
…

Stage II Output

Device Model 设备型号: GW-IoT-4200 Edge Gateway
Current FW 当前固件: v4.1.8 (build 20250911)
Target FW 目标固件: v4.2.1 (build 20260401)
Build Date 构建日期: 2026-04-01 08:30:00 UTC
Architecture 架构: ARMv8-A Cortex-A72
Image Size 镜像大小: 48.7 MB

## [NEW] 新增功能 / New Features
…

Case 10 · PureDocBench · Real Degraded

Output Drifts to Unrelated Document

Stage I hallucinated irrelevant Chinese public notices and repetitive mathematical derivations; Stage II recovered the actual newspaper content, though local recognition errors remain.

Overall 31.43 → 94.13TextEdit 0.681 → 0.057 (lower is better)

Stage I Output

## THE FINANCIAL CHRONICLE
…
Vol. 02171, No. 69
…
S&P 500 3,738.92 +42.18 (+0.75%)
…
1. 2017年，公司与上海浦东发展银行股份有限公司签订了《关于使用部分闲置募集资金进行现金管理的协议》。
…
The Ground Truth image displays a single, solid horizontal line. According to Rule 2 (UNDERSCORE & LINE RULES), this is a stylistic or background line, not a placeholder underscore.
…
$\therefore m = \frac{3}{11}$
…

Stage II Output

## THE FINANCIAL CHRONICLE
Vol CXIV, No 20
Sunday, March 17, 2025
New York London Hong Kong Tokyo Frankfurt
DJIA 42.186.57 • 287.41 (+0.65%)
S&P 500 3,738.92 +42.18 (+0.74%)
…
## Fed Holds Rates Steady, Signals Two Cuts Remain on Table for 2025
Powell cites "unusually elevated uncertainty" from tariff policies; dot plot unchanged from December projections
BY JAMES WHITFIELD AND SARAH CHEH | FEDERAL RESERVE CORRESPONDENTS
WASHINGTON — The Federal Reserve held its benchmark interest rate unchanged at 4.25%-4.50% on Wednesday, as widely expected
…

Case 11 · PureDocBench · Real Degraded

Misrecognized Name and Formula Flattened to Text

Stage I misrecognized 「林嘉铭」(Lin Jiaming) as 「林露铭」(Lin Luming) and rendered the penalty formula as plain text; Stage II corrected both.

Overall 64.61 → 97.70FormulaCDM 0.024 → 1.000

Stage I Output

…
第一条 各方基本信息
<table>
…
<tr><td>乙方(受让方)</td><td colspan="3">深圳前海汇智投资合伙企业(有限合伙)</td></tr><tr><td>统一社会信用代码</td><td>91440300MA5FP7TN8K</td><td>执行事务合伙人</td><td>林露铭</td></tr>
…
</table>
…
## 第七条 违约责任

7.1 任何一方违反本协议项下的义务、陈述或保证的，…

违约金 = 股权转让总对价 × 10% + 逾期付款金额 × 0.05% × 逾期天数
…

Stage II Output

…
第一条 各方基本信息
<table>
…
<tr><td>乙方(受让方)</td><td colspan="3">深圳前海汇智投资合伙企业(有限合伙)</td></tr><tr><td>统一社会信用代码</td><td>91440300MA5FP7TN8K</td><td>执行事务合伙人</td><td>林嘉铭</td></tr>
…
</table>
…
## 第七条 违约责任

7.1 任何一方违反本协议项下的义务、陈述或保证的，…

$$
\text{违约金} = \text{股权转让总对价} \times 10 \% + \text{逾期付款金额} \times 0.05 \% \times \text{逾期天数}
$$
…

Case 12 · PureDocBench · Real Degraded

Form Over-Formatted into Table

Stage I forced the clinical form into a table; Stage II restored it to a section-by-section linear structure, though local recognition errors remain (such as misreading 「1a」 as 「la」).

Overall 67.08 → 96.66TextEdit 0.648 → 0.056 (lower is better)

Stage I Output

…
二、直接死亡原因链 Cause of Death Chain
<table><tr><td>1a 直接原因</td><td>感染性休克、多器官功能衰竭(MOF) Septic shock with multiple organ failure</td></tr><tr><td></td><td>↑ 由于 due to</td></tr><tr><td>1b 前因</td><td>重症肺炎(多重耐药菌感染) Severe pneumonia …</td></tr>
…
</table>

三、讨论发言记录
<table><tr><td>汇报医师:李明 住院医师</td></tr>
…

Stage II Output

…
## 二、直接死亡原因链 Cause of Death Chain

la 直接原因 感染性休克、多器官功能衰竭(MOF) …
↑ 由于 due to
lb 前因 重症肺炎(多重耐药菌感染) …
↑ 由于 due to
Ic 根因 社区获得性肺炎，免疫功能低下 …
II 促进因素 2型糖尿病(20年，HbA1c 9.8%) …
…
## 三、讨论发言记录
汇报医师：李明 住院医师
…
感染科会诊意见：王教授 主任医师
…
ICU主治：陈副主任医师
…

Data and images are from the WeVisDoc project homepage 'Interactive Recovery Lab' and technical report Appendix B.1; displayed outputs represent the median-scoring run across three inferences.

Getting Started with Deployment and Practical Usage Advice

Both the GitHub repository and Hugging Face model cards indicate an Apache-2.0 license, open-sourcing weights and inference code for both 2B and 4B models. Training data and data construction pipelines are not open-sourced; while the report provides formulaic expressions for mixing ratios, key coefficients (source balance exponent ρ, residual weight β, bounding multipliers ϑ, hard-example ratio λ, etc.) lack disclosed numerical values, withholding the full parameters required for external replication.

Runtime Environment and Launch Methods

The official recommendation is to use vLLM (version ≥ 0.11.1) for serving, supporting multi-concurrency, Tensor Parallel acceleration, and OpenAI-compatible API protocols. Python 3.10 or higher is required. Because vLLM serving and local Transformers inference may conflict in PyTorch dependency versions, official guidance recommends separate virtual environments; client machines issuing requests require only a lightweight dependencies file.

Example server launch command:

# Launch 2B or 4B service after activating the environment
bash scripts/serve_vllm.sh Tencent/WeVisDoc-4B

For multi-GPU setups handling ultra-long single pages, two-GPU tensor parallelism and expanded sequence length can be enabled via environment variables:

CUDA_VISIBLE_DEVICES=0,1 TENSOR_PARALLEL_SIZE=2 MAX_MODEL_LEN=65536 \
  bash scripts/serve_vllm.sh Tencent/WeVisDoc-4B --dtype bfloat16

Client-side batch processing supports multi-threaded concurrent execution over a directory of images:

# Batch process all page images in specified directory, saving results page by page in Markdown format
python -m wevisdoc.client --image-dir ./doc_images --result-dir ./outputs --workers 4

For lightweight local single-GPU testing, an independent Transformers inference script is also provided:

python -m wevisdoc.local --model Tencent/WeVisDoc-2B --image page.png --output result.md

Four Things to Handle Before Integrating It

• Convert PDFs to images first: WeVisDoc accepts PNG, JPEG, and WebP, not PDF files directly. Render multi-page PDFs page by page before sending the images to the model.
• Handle chart content separately: Text becomes Markdown, formulas use LaTeX, and tables use HTML; standalone illustrations and charts are ignored. Add a chart-analysis step if you need trends or labels from figures.
• Leave enough output space for dense pages: The client defaults to 8,192 generated tokens. Dense papers, legal documents, or large tables may be truncated; increase --max-tokens when memory allows, or reduce image size and concurrency.
• Check critical fields against the source image: Handwriting, historical scans, and severely damaged pages have not been evaluated thoroughly. Preserve page references and verify contract amounts, financial figures, and medical fields against the originals.

The project does not publish GPU, VRAM, or speed requirements. Its output also omits text coordinates and does not automatically join paragraphs or tables across pages; workflows that need source highlighting or cross-page reconstruction must add those capabilities separately.