本研究檢驗大型語言模型在異質訂單文件之關鍵資訊擷取與標準化任務上的可行性, 並透過分層實驗設計, 系統性比較不同文件複雜度與不同輸入型態下的表現差異, 同時針對主要錯誤型態及其改善方式進行更深入的剖析︒為使各實驗層級之輸出能在一致的基準下被比較, 本研究定義了一套統一的結構化 schema, 涵蓋訂單編號︑訂單日期︑幣別︑總金額, 以及品項明細中的料號︑數量︑單位︑單價與單項金額等欄位︒實驗體系由三個層級構成︒Level 1 以 50 份簡單純文字訂單驗證大型語言模型在低複雜度條件下的基本可行性;Level 2 則進一步以 250 份複雜純文字訂單檢驗模型在高異質文字文件情境下的穩定性, 並依複雜性的來源細分為欄位名稱變異︑資訊分散︑複雜明細︑干擾資訊與混合複雜性五個類別;Level 3 則以 Level 2 的訂單內容為母體, 將其轉製為影像形式, 並分別建構 OCR-based 與 Vision Direct 兩條處理路線, 藉此比較影像型文件的不同處理策略︒實驗結果顯示, 在簡單純文字訂單情境下,baseline 版本於 Level 1 取得 100.0% 的document accuracy, 意味著大型語言模型能穩定完成所有目標欄位的擷取與標準化︒在複雜純文字訂單情境中,Level 2 baseline 的 document accuracy 為 95.2%, 顯示即便模型必須同時面對欄位命名變異︑資訊分散與複雜明細等異質條件, 仍能維持相當穩定的整體
表現︒進一步分析可發現,Level 2 的主要錯誤並非來自欄位辨識失敗, 而是高度集中於po_date 的日期格式歧義︒換言之, 模型其實能正確辨識出日期資訊, 但在將其標準化為統一格式時, 仍可能因為 dd/mm/yyyy 與mm/dd/yyyy 兩種月日順序之間的歧義而出錯︒針對這項問題, 本研究進一步設計日期標準化支線實驗︒結果顯示, 僅依賴文件內部日期線索的保守修正策略無法有效改善結果; 相較之下, 固定 mm/dd/yyyy 的預設策略雖然改善幅度有限, 卻不會引入額外的誤修正, 意味著對格式敏感欄位來說, 後處理機制與風險控制其實同等重要︒當文件由純文字轉為影像之後,OCR-based 路線的表現出現明顯下降,clean baseline與 v1 baseline 的 document accuracy 分別為 86.8% 與 82.8%︒與純文字情境相比, 錯誤來源已不再侷限於日期欄位, 而進一步擴展至 po_number︑part_number 與部分item-level 欄位, 反映出 OCR 所引入的文字失真與結構破壞會直接影響下游的擷取結果︒針對其中最具代表性的 OCR 字元混淆問題, 本研究進一步設計欄位導向的後處理支線, 對po_number 與 part_number 施加大寫正規化與 pattern-based correction︒結果顯示,clean 條件下的 document accuracy 由 86.8% 推升至 93.6%,v1 條件下也由 82.8% 推升至 93.2%︒值得特別注意的是,po_number 在兩種條件下均被推到 100.0%, 且全程未觀察到明顯的誤修正, 意味著 OCR-based pipeline 中的錯誤傳遞仍可透過輕量級後處理獲得有效緩解︒相較之下,Vision Direct 路線的表現相當低落,clean baseline_vision 與 v1baseline_vision 分別僅為 1.2% 與 0.8%, 顯示在本研究的設定下, 直接將影像送入多模態大型語言模型, 幾乎無法支撐結構化訂單擷取任務的需求︒綜合來看, 本研究確認大型語言模型在異質純文字訂單文件上具備相當高的可行性;在影像文件情境下,OCR-based pipeline 雖然受前處理品質影響而出現表現衰退, 但仍保有一定的應用價值, 而代表性錯誤更可透過欄位導向後處理顯著降低; 至於 Vision Direct路線, 則突顯出當前多模態模型在細粒度 item-level 結構化擷取上的明顯限制︒基於這些發現,OCR 在現階段異質訂單文件處理流程中仍扮演關鍵中介角色, 而針對格式敏感欄位以及 OCR 脆弱欄位所設計的後處理與驗證機制, 也是未來提升實務可用性的重要方向︒

This study examines the feasibility of applying large language models (LLMs) to key information extraction and standardization in heterogeneous order documents. Through a layered experimental design, the study systematically compares model performance across different document complexities and input modalities, and further analyzes the major error patterns together with their potential remediation strategies. To ensure that the outputs of all experimental levels can be compared on a consistent basis, a unified structured schema is defined, covering the purchase order number, order date, currency, and total amount, as well as the part number, quantity, unit, unit price, and line amount at the item level. The experimental framework is organized into three progressive levels. Level 1 employs 50 simple plain-text orders to verify the basic feasibility of LLMs under low-complexity conditions. Level 2 extends the evaluation to 250 complex plain-text orders, examining model stability in highly heterogeneous textual settings, and partitions the data into five categories according to the source of complexity: field-name variation, scattered fields, complex items, noise information, and mixed complexity. Level 3 then takes the Level 2 orders as the source corpus, renders them into image documents, and constructs two parallel processing routes—OCR-based and Vision Direct—to contrast different strategies for handling image-based documents. The results show that, under the simple plain-text setting, the baseline configuration at Level 1 achieves a document accuracy of 100.0%, indicating that LLMs can stably complete the extraction and standardization of all target fields. In the complex plain-text setting, the Level 2 baseline reaches 95.2% document accuracy, demonstrating that the model retains a high level of overall stability even when confronted with heterogeneous conditions such as field-name variation, scattered information, and complex item expressions. A closer analysis reveals that the main errors at Level 2 do not stem from field-recognition failure; rather, they are highly concentrated in the date-format ambiguity of the po_date field. In other words, the model is in fact able to recognize the date information correctly, yet may still fail when normalizing it into a unified format due to the inherent ambiguity between dd/mm/yyyy and mm/dd/ yyyy. To address this issue, a dedicated date-standardization branch experiment was designed. The results indicate that conservative correction strategies relying solely on internal document cues do not yield meaningful improvement; in contrast, adopting a fixed mm/dd/yyyy default strategy brings only a modest gain but introduces no additional overcorrection. This finding suggests that, for format-sensitive fields, post-processing mechanisms and risk control are of equal importance. When the input modality shifts from plain text to images, the OCR-based route exhibits a clear performance drop, with the clean and v1 baselines achieving document accuracies of 86.8% and 82.8%, respectively. Compared with the plain-text setting, the error sources are no longer confined to the date field but extend further to po_number, part_number, and several item-level fields, reflecting how OCR-induced textual distortion and structural degradation directly affect downstream extraction. To tackle the most representative form of these errors—OCR character confusion—this study designs a field-oriented post-processing branch that applies uppercase normalization and pattern-based correction to po_number and part_number. The results show that document accuracy is lifted from 86.8% to 93.6% under the clean condition, and from 82.8% to 93.2% under the v1 condition. Notably, po_number reaches 100.0% accuracy in both conditions, with no observable overcorrection, indicating that the error propagation within the OCR-based pipeline can be effectively mitigated through lightweight post-processing. By contrast, the Vision Direct route performs poorly, with clean baseline_vision and v1 baseline_vision yielding only 1.2% and 0.8% document accuracy, respectively. This suggests that, under the experimental settings of this study, feeding images directly into a multimodal LLM is still far from sufficient to support the structured order extraction task. Taken together, the findings of this study confirm that LLMs offer a high degree of feasibility for heterogeneous plain-text order documents. In the image-based setting, although the OCR-based pipeline suffers from performance degradation caused by preprocessing quality, it still retains practical value, and its representative errors can be substantially reduced through field-oriented post-processing. The Vision Direct route, on the other hand, highlights the noticeable limitations of current multimodal models in fine-grained item-level structured extraction. On the basis of these observations, OCR continues to play a critical intermediary role in the current pipeline of heterogeneous order document processing, while post-processing and validation mechanisms tailored to format-sensitive fields and OCR-vulnerable fields constitute an important direction for further improving practical usability.

[1] Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R. Manmatha. Docformer: End-to-end transformer for document understanding. arXiv preprint arXiv:2106.11539, 2021.

[2] Yuning Du, Chenxia Li, Ruoyu Guo, et al. PP-OCR: A practical ultra lightweight OCR system. arXiv preprint arXiv:2009.09941, 2020.

[3] Yunshi Hong et al. An empirical study on information extraction using large language models. arXiv preprint arXiv:2305.14450, 2023.

[4] Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. LayoutLMv3: Pre-training for document AI with unified text and image masking. In Proceedings of the 30th ACM International Conference on Multimedia, pages 4083–4091, 2022.

[5] Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. OCR-free document understanding transformer. In Computer Vision – ECCV 2022, pages 498–517. Springer, 2022.

[6] Kenton Lee, Mandar Joshi, Iulia Turc, Hexiang Hu, Fangyu Liu, Julian Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. Pix2Struct: Screenshot parsing as pretraining for visual language understanding. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 18893–18912, 2023.

[7] Minghao Li, Tengchao Lv, Jingye Chen, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, and Furu Wei. TrOCR: Transformer-based optical character recognition with pre-trained models. Proceedings of the AAAI Conference on Artificial Intelligence, 37(11):13094–13102, 2023.

[8] OpenAI. GPT-4o mini model. https://developers.openai.com/api/docs/models/gpt-4o-mini, 2024. Accessed: 2026-04-22.

[9] Bo Peng et al. LayoutLLM: Layout instruction tuning with large language models for document understanding. arXiv preprint arXiv:2404.05225, 2024.

[10] Zhaoqing Tang, Benyou Wang, Fangfang Su, Yuxin Luo, Yuxiao Jin, Lin Ma, Kehai Chen, and Min Zhang. Unifying vision, text, and layout for universal document processing. arXiv preprint arXiv:2212.02623, 2023.

[11] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017.

[12] Zhijie Wang et al. Large language models for generative information extraction: A survey. arXiv preprint arXiv:2312.17617, 2023.

[13] Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. LayoutLM: Pre-training of text and layout for document image understanding. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1192–1200, 2020.

[14] Hao Yang et al. DocGraphLM: Documental graph language model for information extraction from visually-rich documents. arXiv preprint arXiv:2401.02823, 2024.