| 研究生: |
楊幸怡 Yang, Xing-Yi |
|---|---|
| 論文名稱: |
基於視覺化知識圖譜的影像報告智慧檢索與生成 An Approach Based on Visualized Knowledge Graphs for Intelligent Retrieval and Generation of Imaging Reports |
| 指導教授: |
羅崇銘
李沛錞 |
| 口試委員: | 陳淑君 |
| 學位類別: |
碩士
Master |
| 系所名稱: |
文學院 - 圖書資訊與檔案學研究所 Graduate Institute of Library, Information and Archival Studies |
| 論文出版年: | 2026 |
| 畢業學年度: | 115 |
| 語文別: | 中文 |
| 論文頁數: | 60 |
| 中文關鍵詞: | 知識圖譜 、詮釋資料 、超音波影像 、觀察者變異 |
| 外文關鍵詞: | Knowledge graph, Metadata, Ultrasound imaging, Observer variability |
| 相關次數: | 點閱:11 下載:0 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
許多專業領域的報告撰寫皆涉及量測數值、判讀規則與文字敘述三者之間的轉換,但現行資訊系統多僅支援結果記錄,較少將量測值與判讀規則系統化組織為可檢索、可推理與可視覺化的知識結構。知識圖譜(Knowledge Graph, KG)提供了一種以節點與關聯呈現複雜語意關係的途徑,若能結合量化指標與領域規則,將有機會成為跨領域報告生成與決策支援的共同基礎。
本研究提出一個結合量測數值與判讀規則的視覺化知識圖譜框架,並探討其與檢索增強生成(Retrieval-Augmented Generation, RAG)整合後,支援報告檢索與文字生成的可行性。以頸動脈都卜勒超音波為應用場域,透過Neo4j建構知識圖譜,將醫師、血管位置、血流量測指標與狹窄等級之語意關聯結構化,並以兩階段XGBoost分類器之leaf proximity(葉節點相似度)進行相似案例檢索,此相似度定義與觀察者內變異分析採用一致之方法,分類模組採兩階段XGBoost架構,先區分正常與異常,再將異常樣本細分為四個狹窄等級,生成模組以本機部署之大型語言模型透過RAG,以結構化的圖譜脈絡產生具診斷語意格式之報告文字。
蒐集2019年至2024年間臺北榮民總醫院神經內科頸動脈都卜勒超音波影像報告,經匿名化與診斷結果標準化篩選後,從507位患者報告中,取出4,411筆血管量測紀錄納入分析。研究設計涵蓋四項評估面向:醫師診斷行為變異,以個別醫師模型之Kappa差距量化觀察者間診斷行為差異,並以偽陽性案例之leaf proximity分析相似案例在觀察者內標記狹窄的穩定性;分類效能之最佳模型配置,透過特徵群組與類別權重兩階段消融實驗確認;系統操作效率,以各端點回應時間評估;生成內容對檢索脈絡的依附程度,以RAGAS介面忠實性評估。
實驗結果顯示,將六位醫師個別模型的交叉驗證預測結果合併後,最終整體加權Kappa為0.655,依Landis and Koch (1977)標準屬良好一致性,在排除異常案例過少而Kappa較無法參考的醫師後,五位醫師個別訓練模型之Kappa介於0.299至0.821之間,顯示醫師間診斷行為存在差異。觀察者內變異分析顯示,18件偽陽性案例中77.8%(14件)之相似案例,同一位醫師自己亦標記為異常,顯示此類誤判部分源自同一觀察者標記前後不一致,而非完全由分類器邊界模糊造成。系統在相似案例檢索整體中位數回應時間為0.0498秒,在較具可比性的Excel結構化資料人工查詢情境中,系統相似案例檢索端點的時間差距約為723倍,無結構化歷史報告查找平均耗時為74.6秒,顯示結構化知識組織對查詢效率的潛在價值。整體效能瓶頸集中於本地端大型語言模型推論,受評估規模所限,忠實性框架驗證僅得介面忠實性均值0.769,屬概念驗證層次之依附性證據,完整評估有待後續研究擴大規模執行。上述結果支持以視覺化知識圖譜整合影像報告檢索與生成之診斷輔助框架具有初步可行性,可為進行頸動脈影像報告判讀的醫師,提供客觀之參考依據。
Report writing across many professional domains involves translating among quantitative measurements, interpretive rules, and narrative text, yet existing information systems mostly support only outcome recording and rarely organize measurement values and interpretive rules into a structure that is queryable, inferable, and visualizable. Knowledge Graphs(KG) offer a means of representing complex semantic relationships through nodes and relations; when combined with quantitative indicators and domain-specific rules, KGs may serve as a common foundation for cross-domain report generation and decision support.
This study proposes a visualized knowledge graph framework that integrates quantitative measurements with interpretive rules, and examines the feasibility of coupling it with Retrieval-Augmented Generation(RAG) to support report retrieval and text generation. Using carotid duplex ultrasonography as the application domain, a knowledge graph was constructed in Neo4j to structure the semantic relationships among doctors, vessel positions, hemodynamic measurement indicators, and stenosis grades. Similar-case retrieval is performed using leaf proximity derived from a two-stage XGBoost classifier, with the same similarity definition applied consistently in the intra-observer variability analysis. The classification module adopts a two-stage XGBoost architecture that first distinguishes normal from abnormal cases and then further subdivides abnormal samples into four stenosis grades. The generation module employs a locally deployed large language model(LLM) that, driven by RAG, produces report text with diagnostic semantic formatting based on structured graph context.
Carotid duplex ultrasound reports from the Department of Neurology at Taipei Veterans General Hospital, collected between 2019 and 2024, were used as the data source. After anonymization and standardization of diagnostic outcomes, 4,411 vessel-position measurement records from 507 patients were included in the analysis. The study design covers four evaluation dimensions: (1) variability in doctors' diagnostic behavior, quantified through the Kappa discrepancy among individual doctor models, together with a leaf-proximity analysis of similar cases among false-positive instances to assess the stability of intra-observer stenosis labeling; (2) the optimal model configuration for classification performance, determined through two-stage ablation experiments on feature groups and class weighting; (3) system operational efficiency, evaluated via endpoint response time; and (4) the degree to which generated content adheres to the retrieved context, assessed using RAGAS interface faithfulness.
Experimental results show that after pooling the cross-validated predictions of the six doctor-specific models, the overall weighted Kappa reached 0.655, indicating substantial agreement according to the criteria of Landis and Koch (1977). After excluding doctor, whose Kappa was not meaningfully interpretable due to an insufficient number of abnormal cases, the Kappa values of the remaining five individually trained doctor models ranged from 0.299 to 0.821, indicating variability in diagnostic behavior across doctors. The intra-observer variability analysis showed that among 18 false-positive cases, 77.8% (14 cases) had similar cases that the same doctor had also labeled as abnormal, suggesting that such misclassifications partly stem from inconsistent labeling by the same observer over time rather than being entirely attributable to ambiguity at the classifier's decision boundary. The system achieved an overall median response time of 0.0498 seconds for similar-case retrieval; compared with a more directly comparable manual query scenario using structured Excel data, the retrieval endpoint was approximately 723 times faster, while manually searching unstructured historical reports took an average of 74.6 seconds, highlighting the potential value of structured knowledge organization for query efficiency. The primary performance bottleneck lies in local LLM inference. Constrained by the limited evaluation scale, the faithfulness framework validation yielded a mean interface faithfulness score of 0.769, which should be regarded as proof-of-concept evidence of contextual adherence, with full-scale evaluation left to future research. These findings support the preliminary feasibility of a diagnostic support framework that integrates a visualized knowledge graph with report retrieval and generation, offering doctors an objective reference for interpreting carotid ultrasound reports.
謝辭 i
摘要 ii
Abstract iv
目次 vii
圖目錄 ix
表目錄 x
第一章 緒論 1
第一節 研究背景與動機 1
第二節 研究目的與研究問題 4
壹、 研究目的 4
貳、 研究問題 6
第三節 研究範圍與限制 6
壹、 研究範圍 6
貳、 研究限制 7
第二章 文獻探討 8
第三章 研究材料與方法 14
第一節 資料蒐集 14
第二節 視覺化知識圖譜 16
壹、 知識圖譜的節點 17
貳、 知識圖譜的關聯 19
第三節 診斷分類模型建構 23
第四節 影像報告智慧檢索 24
第五節 系統整合架構 25
壹、 三層式服務架構 26
貳、 圖譜增強式生成流程 26
參、 系統介面 27
第六節 評估方法 29
壹、 狹窄等級分類效能評估 30
貳、 消融實驗設計 32
參、 醫師診斷行為變異評估方法 33
肆、 回應時間評估 35
伍、 生成內容忠實性初步驗證 36
第四章 實驗結果 38
第一節 消融實驗結果分析 38
壹、 消融實驗結果 38
貳、 最終模型效能 40
第二節 醫師診斷行為變異分析 41
壹、 個別模型整體效能比較 42
貳、 偽陽性案例之相似案例分析 42
第三節 回應時間評估 45
壹、 測試環境與方法 45
貳、 回應時間評估結果 45
第四節 生成內容忠實性初步驗證 48
第五章 結論與展望 49
第一節 研究結論 49
第二節 研究貢獻 52
第三節 研究限制 53
第四節 未來展望 54
參考文獻 56
American Heart Association. (2025). Global cardiovascular disease statistics: Southeast Asia, East Asia, and Oceania.
Benkortbi, K., Catarino, R., Wisniak, A., Kenfack, B., Foguem, E. T., Venegas, G., Mulindi, M., Horo, A., Jeronimo, J., Vassilakos, P., & Petignat, P. (2023). Inter- and intra-observer agreement in the assessment of the cervical transformation zone (TZ) by visual inspection with acetic acid (VIA) and its implications for a screen and treat approach: a reliability study [Article]. Bmc Womens Health, 23(1), 8, Article 27. https://doi.org/10.1186/s12905-022-02131-z
Chávez, A. I. R., Su, W. C., Han, M. Z., & Lu, Z. P. (2024). Study on Knowledge Graph of Red Cultural Resources from the Perspective of Art Works on the Theme of Cuban Red Revolution History [Article; Early Access]. Libri-International Journal of Libraries and Information Studies, 18. https://doi.org/10.1515/libri-2023-0106
Chen, A. H., Rudin, R. S., Levine, D. M., & Mehrotra, A. (2025). Improving patient understanding of radiology reports using generative artificial intelligence: a vignette study of 2000 US adults [Article; Early Access]. Journal of the American Medical Informatics Association, 10. https://doi.org/10.1093/jamia/ocaf187
Chen, C. M., Witt, B., & Lin, C. Y. (2024). A knowledge graph analysis tool of people and organizations to facilitate digital humanities research [Article; Early Access]. Data Technologies and Applications, 29. https://doi.org/10.1108/dta-01-2024-0009
Deng, P. H., Fu, Y., Chen, M. S., Wang, D., & Si, L. (2023). Temporal trends in inequalities of the burden of cardiovascular disease across 186 countries and territories [Article]. International Journal for Equity in Health, 22(1), 13, Article 164. https://doi.org/10.1186/s12939-023-01988-2
Es, S., James, J., Anke, L. E., & Schockaert, S. (2024). Ragas: Automated evaluation of retrieval augmented generation. Proceedings of the 18th conference of the european chapter of the association for computational linguistics: system demonstrations,
Fareedi, A. A., Ismail, M., Gagnon, S., Ghazanweh, A., & Arooj, Z. (2025). Digital Health Transformation: Leveraging a Knowledge Graph Reasoning Framework and Conversational Agents for Enhanced Knowledge Management [Article]. Systems, 13(2), 38, Article 72. https://doi.org/10.3390/systems13020072
Gao, J., Peng, P., Lu, F., Claramunt, C., Qiu, P., & Xu, Y. (2024). Mining tourist preferences and decision support via tourism-oriented knowledge graph. Information Processing & Management, 61(1), 103523. https://doi.org/https://doi.org/10.1016/j.ipm.2023.103523
Gornik, H. L., Rundek, T., Gardener, H., Benenati, J. F., Dahiya, N., Hamburg, N. M., Kupinski, A. M., Leers, S. A., Lilly, M. P., Lohr, J. M., Pellerito, J. S., Rholl, K. S., Vickery, M. A., Hutchisson, M. S., & Needleman, L. (2021). Optimization of duplex velocity criteria for diagnosis of internal carotid artery (ICA) stenosis: A report of the Intersocietal Accreditation Commission (IAC) Vascular Testing Division Carotid Diagnostic Criteria Committee. Vasc Med, 26(5), 515-525. https://doi.org/10.1177/1358863X211011253
Grant, E. G., Benson, C. B., Moneta, G. L., Alexandrov, A. V., Baker, J. D., Bluth, E. I., Carroll, B. A., Eliasziw, M., Gocke, J., & Hertzberg, B. S. (2003). Carotid artery stenosis: gray-scale and Doppler US diagnosis—Society of Radiologists in Ultrasound Consensus Conference. Radiology, 229(2), 340-346.
Haendel, M. A., Chute, C. G., & Robinson, P. N. (2018). Classification, Ontology, and Precision Medicine [Review]. New England Journal of Medicine, 379(15), 1452-1462. https://doi.org/10.1056/NEJMra1615014
He, Q., Yu, C. Y., Song, W., Jiang, X. Y., Song, L. L., & Wang, J. (2023). ISLKG: The Construction of Island Knowledge Graph and Knowledge Reasoning [Article]. Sustainability, 15(17), 26, Article 13189. https://doi.org/10.3390/su151713189
Huang, J. X., Liang, J. Q., Yang, M. S., & Li, Y. (2023). Visual Preference Analysis and Planning Responses Based on Street View Images: A Case Study of Gulangyu Island, China [Article]. Land, 12(1), 15, Article 129. https://doi.org/10.3390/land12010129
Iqbal, Q., Ahmad, N. H., & Nawaz, R. (2020). Perceived information pollution: conceptualization, measurement, and nomological validity. Online Information Review, 44(3), 705-722. https://doi.org/10.1108/OIR-10-2018-0322
Kamsu-Foguem, B., Tchuenté-Foguem, G., & Foguem, C. (2014). Using conceptual graphs for clinical guidelines representation and knowledge visualization [Article]. Information Systems Frontiers, 16(4), 571-589. https://doi.org/10.1007/s10796-012-9360-2
Kim, J., & Sohn, M. (2022). Graph Representation Learning-Based Early Depression Detection Framework in Smart Home Environments [Article]. Sensors, 22(4), 19, Article 1545. https://doi.org/10.3390/s22041545
Landis, J. R., & Koch, G. G. (1977). MEASUREMENT OF OBSERVER AGREEMENT FOR CATEGORICAL DATA [Article]. Biometrics, 33(1), 159-174. https://doi.org/10.2307/2529310
Li, L. F., Wang, P., Yan, J., Wang, Y., Li, S. M., Jiang, J. P., Sun, Z., Tang, B. Z., Chang, T. H., Wang, S. H., & Liu, Y. T. (2020). Real-world data medical knowledge graph: construction and applications [Article]. Artificial Intelligence in Medicine, 103, 10, Article 101817. https://doi.org/10.1016/j.artmed.2020.101817
Li, W. W., Wang, S. Z., Wu, S., Gu, Z. N., & Tian, Y. Y. (2022). Performance benchmark on semantic web repositories for spatially explicit knowledge graph applications [Article]. Computers Environment and Urban Systems, 98, 14, Article 101884. https://doi.org/10.1016/j.compenvurbsys.2022.101884
Liu, X., Shen, P., Xu, X. Z., Meng, L., Ni, J., Lin, G. L., & Jiao, F. P. (2025). The study of blended teaching based on knowledge graph: the case of clinical microbiology examination course [Article; Early Access]. Education and Information Technologies, 23. https://doi.org/10.1007/s10639-025-13626-1
Liu, Z. H., Qian, Y. X., Lv, W. L., Fang, Y. B., & Liu, S. L. (2024). Multi-feature fusion stock prediction based on knowledge graph. Electronic Library, 42(3), 455-482. https://doi.org/10.1108/el-02-2023-0053
Liu, Z. Y., Gao, X. M., & Li, C. Y. (2022). Modeling COVID-19 Vaccine Adverse Effects with a Visualized Knowledge Graph Database [Article]. Healthcare, 10(8), 14, Article 1419. https://doi.org/10.3390/healthcare10081419
Lu, Y. C., Chen, P. J., Lu, S. N., Liang, F. W., & Chuang, H. Y. (2024). Comparing Carotid Artery Velocities with Current ASCVD Risk Stratification: A Novel Approach to Simpler Risk Assessment [Article]. Journal of Epidemiology and Global Health, 14(4), 1569-1578. https://doi.org/10.1007/s44197-024-00308-3
Lu, Y. H., Tong, X. Y., Xiong, X., & Zhu, H. (2024). Knowledge graph enhanced citation recommendation model for patent examiners [Article]. Scientometrics, 129(4), 2181-2203. https://doi.org/10.1007/s11192-024-04966-9
Ma, B. W., Zhang, C. Z., Wang, Y. Z., & Deng, S. H. (2022). Enhancing identification of structure function of academic articles using contextual information [Article]. Scientometrics, 127(2), 885-925. https://doi.org/10.1007/s11192-021-04225-1
Ma, J. T., Li, K. L., Zhang, F., Wang, Y. J., Luo, X. Y., Li, C. L., & Qiao, Y. Q. (2024). TaReT: Temporal knowledge graph reasoning based on topology-aware dynamic relation graph and temporal fusion. Information Processing & Management, 61(6), Article 103848. https://doi.org/10.1016/j.ipm.2024.103848
Nakao, Y., Ishino, A., Kokubu, K., & Okada, H. (2024). Exploring visual communication in corporate sustainability reporting: Using image recognition with deep learning [Article]. Corporate Social Responsibility and Environmental Management, 31(4), 3210-3234. https://doi.org/10.1002/csr.2735
Nguyen, H. H., Saarakkala, S., Blaschko, M. B., & Tiulpin, A. (2020). Semixup: In- and Out-of-Manifold Regularization for Deep Semi-Supervised Knee Osteoarthritis Severity Grading From Plain Radiographs [Article]. Ieee Transactions on Medical Imaging, 39(12), 4346-4356. https://doi.org/10.1109/tmi.2020.3017007
Nukovic, J. J., Opancina, V., Ciceri, E., Muto, M., Zdravkovic, N., Altin, A., Altaysoy, P., Kastelic, R., Velazquez Mendivil, D. M., Nukovic, J. A., Markovic, N. V., Opancina, M., Prodanovic, T., Nukovic, M., Kostic, J., & Prodanovic, N. (2023). Neuroimaging Modalities Used for Ischemic Stroke Diagnosis and Monitoring. Medicina (Kaunas), 59(11). https://doi.org/10.3390/medicina59111908
Qin, Z. K., Wu, D. Z., Zang, Z. D., Chen, X. L., Zhang, H. F., & Wong, C. U. I. (2025). Building an intelligent diabetes Q&A system with knowledge graphs and large language models [Article]. Frontiers in Public Health, 13, 17, Article 1540946. https://doi.org/10.3389/fpubh.2025.1540946
Sanjak, J., Binder, J., Yadaw, A. S., Zhu, Q., & Mathé, E. A. (2024). Clustering rare diseases within an ontology-enriched knowledge graph [Article]. Journal of the American Medical Informatics Association, 31(1), 154-164. https://doi.org/10.1093/jamia/ocad186
Sebro, R. A., & Kahn, C. E., Jr. (2023). Automated detection of causal relationships among diseases and imaging findings in textual radiology reports [Article]. Journal of the American Medical Informatics Association, 30(10), 1701-1706. https://doi.org/10.1093/jamia/ocad119
Shang, B., Zhao, Y. L., & Liu, J. (2024). Knowledge graph representation learning with relation-guided aggregation and interaction [Article]. Information Processing & Management, 61(4), 15, Article 103752. https://doi.org/10.1016/j.ipm.2024.103752
Sun, H. Q., Xu, H. Z., Wu, J., Sun, S. L., & Wang, S. Y. (2024). Let pictures speak: hotel selection-recommendation method with cognitive image attribute-enhanced knowledge graphs [Article; Early Access]. International Journal of Contemporary Hospitality Management, 23. https://doi.org/10.1108/ijchm-12-2023-1849
Tahmasebi, A. M., Zhu, H. H., Mankovich, G., Prinsen, P., Klassen, P., Pilato, S., van Ommering, R., Patel, P., Gunn, M. L., & Chang, P. (2019). Automatic Normalization of Anatomical Phrases in Radiology Reports Using Unsupervised Learning [Article]. Journal of Digital Imaging, 32(1), 6-18. https://doi.org/10.1007/s10278-018-0116-5
Tamjid, S., Nooshinfard, F., Beheshti, M. S. H., Hariri, N., & Babalhavaeji, F. (2024). Semi-automated ontology development scheme via text mining of scientific records [Article]. Electronic Library, 42(2), 230-254. https://doi.org/10.1108/el-06-2023-0165
Uyar, A., & Aliyu, F. M. (2015). Evaluating search features of Google Knowledge Graph and Bing Satori Entity types, list searches and query interfaces [Article]. Online Information Review, 39(2), 197-213. https://doi.org/10.1108/oir-10-2014-0257
World Health Organization. (2024). Cardiovascular diseases. Retrieved 2024/08/15 from https://www.who.int/health-topics/cardiovascular-diseases#tab=tab_1
World Heart Federation. (2023). World Heart Report 2023. W. H. Federation.
Yim, W. W., Kwan, S. W., & Yetisgen, M. (2017). Classifying tumor event attributes in radiology reports [Article]. Journal of the Association for Information Science and Technology, 68(11), 2662-2674. https://doi.org/10.1002/asi.23937
Zhang, F., Luo, Y. F., Gao, Z. H., & Han, A. H. (2025). Injury degree appraisal of large language model based on retrieval-augmented generation and deep learning [Article]. International Journal of Law and Psychiatry, 100, 14, Article 102070. https://doi.org/10.1016/j.ijlp.2025.102070
全文公開日期 2031/07/23