General – contact@radisentech.com
Sales – sales@radisentech.com
Customer Service – cs@radisentech.com

Partnership – partner@radisentech.com
Investment – invest@radisentech.com

개인정보처리방침

ⓒ 2024 RadiSen Inc. All rights reserved. 

Pretraining Vision-Language Model for Difference Visual Question Answering in Longitudinal Chest X-rays

Published

Medical Imaging with Deep Learning (2024)

Authors

Yeongjae Cho1, Taehee Kim2, Heejun Shin2, Sungzoon Cho1, and Dongmyung Shin2

Affiliations

1Seoul National University
2Artificial Intelligence Engineering Division, Radisen Co. Ltd

Abstract

Difference visual question answering (diff-VQA) is a challenging task that requires answering complex questions based on differences between a pair of images. This task is particularly important in reading chest X-ray images because radiologists often compare multiple images of the same patient taken at different times to track disease progression and changes in its severity in their clinical practice. However, previous works focused on designing specific network architectures for the diff-VQA task, missing opportunities to enhance the model’s performance using a pretrained vision-language model (VLM). Here, we introduce a novel VLM called PLURAL, which is pretrained on natural and longitudinal chest X-ray data for the diff-VQA task. The model is developed using a step-by-step approach, starting with being pretrained on natural images and texts, followed by being trained using longitudinal chest X-ray data. The longitudinal data consist of pairs of X-ray images, along with question-answer sets and radiologist’s reports that describe the changes in lung abnormalities and diseases over time. Our experimental results show that the PLURAL model outperforms state-of-the-art methods not only in diff-VQA for longitudinal X-rays but also in conventional VQA for a single X-ray image. Through extensive experiments, we demonstrate the effectiveness of the proposed VLM architecture and pretraining method in improving the model’s performance. Our code is available at: https://github.com/yjch00/PLURAL

Share:

1. Purpose of collecting and using personal information

Sub-One Co. Ltd. collects personal information according to items of personal information collected to receive customer counseling and process inquiries.


2. Items of personal information collected
– Required: Email
– Selections: Company name, mobile phone number


3. Retention and period of use of personal information
Sub-One Co. Ltd. shall be destroyed after storing the information for 3 years after the purpose of personal information collection and use has been achieved.

 

Provided, That if it is necessary to preserve it under the provisions of the relevant statutes, member information shall be kept for a certain period of time prescribed by the relevant statutes.