publications

2024

1. 

   Medical Unlearnable Examples: Securing Medical Data from Unauthorized Traning via Sparsity-Aware Local Masking

   Weixiang Sun, Yixin Liu, Zhiling Yan, Kaidi Xu, and Lichao Sun

   ICML 2024 NextGenAISafety, 2024

   Abs arXiv PDF

   With the rapid growth of artificial intelligence (AI) in healthcare, there has been a significant increase in the generation and storage of sensitive medical data. This abundance of data, in turn, has propelled the advancement of medical AI technologies. However, concerns about unauthorized data exploitation, such as training commercial AI models, often deter researchers from making their invaluable datasets publicly available. In response to the need to protect this hard-to-collect data while still encouraging medical institutions to share it, one promising solution is to introduce imperceptible noise into the data. This method aims to safeguard the data against unauthorized training by inducing degradation in model generalization. Although existing methods have shown commendable data protection capabilities in general domains, they tend to fall short when applied to biomedical data, mainly due to their failure to account for the sparse nature of medical images. To address this problem, we propose the Sparsity-Aware Local Masking (SALM) method, a novel approach that selectively perturbs significant pixel regions rather than the entire image as previous strategies have done. This simple-yet-effective approach significantly reduces the perturbation search space by concentrating on local regions, thereby improving both the efficiency and effectiveness of data protection for biomedical datasets characterized by sparse features. Besides, we have demonstrated that SALM maintains the essential characteristics of the data, ensuring its clinical utility remains uncompromised. Our extensive experiments across various datasets and model architectures demonstrate that SALM effectively prevents unauthorized training of deep-learning models and outperforms previous state-of-the-art data protection methods.

2. 

   Fine-tuning Text-to-Video Diffusion Transformers via Image Guidance

   Zhengqing Yuan, Ruoxi Chen, Yang Wang, Weixiang Sun, Yanfang Ye, and Lichao Sun

   arxiv preprint, 2024

   Abs

   Customized text-to-video generation, which seeks to produce text-directed videos incorporating specific user-defined subjects, has recently seen a surge in research interest. In this paper, we pioneered to use diffusion transformers for customized video generation. We have adapted DreamBooth to fine-tune text-to-video models effectively, utilizing videos that have been transformed from images by image-to-video models. Our proposed method, VideoAdapter, makes efficient use of a few reference images to capture and faithfully reproduce the essential visual characteristics of the subject. Extensive experiments have verified the performance of \ours in both quantitatively and qualitatively, demonstrating its capability to achieve high visual fidelity in dynamic video content generation.

3. 

   Bora: Biomedical Generalist Video Generation Model

   Weixiang Sun, Xiaocao You, Ruizhe Zheng, Zhengqing Yuan, Xiang Li, Lifang He, Quanzheng Li, and Lichao Sun

   arXiv preprint arXiv:2312.16862, 2024

   Abs arXiv PDF Website

   Generative models hold promise for revolutionizing medical education, robot-assisted surgery, and data augmentation for medical AI development. Diffusion models can now generate realistic images from text prompts, while recent advancements have demonstrated their ability to create diverse, high-quality videos. However, these models often struggle with generating accurate representations of medical procedures and detailed anatomical structures. This paper introduces Bora, the first spatio-temporal diffusion probabilistic model designed for text-guided biomedical video generation. Bora leverages Transformer architecture and is pre-trained on general-purpose video generation tasks. It is fine-tuned through model alignment and instruction tuning using a newly established medical video corpus, which includes paired text-video data from various biomedical fields. To the best of our knowledge, this is the first attempt to establish such a comprehensive annotated biomedical video dataset. Bora is capable of generating high-quality video data across four distinct biomedical domains, adhering to medical expert standards and demonstrating consistency and diversity. This generalist video generative model holds significant potential for enhancing medical consultation and decision-making, particularly in resource-limited settings. Additionally, Bora could pave the way for immersive medical training and procedure planning. Extensive experiments on distinct medical modalities such as endoscopy, ultrasound, MRI, and cell tracking validate the effectiveness of our model in understanding biomedical instructions and its superior performance across subjects compared to state-of-the-art generation models.

4. 

   Biomedical sam 2: Segment anything in biomedical images and videos

   Zhiling Yan, Weixiang Sun, Rong Zhou, Zhengqing Yuan, Kai Zhang, Yiwei Li, Tianming Liu, Quanzheng Li, Xiang Li, Lifang He, and 1 more author

   arXiv preprint arXiv:2408.03286, 2024

   Abs arXiv

   Medical image segmentation and video object segmentation are essential for diagnosing and analyzing diseases by identifying and measuring biological structures. Recent advances in natural domain have been driven by foundation models like the Segment Anything Model 2 (SAM-2). To explore the performance of SAM-2 in biomedical applications, we designed three evaluation pipelines for single-frame 2D image segmentation, multi-frame 3D image segmentation and multi-frame video segmentation with varied prompt designs, revealing SAM-2’s limitations in medical contexts. Consequently, we developed BioSAM-2, an enhanced foundation model optimized for biomedical data based on SAM-2. Our experiments show that BioSAM-2 not only surpasses the performance of existing state-of-the-art foundation models but also matches or even exceeds specialist models, demonstrating its efficacy and potential in the medical domain.

2023

1. 

   Research and application of improved neural network optimization algorithm

   Weixiang Sun

   In Third International Symposium on Computer Engineering and Intelligent Communications (ISCEIC 2022), 2023

   Abs PDF

   In order to improve the computational efficiency and accuracy of the neural network algorithm, this research establishes an optimized neural network algorithm. Firstly, the optimal training function and the optimal number of hidden layer nodes of the neural network are obtained by using the empirical formula method; secondly, the prediction accuracy of the neural network is optimized, and the science of the neural network is improved. Finally, with the network as the core algorithm, the quantitative adjustment rate is used as the weight coefficient to improve the evaluation and calculation of the impact factor MIV.