- Research
- Open access
- Published:
Development and evaluation of a deep learning model for automatic segmentation of non-perfusion area in fundus fluorescein angiography
Journal of Big Data volume 11, Article number: 131 (2024)
Abstract
Diabetic retinopathy (DR) is the most prevalent cause of preventable vision loss worldwide, imposing a significant economic and medical burden on society today, of which early identification is the cornerstones of the management. The diagnosis and severity grading of DR rely on scales based on clinical visualized features, but lack detailed quantitative parameters. Retinal non-perfusion area (NPA) is a pathogenic characteristic of DR that symbolizes retinal hypoxia conditions, and was found to be intimately associated with disease progression, prognosis, and management. However, the practical value of NPA is constrained since it appears on fundus fluorescein angiography (FFA) as distributed, irregularly shaped, darker plaques that are challenging to measure manually. In this study, we propose a deep learning-based method, NPA-Net, for accurate and automatic segmentation of NPAs from FFA images acquired in clinical practice. NPA-Net uses the U-net structure as the basic backbone, which has an encoder-decoder model structure. To enhance the recognition performance of the model for NPA, we adaptively incorporate multi-scale features and contextual information in feature learning and design three modules: Adaptive Encoder Feature Fusion (AEFF) module, Multilayer Deep Supervised Loss, and Atrous Spatial Pyramid Pooling (ASPP) module, which enhance the recognition ability of the model for NPAs of different sizes from different perspectives. We conducted extensive experiments on a clinical dataset with 163 eyes with NPAs manually annotated by ophthalmologists, and NPA-Net achieved better segmentation performance compared to other existing methods with an area under the receiver operating characteristic curve (AUC) of 0.9752, accuracy of 0.9431, sensitivity of 0.8794, specificity of 0.9459, IOU of 0.3876 and Dice of 0.5686. This new automatic segmentation model is useful for identifying NPA in clinical practice, generating quantitative parameters that can be useful for further research as well as guiding DR detection, grading severity, treatment planning, and prognosis.
Introduction
Diabetic retinopathy (DR) is a common complication of diabetes mellitus (DM) [1] and the most prominent cause of avoidable blindness in many nations for working-age individuals [2, 3]. Due to the extensive epidemic of DM [4, 5], the prevalence of DR has reached an alarming level and continues apace [2, 3]. It was estimated that more than 160 million people will suffer from DR by 2045 globally, and 44.82 million are vision-threatening [6], which imposes huge medical and economic burdens. The pathogenesis of DR by elevated plasma glucose levels triggers biochemical environment changes that lead to microvascular damage. One prominent clinical lesion is the non-perfusion area (NPA) in the retina [7]. NPA is a non-perfused capillary patch caused by shunt and viscosity changes of blood secondary to vascular wall damage [8, 9], manifested as the occlusion or closure of local capillaries and dilation of adjoining ones [9]. Current NPA detection relies on fundus fluorescein angiography (FFA) or optical coherence tomography angiography (OCTA). Although OCTA has emerged as a non-invasive examination, its application has been limited because of the high cost and restrictions like artifact correction [10], making FFA remains the gold standard. On FFA, fluorescein induced into the bloodstream allows direct visualization of retinal vascular, whereas NPA is identified as hypofluorescent dark areas surrounded by hyperfluorescent dilated vessels [11], presenting as scattered, irregularly bordered patches in DR patients. Quantification of NPA has been revealed as a biomarker in DR assessment and follow-up [12, 13], with important implications in quantifying disease severity [13], predicting progression [13,14,15], and even guiding treatment scheme [14, 16]. However, manual quantification of NPA is time-consuming and labor-intensive, so auto-quantification is essential to make it practical in real-world clinics. Previous studies have proposed algorithms based on image processing [17, 18] for NPA automatic detection. These methods extracted NPA features as lesser grayscale and monotonic texture compared to regular perfused regions on FFA [18,19,20], inevitably depending on illumination correction and noise removal to minimize disturbances caused by image capturing, building models by over-segmenting in primary regions [19] or using topographic characteristics to designate NPA as valley [20] or pool [18], and provided comparable results with the manual label with an area under the curve (AUC) of around 0.8 [18, 20]. However, these methods will inevitably require manual feature extraction and empirical parameter selection, and lack robustness in practice.
In recent years, deep learning (DL) techniques have made breakthroughs in a variety of fields, such as computer vision, natural language processing and speech recognition, due to their powerful feature representation capabilities [21]. In the area of assisted diagnosis and screening of ophthalmic diseases, DL showed excellent advancement in detecting clinical features such as hemorrhage and exudation [22], diagnosing DR [23], and grading severity [24, 25]. Meanwhile, DL has also been applied to auto-detect some quantifiable indicators, such as vascular segmentation [26] and fluid quantification [27], and a few studies have produced promising outcomes for NPA detection on FFA [28, 29]. Tang et al. [30] proposed a deep learning model for segmenting non-perfusion regions from FFA images using dense atrous and deformable convolution (DADC), dense atrous convolution (DAC) block, and residual multi-kernel pooling to learn better feature representations. Jin et al. [29] explored different deep learning models DenseNet, ResNet50 and VGG16 to simultaneously identify NPAs, microaneurysms and leaks in FFA images. Despite the promising performance achieved, the accurate identification of non-perfusion areas from FFA images remains a challenging task. This is mainly due to several reasons: (1) the shape and size of non-perfusion areas in FFA images are usually irregular and diffuse. Some small, fuzzy boundaries are often missed by the model; and (2) The contrast between the non-perfusion region and the surrounding area is small, making it more difficult for the model to identify it accurately.
To address the above issues, in this paper, we propose a new deep learning model NPA-Net for the accurate identification of non-perfusion regions from FFA images. Firstly, considering the low contrast between the non-perfusion region and the surrounding region, we use a Contrast Limited Adaptive Histogram Equalization (CLAHE) technique [31] to enhance the contrast of the image to improve the recognition performance of the model. Secondly, as the shape of the non-perfusion regions varies, incorporating multi-scale features and contextual information in the model training process may help the model to better cope with irregular non-perfusion regions. To this end, we use an adaptive encoder feature fusion module (AEFF), a multilayer deep supervised loss and an atrous spatial pyramid pooling module (ASPP) to adaptively fuse multi-scale features and contextual information in the segmentation to further improve the recognition performance of the model for the non-perfusion regions. We have conducted extensive experiments on a stitched FFA image dataset acquired from a clinical setting, and the results show that NPA-Net outperforms other traditional and deep learning methods by a large margin.
Methodology
The overall structure of NPA-Net is shown in Fig. 1. Our segmentation model NPA-Net is a U-shaped structure with an encoder-decoder structure, containing a total of four encoders and four corresponding decoders, and finally a classification layer and a softmax function to output the predictions of the model. Each module contains a convolution layer, a batch normalization layer, and a dropout layer with a dropout rate of 0.2. The dropout and batch normalization layers are used to reduce overfitting and accelerate convergence respectively. The number of channels per encoder is 64, 256, 512 and 1024 respectively. Each encoder is followed by a pooling layer with a step size of \(2 \times 2\) to reduce the feature map, while an upsampling layer is applied to enlarge the feature map during decoding. In addition, skip connections are used to combine the feature maps extracted by each encoder with those of the corresponding decoder.
To improve the recognition performance of the model for non-perfusion regions, we introduced three key designs: Adaptive encoder feature fusion module (AEFF), Multilayer deep supervised loss and Atrous spatial pyramid pooling module (ASPP).
Adaptive encoder feature fusion module
In FFA images, non-perfusion regions are usually irregular and of varying shape and size. Traditional deep learning segmentation models such as the U-net cannot effectively handle irregularly regions. We believe that the introduction of multi-scale features and contextual information can help the model to better identify non-perfusion regions. To this end, we propose an adaptive encoder feature fusion module that fuses the multi-scale feature maps generated by different encoder layers and is able to adaptively learn the weights of different scale features (Fig. 2). Specifically, we first obtain the output \(E_{l} (l \in \{1,2,3,4\})\) of each encoder layer, which corresponds to the multi-scale deep feature representation extracted by encoder 1–encoder 4, respectively. As the resolution and scale of the feature representations extracted by different encoders differ, we scale the feature maps of encoder 1, encoder 2 and encoder 3 all to the size of the feature map of encoder 4 uniformly through the convolution and pooling layers. We then introduce an adaptive weighting layer to fuse the feature representations at different scales, and the fused feature representation is:
where \(w_{1}, w_{2}, w_{3}, w_{4}\) represent scalar variables used to weight the feature representations at different scales and \(w_{1}+w_{2}+w_{3}+w_{4}=1\). These weights can be calculated by:
where \(\lambda _{1}, \lambda _{2},\lambda _{3},\lambda _{4}\) represent the learnable parameters and we can update these weight parameters by back propagation.
Multilayer deep supervised loss
To further exploit the multi-scale features to improve the segmentation performance of the model, we further introduce a multi-layer deep supervised loss. Specifically, as shown in Fig. 1, we insert a prediction branch after each decoder layer, and different decoder layers are able to generate segmentation results at different scales. The overall segmentation loss is thus defined as:
where \({\mathbb {L}}_{Bce}\) represents the binary cross-entropy loss and \({\mathbb {L}}_{Dice}\) represents the Dice loss used to mitigate the imbalance problem. \(Y_{i}\) and \(Y_{i}^{\prime }\) represent the ground truth labels and model predictions, respectively, and the ground truth labels at other scales are obtained by downsampling the original ground truth labels.
Atrous spatial pyramid pooling module
We also introduced the atrous spatial pyramid pooling module to further extend the receptive field of the model to extract multi-scale feature representations, which has achieved significant segmentation performance improvements in the field of natural images [32]. As shown in Fig. 3, for an input feature map, we apply four convolution layers with different dilation rates to extract different scales of feature representations, and then fuse the different scales of feature representations to obtain the final output. We add this module to the last layer of the segmentation model to obtain the prediction results.
Experiments and results
To validate the segmentation performance of the proposed deep learning algorithm, we conduct experiments on a clinical medical dataset.
Dataset
FFA images from DR patients with type II diabetes who were referred to the ophthalmology department at Beijing Tsinghua Changgung Hospital between February 2015 and August 2022 were randomly selected. All images were collected by a Heidelberg imaging system (Heidelberg imaging system (SN: Spec-CAM-07889-S1600)). The imaging field is \(55 \times 55\) degrees, and mosaic images were obtained manually (Photoshop, version 22.4.0, Adobe Inc) from images in the venous phase (45 s -3 min). The NPA in the FFA images was annotated by two ophthalmologists and revised by one retinal expert. Mosaic images labeled with NPA were collected as the database for this study, and demographic information was collected from electronic medical records, with no personal information accessed. Approval for retrospective anonymized data collection and analysis from the institutional review board of Beijing Tsinghua Changgung Hospital was obtained. The study was conducted per the tenets of the Declaration of Helsinki.
A total of 163 eyes from 130 patients were included in this study, among which 116 eyes were randomly selected as the training set and 47 eyes as the test set. The demographic characteristics are shown in Table 1.
Pre-processing
We perform a series of pre-processing on the FFA images. Firstly, we normalize each pixel in the original FFA image to a zero mean, unit standard deviation, and then map the normalized pixel values to the [0,255] range. Secondly, we apply the Contrast Limited Adaptive Histogram Equalization (CLAHE) technique [31] to each FFA image to enhance the contrast of the image.
Specifically, we first divide each FFA image into \(8 \times 8\) blocks and then compute the histogram on each block. If any histogram bin has a value higher than 2, these pixels are clipped and evenly distributed to the other bins before histogram equalisation is applied. Compared to traditional histogram equalisation algorithms, CLAHE can focus on local areas and avoid over-amplifying background noise. The example visualization results of the CLAHE algorithm are shown in Fig. 4. It can be seen that after CLAHE, the contrast of some regions of interest in the FFA image is enhanced, which will facilitate more accurate recognition by the segmentation model later.
Finally, given the large size of the original FFA images, direct input to the model would result in out-of-memory, and direct scaling to smaller sizes would lose a large amount of detail information, we use a patch-based training and evaluation strategy. Specifically, we randomly sample 100,000 \(64 \times 64\) patches from the training set and input them to the model for training. In the test phase, we also use a patch-based evaluation strategy, where we extract patches from the test FFA images using a \(64 \times 64\) sliding window in steps of 32, and feed these patches into the model to obtain segmentation results. As the same pixel may appear in different patches, we average the prediction probabilities of different patches to obtain the final prediction probability for each pixel. A similar training and evaluation approach is used for the vessel segmentation task to increase the number of images in the training set [33].
Experimental setup and implementation details
We run all experiments based on the pytorch deep learning framework. We use the Stochastic Gradient Descent optimizer to train the segmentation model, with an initial learning rate set to 0.001 and a weight decay of 0.0003. The number of iterations of the model is set to 100,000, the batch size is 128, and we multiply the learning rate by 0.1 every 10,000 iterations. During the training process, we also use data augmentation strategies such as random rotation, random horizontal/vertical flipping to enhance the model’s robustness and generalization performance. For performance evaluation, we use the following metrics: Area Under the ROC Curve (AUC), accuracy (ACC), sensitivity (SEN), specificity (SPE), intersection over union(IOU), and Dice coefficient (Dice).
Comparison with the state-of-the-art methods
In order to verify the superiority of the proposed model NPA-Net, we compare it with some currently existing segmentation models. Specifically, we implement a traditional segmentation model based on Graph Cuts [34]. In addition, we also implement five deep learning segmentation models: U-net [35], CE-net [36], deeplab [37], ConvNeXt [38] and InternImage [39]. U-net is the most classical deep learning segmentation model that has been successfully applied to several medical image analysis tasks, and CE-net by dense atrous convolution (DAC) block and residual multi-kernel pooling (RMP) block to exploit the contextual spatial information to improve the segmentation performance of the model. ConvNeXt uses a more advanced segmentation model backbone, and deeplab uses atrous convolution to capture multi-scale features. InternImage [39] uses deformable convolution as the core operator, and introduces long-range dependencies and adaptive spatial aggregation to learn stronger and more robust feature representations. Table 2 shows the segmentation performance of different algorithms. It can be seen that the traditional semi-automatic segmentation algorithm Graph Cuts performs the worst, which is mainly because it still requires manual feature design, and lacks robustness and better generalization performance in practical applications. In contrast, the deep learning methods all achieve better segmentation performance, which indicates they have more powerful feature extraction capability and better model generalization performance. Finally, it can be found that our model achieves the best segmentation performance. Compared with the best comparison algorithm InternImage, our model has higher sensitivity and specificity, which indicates its ability to make full use of the contextual information and multi-scale features in FFA images to improve the model’s recognition of non-perfusion regions.
Figure 5 shows the segmentation results of different algorithms. It can be seen that the traditional method is difficult to accurately identify the non-perfusion regions in the FFA images. The deep learning model has better recognition performance due to its ability to perform feature extraction automatically. The better recognition performance of our segmentation model NPA-Net, which greatly reduces the number of false positives and the probability of missing non-perfusion regions, indicates that it is more suitable for segmenting non-perfusion regions from FFA images and is expected to be applied in clinical scenarios.
Figure 6 shows the comparison of the area of the non-perfusion region predicted by NPA-Net and labeled by the doctors on the test set images. It can be seen that NPA-Net accurately identifies non-perfusion regions in global FFA images, with its predicted areas closely matching those annotated by doctors. This shows its potential to automatically identify the non-perfusion region from the FFA image in clinical applications.
We perform ablation experiments to verify the effectiveness of the different components. As shown in Table 3, AEFF denotes adaptive encoder feature fusion module, ASPP denotes Atrous spatial pyramid pooling module, and MDS denotes Multilayer deep supervised loss. It can be observed that all of these components lead to improved segmentation performance, and our segmentation model integrates the different components to achieve the best segmentation performance.
Discussion
Non-perfusion area (NPA) is an important clinical feature of DR and an important link and component of the pathogenesis. With the availability and advancement of examination techniques, numerous studies have confirmed that NPA is closely related to the severity of DR [7, 40]. In cross-sectional studies, larger NPA was frequently observed in more severe eyes, with a more pronounced difference in peripheral retinal regions [41, 42]. Using manually labeled NPA, Antaki F. et al. [43] reported increased NPA associated with macular thickening and visual deterioration, indicating the prognostic value of NPA, which was further supported by a longitudinal study [14], as eyes with larger NPA was reported to have a higher risk for DR exacerbation, in which posterior NPA contributes to a higher hazard ratio in disease worsening.
As a key event in the progression of the disease to the proliferative phase, retinal neovascularization tends to occur when large areas of non-perfusion are present [44, 45]. In addition, NPA size varied in groups with different neovascularization locations [13], and larger NPAs found in individuals with optic disc neovascularization (a risk factor for severe visual loss [46]) compared to elsewhere [13, 47]. Studied further from the mechanism, NPA leads to local retinal hypoxia, increasing oxidate stress, tissue inflammation, and cytokines release [44]; it has also been discovered that the levels of pro-angiogenic factors like VEGF, which promote neovascularization, were correlated with the degree of NPA. As the current first-line DR treatment, trials involving anti-VEGF, including RISE/RIDE [48], VISTA [49], PERMEATE [49], and RECOVERY [16] showed an increased best-corrected visual acuity (BCVA) and paralleled delayed development of NPA in a dose-dependent manner, which provide a theoretical basis for NPA in guiding precision and individualization anti-VEGF schedule and adjustment during follow-up.
Although anti-VEGF is more effective in preserving the retina and can better maintain peripheral and night vision [50], some patients respond poorly to it [51]. At the same time, considering the economic cost of anti-VEGF treatment and the strong dependence on follow-up, laser photocoagulation remains an irreplaceable and effective treatment option with visual benefits not significantly different from anti-VEGF injections [50]. Laser photocoagulation is usually applied when retinal neovascularization is present, but since the appearance of new vessels marks the irreversible progression of DR to a severe stage, if laser is performed only after the appearance of neovascularization is observed, a head start in preventing vision loss is lost. Experienced ophthalmologists often perform laser treatment to control the further progression of DR in the presence of large areas of nonperfusion in the retina. However, there is no specific answer as to what constitutes a ”large area”, making the timing of laser treatment a very subjective decision. With this in mind, NPA quantification, as a biomarker that can sensitively and effectively reflect the state of retinal ischemia, will undoubtedly provide a strong, actionable and objective indicator of the above issues.
In recent years, deep learning techniques have developed rapidly and have achieved great success in several fields. Convolution neural networks based on deep learning techniques are able to automatically extract deep feature representations from the original image and then perform recognition and prediction, avoiding the tedious manual feature extraction step, integrating feature extraction and recognition into a unified framework that can be trained in an end-to-end manner. In the past few years, several automatic segmentation models for NPA have been proposed, but most of them are based on the local NPA segmentation model and do not effectively address some key challenges in the recognition of NPA [28,29,30]. First, in FFA images, the contrast between the NPA and the surrounding regions is low and easily affected by some background noise, and the stitching introduces more disturbances, e.g., inconsistent illumination, making the segmentation task more challenging; secondly, the NPAs are usually irregular and diffuse, and it is difficult for the traditional segmentation models to effectively handle the NPAs with different shapes and sizes. To this end, in this work, we propose a new segmentation model based on deep learning techniques for automatic identification of NPAs from mosaic stitch-based FFA images. We employ different techniques to deal with the above problems separately. Specifically, for the low contrast problem of the NPAs in FFA images, we use the CLAHE technique to enhance the contrast of the images to improve the recognition ability of the model for the NPAs. Then, considering that the NPAs are usually irregular and diffuse, introducing contextual information and multi-scale features in the segmentation model might improve the recognition ability of the model for NPAs of different shapes and sizes. To this end, we propose three modules based on the U-net segmentation backbone model: AEFF, MDS and ASPP to make full use of multiscale features and contextual information, which greatly improves the segmentation performance of the model for NPAs and effectively reduces false positive predictions while avoiding missing some small dispersed NPAs.
We conducted extensive experiments on a dataset of FFA images acquired from a clinical setting and compared our approach with traditional methods and some of the latest deep learning models. The experimental results in Table 2 show that our model NPA-Net greatly outperforms other comparative algorithms, with an AUC 0.9752, accuracy 0.9431, sensitivity 0.8794, specificity 0.9459, IOU 0.3876, and Dice 0.5686 on the test set. As can be seen from the segmentation results in Fig. 5, NPA-Net is able to mitigate the prediction of false positives and avoid the interference of background noise, as well as identify well for those small, diffuse NPAs. In addition, we verified the effectiveness of the different modules through ablation experiments in Table 3. We also quantified the NPAs predicted by the model, as shown in Fig. 6, the area of the NPA predicted by NPA-Net is very close to the area of the NPA labeled by the human physician, which demonstrates its ability to automatically segment the NPA. Future work is to validate the effectiveness of NPA-Net on clinical datasets from additional medical institutions.
Conclusion
In this work, a new DL model NPA-Net was developed to detect NPA in FFA images. We introduced three modules, Adaptive Encoder Feature Fusion (AEFF), Multilayer Deep Supervised Loss, and Atrous Spatial Pyramid Pooling (ASPP) to incorporate multiscale features and contextual information from different perspectives, effectively enhancing the model’s ability to recognize NPA of different sizes. This NPA segmentation model is expected to automatically identify biomarker NPAs from FFA images, provide reference for clinical diagnosis, grading and follow-up of DR patients, and provide evaluation for the formulation of treatment plans such as anti-vascular endothelial growth factor and laser photocoagulation.
Data availability
The dataset underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
Fong DS, Aiello L, Gardner TW, King GL, Blankenship G, Cavallerano JD, Ferris III FL, Klein R, Association AD. Retinopathy in diabetes. Diabetes care 27(suppl_1), 2004;84–87.
Saaddine JB, Honeycutt AA, Narayan KV, Zhang X, Klein R, Boyle JP. Projection of diabetic retinopathy and other major eye diseases among people with diabetes mellitus: United states, 2005–2050. Arch Ophthalmol. 2008;126(12):1740–7.
Li JQ, Welchowski T, Schmid M, Letow J, Wolpers C, Pascual-Camps I, Holz FG, Finger RP. Prevalence, incidence and future projection of diabetic eye disease in europe: a systematic review and meta-analysis. Eur J Epidemiol. 2020;35(1):11–23.
Zimmet PZ, Magliano DJ, Herman WH, Shaw JE. Diabetes: a 21st century challenge. Lancet Diabetes Endocrinol. 2014;2(1):56–64.
Saeedi P, Petersohn I, Salpea P, Malanda B, Karuranga S, Unwin N, Colagiuri S, Guariguata L, Motala AA, Ogurtsova K, et al. Global and regional diabetes prevalence estimates for 2019 and projections for 2030 and 2045: Results from the international diabetes federation diabetes atlas. Diabetes Res Clin Pract. 2019;157: 107843.
Teo ZL, Tham Y-C, Yu M, Chee ML, Rim TH, Cheung N, Bikbov MM, Wang YX, Tang Y, Lu Y, et al. Global prevalence of diabetic retinopathy and projection of burden through 2045: systematic review and meta-analysis. Ophthalmology. 2021;128(11):1580–91.
Wykoff CC, Yu HJ, Avery RL, Ehlers JP, Tadayoni R, Sadda SR. Retinal non-perfusion in diabetic retinopathy. Eye. 2022;36(2):249–56.
Stefánsson E, Chan YK, Bek T, Hardarson SH, Wong D, Wilson D. Laws of physics help explain capillary non-perfusion in diabetic retinopathy. Eye. 2018;32(2):210–2.
Cogan DG, Kuwabara T. Capillary shunts in the pathogenesis of diabetic retinopathy. Diabetes. 1963;12(4):293–300.
De Carlo TE, Romano A, Waheed NK, Duker JS. A review of optical coherence tomography angiography (octa). Int J Retina Vitreous. 2015;1(1):1–15.
Group ETDRSR, et al. Classification of diabetic retinopathy from fluorescein angiograms: Etdrs report number 11. Ophthalmology. 1991;98(5):807–22.
Markan A, Agarwal A, Arora A, Bazgain K, Rana V, Gupta V. Novel imaging biomarkers in diabetic retinopathy and diabetic macular edema. Ther Adv Ophthalmol. 2020;12:2515841420950513.
Nicholson L, Ramu J, Chan EW, Bainbridge JW, Hykin PG, Talks SJ, Sivaprasad S. Retinal nonperfusion characteristics on ultra-widefield angiography in eyes with severe nonproliferative diabetic retinopathy and proliferative diabetic retinopathy. JAMA Ophthalmol. 2019;137(6):626–31.
Silva PS, Marcus DM, Liu D, Aiello LP, Antoszyk A, Elman M, Friedman S, Glassman AR, Googe JM, Jampol LM, et al. Association of ultra-widefield fluorescein angiography–identified retinal nonperfusion and the risk of diabetic retinopathy worsening over time. JAMA Ophthalmol. 2022.
Marcus DM, Silva PS, Liu D, Aiello LP, Antoszyk A, Elman M, Friedman S, Glassman AR, Googe JM, Jampol LM, et al. Association of predominantly peripheral lesions on ultra-widefield imaging and the risk of diabetic retinopathy worsening over time. JAMA Ophthalmol. 2022.
Wykoff CC, Nittala MG, Zhou B, Fan W, Velaga SB, Lampen SI, Rusakevich AM, Ehlers JP, Babiuch A, Brown DM, et al. Intravitreal aflibercept for retinal nonperfusion in proliferative diabetic retinopathy: outcomes from the randomized recovery trial. Ophthalmol Retina. 2019;3(12):1076–86.
Zhao H, Li X, Xiang Y, Zou B, Gao L. Automatic detection of retinal capillary nonperfusion via a new active contour model. In: 2016 International conference on progress in informatics and computing (PIC). IEEE. 2016;321–325.
Rasta SH, Nikfarjam S, Javadzadeh A. Detection of retinal capillary nonperfusion in fundus fluorescein angiogram of diabetic retinopathy. BioImpacts: BI. 2015;5(4):183.
Jasiobedzki P, McLeod D, Taylor CJ. Detection of non-perfused zones in retinal images. In: Computer-Based Medical Systems-Proceedings of the Fourth Annual IEEE Symposium, IEEE Comput Soc. 1991;162–163.
Sivaswamy J, Agarwal A, Chawla M, Rani A, Das T. Extraction of capillary non-perfusion from fundus fluorescein angiogram. In: International joint conference on biomedical engineering systems and technologies. Cham: Springer; 2008. pp.176–188.
LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.
Dai L, Wu L, Li H, Cai C, Wu Q, Kong H, Liu R, Wang X, Hou X, Liu Y, et al. A deep learning system for detecting diabetic retinopathy across the disease spectrum. Nat commun. 2021;12(1):1–11.
Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama. 2016;316(22):2402–10.
Ting DSW, Cheung CY-L, Lim G, Tan GSW, Quang ND, Gan A, Hamzah H, Garcia-Franco R, San Yeo IY, Lee SY, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. Jama 2017;318(22): 2211–2223.
Bhuiyan A, Govindaiah A, Deobhakta A, Hossain M, Rosen R, Smith T. Automated diabetic retinopathy screening for primary care settings using deep learning. Intell Based Med. 2021;5: 100045.
Sevgi DD, Srivastava SK, Wykoff C, Scott AW, Hach J, O’Connell M, Whitney J, Vasanji A, Reese JL, Ehlers JP. Deep learning-enabled ultra-widefield retinal vessel segmentation with an automated quality-optimized angiographic phase selection tool. Eye. 2022;36(9):1783–8.
Moraes G, Fu DJ, Wilson M, Khalid H, Wagner SK, Korot E, Ferraz D, Faes L, Kelly CJ, Spitz T, et al. Quantitative analysis of oct for neovascular age-related macular degeneration using deep learning. Ophthalmology. 2021;128(5):693–705.
Nunez do Rio JM, Sen P, Rasheed R, Bagchi A, Nicholson L, Dubis AM, Bergeles C, Sivaprasad S. Deep learning-based segmentation and quantification of retinal capillary non-perfusion on ultra-wide-field retinal fluorescein angiography. J Clin Med. 2020;9(8):2537.
Jin K, Pan X, You K, Wu J, Liu Z, Cao J, Lou L, Xu Y, Su Z, Yao K, et al. Automatic detection of non-perfusion areas in diabetic macular edema from fundus fluorescein angiography for decision making using deep learning. Sci Rep. 2020;10(1):1–7.
Tang Z, Zhang X, Yang G, Zhang G, Gong Y, Zhao K, Xie J, Hou J, Hou J, Sun B, et al. Automated segmentation of retinal nonperfusion area in fluorescein angiography in retinal vein occlusion using convolutional neural networks. Med Phys. 2021;48(2):648–58.
Zuiderveld K. Contrast limited adaptive histogram equalization. Graphics Gems. 1994;474–485.
Chen L-C, Papandreou G, Schroff F., Adam H. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 2017.
Wu Y, Xia Y, Song Y, Zhang Y, Cai W. Multiscale network followed network model for retinal vessel segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention. Cham: Springer; 2018;119–126.
Boykov YY, Jolly, MP. Interactive graph cuts for optimal boundary and region segmentation of objects in nd images. In: Proceedings Eighth IEEE International conference on computer vision. ICCV 2001. Vancouver: IEEE; 2001;1:105–112.
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention. Cham: Springer; 2015;234–241.
Gu Z, Cheng J, Fu H, Zhou K, Hao H, Zhao Y, Zhang T, Gao S, Liu J. Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans Med Imag. 2019;38(10):2281–92.
Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Analy Mach Intell. 2017;40(4):834–48.
Liu Z, Mao H, Wu C-Y, Feichtenhofer C, Darrell T, Xie S. A convnet for the 2020s. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2022;11976–11986.
Wang W, Dai J, Chen Z, Huang Z, Li Z, Zhu X, Hu X, Lu T, Lu L, Li H, et al. Internimage: Exploring large-scale vision foundation models with deformable convolutions. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2023;14408–14419.
Hwang TS, Zhang M, Bhavsar K, Zhang X, Campbell JP, Lin P, Bailey ST, Flaxel CJ, Lauer AK, Wilson DJ, et al. Visualization of 3 distinct retinal plexuses by projection-resolved optical coherence tomography angiography in diabetic retinopathy. JAMA Ophthalmol. 2016;134(12):1411–9.
Silva PS, Cruz AJD, Ledesma MG, van Hemert J, Radwan A, Cavallerano JD, Aiello LM, Sun JK, Aiello LP. Diabetic retinopathy severity and peripheral lesions are associated with nonperfusion on ultrawide field angiography. Ophthalmology. 2015;122(12):2465–72.
de Carlo TE, Bonini Filho MA, Baumal CR, Reichel E, Rogers A, Witkin AJ, Duker JS, Waheed NK. Evaluation of preretinal neovascularization in proliferative diabetic retinopathy using optical coherence tomography angiography. Ophthal Surg Lasers Imag Retina. 2016;47(2):115–9.
Antaki F, Coussa RG, Mikhail M, Archambault C, Lederer DE. The prognostic value of peripheral retinal nonperfusion in diabetic retinopathy using ultra-widefield fluorescein angiography. Graefe’s Arch Clin Exp Ophthalmol. 2020;258(12):2681–90.
Abraham JR, Wykoff CC, Arepalli S, Lunasco L, Hannah JY, Martin A, Mugnaini C, Hu M, Reese J, Srivastava SK, et al. Exploring the angiographic-biologic phenotype in the imagine study: quantitative uwfa and cytokine expression. Br J Ophthalmol. 2022;106(10):1444–9.
Ra H, Park JH, Baek JU, Baek J. Relationships among retinal nonperfusion, neovascularization, and vascular endothelial growth factor levels in quiescent proliferative diabetic retinopathy. J Clin Med. 2020;9(5):1462.
Rand LI, Prud’homme GJ, Ederer F, Canner P. Factors influencing the development of visual loss in advanced diabetic retinopathy. diabetic retinopathy study (drs) report no. 10. Investig Ophthalmol Vis Sci. 1985;26(7):983–91.
Jung EE, Lin M, Ryu C, Moysidis SN, Burkemper B, Murgai R, Arbabi A, Ameri H. Association of the pattern of retinal capillary non-perfusion and vascular leakage with retinal neovascularization in proliferative diabetic retinopathy. J Curr Ophthalmol. 2021;33(1):56.
Campochiaro PA, Wykoff CC, Shapiro H, Rubio RG, Ehrlich JS. Neutralization of vascular endothelial growth factor slows progression of retinal nonperfusion in patients with diabetic macular edema. Ophthalmology. 2014;121(9):1783–9.
Figueiredo N, Srivastava SK, Singh RP, Babiuch A, Sharma S, Rachitskaya A, Talcott K, Reese J, Hu M, Ehlers JP. Longitudinal panretinal leakage and ischemic indices in retinal vascular disease after aflibercept therapy: the permeate study. Ophthalmology Retina. 2020;4(2):154–63.
Gross JG, Glassman AR, Jampol LM, Inusah S, Aiello LP, Antoszyk AN, Baker CW, Berger BB, Bressler NM, Browning D, et al. Panretinal photocoagulation vs intravitreous ranibizumab for proliferative diabetic retinopathy: a randomized clinical trial. Jama. 2015;314(20):2137–46.
Maggio E, Sartore M, Attanasio M, Maraone G, Guerriero M, Polito A, Pertile G. Anti-vascular endothelial growth factor treatment for diabetic macular edema in a real-world clinical setting. Am J Ophthalmol. 2018;195:209–22.
Acknowledgements
Thanks to Miaochu Wang from the Trafalgar Castle School in Canada for her work on data organization and statistical analysis.
Funding
This project is supported by National Key R&D Program of China (2022YFC2502800) and Tsinghua University Initiative Scientific Research Program of Precision Medicine (10001020106).
Author information
Authors and Affiliations
Contributions
BW obtained the data set for the study and conducted the initial experiments. WF designed the methodology and the model. WF, BW, DS, ML, AC, JW, SL, YZ, BW, ZG, SX and YH were involved in in the revision of the study objectives and methods. All authors were involved in editing and proofreading. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Feng, W., Wang, B., Song, D. et al. Development and evaluation of a deep learning model for automatic segmentation of non-perfusion area in fundus fluorescein angiography. J Big Data 11, 131 (2024). https://doi.org/10.1186/s40537-024-00968-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s40537-024-00968-9