Multispectral Pattern Recognition Reveals a Diversity of Clinical Signs in Intermediate Age-Related Macular Degeneration | IOVS (2024)

This was a retrospective, diagnostic study of 36 eyes of 20 patients with bilateral intermediate AMD who attended the Centre for Eye Health for a macular assessment in 2015 (Supplementary Fig. S1). The Centre for Eye Health is an intermediate-tier eye-care establishment that provides imaging and visual system diagnostic services, operating primarily as a service to eye-health practitioners in commercial practice so that they can optimally manage their patients. Intermediate AMD was defined according to the Beckman initiative for macular research clinical classification scheme, and included any eye with at least one large druse (≥125-μm diameter or exceeding the average width of a retinal vein at the disc margin) within 3 mm of the macular center or pigmentary abnormalities associated with medium drusen in patients aged 55 years or older.² Ten normal eyes (five patients) were also analyzed. Eyes with incomplete or poor image quality, pseudophakia, or macular disease other than intermediate AMD were excluded. Patient written consent was obtained in accordance with the Declaration of Helsinki and approved by a Biomedical Human Research Ethics Advisory Panel of the University of New South Wales, Sydney, Australia.

Methods used for the pattern recognition analysis of images are summarized in Figure 1. Each classified image series, from each eye, included readily available ocular imaging modalities: a reference color fundus photograph, and 8-bit grayscale fundus autofluorescence (FAF, Optos Panoramic 200Tx; Optos, Dunfermline, UK), green (Green532nm, Optos Panoramic 200Tx), and infrared (IR815nm, Spectralis Heidelberg Retina Angiograph 2; Heidelberg Engineering, Heidelberg, Germany) scanning laser ophthalmoscopy images (Fig. 1A). Thus, four images were acquired per eye, resulting in a total analysis of 144 AMD and 40 normal images.

Images were first manually aligned by using eight blood vessel bifurcation reference points in ir-tweak²⁷ (https://www.sci.utah.edu/download/ncrtoolset/; provided in the public domain by the Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT, USA), background corrected by subtracting the Gaussian blurred image from the original in ImageJ (https://imagej.nih.gov/ij/; provided in the public domain by the National Institutes of Health, Bethesda, MD, USA), contrast enhanced, and masked to remove areas beyond the macula (Fig. 1B, panel 1). Reference color fundus photographs (Kowa WX 3D Nonmydriatic retinal camera; Kowa, Nagoya, Japan) were also registered and masked. No further manipulations were performed on the color photographs. The three preprocessed IR815nm, Green532nm, and FAF images were assigned to the red, green, and blue channel, respectively, and preliminarily assessed qualitatively in rgb triplet format by using the merge channels function in ImageJ (Fig. 1B, panel 2).

Multispectral digital analysis uses the spectral data from each pixel in a series of images generated by using narrow bands of light at various imaging wavelengths. Different wavelengths penetrate the fundus to different depths and hence different modalities contain unique information regarding lesion characteristics and location.²⁸ Pixel values (ranging from 0 to 255 for an 8-bit image) from each unit in an image are categorized into common spectral theme classes and the results outputted by using a thematic pseudocolor map or classified image. The clinical meaning of these classes may then be determined a posteriori.²⁹

Grayscale, preprocessed IR815nm, Green532nm, and FAF images from each eye were automatically classified by using unsupervised K-means cluster analysis and commercially available satellite imaging software (http://www.pcigeomatics.com/; provided in the public domain by PCI Remote Sensing, Markham, Ontario, Canada) with the following parameters: 16 maximum classes, 16 maximum iterations, and a 0.01 minimum threshold (Fig. 1B, panel 3). The principles of cluster analysis have been described elsewhere.^25,26,30 Although other clustering algorithms and parameters were explored, we describe herein the best combination as judged by the authors, using simultaneous comparison against the reference fundus photograph (Supplementary Fig. S2). Class separability was calculated by using transformed divergence (D_T; KCLUS algorithm, PCI Remote Sensing), as previously described.^31,32 Briefly, D_T describes a method of calculating the separability or statistical difference (correct classification) between spectral signatures. It derives from measuring the difference in pixel values between all pairs from a predefined number of groups. D_T measures are presented as a value between zero and two: zero indicates extremely poor classification accuracy or overlap between signatures of two classes, while two indicates complete separation between the two classes (analogous to 100% classification accuracy).³² Pattern recognition theory was originally developed for satellite image analysis and encompasses clustering analysis and tests for cluster separability.²⁹ The cluster separability indices include D_T, which is effectively similar to separability indices reported by univariate signal detection theory used in psychology, but D_T is able to report separability in N dimensional space. A larger separability value equates to better classification accuracy, and the statistic is resistant to relatively large deviations from normality. Increasing the number of data channels only increases the value of the statistic if the additional channel contributes further separability.³⁰

Each application of cluster analysis resulted in a classified image, or pseudocolor map, whereby a hue represents a unique class or spectral signature.^30,31,33,34 The two classes with the lowest separability were merged and the process repeated until a minimum D_T cutoff value between all classes was met. We investigated the diagnostic utility of multispectral, unsupervised pattern recognition by using three endpoint criteria—D_T equal to 1.5, 1.7, or 1.9, which translates to a >83%, >90%, and >98% probability of correct classification, respectively (Fig. 1C).³² The final output is a statistically robust, classified image derived from multispectral pattern recognition for each eye.

To evaluate the diagnostic accuracy of the classified images, a ground truth was defined by using two clinical experts (highly trained optometrists with over 15 years of combined optometric experience, with 8 years providing ocular imaging diagnostic services in an optometry-ophthalmology collaborative care setting). The graders manually annotated 36 color fundus photographs for all visible (1) large drusen, (2) hyperpigmentary abnormalities, and (3) hypopigmentary abnormalities (up to a maximum of 20 instances each), using the Cell Counter plugin in ImageJ. Graders also annotated an additional 20 locations each indicating blood vessels and background. The graders were not advised of the diagnosis of any of the study eyes but instructed to use the Beckman initiative definitions of the lesions. Drusen grading circles C0 (63-μm diameter) and C1 (125-μm diameter) were appended to each image for comparative purposes. Disagreements between the two graders were resolved by a third grader (AL). Ten percent of all annotations were also confirmed by a medical retinal specialist (NA). Optical coherence tomography findings were not considered during the grading; however, select images are included in this article for illustrative purposes.

For the classified images, each class was manually categorized as either an AMD lesion (coincident predominantly with large drusen, hyper- or hypopigmentary abnormalities) or background class (blood vessels or background). In the classified images, eyes without an identifiable AMD lesion class were considered poorly classified and excluded from the sensitivity analysis (5/36, 14%; case example shown in Fig. 2C and further described in the Results), resulting in a total of 1693 annotations across 31 eyes with intermediate AMD. Classified images and the annotated color fundus reference photograph for the 31 eyes were then compared for concordance and agreement. True positives of the lesions (which for simplicity we describe as sensitivity) and true negatives (specificity) were calculated (Fig. 1D).

In the normal group, there were three females, three Caucasians, and two Asians aged between 60 and 66 years (Table 1). Macular images from normal, healthy eyes were characterized by relatively low spectral diversity, that is, greater uniformity in the grayscale input images (Fig. 2A). The rgb images consequently displayed a minimal range of colors or phenotype diversity. Comparatively, eyes with AMD showed a more diverse appearance in the classified images (Fig. 2B). Imaging artifacts were identified as a unique signature (Figs. 2A, 2B).

AMD participants were predominantly female (13/20, 65%), Caucasian (17/20, 85%), with ages ranging between 56 and 82 years. Three patients were Asian (3/20, 15%). In the multispectral pattern recognition classified images (also described as pseudocolor maps), each separable spectral signature was identified by a distinct color. Well-classified AMD images showed notable colocalization with the distribution and position of drusen and other AMD lesions, while poorly classified eyes did not (i.e., Fig. 2B versus Fig. 2C). The latter typically also displayed a lower inherent signal to noise ratio (ratio of brightness to darkness of AMD lesions despite preprocessing) in the input images, suggesting the result might be a limitation of the input modalities. Incidentally, the rgb triplet images were also a useful means of validating the accuracy of image registration. Poor registration typically appeared as kerf³⁵ surrounding blood vessels in the rgb image and where visible, prompted realignment.

Across the AMD dataset, the number of unique spectral signatures (colors in the pseudocolor maps) in intermediate AMD ranged between 2 and 11 (Table 2). Classification accuracy for the AMD annotations ranged between 79% and 100% for each eye and depended on the underlying appearance of AMD lesions in the composite imaging. A stricter, that is, higher D_T endpoint, was associated with poorer discrimination of lesions. Consequently, the D_T 1.5 criterion (>83% probability of correct classification) displayed the greatest diagnostic utility. In that classification set, 1543 of 1693 annotations were correctly classified (91% agreement). Two annotations could not be classified owing to convolution by imaging artifacts. Large drusen, hyperpigmentary abnormalities, and hypopigmentary abnormalities were concordantly discriminated in 75%, 68%, and 75% of all annotated instances, respectively, yielding an overall sensitivity of 74% for AMD lesions. By comparison, classification accuracy of blood vessels and background was greater at 99% and 96%, respectively, consistent with a specificity of 98% (Table 1). Without excluding the poorly classified eyes (n = 5), 1743/1932 annotations were correctly classified, resulting in an overall sensitivity and specificity of 68% and 98%, respectively. An additional analysis was conducted to determine if the number of samples from each eye affected overall proportions used to calculate sensitivity, such as to skew the result. There was no statistically significant correlation (P = 0.49) between the percentage of correct classification and the number of AMD lesion annotations on an eye level.

This section describes additional observations attained by using empirical, successive comparison of the grayscale images, rgb images, pseudocolor maps, and the reference fundus photographs. Each color (class) of the pseudocolor map representing a separable spectral signature corresponded to a specific anatomic structure, such as the fundus background, blood vessels, or a distinct type of AMD lesion (drusen, hyper- or hypopigmentary abnormalities; Fig. 3). Using this approach with a D_T criterion of 1.5, a signature corresponding to large drusen was separable in 31 of 35 eyes (89%). Hyperpigmentary abnormalities and hypopigmentary abnormalities were identified in 4 of 10, and 1 of 3 eyes (33%), respectively.

Large drusen in each input, grayscale image typically appeared brighter than the background signal, especially in the Green532nm and FAF images (Fig. 4), and consequently appeared green or blue-green in the rgb images. Interestingly, the drusen often colocalized with more than one theme class in one eye, including changes within or encircling the drusen (intra- or perilesional signatures in 18 eyes; Fig. 4B). Occasionally, several classes represented by various pseudocolors were discovered across drusen (both within and between units in the rgb and classified images), reminiscent of drusen diversity (Fig. 4C). Before pattern recognition, these drusen appeared similar in the fundus images. The three different drusen-related signatures in Figure 4C correlated with differences in the internal reflectivity of the drusen and in the integrity of the inner segment ellipsoid zone overlying the drusen.

If separable, pigmentary abnormalities were typically assigned to a single class (Fig. 5). Hyperpigmentary abnormalities appeared bright in IR815nm and FAF imaging though dark (decreased reflectivity, increased absorption) in the Green532nm images, rendering them bright pink in the rgb images (Figs. 5A, 5B). Hypopigmentary abnormalities appeared least commonly of all the AMD lesions (in three eyes only, as based on expert annotation) and where present, appeared blue-green in rgb (Fig. 5C). A lack of separability occurred twice and may relate to convolution by underlying drusen.

Despite a range of widely available ocular imaging technologies, the diagnosis of AMD and its subtypes is still complex, incongruous, and difficult.^6,17,36,37 Signs of AMD, such as drusen or pigmentary abnormalities, may be hard to identify owing in part to low contrast, interindividual variations, and red saturation of the fundus.³⁸ This study showcased the high specificity of a multispectral, unsupervised pattern recognition approach in classifying intermediate AMD, using multimodal imaging. Specifically, we found AMD lesions have specific spectral signatures that can be displayed on a single multispectral pseudocolor map to simplify multimodal image interpretation.

The strength of this study lies in its capacity to objectively and quantitatively integrate the unique findings of multiple imaging modalities. In particular, this method appeared to capitalize on the advantages of scanning laser ophthalmoscopy—namely, high contrast, averaging, and depth discrimination (axial optical sectioning).^39,40 Evaluating the finite number of colors in the output pseudocolor map is simpler than assessing multiple images and therefore is amenable to interpretation by nonexperts, which might shorten the time and resources required for grading. Input imaging results were also well-tolerated en face modalities that can be acquired undilated.

The steps that allowed useful classification of retinal architecture relied upon (1) accurate image registration, (2) preprocessing, including pseudobackground correction and the contrast enhancement steps, (3) appropriate masking of nonmacular areas, and (4) stepwise, down merging in the unsupervised classification approach. Unprocessed images were widely prone to spurious classifications confounded by nonmacular signatures, physiological variations in visibility of the choroidal vasculature, differences in fundus pigmentation (Supplementary Fig. S1), and background intensity gradient differences across the image (dependent on the acquisition conditions). By beginning with a large predefined number of classes and merging down, the final classified image also appeared more clinically meaningful than the image generated by prespecifying a fixed smaller number of classes (Supplementary Fig. S2).

Our study confirmed that tissue types in the retina have an identifiable spectral signature. In combination with spatial context, these signatures may aid in the detection and diagnosis of disease, including sub- or preclinical features (those preceding or not meeting standard clinical criteria for disease, e.g., dynamic changes in rhodopsin regeneration) or for tracking disease progression (Supplementary Fig. S3).^38,41,42 Light delivered to the ocular tissues undergoes fluorescence, scattering, and absorption, primarily by lipofuscin, hemoglobin, melanin, and water.⁴¹ Macular pigment and choroidal vasculature in our series were never identified as separate spectral signatures, likely owing to the background correction step in preprocessing. An additional finding was that drusen were more often accurately discriminated than pigmentary abnormalities. This may relate to difficulties in grading and the characteristics of these different lesions in the component imaging: drusen appear bright on all modalities with a reflectance peak approximating 560 nm,⁴³ while pigmentary abnormalities appear dark (due to absorption) in green light. Previous studies have also shown that melanin best absorbs light between 450- and 700-nm wavelength, though 450- to 600-nm range light may also be absorbed by other ocular components.⁴¹

As illustrated in Figure 4, our method also appeared to discriminate subtypes of drusen, which may relate to the integrity of the overlying outer retinal layers and may therefore correlate with risk of atrophy. Our observations of unique perilesional spectral signatures may represent a limitation in image registration, that is, residual areas of kerf. Alternatively, it may suggest prescient extension of disease. For example, separable signatures lying just within the margins of drusen might reflect the indistinct border characteristics or mechanical distortion of the retina by soft drusen. Thus, we chose not to remove these signatures in the final analysis.

As with all imaging, the methodology introduced in the foregoing sections should be viewed as adjunctive rather than a replacement of traditional imaging and funduscopy.^44,45 This article reports on a proof-of-concept study and will benefit from future work using a replication sample of larger size. The three modalities (structural images) included were chosen for their ready commercial availability. The sensitivity of our technique might also be improved by varying the number and/or type of input channels (spectral bands) into the algorithm, such as en face images from dye studies, near-infrared FAF, scanning laser tomography, or a dedicated multispectral imaging device.

This study evaluated the appearance and spectral signature separability of typical intermediate AMD lesions (those that are hyperreflective in the input imaging) in Asian and Caucasian eyes only, and did not consider spatial information or lesions of other stages (reticular pseudodrusen, geographic atrophy, signs of neovascular AMD) or ethnicities. The “ground truth” was established via consensus opinion of clinical experts. Although those graders were masked to the diagnosis of the eyes, they were instructed to annotate each eye for lesions in accordance with the Beckman initiative for macular research clinical classification scale. Consequently, some bias was inevitable.

Finally, the application of the described technique to diseases or stages other than intermediate AMD was beyond the scope of this study. Late AMD was not evaluated. The present work described a case control diagnostic accuracy study, and the reported sensitivity and specificity values may indeed be lower in the presence of other confounding retinal diseases. The effect of lens status, for example, pseudophakia, may also introduce other variables, which will require clarification before more widespread adoption of the method. Further research designed as a cohort study applying the technique to patients with these potential confounders is currently underway.

This work represents a novel, proof-of-concept study demonstrating that multispectral, unsupervised pattern recognition can be used to classify fundus images in intermediate AMD and has applications in improving our understanding of phenotypes in AMD and identifying disease progression. This form of image analysis was able to accurately identify AMD lesions with high specificity and could ultimately be incorporated into a fully automated diagnostic tool or clinical decision-making support system suitable for AMD and other diseases, in nonspecialized environments and telemedicine to facilitate the early and timely detection of eye disease.

The authors thank Erica Fletcher and Laura Downie for technical advice relating to pattern recognition, Robert Marc and Rebecca Pfeiffer for technical advice and access to the University of Utah image registration program (ir-tweak), Lindsay Moore and Elizabeth Wong for performing the AMD grading, and Tyson Xu for technical support.

Supported in part by an Australian Government Research Training Program scholarship and a National Health and Medical Research Council (NHMRC) Grant (No. 1033224). Guide Dogs NSW/ACT is a partner in the NHMRC grant and also provided a supplementary PhD scholarship for AL and support for LN-S.

Disclosure: A. Ly, P; L. Nivision-Smith, P; N. Assaad, None; M. Kalloniatis, P