Rationale and Objectives: Cystic lung disease may be accurately diagnosed by imaging interpretation of specialist radiologists, without other information. We hypothesized that with minimal training non-specialists could perform similarly to specialist physicians in the diagnosis of cystic lung disease.
Methods: 72 cystic lung disease cases and 25 cystic lung disease mimics were obtained from three sources: 1) a prospective acquired diffuse lung disease registry, 2) a retrospective search of medical records and 3) teaching files. Cases were anonymized, randomized and interpreted by 7 diffuse lung disease specialists and 15 non-specialist radiologists and pulmonologists. Clinical information other than age and sex was not provided. Prior to interpretation, non-specialists viewed a short PDF training document explaining cystic lung disease interpretation.
Results: Correct first choice diagnosis of 85%-88% may be achieved by high-performing specialist readers and 71%-80% by non-specialists and lower-performing specialists, with mean accuracies in the diagnosis of LAM (91%, p < 0.0001), BHD (93%, p < 0.0001), PLCH (89%, p < 0.0001) and LIP (92%, p < 0.0001). A strategy based on cyst appearance: simple cysts (LAM), peri-septal cysts (BHD), bizarre-shaped cysts (PLCH) and vascular indented cysts (LIP) gave non-specialists accuracies of 90% (p < 0.0001), 94% (p < 0.0001), 92% (p < 0.0001) and 88% (p < 0.0001), respectively, for these diagnoses. Cystic lung abnormalities caused by diseases other than LAM, BHD, PLCH and LIP are rarely accurately diagnosed by imaging alone.
Conclusion: With specific but limited training, non-specialist physicians can diagnose cystic lung diseases from CT appearance alone with similar accuracy to specialists, correctly identifying approximately 75% of cases.
Rationale and Objectives: Cystic lung disease may be accurately diagnosed by imaging interpretation of specialist radiologists, without other information. We hypothesized that with minimal training non-specialists could perform similarly to specialist physicians in the diagnosis of cystic lung disease. Methods: 72 cystic lung disease cases and 25 cystic lung disease mimics were obtained from three sources: 1) a prospective acquired diffuse lung disease registry, 2) a retrospective search of medical records and 3) teaching files. Cases were anonymized, randomized and interpreted by 7 diffuse lung disease specialists and 15 non-specialist radiologists and pulmonologists. Clinical information other than age and sex was not provided. Prior to interpretation, non-specialists viewed a short PDF training document explaining cystic lung disease interpretation. Results: Correct first choice diagnosis of 85% - 88% may be achieved by high-performing specialist readers and 71% - 80% by non-specialists and lower-performing specialists, with mean accuracies in the diagnosis of LAM (91%, p < 0.0001), BHD (93%, p < 0.0001), PLCH (89%, p < 0.0001) and LIP (92%, p < 0.0001). A strategy based on cyst appearance: simple cysts (LAM), peri-septal cysts (BHD), bizarre-shaped cysts (PLCH) and vascular indented cysts (LIP) gave non-specialists accuracies of 90% (p < 0.0001), 94% (p < 0.0001), 92% (p < 0.0001) and 88% (p < 0.0001), respectively, for these diagnoses. Cystic lung abnormalities caused by diseases other than LAM, BHD, PLCH and LIP are rarely accurately diagnosed by imaging alone. Conclusion: With specific but limited training, non-specialist physicians can diagnose cystic lung diseases from CT appearance alone with similar accuracy to specialists, correctly identifying approximately 75% of cases.
Widespread cystic lung disease is an uncommon imaging finding that may be clinically irrelevant or cause significant morbidity and mortality. Pulmonary cysts appear as low-attenuation regions with a surrounding wall and are often round or oval in shape, but other appearances may occur [1] .
The most common causes of cystic lung disease are Lymphangioleiomyomatosis (LAM), Langerhans Cell Histiocytosis (PLCH), Birt-Hogg-Dube (BHD) syndrome and Lymphocytic Interstitial Pneumonia (LIP) [2] [3] [4] [5] [6] . Other causes include metastases, amyloidosis, neurofibromatosis, light-chain deposition disease, pneumocystis pneumonia, hypersensitivity pneumonitis, pulmonary interstitial glycogenosis and nonspecific interstitial pneumonia [2] - [12] .
Most diffuse parenchymal lung diseases require clinical information to establish a diagnosis. However, the imaging appearance of cystic lung diseases is often diagnostic, even without clinical information [3] [13] [14] . Prior studies indicate that experts may be highly accurate in the diagnosis of cystic lung disease based on imaging appearance alone [2] [13] . Previous studies have suggested that pulmonologists and trainees perform less well than chest radiologists [2] . Our hypothesis was that limited training with a simple strategy based on cyst appearance would allow non-specialists to achieve similar accuracy to experts in the diagnosis of cystic lung disease.
2. Materials and Methods
The authors have no conflict of interest, the study is IRB-approved (IRB# 820774) and HIPA compliant. Informed consent was waived by the IRB. We acquired all cases of cystic lung disease available from a single medical center, from three sources: 1) our institution prospectively acquired diffuse lung disease registry, 2) a retrospective search of medical records, and 3) teaching files.
Our institution began a diffuse lung disease registry in January 2013. To be entered into the registry, cases were evaluated jointly by two pulmonologists, a thoracic radiologist and a pulmonary pathologist and classified by cause and confidence in the diagnosis. Thirty-nine cases with moderate or high confidence diagnosis of cystic lung disease were included in this study (Figure 1).
Our radiology information system database was searched using a commercially available search engine (MONTAGETM Search and Analytics, Nuance mPower Clinical Analytics, Nuance Communications, Inc.) from 2012-2018 for 5 thoracic CT exam codes and the following search terms: “lymphangioleiomyomatosis”, “LAM”, “Langerhans cell histiocytosis”, “PLCH”, “eosinophilic granuloma”, “Birt-Hogg-Dube”, “Birt Hogg Dube”, “lymphocytic interstitial pneumonia”, “LIP” and “cystic PCP”. The time frame was chosen to match that of the ILD registry resulting in 270 cases.
CT reports and a limited evaluation of medical records separated cases into those likely or unlikely to meet criteria for a diagnosis of cystic lung disease. Causes for exclusion at this stage included, 1) no evaluation by a pulmonologist (n = 118), 2) duplicate cases (n = 27), 3) other lung disease diagnosed (n = 18), 4) a history of disease without imaging findings (n = 9), 5) superimposed lung disease (n = 6). Cases without a pulmonologist evaluation were excluded because analysis showed these cases were unlikely to have sufficient documentation to prove a diagnosis. Each “likely case” received an extensive review of medical records by two individuals independently, a chest radiologist and pulmonologist specializing in diffuse lung disease to determine if the case met criteria for a diagnosis of a cystic lung disease (LAM and LIP: American Thoracic Society guidelines [14] [15] , PLCH: guidelines by Girschikofsky [16] , BHD: guidelines by Menko [17] ). Both reviewers had to agree on a diagnosis for study inclusion, adding 31 cases to the study. Teaching file case diagnoses were also confirmed by agreement following independent review of the medical record by both reviewers, adding 27 cases to the study (Figure 1).
The database was augmented with cases of emphysema, cystic bronchiectasis and honeycombing that might be confused with cystic lung disease. Proof of diagnosis followed the same protocol as the cystic lung diseases, but cases were selected for a high likelihood of confusion with cystic lung disease.
Searches yielded a total of 72 cystic lung disease cases and 25 cystic mimics.
2.1. Reviewers
Cases were anonymized, randomized and blindly reviewed by 22 individuals of varying experience for the diagnosis of cystic lung disease. Reviewers were recruited from several medical centers in our metropolitan area. Seven reviewers were specialists in thoracic imaging or pulmonologists specializing in diffuse lung diseases. The remaining reviewers were non-specialist radiologists or pulmonologists. The reviewers had not previously been exposed to the any cases used in the study.
2.2. Training Algorithm
Two separate training documents were created and tested. A PDF document of 12 PowerPoint slides, which is a synopsis of the experience of the first author, a radiologist with 24 years of subspecialty experience, outlined a method with imaging examples, for distinguishing cystic lung diseases. The algorithm is similar to one independently proposed by another group [3] and is follows a pattern typically used by chest radiologists. The critical points of the document are as follows:
1) True cysts must be distinguished from honeycombing, emphysema, and cystic bronchiectasis.
2) Most common cystic lung diseases are: LAM, PLCH, BHD and LIP.
3) Simple cysts have round or oval shape and a thin wall.
4) Number of cysts may be helpful in the diagnosis. High profusion simple cysts (defined as ≥100) (Figure 2) is usually LAM. A low profusion of simple cysts (defined as <50) is usually BHD or LIP.
5) Appearance of BHD cysts may be peri-septal or peri-pleural and lenticular in shape (Figure 3).
6) Appearance of PLCH cyst may be irregularly or bizarrely shaped and/or thick walled (Figure 4).
Analysis of results of the first round of cases showed deficiencies in the training document and a second training document was created, with five principal changes: 1) Cyst counting was discarded and LAM was recommended as first choice diagnosis of all simple cysts in women and PLCH as first diagnosis in men, 2) BHD-type cysts were defined as peri-septal (removing peri-pleural from the criteria), 3) PLCH-type cysts were defined as irregularly or bizarrely shaped (removing thick walled as a criterion), 4) cheerio-type cysts (small thick walled cysts) were explained to be caused by a variety of diseases, usually PLCH and metastasis in approximately equal frequency (Figure 5), 5) LIP-type cysts were defined as those containing vascular indentations or septations (Figure 6).
2.3. Image Review
Images were viewed using a DICOM imaging database (Horus, 2019 Horus project, https://horosproject.org/) on each reader’s personal computer. Horus allows for scrollable images, window/level conversion and coronal and axial reconstructions. Reviewers were blinded to clinical information, except age and sex. Reviewers were aware that the database contained a variety of cystic lung diseases and cystic mimics but were unaware of the relative frequencies within the database.
Specialists reviewed the cases without other input. In Round 1, 8 non-specialist reviewers were given the first training document prior to case interpretation. In Round 2, non-specialists numbers 2 and 5 reviewed the second training document and re-evaluated cases for the diagnosis of disease an average of 4 months after the previous interpretation session. An additional 7 new non-specialists were given the second training document prior to evaluation of cases.
Reviewers were asked to provide the 1st, 2nd and 3rd most likely diagnoses for each case to simulate a typical differential diagnosis given in radiology reports. Answers were selected from a drop-down menu: LAM, PLCH, BHD, LIP, honeycombing, bronchiectasis, emphysema and other diagnosis. If “other diagnosis” was selected, an additional free text box was supplied. Non-specialist reviewers in Round 1 were asked to provide the cyst character and cyst number from specified lists. Cyst character choices were: 1) true cyst: thin wall round or oval, 2) true cyst: thick wall and/or bizarre shape, 3) true cyst: lenticular, subpleural and/or peri-septal, 4) true cyst: other, 5) honeycombing, 6) emphysema, 7) cystic bronchiectasis. Cyst number choices: 1) ≥100, 2) <50, 3) 51 - 99, 4) <5, and 5) Not applicable. Reviewers in Round 2 were asked to evaluate for cyst character with choices: 1) Simple-type, 2) BHD-type, 3) PLCH-type, 4) LIP type, 5) cheerio-type, 6) other.
2.4. Statistical Evaluation
Data were imported into the R statistical environment for analyses [18] . Basic statistical tabulations were made to count the number of correct diagnoses for each rater. Diagnostic performance was assessed for each rater by calculating sensitivity, specificity Positive Predictive Value (PPV), Negative Predictive Value (NPV), and accuracy for each diagnosis separately (i.e. LAM, LCH, LIP, BHD, Mimics). Total accuracy (i.e. irrespective of specific diagnoses) was also assessed. For each rater, Receiver Operating Characteristic (ROC) curves were generated for each diagnosis, and the Area under the Curve (AUC) was estimated. In addition to individual measures, group estimates (e.g. pulmonary specialists as a whole) were estimated based on group means. 95% confidence intervals for proportions were estimated based on the formula:
p ^ ± z ∗ p ^ ∗ ( 1 − p ^ ) N
where p ^ represents the proportion, N the sample size, and z = 1.96. Categorical variables were also evaluated for statistical significance with two-tailed Fisher’s exact tests.
3. Results3.1. Characteristics of the Database
The patients’ age ranged from 20 to 86 years with both mean and median ages of 48 years. Women accounted for 71/97, 73% of patients. The causes, frequency and source of cases are listed in Table 1.
3.2. Reader Performance
Table 2 lists the clinical experience of the reviewers and the fraction of correct 1st choice and 1st and 2nd choice diagnoses. Rad-specialists 1 and 2 performed better than all other readers with 1st diagnosis true positive rates of 87% and 82%, respectively. Utilizing the first teaching tool, two non-specialist readers, a 3rd year radiology resident and a general pulmonologist were able to outperform rad-specialists 3 and 4 and the two pulmonary specialists. Four additional non-specialists: a body imaging fellow, interventional radiologist, body imaging radiologist and a 2nd year radiology resident performed similarly to rad-specialists 4 and 5 and the two pulmonary specialists.
Using the second teaching tool, two non-specialists (general pulmonologist, body imaging radiologist) outperformed rad-specialists 3 and 4 and the two pulmonary specialists. Five other non-specialists (two 4th year radiology residents, one 3rd year radiology resident, and two 2nd year radiology residents) performed similarly to rad-specialists 4 and 5 and the two pulmonary specialists.
Table 3 shows the performance of readers for the correct first choice diagnosis of LAM, BHD, LIP, LCH and non-cystic lung disease respectively. Rad-specialists 1 and 2 achieved accuracies of greater than 94% for each of the five specific diagnoses. The accuracy of diagnosis was high, usually >80% for all readers individually. Of the 115 diagnostic accuracies measured (23 readers × 5 categories), 86 (75%) were ≥90% among both specialists and non-specialists. In general, across both rounds, the lowest performance measures for all diseases were the sensitivity and PPV.
Figure 7 shows the mean and standard deviation of the area under the ROC curve for each of the reader groups for each of the diseases, showing overlap in the performance of all readers. However, two rad-specialists, 1 and 2, did generally better than all other readers for all diseases. The performance of rad-specialists 3 and 4, the pulmonary specialists and the non-specialists for each diagnosis is nearly indistinguishable. LIP was the most problematic diagnosis with the lowest average area under the ROC curve and the greatest variance among all readers. The training algorithms in Round 1 and Round 2 performed similarly, with the exception of the diagnosis of LIP, where the second algorithm resulted in substantial improvement in diagnosis.
Causes of cystic lung disease and cystic mimics
Cystic Disease
DLD Registry*
MR Search^
Teaching File
Total
LAM
19
1
10
30
PLCH
4
5
4
13
BHD
2
13
1
16
LIP
2
5
0
7
Cystic Metastasis
0
0
3
3
Amyloidosis
0
0
1
1
Neurofibromatosis
0
1
0
1
Sarcoidosis
1
0
0
1
IPF
0
1
0
1
Total Cystic Disease
28
26
19
73
Cystic Mimics
Emphysema
1
4
4
9
Honeycombing
IPF
2
0
0
2
CTD
3
0
0
3
HP
1
0
1
2
Sarcoidosis
4
1
0
5
Cystic Bronchiectasis
0
0
3
3
Total Mimics
11
5
8
24
Grand Total
39
31
27
97
*Diffuse lung disease registry; ^Medical records search.
Diagnostic accuracy of cystic lung disease diagnosis as a function of training
Reader
Specialty
Years Practice
Pulm ILD+
Dx1 TP*
Dx1-2 TP^
Rad-specialist 1
Radiology (Thoracic)
5
5 yr
83 (86)
87 (91)
Rad-specialist 2
Radiology (Thoracic)
5
5 yr
79 (82)
81 (84)
Rad-specialist 3
Radiology (Thoracic)
8
4 yr
73 (75)
81 (84)
Rad-specialist 4
Radiology (Thoracic)
1
1 yr
67 (70)
78 (81)
Pulm-specialist 1
Pulmonology (DLD specialist)
7
7 yr
66 (69)
71 (74)
Pulm-specialist 2
Pulmonology (DLD specialist)
5
5 yr
68 (71)
74 (77)
Round 1
Non-specialist 1
Radiology (3rd Year Resident)
3
2 mo
78 (81)
80 (83)
Non-specialist 2
Pulmonology (General)
14
6 mo
73(76)
76 (79)
Non-specialist 3
Radiology (Body Imaging Fellow)
6
6 mo
69 (72)
77 (80)
Non-specialist 4
Radiology (Interventional)
6
4 mo
71 (74)
77 (79)
Non-specialist 5
Radiology (Body Imaging)
13
4 mo
69 (72)
74 (77)
Non-specialist 6
Radiology (2nd Year Resident)
1.8
2 mo
67 (70)
75 (78)
Non-specialist 7
Radiology (1st Year Resident)
0.8
1 mo
55 (58)
69 (72)
Non-specialist 8
Radiology (2nd Year Resident)
1.8
3 mo
54 (57)
69 (72)
Round 2
Non-specialist 2
Pulmonology (General)
14
6 mo
77 (80)
83 (87)
Non-specialist 5
Radiology (Body Imaging)
13
4 mo
79 (82)
86 (90)
Non-specialist 9
Radiology (4th Year Resident)
3.3
3 mo
75 (78)
87 (91)
Non-specialist 10
Radiology (4th Year Resident)
3.3
3 mo
72 (75)
76 (79)
Non-specialist 11
Radiology (3rd Year Resident)
2.3
3 mo
72 (75)
73 (76)
Non-specialist 12
Radiology (2nd Year Resident)
1.3
1 mo
73 (76)
87 (91)
Non-specialist 13
Radiology (2nd Year Resident)
1.3
2 mo
69 (72)
82 (86)
Non-specialist 14
Radiology (2nd Year Resident)
1.3
2 mo
64 (66)
75 (78)
Non-specialist 15
Radiology (1st Year Resident)
0.3
1 mo
64 (67)
71 (74)
+Years of specialization in thoracic radiology (radiologists) or pulmonary diffuse infiltrative lung disease (pulmonologists) or months of specific training in thoracic radiology or diffuse infiltrative lung disease. *Frequency of 1st choice correct diagnosis (% correct in parentheses). ^Frequency that correct diagnosis was in the top 2 diagnoses (% correct in parentheses).
3.3. Rare Causes of Cystic Lung Disease
Reader performance in the diagnosis of LAM, BHD, PLCH, LIP and cystic mimics
Reader
Sensitivity
Specificity
PPV
NPV
Accuracy
Lymphangioleiomyomatosis (N = 30)
Rad-specialists 1-2
90 (84 - 96)
99 (96 - 99)
97 (93 - 99)
96 (92 - 99)
96 (92 - 99)
Rad-specialists 3-4
80 (72 - 88)
96 (91 - 99)
89 (83 - 95)
91 (86 - 97)
91 (85 - 96)
Pulmonary Specialists
92 (86 - 97)
92 (86 - 97)
84 (77 - 91)
96 (92 - 99)
92 (86 - 97)
Round 1 Non-specialists
81 (73 - 89)
94 (89 - 99)
86 (79 - 93)
92 (86 - 97)
90 (84 - 96)
Round 2 Non-specialists
86 (79 - 93)
94 (89 - 98)
86 (79 - 93)
94 (89 - 99)
91 (86 - 97)
All Readers
84 (77 - 92)
94 (89 - 99)
87 (80 - 94)
93 (88 - 98)
91 (85 - 97)
Birt Hogg Dube Syndrome (N = 16)
Rad-specialists 1-2
78 (70 - 86)
99 (97 - 99)
94 (89 - 99)
96 (92 - 99)
95 (91 - 99)
Rad-specialists 3-4
56 (46 - 66)
98 (95 - 99)
90 (84 - 96)
92 (87 - 97)
91 (86 - 97)
Pulmonary Specialists
56 (46 - 66)
94 (90 - 99)
68 (58 - 77)
92 (86 - 97)
88 (82 - 95)
Round 1 Non-specialists
81 (73 - 89)
94 (89 - 99)
73 (64 - 82)
96 (92 - 99)
92 (86 - 97)
Round 2 Non-specialists
77 (69 - 85)
97 (94 - 99)
86 (79 - 93)
96 (91 - 99)
94 (89 - 99)
All Readers
75 (66 - 84)
96 (92 - 99)
81 (73 - 89)
95 (91 - 99)
93 (87 - 98)
Langerhans Cell Histiocytosis (N = 13)
Rad-specialists 1-2
88 (82 - 95)
95 (91 - 99)
76 (67 - 84)
98 (96 - 99)
94 (90 - 99)
Rad-specialists 3-4
85 (77 - 92)
92 (87 - 98)
68 (59 - 77)
98 (95 - 99)
91 (86 - 97)
Pulmonary Specialists
58 (48 - 68)
92 (87 - 98)
54 (44 - 64)
93 (88 - 98)
88 (81 - 94)
Round 1 Non-specialists
75 (66 - 84)
86 (79 - 93)
47 (37 - 57)
96 (92 - 99)
85 (78 - 92)
Round 2 Non-specialists
68 (59 - 78)
92 (87 - 97)
59 (49 - 69)
95 (91 - 99)
89 (83 - 95)
All Readers
73 (64 - 82)
90 (85 - 96)
57 (47 - 67)
96 (92 - 99)
88 (82 - 94)
Lymphocytic Interstitial Pneumonia (N = 7)
Rad-specialists 1 - 2
86 (79 - 93)
95 (91 - 99)
61 (51 - 70)
99 (97 - 99)
94 (90 - 99)
Rad-specialists 3 - 4
57 (47 - 67)
92 (87 - 98)
51 (41 - 61)
97 (93 - 99)
90 (84 - 96)
Pulmonary Specialists
64 (55 - 74)
96 (91 - 99)
56 (46 - 66)
97 (94 - 99)
93 (88 - 98)
Round 1 Non-specialists
36 (26 - 45)
97 (94 - 99)
56 (46 - 65)
95 (91 - 99)
93 (88 - 98)
Round 2 Non-specialists
76 (68 - 85)
93 (88 - 98)
47 (37 - 57)
98 (95 - 99)
92 (86 - 97)
All Readers
60 (51 - 70)
95 (90 - 99)
52 (42 - 62)
97 (93 - 99)
92 (87 - 98)
Cystic Lung Disease Mimics (N = 24)
Rad-specialists 1 - 2
96 (92 - 99)
97 (93 - 99)
91 (85 - 97)
99 (96 - 99)
96 (93 - 99)
Rad-specialists 3 - 4
90 (84 - 96)
94 (89 - 99)
83 (76 - 92)
96 (93 - 99)
93 (88 - 98)
Pulmonary Specialists
84 (77 - 91)
94 (90 - 99)
85 (78 - 92)
95 (90 - 99)
92 (86 - 97)
Round 1 Non-specialists
80 (71 - 88)
96 (92 - 99)
88 (81 - 94)
93 (88 - 98)
92 (86 - 97)
Round 2 Non-specialists
77 (69 - 86)
97 (94 - 99)
92 (86 - 97)
93 (87 - 98)
92 (87 - 98)
All Readers
81 (74 - 89)
96 (92 - 99)
89 (83 - 95)
94 (89 - 99)
92 (87 - 98)
Performance of cyst characteristics in diagnosing cystic lung diseases by non-specialists
Cyst Type
Sensitivity
Specificity
PPV
NPV
Accuracy
Simple Cyst (LAM)
Non-specialists
88 (83 - 91)
91 (89 - 93)
83 (78 - 96)
94 (92 - 96)
90 (88 - 92)
Simple Cyst + Female* (LAM)
Non-specialists
91 (86 - 94)
94 (91 - 95)
86 (82 - 89)
96 (94 - 97)
92 (91 - 94)
BHD-type
Non-specialists
76 (68 - 82)
98 (96 - 99)
86 (80 - 91)
95 (94 - 96)
94 (92 - 96)
LIP-type
Non-specialists
73 (60 - 83)
93 (91 - 95)
45 (38 - 52)
98 (97 - 99)
92 (89 - 93)
PLCH-type
Non-specialists
38 (30 - 48)
96 (94 - 97)
59 (49 - 69)
91 (90 - 92)
88 (86 - 90)
There were 7 cases of cystic disease (9.6%) that were not one of the four common causes: 3 cases of cystic metastases and one each of amyloidosis, neurofibromatosis, sarcoidosis and idiopathic pulmonary fibrosis. None of these, except for cystic metastasis, were correctly diagnosed by any reader. Correct diagnosis of cystic metastasis was made in 2/12 (17%) instances by rad-specialists. Non-specialists correctly diagnosed cystic metastasis in 0/24 instances in Round 1 and in 15/27 (55%) instances in Round 2 after specific training regarding cheerio type cysts was given in the second training document.
*If both simple cyst and female as criteria for Dx of LAM
3.4. Causes of Misdiagnosis
There was a subset of cases that accounted for the majority of the remaining diagnostic errors. For 22 cases (23%) in Round 1, ≤50% of observers (0 - 7/14 observers) correctly diagnosed disease. For 22 cases (23%) in Round 2, ≤55% of observers (0 - 5/9 observers) correctly diagnosed disease. Seven cases were the uncommon cystic diseases discussed previously and two were cases where superimposed emphysema confused the case. The majority of misdiagnosed cases were confined to a small number of the common causes of cystic lung disease: LAM, PLCH, BHD and LIP (5 LIP, 3 LAM, 2 BHD and 2 PLCH in Round 1 and 4 BHD, 3 LAM, 3 PLCH, 1 LIP and 4 cystic mimics in Round 2).
3.5. Evaluation of Diagnostic Strategies
We employed two different teaching strategies. Round 1 used a combination of cyst characteristics and cyst number to inform a diagnosis similar to current conventional teaching strategies. Round 2 relied exclusively on the recognition of five cyst types: 1) simple-type, 2) BHD-type, 3) PLCH-type, 4) LIP-type and 5) cheerio-type. Strategy 2 slightly outperformed strategy 1, predominantly because of improved diagnosis of LIP. Two individuals, non-specialists 2 and 5 were involved in both rounds of testing and both improved with the second strategy.
The performance characteristics of the various cyst types for their respective diseases are listed in Table 4. In general, the cyst types are moderately to highly specific for the disease with specificities from 91% - 100%. However, with the exception of simple cysts for the diagnosis of LAM, the sensitivity of cyst types was moderate to low.
4. Discussion
Our study and others [2] [3] [13] [19] indicate that cystic lung diseases can be accurately diagnosed based solely on imaging characteristics, with a correct first-choice diagnosis of as high as 86% by the best-performing specialists. This accuracy is principally due to the diagnosis of the most common cystic lung diseases: LAM (combined accuracy 91%), BHD (combined accuracy of 93%), LIP (combined accuracy of 92%) and PLCH (combined accuracy of 88%). Previous reports have shown similar accuracy in the diagnosis of LAM and LIP but lesser accuracy in the diagnosis of PLCH [2] .
Previous studies have suggested that pulmonologists and trainees perform less well than chest radiologists [2] , a finding we also showed when comparing non-specialists with our highest-performing chest radiologists. However, we have demonstrated that with minimal training, non-specialist radiologists and pulmonologists can also have high performance with the correct first choice diagnosis of as high as 82% which is similar to some specialists.
We devised two training strategies, both of which helped non-specialist readers achieve moderate to high accuracy in the diagnosis of common cystic lung diseases. The second training strategy performed slightly better than the first. This second strategy is simpler and is based on the recognition of 4 cyst types: Simple-type, BHD-type, PLCH-type and LIP-type cysts that are moderate to highly predictive of LAM, BHD, PLCH and LIP respectively.
Misdiagnosis related to the cheerio sign, small thick-walled cysts, was an important cause of reduced accuracy, a finding that has not been noted by prior studies. This sign, which is commonly associated with PLCH, was therefore interpreted as PLCH by nearly all reviewers in Round 1. However, in our database, metastasis, another known cause of the cheerio sign [7] [15] , accounted for half of the cases. In Round 2, the training document specifically noted that the cheerio sign could be caused by both metastasis and PLCH and suggested that lower predominant cheerio signs are likely metastasis and upper predominant cheerio signs are likely PLCH. This strategy reduced, by half, diagnostic errors of cheerio-type cysts. Other reported causes of the cheerio sign include, amyloidosis (present in our database) adenocarcinoma spectrum lesions, primary lung carcinoma, granulomatosis with polyangiitis, rheumatoid nodules and pulmonary meningothelial-like nodules [20] [21] . If our database is representative of the general population, cheerio-type cysts are caused by metastasis and PLCH in approximately equal frequency.
To simulate clinical practice, we included lung diseases such as emphysema, cystic bronchiectasis and severe honeycombing that might be confused with cystic lung disease, a confounding factor that previous studies have not included [2] [3] [13] [19] . We have shown that readers can usually distinguish cystic lung diseases from mimics with a combined accuracy of 92%.
In our database, a significant fraction of diagnostic errors were due to rare causes of cystic lung disease including cystic metastases, amyloidosis, neurofibromatosis, sarcoidosis, and idiopathic pulmonary fibrosis. With the exception of cystic metastasis, none of these diagnoses were correctly diagnosed. Imaging alone is usually not adequate to diagnose these cases.
The majority of the remainder of diagnostic errors occurred in a small subset of cases, 13 - 15 of 97 cases, or 13% - 15% in our database. This finding suggests that in most cases, approximately 85% - 87%, of LAM, BHD, PLCH and LIP produce cysts with the characteristic cyst features outlined in this report and can be identified by trained non-specialists. However, there are small subsets of LAM, BHD, PLCH and LIP that produce cysts that are not characteristic of the disease or for other reasons are easily misclassified.
Our study has strengths related to a large number of cases and large number and varied experiences of readers. In addition, our training algorithm is a teaching tool that can be readily and widely utilized in clinical practice. The principal limitation of the study is the source of the cases. Sixty percent of cases were obtained by retrospective review of the medical records or from teaching files. There is a possibility that biases regarding cyst appearance were introduced in this process. There were no normal exams and the relative frequency of cystic disease and cystic mimics was not controlled. This may have increased reader performance compared with the daily practice where cystic diseases are rarely seen and cystic mimics are more common. The low number of pulmonologists participating in the study is another limitation such that little can be deduced about differences in performance between radiologists and pulmonologists.
5. Conclusion
In conclusion, most cases of cystic lung diseases can be accurately diagnosed from their appearance on thin-section CT utilizing a novel strategy based on 5 cyst appearances: 1) Simple-type, 2) BHD-type, 3) PLCH-type, 4) LIP-type and 5) Cheerio-type. With limited training, non-specialist radiologists and non-specialist pulmonologists can perform as well or better than some diffuse lung disease specialists, although not as well as the highest-performing specialists.
Conflicts of Interest
The authors declare no conflicts of interest regarding the publication of this paper.
Cite this paper
Miller, W.T., Patterson, K.C., Sood, S., Schmitt, J.E., Wani, A.A., Borden, R., Galperin-Aisenberg, M., Porteus, M.K., Hershman, M.L., Hewitt, M., Levy, J., Babatunde, V.D., Glushko, T., Niesen, T.J., Leshchinskiy, S., Sahakyan, K., Desai, K., Gillman, J.A., Reddy, S., Shriver, M., Linna, N.B., Noor, A.M., Buz, A., Biron, M.E. and Simpson, S. (2023) A Validated Model for the Imaging Diagnosis of Cystic Lung Disease. Open Journal of Radiology, 13, 42-57. https://doi.org/10.4236/ojrad.2023.131005
ReferencesHansell, D.M., Bankier, A.A., MacMahon, H., McLoud, T.C., Müller, N.L. and Remy, J. (2008) Fleischner Society: Glossary of Terms for Thoracic Imaging. Radiology, 246, 697-722. https://doi.org/10.1148/radiol.2462070712Gupta, N., Meraj, R., Tanase, D., et al. (2015) Accuracy of Chest High-Resolution Computed Tomography in Diagnosing Diffuse Cystic Lung Diseases. European Respiratory Journal, 46, 1196-1199. https://doi.org/10.1183/13993003.00570-2015Gupta, N., Vassallo, R., Wikenheiser-Brokamp, K.A. and McCormack, F.X. (2015) Diffuse Cystic Lung Disease. Part II. American Journal of Respiratory and Critical Care Medicine, 192, 17-29. https://doi.org/10.1164/rccm.201411-2096CITrotman-Dickenson, B. (2014) Cystic Lung Disease: Achieving a Radiologic Diagnosis. European Journal of Radiology, 83, 39-46. https://doi.org/10.1016/j.ejrad.2013.11.027Xu, K.-F., Feng, R., Cui, H., et al. (2016) Diffuse Cystic Lung Diseases: Diagnostic Considerations. Seminars in Respiratory and Critical Care Medicine, 37, 457-467.https://doi.org/10.1055/s-0036-1580690Lee, J.E., Cha, Y.K., Kim, J.S. and Choi, J.H. (2017) Birt-Hogg-Dubé Syndrome: Characteristic CT Findings Differentiating It from Other Diffuse Cystic Lung Diseases. Diagnostic and Interventional Radiology, 23, 354-359. https://doi.org/10.5152/dir.2017.16606Zhang, J., Zhao, Y.-L., Ye, M.-X., et al. (2012) Rapidly Progressive Diffuse Cystic Lesions as a Radiological Hallmark of Lung Adenocarcinoma. Journal of Thoracic Oncology, 7, 457-458. https://doi.org/10.1097/JTO.0b013e31823c5a39Dumlu, T., Karapolat, B.S., Yıldırım, ü., Güngor, A. and Aydın, L.Y. (2011) Medical Image. Bilateral Diffuse Cystic, Cavitary Lung Metastasis of Adenocarcinoma. The New Zealand Medical Journal, 124, 93-95Zamora, A.C., Collard, H.R., Wolters, P.J., Webb, W.R. and King, T.E. (2006) Neurofibromatosis-Associated Lung Disease: A Case Series and Literature Review. European Respiratory Journal, 29, 210-214.https://doi.org/10.1183/09031936.06.00044006Madhyastha, S.P., Gopalaswamy, V., Reddy, C.T. and Acharya, R.V. (2017) Interesting Association of Neurofibroma with Diffuse Cystic Lung Disease (NF-DLD). BMJ Case Reports, 2017, bcr2016217774.https://doi.org/10.1136/bcr-2016-217774Alves Júnior, S.F., Zanetti, G., Alves de Melo, A.S., et al. (2019) Neurofibromatosis Type 1: State-of-the-Art Review with Emphasis on Pulmonary Involvement. Respiratory Medicine, 149, 9-15. https://doi.org/10.1016/j.rmed.2019.01.002Weinman, J.P., White, C.J., Liptzin, D.R., Deterding, R.R., Galambos, C. and Browne, L.P. (2018) High-Resolution CT Findings of Pulmonary Interstitial Glycogenosis. Pediatric Radiology, 48, 1066-1072. https://doi.org/10.1007/s00247-018-4138-4Raoof, S., Bondalapati, P., Vydyula, R., et al. (2016) Cystic Lung Diseases: Algorithmic Approach. CHEST, 150, 945-965. https://doi.org/10.1016/j.chest.2016.04.026Gupta, N., Finlay, G.A., Kotloff, R.M., et al. (2017) Lymphangioleiomyomatosis Diagnosis and Management: High-Resolution Chest Computed Tomography, Transbronchial Lung Biopsy, and Pleural Disease Management. An Official American Thoracic Society/Japanese Respiratory Society Clinical Practice Guideline. American Journal of Respiratory and Critical Care Medicine, 196, 1337-1348. https://doi.org/10.1164/rccm.201709-1965STKokosi, M.A., Nicholson, A.G., Hansell, D.M. and Wells, A.U. (2016) Rare Idiopathic Interstitial Pneumonias: LIP and PPFE and Rare Histologic Patterns of Interstitial Pneumonias: AFOP and BPIP. Respirology, 21, 600-614. https://doi.org/10.1111/resp.12693Girschikofsky, M., Arico, M., Castillo, D., et al. (2013) Management of Adult Patients with Langerhans Cell Histiocytosis: Recommendations from an Expert Panel on Behalf of Euro-Histio-Net. Orphanet Journal of Rare Diseases, 8, Article No. 72.https://doi.org/10.1186/1750-1172-8-72Menko, F.H., van Steensel, M.A., Giraud, S., et al. (2009) Birt-Hogg-Dubé Syndrome: Diagnosis and Management. The Lancet Oncology, 10, 1199-206.https://doi.org/10.1016/S1470-2045(09)70188-3R Core Team (2020). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna. https://www.R-project.org/Escalon, J.G., Richards, J.C., Koelsch, T., Downey, G.P. and Lynch, D.A. (2019) Isolated Cystic Lung Disease: An Algorithmic Approach to Distinguishing Birt-Hogg-Dubé Syndrome, Lymphangioleiomyomatosis, and Lymphocytic Interstitial Pneumonia. American Journal of Roentgenology, 212, 1260-1264. https://doi.org/10.2214/AJR.18.20920Abbott, G.F., Rosado-de-Christenson, M.L., Franks, T.J., Frazier, A.A. and Galvin, J.R. (2004) From the Archives of the AFIP: Pulmonary Langerhans Cell Histiocytosis. Radiographics, 24, 821-841. https://doi.org/10.1148/rg.243045005Chou, S.-H., Kicska, G., Kanne, J.P. and Pipavath, S. (2013) Cheerio Sign. Journal of Thoracic Imaging, 28, W4. https://doi.org/10.1097/RTI.0b013e31827944d2