1. Introduction

ETSN

E-Health Telecommunication Systems and Networks

2167-9517

Scientific Research Publishing

10.4236/etsn.2021.102003

ETSN-110047

Articles

Computer Science&Communications

Unified Analysis Specific to the Medical Field in the Interpretation of Medical Images through the Use of Deep Learning

Tudor

Florin Ursuleanu

¹^*Andreea

Roxana Luca

²Liliana

Gheorghe

³Roxana

Grigorovici

⁴Stefan

Iancu

⁴Maria

Hlusneac

⁴Cristina

Preda

⁵Alexandru

Grigorovici

⁶

Department of Surgery VI, “Sf. Spiridon” Hospital, Iasi, Romania

Department of Endocrinology, “Sf. Spiridon” Hospital, Iasi, Romania

Department of Surgery I, Regional Institute of Oncology, Iasi, Romania

Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, Iasi, Romania

Department of Radiology, “Sf. Spiridon” Hospital, Iasi, Romania

Department of Obstetrics and Gynecology, Integrated Ambulatory of Hospital “Sf. Spiridon”, Iasi, Romania

24062021

1002417417, May 202121, June 2021 24, June 2021

2014

This work is licensed under the Creative Commons Attribution International License (CC BY). http://creativecommons.org/licenses/by/4.0/

Deep learning (DL) has seen an exponential development in recent years, with major impact in many medical fields, especially in the field of medical image. The purpose of the work converges in determining the importance of each component, describing the specificity and correlations of these elements involved in achieving the precision of interpretation of medical images using DL. The major contribution of this work is primarily to the updated characterisation of the characteristics of the constituent elements of the deep learning process, scientific data, methods of knowledge incorporation, DL models according to the objectives for which they were designed and the presentation of medical applications in accordance with these tasks. Secondly, it describes the specific correlations between the quality, type and volume of data, the deep learning patterns used in the interpretation of diagnostic medical images and their applications in medicine. Finally presents problems and directions of future research. Data quality and volume, annotations and labels, identification and automatic extraction of specific medical terms can help deep learning models perform image analysis tasks. Moreover, the development of models capable of extracting unattended features and easily incorporated into the architecture of DL networks and the development of techniques to search for a certain network architecture according to the objectives set lead to performance in the interpretation of medical images.

Medical Image Analysis Data Types Labels Deep Learning Models

1. Introduction

The medical data most used in medical practice are medical images and for this reason most deep learning algorithms have targeted this category of medical information for the realization of medical applications.

This paper presents a methodical review of the literature [1] with the objective of carrying out an analysis of the importance of the relationship between the types and characteristics of scientific data and their use of deep learning models in the interpretation of medical images. We have defined a methodology for semi-automating the production of relevant articles and eliminating those with low impact in the scientific community, by applying inclusive and exclusive quality criteria in the fields of medicine and information technology [2]. The major contribution of this work lies primarily in the updated characterization of the characteristics of the constituent elements of the process of deep learning from data to applications in medicine. Secondly, it describes the specific correlations between data, deep learning models used in the interpretation of diagnostic medical images and their applications in medicine. Finally presents problems and future research directions [3].

The uniqueness of the work is defined by the description of all the constituent elements, namely: data, identification and extraction of automatic standardization of specific medical terms, representation of medical knowledge, incorporation of medical knowledge labeling, description of deep learning (DL) architectures in relation to the objectives for which they were created and in correlation with the other constituent elements of the DL process, presentation of the applications for which they were constituted. Problems in the analysis of the medical image can be classified as follows: identification and extraction and automatic standardization of specific medical terms; representation of medical knowledge; incorporation of medical knowledge. Problems in medical image analysis are related to the following aspects: medical images provided as data for deep learning models require: quality, volume, specificity, labelling; the provision of data from doctors, descriptive data, labels are ambiguous for the same medical and non-standard references; laborious time in data processing are problems to solve in the future; lack of clinical trials demonstrating the benefits of using DL medical applications in reducing morbidity and mortality and improving patient quality of life [4].

In this paper, we aim to achieve an updated characterization of the specifics of the constituent elements of the deep learning process, scientific data, methods of incorporation of knowledge, DL models according to the objectives for which they were designed and presentation of medical applications according to these tasks. Secondly, we will describe the specific correlations between the quality, type and volume of data and their importance in achieving the performance of the deep learning models used in the interpretation of medical diagnostic images [3]. We will also make a structural and functional description of DL models and their applications in medicine.

A large number of medical images are stored in open access databases have private databases of some ceding institutions. These medical images are filed in connection with imaging reports or medical video image reports and, along with language processing from natural images, they have a great contribution to image analysis [5]. Annotation and labelling of the medical image, representing data from doctors, used through methods of integration into deep learning models, consumes time and requires specialized knowledge [3].

The large volume of training data and properly labeled determines the performance of the deep learning modeling in the interpretation of medical images [3]. Because manual image labelling requires time and specialized training, standardized, organized labelling has been used which has the risk of over-labeling with unnecessary information [2].

In medical image analysis the data types have a high variability and can be exemplified by image captures from different regions [7], different types of data included in a phase [8], different types of images [9], data from doctors have errors and require time for processing [10] small sample sizes [11].

A large number of medical images are stored in open access databases have private databases of some ceding institutions. These medical images are filed in connection with imaging reports or medical video image reports and, along with language processing from natural images, they have a great contribution to image analysis [12]. Annotation and labelling of the medical image, representing data from doctors, used through methods of integration into deep learning models, consumes time and requires specialized knowledge.

The large volume of training data and properly labeled determines the performance of the deep learning modeling in the interpretation of medical images. Because manual image labelling requires time and specialized training, standardized, organized labelling has been used which has the risk of over-labeling with unnecessary information [6].

In the absence of a large amount of data, the problem of over-assembly can be eliminated by adding abandonment. The deep learning model can have increased preformation in these conditions by optimizing a large number of hyper-parameters (size and number of filters, depth, learning rate, activation function, number of hidden layers, etc.) [1] [13]. In medical image analysis the data types have a high variability and can be exemplified by image captures from different regions [7], different types of data included in a phase [14], different types of images [9], data from doctors have errors and require time for processing [10], small sample sizes [15].

Computer-assisted diagnostics (CAD) in medical imaging and diagnostic radiology through the use of deep learning architectures has progressed to satisfactory results with multiple applications, namely, early detection and diagnosis of breast cancer, lung cancer, glaucoma and skin cancer [3] [16] [17] [18].

The types of images used in the analysis of medical images are: CT, MRI, X-ray, Ultra-sound, PET, Wave images, Biopsy, Mammography and Spectrography [1]. In the process of images analysis of the tasks of extracting characteristics, reducing size, augmentation, segmentation, grouping or classification are decisive for the efficiency and precision of integration methods [5] [14] [19] [20].

Larger datasets, compared to the small size of many medical datasets, result in better deep learning models [3] [21].

There are many large-scale and well-annotated data sets, such as ImageNet 1 (over 14 million images tagged in 20 k categories) and COCO 2 (with over 200 images annotated in 80 categories), medical datasets (open source), such as ChestX-ray14 and Deep-Lesion containing medical images tagged over 100 k, the others, contain only a few thousand or even hundreds of medical images [3] (Figure 1), and medical applications have developed properly in the medical fields.

The knowledge of experienced clinical-imaging physicians (radiologists, ophthalmologists and dermatologists, etc.) follows certain characteristics in images, namely, contrast, color, appearance, topology, shape, edges, etc., help and are used by deep learning models to perform the main tasks of medical image analysis [3].

The type and volume of medical data, the labels, the category of field knowledge and the methods of their integration into the DL architectures implicitly determine their performance in medical applications.

2. State of Arts

The current state of performance of deep learning models and architectures (DL) depends on the nature and quality of the data used in their training. This section shows the data types and DL model description and classification according to medical data types used, objectives and performances in medical applications.

2.1. Scientific Data and Dataset

We will further expose, the types of images and medical data used for diagnosis: natural images, medical images, High-level medical data (diagnostic pattern), low-level medical data (areas of images, disease characteristics), manual features used for medical image analysis.

Natural images—from natural datasets, ImageNet 1 (over 14 million images tagged in 20 k categories) and COCO 2 (with over 200 images annotated in 80 categories). Large natural images (ImageNet) are incorporated for the detection of objects in the medical field and are used in applications for the detection of lymph nodes [22], detection of polyp and pulmonary embolism [23], detection of breast tumors [24], detection of colorectal polyps [25] [26]. Natural Images, ImageNet, PASCAL VOC “static data” set, Sports-1M video datasets, which is the largest video classification indicator with 1.1 million sports videos in 487 categories [3] [27].

Medical images from external medical datasets of the same diseases in similar ways (e. g. SFM and DM) [28], medical images from external medical datasets of the same diseases [3] in different ways (DBT and MM, ultrasound) [29] or from different diseases [30]. Medical images are used in multiple applications. Multi-modal medical images, PET images are incorporated for the detection of lesions in CT scans of the liver [31]. Multimodal medical images are also used in another model in the detection of liver tumors [32]. Multimodal medical images (mammographic data) are used to detect breast masses [33]. Medical images, (CT, MRI, angio-CT, butt eye images), annotated retinal images, used to help segment the heart vessel without annotations [3] [34]. External medical data and images of other diseases, such as the union dataset (3DSeg-8) by aggregating eight sets [3] of 3D medical segmentation data [35].

Medical data from doctors: high-level medical data (diagnostic pattern) and low-level medical data (areas of images, disease characteristics). High-level and low-level medical data, i.e. anatomical aspects of the image, shape, position, typology of lesions integrated into segmentation tasks, example of the ISBI 2017 dataset used in skin injury segmentation. The use of additional medical datasets in different ways has also proven to be useful, although most applications are limited in using MRI to help segmentation tasks in CT images [3] [36]. Specific data identified by doctors (attention maps, hand-highlighted features) increase the diagnostic performance of deep learning networks (no comparative studies have been conducted). Medical data from doctors, handmade features, hand-crafted features, invariant LBP, as well as H & Components, are calculated first from the images [3]. The use of the BRATS2015 data set in applications in which these features are used is achieved performance in image segmentation by input-level fusion. However, anatomical priorities are only suitable for segmentation of fixed-shaped organs [3] such as the heart or lungs [35].

Manual features used for medical image analysis is a series of measurements (X-ray projections in CT or spatial frequency information in MRI). The methods based on deep learning have been widely applied in this area [37] [38]. Examples: image reconstruction with optical diffuse tomography (DOT), reconstruction of magnetic resonance imaging by compressed detection (CS-MRI) [39], reconstruction of the image with diffuse optical tomography (DOT) of limited-angle breast cancer and limited sources in a strong scattering environment [40], recovery of brain MRI images, target contrast using GAN. Content-based image recovery (CBIR) can be great help to for the clinicians to navigate these large data sets. Some deep learning methods [3] adopt transfer learning to use knowledge from natural images or external medical datasets [41] [42] [43], for example, metadata such as age and sex of patients, characteristics extracted from health areas, decision values of binary traits and texture traits in the process of thoracic X-ray recovery [3].

Medical data used to generate medical reports, subtitling medical images, templates from radiologist reports, visual characteristics of medical images, generating reports using the IU-RR dataset.

2.2. Addressing Label Noise in the Formation of Deep Learning Patterns in Medical Image Analysis

The noise of the label in the formation of deep learning models is important in their performance for medical image analysis. The approach of the label noise was achieved by: cleaning and pre-processing labels, improving the network architecture with noise layer, the endowment of networks with loss functions, data re-weighting, data and label consistency, training procedures.

Cleaning and pre-processing labels

In chest X-ray scans in the classification of thoracic diseases, the smoothing of labels was used to handle noisy labels and led to improvements of up to 0.08 in the area below the characteristic receptor operating curve (ASC) [44].

Network Architectures

In the case of network architectures, the noise layer proposed by [45] improved the accuracy in detecting breast lesions in mammograms.

Loss functions

The enhancement of networks with loss functions that cause annotations to dilate with a small and large structuring element to generate noisy masks for the foreground and background, e.g. parts of the ring union image were marked as unsafe regions that were ignored during training [46].

Re-weighting data

The method of re-weighting data to cope with noisy annotations in cancer detection was achieved by training models on a large group of noisy label patches using calculated features from a small set of clean label patches and increased model performance by 10%. [47]. This strategy was used to classify skin lesions in noisy label images [48], for segmentation of the heart, clavicles and lung in chest X-rays [10], for segmenting the skin lesion from highly inaccurate annotations [49] proposed a specific characteristic of pixels.

Consistency of data and labels

For segmentation of the left atrium in THE MRI from tagged and unlabeled data it was proposed to form two separate models: a teacher model that produced noisy labels and labeled maps with non-certainties on unlabeled images and a student model that was trained using the noisy labels generated, while taking into account the uncertainty of the label and making correct predictions on the clean data set in accordance with the teacher's model on the label, with uncertainty below the threshold.

Training procedures

For segmentation of the bladder, prostate and rectum in MRI, a model was trained on a clean label data set and used it to predict segmentation masks for a separate set of unlabeled data, and a second model was instructed to estimate a confidence map to indicate regions where predicted labels were more likely to be accurate and reliable paper used to sample the main model with a 3% improvement in the Dice similarity coefficient (DSC) [50]. A rather similar method has been used to classify aortic valve defects in MRI [51].

2.3. DL Model Description and Classification According to Medical Data Types Used, Objectives and Performances in Medical Applications

We will synthesize in Figure 2 classification of DL models according to the characteristics and tasks for which they were designed, classification of DL models according to the characteristics and tasks for which they were designed.

DL architectures can be divided into three categories: [1]

· Supervised

· Unsupervised

· Semi-supervised

Supervised DL models: [1]

· Recurent neural networks (RNN), short-term memory (LSTM), closed recurring unit (GRU),

· Convolutional neural networks (CNN) and

· Network of generational opponents (GAN).

Unsupervised deep learnirng models: [1]

· Deep Faith Networks (DBN),

· Deep Transfer Network (DTN),

· Tensor Deep Stack Networks (TDSN),

· Autoencoders (AE). [1]

2.3.1. Below We Describe the DL Models

CNN (convolutional neural network) are popular in areas where the shape of an object is an important feature, such as image analysis [4] [52] [53] [54] [55] [56], particularly in the study of cancers and bodily injuries in the medical sector [57] [58] and video analysis [4] [59].

CNN contains convolutive layers, grouping layers, dropout layers, and an output layer, hierarchically positioned that each learn stun specific characteristics in the image [14].

CNN in image analysis has low performance when high-resolution datasets are considered [60] and when localization over large patches is required, especially in medical images [61].

Image analysis performance is enhanced by the use of the following architectures: AlexNet, VGGNet and ResNet, YOLO or U-net that we describe below:

AlexNet was proposed by [58] [59] for the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2012 [4].

AlexNet consists of 8 layers, 5 layers of convolution and 3 dense, fully connected layers, overlapping overlay, abandonment, data augmentation, ReLU activations after each convolutive layer and fully connected, SGD with impulse [1] [62]. AlexNet is used for image recognition in image analysis and is usually applied to issues involving semantic segmentation and high-resolution data classification tasks [63] [64].

VGG (Visual Geometry Group): Consists of 13 convolution layers (in VGG16) & 16 convolution layers (in VGG19), 3 dense layers, pooling and three RELU units, very small responsive fields [1] [65]. VGG is used for object recognition, classification of medical images [66] [67] and image segmentation [68]. VGG loses accuracy when the depth becomes too high.

ResNet (Residual Neural Network): Contains closed units or closed recurring units and has a strong similarity to recent successful elements applied in RNNs [1]. ResNet is characterized by: residual mapping, identity function, and a two-layer residual block, one layer learns from the residue, the other layer learns from the same function and has high level of performance in image classification (Saravanan et al., Saravanan) and audio analysis tasks [4] [69].

GoogLeNet is built from 22 deep LAYERS CNN and 4 million parameters and contains several layer filters and stacked convolution layers [70] It was used for batch normalization, image distortions, and RMSprop [1].

U-Net, developed by Ronneberger [4] [61], addresses the problem of locating images of a standard CNN by extracting data features followed by reconstruction of the original dimension through an up-sampling operation. U-Net is a type of Enconder-Decoder network in which the codoficator output belongs to the input space [4]. U-Net is used in single-stage segmentation and classification [71], specifically in the locatio;n of cancerous lesions [72] [73] [74]. SegNet [75] is a U-Net variant that uses maximum grouping indices in the upsampling step that reduces the complexity of U-Net space.

RNNs were developed by Rumelhart et al. [4] [76] using with efficiency the correlations existing between input data of a prediction problem, through which they process sequential data in relation to text analysis [77] [78] [79], in electronic medical records to predict diseases [4] [80] [81] and speech recognition [82]. RnN variants are: one-way, learning from the past and predicting the future and bidirectional that uses the future to restore the past. RNN has the following variants: short-term memory (LSTM) and closed recurring units (GRU), recursive neural networks (Recursive NNs), two-way RNNs (BiRNN). Short-term memory LSTMs were introduced by [4] [67] [83] and consist of: the gate of oblivion that alleviates the escape and explosion gradient, the entrance gate and the exit gate, the last two track the flow of data coming in and out of the cell. They were used in speech recognition [84], path prediction [85] and medical diagnosis [86], in which the authors proposed an LSTM network, called DeepCare, combining different types of data to identify clinical diseases.

GURs (recurring unit gated) created by [87] [88] solve the problem of increasing the time complexity of LSTM, when large amounts of data are used [4]. The GRU consists of a reset gate in which it is decided how much information from the past is transmitted in the future, and an update gate that decides how much information from the past can be forgotten. GRU and LSTMs have similar applications especially in speech recognition [89].

The two-way recurring neural network and the Boltzmann BRNNs [4] introduced by [90] [91] are characterized by the fact that the hidden state is updated by using past information, as in a classic RNN, and by using information related to future moments [4]. They were applied in handwriting and speech recognition, where they are used to detect missing parts of a sentence in a knowledge of the other words [92] [93].

BM models, introduced by [94] [95], are a family of RNNs that are easy to implement and that reproduce many probability distributions, BMs are used in image classification [4]. BMs combined with other models are used to locate objects, [96] [97]. In the classification of images, BMs are used to identify the presence of a tumor [98]. BM models are slow and ineffective when the data size increases exponentially due to the complete connection between neurons [99]. A restricted BM was proposed in which relaxing the connections between neurons of the same or one-way connection between neurons would solve the problem of the classic BM model [100].

AEs, developed by [101] [102], consisting of encoder and decoder, with the aim of reducing the size of the data through significant representations and learning data characteristics for the reconstruction of outputs. They are used in applications in medical image analysis [4] [103], natural language processing [104] and video analysis [105].

Additional variants of AE that can be found in the literature are variational AE (VAE). In a VAE, the encoder is represented by the probability density function of the input into the feature space and, after the encoding stage, a sampling of the new data using the PDF is added. Differently from the DAE and the SAE, a VAE is not a regularized AE, but is part of the generation class [4].

GAN it is used to generate synthetic training data from original data using latent distribution [1] [106]. It consisted of two networks, a generator estimates false data from input data, and a discriminator, which differentiates fake data from real data and separates it in order to increase the quality of the data generated. GAN has two problems: the problem of the collapse of the mode, and the fact that, can become very unstable.

The DBN (Deep Network of Beliefs), created by Hinton [107], consists of two networks that build each other: of beliefs represented by an acyclic graph composed of layers of stochastic binary units with weighted and respectively weighted connections, restricted Boltzmann Machines which is a stochastic [1]. DBNs are applied in image recognition and speech recognition, in classification to detect lesions in medical diagnosis and, in video recognition to identify the presence of persons [108], in speech recognition to understand missing words in a sentence [109] and in application on physiological signals to recognize human emotion [110].

DTN contains a characteristic extraction layer, which teaches a shared feature subspace in which marginal source distributions and target samples are drawn close and a layer [1] of discrimination that match conditional distributions by classified transduction [111]. It is used for large-scale problems [1].

TDSN contains two parallel hidden representations that are combined using a bilinear mapping [1] [112]. This arrangement provides better generalization compared to the architecture of a single module. The prejudices of the generalisers with regard to the learning set shall be inferred. It works effectively and better than an eco-validation strategy when used with multiple generalisers compared to individual generalisers [1].

Deep InfoMax (DIM): Maximizes mutual information between an input and output of a highly flexible convolutive encoder [113] by forming another neural network that maximizes a lower limit on a divergence between the marginal product of encoder input and output. Estimates obtained by another network can be used to maximize the reciprocal information of the features in the input encoder. The memory requirement of the DIM is lower because it requires only encoder not decoder [1].

2.3.2. Combinations of Different DL Models Depending on the Type of Data Involved in the Problem to Be Solved

DL models can be combined in five different ways depending on the type of data involved in the problem to be solved [1] [4]. Of these, three types of HA (hybrid architectures), namely the integrated model, the built-in model and the whole model.

In the integrated model, the output of the convolution layer is transmitted directly as input to other architectures to the residual attention network, the recurrent convolutive neural network (RCNN) and the model of the recurrent residual convolutive neural network (IRRCNN) [114].

In the built-in model (the improved common hybrid CNNBiLSTM), the size reduction model and the classification model perform together, the results of one represent the inputs for the other model. In the model (EJH-CNN-BiLTM), several basic models are combined.

In the learning transfer model (TL) is trained and uses the same type of problem. CNN models that use the TL model are VGG (e.g. VGG16 or VGG19), GoogLeNet (e.g. InceptionV3), Inception Network (Inception-v4), Repiuled Neural Network (e.g. ResNet50), AlexNet. Joint AB based DL combines max pooling, and careful sharing [1].

2.3.3. Combinations of Different DL Models to Benefit from the Characteristics of Each Model with Medical Applications Are: CNN + RNN, AE + CNN and GAN + CNN

CNN + RNN are used for the capabilities of the CNN feature extraction model and the RNNs [15]. Because the result of a CNN is a 3D value and an RNN works with 2D-data, a remodeling layer is, associated between CNN and RNN, to convert THE production of CNN into an array [4]. CNN + RNN have been successfully applied in text analysis to identify missing words [115] and image analysis to increase the speed of magnetic resonance image storage [116] [117]. CNN + RNN variants are obtained by replacing the Standard RNN component [4] with an LSTM component [24] [118].

The AE + CNN architecture combines AE as a pre-training model when using data with high noise levels, and a CNN as a feature extractor model [4]. AE + NVs have an application in image analysis to classify noisy medical images [119] and in the reconstruction of medical images [120] [121].

GAN + CNN combines GAN as a pre-workout model to moderate the problem of over-mounting, and a CNN, used as a feature extractor [4]. It has applications in image analysis [11] [122].

The DL architectures applied especially in image analysis are CNN, AE and GAN. NVs preserve the spatial structure of the data, and are used as feature extractors (especially U-Net), AEs reduce the characteristics of complex images in the analysis process, and GANs are pre-training architectures that select input categories to control overfitting.

2.4. Applications in Medicine and the Performance of DL Models Depending on the Therapeutic Areas in Which They Were Used

We further highlight the acquisitions in the study of deep learning and its applications in the analysis of the medical image, between 2017 and 2020 [4]. You can easily identify references to image labeling and annotation, developing new deep learning models with increased performance, and new approaches to medical image processing:

· diagnosis of cancer by using CNN with different number of layers [123],

· studying deep learning optimization methods and applying in the analysis of medical images [124],

· development of techniques used for endoscopic navigation [125],

· highlighting the importance of data labelling and annotation and knowledge of model performance [126] [127],

· perfecting the layer-wise architecture of convolution networks [1], lesson the cost and calculation time for processor training [128],

· description of the use of AI and its applications in the analysis [1] of medical images [129],

· diagnosis in degenerative disorder using depp learning techniques [130] and,

· detection of cancer by processing medical images using the medium change filter technique [131],

· classification of cancer using histopathological images and highlighting the rapidity of Theano, superior tensor flow [131],

· development of two-channel computational algorithms using DL (segmentation, extraction of characteristics, selection of characteristics and classification and classification, extraction of high-level captures respectively) [132],

· malaria detection using a deep neural network (MM-ResNet) [8].

We will exemplify in Table 1 [2] applications in medicine and the performance of DL models depending on types of medical images and the therapeutic areas in which they were used. We included most relevant papers about the most used medical investigations, respectively medical images.

Table 1 Applications and the performances of the DL models depending on the types of medical images and the therapeutic area [<xref ref-type="bibr" rid="scirp.110047-ref2">2</xref>]. Acronyms: AMD, age-related Macular Degeneration, CAD, Computer Aided Diagnosis, CNN, Con-volutional Neural Network, MRI, Magnetic Resonance Images, PET, Photon Emission Tomogra-phy, CT, Computed Tomography, OCT, Optical Coherence Tomography, D, dimensions, AUC, Area Under the Curve, MSE, Mean Squared Error, RMSE, Root Mean Square Error, DSC, Dice Similarity Coefficient [<xref ref-type="bibr" rid="scirp.110047-ref2">2</xref>]

Imaging		Number of Images	Type	Purpose	Name Datasets
Multiple		1921 patients	Brain	Classification	ADNI
MRI		539 patients	Brain	Classification	ABIDE
MRI		150 patients	Cardiac	Classification	ACDC
X-ray		112,120 images - 30,805 patients	Chest	Detection	Chest X-ray14
CT	X-ray	1018 patients	Lung	Detection	LIDC-IDRI
CT		888 images	Lung	Detection	LUNA16
X-ray		40,895 images - 14,982 patients	Musculo- skeletal	Detection	MURA
MRI		542 images	Brain	Segmentation	BraTS2018
SLO		400 images	Eye	Segmentation	STARE
Mammography		2500 patients	Breast	Classification Detection	DDSM
CT		32,735 images - 4427 patients	Multiple	Classification Detection	Deep-Lesion
MRI		7980 images - 33 cases	Cardiac	Classification Segmentation	Cardiac MRI
Dermoscopy		13,000 images	Skin	Classification Detection Segmentation	ISIC 2018

	Type of Data	Sample	Objective	Model Design	Results	Therapeutic Area	Paper
Mammography	Mammography images	45,000 images	Detect malign solid lesions and prevent overtreatment in false positives [2]	CNN	AUC of 0.90	Oncology	[89]
	Mammography	667 benign and 333 malignant	Mammography diagnosis of early malignant breast	Stacked AE	Accuracy of 0.89	Oncology	[96]
	Digital Mammography images and the biopsy result of the lesions [2]	1000 malignant masses and 600 cysts images and their biopsy [2]	Discriminate benign cysts from malignant masses	CNN	AUC of 0.80	Oncology	[97]
	Mammography images	840 images of mammograms from 210 different patients	Breast arterial calcification on mammograms classifier to evaluate the risk of coronary disease [2]	CNN	Misclassfied cases of 6%	Cardiovascular	[101]
	Digital mammograms	661 from 444 patients	Computer automated estimation of breast percentage density [2]	CNN	AUC of 0.981	Oncology	[151]
	Mammography images	Mammograms from 604 women	Segment areas of dense fibroglandular tissue in the breast [2]	CNN	Accuracy of 0.66	Oncology	[116]
	Digital mammograms images	29,107 left mediolateral oblique, right mediolateral oblique, left cranial-caudal and right cranial-caudal mammograms images	Probability of cancer on mammograms [2]	CNN	AUC of 0.90	Oncology	[121]
Ultrasound	Image of the heart 2D	400 images with five different heart diseases and 80 normal echocardiogram images	Segment left ventricle images with greater precision	Deep belief networks	Hammoude distance of 0.80	Cardiovascular	[152]
	Ultrasound imaging	306 malignant and 136 benign tumors images	CAD system to detect and differentiate breast lesions with ultrasound	CNNs inspired in AlexNet, U-Net and LeNet	Best F-measure of 0.91 and 0.89 depending on the data	Oncology	[2] [24]
	Transesophageal ultrasound volume and 3D geometry of the aortic valve images	3795 volumes from the aortic valves from 150 patients	Diagnose, stratification and treatment planning for patients with aortic valve pathologies	Marginal space deep learning	Position error of 1.66 mms and mean corner distance error of 3.29 mms	Cardiovascular	[2] [84]

Radiography	Radiography images	7821 subjects with 6 monitoring phases	CAD for diagnosis of knee osteoarthritis	Deep Siamese	Accuracy of 0.66	Traumatology	[115]
	Radiography images	420 radiography images (219 control group, 201 ostearthritis) [2]	Radiographies CAD for hip osteoarthritis diagnosis	CNN	Accuracy of 0.92	Traumatology	[153]
	Radiography images	112,120 frontal view chest radiographs from 30,805 patients and 17,202 frontal view chest radiographs with a binary class label for normal vs abnormal	Abnormality detection in chest radiographs	CNN	AUROCs of 0.960 and 0.951. AUROCs of 0.900 and 0.893	Radiology	[145]
Slide image	Pathology cancer images (hematoxylin and eosin)	5202 images tumor infiltrating lymphocytes	Study of tumor tissue samples. Localize areas of necrosis and lymphocyte infiltration	Two CNNs	AUC of 0.95	Oncology	[2] ; [76]
	Giemsa-stained thin blood smear slides cell images	27,558 cell images 150 infected and 50 healthy patients	Create a screening system for Malaria	CNN	Accuracy of 0.94	Infectious Disease	[80]
	Microscopy image patches	249 images belonging to 20 histologic categories	Classification of breast cancer histology microscopy images	CNN with a Support Vector Machine (SVM)	Accuracy of 0.77 for four class classification and an accuracy of 0.83 for carcinoma /noncarcinoma classification	Oncology	[109]
	Microscopy histopathological images	7909 images of eight subclasses of breast cancers	CAD for breast cancer histopathological diagnosis	CNN	Accuracy of 0.93	Oncology	[110]
	Microscope images	200 female subjects aged from 22 to 64	Cervix cancer screening	Multiscale CNN	Mean and standard deviation of 0.95 and 0.18	Oncology	[122]
	Whole-slide prostate histopathology images	2663 images from 32 whole slide prostate histopathology images	Whole-slide histopathology images to outline the malignant regions	CNN	Dice coefficient of 0.72	Oncology	[2] ; [154]
Ocular fundus	2D Ocular fundus images	243 retina images	Diagnose retinal lesions	CNN	Precision recall curve of 0.86 in microaneurysms and 0.64 in exudates	Ophthalmology	[2] ; [78]
Ocular fundus	Ocular fundus images 2D	Over 85,000 images	Diabetic retinopathy detection and stage classification	Bayesian CNN	AUC value of 0.99	Ophthalmology	[88]

	Color ocular fundus images	6679 random sampling images from Kaggle’s Diabetic Retinopathy Detection	Detect retinal hemorrhages	CNN	AUC of 0.894 and 0.972	Ophthalmology	[95]
	Ocular fundus images	168 images with glaucoma and 428 control	System to detect and evaluate glaucoma	CNN: ResNet and U-Net	AUC of 0.91 and 0.84 respectively	Ophthalmology	[98]
	Ocular fundus images	90,000 images with their diagnoses	Predict the evolution of diabetic retinopathy with fundus images	CNN	AUC of 0.95	Ophthalmology	[155]
	Fundus images	7000 colour fundus images	Image quality in the context of diabetic retinopathy	CNN	Accuracy of 100%	Ophthalmology	[156]
	AREDS (age related eye disease study) image	130,000 fundus images	Diagnosis of Age-related Macular Degeneration	CNN	94.97 sensitivity and 98.32% specificity	Ophthalmology	[157]
	Fundus images	219,302 from normal participants without hypertension, diabetes mellitus (DM), and any smoking history	Predict age and sex from retinal fundus images	CNN	AUC 0.96	Ophthalmology	[2] ; [158]
Dermoscopy	Dermoscopy images	350 images of melanomas and 374 benign nevus	Dermoscopy CAD system for acral lentiginous melanoma diagnosis	CNN	Accuracy of over 0.80	Oncology	[2] ; [99]
	Patient demographics and clinical images	49,567 images	Recognize nails nychomycosis lesions	Region-based- CNN	AUC of 0.98, AUC of 0.95, AUC of 0.93, AUC value of 0.82 in the different datasets	Dermatology	[120]
	Stress 99mTc-sestamibi or tetrofosmin myocardial perfusion images	1638 patients	Obstructive coronary disease automatic prediction system	CNN	Sensitivity value of 0.82 and 0.69 for both use cases	Cardiovascular	[159]
Arterial labeling	Arterial spin labeling (ASL) perfusion images	140 subjects	Monitoring cerebral arterial perfusion via spin labeling	CNN	AUC of 0.94	Cardiovascular	[91]
Frames from endoscopy	Frames from endoscopy videos	205 normal and 360 abnormal images	Detection and localization system of gastrointestinal anomalies via endoscopy	CNN	AUC of over 0.80	Gastroenterology	[103]

Tracking dataset multi-instrument Endo-Visceral Surgery and multi-instrument in vivo	Single-instrument Retinal Microsurgery Instrument Tracking dataset, Multi-instrument Endo-Visceral surgery and multi-instrument in vivo images	940 frames of the training data (4479 frames) and 910 frames for the test data (4495 frames)	Detect the two-dimensional position of different medical instruments in endoscopy and microscopy surge	Convolutional Detection regression network	Accuracy of 0.94	Robotic Surgery	[2] : [119]
CT/PET- CT/SPECT	Nuclear MRIs 3D	124 double echo steady state from 17 patients	Diagnose possible soft tissue injuries	DeepResolve, a 3D-CNN model	MSE of 0.008	Traumatology	[2] [160]
	Retinal 3D images obtained by Optical Coherence Tomography	269 patients with AMD, 115 control patients	Retina age-related macular degeneration diagnostic	CNN	AUC of 0	Ophthalmology	[77]
	123I-fluoropropyl carbomethoxy-iodophenyl nortropane single-photon emission computed tomography (FP-CIT SPECT) 2D images	431 patient cases	Automatic interpretation system in Parkinson’s disease	CNN	Accuracy of 0.96	Neurology- Psychiatry	[79]
	Abdominal CT 3D images	231 computed abdominal	CAD system to classify tomographyes and evaluate the malignity degree in gastro-intestinal stromal tumors (GISTs)	Hybrid system between convolutional networks and radiomics	AUC of 0.882	Oncology	[161]
	CT image patches 2D	14,696 images from 120 patients with proven diagnose	CAD system to diagnose interstitial lung disease	CNN	Accuracy of 0.85	Pneumology	[85]
	3D MRI and PET	93 Alzheimer Disease, 204 MCI Mild Cognitive Impairment converters and normal control subjects	CAD for early Alzheimer disease stages	Multimodal DBM	Accuracy of 0.95, 0.85 and 0.75 for the three use cases	Neurology- Psychiatry	[2] ; [92]
CT/PET- CT/SPECT	CT images, MRI images and PET images	6776 images for training and 4166 for tests	Classify medical diagnostic images according to the modality they were produced and classify illustrations according to their production attributes	CNN and a synergic signal system	Accuracy of 0.86	Various	[2] [108]

	CT image 2D	63,890 patients with cancer and 171,345 healthy	Discriminate lung cancer lesions in adenocarcinoma, squamous and small cell carcinoma	CNN	Log-Loss error of 0.66 with a sensitivity of 0.87	Oncology	[118]
	CT 2D images	3059 images from several parts of human body	Speed up CT images collection and rebuild the data	Dense Net and a deconvolution model	RMSE of 0.00048	Various	[11]
	CT images 3D	6960 lung nodule regions, 3480 of which were positive samples and rest were negative samples (nonnodule)	CAD to diagnose lung cancer in low-dosage computed tomography	Eye tracking sparse attentional model and convolutional neural network	Accuracy of 0.97	Oncology	[162]
	CT images 2D and text (reports)	9000 training and 1000 testing images	Processing text from CT reports in order to classify their respective images	CNN	Accuracy of 0.95, 0.70 and 0.58 respectively for the three use cases	Various	[12]
	Computed tomography (CT)	Three datasets: 224,316, 112,120 and 15,783	Binary classification of posteroanterior chest xray	CNN	92% accuracy	Radiology	[2] [163]
MRI	Diffusion- weighted imaging maps using MRI	222 patients. 187 treated with rtPA (recombinant tissue-type plasminogen activator)	Decide Acute Ischemic Stroke patients’ treatment through volume lesions prediction	CNN	AUC of 0.88	Neurology- Psychiatry	[2] ; [81]
	Magnetic resonance images	474 patients with schizophrenia and 607 healthy subjects	Schizophrenia detection	Deep discriminant autoencoder network	Accuracy over 0.8	Neurology- Psychiatry	[83]
	Gadoxetic acid-enhanced 2D MRI	144,180 images from 634 patients	Staging liver fibrosis through MR	CNN	AUC values of 0.84, 0.84, and 0.85 for each stage	Gastroenterology	[86]
	Resting state functional magnetic resonance imaging (rs-fMRI), T1 structural cerebral images and phenotypic information	505 individuals with autism and 520 matched typical controls	Identify different autism spectrum disorders	Denoising AE	Accuracy of 0.70	Neurology- Psychiatry	[92]
	3D MRI and PET	93 Alzheimer Disease, 204 MCI Mild Cognitive Impairment converters and normal control subjects	CAD for early Alzheimer disease stages	Multimodal DBM	Accuracy of 0.95, 0.85 and 0.75 for the three use cases	Neurology- Psychiatry	[2] ; [93]

3. Conclusions

Doctors interpret images descriptively (contour, contrast, appearance, localization, etc.) by using data from different excipients and successive stages in the analysis of medical images. These handcrafted features consume time and do not have a standardized character.

Data quality and volume, annotations and labels, identification and automatic extraction of specific medical terms can help deep learning models perform in the tasks of image analysis [3]. Incorporating these features, labels, into DL architectures increases their performance.

High-level domain knowledge is incorporated as input images [3], and low-level domain knowledge is learned using specific network structures [35] and, together with direct networking, low-level domain knowledge information can also be used to design training commands when combined with the easy-to-use training model [3] [133].

DL can be a support in solving complex problems of interpretation of medical images and provides the doctor with support in making medical decisions and time for patient care.

4. Research Problems

Problems in medical image analysis can be categorized as follows:

· identification and automatic extraction and standardization of specific medical terms,

· representation of medical knowledge,

· incorporation of medical knowledge.

Problems in medical image analysis are related to:

· medical images provided as data for deep-street models require: quality, volume, specificity, labelling.

· providing data from doctors, descriptive data, labels are ambiguous for the same medical and non-standard references.

· laborious time in data processing are problems to solve in the future.

· lack of clinical trials demonstrating the benefits of using DL medical applications in reducing morbidity and mortality and improving patient quality of life [4].

5. Future Challenges

These consist of adapting the domain consisting of transferring data from one domain to another domain by using labels; knowledge graph characterized by the incorporation of multimodal medical data; generating models capable of extracting features unsupervised and easily incorporated into the architecture of DL networks; techniques to search for a particular network architecture according to the defined objectives.

The adaptation of the domain consisted of transferring information from a source domain to a target domain [3], such as adversarial learning [134], and makes it restrict the domain change between source and target [3] domain in input space [135], feature space [136] [137] and output space [138] [139]. It can be used to transfer knowledge about one set of medical data to another [3] [140], even when they have different modes [3] of imaging or belong to different diseases [141] [142]. UDA (unsupervised domain adaptation), which uses medical labels, has demonstrated performance in disease diagnosis and organ segmentation [3] [140] [143] [144] [145].

The knowledge graph has the specifics of incorporating multimodal medical data and achieves performance in medical image analysis [3] and the creation of medical reports [146]. The graphs of medical knowledge describing, the relationship between different types of knowledge, the relationship between different diseases, the relationship between medical datasets and a type of medical data, help deep learning models work [147].

Generating models, GAN and AE are mainly used for segmentation activities. GAN uses MRI datasets to segment CT images [142] [143]. GAN is a type of unsupervised deep learning network used in medical image analysis [3] [167]. AE are used in extracting features, shape priorities in objects such as organs or lesions, completely unsupervised and are easily incorporated into the network formation process [35] [148].

Network Architecture Search Technique (NAS) can automatically identify a specific network architecture in computer tasks [149] and promises that utility and performance in the medical field [150].

Funding

Scientific research funded by the University of Medicine and Pharmacy “Gr. T. Popa” of Iasi, based on contract number 4714.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

Cite this paper

Ursuleanu, T.F., Luca, A.R., Gheorghe, L., Grigorovici, R., Iancu, S., Hlusneac, M., Preda, C. and Grigorovici, A. (2021) Unified Analysis Specific to the Medical Field in the Interpretation of Medical Images through the Use of Deep Learning. E-Health Telecommunication Systems and Networks, 10, 41-74. https://doi.org/10.4236/etsn.2021.102003

References1

Pandey, B., Pandey, D.K., Mishra, B.P. and Rhmann, W. (2021) A Comprehensive Survey of Deep Learning in the Field of Medical Imaging and Medical Natural Language Processing: Challenges and Research Directions. Journal of King Saud University—Computer and Information Sciences, in press. https://doi.org/10.1016/j.jksuci.2021.01.007

Nogales, A., García-Tejedor, á.J., Monge, D., Vara, J.S. and Antón, C. (2021) A Survey of Deep Learning Models in Medical Therapeutic Areas. Artificial Intelligence in Medicine, 112, Article ID: 102020. https://doi.org/10.1016/j.artmed.2021.102020

Xie, X.Z., Niu, J.W., Liu, X.F., Chen, Z.S., Tang, S.J. and Yu, S. (2021) A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis. Medical Image Analysis, 69, Article ID: 101985. https://doi.org/10.1016/j.media.2021.101985

Piccialli, F., Di Somma, V., Giampaolo, F., Cuomo, S. and Fortino, G. (2021) A Survey on Deep Learning in Medicine: Why, How and When? Information Fusion, 66, 111-137. https://doi.org/10.1016/j.inffus.2020.09.006

Shin, H.-C., Lu, L. and Summers, R.M. (2017) Chapter 17. Natural Language Processing for Large-Scale Medical Image Analysis Using Deep Learning. In: Zhou, S.K., Greenspan, H. and Shen, D.G., Eds., Deep Learning for Medical Image Analysis, Academic Press, Cambridge, MA, 405-421. https://doi.org/10.1016/B978-0-12-810408-8.00023-7

Wang, X., Yang, X., Dou, H.R., Li, S.L., Heng, P. and Ni, D. (2019) Joint Segmentation and Landmark Localization of Fetal Femur in Ultrasound Volumes. IEEE EMBS International Conference on Biomedical & Health Informatics (BHI), Chicago, IL, 19-22 May 2019, 1-5. https://doi.org/10.1109/BHI.2019.8834615

Sharma, S. and Mehra, R. (2020) Conventional Machine Learning and Deep Learning Approach for Multi-Classification of Breast Cancer Histopathology Images—A Comparative Insight. Journal of Digital Imaging, 33, 632-654. https://doi.org/10.1007/s10278-019-00307-y

Pattanaik, P.A., Mittal, M., Khan, M.Z. and Panda, S.N. (2020) Malaria Detection Using Deep Residual Networks with Mobile Microscopy. Journal of King Saud University—Computer and Information Sciences, in Press. https://doi.org/10.1016/j.jksuci.2020.07.003

He, Y.T., Yang, G.Y., Chen, Y., Kong, Y.Y., Wu, J.S., et al. (2019) DPA-DenseBiasNet: Semi-Supervised 3D Fine Renal Artery Segmentation with Dense Biased Network and Deep Priori Anatomy. In: Shen, D., et al., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science, Vol. 11769, Springer, Cham, 139-147. https://doi.org/10.1007/978-3-030-32226-7_16

Zhu, Y., Wang, M.D., Tong, L. and Deshpande, S.R. (2019) Improved Prediction on Heart Transplant Rejection Using Convolutional Autoencoder and Multiple Instance Learning on Whole-Slide Imaging. IEEE EMBS International Conference on Biomedical and Health Informatics, Chicago, IL, 19-22 May 2019, 1-4. https://doi.org/10.1109/BHI.2019.8834632

Zhang, Z., Liang, X., Dong, X., Xie, Y. and Cao, G. (2018) A Sparse-View CT Reconstruction Method Based on Combination of DenseNet and Deconvolution. IEEE Transactions on Medical Imaging, 37, 1407-1417. https://doi.org/10.1109/TMI.2018.2823338

Shin, H.-C., Lu, L., Kim, L., Seff, A., Yao, J.H. and Summers, R.M. (2016) Interleaved Text/Image Deep Mining on a Large-Scale Radiology Database for Automated Image Interpretation. Journal of Machine Learning Research, 17, 1-31.

Ravi, D., Wong, C., Deligianni, F., Berthelot, M., Andreu-Perez, J., Lo, B. and Yang, G.Z. (2017) Deep Learning for Health Informatics. IEEE Journal of Biomedical and Health Informatics, 21, 4-21. https://doi.org/10.1109/JBHI.2016.2636665

Vizcarra, J., Place, R., Tong, L., Gutman, D. and Wang, M.D. (2019) Fusion in Breast Cancer Histology Classification. Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, Niagara Falls, NY, 7-10 September 2019, 485-493. https://doi.org/10.1145/3307339.3342166

Zhao, Y., Dong, Q., Zhang, S., Zhang, W., Chen, H., Jiang, X., Guo, L., Hu, X., Han, J. and Liu, T. (2018) Automatic Recognition of fMRI-Derived Functional Networks Using 3-D Convolutional Neural Networks. IEEE Transactions on Biomedical Engineering, 65, pp1975-1984. https://doi.org/10.1109/TBME.2017.2715281

Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M. and Thrun, S. (2017) Dermatologist-Level Classification of Skin Cancer with Deep Neural Networks. Nature. Nature, 542, 115-118. https://doi.org/10.1038/nature21056

Zhou, H., Sun, J., Yacoob, Y. and Jacobs, D.W. (2018) Label Denoising Adversarial Network (LDAN) for Inverse Lighting of Faces. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 6238-6247. https://doi.org/10.1109/CVPR.2018.00653

Zhu, W.T., Liu, C.C., Fan, W. and Xie, X. (2018) DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification. 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, 12-15 March 2018, 673-681. https://doi.org/10.1109/WACV.2018.00079

Zhang, R., Zheng, Y., Poon, C.C.Y., Shen, D. and Lau, J.Y.W. (2018) Polyp Detection during Colonoscopy Using a Regression-Based Convolutional Neural Network with a Tracker. Pattern Recognition, 83, 209-219. https://doi.org/10.1016/j.patcog.2018.05.026

Cheimariotis, G.A., Riga, M., Toutouzas, K., Tousoulis, D., Katsaggelos, A. and Maglaveras, N. (2019) Deep Learning Method to Detect Plaques in IVOCT Images. In: Lin, K.-P., Magjarevic, R. and de Carvalho, P., Eds., Future Trends in Biomedical and Health Informatics and Cybersecurity in Medical Devices. ICBHI 2019. IFMBE Proceedings, Vol. 74, Springer, Cham, 389-395. https://doi.org/10.1007/978-3-030-30636-6_53

Halevy, A., Norvig, P. and Pereira, F. (2009) The Unreasonable Effectiveness of Data. IEEE Intelligent Systems, 24, 8-12. https://doi.org/10.1109/MIS.2009.36

Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., et al. (2016) Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning. IEEE Transactions on Medical Imaging, 35, 1285-1298. https://doi.org/10.1109/TMI.2016.2528162

Tajbakhsh, N., Shin, J.Y., Gurudu, S.R., Hurst, R.T., Kendall, C.B., Gotway, M.B. and Liang, J.M. (2016) Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning. IEEE Transactions on Medical Imaging, 35, 1299-1312. https://doi.org/10.1109/TMI.2016.2535302

Yap, M.H., Pons, G., Marti, J., Ganau, S., Sentis, M., Zwiggelaar, R., Davison, A.K., Marti, R., Yap, M.H., Pons, G., Marti, J., Ganau, S., Sentis, M., Zwiggelaar, R., Davison, A.K. and Marti, R. (2018) Automated Breast Ultrasound Lesions Detection Using Convolutional Neural Networks. IEEE Journal of Biomedical and Health Informatics, 22, 1218-1226. https://doi.org/10.1109/JBHI.2017.2731873

Näppi, J.J., Hironaka, T., Regge, D. and Yoshida, H. (2016) Deep Transfer Learning of Virtual Endoluminal Views for the Detection of Polyps in CT Colonography. Medical Imaging 2016: Computer-Aided Diagnosis, 9785, 97852B. https://doi.org/10.1117/12.2217260

Zhang, R., Zheng, Y., Mak, T.W., Yu, R., Wong, S.H., Lau, J.Y. and Poon, C.C. (2017) Automatic Detection and Classification of Colorectal Polyps by Transferring Low-Level CNN Features From Nonmedical Domain. IEEE Journal of Biomedical and Health Informatics, 21, 41-47. https://doi.org/10.1109/JBHI.2016.2635662

Tran, D., Bourdev, L., Fergus, R., Torresani, L. and Paluri, M. (2015) Learning Spatiotemporal Features with 3D Convolutional Networks. IEEE International Conference on Computer Vision (ICCV), Santiago, 7-13 December 2015, 4489-4497. https://doi.org/10.1109/ICCV.2015.510

Samala, R.K., Chan, H.P., Hadjiiski, L.M., Helvie, M.A., Cha, K.H. and Richter, C.D. (2017) Multi-Task Transfer Learning Deep Convolutional Neural Network: Application to Computer-Aided Diagnosis of Breast Cancer on Mammograms. Physics in Medicine & Biology, 62, 8894-8908. https://doi.org/10.1088/1361-6560/aa93d4

Samala, R.K., Chan, H.-P., Hadjiiski, L., Helvie, M.A., Richter, C.D. and Cha, K.H. (2019) Breast Cancer Diagnosis in Digital Breast Tomosynthesis: Effects of Training Sample Size on Multi-Stage Transfer Learning Using Deep Neural Nets. IEEE Transactions on Medical Imaging, 38, 686-696. https://doi.org/10.1109/TMI.2018.2870343

Liao, Q., Ding, Y., Jiang, Z.L., Wang, X., Zhang, C.K. and Zhang, Q. (2019) Multi-Task Deep Convolutional Neural Network for Cancer Diagnosis. Neurocomputing, 348, 66-73. https://doi.org/10.1016/j.neucom.2018.06.084

Ben-Cohen, A., Klang, E., Raskin, S.P., Soffer, S., Ben-Haim, S., Konen, E., Amitai, M.M. and Greenspan, H. (2019) Cross-Modality Synthesis from CT to PET Using FCN and GAN Networks for Improved Automated Lesion Detection. Engineering Applications of Artificial Intelligence, 78, 186-194. https://doi.org/10.1016/j.engappai.2018.11.013

Zhao, J., Li, D., Kassam, Z., Howey, J., Chong, J., Chen, B. and Li, S. (2020) Tripartite-GAN: Synthesizing Liver Contrast-Enhanced MRI to Improve Tumor Detection. Medical Image Analysis, 63, Article ID: 101667. https://doi.org/10.1016/j.media.2020.101667

Zhang, J., Saha, A., Zhu, Z. and Mazurowski, M.A. (2019) Hierarchical Convolutional Neural Networks for Segmentation of Breast Tumors in MRI with Application to Radiogenomics. IEEE Transactions on Medical Imaging, 38, 435-447. https://doi.org/10.1109/TMI.2018.2865671

Yu, F., Zhao, J., Gong, Y., Wang, Z., Li, Y., Yang, F. and Zhang, L. (2019) Annotation-Free Cardiac Vessel Segmentation via Knowledge Transfer from Retinal Images. In: Shen, D., et al., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science, Vol. 11765, Springer, Cham, 714-722. https://doi.org/10.1007/978-3-030-32245-8_79

Chen, C., Biffi, C., Tarroni, G., Petersen, S., Bai, W. and Rueckert, D. (2019) Learning Shape Priors for Robust Cardiac MR Segmentation from Multi-View Images. In: Shen, D., et al., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science, Vol. 11765, Springer, Cham, 523-531. https://doi.org/10.1007/978-3-030-32245-8_58

Valindria, V.V., et al. (2018) Multi-Modal Learning from Unpaired Images: Application to Multi-Organ Segmentation in CT and MRI. 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, 12-15 March 2018, 547-556. https://doi.org/10.1109/WACV.2018.00066

Qin, C., Schlemper, J., Caballero, J., Price, A.N., Hajnal, J.V. and Rueckert, D. (2019) Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction. IEEE Transactions on Medical Imaging, 38, 280-290. https://doi.org/10.1109/TMI.2018.2863670

Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N. and Rueckert, D. (2018) A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. IEEE Transactions on Medical Imaging, 37, 491-503. https://doi.org/10.1109/TMI.2017.2760978

Yang, G., Yu, S., Dong, H., Slabaugh, G., Dragotti, P.L., Ye, X., Liu, F., Arridge, S., Keegan, J., Guo, Y., Firmin, D., Keegan, J., Slabaugh, G., Arridge, S., Ye, X., Guo, Y., Yu, S., Liu, F., Firmin, D., Dragotti, P.L., Yang, G. and Dong, H. (2018) DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction. IEEE Transactions on Medical Imaging, 37, 1310-1321. https://doi.org/10.1109/TMI.2017.2785879

Ben Yedder, H., Shokoufi, M., Cardoen, B., Golnaraghi, F. and Hamarneh, G. (2019) Limited-Angle Diffuse Optical Tomography Image Reconstruction Using Deep Learning. In: Shen, D., et al., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science, Vol. 11764, Springer, Cham, 66-74. https://doi.org/10.1007/978-3-030-32239-7_8

Ahmad, J., Sajjad, M., Mehmood, I. and Baik, S.W. (2017) SiNC: Saliency-Injected Neural Codes for Representation and Efficient Retrieval of Medical Radiographs. PLoS ONE, 12, e0181707. https://doi.org/10.1371/journal.pone.0181707

Khatami, A., Babaie, M., Tizhoosh, H.R., Khosravi, A., Nguyen, T. and Nahavandi, S. (2018) A Sequential Search-Space Shrinking Using CNN Transfer Learning and a Radon Projection Pool for Medical Image Retrieval. Expert Systems with Applications, 100, 224-233. https://doi.org/10.1016/j.eswa.2018.01.056

Swati, Z.N.K., Zhao, Q., Kabir, M., Ali, F., Ali, Z., Ahmed, S. and Lu, J. (2019) Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning. IEEE Access, 7, 17809-17822. https://doi.org/10.1109/ACCESS.2019.2892455

Pham, H.H., Le, T.T., Tran, D.Q., Ngo, D.T. and Nguyen, H.Q. (2019) Interpreting Chest X-Rays via CNNs That Exploit Hierarchical Disease Dependencies and Uncertainty Labels. arXiv: 1911.06475 https://doi.org/10.1101/19013342

Bekker, A.J. and Goldberger, J. (2016) Training Deep Neural-Networks Based on Unreliable Labels. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, 20-25 March 2016, 2682-2686. https://doi.org/10.1109/ICASSP.2016.7472164

Matuszewski, D.J. and Sintorn, I.M. (2018) Minimal Annotation Training for Segmentation of Microscopy Images. 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington DC, 4-7 April 2018, 387-390. https://doi.org/10.1109/ISBI.2018.8363599

Ren, M., Zeng, W., Yang, B. and Urtasun, R. (2018) Learning to Reweight Examples for Robust Deep Learning. Proceedings of the 35th International Conference on Machine Learning, PMLR, 80, 4334-4343.

Xue, C., Dou, Q., Shi, X., Chen, H. and Heng, P.A. (2019) Robust Learning at Noisy Labeled Medical Images: Applied to Skin Lesion Classification. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, 8-11 April 2019, 1280-1283. https://doi.org/10.1109/ISBI.2019.8759203

Mirikharaji, Z., Yan, Y. and Hamarneh, G. (2019) Learning to Segment Skin Lesions from Noisy Annotations. In: Wang, Q., et al., Eds., Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data. DART 2019, MIL3ID 2019. Lecture Notes in Computer Science, Vol. 11795, Springer, Cham, 207-215. https://doi.org/10.1007/978-3-030-33391-1_24

Nie, D., Gao, Y., Wang, L. and Shen, D. (2018) ASDNet: Attention Based Semi-Supervised Deep Networks for Medical Image Segmentation. In: Frangi, A., Schnabel, J., Davatzikos, C., Alberola-López, C. and Fichtinger, G., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2018. MICCAI 2018. Lecture Notes in Computer Science, Vol. 11073, Springer, Cham, 370-378. https://doi.org/10.1007/978-3-030-00937-3_43

Fries, J.A., Varma, P., Chen, V.S., Xiao, K., Tejeda, H., Saha, P., Dunnmon, J., Chubb, H., Maskatia, S., Fiterau, M., Delp, S., Ashley, E., Ré, C. and Priest, J.R. (2019) Weakly Supervised Classification of Aortic Valve Malformations Using Unlabeled Cardiac MRI Sequences. Nature Communications, 10, Article No. 3111. https://doi.org/10.1038/s41467-019-11012-3

Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H. and Wang, Y. (2017) Artificial Intelligence in Healthcare: Past, Present and Future. Stroke and Vascular Neurology, 2, 230-243. https://doi.org/10.1136/svn-2017-000101

Miller, D.D. and Brown, E.W. (2018) Artificial Intelligence in Medical Practice: The Question to the Answer. The American Journal of Medicine, 131, 129-133. https://doi.org/10.1016/j.amjmed.2017.10.035

Jang, H.J. and Cho, K.O. (2019) Applications of Deep Learning for the Analysis of Medical Data. Archives of Pharmacal Research, 42, 492-504. https://doi.org/10.1007/s12272-019-01162-9

Bakator, M. and Radosav, D. (2018) Deep Learning and Medical Diagnosis: A Review of Literature. Multimodal Technologies and Interaction, 2, 47. https://doi.org/10.3390/mti2030047

Lundervold, A.S. and Lundervold, A. (2019) An Overview of Deep Learning in Medical Imaging Focusing on MRI. Zeitschrift für Medizinische Physik, 29, 102-127. https://doi.org/10.1016/j.zemedi.2018.11.002

Hecht-Nielsen, R. (1988) Neurocomputing: Picking the Human Brain. IEEE Spectrum, 25, 36-41. https://doi.org/10.1109/6.4520

Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012) ImageNet Classification with Deep Convolutional Neural Networks. In: Pereira, F., Burges, C.J.C., Bottou, L. and Weinberger, K.Q., Eds., Advances in Neural Information Processing Systems, Vol. 25, Curran Associates, Inc., Red Hook, NY, 1097-1105.

Arasu, A. and Garcia-Molina, H. (2003) Extracting Structured Data from Web Pages. Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, San Diego, CA, 9-12 June 2003, 337-348. https://doi.org/10.1145/872757.872799

Velicer, W.F. and Molenaar, P.C. (2012) Time Series Analysis for Psychological Research. In: Weiner, I., Schinka, J.A. and Velicer, W.F., Eds., Handbook of Psychology, 2nd Edition, John Wiley & Sons, Inc, Hoboken. https://doi.org/10.1002/9781118133880.hop202022

LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W. and Jackel, L.D. (1989) Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1, 541-551. https://doi.org/10.1162/neco.1989.1.4.541

Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2017) ImageNet Classification with Deep Convolutional Neural Networks. Communications of the ACM, 60, 84-90. https://doi.org/10.1145/3065386

Kipf, T.N. and Welling, M. (2016) Semi-Supervised Classification with Graph Convolutional Networks. 5th International Conference on Learning Representations (ICLR-17). arXiv: 1609.02907.

Elman, J.L. (1990) Finding Structure in Time. Cognitive Science, 14, 179-211. https://doi.org/10.1207/s15516709cog1402_1

Simonyan, K. and Zisserman, A. (2014) Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv: 1409.1556.

León, J., Escobar, J.J., Ortiz, A., Ortega, J., González, J., Martín-Smith, P., Gan, J.Q. and Damas, M. (2020) Deep Learning for EEG-Based Motor Imagery Classification: Accuracy-Cost Trade-Off. PLoS ONE, 15, e0234178. https://doi.org/10.1371/journal.pone.0234178

Hochreiter, S. and Schmidhuber, J. (1997) Long Short-Term Memory. Neural Computation, 9, 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735

Sathya, R. and Abraham, A. (2013) Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification. International Journal of Advanced Research in Artificial Intelligence (IJARAI), 2, 34-38. https://doi.org/10.14569/IJARAI.2013.020206

Gondara, L. (2016) Medical Image Denoising Using Convolutional Denoising Autoencoders. 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), Barcelona, 12-15 December 2016, 241-246. https://doi.org/10.1109/ICDMW.2016.0041

Zhou, B.L., Khosla, A., Lapedriza, A., Torralba, A. and Oliva, A. (2016) Places: An Image Database for Deep Scene Understanding. arXiv: 1610.02055

Nowling, R.J., et al. (2019) Classification before Segmentation: Improved U-Net Prostate Segmentation. 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI), Chicago, IL, 19-22 May 2019, 1-4. https://doi.org/10.1109/BHI.2019.8834494

Pesteie, M, Abolmaesumi, P. and Rohling, R.N. (2019) Adaptive Augmentation of Medical Data Using Independently Conditional Variational Auto-Encoders. IEEE Transactions on Medical Imaging, 38, 2807-2820. https://doi.org/10.1109/TMI.2019.2914656

Yu, E.M., Iglesias, J.E., Dalca, A.V. and Sabuncu, M.R. (2020) An Auto-Encoder Strategy for Adaptive Image Segmentation. Proceedings of the Third Conference on Medical Imaging with Deep Learning, PMLR, 121, 881-891.

Uzunova, H., Schultz, S., Handels, H., et al. (2019) Unsupervised Pathology Detection in Medical Images Using Conditional Variational Autoencoders. International Journal of Computer Assisted Radiology and Surgery 14, 451-461. https://doi.org/10.1007/s11548-018-1898-0

Chen, M., Shi, X., Zhang, Y., Wu, D. and Guizani, M. (2017) Deep Features Learning for Medical Image Analysis with Convolutional Autoencoder Neural Network. IEEE Transactions on Big Data.

Saltz, J., et al. (2018) Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images. Cell Reports, 23, 181-193.e7.

Apostolopoulos, S., Ciller, C., De Zanet, S., Wolf, S. and Sznitman, R. (2017) RetiNet: Automatic AMD Identification in OCT Volumetric Data. Investigative Ophthalmology & Visual Science, 58, 387.

Lam, C., Yu, C., Huang, L. and Rubin, D. (2018) Retinal Lesion Detection with Deep Learning Using Image Patches. Investigative Ophthalmology & Visual Science, 59, 590-596. https://doi.org/10.1167/iovs.17-22721

Choi, H., Ha, S., Im, H.J., Paek, S.H. and Lee, D.S. (2017) Refining Diagnosis of Parkinson’s Disease with Deep Learning-Based Interpretation of Dopamine Transporter Imaging. NeuroImage: Clinical, 16, 586-594. https://doi.org/10.1016/j.nicl.2017.09.010

Rajaraman, S., Antani, S.K., Poostchi, M., Silamut, K., Hossain, M.A., Maude, R.J., Jaeger, S. and Thoma, G.R. (2018) Pre-Trained Convolutional Neural Networks as Feature Extractors toward Improved Malaria Parasite Detection in Thin Blood Smear Images. PeerJ, 6, e4568. https://doi.org/10.7717/peerj.4568

Nielsen, A., Hansen, M.B., Tietze, A. and Mouridsen, K. (2018) Prediction of Tissue Outcome and Assessment of Treatment Effect in Acute Ischemic Stroke Using Deep Learning. Stroke, 49, 1394-1401. https://doi.org/10.1161/STROKEAHA.117.019740

Lee, H.C., Ryu, H.G., Chung, E.J. and Jung, C.W. (2018) Prediction of Bispectral Index during Target-Controlled Infusion of Propofol and Remifentanil: A Deep Learning Approach. Anesthesiology, 128, 492-501. https://doi.org/10.1097/ALN.0000000000001892

Zeng, L.L., Wang, H., Hu, P., Yang, B., Pu, W., Shen, H., Chen, X., Liu, Z., Yin, H., Tan, Q., Wang, K. and Hu, D. (2018) Multi-Site Diagnostic Classification of Schizophrenia Using Discriminant Deep Learning with Functional Connectivity MRI. EBioMedicine, 30, 74-85. https://doi.org/10.1016/j.ebiom.2018.03.017

Ghesu, F.C., Georgescu, B., Zheng, Y., Hornegger, J. and Comaniciu, D. (2015) Marginal Space Deep Learning: Efficient Architecture for Detection in Volumetric Image Data. In: Navab, N., Hornegger, J., Wells, W. and Frangi, A., Eds., Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science, Vol. 9349, Springer, Cham, 710-718. https://doi.org/10.1007/978-3-319-24553-9_87

Anthimopoulos, M., Christodoulidis, S., Ebner, L., Christe, A. and Mougiakakou, S. (2016) Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network. IEEE Transactions on Medical Imaging, 35, 1207-1216. https://doi.org/10.1109/TMI.2016.2535865

Yasaka, K., Akai, H., Kunimatsu, A., Abe, O. and Kiryu, S. (2018) Liver Fibrosis: Deep Convolutional Neural Network for Staging by Using Gadoxetic Acid-Enhanced Hepatobiliary Phase MR Images. Radiology, 287, 146-155. https://doi.org/10.1148/radiol.2017171928

Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H. and Bengio, Y. (2014) Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, October 2014, 1724-1734. https://doi.org/10.3115/v1/D14-1179

Leibig, C., Allken, V., Ayhan, M.S., Berens, P. and Wahl, S. (2017) Leveraging Uncertainty Information from Deep Neural Networks for Disease Detection. Scientific Reports, 7, Article No. 17816. https://doi.org/10.1038/s41598-017-17876-z

Kooi, T., Litjens, G., van Ginneken, B., Gubern-Mérida, A., Sánchez, C.I., Mann, R., den Heeten, A. and Karssemeijer, N. (2017) Large Scale Deep Learning for Computer Aided Detection of Mammographic Lesions. Medical Image Analysis, 35, 303-312. https://doi.org/10.1016/j.media.2016.07.007

Schuster, M. and Paliwal, K.K. (1997) Bidirectional Recurrent Neural Networks. IEEE Transactions on Signal Processing, 45, 2673-2681. https://doi.org/10.1109/78.650093

Kim, E.K., Kim, H.E., Han, K., Kang, B.J., Sohn, Y.M., Woo, O.H. and Lee, C.W. (2018) Applying Data-Driven Imaging Biomarker in Mammography for Breast Cancer Screening: Preliminary Study. Scientific Reports, 8, Article No. 2762. https://doi.org/10.1038/s41598-018-21215-1

Heinsfeld, A.S., Franco, A.R., Craddock, R.C., Buchweitz, A. and Meneguzzi, F. (2018) Identification of Autism Spectrum Disorder Using Deep Learning and the ABIDE Dataset. NeuroImage: Clinical, 17, 16-23. https://doi.org/10.1016/j.nicl.2017.08.017

Suk, H.I., Lee, S.W. and Shen, D. (2014) Hierarchical Feature Representation and Multimodal Fusion with Deep Learning for AD/MCI Diagnosis. NeuroImage, 101, 569-582. https://doi.org/10.1016/j.neuroimage.2014.06.077

Hinton, G., Vinyals, O. and Dean, J. (2015) Distilling the Knowledge in a Neural Network. arXiv: 1503.02531

van Grinsven, M.J., van Ginneken, B., Hoyng, C.B., Theelen, T. and Sanchez, C.I. (2016) Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images. IEEE Transactions on Medical Imaging, 35, 1273-1284. https://doi.org/10.1109/TMI.2016.2526689

Wang, J., Yang, X., Cai, H., et al. (2016) Discrimination of Breast Cancer with Microcalcifications on Mammography by Deep Learning. Scientific Reports, 6, Article No. 27327. https://doi.org/10.1038/srep27327

Kooi, T., van Ginneken, B., Karssemeijer, N. and den Heeten, A. (2017) Discriminating Solitary Cysts from Soft Tissue Lesions in Mammography Using a Pretrained Deep Convolutional Neural Network. Medical Physics, 44, 1017-1027. https://doi.org/10.1002/mp.12110

Fu, H., Cheng, J., Xu, Y., Zhang, C., Wong, D.W.K., Liu, J. and Cao, X. (2018) Disc-Aware Ensemble Network for Glaucoma Screening from Fundus Image. IEEE Transactions on Medical Imaging, 37, 2493-2501. https://doi.org/10.1109/TMI.2018.2837012

Yu, C., Yang, S., Kim, W., Jung, J., Chung, K.Y., Lee, S.W. and Oh, B. (2018) Acral Melanoma Detection Using a Convolutional Neural Network for Dermoscopy Images. PLoS ONE, 13, e0193321. https://doi.org/10.1371/journal.pone.0193321

100

Topol, E. (2019) Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again. Basic Books, Hachette, UK.

101

Wang, J., Ding, H., Bidgoli, F.A., Zhou, B., Iribarren, C., Molloi, S. and Baldi, P. (2017) Detecting Cardiovascular Disease from Mammograms with Deep Learning. IEEE Transactions on Medical Imaging, 36, 1172-1181. https://doi.org/10.1109/TMI.2017.2655486

102

Rumelhart, D.E., Hinton, G.E. and Williams, R.J. (1985) Learning Internal Representations by Error Propagation. California Univ San Diego La Jolla Inst for Cognitive Science.

103

Iakovidis, D.K., Georgakopoulos, S.V., Vasilakakis, M., Koulaouzidis, A. and Plagianakos, V.P. (2018) Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification. IEEE Transactions on Medical Imaging, 37, 2196-2210. https://doi.org/10.1109/TMI.2018.2837002

104

van der Burgh, H.K., Schmidt, R., Westeneng, H.J., de Reus, M.A., van den Berg, L.H. and van den Heuvel, M.P. (2016) Deep Learning Predictions of Survival Based on MRI in Amyotrophic Lateral Sclerosis. NeuroImage: Clinical, 13, 361-369. https://doi.org/10.1016/j.nicl.2016.10.008

105

Lee, C.S., Baughman, D.M. and Lee, A.Y. (2017) Deep Learning Is Effective for the Classification of OCT Images of Normal Versus Age-Related Macular Degeneration. Ophthalmology Retina, 1, 322-327. https://doi.org/10.1016/j.oret.2016.12.009

106

Hsieh, Y.J., Tseng, H.C., Chin, C.L., Shao, Y.H. and Tsai, T.Y. (2020) Based on DICOM RT Structure and Multiple Loss Function Deep Learning Algorithm in Organ Segmentation of Head and Neck Image. In: Lin, K.P., Magjarevic, R. and de Carvalho, P., Eds., Future Trends in Biomedical and Health Informatics and Cybersecurity in Medical Devices. ICBHI 2019. IFMBE Proceedings, Vol. 74, Springer, Cham, 428-435. https://doi.org/10.1007/978-3-030-30636-6_58

107

Ngo, T.A., Lu, Z. and Carneiro, G. (2017) Combining Deep Learning and Level Set for the Automated Segmentation of the Left Ventricle of the Heart from Cardiac Cine Magnetic Resonance. Medical Image Analysis, 35, 159-171. https://doi.org/10.1016/j.media.2016.05.009

108

Zhang, J., Xia, Y., Wu, Q. and Xie, Y.T. (2017) Classification of Medical Images and Illustrations in the Biomedical Literature Using Synergic Deep Learning. arXiv: 1706.09092

109

Araújo, T., Aresta, G., Castro, E., Rouco, J., Aguiar, P., Eloy, C., Polónia, A. and Campilho, A. (2017) Classification of Breast Cancer Histology Images Using Convolutional Neural Networks. PLoS ONE, 12, e0177544. https://doi.org/10.1371/journal.pone.0177544

110

Han, Z., Wei, B., Zheng, Y., Yin, Y., Li, K. and Li, S. (2017) Breast Cancer Multi-Classification from Histopathological Images with Structured Deep Learning Model. Scientific Reports, 7, Article No. 4172. https://doi.org/10.1038/s41598-017-04075-z

111

Zhang, X., Yu, F.X., Chang, S.-F. and Wang, S.J. (2015) Deep Transfer Network: Unsupervised Domain Adaptation. arXiv: 1503.00591

112

Hutchinson, B., Deng, L. and Yu, D. (2013) Tensor Deep Stacking Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 1944-1957. https://doi.org/10.1109/TPAMI.2012.268

113

Hjelm, R.D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A. and Bengio, Y. (2018) Learning Deep Representations by Mutual Information Estimation and Maximization. arXiv: 1808.06670

114

Alom, M.Z., Yakopcic, C., Nasrin, M.S., Taha, T.M. and Asari, V.K. (2019) Breast Cancer Classification from Histopathological Images with Inception Recurrent Residual Convolutional Neural Network. Journal of Digital Imaging, 32, 605-617. https://doi.org/10.1007/s10278-019-00182-7

115

Tiulpin, A., Thevenot, J., Rahtu, E., Lehenkari, P. and Saarakkala, S. (2018) Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach. Scientific Reports, 8, Article No. 1727. https://doi.org/10.1038/s41598-018-20132-7

116

Lee, J. and Nishikawa, R.M. (2018) Automated Mammographic Breast Density Estimation Using a Fully Convolutional Network. Medical Physics, 45, 1178-1190. https://doi.org/10.1002/mp.12763

117

Esses, S.J., Lu, X., Zhao, T., Shanbhogue, K., Dane, B., Bruno, M. and Chandarana, H. (2018) Automated Image Quality Evaluation of T2-Weighted Liver MRI Utilizing Deep Learning Architecture. Journal of Magnetic Resonance Imaging, 47, 723-728. https://doi.org/10.1002/jmri.25779

118

Serj, M.F., Lavi, B., Hoff, G. and Valls, D.P. (2018) A Deep Convolutional Neural Network for Lung Cancer Diagnostic. arXiv: 1804.08170

119

Du, X., et al. (2018) Articulated Multi-Instrument 2-D Pose Estimation Using Fully Convolutional Networks. IEEE Transactions on Medical Imaging, 37, 1276-1287. https://doi.org/10.1109/TMI.2017.2787672

120

Han, S.S., Park, G.H., Lim, W., Kim, M.S., Na, J.I., Park, I. and Chang, S.E. (2018) Deep Neural Networks Show an Equivalent and Often Superior Performance to Dermatologists in Onychomycosis Diagnosis: Automatic Construction of Onychomycosis Datasets by Region-Based Convolutional Deep Neural Network. PLoS ONE, 13, e0191493. https://doi.org/10.1371/journal.pone.0191493

121

Kim, K.H., Choi, S.H. and Park, S.H. (2018) Improving Arterial Spin Labeling by Using Deep Learning. Radiology, 287, 658-666. https://doi.org/10.1148/radiol.2017171154

122

Song, Y., Zhang, L., Chen, S., Ni, D., Lei, B. and Wang, T. (2015) Accurate Segmentation of Cervical Cytoplasm and Nuclei Based on Multiscale Convolutional Network and Graph Partitioning. IEEE Transactions on Biomedical Engineering, 62, 2421-2433. https://doi.org/10.1109/TBME.2015.2430895

123

Haryanto, T., Wasito, I. and Suhartanto, H. (2017) Convolutional Neural Network (CNN) for Gland Images Classification. 2017 11th International Conference on Information & Communication Technology and System (ICTS), Surabaya, 31-31 October 2017, 55-60. https://doi.org/10.1109/ICTS.2017.8265646

124

Cao, H., Bernard, S., Heutte, L. and Sabourin, R. (2018) Improve the Performance of Transfer Learning without Fine-Tuning Using Dissimilarity-Based Multi-View Learning for Breast Cancer Histology Images. In: Campilho, A., Karray, F. and ter Haar Romeny, B., Eds., Image Analysis and Recognition. ICIAR 2018. Lecture Notes in Computer Science, Vol. 10882, Springer, Cham, 779-787. https://doi.org/10.1007/978-3-319-93000-8_88

125

Luo, X., Mori, K. and Peters, T.M. (2018) Advanced Endoscopic Navigation: Surgical Big Data, Methodology, and Applications. Annual Review of Biomedical Engineering, 20, 221-251. https://doi.org/10.1146/annurev-bioeng-062117-120917

126

Xiao, C., Choi, E. and Sun, J. (2018) Opportunities and Challenges in Developing Deep Learning Models Using Electronic Health Records Data: A Systematic Review. Journal of the American Medical Informatics Association, 25, 1419-1428. https://doi.org/10.1093/jamia/ocy068

127

Shickel, B., Tighe, P.J., Bihorac, A. and Rashidi, P. (2018) Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis. IEEE Journal of Biomedical and Health Informatics, 22, 1589-1604. https://doi.org/10.1109/JBHI.2017.2767063

128

Karkra, S., Singh, P. and Kaur, K. (2019) Convolution Neural Network: A Shallow Dive into Deep Neural Net Technology. International Journal of Recent Technology and Engineering (IJRTE), 8, 487-495.

129

Ranschaert, E.R., Morozov, S. and Algra, P.R. (2019) Artificial Intelligence in Medical Imaging: Opportunities, Applications and Risks. Springer, Berlin. https://doi.org/10.1007/978-3-319-94878-2

130

Tsang, G., Xie, X. and Zhou, S.M. (2020) Harnessing the Power of Machine Learning in Dementia Informatics Research: Issues, Opportunities, and Challenges. IEEE Reviews in Biomedical Engineering, 13, 113-129. https://doi.org/10.1109/RBME.2019.2904488

131

Haryanto, T., Suhartanto, H., Murni, A. and Kusmardi, K. (2019) Strategies to Improve Performance of Convolutional Neural Network on Histopathological Images Classification. 2019 International Conference on Advanced Computer Science and information Systems (ICACSIS), Bali, Indonesia, 12-13 October 2019, 125-132. https://doi.org/10.1109/ICACSIS47736.2019.8979740

132

Das, A., Nair, M.S. and Peter, S.D. (2020) Computer-Aided Histopathological Image Analysis Techniques for Automated Nuclear Atypia Scoring of Breast Cancer: A Review. Journal of Digital Imaging, 33, 1091-1121. https://doi.org/10.1007/s10278-019-00295-z

133

Tang, Y., Wang, X., Harrison, A.P., Lu, L., Xiao, J. and Summers, R.M. (2018) Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs. In: Shi, Y., Suk, H.I. and Liu, M., Eds., Machine Learning in Medical Imaging. MLMI 2018. Lecture Notes in Computer Science, Vol. 11046, Springer, Cham, 249-258. https://doi.org/10.1007/978-3-030-00919-9_29

134

Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014) Generative Adversarial Nets. Proceedings of the 27th International Conference on Neural Information Processing Systems, 2, 2672-2680.

135

Hoffman, J., Tzeng, E., Park, T., Zhu, J.Y., Isola, P., Saenko, K., et al. (2018) Cycada: Cycle-Consistent Adversarial Domain Adaptation. Proceedings of the 35th International Conference on Machine Learning, PMLR, 80, 1989-1998.

136

Long, M., Zhu, H., Wang, J. and Jordan, M.I. (2016) Unsupervised Domain Adaptation with Residual Transfer Networks. arXiv: 1602.04433

137

Tzeng, E., Hoffman, J., Saenko, K. and Darrell, T. (2017) Adversarial Discriminative Domain Adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, 21-26 July 2017, 2962-2971. https://doi.org/10.1109/CVPR.2017.316

138

Luo, Y., Zheng, L., Guan, T., Yu, J. and Yang, Y. (2019) Taking a Closer Look at Domain Shift: Category-Level Adversaries for Semantics Consistent Domain Adaptation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, 15-20 June 2019, 2502-2511. https://doi.org/10.1109/CVPR.2019.00261

139

Tsai, Y.H., Hung, W.C., Schulter, S., Sohn, K., Yang, M.H. and Chandraker, M. (2018) Learning to Adapt Structured Output Space for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7472-7481. https://doi.org/10.1109/CVPR.2018.00780

140

Liu, D., Zhang, D., Song, Y., Zhang, F., O’Donnell, L., Huang, H., Chen, M. and Cai, W. (2021) PDAM: A Panoptic-Level Feature Alignment Framework for Unsupervised Domain Adaptive Instance Segmentation in Microscopy Images. IEEE Transactions on Medical Imaging, 40, 154-165. https://doi.org/10.1109/TMI.2020.3023466

141

Ghafoorian, M., et al. (2017) Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D. and Duchesne, S., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2017. MICCAI 2017. Lecture Notes in Computer Science, Vol. 10435, Springer, Cham, 516-524.

142

Jiang, J., Hu, Y.C., Tyagi, N., Zhang, P., Rimner, A., Mageras, G.S., Deasy, J.O. and Veeraraghavan, H. (2018) Tumor-Aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation. In: Frangi, A., Schnabel, J., Davatzikos, C., Alberola-López, C. and Fichtinger, G., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2018. MICCAI 2018. Lecture Notes in Computer Science, Vol. 11071, Springer, Cham, 777-785. https://doi.org/10.1007/978-3-030-00934-2_86

143

Chen, C., Dou, Q., Chen, H. and Heng, P.A. (2018) Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-Ray Segmentation. In: Shi, Y., Suk, H.I. and Liu, M., Eds., Machine Learning in Medical Imaging. MLMI 2018. Lecture Notes in Computer Science, Vol. 11046, Springer, Cham, 143-151. https://doi.org/10.1007/978-3-030-00919-9_17

144

Yang, J., Dvornek, N.C., Zhang, F., Chapiro, J., Lin, M. and Duncan, J.S. (2019) Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation. In: Shen, D., et al., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science, Vol. 11765, Springer, Cham, 255-263. https://doi.org/10.1007/978-3-030-32245-8_29

145

Zhang, C., Wu, S., Lu, Z., Shen, Y., Wang, J., Huang, P., Lou, J., Liu, C., Xing, L., Zhang, J., Xue, J. and Li, D. (2020) Hybrid Adversarial-Discriminative Network for Leukocyte Classification in Leukemia. Medical Physics, 47, 3732-3744. https://doi.org/10.1002/mp.14144

146

Li, C.Y., Liang, X., Hu, Z. and Xing, E.P. (2019) Knowledge-Driven Encode, Retrieve, Paraphrase for Medical Image Report Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 6666-6673. https://doi.org/10.1609/aaai.v33i01.33016666

147

Wang, Z., Zhang, J., Feng, J. and Chen, Z. (2014) Knowledge Graph and Text Jointly Embedding. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, October 2014, 1591-1601. https://doi.org/10.3115/v1/D14-1167

148

Luo, B.N., Shen, J., Cheng, S.Y., Wang, Y.J. and Pantic, M. (2020) Shape Constrained Network for Eye Segmentation in the Wild. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Snowmass, CO, 1-5 March 2020, 1952-1960. https://doi.org/10.1109/WACV45572.2020.9093483

149

Wistuba, M., Rawat, A. and Pedapati, T. (2019) A Survey on Neural Architecture Search. arXiv: 1905. 01392.

150

Guo, D., Jin, D., Zhu, Z., Ho, T.Y., Harrison, A.P., Chao, C.H., et al. (2020) Organ at Risk Segmentation for Head and Neck Cancer Using Stratified Learning and Neural Architecture Search. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, 13-19 June 2020, 4222-4231. https://doi.org/10.1109/CVPR42600.2020.00428

151

Li, S., Wei, J., Chan, H.P., Helvie, M.A., Roubidoux, M.A., Lu, Y., Zhou, C., Hadjiiski, L.M. and Samala, R.K. (2018) Computer-Aided Assessment of Breast Density: Comparison of Supervised Deep Learning and Feature-Based Statistical Learning. Physics in Medicine & Biology, 63, Article ID: 025005. https://doi.org/10.1088/1361-6560/aa9f87

152

Carneiro, G., Nascimento, J.C. and Freitas, A. (2012) The Segmentation of the Left Ventricle of the Heart From Ultrasound Data Using Deep Learning Architectures and Derivative-Based Search Methods. IEEE Transactions on Image Processing, 21, 968-982. https://doi.org/10.1109/TIP.2011.2169273

153

Xue, Y., Zhang, R., Deng, Y., Chen, K. and Jiang, T. (2017) A Preliminary Examination of the Diagnostic Value of Deep Learning in Hip Osteoarthritis. PLoS ONE, 12, e0178992. https://doi.org/10.1371/journal.pone.0178992

154

Chen, C.-M., Huang, Y.-S., Fang, P.-W., Liang, C.-W. and Chang, R.-F. (2020) A Computer-Aided Diagnosis System for Differentiation and Delineation of Malignant Regions on Whole-Slide Prostate Histopathology Image Using Spatial Statistics and Multidimensional DenseNet. Medical Physics, 47, 1021-1033. https://doi.org/10.1002/mp.13964

155

Quellec, G., Charrière, K., Boudi, Y., Cochener, B. and Lamard, M. (2017) Deep Image Mining for Diabetic Retinopathy Screening. Medical Image Analysis, 39, 178-193. https://doi.org/10.1016/j.media.2017.04.012

156

Saha, S.K., Fernando, B., Cuadros, J., Xiao, D. and Kanagasingam, Y. (2018) Automated Quality Assessment of Colour Fundus Images for Diabetic Retinopathy Screening in Telemedicine. Journal of Digital Imaging, 31, 869-878. https://doi.org/10.1007/s10278-018-0084-9

157

Das, A., Rad, P., Choo, K.R., Nouhi, B., Lish, J. and Martel, J. (2019) Distributed Machine Learning Cloud Teleophthalmology IoT for Predicting AMD Disease Progression. Future Generation Computer Systems, 93, 486-498. https://doi.org/10.1016/j.future.2018.10.050

158

Kim, Y D., Noh, K.J., Byun, S.J., et al. (2020) Effects of Hypertension, Diabetes, and Smoking on Age and Sex Prediction from Retinal Fundus Images. Scientific Reports, 10, Article No. 4623. https://doi.org/10.1038/s41598-020-61519-9

159

Betancur, J., Commandeur, F., Motlagh, M., Sharir, T., Einstein, A.J., et al. (2018) Deep Learning for Prediction of Obstructive Disease from Fast Myocardial Perfusion SPECT: A Multicenter Study. JACC: Cardiovascular Imaging, 11, 1654-1663. https://doi.org/10.1016/j.jcmg.2018.01.020

160

Chaudhari, A.S., Fang, Z., Kogan, F., Wood, J., Stevens, K.J., Gibbons, E.K., Lee, J.H., Gold, G.E. and Hargreaves, B.A. (2018) Super-Resolution Musculoskeletal MRI Using Deep Learning. Magnetic Resonance in Medicine, 80, 2139-2154. https://doi.org/10.1002/mrm.27178

161

Ning, Z., Luo, J., Li, Y., Han, S., Feng, Q., Xu, Y., Chen, W., Chen, T. and Zhang, Y. (2019) Pattern Classification for Gastrointestinal Stromal Tumors by Integration of Radiomics and Deep Convolutional Features. IEEE Journal of Biomedical and Health Informatics, 23, 1181-1191. https://doi.org/10.1109/JBHI.2018.2841992

162

Khosravan, N., Celik, H., Turkbey, B., Jones, E.C., Wood, B. and Bagci, U. (2019) A Collaborative Computer Aided Diagnosis (C-CAD) System with Eye-Tracking, Sparse Attentional Model, and Deep Learning. Medical Image Analysis, 51, 101-115. https://doi.org/10.1016/j.media.2018.10.010

163

Jang, R., Kim, N., Jang, M., Lee, K.H., Lee, S.M., Lee, K.H., Noh, H.N. and Seo, J.B. (2020) Assessment of the Robustness of Convolutional Neural Networks in Labeling Noise by Using Chest X-Ray Images from Multiple Centers. JMIR Medical Informatics, 8, e18089. https://doi.org/10.2196/18089

164

Cheng, J.Z., Ni, D., Chou, Y.H., Qin, J., Tiu, C.M., Chang, Y.C., Huang, C.S., Shen, D. and Chen, C.M. (2016) Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans. Scientific Reports, 6, Article No. 24454. https://doi.org/10.1038/srep24454

165

Song, Y., Zhang, Y.D., Yan, X., Liu, H., Zhou, M., Hu, B. and Yang, G. (2018) Computer-Aided Diagnosis of Prostate Cancer Using a Deep Convolutional Neural Network from Multiparametric MRI. Journal of Magnetic Resonance Imaging, 48, 1570-1577. https://doi.org/10.1002/jmri.26047

166

Sujit, S.J., Coronado, I., Kamali, A., Narayana, P.A. and Gabr, R.E. (2019) Automated Image Quality Evaluation of Structural Brain MRI Using an Ensemble of Deep Learning Networks. Journal of Magnetic Resonance Imaging, 50, 1260-1267. https://doi.org/10.1002/jmri.26693

167

Dar, S.U.H., Yurt, M., Shahdloo, M., Ildız, M.E. and Çukur, T. (2018) Synergistic Reconstruction and Synthesis via Generative Adversarial Networks for Accelerated Multi-Contrast MRI. Computer Vision and Pattern Recognition. arXiv: 1805.10704.