As a raw corpus, we used a subset of PubMed abstracts downloaded in October 2013, filtered by the keyword “cancer”. From 3,099,076 abstracts, 14,847,050 sentences were extracted.

Enju [9] was used for POS recognition of words and conversion into base forms. Since the sentences were extracted from biomedical abstracts, “-genia” option was specified. As a result, part-of-speech (POS) and base form are recognized for each words.

So that word2vec can differently treat the same word with different POS categories, they were attached right after the base form of words (e.g. “care” ->“care(V)”). For readability, nouns are kept as is. To simplify the input for word2vec, we removed all words except nouns, adjectives, adverbs, and verbs.

Biological terms typically consist of two or more words. In addition, they have many synonyms. Since word2vec basically treats a sentence as a sequence of words, it is needed to recognize biological synonyms, aggregate them into primary terms, and convert them into single words (e.g. “yolk sac tumor” ->“endodermal sinus tumor” ->“endodermal_sinus_tumor”). In this study, primary names and synonyms of drugs, diseases, and genes were extracted from PharmGKB [10] and used for recognition and aggregation (genes are used only for showing distribution of word vectors). For each converted single words, prefixes indicating their semantic categories were attached for later processing (e.g. “endodermal_sinus_tumor” ->“disease::endodermal_sinus_tumor”).

Related to the conversion above, we need to consider about the existence of original single words. Firstly, if a synonym word is aggregated into primary word, the original word disappears and is not used for word embedding. Secondly, if a multi-word term is converted into a single word, all the original single words in the multi-word term disappear. Thirdly, if two multi-word term occur in a sentence with overlapping, it is impossible to replace both of them at once. To avoid there problems, a sentence is converted into the sentences which containing at most one converted word per sentence. For example, if a sentence contains two terms to be converted, three sentences including original one are generated. After all conversion, 14,847,050 sentences are expanded to 45,264,480.

In the field of natural language processing and text mining, computational representation of a linguistic unit (e.g. documents, paragraphs, sentences, terms, and words) is essential. The simplest one for document is bag-of-word model, which represents each document as a vector of word frequencies in it. In case of word representation, only the neighboring words in the same sentence are counted. For better analysis, stop-words are removed and raw frequencies are modified by term weighting such as tf-idf. After that, these vectors are used to evaluate the characteristics of the units and similarities between them (vector space model).

One of the serious problems in such a representation and analysis is high dimensionality and sparseness of vectors. For instance, 10 millions of sentences may contain one million of different words, then the dimension of a vector is also one million. In addition, since frequency of word follows Zipf’s law, most of the one million of words only occur a few times, which makes the vectors quite sparse. Though there exist traditional algorithms for dimension reduction or compression like Principal Component Analysis (PCA) and Latent Semantic Analysis (LSA), this problem is not fully solved.

Word embedding for distributed representation of word sense is a new approach to this problem. Based on neural network algorithm, reasonably short numerical vectors (e.g. 100 dimensions) are calculated for all words in a set of sentences. Through the application studies, it is proved that the vector space constructed by word embedding represents word senses and distances (similarities) between them quite well. Additionally, in this space of word sense, word analogy works well in some domains. For example, given three words “man”, “woman”, and “king”, word analogy could predict “queen” by calculating vector(“man”) − vector(“woman”) + vector(“king”) and searching for the nearest word vector vector(“queen”). Though word analogy might allow wide variety of applications, the most desired one is discovery of unknown relations.

In this study, we used word2vec software, a de facto standard implementation of word embedding algorithm, with the following parameters by default.

Ÿ vector size = 200

Ÿ window size = 8

Ÿ minimum count of words to be embedded = 1 (i.e. all words)

Ÿ model = continuous bag of words

As a result, 1,772,186 words were embedded into word vectors (2303 for drugs, 3069 for diseases, 8703 for genes, and 1,758,111 for others).

For the evaluation of clustering results, ATC codes [11] and MeSH tree numbers [12] were attached to drug and disease names, respectively. ATC codes were extracted from DrugBank [13] . Due to the incompleteness of data annotation, only 1253 drugs out of 2303 and 2745 diseases out of 3069 have such classification.

For the evaluation of difference vectors between drugs and diseases, relations between drugs and diseases occurring in the corpus were extracted from CTD [14] . Only the 12,462 relations with therapeutic evidences were adopted for obtaining trustable results. In the set of relations, the mapping from drugs to diseases is many-to- many. For example, “drug::gefitinib” is related to 17 different diseases, and “disease::lung_neoplasm” is mapped from 60 different drugs.

In order to conduct detailed analysis on cancer-related drugs and diseases, 12,462 extracted drug-disease relations were further filtered so that both of drug and disease names in each relation are attached to an ATC code and a MeSH tree number beginning with “L” (Antineoplastic and immunomodulating agents) and “C04” (Neoplasms), respectively. As a result, 1097 relations consist of 104 anti-cancer drugs and 107 cancer-related diseases were extracted for detailed analysis.

For visual evaluation of word vector quality, we performed hierarchical clustering with cosine distance and Ward’s method [15] . Before the clustering, 2303 drugs and 3069 diseases occurring in the corpus were reduced to 1282 and 1051, respectively, since other drugs and diseases did not occur in CTD.

Support Vector Machine (SVM) was adopted for learning and predicting possible relations between drugs and diseases. As an implementation, ksvm function included in kernlab package for R software was used with default parameters.

Figure 2 illustrates the 3D plot of vectors corresponding to 2303 drugs, 3069 diseases, and 8703 for genes. For visualization, the dimension of vector was reduced from 200 to 3 by PCA. In the left panel of the figure, it is

shown that the distributions of word vectors in three categories are clearly separated. In the right panel, it is also shown that the frequent words have clear separation, whereas it is relatively difficult to discriminate the categories of rare words.

Figure 3 and Figure 4 show the result of hierarchical clustering for drugs and diseases, respectively. In the right panels of them, entire pictures of clustering results for 1282 drugs and 1051 diseases are shown. In the right panel of Figure 3, it can be seen that most of the anti-cancer drugs are condensed in the ninth cluster from the top (left panel for more detail). It indicates that the word vectors for drugs well represent the characteristics of corresponding drugs. Also in Figure 4, we can see that the seventh cluster from the top contains a number of cancer-related diseases, however, also in sixth and ninth clusters. The difference between these results might come from the fact that diseases can be classified from different perspectives (tissues, mechanism, etc.).

Though word analogy is quite attractive, it does not always works well. To evaluate the applicability of word analogy to the discovery of new relation between drug and disease, we checked whether most of the displacement vectors between confirmed drug-disease pairs (i.e. correct relations) are similar in length and parallel to each other or not. Unfortunately, as shown in Figure 5, the displacement vectors have wide range of lengths and directions. It indicates that the simple application of word analogy to drug repositioning cannot achieve high performance.

Instead of simple application of word analogy, we constructed a classification model using SVM. For all combinations of 104 anti-cancer drugs and 107 cancer-related diseases (i.e. 11,128 drug-disease pairs), drug vectors and disease vectors were concatenated and binary class labels (i.e. positive or negative) were added according to 1097 correct drug-disease relations extracted from CTD. Due to the imbalance of two classes, 1097 out of 10,031 negative examples were randomly selected so that the numbers of positive and negative examples are balanced.

The result of performance evaluation is shown in Table 1. Each accuracy is an average of 100 times 10-fold cross-validation with different subsets of negative examples. In this table, it was revealed that the performance was not so affected by vector size and window size, and the best accuracy was 87.6%. For exploratory use of the classification model to discover candidate drug-disease pairs, it means sufficiently high performance.

Finally, we tested all combinations of 2199 drugs not used in training and 107 cancer-related diseases (in total, 235,293 drug-disease pairs). In case of the classification model trained by 11,128 examples, only 64 test examples were predicted as positive, and all the drugs in the examples were anti-cancer drugs (but not included in 104 anti-cancer drugs used for training). By controlling the degree of class imbalance in training data, it is possible to predict a pair of non-anti-cancer drug and cancer-related disease as positive. For example, using the classification model trained by 1097 positive and 8776 negative examples (degree of imbalance is 1:8), 10 times training and test by 235,293 drug-disease pairs discovered the following candidate drugs for repositioning to cancer treatment, where the numbers indicate how many times they were discovered in 10 times training and test.

drug::urokinase (10), drug::photodynamic_therapy (10), drug::oxygen (10), drug::nonoxynol-9 (10),

drug::nitroglycerin (10), drug::nitrogen (10), drug::l-phenylalanine (10), drug::l-methionine (10),

drug::l-glutamine (10), drug::l-cysteine (10), drug::glutathione (10), drug::glucose (10),

drug::epoxide (10), drug::enzyme (10), drug::collagenase (10), drug::bisphosphonate (10),

drug::amino_acid (10), drug::amide (10), drug::clarithromycin (9), drug::vitamin (8), drug::l-proline (7),

drug::vitamin_e (6), drug::xanthophylls (4), drug::phospholipid (4), drug::palifermin (4), drug::ether (4),

drug::ethacrynic_acid (4), drug::denosumab (4), drug::egfr_inhibitor (2), drug::pyruvic_acid (1).

Besides too general names like “drug::enzyme” and “drug::amide”, it is notable that the above list includes approved anti-cancer drugs (e.g. “drug::denosumab”), anti-cancer drugs under investigation (e.g. “drug:: clarithromycin”, “drug::bisphosphonate”, and “drug::xanthophyll”), and drugs potentially promote cancer (e.g. “drug::urokinase” and “drug::collagenase”). Especially, it should be emphasized that repositioning of clarithro- mycin to anti-cancer agent has been reported in 2015 [16] , despite the fact that the corpus was downloaded in 2013. Though further screening based on expert’s knowledge is necessary, this result demonstrate that the classification of concatenated word vector is a promising approach to in-silico screening of drug-disease relations for drug repositioning.