Numerical features included age, lymph nodes examined positive, mutation count, Nottingham Prognostic Index, overall survival months and tumor size. Age at diagnosis means the age of the patient at diagnosis time. Lymph nodes examined positive refer to the number of the positive lymph nodes after histopathological exam. Mutation count means the number of genes which have mutations. Nottingham Prognostic Index is used as an index related to concerning grade, tumor size, and lymph node status, each weighted according to regression coefficients of a Cox proportional hazard analysis [14] . Overall survival months means the duration from the time of the intervention to death. Tumor size shows the tumor size measured by imaging techniques.

Categorical features included 22 features. Cancer type is the type of cancer which can be separated into 2 types: Breast Cancer and Breast Sarcoma. Type of breast surgery refers to breast cancer surgery type. And the main treatment method of breast cancer continues to be surgical: breast-conserving surgery, typically with adjuvant radiation, or mastectomy. They are the main two types [15] . “Mastectomy” refers to a surgery to remove all breast tissue from breast as a way to treat or prevent breast cancer. “Breast conserving” refers to a surgery where only the part of the breast that has cancer is removed. Cancer type detailed is the more detailed cancer types. It can be separated into: Breast Invasive Ductal Carcinoma, Breast Mixed Ductal and Lobular Carcinoma Breast Invasive Lobular Carcinoma, Breast Invasive Mixed Mucinous Carcinoma and Metaplastic Breast Cancer. Cellularity refers to the amount of tumor cells in the specimen and their arrangement into clusters. According to the amount level, we can separate it into three groups: low, moderate and high. Chemotherapy is whether the patient had chemotherapy as a treatment or not. “1” refers to yes and “0” refers to no. For Pam50 + claudin-low subtype, Pam 50 is a tumor profiling test that helps show whether some estrogen receptor-positive (ER-positive), HER2-negative breast cancers are likely to metastasize (when breast cancer spreads to other organs). According to Perou et al., there are initially four breast cancer intrinsic subtypes including basal-like, HER2-enriched, luminal and normal-like, which can display gene expression patterns. Subsequent studies have led to the sub-stratification of luminal breast cancers into luminal A and luminal B, and shown that this classification system is of prognostic significance [16] . The classified information was shown in Table 1.

Cohort refers to a group of subjects who share a defining characteristic (It takes a value from 1 to 5). Er status measured by ihc is by using the IHC method, the tumour was considered oestrogen receptor (ER) positive if ≥10% of the neoplastic cells showed nuclear staining, otherwise it will be negative [17] . Er status refers to the estrogen receptor-alpha (ER-α), the binary ER status is predictive for treatment response and prognostic for outcome [18] . Neoplasm histologic grade is determined by pathology by looking the nature of the cells, to judge they look aggressive or not (It takes a value from 1 to 3). Her2 status measured by snp6 is used to assess if the cancer positive for HER2 or not by

ER: oestrogen receptor; PR: progesterone receptor; HER2: human epidermal growth factor receptor 2; Ki-67: a kind of proliferation marker associated with the expression of ER-associated genes.

using advance molecular techniques (type of next generation sequencing). Her2 status is whether the cancer is positive or negative for HER2. Tumor other histologic subtype is the type of the cancer based on microscopic examination of the cancer tissue (It takes a value of “Ductal/NST”, “Mixed”, “Lobular”, “Tubular/cribriform”, “Mucinous”, “Medullary”, “Other”, “Metaplastic”). Hormone therapy refers to whether or not the patient had hormonal as a treatment (1-yes/0-no). Inferred menopausal state is whether the patient postmenopausal or not (post/pre). Integrative cluster is the molecular subtype of the cancer based on some gene expression (it takes a value from “4ER+”, “3”, “9”, “7”, “4ER−”, “5”, “8”, “10”, “1”, “2”, “6”). Primary tumor laterality is whether the right breast or the left breast is involved. For Oncotree code, the OncoTree is an open-source ontology that was developed at Memorial Sloan Kettering Cancer Center (MSK) for standardizing cancer type diagnosis from a clinical perspective by assigning each diagnosis a unique OncoTree code. Pr status refers to Cancer cells are positive or negative for progesterone receptors. Radio therapy is whether or not the patient had radio as a treatment (yes-1/no-0). 3-gene classifier subtype is the three Gene classifier subtype. It takes a value from “ER−/HER2−”, “ER+/HER2− High Prolif”, “ER+/HER2− Low Prolif”, “HER2+”. Tumor stage refers to the stage of the cancer based on the involvement of surrounding structures, lymph nodes and distant spread.

This part of data included mRNA levels Z-score for 331 genes collected from present research. They were proved to have relationship with breast cancer [13] . mRNA expression data were used to calculate the relative expression of an individual gene and tumor to the gene’s expression distribution. For all samples in the returned value indicated the number of standard deviations away from the mean of expression in the reference population (Z-score). This measure was useful to determine whether a gene is up- or down-regulated relative to the normal samples or all other tumor samples. The formula was

Z = expressionintumorsample − meanexpressioninreferencesample standarddeviationofexpressioninreferencesample .

This part of data included mutation information for 175 genes which were related to breast cancer [13] . “1” means there are mutations on the gene while “0” means there are no mutations.

Overall survival refers to whether the patient is alive (1) or dead (0).

Through two-sample t-tests on numerical feature based on survival and non-survival group (There are 801 samples in survival group and 1103 samples in non-survival group), all 6 numerical features were significantly different between survival and non-survival group. Age at diagnosis was statistically significantly lower in the survival group (survival = 56.46 ± 11.37 yr, non-survival = 64.44 ± 13.04 yr, P = 6.61 × 10⁻⁴²), lymph nodes examined was statistically significantly lower in the survival group (survival = 1.21 ± 2.72, non-survival = 2.57 ± 4.75, P = 5.11 × 10⁻¹³), mutation count was significantly different between survival and non-survival group (survival = 5.32 ± 3.34, non-survival = 5.96 ± 4.48, P = 7.94 × 10⁻⁴), Nottingham Prognostic Index was statistically significantly lower in the survival group (survival = 3.85 ± 1.07, non-survival = 4.17 ± 1.18, P = 1.48 × 10⁻⁹), overall survival months was statistically significantly lower in the non-survival group (survival = 159.55 ± 71.65 months, non-survival = 100.12 ± 69.57 months, P = 4.11 × 10⁻⁶⁸), and tumor size was statistically significantly lower in the survival group (survival = 23.32 ± 13.05, non-survival = 28.36 ± 16.19, P = 7.06 × 10⁻¹³).

χ² analysis was also used on categorical feature based on survival and non-survival groups. Nine out of all 22 categorical features were significantly different between survival and non-survival groups, including type of breast surgery, cancer type detailed, chemotherapy Pam50 + claudin-low subtype, and so on. Type of breast surgery is statistically different between survival and non-survival groups (P = 4.24 × 10⁻¹⁶). Cancer type is not statistically different between survival and non-survival groups (P = 0.87). Cancer type detailed is statistically different between survival and non-survival groups (P = 0.04). Cellularity is not statistically different between survival and non-survival groups (P = 0.32). Chemotherapy is not statistically different between survival and non-survival groups (P = 0.05). Pam50 + claudin-low subtype is statistically different between survival and non-survival groups P = 2.57 × 10⁻³). Cohort is statistically different between survival and non-survival groups (P = 0.03). Er status measured by ihc is not statistically different between survival and non-survival groups (P = 0.38). Er status is not statistically different between survival and non-survival groups (P = 0.42). Neoplasm histologic grade is statistically different between survival and non-survival groups (P = 0.02). Her2 status measured by snp6 is not statistically different between survival and non-survival groups (P = 0.08). Her2 status is not statistically different between survival and non-survival groups (P = 0.17). Tumor other histologic subtype is not statistically different between survival and non-survival groups (P = 0.89). Hormone therapy is not statistically different between survival and non-survival groups (P = 0.20). Inferred menopausal state is statistically different between survival and non-survival groups (P = 1.35 × 10⁻¹³). Integrative cluster is not statistically different between survival and non-survival groups (P = 0.11). Primary tumor laterality is not statistically different between survival and non-survival groups (P = 0.06). Oncotree code is statistically different between survival and non-survival groups (P = 0.04). Pr status is not statistically different between survival and non-survival groups (P = 0.35). Radio therapy is statistically different between survival and non-survival groups (P = 1.28 × 10⁻⁶). 3-gene classifier subtype is statistically different between survival and non-survival groups (P = 1.86 × 10⁻⁶). Tumor stage is not statistically different between survival and non-survival groups (P = 0.76).

For all features, whether it has missing value was detected, and the percentage of missing value was calculated. The information was shown in Table 2. According to the percentage, features were sorted in descending order.

All percent of missing value < 30%, the samples which include missing value were removed in the process of modeling.

For all clinical features, they were split into two groups: numerical features and categorical features.

All numerical features were selected out and checked, and their basic information was shown in Table 3.

For every numerical feature, its histogram and boxplot were drawn to see its distribution and frequency (Figure 1 and Figure 2).

All categorical clinical features were selected out. For every categorical feature, the numbers of different kinds in the categorical features were shown (Figures 3-6).

Besides clinical features, genetic features and mutation features were also included in analysis (not shown here because of the size).

In the training part, overall survival was the target feature; all other features were training features. For the 1980 primary breast cancer samples, 20% of them were chosen as test data. What is more, aiming at splitting the data in the same way every time to avoid the bias brought by different split ways, a seeded random state was set.

In this part, pycaret module was imported to compare different models by using accuracy, AUC, recall and other evaluation indexes. The result was shown in Table 4. Drawing these evaluation indexes of different models (Figure 13), lightgbm (Light Gradient Boosting Machine) model was selected as the training model according to the best comprehensive efficiency. Lightgbm model was used in the subsequent part of training and analysis.

After evaluating the model performance, the lightgbm model was used to train the data and the optimal values for the models were determined by hyperparameter tuning.

In the hyperparameter tuning part, learning rate, feature fraction, num leaves, max depth were tuned for better efficiency. Learning rate: the default setting is 0.1, and the general setting is between 0.05 and 0.1. Choosing a smaller learning rate can obtain more stable and better model performance. Feature fraction: the default setting is 1.0, the value of feature_fraction lies between 0.0 and 1.0, if feature fraction is less than 1.0, the model lightgbm will randomly select some features in each iteration. For example, if it is set to 0.8, 80% features will be

LightGBM: Light Gradient Boosting Machine; GBC: Gradient Boosting Classifier; KNN: K Neighbors Classifier; ADA: Ada Boost Classifier; LR: Logistic Regression; RF: Random Forest Classifier; SVM: SVM—Linear Kernel; ET: Extra Trees Classifier; DT: Decision Tree Classifier; Ridge: Ridge Classifier; LDA: Linear Discriminant Analysis; QDA: Quadratic Discriminant Analysis; NB: Naive Bayes.

selected before each tree is trained. By this way, the training can be accelerated. It can also be used to prevent over-fitting. Num leaves: The number of leaf nodes on a tree. The default setting is 31, which cooperates with max depth to empty the shape of the value tree. It is a key parameter that needs to be adjusted, which has a great influence on the model performance. Max depth: Maximum depth of the tree model, which is the most important parameter to prevent over-fitting. It is also the core parameter that needs to be adjusted, which plays a decisive role in model performance and generalization ability.

In the process of tune-up, several values of every model hyperparameter were selected and tuned as following: learning rate = [0.1, 0.3, 0.6], feature fraction = [0.5, 0.8, 1], num leaves = [16, 32, 64], max depth = [−1, 2, 3, 4]. Gridsearchcv was mainly applied in this part. Grid search was to traverse every intersection in the grid to find the best combination. The dimension of the grid was the number of hyperparameters. And by using cross validation = 5, all datasets were divided into five parts, one part was taken as the test set each time without repetition, and the other four parts were trained to create the model. The best model with the least mean-square error was selected out.

According to the part analysis on feature separated by target feature, only some genetic feature and mutation feature were significantly different between the survival and non-survival group. Therefore, for genetic and mutation feature of the new training data, only genetic and mutation features which were significantly different between the two groups were included. By comparing the model performance evaluation parameters of the original and new data, the better one was chosen and used for training.

In summary, for original data without selection and new data with selection, two models (before tune-up and after tune-up) were built separately. In total, four models and their model evaluation index were shown in Table 5.

From the four models selected, the lightgbm model after tune-up which used data with feature selection had the best comprehensive efficiency. The accuracy of the test data set of it was 0.8215, precision of the model was 0.8219, recall was 0.8215, F1 score was 0.8217, Roc_auc score was 0.8708 and the roc_auc curve was shown (Figure 14).

The average value of how much contribution each feature had made to each tree in the random forest was calculated and compared (Figure 15). In this part, gini index was used as the calculation method of contribution. G i n i ( p ) = ∑ k = 1 K p k ( 1 − p k ) (K categories, p k is the sample weight of category k). In this figure, the 5 most important features were age at diagnosis, cohort, Nottingham Prognostic Index, rheb and nr3c1.

Partial dependence curve is a visual analysis method to evaluate the influence of a certain dimension feature on the model output, and this method is also independent of the global model. In general, this method gives the average model output score for all possible values of the features input by the model, and a curve is drawn. This method is based on a fundamental assumption: the input of the model is independent.

The trend of partial dependence curve can only represent whether the influence of this feature on the output of the model is positive or negative from the average point of view. Features that can score a larger peak gap tend to be more important.

Through analysis, age at diagnosis, Nottingham Prognostic Index, cohort, rheb and nr3c1 were important to the target feature (Figure 16).