LLMs are advanced artificial intelligence models built on deep learning architectures. These models undergo pre-training on massive text corpora, allowing them to learn complex linguistic structures and patterns. Typically comprising hundreds of millions to hundreds of billions of parameters, they exhibit exceptional language comprehension and generation capabilities, allowing them to perform complex natural language processing tasks.

In 2018, OpenAI introduced GPT (Generative Pre-trained Transformer) [11], a language model with 150 million parameters that pioneered the era of large-scale pre-trained language models. Subsequently, models such as GPT2 [12] and GPT3 [13] were released, continuously pushing the boundaries of training dataset sizes, parameter scales, and performance, thereby advancing the field of NLP. By 2023, multiple companies including Meta, Baidu, Alibaba, and DeepSeek had launched LLMs with parameter counts exceeding tens of billions. Among them, several globally recognized large language models were trained on datasets ranging from hundreds of billions to trillions of tokens.

Compared to traditional deep learning models, LLMs exhibit two distinctive characteristics: first, their parameter complexity is extremely high, with counts ranging from billions to hundreds of billions; second, the scale of their training datasets is immense. These two core features endow LLMs with remarkable comprehension and generation capabilities. Taking the GPT series as an example, GPT1 had 117 million parameters [11], GPT2 increased to 1.5 billion parameters [12], and GPT3 reached a staggering 175 billion parameters [13]. Alongside the significant growth in parameter scale, the training datasets also expanded accordingly. GPT1 was trained on BookCorpus, a dataset containing approximately 7,000 unpublished books with a data volume of about 4.6 GB [11]. GPT2 used WebText, a dataset constructed from 8 million Reddit articles that received at three upvotes, comprising 40 GB of text data [12]. GPT3 utilized an even more extensive training dataset, including two undisclosed book datasets, an expanded version of WebText, and a filtered Common Crawl dataset, totaling over 570 GB of text data [13].

The training corpora for LLMs encompass diverse resources such as books, internet text, news articles, and forum posts. During training, the models learn linguistic rules and knowledge from these corpora and store this information internally as parameters. Consequently, when a model receives a text sequence as input, it can generate corresponding outputs based on these parameters.

In-Context Learning (ICL) enables large language models to perform tasks directly through natural language prompts without requiring parameter updates or additional training. When provided with appropriate task descriptions and representative demonstrations, LLMs can effectively interpret task requirements and generate appropriate responses. The carefully designed input sequence used to guide the model, known as a prompt, typically comprises three essential components: the task specification, demonstration examples, and the target query.

For effective ICL, the task specification must clearly define the objective and expectations, while the demonstration set consists of input-output pairs illustrating similar problems and their solutions. For example, in machine translation, demonstrations would show source texts with their corresponding translations. These illustrative examples help the model recognize task-specific patterns and expected output formats.

Research has shown that model performance strongly depends on prompt construction strategies, including both the formulation of task instructions and the selection of demonstrations. Recent studies indicate that curating examples based on semantic similarity to the target problem significantly enhances performance in reasoning tasks. The LENS methodology further advances this approach by employing support vector selection for demonstration assembly, demonstrating consistent accuracy improvements across various text classification benchmarks.

This paper aims to build an automated fact verification and question answering system based on LLM-ICL technology for tabular data analysis. Our research initiative serves two primary purposes: firstly, it provides the basic functional framework for subsequent research; secondly, it explores the potential of current LLM-ICL tabular reasoning methods in practical applications and identifies potential problems. A key technical challenge we address is the contradiction between token restrictions and reasoning with large tabular data. To systematically tackle this challenge, our research can be divided into eight sub-tasks, unfolding in three phases, with the specific technical roadmap shown in Figure 1Figure 1.

4.2.1. Data Preparation

This paper uses the IPUMS dataset as an example to validate the effectiveness of the structured data prediction algorithm based on large language models. The dataset originates from the Integrated Public Use Microdata Series (IPUMS) database and includes various features describing individual social attributes such as age, occupation category, education level, and profession. The prediction target is to determine whether an individual’s annual income exceeds $50,000.

Relying on social common sense, one can make a reasonable prediction of an individual’s income level based solely on social attributes provided by the dataset, such as “age” and “occupation category”, without the need for complex data analysis. This characteristic provides an experimental basis for investigating whether the prior knowledge embedded in the parameters of large language models can positively influence the prediction task.

Figure 1. Technical roadmap for LLM-ICL-Based table reasoning system development.

Specifically, the dataset includes 14 attribute variables, i.e., d = 14 , with the feature vector X → ∈ ℝ 14 , where each component corresponds to an attribute in Table 2 (one row). Among these, eight are discrete variables, including work unit nature, education level, marital status, occupation, social relationship, race, gender, and nationality; six are continuous variables, including age, census enumerator serial number, years of education, capital gains, capital losses, and hours worked per week. Table 2 provides the variable types and specific meanings. The prediction target is whether an individual’s annual income exceeds $50,000, i.e., Y ∈ ℝ , where Y = 1 indicates that the individual’s annual income exceeds $50,000, and Y = 0 indicates that it does not.

Table 2.Variable types and descriptions of the IPUMS dataset.

4.2.2. Selection of LLM Models

Given the diversity of Large Language Models (LLMs), several different LLM models-including GPT-3.5, Ernie-lite,Qwen-plus, Deepseek-V3.1, and Llama-4-will be used for systematic evaluation. A performance comparison will be conducted to determine the performance of each model across various specific tasks, including processing speed, accuracy, and adaptability to the economic and social development data of Xiangxi.

4.2.3. System Setup

Based on the performance evaluation results, the selected models will be integrated and optimized to improve the input-output process and adapt to the economic and social development data of western Hunan. Furthermore, multi-model fusion strategies will be explored so that the system can automatically select the most suitable model or model combination based on the nature of the question. Through model selection and strategy optimization, an efficient and accurate question-answering system prototype will be built, and system construction and preliminary testing will be conducted to ensure its stability and response speed, laying the foundation for the next stage of technological exploration and iterative improvement.

To improve the performance of table reasoning, this study will explore technologies from several aspects, including prompting strategies, selection of contextual content, and table serialization methods, to address challenges such as token quantity limitations. The accuracy of the system’s fact verification and question answering based on the evaluation indicators determined in the previous stage will be iteratively optimized.

Prompting Strategies: By designing effective prompting strategies, the model can be guided to reason along the correct thought path, thereby improving the quality and accuracy of the answers. A prompting strategy can be selected at runtime based on the specific statement/question. Various prompting strategies, including direct prediction, self-consistent decoding, thought chain, thought tree, and thought graph, will be tested.

Contextual Content Selection: Since LLMs typically have input token length limitations and computational costs are positively correlated with the number of tokens, how to fit as much accurate and relevant contextual content as possible into a limited input space is a key issue. Solving this problem is of great significance for complex data and large table data application scenarios. The plan is to draw on the large table processing methods in research and the vector database-similarity search method commonly used in current research.

Table serialization refers to transforming complex tabular data into a format more suitable for language models to process, thereby improving the LLM’s ability to understand and process table content. Currently testable and improveable table serialization methods include table-to-text description conversion, key-value pair conversion, entity relation extraction, and cell coordinate encoding.

To rigorously evaluate the predictive performance of LLM-ICL in few-shot settings, this study employs a dual baseline approach, using XGBoost as the representative parametric machine learning algorithm and KNN for non-parametric approaches.

XGBoost: Developed by Chen Tianqi and colleagues, XGBoost [14] is an open-source machine learning tool based on the second-order tree algorithm GBDT (Gradient Boosting Decision Tree). The tool incorporates significant improvements to the original GBDT method at both algorithmic and engineering levels. In recent years, XGBoost has emerged as one of the most stable and effective machine learning tools available.

KNN: The K-Nearest Neighbors algorithm, introduced by Cover et al. [15], represents a non-parametric machine learning approach. This method identifies the K closest samples to the target instance in the feature space and determines the classification based on the majority class among these neighbors. KNN’s strength lies in its simplicity and intuitive nature, as it operates without relying on specific assumptions or model structures.

To assess LLM-ICLs’ predictive capability with limited samples, this research randomly selects 10 samples from the IPUMS dataset as the training set for both XGBoost and KNN algorithms, denoted as D 1 = { ( X i , Y i ) , i = 1 , 2 , ⋯ , 10 } , while using 100 randomly chosen samples as the test set, represented as D 2 = { ( X i , Y i ) , i = 1 , 2 , ⋯ , 100 } . A grid search strategy is implemented to optimize hyperparameters for both XGBoost and KNN algorithms. The optimal hyperparameter values are determined through five-fold cross-validation on the training dataset.

When applying the proposed LLM-ICL-based prediction framework, the “demonstration dataset” in the prompt consists of exactly 10 samples, meaning the contextual learning for LLM-ICLs is constrained to this limited sample size. For comprehensive evaluation of LLM-ICL performance, this study examines several prominent models: OpenAI’s GPT-3.5-turbo [16], Baidu’s Ernie-lite [17], Alibaba’s Qwen-plus [18], DeepSeek’s Deepseek-V3.1 [19], and Meta’s Llama-4 [20].

The evaluation employs accuracy, the most commonly used metric for binary classification tasks, defined as the proportion of correctly predicted samples. LLM-ICL outputs of “yes” correspond to predictions of “annual income exceeding $50,000”, while “no” indicates “annual income below $50,000”. If an LLM-ICL generates “uncertain” as output, it is considered an incorrect prediction regardless of the actual income level. To mitigate random variability, the study adopts a multiple random sampling strategy, with the final accuracy metric representing the average across all test iterations.

Table 3 presents the accuracy performance of XGBoost and KNN on the test dataset with equivalent training sample sizes, compared against LLM-ICLs utilizing the same 10 samples as demonstration examples. With a training sample size of N = 10, The XGBoost and KNN models achieve accuracy rates of 0.735 and 0.711, respectively. Although performance fluctuates across different sample sets, this value consistently falls below the accuracy levels attained by LLM-ICLs using equivalent sample quantities. These results demonstrate that LLM-ICLs substantially outperform supervised machine learning algorithms in scenarios with limited training data.

Further analysis reveals that when employing only 10 demonstration samples, the top-performing Qwen-plus model achieves an accuracy of 0.794. Remarkably, KNN and XGBoost require approximately 500 and 16,000 samples respectively to reach comparable accuracy levels. This finding underscores LLM-ICLs’ exceptional capability in handling structured data prediction tasks. Even with minimal samples, LLM-ICLs leverage their powerful learning capacities to achieve accuracy comparable to supervised machine learning models trained on substantially larger datasets, highlighting their significant potential for structured data prediction applications.

Table 3. Accuracy comparison of different algorithms on test dataset (presented as mean ± standard deviation).

Table 4 presents a systematic comparative analysis of how “random sampling” and “nearest neighbor selection” strategies affect large language model prediction performance across varying sample sizes. In this carefully controlled experimental series, all data features were converted using natural language descriptions to ensure fair comparison. The experimental results unequivocally demonstrate that when maintaining identical sample quantities, models utilizing the nearest neighbor strategy for constructing demonstration sets significantly outperform those using random sampling. This consistent pattern confirms that selecting demonstration samples with high similarity to target samples effectively enhances model performance in structured data prediction tasks, thereby aligning with and extending findings observed in natural language processing applications.

As the number of demonstration samples increased from 3 to 10, all models exhibited steady improvement in prediction accuracy, regardless of the selection strategy employed. This indicates that providing more examples initially helps the model better understand the task pattern.

However, when sample size was further expanded to 15, certain models showed performance degradation. This phenomenon reveals a crucial characteristic of LLMs’ in-context learning mechanism: model performance does not continuously improve with increasing sample size.

Potential explanations for this pattern include: first, the “lost-in-the-middle” effect, where excessively long input sequences may impair models’ ability to identify critical information; second, an overabundance of demonstration examples could complicate pattern recognition processes, as the model struggles to identify the most relevant reasoning path among too many examples.

Additionally, the fixed context window of Transformer architectures forces a trade-off: more demonstration samples consume limited context resources, potentially leaving insufficient space for the model to effectively process the target query.

Table 4. Prediction accuracy under different sample selection strategies and sample sizes.