The identified issues in the ETL solutions that exist today are:

Addressing these challenges, especially for structured data saved in formats such as PDFs, requires an approach that utilizes LLM along with the SQL querying and RAG generating capabilities to make sense of the data and also query as necessary [4] . Leveraging both LLM-RAG and SQL agents makes processes more specific to operations. LLMs, trained on vast datasets, excel in processing prompts and generating outputs, interpreting user requests, assigning tasks to specialized agents, and creating queries for structured data. SQL excels in retrieving structured data from relational databases, while RAG efficiently retrieves data from unstructured documents and formulates responses. Using SQL and RAG agents based on data type, user prompts, and storage location enhances performance.

This paper introduces a novel method for extracting both structured and unstructured data and performing a web search for the entered prompt to generate the most relevant response. The project structure is shown in Figure 2 . The pipeline employs three LLMs; a primary LLM analyzes prompts alongside a knowledge graph for context. Details of the SQL agent, RAG agent, and Tavily web search agent are provided in later sections.

These existing instruments facilitate the extraction and conversion of raw data into JSON or CSV formats; however, they do not retain data in SQL-compatible formats. This limitation hinders their capability to support relational queries and joins with other structured data sources. Furthermore, unstract solely accepts data in PDF format.

The current tools are primarily designed for field-level extraction; however, they lack the capability to directly facilitate advanced relational queries, including conditional joins or aggregations across various tabular fields derived from multiple documents within the database [10] . This limitation further constrains the potential for bidirectional interaction with the database. JSON-based outputs necessitate additional transformations to integrate effectively with SQL databases, thereby introducing further complexity.

Moreover, the majority of extant solutions depend on predefined extraction models, which lack the capacity for context-sensitive processing [11] [12] .

Organizations require a comprehensive pipeline that can:

1) Extract and store structured PDF data as relational data to facilitate SQL-based analytics.

2) Seamlessly merge and query structured and unstructured data without needing additional ETL processes.

3) Validate and standardize extracted data using AI-powered models to ensure accuracy and usability.

We now identify the main features of our solution, Intelligent ETL:

We developed a Python-based workflow specifically designed for the detection and extraction of structured tabular data from PDF documents. This facilitates the seamless integration of extracted PDF data into pre-existing SQL databases, thereby enabling the execution of relational operations and complex queries. Additionally, we incorporate procedures for the extraction of unstructured data within our pipeline.

By incorporating large language models (LLMs) such as GPT-3.5-turbo, our method facilitates dynamic detection and verification of fields, thereby ensuring precise identification of essential fields. This approach affords greater flexibility and adaptability compared to predefined extraction models. Additionally, we utilize the primary LLM for output validation by scrutinizing it against the input prompt and activating the suitable tool when necessary. LLMs are observed to be the best tool for taking context-aware computations into account [13] - [16] .

We achieve a convergence of the ETL pipeline that processes both structured SQL data and unstructured PDFs, integrating them within a unified relational data model for analytical purposes. This integration represents a novel advancement, as traditionally, unstructured and structured datasets necessitate distinct pipeline processes, despite the possibility of PDFs harboring inherently structured data. In contrast to tools like unstract, our approach preserves the relational aspects of PDF data by embedding them within SQL tables. This methodology facilitates the direct interrogation of data using SQL, thus eliminating the requirement for additional transformations. This methodology reduces the requirement for separate ETL processes. Furthermore, it enhances the capacity for sophisticated analytics by enabling the execution of complex SQL queries on both structured and unstructured data, while maintaining data integrity through the normalization of extracted data into relational formats. The proposed solution is designed to be scalable, accommodating large datasets, and adaptable for implementation across various projects. Additionally, it supports the storage of xlsx, csv, pdf, and txt formats within databases, thus enabling the Intelligent ETL pipeline to conduct analyses on the encompassed data. In contrast to the unstract approach, this pipeline stores SQL data as opposed to JSON formatted data, thereby facilitating advanced, context-aware querying.

Through the integration of structured and unstructured data within a unified relational model within IntelligentETL, organizations can be empowered to conduct sophisticated analytics and reporting with ease. The implementation of GPT-based AI models introduces an additional layer of intelligence, rendering the system flexible and responsive to diverse user queries. We develop a chatbot-based procurement application utilizing Gradio and LangChain technologies and subsequently evaluate our assertions with respect to “unstrict”.

The reliance on LLMs especially under high-load scenarios, poses a possibility of encountering computational challenges such as computational cost and performance efficiency. This is because LLMs are resource-intensive and may struggle to maintain minimum latency while dealing with large datasets or while processing concurrent queries.

The mentioned challenges are mitigated in the proposed pipeline as the architecture incorporates a modular design that enables task-specific agents such as SQL and RAG and utilizes capabilities of the knowledge graph to handle discrete operations without fully engaging all the LLMs for every step of the process and for every query in the subsequent chat progression. The knowledge graph works as an efficient caching mechanism thus enabling the pipeline to produce accurate answers for context-relevant queries. Additionally, it also nullifies the requirement of even activating respective LLM-powered agents in the case of context-ready or repetitive queries, thus enabling the pipeline to be scalable.

This enhancement of data generation capabilities, coupled with the ongoing advancements in data storage, suggests an increasing importance of LLM-directed query agents and user-database interactions structured around meticulously designed knowledge graphs in the foreseeable future. Knowledge graphs elucidate the trajectory of system control flow, thereby guiding both developers and users in the creation of applications that encourage question formulation in a manner that, when processed through advanced ETL systems, is likely to yield precise cost-effective outcomes [17] - [19] .

Distilled versions of GPT could be further researched for further enhancement in the direction of making the pipeline ready for enterprise-scale deployment.

An in-detail workflow of an automated layout that detects tabular data in PDFs, extracts and transfers it to an SQLite database(s), and transfers it to the destination database that is specified, is shown in Algorithm 1. The solution leverages time-tested Python libraries to handle structured and unstructured data efficiently. pdfplumber is used to extract plain text from multiple PDF sources, camelot is used to extract detected tables from PDFs, pandas is used to refine tabular data and bring it into a structure, and sqlite3 and create _engine from sqlalchemy is used to connect the source and the destination of the data transfer and to save each data-frame in the database respectively. The pipeline is not only designed for scalable data ingestion but also for a versatile merger of multiple databases. Here an end-to-end PDF data merging to a predefined database is achieved. The Data Converter from Figure 2 depicts this workflow. Enterprise solutions exist that carry out similar tasks. This ETL pipeline is flexible enough to be used with those solutions as well. However, this simple workflow achieves accurate results with comparatively less to none error expectancy. Analytics of structured data presented in PDF is then ready to be carried out based on user prompts, the methodology will be explained in subsequent sections of this paper.

To mathematically represent the data matching and database merging operations for two data sets in different formats, let us consider the following model:

First, the tables are identified by the module and extracted from PDFs using efficient Python-AI libraries. Then:

For a given set of DataFrames, , each DataFrame is saved in an SQLite database:

(1)

(2)

where:

Set theory is used to model the merger of source and destination databases:

(3)

(4)

where:

Finally, the extracted text from each page is concatenated:

(5)

where:

The algorithm block of prompt inspection and agent identification is an intermediate block in the system and the next step after the structured and unstructured content ingestion. Three LangChain-linked LLM-powered tools have been designed for the system to function as intended. These are respectively, the SQL agent tool block, the RAG-based retrieval module tool block, and a Tavily-based web search tool block. Each block has its own LLM agents for content retrieval. However, this block employs a GPT-4o-mini primary agent LLM to dynamically study the prompt’s intent as well as content and then identify the tool that is most probable for handling the database with respect to the user’s input prompt and forwards the user prompt to the respective agent block. It also takes into account the previous query history based on the knowledge graph for context understanding. Additionally, a feature that allows users to upload documents containing structured information adds flexibility for user interaction. The structured data is extracted and stored in the database. The LLM agent ensures that the extracted data aligns precisely with both the entered prompt and the associated data from the PDF document uploaded by the user. The primary LLM agent’s logic for prompt inspection is enclosed within the route_tools() method as seen in Figure 3 . This method checks whether the question is associated with:

Each agent block is correlated with a specific data type as determined by the user’s prompt. For instance, operations conducted at the field level within a relational model—such as calculating the aggregate sum within a table or identifying the minimum and maximum values across all tables in a database—necessitate the engagement of the secondary agent from the SQL agent module to access structured data. Such operations are unattainable using solely a Large Language Model (LLM) agent that employs Retrieval-Augmented Generation (RAG) or functions independently of any auxiliary tool. In contrast, a user prompt necessitating a sequential response through policy search will activate the secondary LLM within the RAG tool module to generate an appropriate response by scrutinizing PDF documents. Moreover, the primary LLM ensures the adequacy of the extracted data in addressing the query before transmitting the final output to the user.

The core code for the SQL agent tool is as seen in Figure 4 . This agent block handles database-related queries by accessing the relational data stored in the database. As seen in the code, the SQLAgent is implemented to produce queries. The agent, a distinct tool, converts prompts into SQL queries with the guidance of a GPT-3.5-turbo LLM, which interprets the input from the primary LLM. Tests show GPT-3.5-turbo excels in writing and performing database queries compared to GPT-4o-mini. As seen in Figure 5 , the mentioned LLM is also responsible for analyzing the retrieved data that is returned in a structured format, against the input prompt and the uploaded document(s), and refactoring the SQL agent to design a more appropriate SQL query if needed. This ensures accurate output aligned with the user’s question and related uploaded documents in the database.

The SQL agent module performs operations like joins, min/max calculations, and aggregations. For instance, it joins invoice and product tables to calculate total sales per product, which enterprise solutions like Unstract cannot achieve, as discussed in the paper.

The core code for the RAG agent tool is as seen in Figure 6 . The RAG agent module utilizes a Chroma-vector-based search mechanism for the extraction of pertinent unstructured data from diverse sources. Its efficacy is particularly pronounced in the context of sequential text extraction, necessitated by the nature of text vector embeddings and the corresponding text, which is conventionally documented in JSON format. This agent is developed as an independent tool, tasked with the processing of text-based queries via the implemented pdfRAGTool. When the GPT-4o-mini agent identifies PDF-based policy data extraction needs, it triggers the tool in LangChain.

The RAG agent block is as seen in Figure 7 . The agent module embeds the document and prompt into vectors using OpenAI’s text-embedding-3-small model and stores them in JSON. It queries Chroma-based data for prompt-related information, which is then used by GPT-4o-mini to create a clear user response.

When a prompt involves an uploaded document with RAG models and LangChain, the tool uses vector embeddings to find and extract the most relevant sequence with GPT-4o-mini. The output is then summarized in human-readable format by the LLM within the RAG agent module. RAG is effective at synthesizing information from unstructured data, even in PDFs. GPT-4o-mini is used as both the primary and secondary LLM for its efficiency, lightweight nature, and free availability.

The workflow also integrates the Tavily tool which is most efficient for web-based searching capability. Tavily is an exclusive web search tool, designed to be easily integrated into AI-powered applications. If the primary agent deduces that the entered prompt requires web-based search, this tool is invoked. The Tavily tool in this workflow is initiated to return the 3 most relevant website links that most match the user prompt based on the analysis conducted by the primary agent.

The core code for control flow as depicted by the knowledge graph is as seen in Figure 8 . A knowledge graph effectively represents the control flow of ensemble systems, offering a concise view of the workflow and modules while aiding in analyzing interactions for context-accurate queries. This is crucial for designing and optimizing AI applications focused on timely accuracy. All the designed agents along with their respective tools and the user prompts are represented as nodes in a knowledge graph. The control transition between these nodes as seen in Figure 8 , is modeled as edges that denote the control flow of the system. The knowledge graph is visualized through high-level control flow modules as seen in the block diagram of the generated knowledge graph in Figure 9 . The State graph or knowledge graph is generated by the primary LLM denoting the control flow from the user prompts and the tools invoked, to the output prompt in the chatbot. The memory is saved for the previous state retention and routing in the MemorySaver object from langgraph.checkpoint module by logging checkpoints of the previous states of interaction.

With each interaction by the user, the knowledge graph undergoes dynamic expansion. The StateGraph object is configured and constructed by the principal LLM according to the flow of control across various tools and states. Performance

metrics are available for examination on the LangChain dashboard. The sequence of inputs, the agents invoked consequently, and the output delivered can thus be scrutinized. The memory preservation mechanism is activated to enable the system to record prior interactions, thereby enhancing the content-awareness and adaptability of ensuing dialogues. This process is referred to as state maintenance within the framework of a knowledge graph representation.

The modular design of the system, is suitable for integrating the pipeline with enterprise-grade software ecosystems such as Enterprise Resource Planning (ERP) via standardized APIs.

The pipeline as a module is also suitable to be integrated with the sub-modules of ERP such as Customer Relationship Management (CRM) for automating individual parts of the ecosystem.

The pipeline integrates into the ERP as seen in Figure 10 . The results for these functionalities are discussed in the subsequent section of the paper. The modules that the IntelligentETL pipeline replaces are:

1) Supply Chain Management (SCM) and CRM: The pipeline is responsible for extracting, validating, and integrating vendor contracts, invoices, purchase orders, and customer requirements into SQL-compatible formats. It enables analytics such as customer order history, and customer behavior insights based on the knowledge graph formation for each customer interaction.

2) Data Warehousing: The pipeline efficiently ingests and centralizes structured and unstructured data into a relational model. This facilitates not just efficient querying but also analytics-ready storage and dynamic data retrieval for enterprise-wide reporting.

3) Financial Resource Management (FRM): The extraction and validation of invoices, customer requirements, and payment data support advanced financial reconciliations through robust and accurate SQL query formulation.

The modules where the IntelligentETL pipeline lacks direct applicability are: Human Resource Planning (HRP), and Manufacturing Resource Planning (MRP). The pipeline supports inventory and supply chain-related data but does not cater to functionalities specific to MRP and manufacturing processes such as production scheduling, and shop control. Also, it does not address HRP-specific functionalities such as employee records, recruitment processes, and payroll analytics in its current architecture. Designing additional modules or integrations tailored to HRP and MRP use cases is necessary to extend the pipeline’s applicability to enterprise applications dealing with the automation of these processes. An additional module specifically designed for resource planning is then necessary to be integrated with the developed pipeline to account for HRP and MRP as per respective application requirements based on the environments such as financial auditing, supply chain management, and legal documentation workflows.

The validation process for data conversion speed is conducted herein. Subsequent to this, the accuracy of structured data extraction will be evaluated, and the resulting metrics will be compared with those derived from unstract.

The time taken for each of the operations is as given:

(6)

where:

The total time for the complete transformation workflow is given as:

(7)

where:

As seen in Figure 11 , the time analysis result for all the operations for data transformation explained above sums up to 0.07 seconds for an A4-sized PDF. This is on par with the enterprise-level software that performs the task of only data conversion.

Figure 12 shows the performance of the LLM-backed SQL agent. The application was structured using LangChain. As such, LangChain offers excellent performance metrics reporting services.

The performance metrics are displayed as seen in Figure 13 . The metrics were obtained for the SQL agent deployed via the LLM to facilitate database querying. Upon conducting tests with a variety of prompts, it was noted that the performance of the LLM-based RAG and Tavily agents was comparable to that of the LLM-based SQL agent. Moreover, the performance metrics for any future tools that might be integrated into this pipeline through the LLM could be similarly analyzed. The adaptability to incorporate additional tools seamlessly, coupled with the ease of performance assessment via LangChain capabilities, significantly enhances the pipeline’s customizability—an aspect that is notably absent in the unstract framework.

The query was initiated to determine the minimum and maximum amounts across all invoices within the database. The agent efficiently parsed through the extensive database, providing a 100% accurate response in only 6.50 seconds, based on 1399 generated tokens. Including the duration required for extracting content from PDFs and integrating it into the existing database, the entire pipeline process was completed in a total of 6.57 seconds.

This performance was evaluated against that of Unstract, which utilized a state-of-the-art GPT-3.5-based large language model (LLM) and LLM extractor, alongside a Qdrant database and an enterprise-provided LLM whisperer. As seen in Figure 14 , the prompts evaluated in this study were designed to extract the minimum and maximum values of the products, identify the product with the highest purchase frequency, and ascertain the individual who purchased the largest quantity of products alongside the total expenditure incurred. For an LLM-SQL agent, these queries are adequate to efficiently traverse the entire database, regardless of the volume of data stored, and produce precise results. Nonetheless, when executed on Unstract, the majority of these prompts either exceeded the designated time threshold or yielded inaccurate outcomes due to its sole reliance on LLM not only for prompt processing but also for data analysis. Observations indicate that Unstract requires an indefinite amount of time to execute complex field-based relational operations, defined in this context as operations taking more than 100 seconds. The time ellipsoid performance plot depicting the performance of one such comparative instance, as referenced herein, is illustrated in Figure 15 .

Having noticed these differences between the approach that we present and existing enterprise-ready software, we document these differences in a tabular structure as is seen in Table 1 . Consequently, Unstract demonstrates limitations in delivering persuasive outcomes for multi-condition filtering. Specifically, implementing a combination of functions such as WHERE(), HAVING(), and GROUP BY()—commonly compliant with SQL—is not effectively handled. Furthermore, the accurate output of specific mathematical function combinations, including MIN(), MAX(), SUM(), AVG(), and COUNT(), poses challenges for the software,

particularly when dealing with a single multi-paged document or multiple documents. Additionally, in scenarios where numerical and field comparisons are critical, such as in detecting inconsistencies between purchase orders and vendor quotes, Language Learning Models (LLMs) tend to be error-prone and necessitate human intervention for validation. However, an LLM-powered SQL agent is capable of supporting complex multi-field numerical comparisons, such as assessing discrepancies in price, quantity, and discount, to thoroughly analyze data mismatches.

As previously noted, the software is limited to processing PDF inputs and rendering outputs in JSON format, resulting in a significant inefficiency for subsequent operations on the database by utilizing the intermediate results. In contrast, the proposed pipeline incorporates an SQL agent that facilitates adjustments to administrative permissions, thereby permitting modifications to the data stored within the database. More conveniently, it can be deduced from Table 2 , that the level of operational flexibility that an LLM-powered SQL agent allows is not achieved with solutions that are developed leveraging the LLMs alone. Also, these solutions are always prone to certain error occurrences. Table 2 compares specific operation chains for structured data analysis which is not possible for Unstract to perform given its complete reliance on LLM alone.