TITLE:
Smaller & Smarter: Score-Driven Network Chaining of Smaller Language Models
AUTHORS:
Gunika Dhingra, Siddansh Chawla, Vijay K. Madisetti, Arshdeep Bahga
KEYWORDS:
Large Language Models (LLMs), Smaller Language Models (SLMs), Finance, Networking, Supervisor Model, Scoring Function
JOURNAL NAME:
Journal of Software Engineering and Applications,
Vol.17 No.1,
January
23,
2024
ABSTRACT: With the continuous evolution and expanding applications of Large Language Models (LLMs), there has been a noticeable surge in the size of the emerging models. It is not solely the growth in model size, primarily measured by the number of parameters, but also the subsequent escalation in computational demands, hardware and software prerequisites for training, all culminating in a substantial financial investment as well. In this paper, we present novel techniques like supervision, parallelization, and scoring functions to get better results out of chains of smaller language models, rather than relying solely on scaling up model size. Firstly, we propose an approach to quantify the performance of a Smaller Language Models (SLM) by introducing a corresponding supervisor model that incrementally corrects the encountered errors. Secondly, we propose an approach to utilize two smaller language models (in a network) performing the same task and retrieving the best relevant output from the two, ensuring peak performance for a specific task. Experimental evaluations establish the quantitative accuracy improvements on financial reasoning and arithmetic calculation tasks from utilizing techniques like supervisor models (in a network of model scenario), threshold scoring and parallel processing over a baseline study.