Linear programming is a tool for optimizing numerous real-world problems such as the allocation problem. Consider a general linear program (LP) as the following problem 
where 
represents the objective function for 
variables
and the expression (2) describes 
rows of constraints for 
variables
.
Furthermore, the vector 0 in (3) is a column vector of zeros of appropriate dimension according to context. The dual of 
is considered the standard minimization LP problem. We focus here on the maximization case.
A COST RAD [
and 
and
. However in LPs, the components of 
and 
are not restricted to be nonnegative numbers.
Though simplex pivoting algorithms and polynomial interior-point barrier-function methods represent the two principal solution approaches to solve problem 
[
for which the method performs poorly [
An active-set framework for solving LPs will be analogous to that of Saito et al. [
, with a single artificial bounding constraint such as 
or 
for sufficiently large 
so as not to reduce the feasible region of
.
A series of relaxations 
of 
is formed by adding one or more violating constraints from set (2). The constraints that have been added are called operative constraints, while constraints that still remain in (2) are called inoperative constraints. Eventually a solution 
of 
is obtained when none of the inoperative constraints are violated; i.e., for no inoperative constraint 
is
. In the LP COSTs of this paper we explore 1) the ordering of a set of inoperable constraints for possibly adding them to the current operable constraints and 2) the actual selection of a group of such constraints to be added at an iteration.
Active-set approaches have been studied in the past, including those by Stone [
to form
. These methods are called SUB and VIOL here, respectively, as in Saito et al. [
More recent work on constraint selection has focused on choosing the violated inoperative constraints considered most likely to be binding at optimality for the original problem 
according to a particular constraint selection criterion. In the cosine criterion, the angle between normal vector 
of (2) and normal vector 
of (1) as measured by the cosine,
is considered. Naylor and Sell ([
a single inoperative constraint 
of 
violating 
and having the largest
.
In Saito et al. [
formed with the normal vector 
of the objective function. Factor II is the depth of the cut that constraint 
removes as a violated inoperative constraint of
. From these two factors the constraint selection metric
was developed. This constraint selection metric was utilized in conjunction with a multi-bound and multi-cut technique [
. In this paper we rename the COST RAD of [
Although NRAD performs extremely well for NNLPs, we show here that its superiority over traditional algorithms is less dramatic for the general LP (1)-(3). Hence, the contribution of this paper includes using the principles of NRAD to develop a new COST for the general LP, which we refer to as GRAD. Even though GRAD and NRAD share similar principles, GRAD is a significant modification of NRAD. Indeed, GRAD solves general LPs seven times faster on the average than NRAD in our computational experiments.
The remainder of this paper is organized as follows. GRAD with multi-cuts is developed in Section 2, and an interpretation is given. In Section 3, we present computational results where GRAD is compared to the CPLEX simplex methods, CPLEX barrier method, and the activeset approaches SUB, VIOL, and NRAD. In Section 4 we offer conclusions and discuss future research.
is no longer guaranteed to be feasible for LP problems. Moreover, the optimal solution 
may not lie in the same orthant as the normal 
to a constraint. We must thus modify NRAD for NNLP to GRAD for LP in order to emulate the underlying reasoning of NRAD based on Factors I and II efficiently.
is a zero vector. However, this is not the case for LP. Therefore an initial bounded problem 
is formed by adding a bounding constraint such as
, along with some constraints from (2), as described in Section 2.3. 
is then solved to obtain an initial solution
. 
is generated by adding one or more inoperative constraints of 
that maximize the constraint selection metric for LP among all inoperative constraints of 
violating
. Define this constraint selection metric as
is a small positive constant. Thus GRAD seeks 
such that
are shifted by 
if the minimum value is nonpositive. Hence 
is always positive, and each term in (5) contributes additively to the criterion. GRAD (5) becomes the same as NRAD (4) when 
and
. Therefore it could be utilized to effectively solve NNLPs as well. Although equality constraints are not considered here, it should be noted that equality constraints could be included in
.
for NNLP is obtained by adding multiple constraints from (2) ordered by decreasing value of NRAD until no column of 
is a zero vector. Although this approach does not guarantee boundedness for LP, a generalization was found to be effective here.
is formed by adding an artificial bounding constraint such as
, as well as multiple constraints from (2) ordered by decreasing value of GRAD, until each column of 
has at least one positive and at least one negative coefficient (Step 1). After an optimal solution to the initial bounded problem is obtained by the primal simplex method (Step 2), subsequent iterations are solved by the dual simplex method (Step 3). Moreover, after the solution of 
and each subsequent
, constraints are again added in groups. 
is formed by selecting inoperative constraints in decreasing order of GRAD until both a positive coefficient and a negative coefficient are included for each variable 
(Step 3, lines 7-27). The following pseudocode describes the COST GRAD with the new multi-cut technique.
.
do 2: if 
then 3: 
then 6: 
, 
, 
,
and 
and 
do 11: Let 
and 
then 14: 
then 16: 
//case if there are no more constraints with 
and 
then 22: 
then 24: 
// case if there are no more constraints with 
, 
for the initial bounded problem 
is found.
false 2: while 
false do 3: if 
then 4: 
//
is an optimal solution to
.
, 
, 
and 
and 
do 8: Let 
and 
then
then 13: 
//case if there are no more constraints with 
and 
then 19: 
then 21: 
case if there are no more constraints with 
, 
defined by (7)-(10) using the dual simplex method to obtain
.
. However for LP of (1)-(3), the intersection of
, drawn from the origin, and 
may not necessarily lie in the feasible region. Moreover, the 
of NNLP are all positive. The GRAD constraint selection metric of (5) thus must be modified from 
in (4) to account for these facts.
is maximized when 
and 
are large. Hence, a larger value of 
is more likely to yield a larger value of
. This relationship implies that the left-hand side of the constraint is likely to be larger for larger values of
with
, it is hard to predict the value of 
in a solution. Consequently, we assume that the 
in which 
are all equally likely and have the nominal value 1. The left-hand side is now
makes a constraint more likely to be binding. We thus divide the left-hand side by 
to measure the 
constraint’s likelihood of being binding at optimality, resulting in
which is essentially GRAD.
for which the 
are negative. Consider (11) below.
and 
are both negative, and 
is large. However, it is found in Section 3.3.1 that the constraint selection metric performed better when the second term was
as shown in (5).
.
and 200,000 constraints 
having various densities ranging from 0.005 to 1. Randomly generated real numbers between 1 and 5, or between −1 and −5 were assigned to elements of
. To assure that the randomly generated LP had a feasible solution, a feasible solution 
(not all elements of 
were nonzero) was randomly generated to derive random
, where
. Then a feasible solution 
(having the same number of nonzero elements as
) was randomly generated to derive random
, where
. The ratio of the number of positive and negative elements of 
was one. The number of nonzero 
in each constraint was binomially distributed
. Additionally, we required each constraint to have at least two nonzero 
so that a constraint would not become a simple upper or lower bound on a variable. At each of the 21 densities, 5 random LP were generated. 
(ON) and 
(AUTO) were used for CPU times of the CPLEX primal simplex method, the CPLEX dual simplex method, and the CPLEX barrier method. No CPLEX preprocessing was implemented [used PREIND = 0 (OFF) and PREDUAL = −1 (OFF)] by the CPLEX primal simplex and dual simplex solvers as part of GRAD, NRAD, SUB, and VIOL.
and large positive 
with small 
as possible,” or “as small negative 
and small negative 
with large 
as possible.” However, when running LP problems utilizing NRAD (4), the constraint selection metric is
and 
are explicitly written out. The first term follows the above general strategyexcept when 
becomes negative. The second term should be subtracted, instead of added, from the first term in order for the constraint selection metric to take a higher value when giving priority to “as small negative 
and small negative 
with large 
as possible.” The 
in the second term should also be positive for the metric to work additively.
term of (12). The test problems of 
therefore the case was not specially handled. The second version,
, which was defined in (6). Variation 3,
was used for all results presented.
for every variable 
in forming a set of cuts for each iteration of an active-set method for LP. The rest of the comparisons are made with those methods utilizing the multi-cut method for LP.
in (14), (5), and (11) significantly improved the CPU times over (13). Between (11) and (14), (14) which only considered the 
term was faster. Going from (14) to GRAD, utilizing the term 
improved the CPU time slightly more, 5.7% on average.
if there are no constraints with negative 
in the inoperative set.
set, but not the artificial bounding constraint
. To implement SUB and VIOL as in previous work, a single bounding constraint 
was used.
†Average of 5 instances of LP at each density; ‡One constraint was added per iteration 
[
was used as the bounding constraint; ‖Multi-cut technique for LP was applied with 
as the bounding constraint; §One constraint was added per iteration 
[
was used as the bounding constraint; ¶One constraint was added per iteration r. 
was used as the bounding constraint.
The data in 
for a prior method, i.e. GRAD, became apparent when multi-cut is applied. In multi-cut VIOL, violating inoperative constraints had to be resorted in descending order of violation at every iteration 
. For densities between 0.02 and 1, GRAD outperformed the CPLEX primal simplex, dual simplex, and barrier solvers for LP (maximization) with long-and-narrow 
matrices. Moreover, GRAD for LP still maintains the attractive features of the dual simplex method such as a basis, shadow prices, reduced costs, and sensitivity analysis [
in addition to the global GRAD information for constraints obtained prior to the active-set iterations. It is conceivable that the rationale behind NRAD and GRAD could also lead to better integer programming cutting planes. Finally, it should be noted that any COST such as GRAD is a polynomial algorithm if the CPLEX barrier solver is used to solve each new subproblem with added constraints instead of the primal simplex or the dual simplex. Such a COST, however, performs extremely poorly in practice.