SolitaryRoad.com

Website owner:  James Miller


[ Home ] [ Up ] [ Info ] [ Mail ]

  MAXIMA AND MINIMA OF FUNCTIONS OF SEVERAL VARIABLES, STATIONARY POINT, LAGRANGE’S METHOD OF MULTIPLIERS



Def. Stationary (or critical) point. For a function y = f(x) of a single variable, a stationary (or critical) point is a point at which dy/dx = 0; for a function u = f(x1, x2, ... , xn) of n variables it is a point at which


ole.gif


In the case of a function y = f(x) of a single variable, a stationary point corresponds to a point on the curve at which the tangent to the curve is horizontal. In the case of a function y = f(x, y) of two variables a stationary point corresponds to a point on the surface at which the tangent plane to the surface is horizontal.

 

In the case of a function y = f(x) of a single variable, a stationary point can be any of the following three: a maximum point, a minimum point or an inflection point. For a function y = f(x, y) of two variables, a stationary point can be a maximum point, a minimum point or a saddle point. For a function of n variables it can be a maximum point, a minimum point or a point that is analogous to an inflection or saddle point.


Maxima and minima of functions of several variables. A function f(x, y) of two independent variables has a maximum at a point (x0, y0) if f(x0, y0) ole1.gif f(x, y) for all points (x, y) in the neighborhood of (x0, y0). Such a function has a minimum at a point (x0, y0) if f(x0, y0) ole2.gif f(x, y) for all points (x, y) in the neighborhood of (x0, y0).


A function f(x1, x2, ... , xn) of n independent variables has a maximum at a point (x1', x2', ... , xn') if f(x1', x2', ... , xn') ole3.gif f(x1, x2, ... , xn) at all points in the neighborhood of (x1', x2', ... , xn'). Such a function has a minimum at a point (x1', x2', ... , xn') if f(x1', x2', ... , xn') ole4.gif f(x1, x2, ... , xn) at all points in the neighborhood of (x1', x2', ... , xn').



Necessary condition for a maxima or minima. A necessary condition for a function f(x, y) of two variables to have a maxima or minima at point (x0, y0) is that


             ole5.gif


at the point (i.e. that the point be a stationary point).


In the case of a function f(x1, x2, ... , xn) of n variables, the condition for the function to have a maximum or minimum at point (x1', x2', ... , xn') is that


             ole6.gif


at that point (i.e. that the point be a stationary point).



To find the maximum or minimum points of a function we first locate the stationary points using 1) above. After locating the stationary points we then examine each stationary point to determine if it is a maximum or minimum. To determine if a point is a maximum or minimum we may consider values of the function in the neighborhood of the point as well as the values of its first and second partial derivatives. We also may be able to establish what it is by arguments of one kind or other. The following theorem may be useful in establishing maximums and minimums for the case of functions of two variables.


Sufficient condition for a maximum or minimum of a function z = f(x, y). Let z = f(x, y) have continuous first and second partial derivatives in the neighborhood of point (x0, y0). If at the point (x0, y0)


                         ole7.gif


and


             ole8.gif


then there is a maximum at (x0, y0) if


             ole9.gif


and a minimum if


             ole10.gif


If Δ > 0 , point (x0, y0) is a saddle point (neither maximum nor minimum). If Δ = 0 , the nature of point (x0, y0) is undecided. More investigation is necessary.



Example. Find the maxima and minima of function z = x2 + xy + y2 - y .


Solution..


             ole11.gif


             ole12.gif



2x + y = 0

x + 2y = 1


x = -1/3 , y = 2/3


This is the stationary point. At this point Δ > 0 and


             ole13.gif


and the point is a minimum. The minimum value of the function is - 1/3.




ole14.gif

Maxima and minima of functions subject to constraints. Let us set ourselves the following problem: Let F(x, y) and G(x, y) be functions defined over some region R of the x-y plane. Find the points at which the function F(x, y) has maximums subject to the side condition G(x, y) = 0. Basically we are asking the question: At what points on the solution set of G(x, y) = 0 does F(x, y) have maximums? The solution set of G(x, y) = 0 corresponds to some curve in the plane. See Figure 1. The solution set (i.e. locus) of G(x, y) = 0 is shown in red. Figure 2 shows the situation in three dimensions where function z = F(x, y) is shown rising up above the x-y plane along the curve G(x, y) = 0. The problem is to find the maximums of z = F(x, y) along the curve G(x, y) = 0.

 


ole15.gif
ole16.gif

Let us now consider the same problem in three variables. Let F(x, y, z) and G(x, y, z) be functions defined over some region R of space. Find the points at which the function F(x, y, z) has maximums subject to the side condition G(x, y, z) = 0. Basically we are asking the question: At what points on the solution set of G(x, y, z) = 0 does F(x, y, z) have maximums? G(x, y, z) = 0 represents some surface in space. In Figure 3, G(x, y, z) = 0 is depicted as a spheroid in space. The problem then is to find the maximums of the function F(x, y, z) as evaluated on this spheroidal surface.                                                             

Let us now consider another problem. Suppose instead of one side condition we have two. Let F(x, y, z), G(x, y, z) and H(x, y, z) be functions defined over some region R of space. Find the points at which the function F(x, y, z) has maximums subject to the side conditions


2)        G(x, y, z) = 0

3)        H(x, y, z) = 0.


ole17.gif

Here we wish to find the maximum values of F(x, y, z) on that set of points that satisfy both equations 2) and 3). Thus if D represents the solution set of G(x, y, z) = 0 and E represents the solution set of H(x, y, z) = 0 we wish to find the maximum points of F(x, y, z) as evaluated on set F = D ole18.gif E (i.e. the intersection of sets D and E). In Fig. 4 G(x, y, z) = 0 is depicted as an ellipsoid and H(x, y, z) = 0 as a plane. The intersection of the ellipsoid and the plane is the set F on which F(x, y, z) is to be evaluated.                                            

The above can be generalized to functions of n variables F(x1, x2, ... , xn), G(x1, x2, ... , xn), etc. and m side conditions.  

 



Methods for finding maxima and minima of functions subject to constraints.


1. Method of direct elimination. Suppose we wish to find the maxima or minima of a function F(x, y) with the constraint Φ(x, y) = 0. Suppose we are so lucky that Φ(x, y) = 0 can be solved explicitly for y, giving y = g(x). We can then substitute g(x) for y in F(x, y) and then find the maximums and minimums of F(x, g(x)) by standard methods. In some cases, it may be possible to do this kind of thing. We express some of the variables in the equations of constraint in terms of other variables and then substitute into the function whose extrema are sought, and find the extrema by standard methods.


2. Method of implicit functions. Suppose we wish to find the maxima or minima of a function u = F(x, y, z) with the constraint Φ(x, y, z) = 0. We note that Φ(x, y, z) = 0 defines z implicitly as a function of x and y i.e. z = f(x, y). We thus seek the extrema of the quantity


            u = F(x, y, f(x, y)) .


The necessary condition for a stationary point, as given by 1) above, becomes

 

ole19.gif

 

(where F1 represents the partial of F with respect to x, etc.)


Taking partials of Φ with respect to x and y it follows that


ole20.gif

 

(since the partial derivative of a function that is constant is zero).                                         

From the pair of equations consisting of the first equation in 4) and 5) we can eliminate ∂z / ∂x giving

 

6)        F1Φ3 - F3Φ1 = 0


From the pair of equations consisting of the second equation in 4) and 5) we can eliminate ∂z / ∂y giving

 

7)        F2Φ3 - F3Φ2 = 0


Equations 6) and 7) can be written in determinant form as



ole21.gif


Equations 8) combined with the equation Φ(x, y, z) = 0 give us three equations which we can solve simultaneously for x, y, z to obtain the stationary points of function F(x, y, z). The maxima and minima will be among the stationary points.


This same method can be used for functions of an arbitrary number of variables and an arbitrary number of side conditions (smaller than the number of variables).


Extrema for a function of four variables with two auxiliary equations. Suppose we wish to find the maxima or minima of a function


            u = F(x, y, z, t)


with the side conditions


9)        Φ(x, y, z, t) = 0            ψ(x, y, z, t) = 0.


Equations 9) define variables z and t implicitly as functions of x and y i.e.


10)      z = f1(x,y)       t = f2(x, y) .


We thus seek the extrema of the quantity


            u = F(x, y, f1(x, y), f2(x, y)) .



The necessary condition for a stationary point, as given by 1) above, becomes

 

ole22.gif

 

Taking partials of Φ with respect to x and y it follows that


ole23.gif

 

Taking partials of ψ with respect to x and y it follows that


ole24.gif

 

From 12) and 13) we can derive the conditions


ole25.gif


Equations 14) combined with the auxiliary equations Φ(x, y, z, t) = 0 and ψ(x, y, z, t) = 0 give us four equations which we can solve simultaneously for x, y, z, t to obtain the stationary points of function F(x, y, z, t). The maxima and minima will be among the stationary points.



Extrema for a function of n variables with p auxiliary equations. The p equations corresponding to equation 14) above for the case of a function of n variables


                        u = F(x1, x2, ... .xn)


and p auxiliary equations (i.e. side conditions)


            Φ(x1, x2, ... , xn) = 0

            Ψ(x1, x2, ... , xn) = 0

            .................................


            Ω(x1, x2, ... , xn) = 0


are


 

ole26.gif


These p equations along with the p auxiliary equations


            Φ(x1, x2, ... , xn) = 0

            Ψ(x1, x2, ... , xn) = 0

            .................................


            Ω(x1, x2, ... , xn) = 0


can be solved simultaneously for the n variables x1, x2, ... .xn to obtain the stationary points of F(x1, x2, ... .xn). The maxima and minima will be among the stationary points.




*********************************                                                                               


Geometrical interpretation for extrema of function F(x, y, z) with a constraint. We shall now present a theorem that gives a geometrical interpretation for the case of extremal values of functions of type F(x, y, z) with a constraint.


Theorem 1. Suppose the functions F(x, y, z) and Φ(x, y, z) have continuous first partial derivatives throughout a certain region R of space. Let the equation Φ(x, y, z) = 0 define a surface S, every point of which is in the interior of R, and suppose that the three partial derivatives Φ1, Φ2, Φ3 are never simultaneously zero at a point of S. Then a necessary condition for the values of F(x, y, z) on S to attain an extreme value (either relative or absolute) at a point of S is that F1, F2, F3 be proportional to Φ1, Φ2, Φ3 at that point. If C is the value of F at the point, and if the constant of proportionality is not zero, the geometric meaning of the proportionality is that the surface S and the surface F(x, y, z) = C are tangent at the point in question.


Rationale behind theorem. From 8) above, a necessary condition for F(x, y, z) to attain a maxima or minima (i.e. a condition for a stationary point) at a point P is that


            F1Φ3 - F3Φ1 = 0                      F2Φ3 - F3Φ2 = 0


or


ole27.gif


Thus at a stationary point the partial derivatives F1, F2, F3 and Φ1, Φ2, Φ3 are proportional. Now the partial derivatives F1, F2, F3 and Φ1, Φ2, Φ3 represent the gradients of the functions F and Φ; and the gradient, at any point P, of a scalar point function ψ(x, y, z) is a vector that is normal to that level surface of ψ(x, y, z) that passes through point P. If C is the value of F at the stationary point P, then the vector (F1, F2, F3) at point P is normal to the surface F(x, y, z) = C at P. Similarly, the vector (Φ1, Φ2, Φ3) at point P is normal to the surface Φ(x, y, z) = 0 at P. Since the partial derivatives F1, F2, F3 and Φ1, Φ2, Φ3 are proportional, the normals to the two surfaces point in the same direction at P and the surfaces must be tangent at point P.


Example. Consider the maximum and minimum values of F(x, y, z) = x2 + y2 + z2 on the surface of the ellipsoid


             ole28.gif


Since F(x, y, z) is the square of the distance from (x, y, z) to the origin, it is clear that we are looking for the points at maximum and minimum distances from the center of the ellipsoid. The maximum occurs at the ends of the longest principal axis, namely at ( ole29.gif 8, 0, 0). The minimum occurs at the ends of the shortest principal axis, namely at (0, 0, ole30.gif 5). Consider the maximum point (8, 0, 0). The value of F at this point is 64, and the surface F(x, y, z) = 64 is a sphere. The sphere and the ellipsoid are tangent at (8, 0, 0) as asserted by the theorem. In this case the ratios G1:G2:G3 and F1:F2:F3 at (8, 0, 0) are 1/4 : 0 : 0 and 16 : 0 : 0 respectively.

 

This example brings out the fact that the tangency of the surfaces (or the proportionality of the two sets of ratios), is a necessary but not a sufficient condition for a maximum or minimum value of F, for we note that the condition of proportionality exists at the points (0, ole31.gif 6, 0), which are the ends of the principal axis of intermediate length. But the value of F in neither a maximum nor a minimum at this point.

 


Case of extrema of function F(x, y) with a constraint. A similar geometrical interpretation can be given to the problem of extremal values for F(x, y) subject to the constraint Φ(x, y) = 0. Here we have a curve defined by the constraint, and a one-parameter family of curves F(x, y) = C. At a point of extremal value of F the curve F(x, y) = C through the point will be tangent to the curve defined by the constraint.

 





3. Lagrange’s Method of Multipiers. Let F(x, y, z) and Φ(x, y, z) be functions defined over some region R of space. Find the points at which the function F(x, y, z) has maximums and minimums subject to the side condition Φ(x, y, z) = 0. Lagrange’s method for solving this problem consists of forming a third function G(x, y, z) given by


17)      G(x, y, z) = F(x, y, z) + λΦ(x, y, z) ,


where λ is a constant (i.e. a parameter) to which we will later assign a value, and then finding the maxima and minima of the function G(x, y, z). A reader might quickly ask, “Of what interest are the maxima and minima of the function G(x, y, z)? How does this help us solve the problem of finding the maxima and minima of F(x, y, z)?” The answer is that examination of 17) shows that for those points corresponding to the solution set of Φ(x, y, z) = 0 the function G(x, y, z) is equal to the function F(x, y, z) since at those points equation 17) becomes


            G(x, y, z) = F(x, y, z) + λ·0 .

Thus, for the points on the surface Φ(x, y, z) = 0, functions F and G are equal so the maxima and minima of G are also the maxima and minima of F. The procedure for finding the maxima and minima of G(x, y, z) is as follows: We regard G(x, y, z) as a function of three independent variables and write down the necessary conditions for a stationary point using 1) above:


18)      F1 + λΦ1 = 0                F2 + λΦ2 = 0                F3 + λΦ3 = 0


We then solve these three equations along with the equation of constraint Φ(x, y, z) = 0 to find the values of the four quantities x, y, z, λ. More than one point can be found in this way and this will give us the locations of the stationary points. The maxima and minima will be among the stationary points thus found.


Let us now observe something. If equations 18) are to hold simultaneously, then it follows from the third of them that λ must have the value


             ole32.gif


If we substitute this value of λ into the first two equations of 18) we obtain


            F1Φ3 - F3Φ1 = 0                      F2Φ3 - F3Φ2 = 0 


or


  ole33.gif


We note that the two equations of 19) are identically the same conditions as 8) above for the previous method. Thus using equations 19) along with the equation of constraint Φ(x, y, z) = 0 is exactly the same procedure as the previous method in which we used equations 8) and the same constraint.


One of the great advantages of Lagrange’s method over the method of implicit functions or the method of direct elimination is that it enables us to avoid making a choice of independent variables. This is sometimes very important; it permits the retention of symmetry in a problem where the variables enter symmetrically at the outset.


Lagrange’s method can be used with functions of any number of variables and any number of constraints (smaller than the number of variables). In general, given a function F(x1, x2, ... , xn) of n variables and h side conditions Φ1 = 0, Φ2 = 0, .... , Φh = 0, for which this function may have a maximum or minimum, equate to zero the partial derivatives of the auxiliary function F + λ1Φ1 + λ2Φ2 + ...... + λhΦh with respect to x1, x2, ... , xn , regarding λ1, λ2, ..... ,λh as constants, and solve these n equations simultaneously with the given h side conditions, treating the λ’s as unknowns to be eliminated.


The parameter λ in Lagrange’s method is called Lagrange’s multiplier.



References.

  Taylor. Advanced Calculus

  Osgood. Advanced Calculus.

  James and James. Mathematics Dictionary.

  Mathematics, Its Content, Methods and Meaning. Vol. I



More from SolitaryRoad.com:

The Way of Truth and Life

God's message to the world

Jesus Christ and His Teachings

Words of Wisdom

Way of enlightenment, wisdom, and understanding

Way of true Christianity

America, a corrupt, depraved, shameless country

On integrity and the lack of it

The test of a person's Christianity is what he is

Who will go to heaven?

The superior person

On faith and works

Ninety five percent of the problems that most people have come from personal foolishness

Liberalism, socialism and the modern welfare state

The desire to harm, a motivation for conduct

The teaching is:

On modern intellectualism

On Homosexuality

On Self-sufficient Country Living, Homesteading

Principles for Living Life

Topically Arranged Proverbs, Precepts, Quotations. Common Sayings. Poor Richard's Almanac.

America has lost her way

The really big sins

Theory on the Formation of Character

Moral Perversion

You are what you eat

People are like radio tuners --- they pick out and listen to one wavelength and ignore the rest

Cause of Character Traits --- According to Aristotle

These things go together

Television

We are what we eat --- living under the discipline of a diet

Avoiding problems and trouble in life

Role of habit in formation of character

The True Christian

What is true Christianity?

Personal attributes of the true Christian

What determines a person's character?

Love of God and love of virtue are closely united

Walking a solitary road

Intellectual disparities among people and the power in good habits

Tools of Satan. Tactics and Tricks used by the Devil.

On responding to wrongs

Real Christian Faith

The Natural Way -- The Unnatural Way

Wisdom, Reason and Virtue are closely related

Knowledge is one thing, wisdom is another

My views on Christianity in America

The most important thing in life is understanding

Sizing up people

We are all examples --- for good or for bad

Television --- spiritual poison

The Prime Mover that decides "What We Are"

Where do our outlooks, attitudes and values come from?

Sin is serious business. The punishment for it is real. Hell is real.

Self-imposed discipline and regimentation

Achieving happiness in life --- a matter of the right strategies

Self-discipline

Self-control, self-restraint, self-discipline basic to so much in life

We are our habits

What creates moral character?


[ Home ] [ Up ] [ Info ] [ Mail ]