Nonlinear Programming, 3rd Edition, 2016

Convex Optimization Algorithms, 2015

Convex Optimization Theory, 2009


This series of complementary textbooks cover all aspects of continuous optimization, and its connections with discrete optimization via duality. The two convex optimization books deal primarily with convex, possibly nondifferentiable, problems and rely on convex analysis. By contrast the nonlinear programming book focuses primarily on analytical and computational methods for possibly nonconvex differentiable problems. It relies primarily on calculus and variational analysis, yet it still contains a detailed presentation of duality theory and its uses for both convex and nonconvex problems.

The books share notation, style, and prerequisites, and are aimed at students, researchers, and practitioners, roughly at the first year graduate level. They rely on rigorous analysis, but they also aim at an intuitive exposition that makes use of geometric visualization. They are supported by substantial internet-accessible material, including exercises with solutions, lecture slides, and course content from MIT Open Courseware. Three other complementary and more specialized books of the author can also be freely downloaded: the 1998 Network Optimization book, the 1989 Parallel and Distributed Computation book (coauthored with J. Tsitsiklis), and the 1982 Constrained Optimization and Lagrange Multiplier Methods book.

Instructors that adopt any one book as a main textbook for a course may contact the publishing company Athena Scientific for a free adoption copy.


nonlincover.jpg

This book provides an up-to-date, comprehensive, and rigorous account of nonlinear programming at the first year graduate student level. It covers descent algorithms for unconstrained and constrained optimization, Lagrange multiplier theory, interior point and augmented Lagrangian methods for linear and nonlinear programs, duality theory, and major aspects of large-scale optimization.

The third edition of the book is a thoroughly rewritten version of the 1999 second edition. New material was included, some of the old material was discarded, and a large portion of the remainder was reorganized or revised. The new material covers a variety of topics, such as first order methods, proximal algorithms, alternating direction methods of multipliers, and conic programming; also large-scale optimization topics of much current interest, such as incremental methods, and distributed asynchronous computation, and their applications in machine learning, signal processing, neural network training, and big data applications.

Another objective of the third edition to integrate its contents with those of the other books, together with internet-accessible material, so that they complement each other and form a unified whole. To this end, bridges to the other books have been provided, with extensive references to generalizations, discussions, and elaborations of the analysis.


Click here to visit the book's web site at Athena Scientific for exercise solutions, slides, and other instructional material, or to order the book directly from the publisher for faster service.

Click here to download the Table of Contents and the Preface.

Click here to order the book from amazon.com.

Click here to download the Newton Method code referenced in the last appendix of the book.


convexalg_cover.jpg

This book provides a comprehensive and accessible presentation of algorithms for solving convex optimization problems. It relies on rigorous mathematical analysis, but also aims at an intuitive exposition that makes use of visualization where possible. This is facilitated by the extensive use of analytical and algorithmic concepts of duality, which by nature lend themselves to geometrical interpretation. The book places particular emphasis on modern developments, and their widespread applications in fields such as large-scale resource allocation problems, signal processing, and machine learning.

The book is aimed at students, researchers, and practitioners, roughly at the first year graduate level. It is similar in style to the author's 2009 Convex Optimization Theory book, but can be read independently. The latter book focuses on convexity theory and optimization duality, while the 2015 Convex Optimization Algorithms book focuses on algorithmic issues. The two books share notation, and together cover the entire finite-dimensional convex optimization methodology. To facilitate readability, the statements of definitions and results of the "theory book" are reproduced without proofs in Appendix B.


Click here to visit the book's web site at Athena Scientific for exercises with solutions, slides, and other instructional material, or to order the book directly from the publisher for faster service.

Click here to download Preface and Table of Contents, and Chapter 1

Click here to order the book from amazon.com.


convexdualitycover.jpg

This book aims at an insightful, concise, and rigorous treatment of the basic theory of convex sets and functions in finite dimensions, and the analytical/geometrical foundations of convex optimization and duality theory. It provides much of the mathematical background needed for the in-depth reading of the Nonlinear Programming and Convex Optimization Algorithms books.

Convexity theory is first developed in a simple accessible manner, using easily visualized proofs. Then the focus shifts to a transparent geometrical line of analysis to develop the fundamental duality between descriptions of convex functions in terms of points, and in terms of hyperplanes. Finally, convexity theory and abstract duality are applied to problems of constrained optimization, Fenchel and conic duality, and game theory to develop the sharpest possible duality results within a highly visual geometric framework.

The book is aimed at students, researchers, and practitioners, roughly at the first year graduate level. It is similar in style to the author's 2015 Convex Optimization Algorithms book, but can be read independently. The latter book focuses on algorithmic issues, while the 2009 Convex Optimization Theory book focuses on convexity theory and optimization duality. The two books share notation, and together cover the entire finite-dimensional convex optimization methodology.


Click here to visit the book's web site at Athena Scientific slides, and other instructional material, or to order the book directly from the publisher for faster service.

Click here to download the Entire Book.

Click here to visit the book's web site to obtain an Extensive Set of Exercises with Statements and Complete Solutions.

Click here for the Statements of all the Mathematical Results.

Click here to order the book from amazon.com.


Constrained Optimization and Lagrange Multiplier Methods

Dimitri P. Bertsekas


This reference textbook, first published in 1982 by Academic Press, is a comprehensive treatment of some of the most widely used constrained optimization methods, including the augmented Lagrangian/multiplier and sequential quadratic programming methods.


Review :

"This is an excellent reference book. The author has done a great job in at least three directions. First, he expertly, systematically and with ever-present authority guides the reader through complicated areas of numerical optimization. This is achieved by carefully explaining and illustrating (by figures, if necessary) the underlying principles and theory. Second, he provides extensive guidance on the merits of various types of methods. This is extremely useful to practitioners. Finally, this is truly a state of the art book on numerical optimization."
S. Zlobec, McGill University, in SIAM Review


The book may be downloaded from here or can be purchased from the publishing company, Athena Scientific. (ISBN 1-886529--04-3, 400 pages, softcover)


Constrained Optimization and Lagrange Multiplier Methods


  1. Introduction

    1. General Remarks
    2. Notation and Mathematical Background
    3. Unconstrained Optimization
    4. Constrained Minimization
    5. Algorithms for Minimization Subject to Simple Constraints
    6. Notes and Sources

  2. The Method of Multipliers for Equality Constrained Problems

    1. The Quadratic Penalty Function Method
    2. The Original Method of Multipliers
    3. Duality Framework for the Method of Multipliers
    4. Multiplier Methods with Partial Elimination of Constraints
    5. Asymptotically Exact Minimization in the Method of Multipliers
    6. Primal-Dual Methods Not Utilizing a Penalty Function
    7. Label Correcting Methods
    8. Notes and Sources

  3. The Method of Multipliers for Inequality Constrained and Nondifferentiable Optimization Problems

    1. One-Sided Inequality Constraints
    2. Two-Sided Inequality Constraints
    3. Approximation Procedures for Nondifferentiable and Ill-Conditioned Optimization Problems
    4. Notes and Sources

  4. Exact Penalty Methods and Lagrangian Methods

    1. Nondifferentiable Exact Penalty Functions
    2. Linearization Algorithms Based on Nondifferentiable Exact Penalty Functions
    3. Differentiable Exact Penalty Functions
    4. Lagrangian Methods - Local Convergence
    5. Lagrangian Methods - Global Convergence
    6. Notes and Sources

  5. Nonquadratic Penalty Functions - Convex Programming

    1. Classes of Penalty Functions and Corresponding Methods of Multipliers
    2. Convex Programming and Duality
    3. Convergence Analysis of Multiplier Methods
    4. Rate of Convergence Analysis
    5. Conditions for Penalty Methods to be Exact
    6. Large Scale Integer Programming Problems and the Exponential Method of Multipliers
    7. Notes and Sources

  6. References

  7. Index



























































Visits since April 9, 2019