10.34 Numerical Methods Applied to Chemical Engineering

This course is currently part of a trial of the Open Course Ware (OCW) program at MIT (http://web.mit.edu/ocw/). The goal of this project is to make available to the general public lecture material from MIT courses. The in-house development site of the OCW homepage for 10.34 may be found at https://ocw-int.mit.edu/10/10.34/f02/index.html. As the semester progresses, lecture notes from 10.34 will be posted as pdf files under the section titles listed below.

Lecture Material

NOTE: Further program examples may be found in the assignments page

Program Development Document Example: die_flow_1D_program_development.htm

When writing a program from scratch, it is best to organize first the logical flow of data in the program before you begin to write code. To help this planning process, I have attached an example of a Program Development Document, a tool that you can use to help organize the job or writing a computer program. The basic idea is to plan out the structure of the program in terms that are comfortable and familiar to you before you start worrying about how to convert these ideas into computer code. Once you develop a program development document, all you have to do is paste the lines as comments into a file, and add after each comment the code that performs the tasks that you have outlined. Experience teaches that this is a more efficient and quicker way to write programs that work than simply to sit and the keyboard and start typing blindly. Of course, programming style is highly individual, so you will not be required to write these program development documents, and you are free adapt this approach to your own way of solving problems.

 

Chapter 1. Linear algebra I

1.1. Linear systems of algebraic equations

1.1.1. Expressing systems of linear algebraic equations as Ax = b

1.1.2. Definitions of vectors, real and complex numbers

1.2.3. Matrix addition and matrix/vector multiplication

1.2. The method of Gaussian elimination

1.2.1. Reduction of Ax = b to triangular form by Gaussian elimination

1.2.2. Gauss-Jordan elimination

1.2.3. Pivoting techniques in Gaussian elimination

1.3. Existence and uniqueness of solutions

1.3.1. Interpreting Ax = b as a linear transformation

1.3.2. Multiplication of matrices / matrix transpose

1.3.3. Basis sets and Gram-Schmidt orthogonalization

1.3.4. Null space (kernel) and existence/uniqueness of solutions

1.3.5. The determinant of a square matrix

1.3.6. The inverse of a square matrix

1.4. Matrix decomposition

1.4.1. LU decomposition

1.4.2. Cholesky decomposition of symmetric matrices

Chapter 2. Systems of nonlinear algebraic equations

2.1. Solution of a single nonlinear algebraic equation

2.1.1. Newton and secant methods for solving a nonlinear algebraic equation and Taylor series in 1D

2.2. Solving multiple nonlinear algebraic equations

2.2.1. Newton’s method for multiple nonlinear algebraic equations

2.2.2. Estimating the Jacobian matrix and quasi-Newton methods

MATLAB Program: approx_Jacobian_FD.m. This routine uses a simple finite difference approach to approximate the Jaobian matrix. An option exists to compute the Jacobian as a sparse matrix of known structure, but no effort is made to employ graph theory to reduce the number of finite difference operations.

2.2.3. Reduced-step Newton’s method

MATLAB PROGRAM: Newton_2D_test1b_v2.m. This program compares the performance of the full-step and reduced-step Newton's methods, where in the latter method a weak line search is implemented to improve robustness. To run this program, you must also save the m-files Newton_2D_test1b_calc_f.m and Newton_2D_test1b_calc_Jac.m, that calculate the vector of function values and the Jacobian for this problem. In addition, the program calls the function reduced_Newton.m that implements the reduced-step Newton's method algorithm.

MATLAB PROGRAM: reduced_Newton.m. This program implements a reduced-step Newton's method algorithm to solve a set of nonlinear algebraic equations. The method is rather robust, but may still "hang up" when the search direction becomes nearly orthogonal to the direction of steepest descent. A possible modification to correct this failure would be to implement a Levenberg-Marquardt type of modification of the Jacobian, adding a small positive scalar times the identify matrix to the Jacobian to push the method toward the direction of steepest descent when the weak line search results in very small step sizes.

2.2.4. Example : CSTR steady-state with two reactions

MATLAB PROGRAM : SS_CSTR_2_rxn_scan_Q.m. This program simulates a CSTR at steady state with the two reactions A + B --> C and B + C --> D. The concentrations of each species are plotted as functions of the volumetric flow rate. The program uses the MATLAB nonlinear solver fsolve(), part of the optimization toolkit.

Chapter 3. Linear algebra II

3.1. Matrix eigenvalue problems

3.1.1. Definition of eigenvalues/eigenvectors

3.1.2. Estimating eigenvalues; Gershorgin’s theorem

MATLAB PROGRAM: test_Gershorgin.m. This program takes an input matrix, compute the bounds on the eigenvalues based on Gershorgin's theorem, and then computes the eigenvalues. A graph is made showing how the eigenvalues fit within the bounds.

3.1.3. Completeness of eigenvector basis sets; eigenvalue properties of special matrices

3.1.4. Numerical calculation of eigenvalues/eigenvectors

MATLAB PROGRAM: QR_eig.m. This program demonstrates the iterative QR approach to calculating the eigenvalues of a matrix. This routine is meant to demonstrate the method only, use the command eig() and eigs() for actual computations.

3.1.5. Eigenvalue technique for polynomial root finding

MATLAB PROGRAM: get_poly_roots.m. This program demonstrates how eigenvalue calculations can be used to compute all of the roots of a polynomial.

3.2. Singular value decomposition

3.2.1. Singular value decomposition and singular problems

MATLAB PROGRAM: analyze_SVD.m. This program analyzes the existence and uniqueness properties of a system of linear algebraic equations using singular value decomposiiton.

3.2.2. SVD of overdetermined systems

MATLAB PROGRAM: test_data_CVD.m. This program demonstrates the use of SVD to perform linear least squares data analysis.

Chapter 4. Function-space discretization

4.1. Introduction to function-space discretization

4.1.1. A single electron trapped in a 1-D well

MATLAB program: quantum_1D.m. This MATLAB program uses a basis function exapansion to solve for the quantum states of a particle trapped in a 1-D infinite well with an arbitrary external potential energy function. The program uses a basis set exapansion is sine function s to convert the eigenvalue differential equation into a matrix eigenvalue problem. The program file also includes various subroutines for specific examples of external potential energy functions. Type "harmonic_V_ext" to compute the energy levels of a harmonic spring.

4.1.2. Applications of functional analysis

4.2. Functional analysis and linear operator theory

4.2.1. Fundamentals of functional analysis

4.2.2. Eigenfunction expansions and the solution of BVP’s in 1 spatial dimension

MATLAB Program: plot_Greens_fcn_1D.m. This program plots the Green's function for the 1-D second derivative with zero-value Dirichlet boundary conditions. The forms obtained from both eigenfunction expansion and the "physical" approach from shell balances are compared.

4.2.3. Eigenfunctions of the Laplacian in multiple Cartesian coordinates

MATLAB Program: plot_2D_duct_flow.m. This program plots the analytical solution, expressed as an eigenfunction expansion, of the steady-state velocity profile for laminar, pressure-driven flow down a rectangular duct.

4.2.4. Eigenfunctions of the Laplacian in cylindrical coordinates

MATLAB Program: plot_Bessel_function.m. This program plots a few leading-order Bessel functions of the first and second kinds.

MATLAB Program: fit_Bessel.m. This program fits a truncated expansion in Bessel functions to a function f(r), demonstrating the use of the orthogonality relations for Bessel functions.

MATLAB Program: Poiseuille_startup.m. This program uses expansion in Bessel functions to compute analytically the time-varying velocity profile of laminar flow in a circular tube upon imposition of an axial pressure gradient.

4.2.5. Eigenfunctions of the Laplacian in spherical coordinates

4.3. Numerical methods in quantum mechanics

4.3.1. Linear operator theory in quantum mechanics

4.4. Orthogonal polynomials

4.4.1. Construction of common polynomials

4.4.2. Gaussian quadrature

MATLAB Program: test_integration.m. This program demonstrates various methods within MATLAB to compute definite integrals numerically.

4.4.3. Orthogonal collocation

4.5. Fourier Analysis

4.5.1 Fourier Series and Transforms in 1D

MATLAB Program: get_Fourier_series.m. This program computes the Fourier coefficients (using sine and cosine basis functions) of an input function measured at various (not necessarily discrete) times.

MATLAB Program: get_Fourier_transform.m. This program computes uses FFT to compute and plot the Fourier transform and power spectrum of a discretely-sampled signal. Optionally, a low-pass digital filter can be applied to the data prior to computing the Fourier transform.

MATLAB Program: results_get_Fourier_transform.m. This script file invokes get_Fourier_transform.m to demonstrate its use.

4.5.2. Convolution and correlations of functions

MATLAB Program: compute_convolution.m. This MATLAB program uses convolution to compute the time-varying output of a system to a given input signal, where the time-domain response function of the system is known. This program automatically performs zero-padding prior to computing Fourier transforms to avoid wraparound problems.

MATLAB Program: damped_spring_response.m. This program demonstrates the response-function behavior of a damped spring.

MATLAB Program: get_response_example.m. This script demonstrates the use of Fourier analysis to estimate the response function of a damped spring from input and output data.

MATLAB Program: get_response.m. This program uses deconvolution to estimate the response function of a system from input and output data.

MATLAB Program: correct_signal_example.m. This program uses deconvolution to correct a measured signal to remove the bias in the detector.

MATLAB Program: correlation_example.m. This program uses Fourier analysis to compute the correlation of two functions.

4.5.3. Fourier transforms in multiple dimensions

MATLAB Program: example_fft2.m This program demonstrates the computation and interpretation of 2-D Fourier transforms.

MATLAB Program: example_2D_scatter.m. This program demonstrates, in 2D, how scattering information can be used to probe spatial structure.

Chapter 5. Real-space discretization

5.1. Introduction to real-space discretization

5.1.1. Overview of methods

5.1.2. Shooting vs. relaxation methods

5.1.3. Finite difference calculation of 1-D fluid flow

5.1.4. Characteristics and types of PDE’s

5.2. Solving sparse systems of linear algebraic equations

5.2.1. Elimination methods for sparse systems

5.2.2. Simple iterative methods

5.2.3. Conjugate-gradient method

5.3. Method of finite differences

5.3.1. Finite difference calculations for 1-D BVP's

MATLAB Program: polymer_flow_1D_v3.m. This program computes the velocity profile of a polymeric shear-thinning fluid in steady, pressure-driven, laminar flow between two parallel, semi-infinite flat-plates. A shear thinning generalized Newton fluid constitutive model is used. Finite differences are used to convert the boundary value problem to a set of nonlinear algebraic equations. These are then solved using the MATLAB fsolve() routine. The user enters the simulation parameters through an input file, an example of which is polymer_flow_1D_input1.m.

5.3.2. Finite difference calculations with multiple spatial dimensions

5.4. Method of finite elements

5.4.1. Finite element method in 1-D

5.4.2. Finite elements in 2-D and 3-D for solid mechanics

5.4.3. Finite elements in 2-D and 3-D for transport problems

5.5. Method of finite volumes

5.5.1. Finite volume method for 2-D transport problems

Chapter 6. Initial value problems

6.1. Introduction to initial value problems

6.1.1. Examples of ODE-IVP’s

6.1.2. Systems of linear first-order ODE’s

6.1.3. Eigenvalue stability analysis

6.2. Explicit vs. implicit methods and stiffness

6.2.1. Explicit vs. implicit Euler methods

6.2.2. Treatment of stiff systems

6.3. Higher-order methods

6.3.1. High-order single-step methods

6.3.2. High-order multi-step methods

6.3.3. Stoer-Bulirsch method

6.4. Methods for DAE (Differential-Algebraic Equation) systems

6.4.1. Predictor-corrector methods for index-1 problems

6.5. Parametric continuation

6.5.1. Pseudo-arc length continuation

6.6. Time-symmetric methods for energy conservation

6.6.1. Time integration algorithms for molecular dynamics

Chapter 7. Probability Theory

7.1. Applications of probability theory

7.1.1. Probability theory in science and engineering

7.2. Basic probability theory

7.2.1. Random variables, joint and conditional probabilities

7.2.2. Markov chains

7.3. Common probability distributions

7.3.1. The Gaussian(normal) and binomial distributions

7.3.2. Random walks

7.3.3. The Poisson distribution

7.4. Monte Carlo methods

7.4.1. Monte Carlo integration

7.4.2. Monte Carlo sampling of a distribution

7.4.3. Kinetic Monte Carlo

Chapter 8. Linear least squares analysis

8.1. Introduction to regression

8.1.1. Regression problems in science and engineering

8.2. Statistical basis of the least squares method

8.2.1. Bayes’ theorem and the c2 parameter

8.2.2. Least-squares estimation and the Gauss-Markov conditions

8.3. Single-response linear least-squares analysis

8.3.1. Linear least squares matrix computations

8.3.2. Covariance of model parameters and predictions

8.3.3. SVD methods applied to linear least squares

8.4. Analysis of residuals

8.4.1. The t and F distributions

8.4.2. Generating confidence intervals for parameters and model predictions

8.5. Design of experiments

8.5.1. Common factorial designs

8.5.2. Use of residual analysis to improve data gathering

8.6. Examples of single-response regression

8.6.1. Hypothesis testing

8.6.2. Comparison of multiple populations

8.6.3. Model parameter estimation

8.7. Multi-response linear least squares analysis

8.7.1. Linear least squares with multiple observed variables

8.8. Treating failure of Gauss-Markov conditions

8.8.1. Scaling methods

8.8.2. Estimating the error covariance

Chapter 9. Nonlinear parameter estimation

9.1 Nonlinear parameter estimation as optimization problem

9.1.1. Minimization of c2 as function of model parameters

9.2. Numerical techniques for nonlinear optimization

9.2.1. Newton and quasi-Newton methods

9.2.2. Steepest descent

9.2.3. Conjugate-gradient method

9.2.4. Levenberg-Marquardt method

9.3. Analysis of residuals

9.3.1. Linearized approach to construction of confidence intervals

9.3.2. Monte Carlo methods

9.4. Nonlinear parameter estimation examples

9.4.1. Fitting of thermodynamic parameters

9.4.2. Fitting of kinetic rate parameters