E21 Computer Engineering Fundamentals
October 29, 2026


\[ \left[\begin{array}{ccc|c} A^*_{11} & A^*_{12} & A^*_{13} & b^*_1 \\ 0 & A^*_{22} & A^*_{23} & b^*_2 \\ 0 & 0 & A^*_{33} & b^*_3 \end{array}\right] \]
\[x_3 = \frac{b_3^*}{A_{33}^*}\] \[x_2 = \frac{b_2^* - A_{23}^* x_3}{A_{22}^*}\] \[x_1 = \frac{b_1^* - A_{13}^* x_3 - A_{12}^* x_2}{A_{11}^*}\]
\[\boxed{x_k = \frac{1}{A^*_{kk}} \left( b^*_k - \sum^n_{j = k +1} \left[ A^*_{kj} x_j \right] \right)}\]
The symbol \(\sum^n_{j = k +1}\) means sum over the term inside square brackets using index \(j\) from \(j = k+1\) to \(j = n\) (inclusive)
The \(*\) reminds us that backward substitution is performed on the row-reduced augmented matrix \(A^* | b^*\), not the originally given problem \(A | b\).
Simplify the given problem to row-reduced form
Solve \(Ax = b\) where \[ A =\begin{bmatrix} 2 & 1 & -1 & 0\\ 1 & 1 & 2 & 0\\ -1 & 2 & 1 & 1\\ 6 & 1 & 1 & -2 \end{bmatrix}, \quad b = \begin{bmatrix} 1 \\ -1 \\ 0 \\ 2 \end{bmatrix}. \]
By first obtaining a row-echelon form \[ \left[ \begin{array}{cccc|c} A_{11} & A_{12} & A_{13} & A_{14} & b_1\\ 0 & A_{22} & A_{23} & A_{24} & b_2 \\ 0 & 0 & A_{33} & A_{34} & b_3 \\ 0 & 0 & 0 & A_{44} & b_4 \end{array} \right] \]
The solution is \[\displaystyle \left[ \begin{array}{cccc|c} {2} & {1} & {-1} & {0} & {1}\\ {0} & {0.5} & {2.5} & {0} & {-1.5} \\ {0} & {0} & {-12} & {1} & {8} \\ {0} & {0} & {0} & {-5/6}& {7/3} \end{array} \right]\]
Copy into your Python workspace
Write one line of code for each step of Gaussian elimination
Original Matrix
\[ \begin{bmatrix} 2 & 1 & -1 & 0 \\ 1 & 1 & 2 & 0 \\ -1 & 2 & 1 & 1 \\ 6 & 1 & 1 & -2 \end{bmatrix} \]
Step 1
\[ \begin{bmatrix} 2 & 1 & -1 & 0 \\ 0 & 1/2 & 5/2 & 0 \\ -1 & 2 & 1 & 1 \\ 6 & 1 & 1 & -2 \end{bmatrix} \]
Step 2
\[ \begin{bmatrix} 2 & 1 & -1 & 0 \\ 0 & 1/2 & 5/2 & 0 \\ 0 & 5/2 & 1/2 & 1 \\ 6 & 1 & 1 & -2 \end{bmatrix} \]
Step 3
\[ \begin{bmatrix} 2 & 1 & -1 & 0 \\ 0 & 1/2 & 5/2 & 0 \\ 0 & 5/2 & 1/2 & 1 \\ 0 & -2 & 4 & -2 \end{bmatrix} \]
Step 4
\[ \begin{bmatrix} 2 & 1 & -1 & 0 \\ 0 & 1/2 & 5/2 & 0 \\ 0 & 0 & -12 & 1 \\ 0 & -2 & 4 & -2 \end{bmatrix} \]
Step 5
\[ \begin{bmatrix} 2 & 1 & -1 & 0 \\ 0 & 1/2 & 5/2 & 0 \\ 0 & 0 & -12 & 1 \\ 0 & 0 & 14 & -2 \end{bmatrix} \]
Step 6
\[ \begin{bmatrix} 2 & 1 & -1 & 0 \\ 0 & \frac{1}{2} & \frac{5}{2} & 0 \\ 0 & 0 & -12 & 1 \\ 0 & 0 & 0 & -\frac{5}{6} \end{bmatrix} \]
Any square matrix \(A\) can be written as the product of an upper triangular matrix and a lower traingular matrix, which we call \(U\) and \(L\) respectively.
\(A = L U\)
\(Ax = b \implies L U x = b\)
Let \(Ux = y\). Then, \(L y = b\)
Why do this?
\[L = \begin{bmatrix} L_{11} & 0 & 0 \\ L_{21} & L_{22} & 0 \\ L_{31} & L_{32} & L_{33} \end{bmatrix}, U = \begin{bmatrix} U_{11} & U_{12} & U_{13} \\ 0 & U_{22} & U_{23} \\ 0 & 0 & U_{33} \end{bmatrix}\]
\[L = \begin{bmatrix} 1 & 0 & 0 \\ L_{21} & 1 & 0 \\ L_{31} & L_{32} & 1 \end{bmatrix}, U = \begin{bmatrix} U_{11} & U_{12} & U_{13} \\ 0 & U_{22} & U_{23} \\ 0 & 0 & U_{33} \end{bmatrix}\]
In engineering, we often encounter banded matrices
\[\begin{bmatrix} A_{11} & A_{12} & 0 & 0 & 0 & 0 \\ A_{21} & A_{22} & A_{23} & 0 & 0 & 0 \\ 0 & A_{32} & A_{33} & A_{34} & 0 & 0 \\ 0 & 0 & A_{43} & A_{44} & A_{45} & 0 \\ 0 & 0 & 0 & A_{54} & A_{55} & A_{56} \\ 0 & 0 & 0 & 0 & A_{65} & A_{66} \end{bmatrix}\]
A tridiagonal matrix has 3 bands surrounding the diagonal.
Specialized solvers can be used for these; don’t waste operations on the zeros!
The \(LU\) Decomposition of a banded matrix preserves the banded structure. e.g.,
\[L = \begin{bmatrix} L_{11} & 0 & 0 & 0 & 0 & 0 \\ L_{21} & L_{22} & 0 & 0 & 0 & 0 \\ 0 & L_{32} & L_{33} & 0 & 0 & 0 \\ 0 & 0 & L_{43} & L_{44} & 0 & 0 \\ 0 & 0 & 0 & L_{54} & L_{55} & 0 \\ 0 & 0 & 0 & 0 & L_{65} & L_{66} \end{bmatrix}, U = \begin{bmatrix} U_{11} & U_{12} & 0 & 0 & 0 & 0 \\ 0 & U_{22} & U_{23} & 0 & 0 & 0 \\ 0 & 0 & U_{33} & U_{34} & 0 & 0 \\ 0 & 0 & 0 & U_{44} & U_{45} & 0 \\ 0 & 0 & 0 & 0 & U_{55} & U_{56} \\ 0 & 0 & 0 & 0 & 0 & U_{66} \end{bmatrix}\]
Solve \(Ax=b\) using LU Decomposition when
\[A = \begin{bmatrix} d_1 & e_1 & 0 & 0 & \cdots & 0 \\ c_1 & d_2 & e_2 & 0 & \cdots & 0 \\ 0 & c_2 & d_3 & e_3 & \cdots & 0 \\ 0 & 0 & c_3 & d_4 & \cdots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 0 & c_{n-1} & d_n \end{bmatrix}\, c = \begin{bmatrix} c_1 \\ c_2 \\ \vdots \\ c_{n-1} \end{bmatrix}\, d = \begin{bmatrix} d_1 \\ d_2 \\ \vdots \\ d_n \end{bmatrix}\, e = \begin{bmatrix} e_1 \\ e_2 \\ \vdots \\ e_{n-1} \end{bmatrix}\]
Gaussian Elimination of second row of \(A\):
\[\boxed{\text{Row k} = \text{Row k} - \frac{A_{21}}{A_{11}} \times \text{Row }(k-1)}\]
Write down a formula for \(c_1\), \(d_2\) and \(e_2\) under one step of Gaussian elimination, in terms of all other \(c\)’s, \(d\)’s and \(e\)’s.
\[ \begin{align} \text{New } c_1 &= c_1 - \frac{c_1}{d_1} \times d_1 \implies c_1 = 0 \\ \text{New } d_2 &= d_2 - \frac{c_1}{d_1} \times e_1 \implies d_2 = d_2 - \frac{c_1}{d_1} e_1 \\ \text{New } e_2 &= e_2 - \frac{c_1}{d_1} \times 0 \implies e_2 = e_2 \end{align} \]
The \(LU\) Decomposition of \(A\) is therefore
\[L = \begin{bmatrix} 1 & 0 & 0 & 0 & \cdots & 0 \\ \lambda_1 & 1 & 0 & 0 & \cdots & 0 \\ 0 & \lambda_2 & 1 & 0 & \cdots & 0 \\ 0 & 0 & \lambda_3 & 1 & \cdots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 0 & \lambda_{n-1} & 1 \end{bmatrix}, \quad U = \begin{bmatrix} d_1^* & e_1 & 0 & 0 & \cdots & 0 \\ 0 & d_2^* & e_2 & 0 & \cdots & 0 \\ 0 & 0 & d_3^* & e_3 & \cdots & 0 \\ 0 & 0 & 0 & d_4^* & \cdots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 0 & 0 & d_n^* \end{bmatrix}\]
Recall that \(LUx = b\) can be solved in two steps:
Forward substitution of \(Ly = b\)
\(\left[\begin{array}{cccccc|c} 1 & 0 & 0 & 0 & \cdots & 0 & b_1 \\ \lambda_1 & 1 & 0 & 0 & \cdots & 0 & b_2 \\ 0 & \lambda_2 & 1 & 0 & \cdots & 0 & b_3 \\ 0 & 0 & \lambda_3 & 1 & \cdots & 0 & b_4 \\ \vdots & \vdots & \vdots & \vdots & \cdots & \vdots & \vdots \\ 0 & 0 & \cdots & 0 & \lambda_{n-1} & 1 & b_n \end{array}\right]\)
Backward substitution of \(Ux = y\)
\(\left[\begin{array}{cccccc|c} d_1^* & e_1 & 0 & \cdots & 0 & 0 & y_1 \\ 0 & d_2^* & e_2 & \cdots & 0 & 0 & y_2 \\ 0 & 0 & d_3^* & \cdots & 0 & 0 & y_3 \\ \vdots & \vdots & \vdots & & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & \cdots & d_{n-1}^* & e_{n-1} & y_{n-1} \\ 0 & 0 & 0 & \cdots & 0 & d_n^* & y_n \end{array}\right]\)
E21 • Fall 2026 • Lecture 16 • October 29, 2026 • ↩︎