Lecture 15

E21 Computer Engineering Fundamentals

Emad Masroor

October 27, 2026

Systems of Linear Equations

Linear systems of the form \[\boldsymbol{A} \boldsymbol{x} = \boldsymbol{b}\] are extremely common in engineering.

\[ \begin{bmatrix} A_{11} & A_{12} & A_{13} \\ A_{21} & A_{22} & A_{23} \\ A_{31} & A_{32} & A_{33} \\ A_{41} & A_{42} & A_{43} \\ A_{51} & A_{52} & A_{53} \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \\ b_4 \\ b_5 \end{bmatrix} \]

\[ \begin{aligned} A_{11}x_1 + A_{12}x_2 + A_{13}x_3 &= b_1 \\ A_{21}x_1 + A_{22}x_2 + A_{23}x_3 &= b_2 \\ A_{31}x_1 + A_{32}x_2 + A_{33}x_3 &= b_3 \\ A_{41}x_1 + A_{42}x_2 + A_{43}x_3 &= b_4 \\ A_{51}x_1 + A_{52}x_2 + A_{53}x_3 &= b_5 \end{aligned} \]

Motivating Example from E6

Find the reaction forces \(H_2\), \(V_2\) and \(V_3\) and all internal forces

Assembling a matrix-vector equation

The matrix \(\boldsymbol{A}\) multiplies a vector of unknown quantities \(\boldsymbol{x}\) and equals a right-hand-side vector \(\boldsymbol{b}\)

\[ \begin{bmatrix} 1 & 0 & 0 & \cos 30^\circ & 1 & 0 \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & \sin 30^\circ & 0 & \sin 60^\circ \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ \end{bmatrix} \begin{bmatrix} H_2 \\ V_2 \\ V_3 \\ F_{21} \\ F_{23} \\ F_{31} \end{bmatrix} = \begin{bmatrix} 0 \\ \dots \\ \dots \\ F_{1,\text{ext}} \\ \dots \\ \dots \end{bmatrix} \]

Six equations for six unknowns (if the problem is well-posed)

Motivating Example from E11

Find the currents in this circuit.

Solving sysems of the form \(A x = b\)

Equivalent to inverting the system matrix \(A\) because \(Ax = b \implies x = A^{-1}b\)
The same \(A^{-1}\) can be used with multiple right-hand side vectors.
- \(Ax = b\) and \(Ax = c\) have solutions: \(x = A^{-1}b\) and \(x = A^{-1}c\) respectively.
Requires number of equations to equal number of unknowns.
- If unknowns > equations: probably need additional physical constraints
- If equations > unknowns: problem is overdetermined
- Cannot double-count two equations that express the same information in different ways.
- The above is equivalent to saying the equations need to be linearly independent.

Gaussian Elimination followed by substitution

\[\phantom{+}2x_1 + 3x_2 = 4 \tag{1}\] \[-\phantom{2}x_1 + 2x_2 = 5 \tag{2}\]

Idea: replace (2) with an equation that only has \(x_2\), not \(x_1\)
Procedure: Find a linear combination of eqs. (1) and (2) that has \(0x_1\).

e.g.

\((1) + 2 \times (2) \rightarrow \text{ new } (2)\)

The new equation is \(\boxed{0 x_1 + 7 x_2 = 14}\) \((2^*)\)

Resulting equations after Gaussian elimination

Original equations: \[\phantom{+}2x_1 + 3x_2 = 4\] \[-\phantom{2}x_1 + 2x_2 = 5\]

Equivalent equations: \[\phantom{+}2x_1 + 3x_2 = 4 \tag{3}\] \[\phantom{-}0x_1 + 7x_2 = 14 \tag{4}\]

1. and (4) are easy to solve for \(x_1\) and \(x_2\).
First solve for \(x_2\) using Equation 4
Then substitute value of \(x_2\) into Equation 3. Now, Equation 3 has only one unknown
Solve Equation 3 for \(x_1\).

Gaussian elimination followed by substitution \(\rightarrow\) solution!

Process of Gaussian Elimination

Start with \(\boldsymbol{A} \boldsymbol{x} = \boldsymbol{b}\)

\[ \begin{bmatrix} A_{11} & A_{12} & A_{13} \\ A_{21} & A_{22} & A_{23} \\ A_{31} & A_{32} & A_{33} \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix} \]

Construct a so-called augmented matrix with one extra column, the r.h.s. of the equation.

\[ \left[\begin{array}{ccc|c} A_{11} & A_{12} & A_{13} & b_1 \\ A_{21} & A_{22} & A_{23} & b_2 \\ A_{31} & A_{32} & A_{33} & b_3 \end{array}\right] \quad \]

Perform row operations until an augmented matrix of the following form survives.

This is known as the row echelon form.

\[ \left[\begin{array}{ccc|c} A^*_{11} & A^*_{12} & A^*_{13} & b^*_1 \\ 0 & A^*_{22} & A^*_{23} & b^*_2 \\ 0 & 0 & A^*_{33} & b^*_3 \end{array}\right] \quad \]

Recognize that the above is shorthand for the following equation:

\[ \begin{aligned} A_{11}^* x_1 + A_{12}^* x_2 + A_{13}^* x_3 &= b^*_1 \\ \phantom{A_{21}x_1 +} A^*_{22}x_2 + A^*_{23}x_3 &= b^*_2 \\ \phantom{A_{31}x_1 + A_{32}x_2 +} A_{33}^* x_3 &= b^*_3 \\ \end{aligned} \]

Solve, starting from the last.

\[x_3 = \frac{b_3^*}{A_{33}}^*\] \[x_2 = \frac{b_2^* - A_{23}^* x_3}{A_{22}^*}\] \[x_1 = \frac{b_1^* - A_{13}^* x_3 - A_{12}^* x_2}{A_{11}^*}\]

What are row operations ?

Swap row \(m\) with row \(n\)
Multiply a row by a non-zero constant
Add one row to another