An iterative solution to a system of equations \(Ax=b\) starts from a (usually arbitrary) guess and progressively improves the solution until the true solution is reached.
The ‘true’ answer is only reached in an approximate sense, but our approximate answer can be made as accurate as we would like — convergence.
Convergence is only guaranteed when the matrix \(A\) is nice.
In contrast, Direct methods like Gaussian Elimination + Substitution follow a procedure that takes us directly to the true answer (but could be more expensive)
The Jacobi Method
Use the worksheet to calculate a few steps of Jacobi’s method by hand/calculator to solve for the three components of the vector \(x\) in the following two systems.
for k inrange(n_steps):# One for loop to go over each component of xfor j inrange(len(xold)): total =0for i inrange(len(xold)):if i != j: total += A[j,i] * xold[i] xnew[j] = (1/(A[j,j])) * (b[j] - total)print(xnew) xold = xnew.copy()
The need for copy
We often need a statement of the kind xold = xnew at the end of an iteration
The correct way to make this assignment is: xold = xnew.copy()
original = [1, 2, 3]# "Make a copy" then append only to the new oneassigned = originalassigned.append(4)print(f"Original (via =): {original}") print(f"Assigned: {assigned}")# Make a copy using .copy() then append only to the new onecopied = original.copy()copied.append(99)print(f"\nOriginal (via .copy): {original}")print(f"Copied: {copied}")
More generally, the norm of an \(n\)-dimensional vector \(\boldsymbol{u} = \begin{bmatrix} u_1 \\ u_2 \\ \dots \\ u_n \end{bmatrix}\) is \(\sqrt{u_1^2 + u_2^2 + \dots + u_n^2}\)
The Residual vector and iterative methods
Iterative methods produce successive candidate solutions
If the method is good, the candidates should get better with more iterations
How do we define goodness of a candidate solution?
The equation \(A x = b\) can be used to numerically evaluate a candidate solution \(x^*\) by evaluating \(\boxed{\left| \left| b - A x^* \right| \right| }\)
This quantity is known as the (norm of the) residual corresponding to the candidate solution \(x^*\).
The smaller the (norm of the) residual, the better a candidate solution is.
Jacobi and Gauss-Seidel are guaranteed to converge if \(A\) is diagonally dominant.
Diagonal Dominance: if, for every row,the absolute value of the entry on the main diagonal (\(A_{11}, A_{22}\), etc.)is greater than the sum ofthe absolute values of all off-diagonal entries in that row,then the matrix is Diagonally Dominant.
Diagonal Dominance is a sufficient, but not necessary, condition for convergence of both Gauss-Seidel and Jacobi.