Row operations change the equations without changing the solution set, so we simplify until the answer is readable.
Yesterday, asking whether b is in a span turned into a set of equations to solve, and we solved each one with whatever trick fit. Bigger systems need a method that always works. Today's method is row reduction. Its goal is to turn a system into one so simple that you can read the solution straight off it, while never changing which points are solutions. First we look at what a solution can look like, so we know what kind of answer to expect.
Each section below opens with a problem. Try it before you open the walkthrough, even if you only get partway.
Three possible outcomes
Solve the system
x1−2x2−x1+3x2=−1=3.
Then change the second equation to −x1+2x2=3 and solve again. Finally, try −x1+2x2=1.
Walk through the solutionHide the walkthrough
The x1 terms are x1 and −x1, so adding the two equations cancels them. That leaves −2x2+3x2=−1+3, which is x2=2. Putting x2=2 into the first equation gives x1−4=−1, so x1=3. The system has exactly one solution, (3,2).
With the second equation changed to −x1+2x2=3, adding the equations gives 0=2. Nothing can make 0 equal 2, so this system has no solution at all.
With −x1+2x2=1, adding gives 0=0, which is always true. The second equation is the first one multiplied by −1, so both describe the same line. Every point on that line is a solution, which means there are infinitely many.
Each equation in two unknowns is a line, and a solution is a point on both lines. The picture shows the three things two lines can do: miss each other, cross once, or coincide.
How many solutions a system can have
A linear equation in x1,…,xn can be written a1x1+⋯+anxn=b. A linear system is a few of these sharing the same unknowns. A solution is a list of numbers that makes every equation true at once, and the solution set is the set of all solutions.
Every linear system has no solution, exactly one, or infinitely many, just as in Problem 1. Two lines cannot share exactly two points. A system with at least one solution is consistent, and a system with none is inconsistent.
Which of these equations is linear?
The augmented matrix
Solving means doing the same few moves many times. Writing every x1 and x2 at each step is slow, so we keep only the numbers.
Write the coefficients of this system in a grid, one row per equation and one column per unknown, and enter it row by row. Then add the right-hand sides as a fourth column.
x1+3x2−x32x2+x3−x1+4x3=4=1=2Walk through the solutionHide the walkthrough
Line up each variable in its own column. The first equation gives the row 1,3,−1. The second equation has no x1, and a missing variable means a coefficient of 0, so its row is 0,2,1. The third equation has no x2, so its row is −1,0,4.
10−1320−114
Adding the right-hand sides 4,1,2 as a last column gives
10−1320−114412
The bar separates the coefficients from the right-hand sides. Each row still says one equation, and each column still belongs to one unknown.
Coefficient and augmented matrices
The coefficient matrix holds the coefficients of each variable in its own column. The augmented matrix adds the right-hand sides as a last column. On Day 2 you built the same grid from a vector equation, with the vectors as columns.
A matrix with m rows and n columns is m×n, rows first. In Problem 2 the coefficient matrix is 3×3 and the augmented matrix is 3×4.
Row operations
The goal is a system like x1=3, x2=2, where each equation hands you one unknown. To get there we need moves that change the equations but never change the solutions.
Start from Problem 1's system, x1−2x2=−1 and −x1+3x2=3, whose solution is (3,2). Replace the second equation by the sum of the two equations. Does (3,2) still satisfy both equations? Could the new system have a solution that the old one did not?
Walk through the solutionHide the walkthrough
The sum of the equations is (x1−x1)+(−2x2+3x2)=−1+3, which is x2=2. The new system is x1−2x2=−1 and x2=2.
The point (3,2) satisfies the first equation, which did not change, and it satisfies x2=2. Any point on both old lines satisfies their sum, so no solution was lost.
For the second question, run the move backward. Subtracting the first equation from x2=2 gives back −x1+3x2=3. So any solution of the new system also satisfies the old one, and no solution was gained. The move is reversible, and that is why the solution set stayed the same.
Three moves that keep every solution
The three elementary row operations change an augmented matrix without changing its solution set, because each one can be undone.
ReplacementInterchangeScalingOperationR2←R2+cR1R1↔R2R2←cR2,c=0Undo it withR2←R2−cR1R1↔R2R2←c1R2
Two matrices are row equivalent, written ∼, when a sequence of row operations turns one into the other. Row-equivalent systems have the same solution set. On Day 8 each of these operations becomes a matrix of its own.
In the picture, a replacement turns one line about the solution point. Use the sliders to make each row mention a single unknown.
Clear the x1 entry of row 2, then the x2 entry of row 1, so each row mentions a single unknown.
Now the matrix reads x1=3 and x2=2. The lines turned the whole time, but they never stopped crossing at the same point.
R2←R2+aR1
R1←R1+bR2
[1−1−23−13]
x1−2x2=−1−x1+3x2=3
The dashed lines are the original system. First R2←R2+R1 turns the blue line flat, and then R1←R1+2R2 stands the yellow line upright.
[1−1−23−13]∼[10−21−12]∼[100132]
Which operation undoes R3←R3−4R1?
Echelon form
We need to know when to stop. The next problem compares two possible end points.
Both matrices below are row equivalent to the same system. Read off the solution from each one.
A=100210−13−6−38−18B=1000100012−13Walk through the solutionHide the walkthrough
Matrix B is the easy one. Its rows say x1=2, x2=−1 and x3=3 directly.
Matrix A takes a little more work, but its shape helps. The last row mentions only x3: it says −6x3=−18, so x3=3. The middle row mentions only x2 and x3, so put in x3=3: x2+9=8, which gives x2=−1. The top row then gives x1+2(−1)−3=−3, so x1=2. Solving from the bottom up like this is called back substitution.
Both give (2,−1,3). A has a staircase of zeros in its lower left, which is enough to finish by back substitution. B goes further, with a single 1 in each column, so it needs no work at all.
Echelon and reduced echelon form
The leading entry of a row is its leftmost nonzero entry. A matrix is in echelon form when it has these three properties.
(1) All nonzero rows are above any rows of zeros.
(2) Each leading entry is in a column to the right of the leading entry in the row above it.
(3) Every entry below a leading entry is zero.
It is in reduced echelon form when it also has these two.
(4) Every leading entry is 1.
(5) Each leading 1 is the only nonzero entry in its column.
Every matrix is row equivalent to exactly one reduced echelon matrix.
In Problem 4, A is in echelon form and B is in reduced echelon form. Echelon form is the halfway target, where you can already tell how many solutions there are. Reduced echelon form is the finish line, where you read the solutions off.
Is 100001203 in echelon form?
The row reduction algorithm
Row reduction has two phases. The forward phase works down and to the right, making zeros below each leading entry until the matrix is in echelon form. The backward phase works up and to the left, scaling each leading entry to 1 and making zeros above it.
Solve by row reduction.
x1+2x2−x32x1+5x2+x3−x1+x2+4x3=−3=2=9Walk through the solutionHide the walkthrough
Each step below says which zero it makes and why that zero comes next.
1
Column 1 is the leftmost column, and its top entry 1 is nonzero, so it serves as the first pivot. Clear the entries below it with R2←R2−2R1 and R3←R3+R1:
12−1251−114−329∼100213−133−386
For example, row 2 becomes (2−2,5−4,1+2,2+6)=(0,1,3,8).
2
Cover row 1 and repeat on the rows below it. The next pivot is the 1 in row 2, column 2. Clear below it with R3←R3−3R2, which gives row 3 as (0,3−3,3−9,6−24)=(0,0,−6,−18):
∼100210−13−6−38−18
This is matrix A from Problem 4, so the forward phase is done.
3
Start the backward phase at the lowest pivot. Scale it to 1 with R3←−61R3, which makes row 3 say x3=3:
∼100210−131−383
4
Clear above that pivot with R2←R2−3R3 and R1←R1+R3. Row 2 becomes (0,1,0,8−9)=(0,1,0,−1) and row 1 becomes (1,2,0,−3+3)=(1,2,0,0):
∼1002100010−13
5
Move one pivot up and to the left. Clear above the pivot in column 2 with R1←R1−2R2, which makes row 1 (1,0,0,0+2)=(1,0,0,2):
∼1000100012−13
The solution is (2,−1,3). Check it in the second original equation: 2(2)+5(−1)+3=2.
The algorithm in five steps
The forward phase repeats steps 1 to 4, and the backward phase is step 5.
Step 1. Find the leftmost column that is not all zeros. Its top position will hold a pivot.
Step 2. If that position holds 0, swap in a row that has a nonzero entry there.
Step 3. Use replacements to make every entry below the pivot zero.
Step 4. Cover that row and repeat steps 1 to 3 on the rows below it.
Step 5. Starting from the lowest pivot and moving up, scale each pivot to 1 and use replacements to make every entry above it zero.
Now run the algorithm yourself. This system starts with a 0 in the top left, so step 2 is needed right away. The operation starts set to that swap.
Use row operations to reach reduced echelon form. The orange entries are the leading entries of each row.
That is the reduced echelon form, and the last column reads x1=4,x2=−1,x3=2.
R1R2R3012125213349
Operation
Row that changes
Row it uses
R1↔R2
0 moves
Pivots and free variables
Sometimes the reduced form has fewer pivots than unknowns. Then some unknowns are not pinned down.
The augmented matrix of a system reduces to
100−2000107−20110.
Describe every solution. Then give the solution with x2=0 and x4=0.
Walk through the solutionHide the walkthrough
Turn the rows back into equations. Row 1 says x1−2x2+7x4=1 and row 2 says x3−2x4=1. Row 3 says 0=0, which is always true and adds no condition.
Each nonzero row starts with a leading 1, in column 1 and column 3. So solve row 1 for x1 and row 2 for x3:
x1=1+2x2−7x4,x3=1+2x4.
Nothing restricts x2 or x4. You may choose them freely, and each choice gives one solution. With x2=0 and x4=0 you get x1=1 and x3=1, so the solution is (1,0,1,0). With x2=1 and x4=0 you get (3,1,1,0), another solution.
The leading 1s of the reduced form sit in pivot positions, and their columns are pivot columns. A variable whose column has a pivot is a basic variable. The others are free variables, and they can take any value.
⎩⎨⎧x1=1+2x2−7x4x2 is freex3=1+2x4x4 is free
The general solution
The list above is the general solution. It solves for each basic variable in terms of the free variables and lists every variable, including the free ones. The free variables act as parameters: each choice of their values gives one solution. Pivots come back often. On Days 12 and 13, counting them tells you whether the columns of a matrix span Rm and whether they are independent.
Pivot positions are the leading entries of any echelon form, so you do not need to finish the reduction to find them. Tap them in this echelon form.
Tap every pivot position in this echelon form, and nothing else.
Those are the pivots. Their columns make x1,x3,x5 basic, shown in teal, and the columns without a pivot make x2,x4 free, shown in orange.
x1
x2
x3
x4
x5
Existence and uniqueness
An echelon form already tells you how many solutions there are, before the backward phase.
For the system x1−2x2=3, 3x1+hx2=k, find the values of h and k that give infinitely many solutions. Which values give no solution, and which give exactly one?
Walk through the solutionHide the walkthrough
Do one step of the forward phase and look at what is left. Clear the 3 below the first pivot with R2←R2−3R1. Row 2 becomes (3−3,h+6,k−9):
[13−2h3k]∼[10−2h+63k−9]
Now read row 2 as the equation (h+6)x2=k−9.
If h=−6, row 2 has a pivot, so x2 is determined and there is exactly one solution, whatever k is.
If h=−6, row 2 says 0=k−9. When k=9 that equation is false, so there is no solution. When k=9 it says 0=0, so x2 is free and there are infinitely many solutions.
Existence and uniqueness theorem
A linear system is consistent exactly when an echelon form of its augmented matrix has no row
[0⋯0b] with b=0.
A consistent system has a unique solution when it has no free variables, and infinitely many when it has at least one.
A row [0⋯0∣b] says 0=b, which nothing satisfies. Below is Problem 7's system, with the elimination step already done in the lower readout. Watch how the blue line and the echelon row change together.
Choose h and k so the system has infinitely many solutions.
That works. Both equations describe the same line, and the echelon row [00∣0] says only 0=0.
x1−2x2=33x1+hx2=k
∼[10−273−7]
The last row says 0= a nonzero number, so there is no solution.
Every row has a pivot, so there is exactly one solution.
The last row is all zeros, so x2 is free and there are infinitely many solutions.
Tomorrow, Day 4 packs a whole system into one equation, Ax=b, and row reduction stays the way to solve it.
Practice
Warm-up
A system has 3 equations in 5 unknowns. What size is its augmented matrix?
Show the solutionHide the solution
There is one row per equation, so 3 rows. There is one column per unknown plus one for the right-hand sides, so 5+1=6 columns. The augmented matrix is 3×6.
Which matrix is in reduced echelon form? (Modeled on Lay §1.2 #1.)
Solve by row reduction. (Modeled on Lay §1.1 #1.)
x1+4x2−3x1−10x2=6=−14Show the solutionHide the solution
Use R2←R2+3R1, then R2←21R2, then R1←R1−4R2.
The second step is R2←−R2 followed by R1←R1−3R2. The pivots are in columns 1 and 3, so x2 is free:
⎩⎨⎧x1=−4+2x2x2 is freex3=3
With x2=0 the solution is (−4,0,3).
For which h is the system with augmented matrix [12h835] inconsistent? (Modeled on Lay §1.1 #19.)
Show a hintHide the hint
Do one replacement to reach echelon form, then ask when the second row reads 0= a nonzero number.
Show the solutionHide the solution
R2←R2−2R1 gives the row (0,8−2h,5−6)=(0,8−2h,−1). When h=4 this row says 0=−1, so the system is inconsistent. For every other h the row has a pivot and the solution is unique.
Do the three lines x1+x2=3, 2x1−x2=0 and x1−2x2=−3 pass through one common point? If so, enter it. (Modeled on Lay §1.1 #17.)
Show a hintHide the hint
Row reduce the 3×3 augmented matrix and look for a row [00∣b] with b=0.
For the system x1+3x2=2, 4x1+hx2=k, find the values of h and k that give infinitely many solutions. (Modeled on Lay §1.2 #19.)
Show a hintHide the hint
Reach echelon form with one replacement. Infinitely many solutions needs a consistent system with a free variable.
Show the solutionHide the solution
R2←R2−4R1 reaches echelon form.
[143h2k]∼[103h−122k−8]
If h=12, row 2 has a pivot and the solution is unique. If h=12 and k=8, row 2 says 0=k−8=0, so there is no solution. If h=12 and k=8, row 2 is all zeros, x2 is free, and there are infinitely many solutions.
True or false: if a linear system has a free variable, then it has infinitely many solutions. (Modeled on Lay §1.2 #22.)
Explain why "multiply a row by 0" is not allowed as a row operation. Give a system where doing it would create solutions that the original system does not have.
Show a hintHide the hint
Take a system with exactly one solution, turn one of its rows into zeros, and count the solutions of what is left.
Show the solutionHide the solution
Every allowed operation can be undone, which is why it keeps the solution set the same. Multiplying a row by 0 replaces it with 0=0, and no operation can recover the original row from zeros, so the argument breaks down.
For example, x1+x2=2, x1−x2=0 has the single solution (1,1). Multiplying row 2 by 0 leaves only x1+x2=2, which is also solved by (2,0), (0,2) and every other point on that line. Solutions were gained, so the new system is not equivalent to the old one.