18.06 pset 0 solutions¶

Problem 1 (13 points)¶

Suppose $A$ is a $3 \times 3$ matrix, $B$ is a $3 \times 2$ matrix, $x$ is a 3-component column vector, and $y$ is a $2$-component column vector.

What is the shape of the output (e.g. scalar, $4\times 3$ matrix, 5-component column vector, etcetera) of the following operations, or say nonsense if the operation doesn't make sense.

$A B$
$B A$
$A B y$
$A x B$
$x^T A y$
$x^T B y$
$x^T y$
$y x^T$
$x^2$
$A^2 = AA$
$B^2 = BB$
$\frac{x}{A}$
$\frac{x}{x}$

Solution 1¶

$3\times 2$ matrix: 2 components in (to $B$) are then fed into $A$, which spits 3 components out, so this process is $\mathrm(3 outputs) \times (2 inputs)$ and hence a $3 \times 2$ matrix
nonsense: $BA$ feeds the outputs of $A$ (3 components) into the input of $B$ (which expects 2 components)
$3$-component column vector: $y$ feeds into $B$, which spits out 3 components, which are fed into $A$, which spits out 3 components
nonsense: $Ax$ is a 3-component vector, but you can't multiply a column vector by a matrix $B$ on the right. (You could do $(Ax)^T B$, which would yield a 2-component row vector.)
nonsense: $Ay$ doesn't make sense, since $y$ has 2 components and $A$ expects 2 inputs.
scalar: this is the dot product of $x$ with the 3-component vector $By$.
nonsense: you can't take the dot product of a 3-component vector witha 2-component vector.
$2 \times 3$ matrix. You can think of this as the linear operator that first takes a dot product with $x$, and then scales $y$ by this factor. You also get the same $2 \times 3$ matrix if you apply the "rows-times-columns" matrix-multiplication rule, treating $y$ as a "1-column matrix" and $x^T$ as a "1-row matrix". (In applied linear algebra, we often gloss over the distinction between a column/row vectors and 1-column/1-row matrices. Here is a video on this glossing-over with more detail than you probably want.)
nonsense. Multiplication of a column vector by a column vector is not a standard operation in linear algebra. (One can define some rule here, but then you need a more explicit notation to denote which kind of product you want. For example, it's quite common to want an elementwise or "Hadamard" product of two column vectors, but this normally uses a special multiplication symbol, e.g. $x \odot x$, or x .* x in Julia.)
$3\times 3$ matrix: $A^2$ is just the linear operator that applies $A$ twice, which is perfectly sensible since the inputs and outputs of $A$ are both 3-component vectors.
nonsense: $B^2$ would apply $B$ twice, but the outputs of $B$ are 3-component vectors while its inputs are 2-component vectors, so there's no way to feed the outputs of $B$ back into another application of $B$.
nonsense. Later in the course, we will see that $A^{-1} x$ is meaningful (also denoted A \ x in Julia or Matlab), but you can only multiply a column vector by a matrix inverse on the left, while the notation $\frac{x}{A}$ is ambiguous about whether we are "dividing" on the left or right. We have to keep left vs. right crystal clear when working with non-commutative things like matrices, unlike real numbers!
nonsense. Just like $x^2$ above, we can't divide vectors by vectors. (Sometimes…not really in 18.06…you may want elementwise division of column vectors, but you need a special symbol for this like x ./ y in Julia.)

Problem 2 (10 points)¶

(From Strang, section 2.2, problem 11.)

A system of linear equations Ax=b cannot have exactly two solutions. An easy way to see why: if two vectors x and y≠x are two solutions (i.e. Ax=b and Ay=b), what is another solution? (Hint: x+y is almost right.)

Solution 2¶

For example, $\boxed{\frac{x+y}{2}}$.

To see this, we could start with hint: let's consider $x+y$, which would us $A(x+y)=Ax+Ay=b+b=2b$. This is almost what we want but not quite a solution, since we get $2b$ rather than $b$. We fix it by dividing by $2$ to get $A\left(\frac{x+y}{2}\right)=A\left(\frac{x}{2}\right)+A\left(\frac{y}{2}\right)=\frac{1}{2}Ax+\frac{1}{2}Ay=\frac{b}{2}+\frac{b}{2}=b$.

The key factor here is the linearity of $A$, which allows us to distribute $A$ through vector-addition and scalar-multiplication operations.

More generally, we could see that $\boxed{\alpha x + (1-\alpha)y}$ is a solution for any real number $\alpha$, since $A[\alpha x + (1-\alpha)y] = \alpha Ax + (1-\alpha) Ay = \alpha b + (1-\alpha) b = b$ by linearity. That means, as soon as a linear equation has two distinct solutions, it must have infinitely many solutions. (We will spend a lot of time on this later in 18.06, because it's quite important in practice.)

Problem 3 (2+4+2+2 points)¶

Either(ideally) download and install Julia and IJulia/Jupyter by following these instructions, or (slower, less convenient) run Julia in the cloud using these instructions.

Create a new Julia notebook. Title it "[your last name] pset 0".
Define a matrix A that takes in any 3-component vector $x$ and outputs ($Ax$) the components in reverse order. For example:

In [ ]:

A = [ 1 2 3 4
      5 6 7 8 ] # this is a 2×4 matrix … not the right answer

3. Multiply A by the vector x = [1,2,3] to check that it is doing what you want.

4. Try computing A*A. The result should be a very simple linear operation: what?

Be sure to include a PDF printout of your notebook with your submitted solutions.

Solution 3¶

2. First, let's set up the matrix. One way to think of it is by row, since each row of the matrix determines the corresponding row of the output vector. That is, the first row has to pluck the third element from the vector so it should be 0 0 1, the second row has to pluck the second element so it is 0 1 0, and the third row has to pluck the first element so it should be 1 0 0.

In [ ]:

A=[0 0 1
   0 1 0
   1 0 0]

(There are other ways to get the same matrix, of course. This is an example of a permutation matrix, and a simple way to get it is to start with the identity matrix $I$ and apply our re-ordering to the rows. We'll think about this more later in 18.06.)

3. Let's try out our A:

Hooray, it reversed the elements in the output as desired!

Think about what $A$ does: it reverses the order of the components of any vector $x$. The matrix $A^2$ reverses the order twice, which means that it results in the original vector, since the second reversal undoes the first. In other words, $A^2$ must be the identity matrix.

Let's try it and see:

(Later in the course, we will say that the matrix $A$ is its own inverse: $A^{-1} = A$. This is very special, and is not true of most matrices!)