Dgdrxrt

Premise: I have an $n × q$ matrix $X$ and a $q × a$ matrix $C$ with $n > q > a$.

I'm interested in the structure of the matrix
$$
M = X X^+ - X_0 X_0^+
$$
where the superscript $^+$ indicates the Moore–Penrose pseudoinverse and
$$
X_0 = X (I_q - C C^+).
$$

I assume that $X$ is of full column rank and therefore $X^+ = (X' X)^-1 X$ (where $'$ indicates the transpose).

_{Background: $X$ is the design matrix of a linear model, $C$ is a contrast, $X_0$ is a reduced design matrix, and $M$ occurs in the definition of standard test statistics.}

$M$ is the difference of two orthogonal projection matrices, where the second projects into a subspace of the subspace the first projects into. This makes the difference an orthogonal projection matrix itself (symmetric and idempotent), which means it has a representation
$$
M = X_Delta X_Delta^+.
$$

Question: How do I obtain $X_Delta$?

_{user1551 has correctly pointed out in an answer that $X_Delta = M$ itself fulfills the equation. However, I'm looking for a "version" of $X$, meaning an $n times q$ matrix of rank $a$.}

My approach: I am guessing that
$$
X_Delta = X - X_0 X_0^+ X,
$$
and this seems to be confirmed by numerical tests. But I am unable to come up with a proof, i.e. to show that
$$
(X - X_0 X_0^+ X) (X - X_0 X_0^+ X)^+ = X X^+ - X_0 X_0^+.
$$

The problem is how to deal with the pseudoinverse of a difference. One can write
$$
X_Delta = (I_n - X_0 X_0^+) X,
$$
and according to Wikipedia, in the pseudoinverse of a product where one factor is an orthogonal projection, the orthogonal projection can be redundantly multiplied to the opposite side, meaning here
$$
X_Delta^+ = [(I_n - X_0 X_0^+) X]^+ = [(I_n - X_0 X_0^+) X]^+ (I_n - X_0 X_0^+) = X_Delta^+ (I_n - X_0 X_0^+),
$$
but that doesn't seem to help.

I can prove that $M$ is symmetric and idempotent, using the relations
$$
X X^+ X_0 = X_0
quad textand quad
X_0 X_0^+ X X^+ = X_0 X_0^+,
$$
which derive from the definition of $X_0$ and the properties of the pseudoinverse. I can also show that
$$
X X_0^+ = X_0 X_0^+
$$
using the property of the pseudoinverse of a product involving an orthogonal projection (see above). But none of that helps either.

With your choice of $X_Delta$, $M$ is indeed equal to $X_Delta X_Delta^+$.

Proof. Let $P=I-CC^+$. Note that the column space of $M=XX^+-(XP)(XP)^+$ is $operatornamecol(X)capoperatornamecol(XP)^perp$, while the column space of $X_Delta X_Delta^+$ is precisely the column space of $X_Delta=left[I-(XP)(XP)^+right]X$.

Since $X_Delta=Xleft[I-P(XP)^+Xright]$, $operatornamecol(X_Delta)subseteqoperatornamecol(X)$. Also, since
beginaligned
(XP)^TX_Delta
&=(X_Delta^TXP)^T\
&=left[X^Tleft(I-(XP)(XP)^+right)XPright]^T\
&=left[X^Tleft(XP-(XP)(XP)^+(XP)right)right]^T\
&=left[X^Tleft(XP-XPright)right]^T=0,
endaligned
we also have $operatornamecol(X_Delta)subseteqoperatornamecol(XP)^perp$. Thus $operatornamecol(X_Delta)subseteqoperatornamecol(M)$.

We now show that the reverse inclusion is also true. Pick any $vinoperatornamecol(M)=operatornamecol(X)capoperatornamecol(XP)^perp$. Since $vinoperatornamecol(X)$, it can be written as $Xb$ for some vector $b$. Thus
$$
X_Delta b=left[I-(XP)(XP)^+right]Xb=v-(XP)(XP)^+v.
$$
However, we also have $vinoperatornamecol(XP)^perp$. Therefore $(XP)(XP)^+v=0$ and in turn $X_Delta b=v$, meaning that $vinoperatornamecol(X_Delta)$.

Thus $operatornamecol(X_Delta X_Delta^+)equivoperatornamecol(X_Delta)=operatornamecol(M)$. Hence $X_Delta X_Delta^+$ and $M$ must be equal, because they are orthogonal projections with identical column spaces.