Pseudo-inverse of a matrix (Moore-Penrose)

The Moore-Penrose pseudo-inverse provides the least-squares solution to problems of the form , even if the matrix is non-invertible or non-square. Additionally, it is often more computationally efficient than exact matrix inversion.

When a matrix has full rank, we can express the pseudo-inverse as

For more detail, see chapter 2 of Math and Architectures of Deep Learning by Chaudhury, et al.