The correct choice is (a) \(\begin{bmatrix}-\frac{1}{7}&\frac{3}{7}\\\frac{4}{7}&-\frac{5}{7}\end{bmatrix}\)
For explanation I would say: Consider the matrix A=\(\begin{bmatrix}5&3\\4&1\end{bmatrix}\)
By using the elementary row operations, we write A=IA
\(\begin{bmatrix}5&3\\4&1\end{bmatrix}\)=\(\begin{bmatrix}1&0\\0&1\end{bmatrix}\)A
Applying R1→R1-3R2 and R1→R1/(-7), we get
\(\begin{bmatrix}1&0\\4&1\end{bmatrix}\)=\(\begin{bmatrix}-\frac{1}{7}&\frac{3}{7}\\0&1\end{bmatrix}\)
Applying R2→R2-4R1, we get
\(\begin{bmatrix}1&0\\0&1\end{bmatrix}\)=\(\begin{bmatrix}-\frac{1}{7}&\frac{3}{7}\\\frac{4}{7}&-\frac{5}{7}\end{bmatrix}\)A
⇒A^-1=\(\begin{bmatrix}-\frac{1}{7}&\frac{3}{7}\\\frac{4}{7}&-\frac{5}{7}\end{bmatrix}\).