Right option is (b) Linear and unstable
To elaborate: \(\frac{d^2 y(t)}{t^2} – \frac{dy}{dt} – 2y(t) = x(t)\)
Now, x (t) –> h (t) so, the system is linear.
Again, taking Laplace Transform with zero initial conditions, we get, s^2 Y(s) – s Y(s) – 2Y(s) = X(s)
Or, H(s) = \(\frac{Y(s)}{X(s)} = \frac{1}{s^2-s-2} = \frac{1}{(s-2)(s+1)} \)
Since pole is at s = +2, the system is unstable.