The correct choice is (c) Ae^4x + Be^-x
To elaborate: d^2y/dx^2 – 3dy/dx + 4y = 0 …..(1)
Let, y = e^mx be a trial solution of (1), then,
=> dy/dx = me^mx and d^2y/dx^2 = m^2e^mx
Clearly, y = e^mx will satisfy equation (1). Hence, we have,
m^2e^mx – 3m * e^mx – 4e^mx = 0
=>m^2 – 3m – 4 = 0 (as e^mx ≠ 0) …….(2)
=> m^2 – 4m + m – 4 = 0
=> m(m – 4) + 1(m – 4) = 0
Or, (m – 4)(m + 1) = 0
Thus, m = 4 or m = -1
Clearly, the roots of the auxiliary equation (2) are real and unequal.
Therefore, the required general solution of (1) is
y = Ae^4x + Be^-x where A and B are constants.