Right answer is (a) \(\frac{10x^3}{3} \left(x^3 logx-\frac{x^3}{3}\right)+C\)
To explain I would say: ∫ 10 logx.x^2 dx=10∫ logx.x^2 dx
By using the formula, ∫ u.v dx=u∫ v dx-∫ u'(∫ v dx), we get
\(10\int logx.x^2 \,dx=10(logx \int x^2 \,dx-\int (logx)’\int \,x^2 \,dx)\)
=\(10 \left(\frac{x^3 logx}{3}-(\int \frac{1}{x}.\frac{x^3}{3} \,dx)\right)+C\)
=\(10 \left(\frac{x^3 logx}{3}-\frac{1}{3} \frac{x^3}{3}\right)+C\)
=\(\frac{10x^3}{3} \left(x^3 logx-\frac{x^3}{3}\right)+C\).